当我们谈论 shared memory parallelism 的时候我们在谈论些什么--前导篇

2020-05-03 约 432 字预计阅读 1 分钟

在目前正在更的Theo和Network系列上又要开启新的系列了呢，也就是目前看到的 parallel computing 系列。因为这个学期刚好选到一个主题是high performance computing (HPC) 的Seminar，然后自己被分配到的题目就是标题所说的"shared memory parallelization"。因为最后需要交一篇paper还要做一个presentation，觉得也可以新开一个系列来整理记录自己看各种资料时候学习到的相关的新知识，当然因为看的资料本身是英语，为了之后写paper方便我有些时候也就直接写英语了。本篇主要内容是最近要交的outline和一些关于并行计算的导引。

General concept
- why parallel?
- what is the idea of shared memory?
- memory modules
Hardware views
- Different architecture
  - UMA Systems
    - Pro
    - Nach
  - NUMA Systems
    - Pro
    - Nach
  - How do interconnection networks work?
    - basic ideas that distiguish from normal networks
    - switch, crossbar….
  - Cache issues
    - Cache coherence:
      - Snooping cache coherence
      - Directory-based cache coherence
    - False sharing:
      - Basic explanaton, will have further discussion in programmer’s view
Programmer’s view
- parallel programming models
  - shard memory without threads
  - threads model:
    - some basic concept and implementation with OpenMP or CUDA
- some example parallel programs written in OpenMP that may concerning to solving flase sharing
Difference in HPC
- Distiguish what’s special about shared memory in HPC (still not quite clear, need further readings)

Materials related:

  Prgramming on Parallel Machines, Norm Matloff, Univercity of California, Davis
An Introduction to Parallel Programming, Peter Pacheco
     
Parallel Computer Architecture A Hardware / Software Approach, David Culler, Jaswinder Paul Singh
     
Parallel Programming for Modern High Performance Computing Systems, Paul Czarnul
     
Slide of an old lecture from TUM: https://www5.in.tum.de/lehre/vorlesungen/parhpp/SS08/

目录

当我们谈论 shared memory parallelism 的时候我们在谈论些什么--前导篇