What we're talking about when we talk about shared memory parallelism - a precursor

I’m going to start a new series on the Theo and Network series that I’m currently working on, which is the parallel computing series that I’ve seen so far. This semester, I was assigned the topic “Shared Memory Parallelization”, which is the title of the seminar. Because I need to submit a paper and do a presentation at the end, I think I can also open a new series to organize and record the new knowledge I learned when I read various materials, of course, because the materials I read are in English, so in order to write a paper afterwards, I sometimes write directly in English. The main content of this article is the recent outline to be submitted and some introduction about parallel computing.

1. General concept
   • why parallel?
   • what is the idea of shared memory?
   • memory modules
2. Hardware views
   • Different architecture
     • UMA Systems
       • Pro
       • Nach
     • NUMA Systems
       • Pro
       • Nach
     • How do interconnection networks work?
       • basic ideas that distiguish from normal networks
       • switch, crossbar….
     • Cache issues
       • Cache coherence:
         • Snooping cache coherence
         • Directory-based cache coherence
       • False sharing:
         • Basic explanaton, will have further discussion in programmer’s view
3. Programmer’s view
   • parallel programming models
     • shard memory without threads
     • threads model:
       • some basic concept and implementation with OpenMP or CUDA
   • some example parallel programs written in OpenMP that may concerning to solving flase sharing
4. Difference in HPC
   • Distiguish what’s special about shared memory in HPC (still not quite clear, need further readings)

Materials related:

  Prgramming on Parallel Machines, Norm Matloff, Univercity of California, Davis
An Introduction to Parallel Programming, Peter Pacheco
     
Parallel Computer Architecture A Hardware / Software Approach, David Culler, Jaswinder Paul Singh
     
Parallel Programming for Modern High Performance Computing Systems, Paul Czarnul
     
Slide of an old lecture from TUM: https://www5.in.tum.de/lehre/vorlesungen/parhpp/SS08/