Parallel Programming Models •Fundamental question: what is the “right” way to write parallel programs –And deal with the complexity of finding parallelism, coarsening granularity, distributing computation and data, synchronizing, optimizing, etc. •Oh, and we want to achieve high performance •And make the program portable across ...

Programming models provide a way to think about the organization of parallel programs (by imposing structure) Shared address space: very little structure to communication

Parallel Computational Models 6.173 Fall 2010 L02 Agarwal-2-Parallel Computational Models Formally, a Computational Model is a Coherent collection of mechanisms for – communication – synchronization – partitioning –placement – scheduling Computational model defined at all levels of abstraction: Machine hardware – e.g., a shared ...

• All processors in a parallel computer can execute different instructions and operate on different data at the same time. • Parallelism achieved by connecting multiple processors

How do we evaluate a parallel program? •Execution time, T p •Speedup, S –S(p, n) = T(1, n) / T(p, n) –Usually, S(p, n) < p –Sometimes S(p, n) > p (superlinear speedup) •Efficiency, E –E(p, n) = S(p, n)/p –Usually, E(p, n) < 1 –Sometimes, greater than 1 •Scalability –Limitations in parallel computing, relation to n and p.

provides a general overview of parallel programming, summarizing the challenges inherent in writing parallel programs, the techniques that can be used to create them, and the metrics used to evaluate these techniques. The next section begins by providing a …

In this book, we focus on this parallel programming, where instructions are neither specified nor expected to be in a single sequence. Further, the execution of these programs is also in a parallel context, where potentially several thousand instructions, or even more, execute at any given time.

Parallel programming models are almost used to integrate parallel software concepts into a sequential code. These models represent an abstraction of the hardware capabilities to the programmer. In fact, a model is a bridge between the application to be parallelized and the machine organization.

We cover shared-memory models, distributed-memory models, models for devices with private memory spaces such as gpus and accelerators, as well as models that combine the aforemen-tioned ones in some manner.

Creating a parallel program Your thought process: 1. Identify work that can be performed in parallel 2. Partition work (and also data associated with the work) 3. Manage data access, communication, and synchronization A common goal is maximizing speedup * For a !xed computation: Speedup( P processors ) = Time (1 processor) Time (P processors)