It’s Parallel Programming 101 • Why can’t we advance from here? • Almost all of the patterns discussed are either intuitive, or covered in introductory courses. • Do these patterns capture most of parallel programs today? • Can these patterns be used to exploit full parallelism?

Structure the UEs (units of execution) of a parallel computation, for better manageability and easier integration with the core computation.

Evaluate Design What is the target architecture? Shared memory, distributed memory, message passing, … Does data sharing have enough special properties (read only, accumulate, temporal constraints) that we can deal with dependences efficiently? If …

There are a number of parallel code patterns that are closely related to the system or hardware that a program is being written for and the software library used to enable parallelism, or concurrent execution.

Parallel Design Patterns¶ There are multiple levels of parallel design patterns that can be applied to a program. At the highest level, algorithmic strategy patterns are strategies for decomposing a problem in its most abstract form.

In particular, this chapter introduces the concept of threads, describes how to use them to write parallel programs, and provides some detailed examples. The most common models for parallel programming make use of the concept of threads.

Since parallel programming has existed for quite a long time, there are many different patterns for programming parallel applications. There are even special programming languages to make programming of specific parallel algorithms easier.

Examine the catalogs of parallel design patterns for structures which match the dependencies in your application. Use the templates and examples for the appropriate patterns to structure and implement your code.

Design Evaluation Pattern • Whether the partition fits the target hardware platform? • Key questions to ask – How many threads can be supported? – How many threads are needed? – How are the data structures shared? – Is there enough work in each thread between synchronizations to make parallel execution worthwhile?

Parallel programming involves writing code that divides a program’s task into parts, works in parallel on different processors, has the processors report back when they are done, and stops in an orderly fashion.