Basics of Parallel Programming

This chapter summarizes basic concepts of parallel computations. It explains what concurrent multitasking and threads are. The chapter describes the risk of using shared objects, and how to protect their access. It shows how to make standard algorithms like std : : transform, std : : sort, and std : : transform_reduce operate in a parallel mode. The chapter discusses programming asynchronous tasks with std : : future and basic construction of the OpenMP library – e.g. how to launch and synchronize multiple tasks, and how to make for operate in parallel to significantly speed up matrix multiplication. It also explains the most important concepts related to the domain of parallel programming: task, multitasking; thread, multithreading; thread pool; team of thread; concurrency; shared objects; private objects; critical section; mutual exclusion; deadlock; atomic instruction; synchronization; and thread safety. The chapter builds parallel systems based on splitting an initial array of data into a number of chunks.