Introduction to OpenMP

Do not miss this exclusive book on Binary Tree Problems. Get it now for free.

Reading time: 15 minutes

OpenMP is an open source library which is used for multi-threaded parallel processing and shared-memory multi-processor (core) computers. In OpenMP, part of program is a single thread and part is multi-threaded.

OpenMP has 3 components:

directives that can be put into C/C++ or Fortran programs
runtime library for setting and querying parallel parameters (number of threads)
environment variables that define runtime parallel parameters (number of threads)

Approach

Two main approaches of OpenMP:

loop-level
parallel regions

Loop-Level Parallelism / fine-grained parallelism

individual loops parallelized
each thread assigned a unique range of the loop index
execution starts on a single serial thread
multiple threads are spawned inside a parallel loop
after parallel loop execution is serial
relatively easy to implement

Parallel Regions Parallelism / coarse-grained parallelism

any sections of codes can be parallelized (not just loops)
using the thread identifier to distribute the work
execution starts on a single serial thread
multiple threads are started for parallel regions (not necessarily at a loop)
ends on a single serial thread

Advantages of OpenMP

Advantages of OpenMP are:

Prevalence of multi-core computers
Requires less code modification than using MPI
OpenMP directives can be treated as comments if OpenMP is not available
Directives can be added incrementally

Disadvantages of OpenMP

Disadvantages of OpenMP are:

OpenMP codes cannot be run on distributed memory computers (exception is Intel's OpenMP)
Requires a compiler that supports OpenMP (most do)
limited by the number of processors available on a single computer
often have lower parallel efficiency
rely more on parallelizable loops
tend to have a higher % of serial code
Amdahl's Law - if 50% of code is serial will only half wall clock time no matter how may processors

Application

OpenMP is used in:

Matlab
Mathematica
Libraries
Intel Math kernel Library (MKL)
GNU Scientific Library (GSL)