Introduction to OpenMP

Reading time: 15 minutes

OpenMP is an open source library which is used for multi-threaded parallel processing and shared-memory multi-processor (core) computers. In OpenMP, part of program is a single thread and part is multi-threaded.

OpenMP has 3 components:

  • directives that can be put into C/C++ or Fortran programs
  • runtime library for setting and querying parallel parameters (number of threads)
  • environment variables that define runtime parallel parameters (number of threads)

Approach

Two main approaches of OpenMP:

  • loop-level
  • parallel regions

Loop-Level Parallelism / fine-grained parallelism

  • individual loops parallelized
  • each thread assigned a unique range of the loop index
  • execution starts on a single serial thread
  • multiple threads are spawned inside a parallel loop
  • after parallel loop execution is serial
  • relatively easy to implement

Parallel Regions Parallelism / coarse-grained parallelism

  • any sections of codes can be parallelized (not just loops)
  • using the thread identifier to distribute the work
  • execution starts on a single serial thread
  • multiple threads are started for parallel regions (not necessarily at a loop)
  • ends on a single serial thread

Advantages of OpenMP

Advantages of OpenMP are:

  • Prevalence of multi-core computers
  • Requires less code modification than using MPI
  • OpenMP directives can be treated as comments if OpenMP is not available
  • Directives can be added incrementally

Disadvantages of OpenMP

Disadvantages of OpenMP are:

  • OpenMP codes cannot be run on distributed memory computers (exception is Intel's OpenMP)
  • Requires a compiler that supports OpenMP (most do)
  • limited by the number of processors available on a single computer
  • often have lower parallel efficiency
  • rely more on parallelizable loops
  • tend to have a higher % of serial code
  • Amdahl's Law - if 50% of code is serial will only half wall clock time no matter how may processors

Application

OpenMP is used in:

  • Matlab
  • Mathematica
  • Libraries
  • Intel Math kernel Library (MKL)
  • GNU Scientific Library (GSL)