Interprocess communication: Shared memory

Do not miss this exclusive book on Binary Tree Problems. Get it now for free.

In this article we give an introduction to interprocess communication in Linux and discuss how processes use shared memory to communicate with each other.

Table of contents.

Introduction.
Shared memory.
Conclusion.
Summary.
References.

Prerequisites.

Introduction.

IPC(Interprocess communication) is the transfer of data between processes. For example the command,

cat hosts.txt | xargs -n1 ping -c 2

will start the concatenation process as well as the ping process and through the pipe(|) symbol data will be transfered from cat process to ping process.
A pipe as used here will allow a one sided communication between the two related processes.

Other than pipes there other different types of interprocess communication namely,

Mapped memory which allow processes to communicate via a shared file in the file system.
Sockets, these support interprocess communication between unrelated processes even on different systems.
Semaphores, are counters which allow multiple threads to synchronize, apart from these there exists an alternate implementation of semaphores referred to as process semaphores or system V semaphores which are used for interprocess communication in Linux.

We categorize IPCs using the following criteria,

Communication restrictions between related processes, unrelated processes sharing a file system or networked computers.
Process' permissions(read, write).
Synchronized communication between processes. e.g if a process will halt to wait for data dependent on another executing process.
Number of processes involved in the communication.

Shared memory.

This type of communication allows processes to access the same memory locations if they are all called malloc and were returned to the same memory space and therefore when a process modifies the memory, this change is seen by all related processes.

This is the fastest form of IPC since all processes share the same memory space.
Access to this memory location is fast since no system calls or kernel entries are required.
Unnecessary data copying is also avoided.

The kernel doesn't synchronize access to shared memory and therefore we must define our own synchronization so as to avoid race conditions. A common strategy is use of semaphores which we have discussed in the next article, the link is given in the references section.

The model.

For a shared memory segment, one process allocates the memory segment and every other process which need to access this segment attaches it to itself.
After using the segment it detaches and at some point the segment is deallocated.

In Linux, each process' virtual memory is split into pages.

Allocating shared memory segments is done by a single process and it causes the creation of virtual memory pages.
Allocating existing segments won't cause the creation of new pages however an identifier to the existing page is returned.

For a process to use a shared memory segment, a process will attach it which will add entries that map from the virtual memory to the segment's shared pages.
When a process is done, the mapping entries are removed and when no other processes need to access shared memory, a single process is given the responsibility for deallocating the virtual memory pages.

Allocation.

The shmget(Shared Memory Get) system call is responsible for allocating shared memory.

It takes an integer key which specifies the segment to create as its first parameter and thus other unrelated processes will use this key to access this shared segment. There may be conflicts if other processes choose this same key, to solve this we use IPC_PRIVATE which is a special constant to guarantee that a new segment is created.

The second parameter is the number of bytes in the segment and since segments are allocated using pages, this number is rounded to an integral multiple of the page size finally, the third parameter is a bitwise or flag values which specify options to shmget.

Flag values may include IPC_CREAT which indicates the creation of a new segment, IPC_EXCL which will cause a shmget to fail if the specified segment exists and Mode flags which is made up of nine bits that represent permissions granted to the owner, group and world to control access to a segment.

For example to create a new shared memory segment with r/w permissions for owners and not users, we write,

int segmentId = shmget(shm_key, getpagesize(), IPC_CREAT | S_IRUSR | S_IWUSER);

If successful we should expect a segment identifier and if the segment exists access permissions are checked to ensure it is not marked for destruction.

Attachment.

The shmat(Shared Memory Attach) call is used by a process to make a shared memory segment available.

It takes the SHMID(Shared memory ID) which we got from shmget as its first argument, a pointer specifying the process' address space where we want to map the shared memory as its second argument and a flag as its third argument.
A flag can be SHM_RND to indicate rounding of the specified address to a page size multiple or SHM_RDONLY to indicate that the segment should only be read and not written to.

If a call is successful an address of the attached shared segment is returned.

Detachment.

After we are done with a shared memory segment, we call the shmdt(Shared Memory Detach) system call which takes the address we got from shmat as its argument.
The segment is removed if it had been deallocated and the current process was the last to use it.

Controlling and Deallocating shared memory.

The shmctl(Shared Memory Control) call is responsible for returning information pertaining a shared memory segment. It is also able to modify it.

It takes the SHMID as its first argument, IPC_STAT as the second argument that is used to obtain the information and a pointer to struct shmid_ds as its third argument.

Deallocating shared memory.

The above call is still used here but instead of passing IPC_STAT as the second argument we pass IPC_RMID and NULL for the third argument.

If successful the segment is removed when the last process formerly attached to it detaches.

Invoking exit or exec can be used to detach memory but not deallocation and thus it is required to use shmctl to explicitly deallocate segments when done with them to avoid violating a systemwide limit on the total number of shared memory segments.

An example of how we use shared memory, shm.c

#include<stdio.h>
#include<sys/shm.h>
#include<sys/stat.h>

int main(){
    int segmentID;
    char* sharedMemory;
    struct shmid_ds shmbuffer;
    int segmentSize;
    const int sharedSegmentSize = 0x6400;

    // allocate shared segment
    segmentID = shmget(IPC_PRIVATE, sharedSegmentSize, IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR);

    // attach
    sharedMemory = (char*) shmat(segmentID, 0, 0);
    printf("Attached at address %p \n", sharedMemory);

    // determine size
    shmctl(segmentID, IPC_STAT, &shmbuffer);
    segmentSize = shmbuffer.shm_segsz;
    printf("Segment size: %d \n", segmentSize);

    // add data to shared memory
    sprintf(sharedMemory, "THIS IS A TEST");

    // detach
    shmdt(sharedMemory);

    // reattach to a different address
    sharedMemory = (char*) shmat(segmentID, (void*) 0x5000000, 0);
    printf("Reattached at address %p \n", sharedMemory);

    // print data from shared memory segment
    shmdt(sharedMemory);

    // deallocate shared memory segment
    shmctl(segmentID, IPC_RMID, 0);

    return 0;
}

Compilation and execution,

gcc shm.c -o shm && ./shm

To obtain information about shared memory write,

$ ipcs -m

The output,

------ Shared Memory Segments --------
key        shmid      owner   perms      bytes      nattch     status
0x00000000 163843     user    600        3002368    2          dest
0x00000000 163844     user    600        3002368    2          dest
0x00000000 32779      user    600        36864      2          dest
0x00000000 32780      user    600        36864      2          dest

We can remove a memory segment by using the ipcrm command as follows,

ipcrm 163844

Where we pass the shared memory id, that is if a program left a segment by error.

Summary.

Shared memory segments allow for fast bidirectional communication among processes.
Users can read/write data but programs must have well defined protocols to prevent race conditions.

Linux doesn't guarantee exclusive access even when using IPC_PRIVATE.
For multiple processes to share a segment they should use the same key.

References.

Execute man shmctl, man shmdt, man shmat, man shmget for manual pages.
IPC Slides from Department of Information and Computer Sciences at the University of Hawaii at Manoa.
Semaphores