Interprocess Communication: Pipes

In this article we discuss how pipes enable both related and unrelated processes communicate in Linux.

Table of contents.

1. Introduction.
2. Pipe creation.
3. Parent-child communication.
4. Redirection.
5. FIFOs.
6. Summary.
7. References.

Prerequisite.

1. Shared Memory
2. Semaphores
3. Mapped Memory

Introduction.

In the prerequisite articles we have discuss three different types of interprocess communications namely, shared memory, semaphores and mapped memory.
Others include,
Sockets, which are bidirectional in nature. They can be used for communication between processes in the same machine or processes running on different machines.

Pipes on the other hand allow unidirectional communication between processes. That is, data that is written to the write end of the pipe is read back from the read end. You can visualize a pipe that has water flowing from one direction to another.
Data in pipes is read in the same way it is written.

In a Linux shell we use the | character to pipe one process' data to another process.
An example

cat hosts.txt | xargs -n1 ping -c 2

The above command will result in two processes, the concatenation process and the ping process. Data from the cat process is communicated to the ping process which uses it to execute its function.

Just like water pipes, data in a pipe is limited whereby if the writer process cat writes faster than the reader process ping consumes the data and assuming the pipe cannot store more data, the writer process blocks until more space is available. On the other hand if the reader process reads faster than the writer process writes, the reader process will block until the data is available.
This is referred to as synchronization in pipes.

Pipe creation.

pipe command is used to create a pipe, it takes an integer array of size two whereby the reading file descriptor is stored at index 0 and the writing file descriptor at index 1.

Consider the code:

int pipe_fds[2];
int read_fd;
int write_fd;

pipe(pipe_fds);
read_fd = pipe_fds[0];
write_fd = pipe_fds[1];

where data that is written in read_fd file descriptor is read back from write_fd file descriptor.

Parent-child communication.

When pipe command is invoked, file descriptors which are only valid within that process are created.

These descriptors can't be passed to unrelated processes although when fork is called by a process, they are copied to the child process.

Therefore, pipes can only facilitate the communication between related processes, however as we shall come to see later we can use FIFO to also have unrelated processes communicate.

An example of using a pipe to communicate with a child process - pipe.c

#include<stdio.h>
#include<unistd.h>
#include<stdlib.h>

// writing COUNT copies to MESSAGE to STREAM
// pause for 1 second between each one
void writer(const char* message, int count, FILE* stream){
    for(; count > 0; --count){
        // write message to stream and send instantly
        fprintf(stream, "%s\n", message);
        fflush(stream);

        // sleep 1 second
        sleep(1);
    }
}

// read random strings from stream
void reader(FILE* stream){
    char buffer[1024];

    // read till end of stream(new line or EOF)
    while(!feof(stream) && !ferror(stream) && fgets(buffer, sizeof(buffer), stream) != NULL)
        fputs(buffer, stdout);
}

int main(){
    int fds[2];
    pid_t pid;

    // create pipe
    pipe(fds);

    // fork child process
    pid = fork();

    if(pid == (pid_t)0){
        FILE* stream;
        
        // close copy of write end of FD
        close(fds[1]);

        // convert read FD to FILE object and read from it
        stream = fdopen(fds[0], "r");
        reader(stream);
        close(fds[0]);
    }else{
        // parent process
        FILE* stream;

        // close copy of read end of FD
        close(fds[0]);

        // convert write FD to FILE object and write to it
        stream = fdopen(fds[1], "w");
        writer("COMMUNICATION WITH CHILD PROCESS THROUGH PIPES!", 7, stream);
        close(fds[1]);
    }
    return 0;
}

Compilation and execution:

$ gcc pipe.c -o pipe && ./pipe

fds is declared as an integer array of size two.
The call to pipe creates a pipe an puts read and write file descriptors in the array as explained earlier.
fork spawns a child process which inherits the pipes file descriptors.
The parent process will write a string to the pipe which is read out by the child process.
fdopen is used to convert the file descriptors into FILE* streams and since we have streams we use printf and fgets high-level standard I/O functions.

After writing in the writer function, the parent process will flush the pipe using fflush call, if not, the string might not be sent through the pipe instantly.

In our case where we cat and ping, two forks will occur each for each single process. Each inherits the pipe file descriptors so as to be able to communicate using a pipe.

Redirection.

It is common to create a child process and set one end of the pipe either as its standard input or standard output.

The dup2 call is used to equate one file descriptor with another e.g to redirect a process' standard input to a file descriptor fd we write,

dup2(fd, STDIN_FILENO);

where the symbolic constant STDIN_FILENO is the file descriptor for the standard input and has a value of 0.

When called, it will first close the standard input and then reopens it as a duplicate of fd so the two may be used interchangeably.

When file descriptors are equated, they will share a similar file position including the same set of file status flags and therefor characters that are read from fd won't be reread from the standard input.

An example of redirecting output from a pipe with dup2 - dup2.c

#include<stdio.h>
#include<unistd.h>
#include<sys/types.h>
#include<sys/wait.h>

int main(){
    int fds[2];
    pid_t pid;

    // create pipe
    pipe(fds);

    // fork child process
    pid = fork();
    if(pid == (pid_t)0){
        // close copy of write end of FD
        close(fds[1]);
        
        // connect read end to stdin
        dup2(fds[0], STDIN_FILENO);

        // replace child process with sort
        execlp("sort", "sort", 0);
    }else{
        // parent process
        FILE* stream;

        // close copy of read end of FD
        close(fds[0]);

        // convert write FD to FILE object and write to it
        stream = fdopen(fds[1], "w");
        fprintf(stream, "M \n");
        fprintf(stream, "Z \n");
        fprintf(stream, "A \n");
        fprintf(stream, "F \n");
        fprintf(stream, "C \n");
        fprintf(stream, "K \n");

        fflush(stream);
        close(fds[1]);

        // wait child process to finish
        waitpid(pid, NULL, 0);
    }
    return 0;
}

Compilation and execution:

gcc dup2.c -o dup2 && ./dup2

dup2 is used to send output from a pipe to the sort command.
After a pipe is created the program forks.

The parent process prints some strings to the pipe while the child process attaches to the read file descriptor of the pipe to its standard input using dup2, then sort is executed.

Pipes are commonly used to send or receive data to/from a program being executed in a subprocess.
popen and pclose functions are used so as to eliminate the need for pipe, exec, dup2, fork and fopen invocations.

Using popen and pclose - popenclose.c.

#include<stdio.h>
#include<unistd.h>

int main(){
    FILE* stream = popen("sort", "w");
    fprintf(stream, "M \n");
    fprintf(stream, "Z \n");
    fprintf(stream, "A \n");
    fprintf(stream, "F \n");
    fprintf(stream, "C \n");
    fprintf(stream, "K \n");
    return pclose(stream);
}

Compilation and execution:

gcc popenclose.c -o popenclose && ./popenclose

When popen is called, it creates a child process to execute the sort command thus replacing pipe, execlp, dup2 and fork calls.
w which is the second argument of popen indicates that the process wants to write to the child process.
The return value of popen is one end of a pipe, the other end is connected to the standard input of the child process.
After writing is done, pclose is used to close the child's stream, it also waits for the termination of the process and it returns the status value.

popen's first argument is executed as a shell command in a subprocess running /bin/sh, usually this the shell will search the PATH environment variable to find a program to execute.
Assuming we pass r as the second argument to popen, it will return the standard output stream of the child process so that the parent can read the output.
If it is w, it will return the standard input of the child process so the parent can send data.
A null pointer is returned incase of an error.

pclose closes the stream returned by popen after which it waits for the termination of the child process.

FIFOs

A FIFO file is a named pipe in the file system that can be opened or closed by any process, these processes might even be unrelated.

mkfifo command is used to make a FIFO.
To create a FIFO in /tmp/fifo we write,

$ mkfifo /tmp/fifo

# Confirm its creation
$ ll /tmp

prw-r--r-- 1 user user    0 Jun  20 02:54 fifo

The p character in the file's permissions indicates that this is a named pipe.

Next open two terminal windows,
In the first terminal window type in the command,

cat < /tmp/fifo

In the second window T2 type,

cat > /tmp/fifo

Now when we write in the second window and press enter, we shall see the text being displayed in the first. This is because the text is being sent though FIFO.

Programmatically we create FIFOs using mkfifo function which takes a path where the FIFO is to be created as its first argument and the pipe's owner, group and world permissions as its second argument.
Note that the permissions must include read and write permissions since the pipe will have a reader and a writer.

The function will return -1 incase of a failure to create a pipe.

As we have seen, access to a FIFO is just like accessing a normal file.
Communication though a FIFO involves a program opening it for writing and another opening it for reading.
We can use low(open, write, read, close etc) or high(fopen, fprintf, fscanf, fclose etc) level I/O functions for this.

Using low level functions we can write a buffer of data to a FIFO as follows,

int fd = open(fifo_path, O_WRONLY);
write(fd, data_length);
close(fd);

Using high level C library I/O functions, we write,

FILE* fifo = fopen(fifo_path, "r");
write(fifo, "%s", buffer);
fclose(fifo);

Bytes from a writer or reader are written to a max size of the PIPE_BUF of 4KB in Linux.
Chunks of simultenous readers or writers can also be interleaved.

Summary.

Named pipes in Win32 are more like sockets in Linux which we discuss in the next article. They connect different processes on different systems through a network and since there can be multiple reader-writer connections, this allows for two-way communication.

FIFO allows unrelated processes to communicate using pipes. It could have multiple readers and writer.
Simultaneous readers and writers can be interleaved.

References.

1. Execute man pipe, man mkfifo for their manual pages.
2. Sockets.