A few weeks ago, I attended a local meetup where someone presented on asynchronous Inter-Process Communication (IPC) using Linux IPC APIs [1]. The speaker showcased different Linux APIs for IPC, such as FIFO, Message Queue, and Ethernet Socket. I find this talk quite interesting and inspire me to explore these features that enable asynchronous inter-process communication on Linux.

Actually this is not my first time I encounter this. In my previous project, the company built their own Remote Procedure Call (RPC) framework. The framework supports both synchronous and asynchronous inter-process communication between processes. It supports not only the request-reply pattern but also notification (subscribe and publish) pattern.

I know how to use the framework to facilitate IPC for our applications. After using it for a while, it made me wonder how it works and how it was built. Admittedly I did not spend time to fully understand how the framework was built. However, after attending the meetup, it becomes clear to me how the puzzle pieces are connected. Now, with the fresh insights from the meetup, I would like to document what I learned in this and upcoming articles.

Linux Inter-Process Communication

Linux provides facilities to enable processes and threads to exchange data with one another and to synchronize their actions. In [2], the author describes the communication facilities available on Linux: Pipe, FIFO, Message Queue, stream socket, shared memory, etc.

I won’t discuss all available facilities in this article and focus only on FIFO, Message Queue, and Stream Socket. I think these are the options commonly used in real world applications. These are also the ones presented in the meetup.

FIFO

FIFO (First-In First-Out) is a mechanism to allow a process to communicate with another process. It operates as a un-directional data channel. A process writes data into the FIFO and another process reads the data from it. This mechanism is suitable in a simple producer-consumer scenario. In order to allow bidirectional communications, two FIFOs will be needed.

A FIFO can be created by calling mkfifo given the path name. Upon created, the FIFO can be found in the file system.

#include <sys/stat.h>

int mkfifo(const char *pathname, mode_t mode);

Message Queue

Message Queue provides a more structured form of communication between two processes. It allows messages which consist of a numeric type plus a body containing data, to be exchanged. Hence, the message boundaries can be preserved between calls.

Compared to FIFO, message queue supports asynchronous messaging. The messages sent by the producer can be queued, and the receiver can retrieve the messages later.

The Message Queue APIs are as follows.

#include <fcntl.h> /* Defines O_* constants */
#include <sys/stat.h> /* Defines mode constants */
#include <mqueue.h>

mqd_t mq_open(const char *name, int oflag, ...
 /* mode_t mode, struct mq_attr *attr */);

int mq_send(mqd_t mqdes, const char *msg_ptr, size_t msg_len,
 unsigned int msg_prio);

ssize_t mq_receive(mqd_t mqdes, char *msg_ptr, size_t msg_len,
 unsigned int *msg_prio);

int mq_close(mqd_t mqdes);

int mq_unlink(const char *name);

Socket

Socket is a method of IPC that allow data to be exchanged bidirectionally either on the same host (UNIX Domain Socket) or on different hosts connected by a network (IPv4 or IPv6). Every sockets implementation provides at least two types: stream and datagram sockets. In the Internet domain, the stream socket uses TCP and the datagram socket uses UDP. The first one is reliable while the second one is not.

The Linux Programming Interface book by Michael Kerrisk shows the overview of system calls used in both types of sockets.

The main advantage of sockets is it accepts multiple client connections on a single server socket. This is useful in server environments where scalability and simultaneous connections are important.

As you can guess, sockets are commonly used in most real-world applications. They provide a robust solution when you need both the reliability of structured communication and the flexibility of managing multiple concurrent connections.

Conclusion

In this article, I have shown some of the commonly used Linux mechanisms for inter-process communication. In the upcoming articles, we will see how to implement IPC using UNIX domain socket and how to use epoll system call to allow asynchronous communication calls.

References

[1] Exploring Linux API slide by Henrique Marks.

[2] The Linux Programming Interface by Michael Kerrisk.