Jots on named pipes (FIFOs in Linuxish)

Major disclaimer

These are pretty random jots that I made while evaluating named pipes as a solution for project. I eventually went for a driver in the kernel for various reasons, so I never got to verify that anything written below is actually correct.

I’ve also written a small post on epoll with named pipes (in Linux, of course) along with a tiny test program.

Linux named pipes (FIFOs)

Section 3.162 of IEEE Std 1003.1-2001, Base Definitions, defines FIFO Special File (or FIFO) trivially, and refers to POSIX IEEE Std 1003.1-2001 for “Other characteristics” related to lseek( ), open( ), read( ), and write( ). In section 3.273 it defines Pipe, and says that it “behaves identically to a FIFO special file”.

From POSIX IEEE Std 1003.1-2001, System Interfaces, Issue 6, line 27153: “When opening a FIFO with O_RDONLY or O_WRONLY set: (…) If O_NONBLOCK is clear, an open( ) for reading-only shall block the calling thread until a thread opens the file for writing. An open( ) for writing-only shall block the calling thread until a thread opens the file for reading.”

Even though the POSIX standard is available for download at a cost of a few hundred dollars, there’s the Open Group Base Specification, which matches it quite well: Base Definitions and System Interfaces.

This is a good source on pipes.

• The buffer for each pipe is 64 kB by default on Linux
• An anonymous pipe (created with pipe() ) is just like a named pipe (i.e. a FIFO special file), only that the node is in pipefs, which only the kernel has access to. There are slight differences in how they’re handled.

Reading the source code (fs/pipe.c)

• This source clearly handles both named and anonymous pipes (the behavior differs slightly).
• There’s F_SETPIPE_SZ and F_GETPIPE_SZ fcntl calls, which clearly allow setting the buffer size. Added in patch 35f3d14dbbc58 from 2010 (v2.6.35).
• fifo_open(): The is_pipe boolean is true if the FIFO is a pipe() (has a magic of PIPE_FS). In other words, if it’s false, it’s a named pipe.
• If a FIFO is opened for read or write (not both), the open() call blocks until there’s a partner, unless O_NONBLOCK is set, in which case the behavior is somewhat messy. The comments imply that this is required by POSIX.
• It’s perfectly fine that a FIFO is opened multiple times both for read and write. Only one reader gets each piece of data, and quite randomly. Writers contribute to the stream independently.
• FIFOs and pipes implement the FIONREAD ioctl() command for telling how many bytes are immediately ready for reading. This is the only ioctl() command implemented however.
• The flags returned by epoll_wait() are as understood from pipe_poll(). Not clear what purpose the list of EPOLL* flags in those calls to wake_up_interruptible_sync_poll() has.

Things to look at:

• What if a file descriptor open for read is closed and there’s no write() blocking on the other side? Is there a way to get a software notification? A zero-length write always succeeds, even if the other side is closed (this case is handled explicitly before the part that produces the EPIPE / SIGPIPE).
• Same in the other direction: A descriptor open for write closes, but there’s no read() to get the EOF. Likewise, a zero-length read() always succeeds, even if the other side is closed (which makes sense, because even if it’s closed, reading should continue until the end).
• Is there a way to wait for the partner without blocking, or must a thread block on it?
• What happens on a close() followed immediately by an open()?
• What about user opening the file in the wrong direction?

open() with O_NONBLOCK? (not)

It’s tempting to open a FIFO with O_NONBLOCK, so there needs not to be a thread blocking while waiting for the other side to be opened. POSIX IEEE Std 1003.1-2001, System Interfaces, Issue 6,says in the part defining open(), page 836:

When opening a FIFO with O_RDONLY or O_WRONLY set:

• If O_NONBLOCK is set, an open( ) for reading-only shall return without delay. An open( ) for writing-only shall return an error if no process currently has the file open for reading.
• If O_NONBLOCK is clear, an open( ) for reading-only shall block the calling thread until a thread opens the file for writing. An open( ) for writing-only shall block the calling thread until a thread opens the file for reading.

In the list of error codes for open(), ENXIO is bound (along with another irrelevant sceniario) to: O_NONBLOCK is set, the named file is a FIFO, O_WRONLY is set, and no process has the file open for reading.

Linux’ implementation of FIFOs follows this exactly.

This rules out using O_NONBLOCK for opening a file for write — it will simply not work. As for opening a file for read, it will work, but the epoll() call won’t wake up the process before there is data to read. Opening the other side only doesn’t generate any event.

Windows named pipes

The basic reading: Microsoft’s introduction and a list of relevant API functions.

General notes:

• The concept of named pipes in Windows seems to resemble UNIX domain sockets in that it’s formed with a client / server terminology. Unlike domain sockets, the client may “connect” with a plain file open() (actually CreateFile and variants). Hence Windows’ named pipes can be truly full duplex between two processes.
• But named pipes can also be accessed remotely. The permissions need to be set explicitly to avoid that.
• The client’s opening of a named pipe is known by the return of ConnectNamedPipe (or a respective event in OVERLAPPED mode).
• Pipe names are not case-sensitive.
• A pipe is created (by the server) with access mode PIPE_ACCESS_INBOUND, PIPE_ACCESS_OUTBOUND or PIPE_ACCESS_DUPLEX, indicating its direction, as well as the number of instances — the number of times the pipe can be opened by clients. There’s PIPE_TYPE_BYTE and PIPE_TYPE_MESSAGE types, the former creating a UNIX-like stream, and the second treats writes to the pipe as atomic messages.
• A named pipe ceases to exist when its number of instances goes to zero. It’s an object, not a file. Hence for a sensible application where a single process generates the named pipe, they vanish when the process terminates.
• Use PIPE_WAIT even when non-blocking behavior is required. Don’t use PIPE_NOWAIT flag on creation for achieving non-blocking use of the pipe. Overlapping access is the correct tool. The former is available to allow compatibility with some Microsoft software accident.
• If a client needs to connect to a pipe which has all its instances already connected, it may wait for its turn with WaitNamedPipe.
• When finishing a connection with a client, FlushFileBuffers() should be called to flush pending written data (if the client didn’t close the connection first) and then DisconnectNamedPipe().
• The suitable mode for working is overlapped I/O. This is the official example.
• There’s a scary remark on this page and this claiming that named pipes are effectively useless for IPC, and that the object’s name has changed, and this is also discussed here. It seems however that this remark doesn’t relate to anything else than UWP apps. Or else it wouldn’t have cause the vulnerability mentioned here, and how could Docker use it this way?

Windows named pipes: Detecting disconnection of client

The word out there is that a disconnection can’t be detected unless there’s an outstanding ReadFile or WriteFile request. This holds true in particular for TCP sockets. So first try this: Make sure there’s only one instance, and let the server call WaitNamedPipeA as soon as a connection is made. This call will hopefully return when the real client disconnects. This will not work if Windows wants the server to disconnect as well before considering the pipe instance vacant enough. It might, because the client is allowed to connect before the server. It all depends on when Windows decrements the reference count and cleans up.