epoll vs io_uring: which Linux async I/O wins?

Key Takeaways

• io_uring reduces syscall overhead by batching operations through shared memory ring buffers
• epoll requires two syscalls per I/O event (epoll_wait + read/write), creating overhead at scale
• io_uring's SQPOLL mode can achieve near-zero syscalls during steady state

A developer and their students rebuilt a reverse proxy three times before landing on io_uring, Linux's modern async I/O interface. Their journey, documented in a post now circulating on Hacker News, illustrates why epoll, the 23-year-old workhorse, struggles at scale. The culprit: syscall overhead that io_uring sidesteps entirely.

The project, called TinyGate, started as a worker-based reverse proxy. It worked, but couldn't touch nginx or haproxy in benchmarks. Version two used epoll and saw dramatic gains. Version three switched to io_uring, requiring a complete rewrite. The performance difference justified the pain.

Why does epoll create syscall overhead?

epoll landed in the Linux kernel in 2002. It tells your application when I/O is possible, not when it's done. This distinction matters. When epoll_wait returns, you still need to call read() or write() yourself. That's two syscalls per I/O event, plus the initial epoll_ctl registration. Each syscall triggers a context switch between user and kernel mode.

At low connection counts, this overhead is negligible. At thousands of concurrent connections, it adds up fast. Every context switch burns CPU cycles and cache coherency.

How io_uring changes the model

io_uring arrived in kernel 5.1 (2019), seventeen years after epoll. Instead of a readiness model, it uses a completion model. The kernel tells you when I/O is done, not when it's possible. No polling loop. Far fewer syscalls.

The architecture uses two ring buffers in shared memory between your application and the kernel. You submit operations to the submission queue (SQ). The kernel posts results to the completion queue (CQ). Both live in memory both sides can access directly.

By default, you call io_uring_enter() to tell the kernel "check the submission queue." But one call can submit a batch of operations and reap a batch of completions. Compare that to epoll's one syscall pair per operation.

SQPOLL: near-zero syscalls at steady state

io_uring offers an aggressive optimization: IORING_SETUP_SQPOLL. This spins up a dedicated kernel thread that continuously polls the submission queue. Your application writes submissions to shared memory; the kernel thread picks them up without any syscall from your side.

The tradeoff is CPU cost. That kernel thread burns cycles polling. For I/O-heavy workloads where syscall overhead dominates, the math often works out. For lighter loads, the spinning thread wastes more than it saves.

What the code looks like

The original post includes C examples for both approaches. The epoll version creates an instance with epoll_create1(), registers a file descriptor with epoll_ctl(), then blocks on epoll_wait(). When it returns, you handle the event with standard read/write calls.

io_uring code (using liburing, the userspace helper library) sets up the ring once, then submits operations by writing to the submission queue. Completions arrive in the completion queue without additional syscalls per operation.

The architectural shift is significant. With epoll, your application orchestrates I/O timing. With io_uring, the kernel handles that work. You describe what you want done; the kernel tells you when it's finished.

When to use which

For systems running kernel 5.1 or newer, io_uring is usually the better choice for network servers handling many concurrent connections. The syscall savings compound as connection count rises.

epoll still makes sense in a few cases. Older kernels that predate io_uring. Applications where I/O isn't the bottleneck. Codebases where the rewrite cost outweighs the performance gain. Simple utilities that don't need the complexity.

The TinyGate team found the rewrite painful but worthwhile. Their benchmarks still trailed nginx and haproxy, both of which have decades of optimization. But the gap narrowed.

Frequently Asked Questions

What kernel version is required for io_uring?

Linux kernel 5.1 or newer, released in 2019. Feature completeness improved significantly in subsequent versions, so 5.10+ is recommended for production use.

Does io_uring replace epoll completely?

Not entirely. epoll remains useful for simpler applications, older kernels, and cases where I/O isn't the bottleneck. io_uring adds complexity that isn't always justified.

What is SQPOLL in io_uring?

SQPOLL (IORING_SETUP_SQPOLL) creates a dedicated kernel thread that polls the submission queue continuously. This eliminates syscalls during steady-state operation but consumes CPU cycles for the polling thread.

Why does epoll require two syscalls per I/O event?

epoll uses a readiness model. epoll_wait tells you I/O is possible, then you must call read() or write() to actually perform it. That's two syscalls, each requiring a user-kernel context switch.

Is io_uring harder to implement than epoll?

Yes. io_uring has a steeper learning curve and requires understanding ring buffer mechanics. Libraries like liburing simplify the interface, but the conceptual model is more complex than epoll's.

Source: Hacker News: Best