IO_uring Fixed Buffer Versus Non-Fixed Buffer Performance Comparison on NVMe

Background

TL;DR

Overview of IO_uring Read Functions and Fixed Buffer Registration

void io_uring_prep_read(struct io_uring_sqe *sqe, int fd,
void *buf, unsigned nbytes, off_t offset);
void io_uring_prep_read_fixed(struct io_uring_sqe *sqe, int fd,
void *buf, unsigned nbytes,
off_t offset, int buf_index);
void io_uring_prep_readv(struct io_uring_sqe *sqe, int fd,
const struct iovec *iovecs,
unsigned nr_vecs, off_t offset);
int io_uring_register_buffers(struct io_uring *ring, 
const struct iovec *iovecs,
unsigned nr_iovecs)
{
int ret;
ret = __sys_io_uring_register(ring->ring_fd,
IORING_REGISTER_BUFFERS,
iovecs, nr_iovecs);
if (ret < 0)
return -errno;
return 0;
}

Test Parameters

Results

IOPs —Single Process, Single Core, Fixed vs. Non-Fixed: 4k, 32k, 128k

Kernel Profile for 4k Fixed Buffer (QueueDepth=128)
Kernel Profile for 4k Non-Fixed Buffer (QueueDepth=128)

IOPs — Two Processes, Two Cores: 4k

IOPs — Single Process, Single Core, Fixed vs. Vectored: 128k

Conclusion and Recommendations

Use Fixed Buffers if Possible

Non-Fixed Buffers are OK Too

Vectored I/O

Distributed Storage Systems Programmer w/ focus on distributed erasure coding, parallel log structuring, and hierarchical storage.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store