Transport-level Testing of NVMe Devices Using VFIO

Explore transport-level testing of NVMe devices using VFIO in this comprehensive conference talk. Dive into the intricacies of NVMe architecture, including submission queue entries, command queues, PCI configuration space, and NVMe controller registers. Learn about setting up admin queues, queue doorbells, PCI doorbells, and completion queue entries. Understand polling techniques, interrupt handling, and freeing completion queue entries. Delve into low-level NVMe operations and practical implementation. Discover the Virtual Function I/O Framework and I/O Memory Management Unit. Examine VFIO's boilerplate requirements and NVMe-specific functionality. Gain insights into emulation, VFIO-based NVMe controller initialization, and the Identify Controller process. Master the submission of commands, queue management, and completion handling. Explore the Controller Memory Buffer and its implications for NVMe device testing.

Intro
Outline
NVMe in N Slides
Submission Queue Entry - 64 bytes
Command Queue
PCI Configuration Space
NVMe Controller Registers (MBAR)
Setting up the Admin Queues
Queue Doorbells
PCI Doorbells
Completion Queue Entries - 16 bytes
Polling for "phase" change
Interrupts
"Free'ing" Completion Queue Entries
Low-level NVMe
Putting the Theory into Practice
How Low Can You Go?
Virtual Function I/O Framework
I/O Memory Management Unit
VFIO is boiler-plate heavy
libvfio-NVMe specific functionality
Emulating this stuff
VFIO-based NVMe Controller Initialization
Identify Controller
Submitting command and kicking the queue
Wait for completion and ack
The Controller Memory Buffer
Rounding up