I have personally found that making even the most primitive efforts at single-writer principle and batching IO in your software can make many orders of magnitude difference.
Saturating an NVMe drive with a single x86 thread is trivial if you change how you play the game. Using async/await and yielding to the OS is not going to cut it anymore. Latency with these drives is measured in microseconds. You are better off doing microbatches of writes (10-1000 uS wide) and pushing these to disk with a single thread that monitors a queue in a busy wait loop (sort of like LMAX Disruptor but even more aggressive).
Thinking about high core count parts, sacrificing an entire thread to busy waiting so you can write your transactions to disk very quickly is not a terrible prospect anymore. This same ideology is also really useful for ultra-precise execution of future timed actions. Approaches in managed lanaguages like Task.Delay or even Thread.Sleep are insanely inaccurate by comparison. The humble while(true) loop is certainly not energy efficient, but it is very responsive and predictable as long as you dont ever yield. What's one core when you have 63 more to go around?
Isn't the use or non-use of async/await a bit orthogonal to the rest of this?
I'm not an expert in this area, but wouldn't it be just as lightweight to have your async workers pushing onto a queue, and then have your async writer only wake up when the queue is at a certain level to create the batched write? Either way, you won't be paying the OS context switching costs associated with blocking a write thread, which I think is most of what you're trying to get out of here.
Right, I agree. I'd go even further and say that async/await is a great fit for a modern asynchronous I/O stack (not read()/write()). Especially with io_uring using polled I/O (the worker thread is in the kernel, all the async runtime has to do is check for completion periodically), or with SPDK if you spin up your own I/O worker thread(s) like @benlwalker explained elsewhere in the thread.
Very interesting. I'm currently desiging and building a system which has a separate MCU just for timing accurate stuff rather than having the burdon of realtime kernel stuff, but I never considered just dedicating a core. Then I could also use that specifically to handle some IO queues too perhaps, so it could do double duty and not necessarily be wasteful. Thanks... now I need to go figure out why I either didn't consider that - or perhaps I did and discarded it for some reason beyond me right now. Hmm... thought provoking post of the day for me
The authors of the article I linked to earlier came to the same conclusions. And so did the SPDK folks. And the kernel community (or axboe :)) when coming up with io_uring.
I'm just hoping that we will see software catching up.
>Latency with these drives is measured in microseconds.
For context and to put numbers around this, the average read latency of the fastest, latest generation PCI 4.0 x4 U.2 enterprise drives is 82-86µs, and the average write latency is 11-16µs.
Saturating an NVMe drive with a single x86 thread is trivial if you change how you play the game. Using async/await and yielding to the OS is not going to cut it anymore. Latency with these drives is measured in microseconds. You are better off doing microbatches of writes (10-1000 uS wide) and pushing these to disk with a single thread that monitors a queue in a busy wait loop (sort of like LMAX Disruptor but even more aggressive).
Thinking about high core count parts, sacrificing an entire thread to busy waiting so you can write your transactions to disk very quickly is not a terrible prospect anymore. This same ideology is also really useful for ultra-precise execution of future timed actions. Approaches in managed lanaguages like Task.Delay or even Thread.Sleep are insanely inaccurate by comparison. The humble while(true) loop is certainly not energy efficient, but it is very responsive and predictable as long as you dont ever yield. What's one core when you have 63 more to go around?