lighty's life

lighty developer blog

Linux AIO and Large Files

The benchmarks only showed results for small files (100kbyte). Time to add larger files to the pool and talk about the chunk-size.

I just push all the work to the kernel and hope that it does it right.
Currently I allow 64 jobs to be pushed to the kernel. Kernel threads are
more light-weight that “real” threads.

Currently I’m working on a posix AIO version. On linux that is using
threads to handle the read(), let’s see how that works out.

I did a third benchmark round against 1000 10Mbyte files. tibco @ IRC is
running a flv-site in china and said that their files are around 12-17Mb.

Client was a win2003-amd64, dual core box connected via Intel Pro/1000
to the server [raid1 … as before].

linux-aio-sendfile: 52Mbyte/s [reading 1Mbyte chunks]

avg-cpu: %user %nice %system %iowait %steal %idle
1.80 0.00 46.20 13.40 0.00 38.60

linux-aio-sendfile: 55Mbyte/s [reading 768kbyte chunks]

avg-cpu: %user %nice %system %iowait %steal %idle
2.99 0.00 56.37 4.58 0.00 36.06

linux-aio-sendfile: 58Mbyte/s [reading 512kbyte chunks]

avg-cpu: %user %nice %system %iowait %steal %idle
1.40 0.00 62.67 5.39 0.00 30.54

linux-aio-sendfile: 54Mbyte/s [reading 384kbyte chunks]

avg-cpu: %user %nice %system %iowait %steal %idle
5.18 0.00 55.38 1.99 0.00 37.45

linux-aio-sendfile: 21Mbyte/s [reading 256kbyte chunks]

avg-cpu: %user %nice %system %iowait %steal %idle
21.00 0.00 28.60 0.80 0.00 49.60

Compared to:

linux-sendfile: 30Mbyte/s

avg-cpu: %user %nice %system %iowait %steal %idle
1.20 0.00 22.20 71.00 0.00 5.60


No matter what, large files or small files, when you disk start to suffer from seeking around AIO will give you, at least in my setup, 80% more throughput.