linux AIO and large files

jan — Tue, 14 Nov 2006 12:27:00 +0000

The benchmarks only showed results for small files (100kbyte). Time to add larger files to the pool and talk about the chunk-size.

I just push all the work to the kernel and hope that it does it right. Currently I allow 64 jobs to be pushed to the kernel. Kernel threads are more light-weight that “real” threads.

Currently I’m working on a posix AIO version. On linux that is using threads to handle the read(), let’s see how that works out.

I did a third benchmark round against 1000 10Mbyte files. tibco @ IRC is running a flv-site in china and said that their files are around 12-17Mb.

Client was a win2003-amd64, dual core box connected via Intel Pro/1000 to the server [raid1 … as before].

linux-aio-sendfile: 52Mbyte/s [reading 1Mbyte chunks]

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.80    0.00   46.20   13.40    0.00   38.60

linux-aio-sendfile: 55Mbyte/s [reading 768kbyte chunks]

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           2.99    0.00   56.37    4.58    0.00   36.06

linux-aio-sendfile: 58Mbyte/s [reading 512kbyte chunks]

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.40    0.00   62.67    5.39    0.00   30.54

linux-aio-sendfile: 54Mbyte/s [reading 384kbyte chunks]

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           5.18    0.00   55.38    1.99    0.00   37.45

linux-aio-sendfile: 21Mbyte/s [reading 256kbyte chunks]

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          21.00    0.00   28.60    0.80    0.00   49.60

Compared to:

linux-sendfile: 30Mbyte/s

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.20    0.00   22.20   71.00    0.00    5.60

Summary

No matter what, large files or small files, when you disk start to suffer from seeking around AIO will give you, at least in my setup, 80% more throughput.

lighty's life: Tag async

linux AIO and large files

Summary