Configuration: - Three disks sd[bcd] - mdadm 3.1.5 providing RAID-0 device /dev/md0 on these (partitionless) - XFS file system created on /dev/md0 and mounted on /mnt/v1 - NFS export created (rw,no_root_squash,async,noatime) for /mnt/v1 - VMware ESXi connectd to NFS export via dedicated GbE - Windows 2003 Server guest on ESXi with volume on the NFS datastore - Aligned partition created in guest - IOMeter used to measure performance thus: - 8K random, 4K aligned, 16 outstanding IOs, 8GB test file, 5 minute duration Results: - 100% read test performs as expected (c.1200 IOPS) - 100% write test performs as expected (c.1200 IOPS) - 70% read 30% write test performs badly c.400 IOPS Monitoring the disks using iostat -xk /dev/sd[bcd] 5 60 during this test I note that the queue depth is at or close to 1 for each disk in the mixed read/write workload. For either 100% read or 100% write, the queue depth is as expected, in line with the IOMeter test parameters. The same results are obtained using Debian (2.6.26), Ubuntu 10.10 (2.6.35) and Fedora 8 (2.6.26). Reason for Bug Filing: Using JFS, ext3 or ext4 instead of XFS in this configuration, the mixed workload slow-down is not observed (and iostat in those cases shows no queue depth throttling). The effect is not present operating direcly with a single device (i.e. without mdadm).
Can you simplify your test configuration down to the simplest config that shows the regression? e.g. is NFS necessary, or can you reproduce the problem directly on the filesystem? If NFS is necessary, can you reproduce the problem directly on the NFS export without VMware in the picture?
Many thanks for the relpy. I'm struggling to test on the FS directly IOMeter always runs with a queue depth of exactly 1 on Linux (removing O_DIRECT from IOTargetDisk.cpp didn't help). I tried mapping to the export directly from a Windows box, but again IOMeter doesn't see the mapped drive (via Windows SFU NFS client). If another tool can be suggested that will test 8K mixed read/write random workload with variable queue depths I would be more than happy to look further. Many thanks indeed.