- XFS wasn’t *really* being formatted optimally for the RAID stripe size
- XFS wasn’t being mounted with the inode64 option which means that all of the inodes are kept in the first 2TB. (Side note: inode64 option is default in newer kernels but not on CentOS 6’s 2.6.32)
- Single threaded testing isn’t entirely accurate because although replication is single threaded, the writes are collected in InnoDB and then writes it to disk using multiple threads governed by innodb_write_io_threads.
Armed with new data, I have – for real – the last round of testing.
To keep things a bit simpler, I will be comparing each file system on 2TB and 27TB, with 4 threads, which matches the default value for innodb_write_io_threads in MySQL 5.5.FS RAID Size Mount Options Transfer/s Requests/s Avg/Request 95%/Request xfs 10 2T noatime,nodiratime,nobarrier,inode64 62.588Mb/sec 4005.66 0.88ms 0.03ms ext4 10 2T noatime,nodiratime,nobarrier 58.667Mb/sec 3754.66 0.87ms 0.19ms FS RAID Size Mount Options Transfer/s Requests/s Avg/Request 95%/Request xfs 10 27T noatime,nodiratime,nobarrier,inode64 64.47Mb/sec 4126.06 0.84ms 0.02ms ext4 10 27T noatime,nodiratime,nobarrier 49.379Mb/sec 3160.26 1.06ms 0.24ms
XFS finally wins out clearly over EXT4. XFS being dramatically slower on 27T earlier really shows how much the worse the performance between inode32 and inode64 is and explains why it was that much better on 2T. Fixing the formatting options pushed XFS over the top easily.
All that’s left to do is setup multiple instances until replication can’t keep up anymore.
PlanetMySQL Voting: Vote UP / Vote DOWN