Disk I/O on a VPS: IOPS, Latency and Filesystem Tuning

Disk I/O is where database and cache performance live or die, and it is also the metric most hostile to guesswork. On a virtualised host you are not benchmarking a physical disk you own; you are measuring a slice of solid-state storage shared with other tenants, behind a scheduler and a cache. To tune it, first measure IOPS and latency at the block level with direct I/O, then align the filesystem to the pattern the workload actually uses.

Block-level truth: IOPS and latency

The two numbers that matter are random IOPS and latency, not raw sequential throughput. Sequential MB/s is interesting for backups; a database does scattered reads. fio with direct I/O and a low queue depth gives a latency, while a deeper queue gives the IOPS ceiling.

# latency-sensitive random read, depth 1
fio --name=lat --rw=randread --bs=4k --size=1G --direct=1   --ioengine=libaio --iodepth=1 --time_based --runtime=15
# throughput-oriented random read, depth 32
fio --name=iops --rw=randread --bs=16k --size=1G --direct=1   --ioengine=libaio --iodepth=32 --time_based --runtime=15   --numjobs=4

The direct=1 flag bypasses page cache, so you measure the device, not your cache. Lower the iodepth if you care about single-request latency; raise it for peak IOPS. Real applications sit somewhere between, so test the depth your database is configured for.

Measuring from inside the OS

Block tools see the host view; for the guest experience use the process view. iostat shows device throughput and queue behaviour, while the latency figures land in the application metrics. Run iostat under your actual load to see if the device queue stays shallow or backs up.

iostat -x 1 10
# watch await (service latency) and %util; pick the workload column
dstat -d 1 10

A device with high %util and a growing await is saturated. But on a shareable host, a high await can mean neighbour contention on solid-state storage rather than your own queue depth. Correlate with steal-style fluctuations to separate your load from theirs.

Where the write cost really hides

Random read latency is easy to measure but writes carry a hidden tax: durability. When a database calls fsync, the storage layer must make sure that data is actually safe, and that flush is where write latency accumulates. On solid-state storage the physical media is fast, but the file system and the database journal negotiate how much must be persisted per write. Measure the cost honestly with an fsync-heavy workload, because that is what a real database issues on every committed transaction.

# simulate the database write pattern: random writes with a forced flush
fio --name=wr --rw=randwrite --bs=4k --size=512M --direct=1   --ioengine=libaio --iodepth=1 --fsync-on-close=1   --time_based --runtime=15

The latency you read from that run is closer to a live database than a throughput benchmark is. If committed-write latency is your pain, the levers are fewer: tune the journal, raise the group commit size, or consider that your durability requirement may not need per-transaction flushes. Whatever you choose, measure the committed-write latency, not the raw write speed, before and after.

Filesystem tuning that moves the needle

The two biggest filesystem levers are journaling behaviour and alignment. For a database you may trade a little write durability for a lot of write latency; for general workloads you want the scheduler to prefer throughput over small-write panic.

# inspect current mount options
mount | grep -E ' / | /var'
# remount with relaxed caches and see what changes (test on a disposable instance)
mount -o remount,noatime /

The noatime option removes a metadata write on every read, which quietly drops write pressure on heavily-read files. Whether to lower journal commits is a durability trade you make consciously, not something to flip casually.

The durability trade, made explicit

Reducing journal commit frequency on ext4 cuts write latency by batching commits, at the cost of losing more committed-but-unflushed data on a hard crash. For a database that already journals its own writes, the filesystem journal adds little; for a filesystem holding ephemeral cache, the trade is usually worth it. Decide with your real RPO, never with a default.

Takeaway

Measure direct I/O first, at the queue depth your app uses, and read the latency columns rather than the throughput headline. Then tune the filesystem to the pattern. For workloads that want low-latency solid-state storage, Netbay provisions instances with fast local solid-state disks; spin one up and benchmark it yourself at netbayhosts.in.