Benchmarking a VPS Honestly: sysbench, iperf and Caveats

Everyone wants the fastest VPS, and benchmarks are how people argue about it. But an honest benchmark is harder than it looks, because on a shared host every number is a snapshot of one moment of contention, one load pattern, and one filesystem state. Learn what each tool measures, what it misses, and how to get numbers that still hold up next week.

Benchmark the resource you care about

A single composite score hides everything. Split the question: is the workload CPU-bound, memory-bound, disk-bound, or network-bound? Then benchmark that resource directly. sysbench covers CPU, memory, and disk with separate modes, and iperf measures network in isolation.

# CPU performance, single-thread
sysbench cpu --threads=1 run
# RAM on a 4 GB block
sysbench memory --memory-block-size=4K --memory-total-size=4G run
# a quick synchronous read latency scan
fio --name=read --rw=randread --bs=4k --size=1G --numjobs=1   --ioengine=libaio --direct=1 --runtime=20 --time_based

Match the benchmark parameters to your real workload. A web cache wants random 4 KiB reads. A database wants random 16 KiB reads with fsync. A media server wants sequential throughput. Running a default test gives you a number that describes neither your disk nor your workload.

Network is a two-sided measurement

iperf reports throughput only as fast as the slower end allows, so the client matters as much as the server. Test both directions and several parallel streams, and run iperf to a machine outside your own infrastructure so you are not benchmarking a single loopback.

# server side
iperf3 -s
# client side, ten seconds, four streams
iperf3 -c <server-ip> -t 10 -P 4
iperf3 -c <server-ip> -t 10 -R

The reverse direction flag measures inbound, which often differs from outbound on asymmetric plans. Run it a few minutes apart to dodge a busy neighbour, because a single iperf run on a shared host is a lottery draw, not a spec sheet.

Make every run reproducible

A benchmark that cannot be re-run identically is worthless as a baseline. Copy-paste a benchmark into a fresh instance and it quietly changes: different queue depth, different block size, cache already warm or already flushed. The discipline is to script the exact command, the exact parameters, and the exact cold/warm state, then run it three times and record the median and the spread.

# pin the parameters, flush the cache, take three samples, report the median
sync; echo 3 > /proc/sys/vm/drop_caches
for i in 1 2 3; do
  fio --name=t$i --rw=randread --bs=16k --size=512M --direct=1     --ioengine=libaio --iodepth=16 --output-format=json 2>/dev/null
done | grep -E '"(read)"|"iops"|"lat"'

Flushing the cache with drop_caches before a direct test is not strictly needed when direct=1 is set, but it removes doubt. Record the date, the instance size, and the kernel version alongside the numbers, because a benchmark compared across different kernels or plans explains more than the raw IOPS ever will. A reproducible measurement beats a fast-sounding one every time.

The variance you should expect

Shared-host benchmarks carry real noise. Two good runs can differ by thirty percent purely from neighbour load, and chasing that difference as a product win is a trap. Establish your expected spread first by running the same benchmark three times at the same hour; then any change outside that spread is meaningful, and anything inside it is noise you ignore.

The caveats that make numbers lie

• **Caching**: the first disk read comes from page cache and looks impossibly fast. Use direct I/O and cold files.
• **Neighbour noise**: another tenant can inflate or deflate a single run. Repeat three times and keep the median.
• **Aggregate scores**: they average CPU, disk, and network into one number that cannot be reproduced in any single workload.
• **Misleading units**: MB/s on a cold random test tells you nothing about a hot sequential workload. Always read the IOPS and latency columns.

Benchmark like you mean it

Pick metrics that match the application, run them at the same time of day across several days, and save the raw output so you can compare after changes. A benchmark you cannot reproduce is a screenshot, not a measurement. The point of benchmarking is a baseline for regression, not a bragging number.

Takeaway

Match the test to the workload, use direct I/O, run both network directions from an outside client, and repeat for a median. When the numbers matter, run them on a fresh instance. Netbay deploys Linux VPS in under 60 seconds at netbayhosts.in so you can benchmark on clean hardware.