Measure the path.
Tune the host.

01 / Establish a baseline

Measure before you tune.

Record the Linux distribution and kernel, adapter model, link rate, connection path, round-trip time, application workload, and the time of the test. Compare one TCP stream with several streams and test both directions. A many-stream total can be useful, but it does not prove that one connection can reach that rate.

uname -r
ip -br address
ip route get 1.1.1.1

iperf3 --version
iperf3 -c iperf3.stealth.net -P 1 -t 10
iperf3 -c iperf3.stealth.net -P 8 -t 10
iperf3 -c iperf3.stealth.net -P 8 -t 10 -R

These examples generate network traffic. Coordinate tests on production links: a 10-second test at a sustained 10 Gbps transfers roughly 12.5 GB in one direction, before overhead. Public test servers and remote links may be busy or rate-limited. A server on the same host can exercise the local stack rather than the external connection.

During each test, record receiver throughput, retransmissions, CPU load, interface counters, and latency under load where appropriate. Then repeat a representative large-file transfer using your actual application, storage, encryption, and destination; record file size and elapsed time. Check disk throughput and utilization at both ends. If iperf3 improves but the file transfer does not, investigate storage, application processing, encryption, and the remote system before changing more network settings. Repeat comparable tests; one high or low result is not a capacity guarantee. See our speed testing guide for test preparation and limits.

02 / Inspect

Look at the connection while it is active.

Replace eth0 with the interface selected for your test. The following commands inspect settings or counters; they do not change them. Tools, fields, and permissions vary across systems and drivers.

ss -tin
nstat -asz
sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem
sysctl net.ipv4.tcp_moderate_rcvbuf

ip -s link show dev eth0
ethtool eth0
ethtool -i eth0
ethtool -S eth0
ethtool -l eth0
ethtool -g eth0
ethtool -k eth0

03 / Bandwidth and delay

How much data can be in flight?

The bandwidth-delay product is the rate multiplied by the round-trip time. It is a useful estimate of how much data must be in flight to sustain a rate across a path, assuming the path and endpoints can support it.

For multiple streams, the total in-flight requirement is shared across the flows; it is not necessarily needed in full for every socket. Record the actual path RTT and the rate you expect a particular flow to sustain.

04 / Socket memory

Understand the buffer limits.

net.ipv4.tcp_rmem and net.ipv4.tcp_wmem each contain three values: a minimum, an initial default, and an automatic tuning maximum, in bytes. With receive autotuning enabled, Linux can adjust TCP receive buffers to match observed conditions up to the applicable limit. TCP send buffering is also subject to the kernel’s configuration and the application.

net.core.rmem_max and net.core.wmem_max constrain normal application requests for socket buffers through SO_RCVBUF and SO_SNDBUF. They are distinct from the TCP autotuning triplets; do not assume that changing one value alone changes every socket in the same way. Explicit socket options can disable automatic sizing for that socket. Privileged override options, accounting, kernel version, and other memory limits affect the result.

A configured maximum is a limit, not a fixed allocation for every connection and not the same thing as the TCP receive window or congestion window. Linux accounts for socket-buffer space and overhead; inspect the active socket and the application’s behavior rather than assuming a one-to-one mapping. High limits can permit substantial aggregate memory consumption across many active sockets.

05 / A controlled example

Try one change, with a way back.

Consider a dedicated test host where evidence suggests the TCP receive autotuning maximum is too small for a long path. For this illustration only, suppose its original setting is 4096 131072 6291456 and your team has chosen to test a 32 MiB maximum. This is not a general recommendation or a setting to copy onto an unrelated server.

During an approved test window, capture the original values in the same shell and save them in your change record. Apply the temporary setting, measure a new connection, and restore the original value from that shell:

if original_tcp_rmem="$(sysctl -n net.ipv4.tcp_rmem)" &&
   printf '%s\n' "$original_tcp_rmem" | awk '
     NF == 3 && $1 == 4096 && $2 == 131072 &&
     $3 ~ /^[0-9]+$/ && $3 >= 4096 && $3 < 33554432 {ok=1}
     END {exit !ok}'
then
  printf 'Original tcp_rmem: %s\n' "$original_tcp_rmem"
  sudo sysctl -w 'net.ipv4.tcp_rmem=4096 131072 33554432'
else
  printf '%s\n' 'No change: the current values do not match this example.'
fi

After testing, restore the original setting in the same shell, whether or not performance improved. Verify the restored values against your change record:

if [ -n "${original_tcp_rmem:-}" ] &&
   [ "$(sysctl -n net.ipv4.tcp_rmem | awk '{$1=$1; print}')" = "4096 131072 33554432" ]; then
  sudo sysctl -w "net.ipv4.tcp_rmem=$original_tcp_rmem"
else
  printf '%s\n' 'No restore: check the saved and current values.'
fi
sysctl net.ipv4.tcp_rmem

The sysctl -w change is temporary; it does not edit a configuration file and can be lost on reboot or overwritten by another manager. Existing sockets may not behave like newly established sockets. If a justified change is approved for persistence, follow the distribution’s configuration precedence and change-control process, then verify the effective value after reboot. Do not apply a collection of large example settings without testing each one.

A second buffer example

Give the sender room, too.

If evidence shows the sending host’s automatically tuned TCP send buffer is constraining a transfer, the IT team can test its maximum separately. For example, suppose tcp_wmem reads 4096 16384 4194304 and a 64 MiB test ceiling is appropriate for this host and path. This is a ceiling in bytes, not memory reserved for every socket or a guaranteed throughput. The application, TCP memory pressure, kernel limits, and other endpoint still matter. An application that explicitly sets SO_SNDBUF disables automatic send-buffer sizing for that socket and is subject to the applicable net.core.wmem_max limit instead; test the application’s actual behavior.

In the same terminal, capture the original values and record them. This example runs only for that exact starting triplet and stops if another value is present:

original_tcp_wmem="$(sysctl -n net.ipv4.tcp_wmem | awk '{$1=$1; print}')"
printf 'Original tcp_wmem: %s\n' "$original_tcp_wmem"
if [ "$original_tcp_wmem" = "4096 16384 4194304" ] &&
   [ "$(sysctl -n net.ipv4.tcp_wmem | awk '{$1=$1; print}')" = "$original_tcp_wmem" ]; then
  sudo sysctl -w 'net.ipv4.tcp_wmem=4096 16384 67108864'
else
  printf '%s\n' 'No change: inspect the current setting.'
fi

Verify with sysctl net.ipv4.tcp_wmem, then compare a new sender-side test with the baseline. After the trial, restore only if the value still matches the trial setting:

if [ "${original_tcp_wmem:-}" = "4096 16384 4194304" ] &&
   [ "$(sysctl -n net.ipv4.tcp_wmem | awk '{$1=$1; print}')" = "4096 16384 67108864" ]; then
  sudo sysctl -w "net.ipv4.tcp_wmem=$original_tcp_wmem"
else
  printf '%s\n' 'No restore: check the saved and current values.'
fi
sysctl net.ipv4.tcp_wmem

Compare the result with the saved original. The shell variable is temporary; retain the original in the change record. These checks do not replace coordination with configuration management or prevent a concurrent change.

06 / The interface & CPU

Check where packets are processed.

At 10 Gbps and above, one busy CPU, an overloaded receive queue, a saturated PCIe path, or a remote host may limit throughput even when average system CPU usage appears low. Check the adapter’s negotiated link speed and PCIe slot generation and lane width against the manufacturer’s requirements; a fast port alone does not establish host capacity. Use per-core monitoring such as mpstat -P ALL 1 when available, and compare interface and driver counters before and after a test.

Receive-side scaling can distribute different flows across hardware queues; it does not guarantee one flow will use every core. Queue count, interrupt placement, driver support, and NUMA locality influence performance. Inspect the actual device and topology first. More queues or a manual interrupt mapping are not automatically better; a service such as irqbalance may also manage placement. For an initial read-only check, compare lscpu -e and cat /sys/class/net/eth0/device/numa_node with the adapter and process placement; a value of -1 means the device’s NUMA node is unknown. Utilities and topology information vary by host. If evidence points to cross-node processing or a busy core, work with your IT team to test supported placement changes and compare the actual file transfer.

Offloads and interrupt coalescing reduce some per-packet work but can change how packet captures and latency appear. Review the driver and application documentation and test changes individually. Jumbo frames require compatible MTU across the relevant path and endpoints; do not assume every Internet path supports them.

07 / Queues & congestion

More buffering is not always better.

net.core.netdev_max_backlog, the adapter’s receive rings, and TCP socket buffers are different queues. Larger queues may help absorb bursts in a measured situation, but they can also use memory, add waiting time, or conceal a bottleneck. Drops and queue behavior need to be interpreted alongside CPU and interface evidence.

Inspect the active queue discipline with tc -s qdisc show dev eth0 and the selected TCP congestion-control algorithm with sysctl net.ipv4.tcp_congestion_control. The configured default queue discipline is not necessarily the one currently active on a particular interface. Algorithms such as CUBIC and BBR have different behavior; do not switch a busy server globally based on a single benchmark. Confirm kernel support, path conditions, and fairness, then test in a controlled environment.

Packet pacing spreads a sender’s traffic over time and can reduce bursts and retransmissions. A controlled test can compare an unpaced baseline with iperf3’s per-stream bitrate pacing; for example, the following limits each of four streams to 1 Gbps, for a nominal aggregate ceiling of 4 Gbps, not a promise of throughput:

iperf3 -c iperf3.stealth.net -P 4 -t 10 -b 1G

Use an approved endpoint and test window. Compare receiver throughput, retransmissions, and latency under load; a deliberately paced test cannot demonstrate an unlimited line rate. iperf3 pacing affects that test, not all applications. For file transfers, check whether the transfer tool offers its own rate or concurrency controls; do not impose a system-wide limit based on one test.

A 100 Gbps test needs capable endpoints, a suitable PCIe and NIC configuration, sufficient CPU and memory, and a path that can sustain the load. Link speed, concurrent tests, and the test server’s limits can dominate the result. If increasing buffers has no measured benefit, restore the original settings and investigate the next constraint.

08 / Before and after

Document the result.

• What real application is slow, in which direction, and under what conditions?
• Which endpoints, interface, source address, route, MTU, RTT, and software versions were tested?
• What did one stream and multiple streams achieve, and what happened to latency and retransmissions?
• Did any interface, queue, CPU, or storage measurement point to a specific constraint?
• Was one setting changed at a time, with the original value and rollback recorded?
• Did representative applications and other services continue to perform well?

For an investigation involving a Stealth service, see preparing for a support request and share timestamps, test paths, and relevant counter changes.

Research / High-speed transfers

What the measurements show.

In a 2026 WLCG preprint, Linux research sites with 25 Gbps or faster adapters tested host tuning for file transfers. One site increased its file-transfer throughput from 60 to 90 Gbps. At another, iperf3 improved from 20 to 90 Gbps while file transfers did not improve because storage was the limiting factor. A third site saw throughput fall after tuning. These are site-specific results, not expected gains for every host.

A 2024 ESnet and AmLight study on 100 Gbps and faster research networks found that Linux version, CPU architecture, pacing, and application-supported zero-copy affected throughput. Its testbed results are not a general tuning profile. Together, the studies make a useful case for measuring the real file transfer alongside a network test and changing one setting at a time.

Related guides

Working with another system?

Explore high-speed network tuning for FreeBSD, macOS, and Windows.