FreeBSD networking.
Measure, then tune.

01 / Establish a baseline

Measure the traffic you care about.

Record the FreeBSD release, adapter, negotiated speed, topology, path, and application. Is the concern one long-distance transfer, many simultaneous users, packet loss, or latency under load? Measure upload and download separately. Use another physical host and a test server you operate or are authorized to use; a test between processes on the same host does not test the external path.

freebsd-version -kru
ifconfig -l
ifconfig ix0

iperf3 -c iperf3.stealth.net -P 1 -t 10
iperf3 -c iperf3.stealth.net -P 8 -t 10
iperf3 -c iperf3.stealth.net -P 8 -t 10 -R

The default direction is from client to server; -R reverses it. Record the receiver rate, retransmissions where reported, elapsed time, CPU, and parallel stream count. Compare like with like and repeat at representative times. A public server may be busy or have its own limitations. See our speed test guide.

Then test the actual large-file workflow between the systems the business uses. Record file size, transfer time and achieved rate in each relevant direction, alongside storage throughput and latency, CPU, encryption, and competing activity on both ends. Use representative files and the normal application; preserve the same test conditions when comparing changes. An iperf3 test isolates aspects of the network path, but does not establish how fast storage or a file-transfer application can read, encrypt, write, and verify the data.

02 / Follow the evidence

Read the interface and host counters.

The following examples are read-only. Capture timestamps and two observations over a known interval; counters may include old events, reset on restart, or represent different events by driver. The interface name below is illustrative.

netstat -I ix0 -b -d
netstat -m
netstat -s -p tcp

vmstat -i
vmstat 1
netstat -w 1 -I ix0
pciconf -lv

Where available, consult your specific adapter’s driver manual and sysctl dev tree for queue and hardware statistics. Check the switch port and optical levels with the equipment owners as well. Do not clear counters before capturing the evidence.

03 / The path matters

How much data is in flight?

The bandwidth-delay product is an estimate of the data needed in flight to fill a path: bottleneck rate in bits per second multiplied by round-trip time in seconds, divided by eight for bytes.

The relevant round-trip time is to the actual peer, and the target rate must be achievable along the complete path. TCP window scaling, congestion control, losses, peer settings, application behavior, and per-socket memory limits all affect the result. Several streams can share the work, but they do not remove a bottleneck.

04 / TCP buffers

Understand what the limits control.

FreeBSD can automatically grow TCP send and receive buffers as connections progress. In the FreeBSD 14.3 tcp(4) manual, net.inet.tcp.recvbuf_auto and net.inet.tcp.sendbuf_auto enable automatic sizing; recvbuf_max and sendbuf_max set their respective automatic-sizing maxima. recvspace and sendspace are initial TCP buffer settings in that manual. Inspect the names and behavior on your release before changing them.

sysctl net.inet.tcp.recvbuf_auto net.inet.tcp.recvbuf_max
sysctl net.inet.tcp.sendbuf_auto net.inet.tcp.sendbuf_max
sysctl kern.ipc.maxsockbuf

A maximum is not memory immediately reserved for every connection. Raising it can nevertheless allow more memory to be used across active connections, and large buffers may increase queueing. Applications can request socket sizes and the system also has socket-buffer limits; verify how they interact on your release. A buffer ceiling alone does not prove that a transfer is limited by that ceiling.

05 / An illustrative experiment

Change a limit only when evidence supports it.

These are conditional recommendations for an administrator, not universal settings. Review each change with your company’s IT team, including memory, testing, and rollback, before applying it.

Suppose an administrator has a FreeBSD 14.3 host receiving a long-running TCP transfer, automatic receive sizing is enabled, and measurements point to its receive-buffer ceiling as a possible constraint. The host has adequate memory, the peer and path can sustain the target rate, and the administrator has reviewed the socket limits and planned a maintenance window. A test can compare the current setting with a modest, preselected ceiling.

In an authorized privileged shell, capture the existing value and write it into the change record. Verify that the read succeeded and produced a numeric value before continuing. The shell variable must remain available in the same session for the restore command.

if old_recvbuf_max=$(sysctl -n net.inet.tcp.recvbuf_max); then
    printf 'Original receive ceiling: %s bytes\n' "$old_recvbuf_max"
else
    unset old_recvbuf_max
    printf 'Could not read the original value. Do not proceed.\n'
fi
sysctl net.inet.tcp.recvbuf_auto kern.ipc.maxsockbuf

After checking the output and prerequisites, run the following in the same privileged shell. It refuses the change if the original value is absent, not numeric, already at least 16 MiB, or has changed since capture. It does not validate your memory budget or other socket limits; those require your review.

if current_recvbuf_max=$(sysctl -n net.inet.tcp.recvbuf_max) &&
   [ -n "${old_recvbuf_max-}" ] &&
   [ "$old_recvbuf_max" -eq "$current_recvbuf_max" ] 2>/dev/null &&
   [ "$old_recvbuf_max" -lt 16777216 ]; then
    sysctl net.inet.tcp.recvbuf_max=16777216
else
    printf 'No change: review the captured and current values.\n'
fi
sysctl net.inet.tcp.recvbuf_max

Repeat the same authorized single-flow receive test and check throughput, retransmissions, CPU, latency, and memory. A new test connection helps make comparisons repeatable. If the result is unchanged or harms another workload, restore the recorded value. The original value below comes from the same shell session:

if current_recvbuf_max=$(sysctl -n net.inet.tcp.recvbuf_max) &&
   [ -n "${old_recvbuf_max-}" ] &&
   [ "$old_recvbuf_max" -eq "$old_recvbuf_max" ] 2>/dev/null &&
   [ "$current_recvbuf_max" -eq 16777216 ] 2>/dev/null; then
    sysctl "net.inet.tcp.recvbuf_max=$old_recvbuf_max"
else
    printf 'No restore: review current and recorded values.\n'
fi
sysctl net.inet.tcp.recvbuf_max

The restore guard refuses to overwrite a value that changed since the experiment. Verify the restored value against the recorded original. A session variable disappears when its shell ends; if that happens, use the independently recorded numeric original. Restoring a ceiling does not necessarily shrink buffers already allocated to existing connections. Do not proceed if the setting is absent or read-only on your release.

More tuning examples / Sending

Give a measured sender more room.

When a FreeBSD host sends to a distant peer, its automatic send-buffer ceiling can be worth investigating alongside the receiver’s window and congestion control. If measurements suggest the send ceiling is restrictive, the following example tests 32 MiB for net.inet.tcp.sendbuf_max. It changes the system-wide ceiling, not the buffer of one application. It is an illustrative starting point, not a guaranteed fit for every 10, 25, or 100 Gbps path.

Run in an authorized privileged shell. This captures the original value, checks that automatic send sizing reports enabled and raises the ceiling only if it is below 32 MiB and has not changed since capture. Keep this shell open and save the printed original value in your change record.

unset old_sendbuf_max
send_trial_applied=0
if old_sendbuf_max=$(sysctl -n net.inet.tcp.sendbuf_max) &&
   [ "$(sysctl -n net.inet.tcp.sendbuf_auto)" = 1 ] &&
   [ "$old_sendbuf_max" -ge 0 ] 2>/dev/null &&
   [ "$old_sendbuf_max" -lt 33554432 ] &&
   [ "$(sysctl -n net.inet.tcp.sendbuf_max)" = "$old_sendbuf_max" ]; then
    printf 'Original send ceiling: %s bytes\n' "$old_sendbuf_max"
    if sysctl net.inet.tcp.sendbuf_max=33554432; then
        send_trial_applied=1
    fi
else
    printf 'No change: review automatic sizing and the current limit.\n'
fi
sysctl net.inet.tcp.sendbuf_auto net.inet.tcp.sendbuf_max kern.ipc.maxsockbuf

Confirm the reported value, establish a new connection, and repeat the same approved sender test. Check throughput, retransmissions, memory use and latency. Roll back in the same shell when the trial is complete, or sooner if the result is worse:

if [ "${send_trial_applied-0}" = 1 ] &&
   [ "${old_sendbuf_max-}" -ge 0 ] 2>/dev/null &&
   [ "$(sysctl -n net.inet.tcp.sendbuf_max)" = 33554432 ]; then
    sysctl "net.inet.tcp.sendbuf_max=$old_sendbuf_max"
else
    printf 'No restore: review the recorded and current values.\n'
fi
sysctl net.inet.tcp.sendbuf_max

Verify the final value against your record. If the shell has closed or another administrator changed the setting, review the saved value before restoring it manually. A higher ceiling does not force an application to use it, and other socket limits, the peer, memory, or the network may still constrain throughput.

More tuning examples / Socket limits

Check the system-wide socket limit.

Applications may explicitly request socket buffers through SO_SNDBUF or SO_RCVBUF. FreeBSD documents kern.ipc.maxsockbuf as an upper limit on these requests. If testing and your release documentation indicate that this limit is blocking an approved workload, a team might test a 64 MiB ceiling. This setting affects sockets system-wide; kernel accounting and other limits can make the effective usable buffer smaller. Confirm available memory and the impact on other applications first.

The following authorized, temporary example captures the current value and changes it only if lower than 64 MiB. It does not reserve 64 MiB for every socket, and it does not automatically change application buffer requests.

unset old_maxsockbuf
socket_trial_applied=0
if old_maxsockbuf=$(sysctl -n kern.ipc.maxsockbuf) &&
   [ "$old_maxsockbuf" -ge 0 ] 2>/dev/null &&
   [ "$old_maxsockbuf" -lt 67108864 ] &&
   [ "$(sysctl -n kern.ipc.maxsockbuf)" = "$old_maxsockbuf" ]; then
    printf 'Original socket limit: %s bytes\n' "$old_maxsockbuf"
    if sysctl kern.ipc.maxsockbuf=67108864; then
        socket_trial_applied=1
    fi
else
    printf 'No change: review the current socket limit.\n'
fi
sysctl kern.ipc.maxsockbuf

After a new, comparable test, restore the captured value in the same privileged shell if the change is not retained. Check memory usage and other workloads during the test.

if [ "${socket_trial_applied-0}" = 1 ] &&
   [ "${old_maxsockbuf-}" -ge 0 ] 2>/dev/null &&
   [ "$(sysctl -n kern.ipc.maxsockbuf)" = 67108864 ]; then
    sysctl "kern.ipc.maxsockbuf=$old_maxsockbuf"
else
    printf 'No restore: review the recorded and current values.\n'
fi
sysctl kern.ipc.maxsockbuf

Verify the restored value against the change record. Existing sockets and application-set buffers may not immediately reflect a lower limit. Never rely on these sample numbers without checking the operating system release, the application and your IT team’s guidance.

06 / Network memory

What are mbufs?

FreeBSD uses mbufs to represent network data and associated metadata. Mbuf clusters provide larger storage for packet data. Drivers and socket processing use these resources in different ways; a high number in use is not by itself proof of exhaustion.

netstat -m provides an overview of mbuf resources and allocation failures. Compare observations during the workload and investigate growing failure counts, relevant device drops, and available memory together. A driver can expose additional queue statistics.

Increasing a limit can allow more memory consumption and may require a reboot for a boot-time tunable. Confirm the supported mechanism in your installed system’s documentation and arrange console or out-of-band recovery before testing boot changes.

07 / The rest of the host

Check the adapter, queues, and CPU.

A negotiated 10, 25, or 100 Gbps link does not by itself establish host capacity. Check that the adapter and its PCIe slot generation and lane width meet the manufacturer’s requirements. Driver queues, interrupt distribution, CPU saturation, memory locality, PCIe bandwidth, encryption, and application threads can all affect throughput. A single flow and many parallel flows may use the system differently. If the test host has multiple NUMA domains, work with your IT team and adapter vendor to inspect the adapter’s locality, application threads, and interrupt distribution before changing CPU placement. Compare the actual transfer as well as the benchmark.

Some transfer tools support pacing, which spreads traffic over time and can reduce bursts. Where the tool and environment support it, compare a paced test with the original test, measuring completion time, retransmissions and the effect on other traffic. An overly low rate can slow transfers; do not apply a generic rate or alter system-wide scheduling without an approved plan.

Inspect the FreeBSD manual for your adapter and release before changing queue counts, interrupt moderation, or receive-side settings. Their names and behavior are driver-specific. Offloads such as checksum processing, segmentation, and receive aggregation can reduce CPU work, but disabling them without evidence can reduce throughput. Routing, bridging, capture, and driver behavior can impose specific restrictions; verify the relevant documentation before changing any feature.

Investigate errors, increasing drops, and packet loss across the path rather than attributing every slow test to a buffer. Large buffers and queues can hide congestion while increasing delay; consider interactive applications and other users during the experiment.

08 / Make it repeatable

Know when a setting is applied.

A writable runtime sysctl change affects the running system; it is not automatically saved across a reboot. FreeBSD reads /etc/sysctl.conf when entering multi-user mode for supported settings. Other settings must be supplied as boot-time tunables through /boot/loader.conf or the documented mechanism for the release and driver. A name visible through sysctl is not necessarily writable at runtime.

Check the setting’s manual page, read/write status, boot ordering and any configuration management that may reapply values. Record the original file contents and effective values. Do not persist an experimental setting until it has passed testing and review; for boot-time changes, prepare console access and a recovery plan before rebooting.

09 / Checklist

Before calling the work complete.

• Did the test use a separate, capable, authorized peer and a representative path?
• Were both directions, one flow and multiple flows, measured under comparable conditions?
• Did interface errors, mbuf allocation failures, CPU, and application behavior point to a specific constraint?
• Were memory and latency under load checked as well as peak throughput?
• Was one setting changed at a time, with an original value and tested rollback recorded?
• Are any persisted settings appropriate for the release, hardware, and expected number of connections?

For diagnostics and what to share with a provider, see preparing for an Internet support request.

Research / Large file transfers

What the research shows.

A 2026 WLCG preprint measured tuning across research data-transfer sites using Linux hosts. At one site, the network benchmark improved from 20 to 90 Gbps while actual file transfers did not improve because storage was the constraint. Another site saw its file-transfer rate rise from 60 to 90 Gbps; tuning reduced throughput at a third. These were Linux results, not FreeBSD benchmarks, but the lesson applies to the whole transfer path: measure the application and storage, not only the network.

For a FreeBSD example, the FreeBSD Foundation’s Netflix case study describes a purpose-built content-delivery system using customized FreeBSD, including work on file sending and TLS. It illustrates the importance of the application and host together; it is not a promise that the same optimizations or rates apply to a general-purpose server.

ESnet’s iperf3 3.16 announcement explains why parallel tests can use multiple CPU cores in newer versions. Use that information to interpret the baseline above, then measure the business transfer that matters.

Related guides

Working with another system?

Explore high-speed network tuning for Linux, macOS, and Windows.