Set the default for UDPRcvBuf and UDPSndBuf to 1M.

It may not be obvious, but due to the way tinc operates (single-threaded
control loop with no intermediate packet buffer), UDP send and receive
buffers can have a massive impact on performance. It is therefore of
paramount importance that the buffers be large enough to prevent packet
drops that could occur while tinc is processing a packet.

Leaving that value to the OS default could be reasonable if we weren't
relying on it so much. Instead, this makes performance somewhat
unpredictable.

In practice, the worst case scenario occurs on Windows, where Microsoft
had the brillant idea of making the buffers 8K in size by default, no
matter what the link speed is. Considering that 8K flies past in a
matter of microseconds on >1G links, this is extremely inappropriate. On
these systems, changing the buffer size to 1M results in *obscene*
raw throughput improvements; I have observed a 10X jump from 40 Mbit/s
to 400 Mbit/s on my system.

In this commit, we stop trusting the OS to get this right and we use a
fixed 1M value instead, which should be enough for <=1G links.

diff --git a/doc/tinc.conf.5.in b/doc/tinc.conf.5.in
index bc481ead83e01a636f241a3edef2e7120ab9e024..2ba1012424cbcb9f5c0d55bff7b822fa5a59b46a 100644 (file)
--- a/doc/tinc.conf.5.in
+++ b/doc/tinc.conf.5.in
@@ -500,12 +500,14 @@ The minimum amount of time between sending UDP ping datagrams to try to establis
 .It Va UDPDiscoveryTimeout Li = Ar seconds Pq 30
 If tinc doesn't receive any UDP ping replies over the specified interval,
 it will assume UDP communication is broken and will fall back to TCP.
-.It Va UDPRcvBuf Li = Ar bytes Pq OS default
+.It Va UDPRcvBuf Li = Ar bytes Pq 1048576
 Sets the socket receive buffer size for the UDP socket, in bytes.
-If unset, the default buffer size will be used by the operating system.
-.It Va UDPSndBuf Li = Ar bytes Pq OS default
+If set to zero, the default buffer size will be used by the operating system.
+Note: this setting can have a significant impact on performance, especially raw throughput.
+.It Va UDPSndBuf Li = Ar bytes Pq 1048576
 Sets the socket send buffer size for the UDP socket, in bytes.
-If unset, the default buffer size will be used by the operating system.
+If set to zero, the default buffer size will be used by the operating system.
+Note: this setting can have a significant impact on performance, especially raw throughput.
 .El
 .Sh HOST CONFIGURATION FILES
 The host configuration files contain all information needed

diff --git a/doc/tinc.texi b/doc/tinc.texi
index 581da6eade277fc6b0ab0e6c36dcdfd1bbc2afa5..3694d6da165f11d8a84f0b9872b5d6fcef34e006 100644 (file)
--- a/doc/tinc.texi
+++ b/doc/tinc.texi
@@ -1254,14 +1254,16 @@ If tinc doesn't receive any UDP ping replies over the specified interval,
 it will assume UDP communication is broken and will fall back to TCP.
 
 @cindex UDPRcvBuf
-@item UDPRcvBuf = <bytes> (OS default)
+@item UDPRcvBuf = <bytes> (1048576)
 Sets the socket receive buffer size for the UDP socket, in bytes.
-If unset, the default buffer size will be used by the operating system.
+If set to zero, the default buffer size will be used by the operating system.
+Note: this setting can have a significant impact on performance, especially raw throughput.
 
 @cindex UDPSndBuf
-@item UDPSndBuf = <bytes> Pq OS default
+@item UDPSndBuf = <bytes> (1048576)
 Sets the socket send buffer size for the UDP socket, in bytes.
-If unset, the default buffer size will be used by the operating system.
+If set to zero, the default buffer size will be used by the operating system.
+Note: this setting can have a significant impact on performance, especially raw throughput.
 
 @end table
 

diff --git a/src/net_setup.c b/src/net_setup.c
index d1d5c04be36ff5bfbbe002b106551a789a1635f2..fee4159fabc16ed9bfb096452e8cd4cb12f14514 100644 (file)
--- a/src/net_setup.c
+++ b/src/net_setup.c
@@ -854,14 +854,14 @@ static bool setup_myself(void) {
        }
 
        if(get_config_int(lookup_config(config_tree, "UDPRcvBuf"), &udp_rcvbuf)) {
-               if(udp_rcvbuf <= 0) {
+               if(udp_rcvbuf < 0) {
                        logger(DEBUG_ALWAYS, LOG_ERR, "UDPRcvBuf cannot be negative!");
                        return false;
                }
        }
 
        if(get_config_int(lookup_config(config_tree, "UDPSndBuf"), &udp_sndbuf)) {
-               if(udp_sndbuf <= 0) {
+               if(udp_sndbuf < 0) {
                        logger(DEBUG_ALWAYS, LOG_ERR, "UDPSndBuf cannot be negative!");
                        return false;
                }

diff --git a/src/net_socket.c b/src/net_socket.c
index 85bc4df56993a4c5eda92c7acf6454100e3dc412..3bf7d757dcf489d0c5c292ccffa641fd900f2783 100644 (file)
--- a/src/net_socket.c
+++ b/src/net_socket.c
@@ -43,8 +43,8 @@
 int addressfamily = AF_UNSPEC;
 int maxtimeout = 900;
 int seconds_till_retry = 5;
-int udp_rcvbuf = 0;
-int udp_sndbuf = 0;
+int udp_rcvbuf = 1024 * 1024;
+int udp_sndbuf = 1024 * 1024;
 int max_connection_burst = 100;
 
 listen_socket_t listen_socket[MAXSOCKETS];