This was the remedy for high latency connections to my ubuntu server. After doing this RTT dropped from average of a good couple hundred ms down to 1 to 2 ms.

# echo 1 > /proc/sys/net/ipv4/tcp_low_latency
# echo 1 > /proc/sys/net/ipv4/tcp_sack
# echo 1 > /proc/sys/net/ipv4/tcp_timestamps
# echo 1 > /proc/sys/net/ipv4/tcp_window_scaling

and once latency went down I did this to get it to stick across reboots:

# echo 'net.ipv4.tcp_low_latency = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_window_scaling = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_timestamps = 1' >> /etc/sysctl.conf
# echo 'net.ipv4.tcp_sack = 1' >> /etc/sysctl.conf

UPDATE

So the latency came back a little while later and reappying these commands again might have brough some relief but the latency kept jumping back up. After swapping the network cable things seem to hovering at a brisk 1ms. Lesson learned: It’s not always your configuration.