基于MPLS的TCP窗口缩放问题

经过大量的故障排除后,为什么MPLS电路上的TCP速度在一个方向上很慢,我发现通过调整TCP窗口大小,情况有了很大的改善。 也许你可以告诉我为什么。

细节:

  • 两台主机之间的ping是7ms
  • 这两台机器运行Debian Wheezy与内核3.2.0-4
  • 两台机器都有默认的TCP设置
  • 两台机器都有10 Gbps的上行链路

使用默认TCP设置的结果:

位置A – >位置B: iperf -c $Location B --> ~2 Gbps

位置B – >位置Aiperf -c $Location A --> 65 - 250 Mbps

将以下更改应用于/etc/sysctl.conf后,我经历了更好的结果

 net.ipv4.tcp_window_scaling = 1 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 16384 16777216 

调整后的TCP设置后的结果:

位置A – >位置B: iperf -c $Location B --> ~2 Gbps

位置B – >位置Aiperf -c $Location A --> 150 Mbps - 2 Gbps (大部分〜1.8 Gbps)

我的问题是什么可能会对TCP窗口缩放产生负面影响? 这是我logging的tcptrace的输出。 也许这有助于确定问题。

感谢您的帮助!

问候,莱昂克莱默

 # # Speed: ~2 Gbps # 1 arg remaining, starting with 'tcpdump.lol' Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004 260307 packets seen, 260307 TCP packets traced elapsed wallclock time: 0:00:17.740455, 14673 pkts/sec analyzed trace file elapsed time: 0:00:03.798052 TCP connection info: 2 TCP connections traced: ================================ TCP connection 2: host c: 10.1.2.5:59242 host d: 10.1.2.2:5201 complete conn: RESET (SYNs: 2) (FINs: 0) first packet: Thu Nov 13 00:05:18.256992 2014 last packet: Thu Nov 13 00:05:21.876883 2014 elapsed time: 0:00:03.619891 total packets: 260276 filename: tcpdump.lol c->d: d->c: total packets: 156463 total packets: 103813 resets sent: 0 resets sent: 154 ack pkts sent: 156462 ack pkts sent: 103660 pure acks sent: 1 pure acks sent: 103658 sack pkts sent: 0 sack pkts sent: 8716 dsack pkts sent: 0 dsack pkts sent: 6 max sack blks/ack: 0 max sack blks/ack: 1 unique bytes sent: 674405781 unique bytes sent: 0 actual data pkts: 156461 actual data pkts: 0 actual data bytes: 674415917 actual data bytes: 0 rexmt data pkts: 6 rexmt data pkts: 0 rexmt data bytes: 10136 rexmt data bytes: 0 zwnd probe pkts: 0 zwnd probe pkts: 0 zwnd probe bytes: 0 zwnd probe bytes: 0 outoforder pkts: 3784 outoforder pkts: 0 pushed data pkts: 28 pushed data pkts: 0 SYN/FIN pkts sent: 1/0 SYN/FIN pkts sent: 1/0 req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y adv wind scale: 7 adv wind scale: 7 req sack: Y req sack: Y sacks sent: 0 sacks sent: 8716 urgent data pkts: 0 pkts urgent data pkts: 0 pkts urgent data bytes: 0 bytes urgent data bytes: 0 bytes mss requested: 1460 bytes mss requested: 1460 bytes max segm size: 14480 bytes max segm size: 0 bytes min segm size: 37 bytes min segm size: 0 bytes avg segm size: 4310 bytes avg segm size: 0 bytes max win adv: 14720 bytes max win adv: 3145728 bytes min win adv: 14720 bytes min win adv: 14592 bytes zero win adv: 0 times zero win adv: 0 times avg win adv: 14720 bytes avg win adv: 3123096 bytes initial window: 37 bytes initial window: 0 bytes initial window: 1 pkts initial window: 0 pkts ttl stream length: NA ttl stream length: NA missed data: NA missed data: NA truncated data: 0 bytes truncated data: 0 bytes truncated packets: 0 pkts truncated packets: 0 pkts data xmit time: 3.612 secs data xmit time: 0.000 secs idletime max: 43.3 ms idletime max: 43.3 ms throughput: 186305549 Bps throughput: 0 Bps ==================================================================================== # # Speed: ~65 Mbps # 1 arg remaining, starting with 'tcpdump.lol' Ostermann's tcptrace -- version 6.6.7 -- Thu Nov 4, 2004 652370 packets seen, 652370 TCP packets traced elapsed wallclock time: 0:00:07.985123, 81698 pkts/sec analyzed trace file elapsed time: 0:02:13.121598 TCP connection info: 2 TCP connections traced: ================================ TCP connection 2: host c: 10.1.2.2:45259 host d: 10.1.2.5:5201 complete conn: RESET (SYNs: 2) (FINs: 1) first packet: Wed Nov 12 22:42:06.606682 2014 last packet: Wed Nov 12 22:44:19.553859 2014 elapsed time: 0:02:12.947177 total packets: 652339 filename: tcpdump.lol c->d: d->c: total packets: 446774 total packets: 205565 resets sent: 0 resets sent: 22 ack pkts sent: 446773 ack pkts sent: 205543 pure acks sent: 1 pure acks sent: 205541 sack pkts sent: 0 sack pkts sent: 9322 dsack pkts sent: 0 dsack pkts sent: 2 max sack blks/ack: 0 max sack blks/ack: 3 unique bytes sent: 1058366405 unique bytes sent: 0 actual data pkts: 446772 actual data pkts: 0 actual data bytes: 1058379437 actual data bytes: 0 rexmt data pkts: 5 rexmt data pkts: 0 rexmt data bytes: 13032 rexmt data bytes: 0 zwnd probe pkts: 0 zwnd probe pkts: 0 zwnd probe bytes: 0 zwnd probe bytes: 0 outoforder pkts: 1577 outoforder pkts: 0 pushed data pkts: 5733 pushed data pkts: 0 SYN/FIN pkts sent: 1/0 SYN/FIN pkts sent: 1/1 req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y adv wind scale: 7 adv wind scale: 7 req sack: Y req sack: Y sacks sent: 0 sacks sent: 9322 urgent data pkts: 0 pkts urgent data pkts: 0 pkts urgent data bytes: 0 bytes urgent data bytes: 0 bytes mss requested: 1460 bytes mss requested: 1460 bytes max segm size: 18824 bytes max segm size: 0 bytes min segm size: 37 bytes min segm size: 0 bytes avg segm size: 2368 bytes avg segm size: 0 bytes max win adv: 14720 bytes max win adv: 561920 bytes min win adv: 14720 bytes min win adv: 14592 bytes zero win adv: 0 times zero win adv: 0 times avg win adv: 14720 bytes avg win adv: 533768 bytes initial window: 37 bytes initial window: 0 bytes initial window: 1 pkts initial window: 0 pkts ttl stream length: NA ttl stream length: 0 bytes missed data: NA missed data: 0 bytes truncated data: 0 bytes truncated data: 0 bytes truncated packets: 0 pkts truncated packets: 0 pkts data xmit time: 132.940 secs data xmit time: 0.000 secs idletime max: 37.8 ms idletime max: 37.8 ms throughput: 7960804 Bps throughput: 0 Bps