数据包队列性能与BIND域名服务器的差异

背景:

我已经inheritance了大容量caching域名服务器环境(Redhat Enterprise Linux 5.8,IBM System x3550),其环形缓冲区设置不一致:eth0为1020,eth1为255。 eth0连接到本地数据中心的交换机1,eth1连接到交换机2。 集群中的每台服务器在eth0或eth1是否为活动接口之间进行切换,并且每个集群位于不同的区域。 环形缓冲区显然需要保持一致。

这里是事情变得更加棘手的地方:当研究为什么一些名称服务器频繁地logging“错误发送响应:未设置”错误, ISC知识库build议与出站拥塞有关时,我发现了上述问题 。 具有较高环形缓冲区设置(1020)的服务器会在ifconfig上丢弃更less的数据包(正如人们所期望的那样),但往往会以高频率logging上述错误,在我的最高负载组中,每天约20千次。 我们称之为“第一组”。 具有较低环形缓冲区(255)设置的服务器每天的入站数据包数量明显减less(同样是预期的),但是在同一个负载组中,BIND错误的实例通常为0-150。

这里也不是一个很大的谜团。 cachingDNS是recursion服务:如果某些东西没有被caching,服务器必须代表这个问题进行多个查询,直到它最终能够返回一个答案。 这是(一) – >(多出)查询关系。 修复RX环形缓冲区应该使这个数字在板子上等于一个新的值,从那里调整proc(wmem_max / wmem_default)中的内核出站networking队列可能是个好主意。


我喜欢能够评估configuration变化对性能问题的影响,因此我在写作之前编写了一份报告来收集一些数据,然后才开始进行生产更改。 以下是组1中前两台服务器的输出示例:

group1-01 RX: 7166.27/sec av. TX: 7432.57/sec av. RXDROP: 7.43/sec av. unset_err: 27633 group1-02 RX: 7137.37/sec av. TX: 7398.50/sec av. RXDROP: 9.94/sec av. unset_err: 107 

这些是公式。 请注意,这是一个本地脚本,不需要依赖每个服务器必须维护的shell脚本。

  RXPACK=$(ssh $server "sar -n DEV -f /var/log/sa/sa$(date --date=yesterday '+%d') | grep \"Average: .*\$(awk '{if (\$2 == "00000000") { print \$1 }}' /proc/net/route)\" | awk '{print \$3}'" 2>/dev/null) TXPACK=$(ssh $server "sar -n DEV -f /var/log/sa/sa$(date --date=yesterday '+%d') | grep \"Average: .*\$(awk '{if (\$2 == "00000000") { print \$1 }}' /proc/net/route)\" | awk '{print \$4}'" 2>/dev/null) RXDROP=$(ssh $server "sar -n EDEV -f /var/log/sa/sa$(date --date=yesterday '+%d') | grep \"Average: .*\$(awk '{if (\$2 == "00000000") { print \$1 }}' /proc/net/route)\" | awk '{print \$6}'" 2>/dev/null) TXDROP=$(ssh $server "sudo grep 'error sending response: unset' /var/log/dns_named.1" 2>/dev/null | wc -l) 

一旦我开始在所有的cachingDNS环境中运行这个报告,我注意到另外一个具有几乎相同数据包负载的组,我们称之为组2, 根本没有任何问题

 group2-01 RX: 7066.44/sec av. TX: 7345.95/sec av. RXDROP: 0.00/sec av. unset_err: 0 group2-02 RX: 7019.18/sec av. TX: 7312.47/sec av. RXDROP: 0.00/sec av. unset_err: 0 

问题是:

为什么group2的行为不需要进一步调整RX环形缓冲区或net.core.wmem_default / net.core.wmem_max ? 我将需要规范环缓冲区无论如何,但我想知道还有什么事情在这里之前,我开始玩wmem值在/ proc。

我唯一能想到的就是队列被应用程序更快地清空了,但是networking堆栈调整并不是我有很多实际操作经验,我想得到第二个意见。 (我的眼睛盯着一些ethtool柜台的名字,我不会否认)

我已经消除了以下可能性。 证据跟在分隔线之后。

  • 环形缓冲区布局是相同的。 (组1和组2的第一个服务器configuration相同,组1和组2的第二个服务器configuration相同)
  • 默认的网关布局是一样的。
  • 网卡是一样的。 (Broadcom BCM5708)
  • ethtool报告的固件版本相同。 (bc 4.0.3 ipms 1.6.0)
  • sysctl -a输出在两组的第一台服务器和第二台服务器之间匹配。 (不包括内核和fs部分)
  • 组1和组2中的服务器总数相同。 (10)

出于机密原因,我无法显示原始的named.conf,或者我用来排除信息的grepfilter。 您将不得不承认,所有四台服务器之间的以下configuration参数是不变的:

  notify no; allow-transfer { none; }; allow-recursion { any; }; allow-query { any; }; allow-query-cache { any; }; recursive-clients 100000; max-cache-size 2G; max-ncache-ttl 900; 

以下是大量的系统信息。 “hosthash”只是为了certificate循环的每一次迭代实际上是在不透露实际主机名的情况下触发不同的服务器。

主机散列:

 group1-1: dc78abcb154b74c87feecb3f35222263d40c028c group1-2: 9fe491d58fd1e7d4e21e5bf10c164e4cf66e884b group2-1: fc76bb3ee1ff580c6aba0d685713bb4145bd5fe3 group2-2: b7550c65d37622a131b1e47f066773defbb4d817 for server in $group1_1 $group1_2 $group2_1 $group2_2 do echo ____________________ ssh $server "echo -en hosthash: \$(echo \$HOSTNAME | sha1sum)\\\n\\\n && SARFILE=/var/log/sa/sa\$(date --date=yesterday '+%d') && uname -srvmpio && sudo /usr/sbin/dmidecode -s system-product-name dmesg | grep Broadcom && head /proc/cpuinfo && GWIF=\$(awk '{if (\$2 == 00000000) { print \$1 }}' /proc/net/route) && sar -n DEV -f \$SARFILE | egrep '(IFACE|Average)' && sar -n EDEV -f \$SARFILE | egrep '(IFACE|Average)' && sudo /sbin/ethtool \$GWIF && sudo /sbin/ethtool -i \$GWIF && sudo /sbin/ethtool -g \$GWIF && sudo /sbin/ethtool -c \$GWIF && sudo /sbin/ethtool -S \$GWIF && echo sysctl linecount: \$(sudo /sbin/sysctl -a | egrep -v '^(fs|kernel)' | wc -l) && echo sysctl hash: \$(sudo /sbin/sysctl -a | egrep -v '^(fs|kernel)' | sha1sum)" done 

输出:

 ____________________ hosthash: dc78abcb154b74c87feecb3f35222263d40c028c - Linux 2.6.18-308.16.1.el5 #1 SMP Tue Sep 18 07:21:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux IBM System x3550 -[7978AC1]- bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.1.11 (July 20, 2011) eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem c8000000, IRQ 90, node addr 001a649db00e eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem ce000000, IRQ 177, node addr 001a649db010 cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.7 (July 20, 2011) Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.0.3 (Aug 04, 2011) processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5420 @ 2.50GHz stepping : 6 cpu MHz : 2493.750 cache size : 6144 KB physical id : 0 siblings : 4 12:00:01 AM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s Average: lo 1269.15 1269.15 206600.39 206600.39 0.00 0.00 0.00 Average: eth0 7166.27 7432.57 704051.80 2419779.42 0.00 0.00 0.94 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:00:01 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 7.43 0.00 0.00 0.00 0.00 0.00 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 driver: bnx2 version: 2.1.11 firmware-version: bc 4.0.3 ipms 1.6.0 bus-info: 0000:04:00.0 Ring parameters for eth0: Pre-set maximums: RX: 2040 RX Mini: 0 RX Jumbo: 8160 TX: 255 Current hardware settings: RX: 1020 RX Mini: 0 RX Jumbo: 0 TX: 255 Coalesce parameters for eth0: Adaptive RX: off TX: off stats-block-usecs: 999936 sample-interval: 0 pkt-rate-low: 0 pkt-rate-high: 0 rx-usecs: 18 rx-frames: 12 rx-usecs-irq: 18 rx-frames-irq: 2 tx-usecs: 80 tx-frames: 20 tx-usecs-irq: 18 tx-frames-irq: 2 rx-usecs-low: 0 rx-frame-low: 0 tx-usecs-low: 0 tx-frame-low: 0 rx-usecs-high: 0 rx-frame-high: 0 tx-usecs-high: 0 tx-frame-high: 0 NIC statistics: rx_bytes: 1505439501410 rx_error_bytes: 0 tx_bytes: 4672574845104 tx_error_bytes: 0 rx_ucast_packets: 15315548049 rx_mcast_packets: 2035415 rx_bcast_packets: 1101989 tx_ucast_packets: 15505474251 tx_mcast_packets: 40018 tx_bcast_packets: 36019 tx_mac_errors: 0 tx_carrier_errors: 0 rx_crc_errors: 0 rx_align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 tx_deferred: 0 tx_excess_collisions: 0 tx_late_collisions: 0 tx_total_collisions: 0 rx_fragments: 0 rx_jabbers: 0 rx_undersize_packets: 0 rx_oversize_packets: 0 rx_64_byte_packets: 92309552 rx_65_to_127_byte_packets: 1243637891 rx_128_to_255_byte_packets: 790117566 rx_256_to_511_byte_packets: 127197337 rx_512_to_1023_byte_packets: 168929387 rx_1024_to_1522_byte_packets: 11591832 rx_1523_to_9022_byte_packets: 0 tx_64_byte_packets: 60586118 tx_65_to_127_byte_packets: 1976738758 tx_128_to_255_byte_packets: 2830395753 tx_256_to_511_byte_packets: 157607989 tx_512_to_1023_byte_packets: 1483716940 tx_1024_to_1522_byte_packets: 406821340 tx_1523_to_9022_byte_packets: 0 rx_xon_frames: 0 rx_xoff_frames: 0 tx_xon_frames: 116422 tx_xoff_frames: 134780 rx_mac_ctrl_frames: 0 rx_filtered_packets: 0 rx_ftq_discards: 0 rx_discards: 0 rx_fw_discards: 14015105 sysctl linecount: 504 sysctl hash: dd6aab90d0fd9ae90742c5f812a78734e2f2ff1c - ____________________ hosthash: 9fe491d58fd1e7d4e21e5bf10c164e4cf66e884b - Linux 2.6.18-308.16.1.el5 #1 SMP Tue Sep 18 07:21:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux IBM System x3550 -[7978EHU]- bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.1.11 (July 20, 2011) eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem c8000000, IRQ 90, node addr 001a6479655c eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem ce000000, IRQ 177, node addr 001a6479655e cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.7 (July 20, 2011) Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.0.3 (Aug 04, 2011) processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5420 @ 2.50GHz stepping : 6 cpu MHz : 2493.746 cache size : 6144 KB physical id : 0 siblings : 4 12:00:01 AM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s Average: lo 1261.04 1261.04 205548.08 205548.08 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth1 7137.37 7398.50 702340.35 2409580.71 0.00 0.00 0.97 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:00:01 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth1 0.00 0.00 0.00 9.94 0.00 0.00 0.00 0.00 0.00 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 driver: bnx2 version: 2.1.11 firmware-version: bc 4.0.3 ipms 1.6.0 bus-info: 0000:06:00.0 Ring parameters for eth1: Pre-set maximums: RX: 2040 RX Mini: 0 RX Jumbo: 8160 TX: 255 Current hardware settings: RX: 255 RX Mini: 0 RX Jumbo: 0 TX: 255 Coalesce parameters for eth1: Adaptive RX: off TX: off stats-block-usecs: 999936 sample-interval: 0 pkt-rate-low: 0 pkt-rate-high: 0 rx-usecs: 18 rx-frames: 12 rx-usecs-irq: 18 rx-frames-irq: 2 tx-usecs: 80 tx-frames: 20 tx-usecs-irq: 18 tx-frames-irq: 2 rx-usecs-low: 0 rx-frame-low: 0 tx-usecs-low: 0 tx-frame-low: 0 rx-usecs-high: 0 rx-frame-high: 0 tx-usecs-high: 0 tx-frame-high: 0 NIC statistics: rx_bytes: 1501719289640 rx_error_bytes: 0 tx_bytes: 4654179094291 tx_error_bytes: 0 rx_ucast_packets: 15253610508 rx_mcast_packets: 2108112 rx_bcast_packets: 1136240 tx_ucast_packets: 15438361249 tx_mcast_packets: 40135 tx_bcast_packets: 1721 tx_mac_errors: 0 tx_carrier_errors: 0 rx_crc_errors: 0 rx_align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 tx_deferred: 0 tx_excess_collisions: 0 tx_late_collisions: 0 tx_total_collisions: 0 rx_fragments: 0 rx_jabbers: 0 rx_undersize_packets: 0 rx_oversize_packets: 0 rx_64_byte_packets: 92376678 rx_65_to_127_byte_packets: 1183040190 rx_128_to_255_byte_packets: 788176623 rx_256_to_511_byte_packets: 126838328 rx_512_to_1023_byte_packets: 168170816 rx_1024_to_1522_byte_packets: 13350337 rx_1523_to_9022_byte_packets: 0 tx_64_byte_packets: 60806588 tx_65_to_127_byte_packets: 1955234150 tx_128_to_255_byte_packets: 2806601346 tx_256_to_511_byte_packets: 154015585 tx_512_to_1023_byte_packets: 1466206531 tx_1024_to_1522_byte_packets: 405928513 tx_1523_to_9022_byte_packets: 0 rx_xon_frames: 0 rx_xoff_frames: 0 tx_xon_frames: 150648 tx_xoff_frames: 173552 rx_mac_ctrl_frames: 0 rx_filtered_packets: 1 rx_ftq_discards: 0 rx_discards: 0 rx_fw_discards: 19605427 sysctl linecount: 504 sysctl hash: 4626e3788c72e091487afe1e3a7cfd32278ab07d - ____________________ hosthash: fc76bb3ee1ff580c6aba0d685713bb4145bd5fe3 - Linux 2.6.18-308.16.1.el5 #1 SMP Tue Sep 18 07:21:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux IBM System x3550 -[7978AC1]- bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.1.11 (July 20, 2011) eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem c8000000, IRQ 90, node addr 001a649dc68a eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem ce000000, IRQ 177, node addr 001a649dc68c cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.7 (July 20, 2011) Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.0.3 (Aug 04, 2011) processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5420 @ 2.50GHz stepping : 6 cpu MHz : 2493.750 cache size : 6144 KB physical id : 0 siblings : 4 12:00:01 AM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s Average: lo 1891.67 1891.67 266593.77 266593.77 0.00 0.00 0.00 Average: eth0 7066.44 7345.95 730519.41 2215508.99 0.00 0.00 4.37 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:00:01 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 driver: bnx2 version: 2.1.11 firmware-version: bc 4.0.3 ipms 1.6.0 bus-info: 0000:04:00.0 Ring parameters for eth0: Pre-set maximums: RX: 2040 RX Mini: 0 RX Jumbo: 8160 TX: 255 Current hardware settings: RX: 1020 RX Mini: 0 RX Jumbo: 0 TX: 255 Coalesce parameters for eth0: Adaptive RX: off TX: off stats-block-usecs: 999936 sample-interval: 0 pkt-rate-low: 0 pkt-rate-high: 0 rx-usecs: 18 rx-frames: 12 rx-usecs-irq: 18 rx-frames-irq: 2 tx-usecs: 80 tx-frames: 20 tx-usecs-irq: 18 tx-frames-irq: 2 rx-usecs-low: 0 rx-frame-low: 0 tx-usecs-low: 0 tx-frame-low: 0 rx-usecs-high: 0 rx-frame-high: 0 tx-usecs-high: 0 tx-frame-high: 0 NIC statistics: rx_bytes: 4640887074833 rx_error_bytes: 0 tx_bytes: 12640942400790 tx_error_bytes: 0 rx_ucast_packets: 46405845860 rx_mcast_packets: 14487857 rx_bcast_packets: 3476467 tx_ucast_packets: 47159091638 tx_mcast_packets: 118147 tx_bcast_packets: 5504 tx_mac_errors: 0 tx_carrier_errors: 0 rx_crc_errors: 0 rx_align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 tx_deferred: 0 tx_excess_collisions: 0 tx_late_collisions: 0 tx_total_collisions: 0 rx_fragments: 0 rx_jabbers: 0 rx_undersize_packets: 0 rx_oversize_packets: 0 rx_64_byte_packets: 136463411 rx_65_to_127_byte_packets: 4245502343 rx_128_to_255_byte_packets: 2357984838 rx_256_to_511_byte_packets: 355610202 rx_512_to_1023_byte_packets: 608223572 rx_1024_to_1522_byte_packets: 65320154 rx_1523_to_9022_byte_packets: 0 tx_64_byte_packets: 112166114 tx_65_to_127_byte_packets: 3010346100 tx_128_to_255_byte_packets: 4087240164 tx_256_to_511_byte_packets: 1625596725 tx_512_to_1023_byte_packets: 3037109096 tx_1024_to_1522_byte_packets: 927187571 tx_1523_to_9022_byte_packets: 0 rx_xon_frames: 0 rx_xoff_frames: 0 tx_xon_frames: 79164 tx_xoff_frames: 89685 rx_mac_ctrl_frames: 0 rx_filtered_packets: 1 rx_ftq_discards: 0 rx_discards: 0 rx_fw_discards: 6857729 sysctl linecount: 504 sysctl hash: dd6aab90d0fd9ae90742c5f812a78734e2f2ff1c - ____________________ hosthash: b7550c65d37622a131b1e47f066773defbb4d817 - Linux 2.6.18-308.16.1.el5 #1 SMP Tue Sep 18 07:21:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux IBM System x3550 -[7978EHU]- bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.1.11 (July 20, 2011) eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem c8000000, IRQ 90, node addr 00215e3f1ec4 eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X 64-bit 133MHz found at mem ce000000, IRQ 177, node addr 00215e3f1ec6 processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 23 model name : Intel(R) Xeon(R) CPU E5420 @ 2.50GHz stepping : 6 cpu MHz : 2493.753 cache size : 6144 KB physical id : 1 siblings : 4 12:00:01 AM IFACE rxpck/s txpck/s rxbyt/s txbyt/s rxcmp/s txcmp/s rxmcst/s Average: lo 1883.04 1883.04 263726.79 263726.79 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth1 7019.18 7312.47 720911.92 2214861.10 0.00 0.00 1.02 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:00:01 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s txcarr/s rxfram/s rxfifo/s txfifo/s Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: sit0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 driver: bnx2 version: 2.1.11 firmware-version: bc 4.0.3 ipms 1.6.0 bus-info: 0000:06:00.0 Ring parameters for eth1: Pre-set maximums: RX: 2040 RX Mini: 0 RX Jumbo: 8160 TX: 255 Current hardware settings: RX: 255 RX Mini: 0 RX Jumbo: 0 TX: 255 Coalesce parameters for eth1: Adaptive RX: off TX: off stats-block-usecs: 999936 sample-interval: 0 pkt-rate-low: 0 pkt-rate-high: 0 rx-usecs: 18 rx-frames: 12 rx-usecs-irq: 18 rx-frames-irq: 2 tx-usecs: 80 tx-frames: 20 tx-usecs-irq: 18 tx-frames-irq: 2 rx-usecs-low: 0 rx-frame-low: 0 tx-usecs-low: 0 tx-frame-low: 0 rx-usecs-high: 0 rx-frame-high: 0 tx-usecs-high: 0 tx-frame-high: 0 NIC statistics: rx_bytes: 4621548539323 rx_error_bytes: 0 tx_bytes: 12598031299743 tx_error_bytes: 0 rx_ucast_packets: 46260356368 rx_mcast_packets: 5352446 rx_bcast_packets: 3474589 tx_ucast_packets: 47008853953 tx_mcast_packets: 118164 tx_bcast_packets: 5471 tx_mac_errors: 0 tx_carrier_errors: 0 rx_crc_errors: 0 rx_align_errors: 0 tx_single_collisions: 0 tx_multi_collisions: 0 tx_deferred: 0 tx_excess_collisions: 0 tx_late_collisions: 0 tx_total_collisions: 0 rx_fragments: 0 rx_jabbers: 0 rx_undersize_packets: 0 rx_oversize_packets: 0 rx_64_byte_packets: 126851062 rx_65_to_127_byte_packets: 4117708205 rx_128_to_255_byte_packets: 2346047550 rx_256_to_511_byte_packets: 356266112 rx_512_to_1023_byte_packets: 604666332 rx_1024_to_1522_byte_packets: 62938478 rx_1523_to_9022_byte_packets: 0 tx_64_byte_packets: 111216848 tx_65_to_127_byte_packets: 2984505931 tx_128_to_255_byte_packets: 4027485330 tx_256_to_511_byte_packets: 1577669672 tx_512_to_1023_byte_packets: 3015060448 tx_1024_to_1522_byte_packets: 933575954 tx_1523_to_9022_byte_packets: 0 rx_xon_frames: 0 rx_xoff_frames: 0 tx_xon_frames: 129873 tx_xoff_frames: 145090 rx_mac_ctrl_frames: 0 rx_filtered_packets: 1 rx_ftq_discards: 0 rx_discards: 0 rx_fw_discards: 6752713 sysctl linecount: 504 sysctl hash: 4626e3788c72e091487afe1e3a7cfd32278ab07d - 

想知道这个盒子是戴尔吗? 戴尔出货的bnx2i驱动程序和芯片组有一个众所周知的问题。 结果是在networking负荷很重的情况下随机丢弃了数据包 看起来合乎逻辑的是,调整后的环形缓冲区可以触发它,如果是这样的话。

我相信戴尔提供他们自己的驱动程序版本作为修复。 另一个解决方法是在modprobe.conf中这样做:

选项bnx2i disable_msi = 1

无论如何不能伤害尝试。 而x2是什么kce说的。 这是我见过的最好的书面问题之一。

即使您确定您的服务器上有完整的负载均衡器VIP列表,仍然可以运行数据包捕获。 仅仅因为你的机器不会响应ARP的IP地址,并不意味着这些伪造的数据包不能被发送到它。 确保发送到您的MAC地址的stream量与configuration的IP地址匹配。

我很欣赏人们提出这个问题的时候,但是我自己的尽职调查在这里是缺乏的。 事后看来,我需要build立一个像这样的PCAPfilter:

 tcpdump -i eth0 -n 'ether dst aa:bb:cc:dd:ee:ff and not (dst host 1.2.3.4 or dst host 5.6.7.8 or...)' 

哪里:

 aa:bb:cc:dd:ee:ff = HW addr of eth0 1.2.3.4, 5.6.7.8 = list of destination addresses that traffic is expected on 

有一些负载平衡器的VIP没有给我(我不控制LB),他们通过TCP端口53传递stream量的方式会导致RX丢弃。 这些传统IP上的stream量非常低,pipe理员不大可能注意到networking上的stream量。