Netfilter:通过VPN代理无效状态

我的盒子A和远程服务器B(都是Debian jessie)之间有一个OpenVPN。

在服务器B上,我也运行Squid3。

然后,我将A浏览器configuration为使用带有B服务器(10.2.0.1:3128)的VPN地址的远程代理服务器,仅用于端口80(无SSL)。

它工作正常,我可以浏览互联网。 我可以通过whatismyip.com看到请求源自公共B服务器地址。

我还在B服务器上有一个防火墙,其规则如下:

-A INPUT-m state --state INVALID -j NFLOG --nflog-prefix "fp=bad_packets:1" -A INPUT-m state --state INVALID -j DROP 

然后我有时会注意到一些数据包被丢弃了:

Jun 12 09:16:20 ks1 fp = bad_packets:1 IN = tun0 OUT = MAC = SRC = 10.2.0.9 DST = 10.2.0.1 LEN = 40 TOS = 00 PREC = 0x00 TTL = 64 ID = 58858 DF PROTO = TCP SPT = 34458 DPT = 3128 SEQ = 3975722279 ACK = 0 WINDOW = 0 RST URGP = 0 MARK = 0

这并不妨碍浏览,但我猜一定有一些错误的地方,我只是不知道如何find它们。

这发生在Firefox上,也与Opera有关。

这不是什么“问题”,它主要是烦人的,我想知道什么是污染我的防火墙日志;)

编辑

OK,我运行了一个tcpdump,刷新了一个触发DROP的网站:我明白你的意思,但是没有任何解释:DROP发生在45310端口,所以我在这个端口上做了grep:看到RESET到达,几乎都在同一个“seq”(sequence?):3360258722:第一个似乎是合法的,之后有6个…不明白为什么:下一个调查步骤是什么?

 09:58:02.133295 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 104724:105019, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 295 09:58:02.134079 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 105019:106375, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134091 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 106375:107731, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134127 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 107731:109087, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134133 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 109087:109115, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 28 09:58:02.134764 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 109115:110471, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134779 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 110471:111827, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134809 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 111827:112081, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 254 09:58:02.258393 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 106375, win 1444, options [nop,nop,TS val 1013362 ecr 128061282], length 0 09:58:02.264169 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 109087, win 1444, options [nop,nop,TS val 1013363 ecr 128061282], length 0 09:58:02.268597 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 110471, win 1444, options [nop,nop,TS val 1013364 ecr 128061282], length 0 09:58:02.273283 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 112081, win 1444, options [nop,nop,TS val 1013365 ecr 128061282], length 0 09:58:02.557676 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [P.], seq 12580:13402, ack 112081, win 1444, options [nop,nop,TS val 1013436 ecr 128061282], length 822 09:58:02.577739 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112081:112389, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 308 09:58:02.577964 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112389:113541, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 1152 09:58:02.593245 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 113541, win 1444, options [nop,nop,TS val 1013445 ecr 128061393], length 0 09:58:02.596042 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 113541:114897, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596081 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 114897:115211, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 314 09:58:02.596441 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 115211:116567, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596467 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 116567:117923, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596530 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 117923:118114, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 191 09:58:02.596814 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 118114:119470, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596837 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 119470:120826, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596890 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 120826:121063, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 237 09:58:02.611183 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [F.], seq 13402, ack 114897, win 1444, options [nop,nop,TS val 1013450 ecr 128061398], length 0 09:58:02.611740 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.611821 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [F.], seq 121063, ack 13403, win 678, options [nop,nop,TS val 128061402 ecr 1013450], length 0 09:58:02.614315 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.617520 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.618641 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.621415 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.624966 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.625256 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.631926 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258723, win 0, length 0 Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19768 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19769 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19770 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19771 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19772 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 Jun 15 09:58:02 ks1 fp=bad_packets:1 IN=tun0 OUT= MAC= SRC=10.85.0.9 DST=10.85.0.1 LEN=40 TOS=00 PREC=0x00 TTL=64 ID=19773 DF PROTO=TCP SPT=45310 DPT=3128 SEQ=3360258722 ACK=0 WINDOW=0 RST URGP=0 MARK=0 

编辑2我已经上传了一个.zip文件与双方的tcpdump捕获:

http://expirebox.com/download/80b335fb34fab6824d4e35dd3609f628.html md5sum:ba748e6c6e787e81e765b6ef83b7b2cf

希望这会有所帮助

注意:我会编辑这个消息一千次,我认为有正确的语法…(我是一个serverfault.com初学者;)

注意到RST标志正在打开。 看来您的浏览器发送RST数据包到一个已经closures的连接到您的代理服务器。 根据提供的信息很难说明为什么会发生这种情况。 也许如果你设法通过wireshark / tcpdump捕获导致这个连接的连接,你可以find。

我们从正常的数据交换开始,服务器向客户端发送一些数据:

 09:58:02.133295 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 104724:105019, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 295 09:58:02.134079 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 105019:106375, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134091 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 106375:107731, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134127 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 107731:109087, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134133 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 109087:109115, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 28 09:58:02.134764 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 109115:110471, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134779 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 110471:111827, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 1356 09:58:02.134809 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 111827:112081, ack 12580, win 656, options [nop,nop,TS val 128061282 ecr 1013310], length 254 

120ms暂停,客户端正在收到所有收到的数据包,虽然客户端窗口很短(1444),但仍然看起来很好,这可能暗示有一些拥塞:

 09:58:02.258393 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 106375, win 1444, options [nop,nop,TS val 1013362 ecr 128061282], length 0 09:58:02.264169 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 109087, win 1444, options [nop,nop,TS val 1013363 ecr 128061282], length 0 09:58:02.268597 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 110471, win 1444, options [nop,nop,TS val 1013364 ecr 128061282], length 0 09:58:02.273283 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 112081, win 1444, options [nop,nop,TS val 1013365 ecr 128061282], length 0 

270毫秒暂停和另一轮数据交换,现在两种方式,数据确认(括号内)两种方式:

 09:58:02.557676 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [P.], seq 12580:13402, ack 112081, win 1444, options [nop,nop,TS val 1013436 ecr 128061282], length 822 09:58:02.577739 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112081:112389, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 308 09:58:02.577964 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 112389:113541, ack 13402, win 678, options [nop,nop,TS val 128061393 ecr 1013436], length 1152 09:58:02.593245 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [.], ack 113541, win 1444, options [nop,nop,TS val 1013445 ecr 128061393], length 0 

现在变得棘手。 从服务器到客户端有很多数据(7522字节),即使窗口只有1444字节,注意到它是8个数据包:

 09:58:02.596042 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 113541:114897, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596081 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 114897:115211, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 314 09:58:02.596441 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 115211:116567, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596467 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 116567:117923, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596530 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 117923:118114, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 191 09:58:02.596814 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 118114:119470, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596837 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [.], seq 119470:120826, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 1356 09:58:02.596890 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [P.], seq 120826:121063, ack 13402, win 678, options [nop,nop,TS val 128061398 ecr 1013445], length 237 

客户端只收取第一个1356字节(第一个数据包)并closures(FIN)连接(括号中的F)

 09:58:02.611183 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [F.], seq 13402, ack 114897, win 1444, options [nop,nop,TS val 1013450 ecr 128061398], length 0 

现在客户端发送一个不一致的seq的RST。 TBH我不知道为什么seq是这个错误。

 09:58:02.611740 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 

服务器确认[。]连接closures(FIN)[F]。

 09:58:02.611821 IP 10.85.0.1.3128 > 10.85.0.9.45310: Flags [F.], seq 121063, ack 13403, win 678, options [nop,nop,TS val 128061402 ecr 1013450], length 0 

客户端拒绝剩余七个发送出窗口的数据包。 这些进入正确closures的连接(FIN,FIN / ACK),所以它们不匹配有效状态(因此logging为无效)。

 09:58:02.614315 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.617520 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.618641 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.621415 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.624966 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.625256 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258722, win 0, length 0 09:58:02.631926 IP 10.85.0.9.45310 > 10.85.0.1.3128: Flags [R], seq 3360258723, win 0, length 0 

我不是安全专家,但是从RST范围以外的数据和通过窗口发送的数据判断,你的堆栈似乎有些问题。 这可能只是一些调整疯狂,或一些错误的filter/ mangling规则,但如果没有正当理由,我会检查我的系统的恶意代码在这一点上。 或者路上可能会有一些stream氓networking硬件。

连接的客户端上的转储可能提供一些洞察发生的事情。