透明的HTTP代理与Squid 3.5

几天来,我试图找出如何获得一个透明的HTTPs代理与Squid工作。 我试图实现的是一个代理接受来自端口80和443的互联网stream量,通过Squid将它们路由到Privoxy,最后通过Tor并返回数据。 所以基本上我想通过Tor“自动”还原一些stream量,而用户不需要为其连接添加代理。

我知道如何设置Privoxy和Tor部分,但我正在努力与Squid&IP表configuration。

这是我的设置

下载最新版本

curl -O http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.22.tar.gz && tar zxvf squid-3.5.22.tar.gz && cd squid-3.5.22 

安装所有需要的包

 apt install devscripts build-essential openssl libssl-dev fakeroot libcppunit-dev libsasl2-dev cdbs ccze libfile-readbackwards-perl libcap2 libcap-dev libcap2-dev libnetfilter-conntrack-dev htop ccze sysv-rc-conf -y 

configuration构build,并进行安装

 ./configure \ CHOST="x86_64-pc-linux-gnu" \ CFLAGS="-march=core2 -O2 -pipe" \ CXXFLAGS="${CFLAGS}" \ --build=x86_64-linux-gnu \ --prefix=/usr \ --exec-prefix=/usr \ --bindir=/usr/bin \ --sbindir=/usr/sbin \ --libdir=/usr/lib \ --sharedstatedir=/usr/com \ --includedir=/usr/include \ --localstatedir=/var \ --libexecdir=/usr/lib/squid \ --srcdir=. \ --datadir=/usr/share/squid \ --sysconfdir=/etc/squid \ --infodir=/usr/share/info \ --mandir=/usr/share/man \ --x-includes=/usr/include \ --x-libraries=/usr/lib \ --with-default-user=proxy \ --with-logdir=/var/log/squid \ --with-pidfile=/var/run/squid.pid \ --enable-err-languages=English \ --enable-default-err-language=English \ --enable-storeio=ufs,aufs,diskd \ --enable-linux-netfilter \ --enable-removal-policies=lru,heap \ --enable-gnuregex \ --enable-follow-x-forwarded-for \ --enable-x-accelerator-vary \ --enable-zph-qos \ --enable-delay-pools \ --enable-snmp \ --enable-underscores \ --with-openssl \ --enable-ssl-crtd \ --enable-http-violations \ --enable-async-io=24 \ --enable-storeid-rewrite-helpers \ --with-large-files \ --with-libcap \ --with-netfilter-conntrack \ --with-included-ltdl \ --with-maxfd=65536 \ --with-filedescriptors=65536 \ --with-pthreads \ --without-gnutls \ --without-mit-krb5 \ --without-heimdal-krb5 \ --without-gnugss \ --disable-icap-client \ --disable-wccp \ --disable-wccpv2 \ --disable-dependency-tracking \ --disable-auth --disable-epoll \ --disable-ident-lookups \ --disable-icmp 

允许ip4转发

 echo -e "net.ipv4.ip_forward = 1\nnet.ipv4.conf.default.rp_filter = 0\nnet.ipv4.conf.all.rp_filter = 0\nnet.ipv4.conf.eth0.rp_filter = 0\n" >> /etc/sysctl.conf 

生成证书

 mkdir /etc/squid/ssl_certs && cd /etc/squid/ssl_certs openssl genrsa -out squid.key 2048 openssl req -new -key squid.key -out squid.csr -nodes openssl x509 -req -days 3652 -in squid.csr -signkey squid.key -out squid.crt cat squid.crt squid.key > squid.pem 

生成证书caching

 mkdir /var/lib/squid && chown -R proxy:proxy /var/lib/squid/ /usr/lib/squid/ssl_crtd -c -s /var/lib/squid/ssl_db 

改变文件夹的所有权和权利

 mkdir -p /var/spool/squid chown -R proxy:proxy /etc/squid/squid.conf | chown -R proxy:proxy /usr/lib/squid | chown -R proxy:proxy /var/lib/squid/ssl_db/ | chown -R proxy:proxy /var/spool/squid | chown -R proxy:proxy /var/log/squid | chmod 777 /var/spool/squid | chmod 777 /var/log/squid | chmod 755 /var/lib/squid/ssl_db/certs | chown proxy:proxy /var/log/squid/ 

更改configuration(下图)并初始化caching

 squid -f /etc/squid/squid.conf -z 

redirect端口80和443

 iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3128 iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 3129 

我的实际鱿鱼configuration

 acl localnet src all acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT never_direct allow all always_direct allow all # Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost debug_options ALL,2 visible_hostname squid # stop squid taking forever to restart. shutdown_lifetime 3 # for clients with a configured proxy. http_port 3127 # for clients who are sent here via iptables ... REDIRECT. http_port 3128 tproxy # for https clients who are sent here via iptables ... REDIRECT https_port 3129 tproxy ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.pem sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB sslcrtd_children 8 startup=1 idle=1 # acl step1 at_step SslBump1 # ssl_bump peek step1 # ssl_bump bump all ssl_bump server-first all sslproxy_cert_error allow all sslproxy_flags DONT_VERIFY_PEER via off forwarded_for off request_header_access From deny all request_header_access Server deny all request_header_access WWW-Authenticate deny all request_header_access Link deny all request_header_access Cache-Control deny all request_header_access Proxy-Connection deny all request_header_access X-Cache deny all request_header_access X-Cache-Lookup deny all request_header_access Via deny all request_header_access X-Forwarded-For deny all request_header_access Pragma deny all request_header_access Keep-Alive deny all cache_dir ufs /var/spool/squid 1024 16 256 coredump_dir /var/cache/squid refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 

你可以注意到我是如何与Squid的设置仁慈。 这只是为了testing。

所以现在我所得到的是不拦截也不是tproxy的作品。 如果我使用accel来处理非HTTPSstream量,那么可以,但是没有别的。 如果我照原样使用它,结果是它会挂起客户端的超时时间,然后超时。

这是一个例子。 我在/etc/hosts更改了httpbin.org的IP,并通过squid框redirect。

 ❯ curl -vk https://httpbin.org/ip * Trying *******... * Connected to httpbin.org (*******) port 443 (#0) * TLS 1.2 connection using TLS_RSA_WITH_AES_256_GCM_SHA384 * Server certificate: ****** * Server certificate: Universe > GET /ip HTTP/1.1 > Host: httpbin.org > User-Agent: curl/7.49.1 > Accept: */* > < HTTP/1.1 503 Service Unavailable < Server: squid/3.5.22 < Mime-Version: 1.0 < Date: Mon, 05 Dec 2016 05:43:50 GMT < Content-Type: text/html;charset=utf-8 < Content-Length: 3498 < X-Squid-Error: ERR_CONNECT_FAIL 110 < Vary: Accept-Language < Content-Language: en < X-Cache: MISS from pipik < Connection: close 

在鱿鱼方面

 2016/12/05 05:42:50.362 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 28 2016/12/05 05:42:50.362 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3129 remote=[::] FD 28 flags=25 2016/12/05 05:42:50.363 kid1| 33,2| client_side.cc(3911) httpsSslBumpAccessCheckDone: sslBump needed for local=*******:3129 remote=############# FD 11 flags=17 method 3 2016/12/05 05:42:50.363 kid1| 11,2| client_side.cc(2347) parseHttpRequest: HTTP Client local=*******:3129 remote=############# FD 11 flags=17 2016/12/05 05:42:50.363 kid1| 11,2| client_side.cc(2348) parseHttpRequest: HTTP Client REQUEST: --------- CONNECT *******:3129 HTTP/1.1 Host: *******:3129 ---------- 2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(744) clientAccessCheckDone: The request CONNECT *******:3129 is ALLOWED; last ACL checked: localnet 2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(720) clientAccessCheck2: No adapted_http_access configuration. default: ALLOW 2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(744) clientAccessCheckDone: The request CONNECT *******:3129 is ALLOWED; last ACL checked: localnet 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed. 2016/12/05 05:42:50.379 kid1| 17,2| FwdState.cc(133) FwdState: Forwarding client request local=*******:3129 remote=############# FD 11 flags=17, url=*******:3129 2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for '*******:3129' 2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = ALLOWED 2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = DUNNO 2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(288) peerSelectDnsPaths: ORIGINAL_DST = local=############# remote=*******:3129 flags=25 2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 0 2016/12/05 05:43:50.645 kid1| 4,2| errorpage.cc(1261) BuildContent: No existing error page language negotiated for ERR_CONNECT_FAIL. Using default error file. 2016/12/05 05:43:50.645 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable 2016/12/05 05:43:50.645 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable 2016/12/05 05:43:50.845 kid1| 83,2| client_side.cc(3811) clientNegotiateSSL: clientNegotiateSSL: New session 0x29dda60 on FD 11 (#############:59117) 2016/12/05 05:43:50.943 kid1| 11,2| client_side.cc(2347) parseHttpRequest: HTTP Client local=*******:3129 remote=############# FD 11 flags=17 2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(2348) parseHttpRequest: HTTP Client REQUEST: --------- GET /ip HTTP/1.1 Host: httpbin.org User-Agent: curl/7.49.1 Accept: */* ---------- 2016/12/05 05:43:50.944 kid1| 33,2| QosConfig.cc(145) doTosLocalMiss: QOS: Preserving TOS on miss, TOS=0 2016/12/05 05:43:50.944 kid1| 33,2| client_side_reply.cc(1534) buildReplyHeader: clientBuildReplyHeader: Connection Keep-Alive not requested by admin or client 2016/12/05 05:43:50.944 kid1| 88,2| client_side_reply.cc(2051) processReplyAccessResult: The reply for GET https://httpbin.org/ip is ALLOWED, because it matched (access_log daemon:/var/log/squid/access.log line) 2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(1393) sendStartOfMessage: HTTP Client local=*******:3129 remote=############# FD 11 flags=17 2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(1394) sendStartOfMessage: HTTP Client REPLY: --------- HTTP/1.1 503 Service Unavailable Server: squid/3.5.22 Mime-Version: 1.0 Date: Mon, 05 Dec 2016 05:43:50 GMT Content-Type: text/html;charset=utf-8 Content-Length: 3498 X-Squid-Error: ERR_CONNECT_FAIL 110 Vary: Accept-Language Content-Language: en X-Cache: MISS from squid Connection: close ---------- 2016/12/05 05:43:50.944 kid1| 33,2| client_side.cc(817) swanSong: local=*******:3129 remote=############# flags=17 2016/12/05 05:43:50.944 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable 2016/12/05 05:43:50.944 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable 

我尝试了很多不同的configuration,我已经失去了什么工作,什么不工作。 我可能不理解iptables和squid之间的连接,但是不pipe我读什么,我总是在这里结束。

我感谢任何build议。