我正在尝试为一个项目build立一个LVS群集,而且我在这些networking层面上没有太多的知识。 我遇到了一个问题,即真正的服务器拒绝来自director的任何数据包,但会允许数据包从我的客户端到同一个端口。 我在2.6.16.60-0.21默认内核上使用suse linux。
导演脚本:
#!/bin/bash #----------mini-rc.lvs_dr-director------------------------ #set ip_forward OFF for lvs-dr director (1 on, 0 off) #(there is no forwarding in the conventional sense for LVS-DR) cat /proc/sys/net/ipv4/ip_forward echo "0" >/proc/sys/net/ipv4/ip_forward #director is not gw for realservers: leave icmp redirects on echo 'setting icmp redirects (1 on, 0 off) ' echo "1" >/proc/sys/net/ipv4/conf/all/send_redirects cat /proc/sys/net/ipv4/conf/all/send_redirects echo "1" >/proc/sys/net/ipv4/conf/default/send_redirects cat /proc/sys/net/ipv4/conf/default/send_redirects echo "1" >/proc/sys/net/ipv4/conf/eth1/send_redirects cat /proc/sys/net/ipv4/conf/eth1/send_redirects #add ethernet device and routing for VIP 192.168.1.110 /sbin/ifconfig eth1:110 172.18.9.192 broadcast 172.18.9.192 netmask 255.255.255.255 /sbin/route add -host 172.18.9.192 dev eth1:110 #listing ifconfig info for VIP 192.168.1.110 /sbin/ifconfig eth1:110 #check VIP 192.168.1.110 is reachable from self (director) /bin/ping -c 1 172.18.9.192 #listing routing info for VIP 192.168.1.110 /bin/netstat -rn #setup_ipvsadm_table #clear ipvsadm table /sbin/ipvsadm -C #installing LVS services with ipvsadm #add telnet to VIP with round robin scheduling /sbin/ipvsadm -A -t 172.18.9.192:8080 -s rr #forward telnet to realserver using direct routing with weight 1 #/sbin/ipvsadm -a -t 172.18.9.192:telnet -r 172.18.9.191 -g -w 1 /sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.9.191:8080 -g -w 1 #check realserver reachable from director ping -c 1 172.18.9.191 #forward telnet to realserver using direct routing with weight 1 /sbin/ipvsadm -a -t 172.18.9.192:8080 -r 172.18.1.136:8080 -g -w 1 #check realserver reachable from director ping -c 1 172.18.1.136 #displaying ipvsadm settings /sbin/ipvsadm #not installing a default gw for LVS_TYPE vs-dr #---------------mini-rc.lvs_dr-director------------------------
真正的服务器脚本
#!/bin/bash #----------mini-rc.lvs_dr-realserver------------------ #installing default gw 192.168.1.254 for vs-dr #/sbin/route add default gw 192.168.1.254 /sbin/route add default gw 172.18.9.254 #showing routing table /bin/netstat -rn #checking if DEFAULT_GW 192.168.1.254 is reachable ping -c 1 172.18.9.254 #set_realserver_ip_forwarding to OFF (1 on, 0 off). echo "0" >/proc/sys/net/ipv4/ip_forward cat /proc/sys/net/ipv4/ip_forward #looking for DIP 192.168.1.9 ping -c 1 172.18.9.110 #looking for VIP (will be on director) ping -c 1 172.18.9.192 #install_realserver_vip /sbin/ifconfig lo 172.18.9.192 broadcast 172.18.9.192 netmask 0xffffffff up #ifconfig output /sbin/ifconfig lo:110 #installing route for VIP 192.168.1.110 on device lo:110 /sbin/route add -host 172.18.9.192 dev lo:110 #listing routing info for VIP 192.168.1.110 /bin/netstat -rn #hiding interface lo:0, will not arp echo "1" >/proc/sys/net/ipv4/conf/all/hidden cat /proc/sys/net/ipv4/conf/all/hidden echo "1" >/proc/sys/net/ipv4/conf/lo/hidden cat /proc/sys/net/ipv4/conf/lo/hidden #----------mini-rc.lvs_dr-realserver------------------
上面的arp修复程序不运行,因为/ hidden不存在。 我修改了/etc/sysctl1.conf来添加:
net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 net.ipv4.conf.all.arp_ignore = 1 net.ipv4.conf.all.arp_announce = 2
并运行sysctl -p
当我运行VIP时,在真实服务器上使用tcpdump的数据包看起来像:
sudo /usr/sbin/tcpdump -n -i any port 8080 tcpdump: WARNING: Promiscuous mode not supported on the "any" device tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on any, link-type LINUX_SLL (Linux cooked), capture size 96 bytes 14:37:53.617753 IP 172.16.10.74.4592 > 172.18.9.192.8080: S 3169956976:316995697 6(0) win 65535 14:37:53.617851 IP 172.18.9.192.8080 > 172.16.10.74.4592: R 0:0(0) ack 316995697 7 win 0
当我直接运行到真正的服务器的数据包被推送:
14:58:23.595729 IP 172.16.10.74.4887 > 172.18.9.191.8080: S 2691152412:2691152412(0) win 65535 14:58:23.595794 IP 172.18.9.191.8080 > 172.16.10.74.4887: S 3988256836:3988256836(0) ack 2691152413 win 5840 14:58:23.602995 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 1 win 65535 14:58:23.600264 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 1:483(482) ack 1 win 65535 14:58:23.600275 IP 172.18.9.191.8080 > 172.16.10.74.4887: . ack 483 win 6432 14:58:23.605755 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 1:175(174) ack 483 win 6432 14:58:23.622787 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 483:969(486) ack 175 win 65361 14:58:23.619738 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 175:349(174) ack 969 win 7504 14:58:23.674299 IP 172.16.10.74.4887 > 172.18.9.191.8080: P 969:1473(504) ack 349 win 65187 14:58:23.671410 IP 172.18.9.191.8080 > 172.16.10.74.4887: P 349:524(175) ack 1473 win 8576 14:58:23.852775 IP 172.16.10.74.4887 > 172.18.9.191.8080: . ack 524 win 65012
如果有更多的信息需要帮助debugging这个问题,请让我知道。
发表我的问题在LVS邮件列表和第一个答复问:
在realserver上是httpd在172.18.9.192:8080上监听?
事实certificate,我已经正确地设置了LVS集群,但是JBoss没有设置来处理VIP。