|
Hi All,
I'm attempting to setup LVS to load balance a couple of servers. I'm
following the steps in the mini-HOWTO and everything goes well until I run
the rc.lvs_dr script. After running this script on the director, my ssh
connection drops and I can't connect to that machine from the outside
world. I *can* connect to it from other machines on the same hub. I'm
running RH 7.3 with a newly compiled 2.4.30smp kernel, dual 2.4GHz Xeon,
1GB RAM. Here's my setup:
My director has 2 NICs
eth0 81.20.71.80 (main IP of the box)
eth1 172.16.16.33 (main internal network IP)
eth1:250 172.16.16.250 (DIP)
I plan on using 81.20.71.94 as my VIP. I have a router at 81.20.71.65 that
acts as the gateway for my public network.
--------- Director ---------------------------------------
eth0 Link encap:Ethernet HWaddr 00:30:48:2B:27:FC
inet addr:81.20.71.80 Bcast:81.20.71.95 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:957911 errors:0 dropped:0 overruns:0 frame:0
TX packets:82926 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:64484323 (61.4 Mb) TX bytes:7544234 (7.1 Mb)
Base address:0x3000 Memory:fc200000-fc220000
eth1 Link encap:Ethernet HWaddr 00:30:48:2B:27:FD
inet addr:172.16.16.33 Bcast:172.16.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:38577 errors:0 dropped:0 overruns:0 frame:0
TX packets:37684 errors:3 dropped:0 overruns:0 carrier:3
collisions:4 txqueuelen:1000
RX bytes:4535466 (4.3 Mb) TX bytes:3387042 (3.2 Mb)
Base address:0x3040 Memory:fc220000-fc240000
eth1:250 Link encap:Ethernet HWaddr 00:30:48:2B:27:FD
inet addr:172.16.16.250 Bcast:172.16.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Base address:0x3040 Memory:fc220000-fc240000
------------------------------------------------
Each realserver has 2 NICs
RS1
eth0 81.20.71.70
eth1 172.16.16.23
--------- RS1 --------------------------------------
eth0 Link encap:Ethernet HWaddr 00:30:48:42:3C:5E
inet addr:81.20.71.70 Bcast:81.20.71.95 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:427726531 errors:0 dropped:0 overruns:0 frame:0
TX packets:410730490 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1113362220 (1061.7 Mb) TX bytes:3461577296 (3301.2 Mb)
Base address:0xa000 Memory:ec000000-ec020000
eth1 Link encap:Ethernet HWaddr 00:30:48:42:3C:5F
inet addr:172.16.16.23 Bcast:172.16.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:110170061 errors:0 dropped:0 overruns:0 frame:0
TX packets:66267121 errors:258504 dropped:0 overruns:0
carrier:258504
collisions:518010 txqueuelen:1000
RX bytes:225675841 (215.2 Mb) TX bytes:2360208878 (2250.8 Mb)
Base address:0xa400 Memory:ec020000-ec040000
------------------------------------------------
RS2
eth0 81.20.71.71
eth1 172.16.16.24
--------- RS2 ---------------------------------------
eth0 Link encap:Ethernet HWaddr 00:30:48:42:3C:20
inet addr:81.20.71.71 Bcast:81.20.71.95 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:165097990 errors:0 dropped:0 overruns:0 frame:0
TX packets:149614276 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2232363075 (2128.9 Mb) TX bytes:4158841509 (3966.1 Mb)
Base address:0xa000 Memory:ec000000-ec020000
eth1 Link encap:Ethernet HWaddr 00:30:48:42:3C:21
inet addr:172.16.16.24 Bcast:172.16.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:34146810 errors:0 dropped:0 overruns:0 frame:0
TX packets:15618345 errors:34105 dropped:0 overruns:0 carrier:34105
collisions:112275 txqueuelen:1000
RX bytes:2764274520 (2636.2 Mb) TX bytes:1444453237 (1377.5 Mb)
Base address:0xa400 Memory:ec020000-ec040000
------------------------------------------------
Here is my lvs_dr.conf:
#----------lvs_dr.conf------------------------------------
LVSCONF_FORMAT=1.1
LVS_TYPE=VS_DR
INITIAL_STATE=on
CLEAR_IPVS_TABLES=yes
VIP=eth0:94 81.20.71.94 255.255.255.255 81.20.71.94
DIP=eth1:250 172.16.16.250 172.16.16.0 255.255.255.0 172.16.16.255
#DIRECTOR_GW=
#SERVICE=t ftp rr RS1 RS2
#SERVICE=t telnet rr RS1,1 RS2,2
SERVICE=t http rr 172.16.16.23 172.16.16.24
#SERVICE=t https rr RS1 RS2
#SERVICE=t nntp wlc RS3
SERVER_VIP_DEVICE=lo:94
SERVER_NET_DEVICE=eth1
SERVER_GW=81.20.71.65
#----------end lvs_dr.conf------------------------------------
Here is the output when I run the rc.lvs_dr script:
-----------rc.lvs_dr-----------------------------------
looking for standard utilities
$FALSE=/bin/false
$PING=/bin/ping -c 1
testing ping
ping can send one packet. is OK.
$FPING=/bin/ping -c 1
$IFCONFIG=/sbin/ifconfig
$NETSTAT=/bin/netstat
$ROUTE=/sbin/route
$AWK=/bin/awk
$AWK=/usr/bin/awk
$GREP=/bin/grep
$HOSTNAME_CMD=/bin/hostname
$UNAME_CMD=/bin/uname
$CAT=/bin/cat
$CUT=/bin/cut
$CUT=/usr/bin/cut
$TAIL=/usr/bin/tail
$XARGS=/usr/bin/xargs
$PS=/bin/ps
$KILL=/bin/kill
$WC=/usr/bin/wc
$TRACEROUTE=/usr/sbin/traceroute
$ARP=/sbin/arp
$TR=/usr/bin/tr
$EXPR=/usr/bin/expr
$CHMOD=/bin/chmod
$MV=/bin/mv
$LM=/bin/ln
$RM=/bin/rm
$MKDIR=/bin/mkdir
$SSH=/usr/bin/ssh
Notice: optional executable ntpd not found
ntp is available from http://www.ntp.org
ntp will keep your servers and director synchronised to within 3msec.
$IP=/sbin/ip
rc.lvs version 0.9.4 Sep 2002
(C) 2000-2002 Joseph Mack jmack@xxxxxxxx, distributed under GPL license
This file is part of the LVS project http://www.linuxvirtualserver.org
setting up ms-13-1000.ganymedemarketing.com
find_System_map
System.map
$SYSTEM_MAP=/boot/System.map
$IPTABLES=/sbin/iptables
$LSMOD=/sbin/lsmod
$RMMOD=/sbin/rmmod
$INSMOD=/sbin/insmod
number nics on director 2
LVS director
find_kernel_function_name_2_4
find_kernel_name_2_4: parameter ip_vs
check_function_in_kernel
function ip_vs_init in kernel.
won't look for module.
searching for ipchains
ipchains not loaded, good
loading ip_tables module
find_kernel_function_name_2_4
find_kernel_name_2_4: parameter ip_tables
check_function_in_kernel
function ipt_tables in kernel.
won't look for module.
setting default policy to ACCEPT for LVS devices
clearing iptables/ipchain rules
showing iptables nat rules
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
showing iptables rules
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
set_director_ip_forwarding on vs-dr director (1 on, 0 off).
set ip_forward OFF for vs-dr director (1 on, 0 off).
proc/sys/net/ipv4/ip_forward 0
director is not gw for realservers: leave icmp redirects on.
setting icmp redirects (1 on, 0 off)
/proc/sys/net/ipv4/conf/all/send_redirects 1
/proc/sys/net/ipv4/conf/default/send_redirects 1
/proc/sys/net/ipv4/conf/eth1/send_redirects 1
81.20.71.94 not found on any network devices, good
adding ethernet device and routing for VIP 81.20.71.94
listing ifconfig info for VIP 81.20.71.94
eth0:94 Link encap:Ethernet HWaddr 00:30:48:2B:27:FC
inet addr:81.20.71.94 Bcast:81.20.71.94 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Base address:0x3000 Memory:fc200000-fc220000
checking VIP 81.20.71.94 is reachable from self (director): PING
81.20.71.94 (81.20.71.94) from 81.20.71.94 : 56(84) bytes of data.
64 bytes from 81.20.71.94: icmp_seq=1 ttl=64 time=0.062 ms
--- 81.20.71.94 ping statistics ---
1 packets transmitted, 1 received, 0% loss, time 0ms
rtt min/avg/max/mdev = 0.062/0.062/0.062/0.000 ms
listing routing info for VIP 81.20.71.94
81.20.71.94 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
clearing ipvsadm table
installing LVS services with ipvsadm
print_service_lines: adding non-persistent service 81.20.71.94:http
checking realserver 172.16.16.23 reachable from director -
PING 172.16.16.23 (172.16.16.23) from 172.16.16.33 : 56(84) bytes of data.
64 bytes from 172.16.16.23: icmp_seq=1 ttl=64 time=0.289 ms
--- 172.16.16.23 ping statistics ---
1 packets transmitted, 1 received, 0% loss, time 0ms
rtt min/avg/max/mdev = 0.289/0.289/0.289/0.000 ms
director starting in ON state
checking realserver 172.16.16.24 reachable from director -
PING 172.16.16.24 (172.16.16.24) from 172.16.16.33 : 56(84) bytes of data.
64 bytes from 172.16.16.24: icmp_seq=1 ttl=64 time=0.241 ms
--- 172.16.16.24 ping statistics ---
1 packets transmitted, 1 received, 0% loss, time 0ms
rtt min/avg/max/mdev = 0.241/0.241/0.241/0.000 ms
director starting in ON state
displaying ipvsadm settings
IP Virtual Server version 1.0.12 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 81.20.71.94:http rr
-> 172.16.16.24:http Route 1 0 0
-> 172.16.16.23:http Route 1 0 0
checking for valid server_gw for vs-dr LVS.
director_vip_device=eth0:94, server_gw=81.20.71.65
default gw for the vs-dr servers is NOT on director, good
DIRECTOR_GW not set, good.
DIRECTOR_GW=
installing default gw for vs-dr
number of default gw 1, first gw 81.20.71.65
deleting current default gw 81.20.71.65
number of default gw 0, first gw
not installing a default gw for LVS_TYPE vs-dr
DIRECTOR_GW=
masquerading handled by LVS code.
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
not adding filter rules.
ntpd not running, won't be restarted
The location of the output file rc.lvs is the default = ./
You can change this (and other output files) by editing the variables
$rc_lvs_home, $rc_mon_home, $rc_ntp_home
Remember to run this rc.lvs script on the real-servers too.
This script will use ssh to do this for you if you run it with
./configure lvs_xx.conf -i
----------------------------------------------
As soon as this script finishes, my ssh connection drops and I can't reach
the director from the outside network on it's main IP (81.20.71.80). As I
said, I *can* reach it from other machines on the same hub and they have
no problem pinging or ssh'ing into the director. To reconnect to the
director from the outside world, I have to do a /etc/init.d/network
restart. As soon as the network is reset, the box is reachable again. Here
is the output of ifconfig after running the rc script on the director:
----------------------------------------------
eth0 Link encap:Ethernet HWaddr 00:30:48:2B:27:FC
inet addr:81.20.71.80 Bcast:81.20.71.95 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:972414 errors:0 dropped:0 overruns:0 frame:0
TX packets:88456 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:65442345 (62.4 Mb) TX bytes:7933913 (7.5 Mb)
Base address:0x3000 Memory:fc200000-fc220000
eth0:94 Link encap:Ethernet HWaddr 00:30:48:2B:27:FC
inet addr:81.20.71.94 Bcast:81.20.71.94 Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Base address:0x3000 Memory:fc200000-fc220000
eth1 Link encap:Ethernet HWaddr 00:30:48:2B:27:FD
inet addr:172.16.16.33 Bcast:172.16.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:40365 errors:0 dropped:0 overruns:0 frame:0
TX packets:39498 errors:4 dropped:0 overruns:0 carrier:4
collisions:8 txqueuelen:1000
RX bytes:4747379 (4.5 Mb) TX bytes:3566720 (3.4 Mb)
Base address:0x3040 Memory:fc220000-fc240000
eth1:250 Link encap:Ethernet HWaddr 00:30:48:2B:27:FD
inet addr:172.16.16.250 Bcast:172.16.16.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Base address:0x3040 Memory:fc220000-fc240000
----------------------------------------------
I don't see any errors in the rc.lvs_dr script output. Can somebody please
shed some light on what's happening? I am testing this using http, but
would eventually like to load balance a couple of DNS and mail servers.
Any help would be appreciated!!
Thanks!
--jeff
|
