|
Hi,
On Tue, 12 Aug 2003, Ian G Batten wrote:
> [[ Gareth, for information ]]
>
> I am slightly confused as to what I have done wrong here. I suspect the
> problem may be that I am over-complicating things by trying to avoid
> making changes to existing machines in my DMZ, but I am puzzled as to
> why I am seeing no LVS activity at all.
>
> I have a network, 192.65.220.0/24, which is my DMZ. I want to use LVS
> to protect a few services, but I would like to avoid subnetting
> 192.65.220.0/24 if I can (it involves modifying some production
> machines). My initial test case is to create a virtual IP number
> 192.65.220.53 which load balances over SMTP servers on 192.65.220.98 and
> 192.65.220.99. The director machine is 192.65.220.19.
>
> I have installed LVS on 192.65.220.19, which is running kernel 2.4.21
> pulled from kernel.org patched with linux-2.4.21-ipvs-1.0.10.patch, laid
> over RedHat 9.0. I have installed ipvsadm again from source, and it
> reports ``IP Virtual Server version 1.0.10 (size=4096)''.
>
> I like to have failover to a spare director, and I have OSPF running
> over all machines in my DMZ. All the machines in the 192.65.220.0/24
> network support variable subnet masks. My plan is to have a standby
> director machine with its interface bandwidth set slightly lower, so
> that it provides failover. This hopefully saves me having to get vsrp
> or whatever it's called working.
>
> I therefore proceeded as follows:
>
> * On 192.65.220.98 and 192.65.220.99 I created a loopback interface,
> lo:1, numbered 192.65.220.53/30. I ensured that the OSPF process
> (zebra/ospfd) did not advertise these interfaces.
>
Well, OSPF probably cannot control the kernel of 192.65.220.98 and
192.65.220.99 not to do ARP response for 192.65.220.53. Please have a look
at http://linuxx-vs.org/docs/arp.html or LVS-HOWTO for ARP issues in the
Direct Routing method.
> * On 192.65.220.19, I created an interface eth0:1, numbered
> 192.65.220.52/30. This provides an advertised for the VIP subnet.
>
> * On 192.65.220.19, I created an iptables rule to tag incoming traffic
> to the VIP:
>
> [root@director1 tmp]# /sbin/iptables -F -t mangle
> [root@director1 tmp]# /sbin/iptables -t mangle -A PREROUTING \
> -i eth0 -p tcp -s 0.0.0.0/0 -d 192.65.220.53/32 \
> --dport 25 -j MARK --set-mark 1
> [root@director1 tmp]# /sbin/iptables -t mangle -L -v
> Chain PREROUTING (policy ACCEPT 8382 packets, 1010K bytes)
> pkts bytes target prot opt in out source
> destination
> 0 0 MARK tcp -- eth0 any anywhere
> 192.65.220.53 tcp dpt:smtp MARK set 0x1
>
> [...]
>
> I created what I think is the appropriate ipvsadm configuration:
>
> [root@director1 tmp]# ipvsadm --clear
> [root@director1 tmp]# ipvsadm -A -f 1 -s rr
> [root@director1 tmp]# ipvsadm -a -f 1 -r 192.65.220.99:25 -g -w 1
> [root@director1 tmp]# ipvsadm -a -f 1 -r 192.65.220.98:25 -g -w 1
> [root@director1 tmp]# ipvsadm --list
> IP Virtual Server version 1.0.10 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> FWM 1 rr
> -> 192.65.220.98:25 Route 1 0 0
> -> 192.65.220.99:25 Route 1 0 0
> [root@director1 tmp]#
>
> The reason I have only put 192.65.220.99 into the pool will become
> apparent in a moment.
>
> Looking at the routing tables on the routers pointing into the
> 192.65.220.0/24 network, they appear correct:
>
> Routing entry for 192.65.220.0/24, 2 known subnets
> Attached (1 connections)
> Variably subnetted with 2 masks
> Redistributing via ospf 1, rip
>
> C 192.65.220.0/24 is directly connected, FastEthernet0/0
> O 192.65.220.52/30 [110/2] via 192.65.220.19, 00:10:02, FastEthernet0/0
>
> Now, telneting to port 25 on 192.65.220.53 yields this result:
>
> elara:/u/igb 10:29:29 (512)
> $ telnet 192.65.220.53 25
> Trying 192.65.220.53...
> Connected to 192.65.220.53.
> Escape character is '^]'.
> 220 ext-proxy-2.ftel.co.uk ESMTP at Tue, 12 Aug 2003 10:29:33 +0100
>
> That's promising, but unfortunately it _always_, from all client
> machines, gets ext-proxy-2 (192.65.220.98) --- a faster machine than
> 192.65.220.99. And looking again at ipvsadm, there is no sign a
> connection has been seen:
>
> [root@director1 tmp]# ipvsadm --list -n
> IP Virtual Server version 1.0.10 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> FWM 1 rr
> -> 192.65.220.98:25 Route 1 0 0
> -> 192.65.220.99:25 Route 1 0 0
> [root@director1 tmp]#
>
> But the iptables rule is definitely matching:
>
> [root@director1 tmp]# /sbin/iptables -L -v -n -t mangle
> Chain PREROUTING (policy ACCEPT 9405 packets, 1107K bytes)
> pkts bytes target prot opt in out source
> destination
> 14 588 MARK tcp -- eth0 * 0.0.0.0/0
> 192.65.220.53 tcp dpt:25 MARK set 0x1
> [...]
> [root@director1 tmp]#
>
> I presume the ipvs isn't being used, and what I've built is a slightly
> complex router. Traceroute confirms this:
>
> $ traceroute 192.65.220.53
> traceroute to 192.65.220.53 (192.65.220.53), 30 hops max, 40 byte packets
> 1 extreme6808 (172.16.2.111) 1.307 ms 0.561 ms 0.518 ms
> 2 inside-firewall-2 (192.150.140.3) 1.408 ms 1.259 ms 1.314 ms
> 3 director1 (192.65.220.19) 2.898 ms 2.711 ms 2.723 ms
> 4 192.65.220.53 (192.65.220.53) 2.793 ms 2.78 ms 2.768 ms
>
> One suspicion is that the eth0:1 interface on 192.65.220.19, numbered
> 192.65.220.52/30, is my problem. It's only there to make sure a route
> is advertised, and perhaps it breaks however ipvs obtains the packets.
> But I'm overall lost, and I'd appreciate any advice.
>
Maybe you can run ethereal or tcpdump on the host 192.65.220.19, and see
if it receives and sends the packet destined for 192.65.220.53.
Regards,
Wensong
|
