|
On 14/09/05, Horms <horms@xxxxxxxxxxxx> wrote:
> On Tue, Sep 13, 2005 at 01:46:55PM +0200, mquich wrote:
> > Hi!
> >
> > I'm new to this list, though I've been a reader for long time.
> >
> > I'm trying to set up a LVS but I can't get it working. I've read the
> > HOWTO's, used tcpdump and I can't get what I want.
> >
> > My topology is as follows:
> >
> > -------------------------
> > | INTERNET |
> > -------------------------
> > |
> > -------------------------
> > | <DSL router> |
> > | 192.168.5.10 |
> > -------------------------
> > |
> > -------------------------
> > | SWITCH |
> > -------------------------
> > | |
> > | |
> > ------------------------- -------------------------
> > | <LVS1> | | <LVS2> |
> > | 192.168.5.111 | | 192.168.5.221 |
> > ------------------------- -------------------------
> > | |
> > | |
> > -------------------------
> > | SWITCH |
> > -------------------------
> > |
> > -------------------------
> > | <CLIENTS> |
> > | 192.168.5.0/24 |
> > -------------------------
> >
> > and my configuration is this:
> >
> > LVS1
> > --------
> > IP/MASK: 192.168.5.111/24 (No eth, just br0 -BRIDGE-)
> > GW: 192.168.5.10
> > DIRECTOR & REALSERVER
> > LVS-DR
> > SQUID
> > IPTABLES:
> > iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
> > MARK --set-mark 80
> > iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d
> > ! 127.0.0.1 --dport 80 -j REDIRECT --to-ports 3128
> >
> > LVS2
> > --------
> > IP/MASK: 192.168.5.221/24 (No eth, just br0 -BRIDGE-)
> > GW: 192.168.5.10
> > DIRECTOR & REALSERVER
> > LVS-DR
> > SQUID
> > IPTABLES:
> > iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
> > MARK --set-mark 81
> > iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d
> > ! 127.0.0.1 --dport 80 -j REDIRECT --to-ports 3128
> >
> > CLIENTS
> > --------------
> > IP/MASK: 192.168.5.0/24 (ethx, just one & no br0 -BRIDGE-)
> > GW: 192.168.5.10
> >
> > Both LVS boxes are transparent to clients, nothing is configured in
> > the clients with such IP's, but they are forced to pass trough the LVS
> > boxes to get outside!. So when a client wants a web page from the
> > outside world, I want that LVS balances so that one page is get by
> > squid on LVS1 and others by squid on LVS2
> >
> > I haven't been able to get the ipvsadm commands for this to work as I want
> > :-(
> > I've tried "hidden patch" to solve arp problem but I can't get it working.
> >
> > Anyone could help me?
>
> What does your LVS setup look like?
Exactly, my configuration is more complex than this, but I've tried to
make it simpler. I use several more programs (keepalived,
dansguardian, squid ... and more) but as I've told, let's simplify the
problem. From now on, I'm going to put my real configuration files
Here's my setup:
LVS1
---------
ipvsadm -A -f 80 -s rr
ipvsadm -a -f 80 -r 192.168.5.221:0 -g -w 1
ipvsadm -a -f 80 -r 192.168.5.111:0 -g -w 1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
MARK --set-mark 80
iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d !
127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080 ==> 8080=dansguardian
port!!!
LVS2
---------
ipvsadm -A -f 81 -s rr
ipvsadm -a -f 81 -r 192.168.5.111:0 -g -w 1
ipvsadm -a -f 81 -r 192.168.5.221:0 -g -w 1
iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
MARK --set-mark 81
iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d !
127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080 ==> 8080=dansguardian
port!!!
>From a client, I do:
lynx www.virtualserver.org
LVS1# ipvsadm -l
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 80 rr
-> 192.168.5.221:0 Route 1 0 1
-> 192.168.5.111:0 Local 1 0 0
LVS2# ipvsadm -l
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 81 rr
-> 192.168.5.111:0 Route 1 0 0
-> 192.168.5.221:0 Local 1 0 0
This time, LVS1 has taken the decision to give the packet to LVS2 but
the packet arrives to it (I can see it with tcpdump) but with
dst=LVS1_IP and LVS does nothing, even "ipvsadm -l" doesn't show the
connection. When LVS decides to give the packet to himself, everything
is ok:
# lynx www.virtualserver.org
LVS1# ipvsadm -l
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 80 rr
-> 192.168.5.221:0 Route 1 0 0
-> 192.168.5.111:0 Local 1 0 1
This shows the page on lynx.
Another configuration issue is that the bridges have STP=on in order
to avoid loops and one of the ports is in "blocking" state, that why
LVS always acts as director:
|
-------------------------
| SWITCH |
-------------------------
| |
| |
------------------------- -------------------------
| <LVS1> | | <LVS2> |
| 192.168.5.111 | | 192.168.5.221 |
------------------------- -------------------------
| | ==> br0
this port is in "blocking" state
| |
-------------------------
| SWITCH |
-------------------------
|
> Are the packets going to LOCAL_IN?
Yes, with tcpdump I can see the packets getting to the LVSx (I use rr
and it's "this is for me & the next is for you" -ok, not always!!!-)
> Does enabling /proc/sys/net/ipv4/vs/debug
> and monitoring the kernel logs yeild anything?
I haven't got this /proc/sys/net/ipv4/vs/debug on my machines ...
maybe I must recompile the kernel? Which option must I use?
My machine is Fedora Core 3 with kernel 2.6.9-1.667
Thanks!
|
