|
On Wed, Sep 14, 2005 at 09:48:57AM +0200, mquich wrote:
> On 14/09/05, Horms <horms@xxxxxxxxxxxx> wrote:
> > On Tue, Sep 13, 2005 at 01:46:55PM +0200, mquich wrote:
> > > Hi!
> > >
> > > I'm new to this list, though I've been a reader for long time.
> > >
> > > I'm trying to set up a LVS but I can't get it working. I've read the
> > > HOWTO's, used tcpdump and I can't get what I want.
> > >
> > > My topology is as follows:
> > >
> > > -------------------------
> > > | INTERNET |
> > > -------------------------
> > > |
> > > -------------------------
> > > | <DSL router> |
> > > | 192.168.5.10 |
> > > -------------------------
> > > |
> > > -------------------------
> > > | SWITCH |
> > > -------------------------
> > > | |
> > > | |
> > > -------------------------
> > > -------------------------
> > > | <LVS1> | | <LVS2> |
> > > | 192.168.5.111 | | 192.168.5.221 |
> > > -------------------------
> > > -------------------------
> > > | |
> > > | |
> > > -------------------------
> > > | SWITCH |
> > > -------------------------
> > > |
> > > -------------------------
> > > | <CLIENTS> |
> > > | 192.168.5.0/24 |
> > > -------------------------
> > >
> > > and my configuration is this:
> > >
> > > LVS1
> > > --------
> > > IP/MASK: 192.168.5.111/24 (No eth, just br0 -BRIDGE-)
> > > GW: 192.168.5.10
> > > DIRECTOR & REALSERVER
> > > LVS-DR
> > > SQUID
> > > IPTABLES:
> > > iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
> > > MARK --set-mark 80
> > > iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d
> > > ! 127.0.0.1 --dport 80 -j REDIRECT --to-ports 3128
> > >
> > > LVS2
> > > --------
> > > IP/MASK: 192.168.5.221/24 (No eth, just br0 -BRIDGE-)
> > > GW: 192.168.5.10
> > > DIRECTOR & REALSERVER
> > > LVS-DR
> > > SQUID
> > > IPTABLES:
> > > iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
> > > MARK --set-mark 81
> > > iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d
> > > ! 127.0.0.1 --dport 80 -j REDIRECT --to-ports 3128
> > >
> > > CLIENTS
> > > --------------
> > > IP/MASK: 192.168.5.0/24 (ethx, just one & no br0 -BRIDGE-)
> > > GW: 192.168.5.10
> > >
> > > Both LVS boxes are transparent to clients, nothing is configured in
> > > the clients with such IP's, but they are forced to pass trough the LVS
> > > boxes to get outside!. So when a client wants a web page from the
> > > outside world, I want that LVS balances so that one page is get by
> > > squid on LVS1 and others by squid on LVS2
> > >
> > > I haven't been able to get the ipvsadm commands for this to work as I
> > > want :-(
> > > I've tried "hidden patch" to solve arp problem but I can't get it working.
> > >
> > > Anyone could help me?
> >
> > What does your LVS setup look like?
>
> Exactly, my configuration is more complex than this, but I've tried to
> make it simpler. I use several more programs (keepalived,
> dansguardian, squid ... and more) but as I've told, let's simplify the
> problem. From now on, I'm going to put my real configuration files
>
> Here's my setup:
>
> LVS1
> ---------
> ipvsadm -A -f 80 -s rr
> ipvsadm -a -f 80 -r 192.168.5.221:0 -g -w 1
> ipvsadm -a -f 80 -r 192.168.5.111:0 -g -w 1
> iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
> MARK --set-mark 80
> iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d !
> 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080 ==> 8080=dansguardian
> port!!!
>
> LVS2
> ---------
> ipvsadm -A -f 81 -s rr
> ipvsadm -a -f 81 -r 192.168.5.111:0 -g -w 1
> ipvsadm -a -f 81 -r 192.168.5.221:0 -g -w 1
> iptables -t mangle -A PREROUTING -i br0 -p tcp -m tcp --dport 80 -j
> MARK --set-mark 81
> iptables -t nat -A PREROUTING -i br0 -p tcp -m tcp -s ! 127.0.0.1 -d !
> 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080 ==> 8080=dansguardian
> port!!!
>
> >From a client, I do:
>
> lynx www.virtualserver.org
>
> LVS1# ipvsadm -l
> IP Virtual Server version 1.2.0 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> FWM 80 rr
> -> 192.168.5.221:0 Route 1 0 1
> -> 192.168.5.111:0 Local 1 0 0
>
> LVS2# ipvsadm -l
> IP Virtual Server version 1.2.0 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> FWM 81 rr
> -> 192.168.5.111:0 Route 1 0 0
> -> 192.168.5.221:0 Local 1 0 0
>
> This time, LVS1 has taken the decision to give the packet to LVS2 but
> the packet arrives to it (I can see it with tcpdump) but with
> dst=LVS1_IP and LVS does nothing, even "ipvsadm -l" doesn't show the
> connection. When LVS decides to give the packet to himself, everything
> is ok:
It seems that the problem is that the packets arriving on LVS2 also get
mached by your iptables mark rule. In any case, to simplify things
further, try only running lvs and iptables on one of the hosts.
> # lynx www.virtualserver.org
>
> LVS1# ipvsadm -l
> IP Virtual Server version 1.2.0 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> FWM 80 rr
> -> 192.168.5.221:0 Route 1 0 0
> -> 192.168.5.111:0 Local 1 0 1
>
> This shows the page on lynx.
>
> Another configuration issue is that the bridges have STP=on in order
> to avoid loops and one of the ports is in "blocking" state, that why
> LVS always acts as director:
>
> |
> -------------------------
> | SWITCH |
> -------------------------
> | |
> | |
> ------------------------- -------------------------
> | <LVS1> | | <LVS2> |
> | 192.168.5.111 | | 192.168.5.221 |
> ------------------------- -------------------------
> | | ==> br0
> this port is in "blocking" state
> | |
> -------------------------
> | SWITCH |
> -------------------------
> |
>
>
> > Are the packets going to LOCAL_IN?
>
> Yes, with tcpdump I can see the packets getting to the LVSx (I use rr
> and it's "this is for me & the next is for you" -ok, not always!!!-)
>
> > Does enabling /proc/sys/net/ipv4/vs/debug
> > and monitoring the kernel logs yeild anything?
>
> I haven't got this /proc/sys/net/ipv4/vs/debug on my machines ...
> maybe I must recompile the kernel? Which option must I use?
Yes, there is you need IPVS_DEBUG, or something like that.
--
Horms
|
