|
Julian Anastasov wrote:
> Hello,
>
> On Wed, 21 Jun 2000, Thomas A. Morris wrote:
>
> > I am chasing a problem now where persistent mode seems not to work. I'm
> > using
> > firewall marks and NAT. consider the following snoop trace where
> > 172.16.0.146
> > is the client, and morrist-2 and morrist-3 are real servers. The initial
> > transactions occur with morrist-3, but suddenly, the MOUNT request goes to
> > morrist-2!:
> > 172.16.0.146 -> morrist-3.cae.crosstor.com TCP D=111 S=907
> > Ack=3651063116
> > Seq=2479735819 Len=0 Win=32120 Options=<nop,nop,tstamp 8970796 2909399>
> > 172.16.0.146 -> morrist-3.cae.crosstor.com PORTMAP C DUMP
> > morrist-3.cae.crosstor.com -> 172.16.0.146 TCP D=907 S=111
> > Ack=2479735863
> > Seq=3651063116 Len=0 Win=10136 Options=<nop,nop,tstamp 2909399 8970796>
> > morrist-3.cae.crosstor.com -> 172.16.0.146 PORTMAP R DUMP 18+ map(s) found
> > 172.16.0.146 -> morrist-3.cae.crosstor.com TCP D=111 S=907
> > Ack=3651063516
> > Seq=2479735863 Len=0 Win=31856 Options=<nop,nop,tstamp 8970796 2909399>
> > morrist-3.cae.crosstor.com -> 172.16.0.146 TCP D=907 S=111
> > Ack=2479735863
> > Seq=3651063516 Len=96 Win=10136 Options=<nop,nop,tstamp 2909399 8970796>
> > 172.16.0.146 -> morrist-3.cae.crosstor.com TCP D=111 S=907 Fin
> > Ack=3651063612
> > Seq=2479735863 Len=0 Win=31856 Options=<nop,nop,tstamp 8970796 2909399>
> > morrist-3.cae.crosstor.com -> 172.16.0.146 TCP D=907 S=111
> > Ack=2479735864
> > Seq=3651063612 Len=0 Win=10136 Options=<nop,nop,tstamp 2909399 8970796>
> > morrist-3.cae.crosstor.com -> 172.16.0.146 TCP D=907 S=111 Fin
> > Ack=2479735864
> > Seq=3651063612 Len=0 Win=10136 Options=<nop,nop,tstamp 2909399 8970796>
> > morrist-2.cae.crosstor.com -> nat1.cae.crosstor.com ARP R 192.168.129.42,
> > morrist-2.cae.crosstor.com is 0:90:27:e0:1d:f6
> > The next line is WRONG!!!!!! Should be going to morrist-3
> > 172.16.0.146 -> morrist-2.cae.crosstor.com MOUNT3 C Mount /Drive2
> > 172.16.0.146 -> morrist-3.cae.crosstor.com TCP D=111 S=907
> > Ack=3651063613
> > Seq=2479735864 Len=0 Win=31856 Options=<nop,nop,tstamp 8970796 2909399>
> >
> > My ipvsadm output:
> > IP Virtual Server version 0.9.13 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > FWM 1 rr
> > -> upquark.cae.crosstor.com:0 Masq 1 0 0
> > -> positron.cae.crosstor.com:0 Masq 1 0 0
> > FWM 2 rr persistent 3600
> > -> morrist-3.cae.crosstor.com:0 Masq 1 0 0
> > -> morrist-2.cae.crosstor.com:0 Masq 1 0 0
> > -> morrist-1.cae.crosstor.com:0 Masq 1 0 0
> >
> > ipchains -L output
> >
> > Chain input (policy ACCEPT):
> > target prot opt source destination ports
> > - all ------ anywhere test n/a
> > - all ------ anywhere morrist n/a
> > Chain forward (policy DENY):
> > target prot opt source destination ports
> > MASQ all ------ positron.cae.crosstor.com anywhere n/a
> > MASQ all ------ upquark.cae.crosstor.com anywhere n/a
> > MASQ all ------ morrist-1.cae.crosstor.com anywhere n/a
> > MASQ all ------ morrist-2.cae.crosstor.com anywhere n/a
> > MASQ all ------ morrist-3.cae.crosstor.com anywhere n/a
> > Chain output (policy ACCEPT):
>
> From your info we can't understand what packets are marked. Do
> you?
>
> > Any Ideas before I go and try to understand the ipvs scheduling code?
>
> I assume you try to mark UDP and TCP packets together in
> a fwmark based service. Right?
Yes.... Here is the command sequence used...
ifconfig eth0:1 morrist netmask 255.255.255.255 mtu 1500 up
/sbin/ipchains -A forward -j MASQ -s morrist-1
/sbin/ipchains -A forward -j MASQ -s morrist-2
/sbin/ipchains -A forward -j MASQ -s morrist-3
/sbin/ipchains -A input -d morrist/32 -m 2
/sbin/ipvsadm -A -f 2 -s rr -p 3600
/sbin/ipvsadm -a -f 2 -r morrist-1:0 -m
/sbin/ipvsadm -a -f 2 -r morrist-2:0 -m
/sbin/ipvsadm -a -f 2 -r morrist-3:0 -m
echo 1 > /proc/sys/net/ipv4/conf/all/hidden
config eth0:1 morrist netmask 255.255.255.255 mtu 1500 up
/sbin/ipchains -A forward -j MASQ -s morrist-1
/sbin/ipchains -A forward -j MASQ -s morrist-2
/sbin/ipchains -A forward -j MASQ -s morrist-3
/sbin/ipchains -A input -d morrist/32 -m 2
/sbin/ipvsadm -A -f 2 -s rr -p 3600
/sbin/ipvsadm -a -f 2 -r morrist-1:0 -m
/sbin/ipvsadm -a -f 2 -r morrist-2:0 -m
/sbin/ipvsadm -a -f 2 -r morrist-3:0 -m
echo 1 > /proc/sys/net/ipv4/conf/all/hidden
echo 1 > /proc/sys/net/ipv4/conf/lo/hidden
>
> We forgot to allow this feature when the fwmark service
> was added. IMO, this can be included in the next LVS version after
> a discussion. This is very good feature.
So does this mean that I cannot have both UDP and TCP from a client persistent
to
the same server?
If so, are there any suggested work-arounds?
>
> The proposed solution: don't use <iph->protocol> as hash
> key when creating the templates and register them with a
> constant protocol value (TCP?). Wensong?
>
> Regards
>
> --
> Julian Anastasov <uli@xxxxxxxxxxxxxxxxxxxxxx>
Tom
|
