|
Just to clarify the packets are going to the loopback of node 1, when they
should be going to node 2. This is shown in the tcpdump output:
Here is the output from the lo device of the first node:
02:10:51.987030 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [.], ack
2970678458, win 115, options [nop,nop,TS val 3044575793 ecr 978483], length
0
02:10:51.987079 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [P.], seq
0:173, ack 1, win 115, options [nop,nop,TS val 3044575793 ecr 978483],
length 173
02:10:51.987426 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [.], ack 2,
win 115, options [nop,nop,TS val 3044575793 ecr 978484], length 0
02:10:51.987480 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [F.], seq
173, ack 2, win 115, options [nop,nop,TS val 3044575793 ecr 978484], length
0
Thanks,
Jacoby
On Wed, Jan 22, 2014 at 7:04 PM, Jacoby Hickerson <hickersonjl@xxxxxxxxx>wrote:
> I spoke to soon about this configuration working, the output of ipvsadm
> lead me to believe connections and packets were being load balanced,
> however they are now all coming from the real server which also is the
> director. I'll call this the 'first node'.
>
> Here is some more info in addition to the details in my previous emails:
> First node routing:
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 172.17.0.16 0.0.0.0 UG 0 0 0
> bond0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> bond0
>
> Second node routing:
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 172.17.0.16 0.0.0.0 UG 0 0 0
> bond0
> 172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0
> bond0
>
> The second node also has the VIP on the loopback device
>
> How I setup the director
> ipvsadm -C
> ipvsadm -A -f 100 -s rr
> ipvsadm -a -f 100 -r 172.17.0.16 -w 100
> ipvsadm -a -f 100 -r 172.17.0.17 -w 100
>
> After conducting a few wget's here are the expected results from ipvsadm
> while not port mapping:
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Conns InPkts OutPkts InBytes
> OutBytes
> -> RemoteAddress:Port
> FWM 100 12 61 0 5416
> 0
> -> 172.17.0.16:0 6 25 0 2422
> 0
> -> 172.17.0.17:0 6 36 0 2994
> 0
>
> Actual results from ipvsadm when using port mapping:
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Conns InPkts OutPkts InBytes
> OutBytes
> -> RemoteAddress:Port
> FWM 100 2 8 4 778
> 216
> -> 172.17.0.16:0 1 4 4 389
> 216
> -> 172.17.0.17:0 1 4 0 389
> 0
>
> Packets are being sent from the RIP of the first node only. From my
> understanding when using DR OutPkts should always be zero.
>
> Output from tcpdump is a bit strange (first pass):
> 02:10:51.986719 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [S], seq
> 1873056683, win 14600, options [mss 1460,sackOK,TS val 3044575792 ecr
> 0,nop,wscale 7], length 0
> 02:10:51.986719 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [S], seq
> 1873056683, win 14600, options [mss 1460,sackOK,TS val 3044575792 ecr
> 0,nop,wscale 7], length 0
> 02:10:51.986719 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [S], seq
> 1873056683, win 14600, options [mss 1460,sackOK,TS val 3044575792 ecr
> 0,nop,wscale 7], length 0
> 02:10:51.986814 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [S.], seq
> 2970678457, ack 1873056684, win 14480, options [mss 1460,sackOK,TS val
> 978483 ecr 3044575792,nop,wscale 7], length 0
> 02:10:51.986919 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [S.], seq
> 2970678457, ack 1873056684, win 14480, options [mss 1460,sackOK,TS val
> 978483 ecr 3044575792,nop,wscale 7], length 0
> 02:10:51.987030 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [.], ack 1,
> win 115, options [nop,nop,TS val 3044575793 ecr 978483], length 0
> 02:10:51.987030 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [.], ack 1,
> win 115, options [nop,nop,TS val 3044575793 ecr 978483], length 0
> *02:10:51.987030 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [.], ack
> 2970678458, win 115, options [nop,nop,TS val 3044575793 ecr 978483], length
> 0*
> 02:10:51.987079 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [P.], seq
> 1:174, ack 1, win 115, options [nop,nop,TS val 3044575793 ecr 978483],
> length 173
> 02:10:51.987079 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [P.], seq
> 1:174, ack 1, win 115, options [nop,nop,TS val 3044575793 ecr 978483],
> length 173
> *02:10:51.987079 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [P.], seq
> 0:173, ack 1, win 115, options [nop,nop,TS val 3044575793 ecr 978483],
> length 173*
> 02:10:51.987126 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [.], ack 174,
> win 122, options [nop,nop,TS val 978484 ecr 3044575793], length 0
> 02:10:51.987128 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [.], ack 174,
> win 122, options [nop,nop,TS val 978484 ecr 3044575793], length 0
> 02:10:51.987241 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [F.], seq 1,
> ack 174, win 122, options [nop,nop,TS val 978484 ecr 3044575793], length 0
> 02:10:51.987245 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [F.], seq 1,
> ack 174, win 122, options [nop,nop,TS val 978484 ecr 3044575793], length 0
> 02:10:51.987426 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [.], ack 2,
> win 115, options [nop,nop,TS val 3044575793 ecr 978484], length 0
> 02:10:51.987426 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [.], ack 2,
> win 115, options [nop,nop,TS val 3044575793 ecr 978484], length 0
> *02:10:51.987426 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [.], ack
> 2, win 115, options [nop,nop,TS val 3044575793 ecr 978484], length 0*
> 02:10:51.987480 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [F.], seq 174,
> ack 2, win 115, options [nop,nop,TS val 3044575793 ecr 978484], length 0
> 02:10:51.987480 IP 172.17.0.2.54276 > 172.17.0.24.80: Flags [F.], seq 174,
> ack 2, win 115, options [nop,nop,TS val 3044575793 ecr 978484], length 0
> *02:10:51.987480 IP 172.17.0.2.54276 > 172.17.0.16.50000: Flags [F.], seq
> 173, ack 2, win 115, options [nop,nop,TS val 3044575793 ecr 978484], length
> 0*
> 02:10:51.987527 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [.], ack 175,
> win 122, options [nop,nop,TS val 978484 ecr 3044575793], length 0
> 02:10:51.987529 IP 172.17.0.24.80 > 172.17.0.2.54276: Flags [.], ack 175,
> win 122, options [nop,nop,TS val 978484 ecr 3044575793], length 0
>
> Second pass:
> 02:10:59.360433 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [S], seq
> 2134630706, win 14600, options [mss 1460,sackOK,TS val 3044583166 ecr
> 0,nop,wscale 7], length 0
> 02:10:59.360433 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [S], seq
> 2134630706, win 14600, options [mss 1460,sackOK,TS val 3044583166 ecr
> 0,nop,wscale 7], length 0
> 02:10:59.360519 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [S.], seq
> 2952275478, ack 2134630707, win 14480, options [mss 1460,sackOK,TS val
> 985857 ecr 3044583166,nop,wscale 7], length 0
> 02:10:59.360524 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [S.], seq
> 2952275478, ack 2134630707, win 14480, options [mss 1460,sackOK,TS val
> 985857 ecr 3044583166,nop,wscale 7], length 0
> 02:10:59.360627 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [.], ack 1,
> win 115, options [nop,nop,TS val 3044583166 ecr 985857], length 0
> 02:10:59.360627 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [.], ack 1,
> win 115, options [nop,nop,TS val 3044583166 ecr 985857], length 0
> 02:10:59.360728 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [P.], seq
> 1:174, ack 1, win 115, options [nop,nop,TS val 3044583166 ecr 985857],
> length 173
> 02:10:59.360728 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [P.], seq
> 1:174, ack 1, win 115, options [nop,nop,TS val 3044583166 ecr 985857],
> length 173
> 02:10:59.360751 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [.], ack 174,
> win 122, options [nop,nop,TS val 985857 ecr 3044583166], length 0
> 02:10:59.360753 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [.], ack 174,
> win 122, options [nop,nop,TS val 985857 ecr 3044583166], length 0
> 02:10:59.361058 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [F.], seq 1,
> ack 174, win 122, options [nop,nop,TS val 985858 ecr 3044583166], length 0
> 02:10:59.361062 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [F.], seq 1,
> ack 174, win 122, options [nop,nop,TS val 985858 ecr 3044583166], length 0
> 02:10:59.361270 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [F.], seq 174,
> ack 2, win 115, options [nop,nop,TS val 3044583167 ecr 985858], length 0
> 02:10:59.361270 IP 172.17.0.2.54277 > 172.17.0.24.80: Flags [F.], seq 174,
> ack 2, win 115, options [nop,nop,TS val 3044583167 ecr 985858], length 0
> 02:10:59.361331 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [.], ack 175,
> win 122, options [nop,nop,TS val 985858 ecr 3044583167], length 0
> 02:10:59.361334 IP 172.17.0.24.80 > 172.17.0.2.54277: Flags [.], ack 175,
> win 122, options [nop,nop,TS val 985858 ecr 3044583167], length 0
>
> The end result is that the packets are always coming from the first node
> and never balanced to the second node.
>
> Thanks for any further help, seems the solution is really close!
>
> Jacoby
>
>
> On Sat, Jan 18, 2014 at 12:44 AM, Julian Anastasov <ja@xxxxxx> wrote:
>
>>
>> Hello,
>>
>> On Fri, 17 Jan 2014, Jacoby Hickerson wrote:
>>
>> > Thanks Julian! After enabling CONFIG_IP_VS_NFCT and setting conntrack
>> to 1
>> > that resolved the problem.
>> > However, how leery should I be with it consuming memory? Is there a
>> test to
>> > monitor this consumption? Currently the nf_conntrack_max is set to the
>> > default: 65536
>>
>> cat /proc/slabinfo | grep nf_conntrack
>> or 'slabtop' can show the object size used by conntracks.
>> It should be 240+ bytes. You can expect one conntrack per
>> IPVS connection. You can also see conntracks with
>> cat /proc/net/nf_conntrack | less
>>
>> cat /proc/sys/net/netfilter/nf_conntrack_count shows
>> the current number of conntracks. You can look for 'count'
>> at __nf_conntrack_alloc() and nf_conntrack_free() to see
>> how it is implemented.
>>
>> Regards
>>
>> --
>> Julian Anastasov <ja@xxxxxx>
>>
>
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
