LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Arp a problem?

To: Julian Anastasov <ja@xxxxxx>
Subject: Re: Arp a problem?
Cc: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
From: Ard van Breemen <ard@xxxxxxxxxxxxxxx>
Date: Wed, 1 Nov 2000 12:32:02 +0100
On Tue, Oct 31, 2000 at 04:26:30PM +0000, Julian Anastasov wrote:
> On Tue, 31 Oct 2000, Ard van Breemen wrote:
> > >   I'm not sure whether you solve problem #2, you don't mention
> > > how you avoid using ARP requests from the real servers.
> > This is the setup:
> > eth0: rs ip in a real server net.
> > eth0:x vip in a vip net. For the vip net, no routers exist, but for
> > the real server net the router exists. So, I guess linux would not
> > be a pain and do arps with an IP that is not on the same logical
> > net (hey, physically they are, but who cares...).
>       Don't guess. I can't believe. Prove it with tcpdump output
> from the real server where the ARP request "who-has GATEWAY tell RIP"
> is showed. I expect from Linux 2.2 "who-has GATEWAY tell VIP"!
> > >   Possible with ARP problem too.
> > Even if a realserver arp's the router, the routing to the vip will
> > go through the director by means of gatewaying (Although arp replies
> > will be replied on the same physical net to the requesting mac).
>       Linux 2.2 does not thing in this way. Every local address
> can be announced in the ARP request if it is not hidden.
Allright, it seems you are right. But still, the question the remains:
is arp a problem? Arp is a lowlevel mac<->ip resolution protocol intended
to resolve local subnet traffic. Since routing happens on the ip layer,
no harm is done...:
Explanation:
Machinelist:
MACHINE    MAC               Real-IP          Virtual IP      Director IP
router     0:d0:b7:26:b7:77  192.168.1.105/24
machine1   0:d0:b7:b5:4:22   192.168.1.159/24 192.168.16.2/24 192.168.1.156/24
machine2   0:d0:b7:b5:6:c2   192.168.1.160/24 192.168.16.2/24
machine3   0:d0:b7:b5:3:e7   192.168.1.162/24 192.168.16.2/24

router has a static route for 192.168.16.0/24 using gw 192.168.1.156

First:
nohup tcpdump -w tcparpdump -epi eth1 'arp or tcp[13] & 3 != 0' &
ab -n 100000 -c 100 http://192.168.16.2/
And later:
tcpdump -ner tcparpdump host 192.168.16.2 or host 192.168.1.156 or host 
192.168.1.159 or host 192.168.1.160 or host 192.168.1.162 > /var/tmp/logje

This is the log (nothing cut except remaining n lines):

Router looks for director:
17:16:31.896279 0:d0:b7:26:b7:77 ff:ff:ff:ff:ff:ff 0806 42: arp who-has 
192.168.1.156 tell 192.168.1.105
17:16:31.896440 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0806 60: arp reply 
192.168.1.156 is-at 0:d0:b7:b5:4:22


Router starts sending to the director:
17:16:31.896463 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3978 > 
192.168.16.2.80: S 2976252717:2976252717(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896468 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3977 > 
192.168.16.2.80: S 2987801419:2987801419(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896473 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3976 > 
192.168.16.2.80: S 2978081276:2978081276(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896483 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3979 > 
192.168.16.2.80: S 2978385484:2978385484(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896534 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3980 > 
192.168.16.2.80: S 2985455161:2985455161(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896577 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3981 > 
192.168.16.2.80: S 2978455149:2978455149(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896617 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3982 > 
192.168.16.2.80: S 2982919402:2982919402(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896656 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3983 > 
192.168.16.2.80: S 2976427106:2976427106(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)

Director arps for real server2:
17:16:31.896671 0:d0:b7:b5:4:22 ff:ff:ff:ff:ff:ff 0806 60: arp who-has 
192.168.1.160 tell 192.168.1.159
Another request to the director:
17:16:31.896726 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3984 > 
192.168.16.2.80: S 2986911779:2986911779(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
Syn-Ack from real server1 (also the director):
17:16:31.896767 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3977: S 2982893252:2982893252(0) ack 2987801420 win 32120 <mss 
1460,sackOK,timestamp 43613365[|tcp]> (DF)
The director arps for real server3:
17:16:31.896794 0:d0:b7:b5:4:22 ff:ff:ff:ff:ff:ff 0806 60: arp who-has 
192.168.1.162 tell 192.168.1.159
Syn-Ack from real server1:
17:16:31.896831 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3980: S 2989863536:2989863536(0) ack 2985455162 win 32120 <mss 
1460,sackOK,timestamp 43613365[|tcp]> (DF)
17:16:31.896902 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3985 > 
192.168.16.2.80: S 2988752868:2988752868(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.896917 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3983: S 2976097478:2976097478(0) ack 2976427107 win 32120 <mss 
1460,sackOK,timestamp 43613365[|tcp]> (DF)
17:16:31.896995 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3986 > 
192.168.16.2.80: S 2981277923:2981277923(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.897036 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3987 > 
192.168.16.2.80: S 2976815413:2976815413(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
Real server3 arps router with source=VIP:
17:16:31.897060 0:d0:b7:b5:3:e7 ff:ff:ff:ff:ff:ff 0806 60: arp who-has 
192.168.1.105 tell 192.168.16.2
Router replies to arp:
17:16:31.897078 0:d0:b7:26:b7:77 0:d0:b7:b5:3:e7 0806 42: arp reply 
192.168.1.105 is-at 0:d0:b7:26:b7:77
17:16:31.897122 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3986: S 2984323950:2984323950(0) ack 2981277924 win 32120 <mss 
1460,sackOK,timestamp 43613365[|tcp]> (DF)
Real server2 arps router with source=VIP:
17:16:31.897136 0:d0:b7:b5:6:c2 ff:ff:ff:ff:ff:ff 0806 60: arp who-has 
192.168.1.105 tell 192.168.16.2
Router replies to arp:
17:16:31.897169 0:d0:b7:26:b7:77 0:d0:b7:b5:6:c2 0806 42: arp reply 
192.168.1.105 is-at 0:d0:b7:26:b7:77

Syn-acks go to the router from the realservers 2 and 3:
17:16:31.897182 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3976: S 2979198624:2979198624(0) ack 2978081277 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.897206 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3981: S 2978333310:2978333310(0) ack 2978455150 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.897210 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3984: S 2988282025:2988282025(0) ack 2986911780 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.897244 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3987: S 2978911792:2978911792(0) ack 2976815414 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.897305 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3985: S 2976978323:2976978323(0) ack 2988752869 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)
17:16:31.897335 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3978: S 2989066025:2989066025(0) ack 2976252718 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)
17:16:31.897338 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3979: S 2977099444:2977099444(0) ack 2978385485 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)

Router keeps sending data to the director:
17:16:31.897502 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3988 > 
192.168.16.2.80: S 2981124386:2981124386(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.897561 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3989 > 
192.168.16.2.80: S 2977136894:2977136894(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.897605 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3990 > 
192.168.16.2.80: S 2981161727:2981161727(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.897643 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3991 > 
192.168.16.2.80: S 2989883160:2989883160(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)

Etcetera:
17:16:31.897670 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3988: S 2989754622:2989754622(0) ack 2981124387 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)
17:16:31.897687 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3989: S 2975866290:2975866290(0) ack 2977136895 win 32120 <mss 
1460,sackOK,timestamp 43613366[|tcp]> (DF)
17:16:31.897765 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3992 > 
192.168.16.2.80: S 2975982620:2975982620(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.897784 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3990: S 2988604073:2988604073(0) ack 2981161728 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.897800 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3991: S 2978058861:2978058861(0) ack 2989883161 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)
17:16:31.897912 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3992: S 2986903148:2986903148(0) ack 2975982621 win 32120 <mss 
1460,sackOK,timestamp 43613366[|tcp]> (DF)
17:16:31.897942 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3993 > 
192.168.16.2.80: S 2978794532:2978794532(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.897988 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3994 > 
192.168.16.2.80: S 2984769440:2984769440(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898026 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3995 > 
192.168.16.2.80: S 2981755588:2981755588(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898064 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3996 > 
192.168.16.2.80: S 2985563767:2985563767(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898102 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3997 > 
192.168.16.2.80: S 2988633199:2988633199(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898122 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3993: S 2986790997:2986790997(0) ack 2978794533 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.898156 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3995: S 2979608669:2979608669(0) ack 2981755589 win 32120 <mss 
1460,sackOK,timestamp 43613366[|tcp]> (DF)
17:16:31.898159 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3994: S 2990644204:2990644204(0) ack 2984769441 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)
17:16:31.898252 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3996: S 2976281453:2976281453(0) ack 2985563768 win 32120 <mss 
1460,sackOK,timestamp 8718131[|tcp]> (DF)
17:16:31.898273 0:d0:b7:b5:6:c2 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3997: S 2981729140:2981729140(0) ack 2988633200 win 32120 <mss 
1460,sackOK,timestamp 8979088[|tcp]> (DF)
17:16:31.898366 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3998 > 
192.168.16.2.80: S 2986404545:2986404545(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898404 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.3999 > 
192.168.16.2.80: S 2982686896:2982686896(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898443 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.4000 > 
192.168.16.2.80: S 2975915787:2975915787(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898480 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.4001 > 
192.168.16.2.80: S 2986209458:2986209458(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898501 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3998: S 2978139877:2978139877(0) ack 2986404546 win 32120 <mss 
1460,sackOK,timestamp 43613366[|tcp]> (DF)
17:16:31.898567 0:d0:b7:26:b7:77 0:d0:b7:b5:4:22 0800 74: 192.168.1.105.4002 > 
192.168.16.2.80: S 2981336260:2981336260(0) win 32120 <mss 
1460,sackOK,timestamp 2637698[|tcp]> (DF)
17:16:31.898618 0:d0:b7:b5:3:e7 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.3999: S 2988383827:2988383827(0) ack 2982686897 win 32120 <mss 
1460,sackOK,timestamp 8718132[|tcp]> (DF)
17:16:31.898621 0:d0:b7:b5:4:22 0:d0:b7:26:b7:77 0800 74: 192.168.16.2.80 > 
192.168.1.105.4001: S 2979048979:2979048979(0) ack 2986209459 win 32120 <mss 
1460,sackOK,timestamp 43613366[|tcp]> (DF)

I want to be right on this that I don't have an arp-problem (I have arps,
but without problems). Because if I am wrong, I have to rethink carefully
what I have to do next. So if you still think I am wrong, I want to know
about it!
-- 
Ard van Breemen, T(elegraaf)E(lektronische)M(edia)
http://www.faqs.org/rfcs/rfc1855.html
**THIS E-MAIL MESSAGE IS VIRUS FREE BY COMPLYING TO THE ASCII STANDARD**


<Prev in Thread] Current Thread [Next in Thread>