|
Hey Malcolm,
Both the Director (10.80.214.72) and RealServer (10.80.214.70) have one
physical interface (eth0). The Director is configured with the VIP
(10.80.214.69) on eth0:0 and the RealServer with the VIP on tunl0. Neither
box has any firewall rules. Both are running kernel 2.6.20.21.
I stumbled upon a number of issues in the mail archives that depict the
issue we¹re experiencing:
1. ³SYN_RECV state symptoms in LVS-DR²
http://lists.graemef.net/pipermail/lvs-users/2004-May/011770.html
2. ³LVS hangs in SYN_RECV state²
http://www.in-addr.de/pipermail/lvs-users/2004-February/011033.html
3. http://archives.free.net.ph/message/20010517.104313.cd1a1128.en.html
4. ³hold at SYN_RECV / LVS-Tun over Internet / one InActConn
³ http://lists.graemef.net/pipermail/lvs-users/2005-September/014786.html
Both the LVS/TUN and LVS/DR setups have worked - - just intermittently. A
ton of SYN_RECV state connections show up in the connection tracking table
when this issue occurs.
Output from ipvsadm on Director:
# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.80.214.69:ssh rr
-> 10.80.214.70:ssh Tunnel 1 0 0
TCP 10.80.214.69:10000 rr
-> 10.80.214.70:10000 Tunnel 1 0 3
TCP 10.80.214.69:60001 rr
-> 10.80.214.70:60001 Tunnel 1 0 391
TCP 10.80.214.69:60000 rr
-> 10.80.214.70:60000 Tunnel 1 0 3
Output from ipvsadm lnc on Director:
# ipvsadm -lnc
IPVS connection entries
pro expire state source virtual destination
TCP 01:00 SYN_RECV 10.80.214.123:3173 10.80.214.69:60001
10.80.214.70:60001
TCP 01:00 SYN_RECV 10.80.214.123:3135 10.80.214.69:60001
10.80.214.70:60001
TCP 01:00 SYN_RECV 10.80.214.120:3206 10.80.214.69:60001
10.80.214.70:60001
.... goes on for many more entries...
TCP 01:00 SYN_RECV 10.80.214.120:3035 10.80.214.69:60001
10.80.214.70:60001
TCP 01:00 SYN_RECV 10.80.214.122:3053 10.80.214.69:60001
10.80.214.70:60001
TCP 01:00 SYN_RECV 10.80.214.123:2916 10.80.214.69:60001
10.80.214.70:60001
Output from route n on Director and RS are identical:
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
10.80.214.64 0.0.0.0 255.255.255.192 U 0 0 0 eth0
0.0.0.0 10.80.214.65 0.0.0.0 UG 0 0 0 eth0
Thoughts? Thanks for your help, Malcolm.
Lee
From: Malcolm Turnbull <malcolm@xxxxxxxxxxxxxxxx>
Reply-To: "LinuxVirtualServer.org users mailing list."
<lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Date: Mon, 11 Aug 2008 13:38:50 -0700
To: "LinuxVirtualServer.org users mailing list."
<lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Recursive SYN packets sent from Director
toRealServer
Lee,
Its unlikely to be a bug.
Sounds like its related to the real server configuration.
Just a thought do you have more than one network card on the LVS box?
You're only blocking ARPs on eth0 at the moment.
Get DR working first and then move on to TUN (have you seen the recent
posts on MTU size?
2008/8/11 Calcote, Lee <lcalcote@xxxxxxxxx>
>
> Hi,
>
> We¹re having intermittent luck using LVS/TUN and LS/DR while trying to
> load-balance web services running on high port numbers: 10000, 60000, and
> 60001. To handle the ARP issue, we¹re using the hidden interface approach
> with the following sysctl settings on the real servers:
>
> > net.ipv4.conf.eth0.arp_ignore = 1
> > net.ipv4.conf.eth0.arp_announce = 2
> > net.ipv4.conf.all.arp_ignore = 1
> > net.ipv4.conf.all.arp_announce = 2
> > net.ipv4.conf.tunl0.arp_ignore = 1
> > net.ipv4.conf.tunl0.arp_announce = 2
>
> We find that client making HTTP requests at <VIP>:10000 (Webmin),
> <VIP>:60000 and <VIP>:60001 (both in-house web services) are able to
> successfully connect to real servers only intermittently. During failed
> requests, we find the Director is generating SYN after SYN request to the
> real server. The real server receives these (many thousand) SYN requests but
> sends no reply (SYN, ACK). One of the mysteries here is that at other times
> the same client will make a request and successfully connect to the web
> service. We¹ve test the load-balancing of SSH and had a 100% success rate.
>
> Does anyone know if this is a bug with LVS or have suggestions on what
> further troubleshooting may be done to identify the issue? Any help would be
> appreciated.
>
> Thanks,
> Lee
>
> -
------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this transmission is
legally privileged and confidential, intended only for the use of the
individual(s) or entities named above. This email and any files transmitted with
it are the property of Pelco. If the reader of this message is not the intended
recipient, or an employee or agent responsible for delivering this message to
the intended recipient, you are hereby notified that any review, disclosure,
copying, distribution, retention, or any action taken or omitted to be taken in
reliance on it is prohibited and may be unlawful. If you receive this
communication in error, please notify us immediately by telephone call to
+1-559-292-1981 or forward the e-mail to administrator@xxxxxxxxx and then
permanently delete the e-mail and destroy all soft and hard copies of the
message and any attachments. Thank you for your cooperation.
> -
------------------------------------------------------------------------------
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
--
Regards,
Malcolm Turnbull.
Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/
- ------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this transmission is
legally privileged and confidential, intended only for the use of the
individual(s) or entities named above. This email and any files transmitted
with it are the property of Pelco. If the reader of this message is not the
intended recipient, or an employee or agent responsible for delivering this
message to the intended recipient, you are hereby notified that any review,
disclosure, copying, distribution, retention, or any action taken or omitted to
be taken in reliance on it is prohibited and may be unlawful. If you receive
this communication in error, please notify us immediately by telephone call to
+1-559-292-1981 or forward the e-mail to administrator@xxxxxxxxx and then
permanently delete the e-mail and destroy all soft and hard copies of the
message and any attachments. Thank you for your cooperation.
- ------------------------------------------------------------------------------
|
