LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: ip_vs_random_dropentry

To: Jacob Coby <jcoby@xxxxxxxxxxxxxxx>
Subject: Re: ip_vs_random_dropentry
Cc: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
From: Julian Anastasov <ja@xxxxxx>
Date: Fri, 30 Sep 2005 08:54:28 +0300 (EEST)
        Hello,

On Thu, 29 Sep 2005, Jacob Coby wrote:

> >     In old days we were sending ICMP error, now it is disabled with
> > nat_icmp_send sysctl. Is it enabled in your setup?
>
> No, it is set to 0.  I'm using DR; would this flag still have effect?

        No, only for NAT. It is difficult to do the right thing,
director has no information from client which SYN is valid, from
real server which SYN is accepted and replied with ACK (NAT can see
SYN+ACK). One option is to limit the SYN+ACK retries if TCP in real
server has such control, Linux has: tcp_synack_retries, etc.

        Another option is someone with spare time to finish the
work for per-state timeout control that is needed in ipvsadm as
the kernel has some unused code to define timeout for every state.
Then you can change SYN timeouts without recompiling kernel.

        Another option is to define short timeout for SYN-only
connections to drop them HZ/x time if the connections don't enter
established state. May be a change needed near this todrop_entry()
usage.

        There are ideas but noone with enough time to implement them.

> I only raise this issue because I've been having trouble with incomplete
> connections due to buggy or overloaded NAT firewalls (or some other
> factor that I can't trace).  I'll see normal traffic and then a flood of
> 45+ SYN packets within 1.5 seconds.  Or it'll generate a flood of EST
> (SYN / SYN+ACK / ACK => EST) connections without sending data.  I'm
> thinking it's a bug (feature?) in the firewall when it gets overloaded
> in that it "forgets" to send RESET or FIN packets.  Or it misunderstands
> the SYN+ACK retries from the realserver, or ...  I'm out of ideas, and
> just want to stop it from bringing down our site.
>
> I'll be adding --syn limits to the iptables rules on the director next
> week, but it still seems weird that LVS will drop a connection on the
> director without letting the real server know it's dead.  It's very
> important, esp. when dealing with apache 1.3 or other server daemons
> that fork and can take up lots of memory (and need to close connections
> ASAP).

        Agreed, may be todrop_entry() needs to work for SYN packets
too.

Regards

--
Julian Anastasov <ja@xxxxxx>

<Prev in Thread] Current Thread [Next in Thread>