LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: IPVS 0.2 crash with SMP kernels (was: ipvs-0.8.0 available)

To: Julian Anastasov <ja@xxxxxx>
Subject: Re: IPVS 0.2 crash with SMP kernels (was: ipvs-0.8.0 available)
Cc: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
From: Radu-Adrian Feurdean <raf@xxxxxxxx>
Date: Wed, 9 May 2001 23:21:43 +0200 (CEST)
On Wed, 9 May 2001, Julian Anastasov wrote:

> 
>       Hello,
> 
> On Wed, 9 May 2001, Radu-Adrian Feurdean wrote:
> 
> > 8 hours, 2 kernel+ipvs upgrades and about 10000 lines of kernel logs after,
> > not so often in the logs (still plenty of):
> >
> > kernel: IPVS: Incoming failed TCP checksum from bla.bla.bla.bla (size=20)!
> 
>       eth model? May be caused from remote attacks? I can see such
> messages in my Linux 2.2 logs too.

Same thing with eepro100 (intel card) and tulip or de4x5 (Quad D-link with DEC
chipset). We had to turn all TCP options off on both director and real
servers, because otherwise each and every packet generated by a linux client
caused a similar message. Cables are good and switches are CISCO on all the
network, so the packets are not trashed on our network (I hope).

> > 2.4.2+ipvs_0.2.6, 2.4.2+ipvs_0.2.7, 2.4.3+ipvs_0.2.8, 2.4.4+ipvs_0.2.11
> > All these combinations (SMP based) crashed in less than 8 hours of high
> > traffic. 2.4.2+0.2.7 resisted over a week-end at low traffic (~2.5 Mbps)
> 
>       We need this crash report! But with the latest versions, please.

We will try to get something at the next director install or when new "SMP
tests" are approved. Now that machine is running fine with UP kernel. The
previous crashes brought the machine to a total lock, so nothing arrived to
syslog. And the admins that were on duty in that period (one week-end,
one night and one legal holiday) didn't have the inspiration to log the
messeages from the serial console.

Unfortunately, IPVS worked great while testing, and all problems appeared only
in production, and unfortunately not during work hours.

Tomorrow we'll start loadbalancing FTP. We'll prepare to get crash logs in
case anything wrong would happen.
 
> > > > And yes, there is netfilter (aka firewall + MASQUERADE) on the same box.
> > > > (here are some other minor problems, like packets - all of them - that 
> > > > don't
> > > > pass through chain OUTPUT, table mangle).
> > >
> > >   The LVS packets? Is this expected behavior? What is shown
> > > in the Netfilter docs? Do you use the Netfilter's IPFW compat code?
> >
> > Both LVS and MASQUERADE packets. It is not the expected behavior and I 
> > haven't
> > found something related in netfilter docs (well, I didn't search very much).
> 
>       Why do you expect the LVS to use the OUTPUT chain? You mention
> the mangle table? Locally generated packets?

Oops I missed that in docs. I need the mangle table in order to mark packets
for CBQ shaping. Unfortunately I see that I can't mark packets based on the
virtual service IP, since we use NAT. So I have to use other less flexible
tricks to do that.


>       It is mentioned in the packet-filter howto. The 2.2 ipchains
> behavior (the netfilter's ipfw compat mode too) is always to use the OUTPUT
> chain when forwarding packets. In the new netfilter framework the OUTPUT
> chain is traversed only from locally generated packets which is not the
> case with LVS. Although LVS differs from the new netfilter's behavior,
> the OUTPUT chain is not traversed. LVS differs in the way the packets
> are forwarded. LVS accepts the out->in traffic in the LOCAL_IN hook
> (always after the INPUT chain) while netfilter forwards it through
> the FORWARD chain. So, you have to revisit your OUTPUT chain usage
> or to drop the iptables and to fallback to the ipchains binary with
> the netfilter's ipfw compat code. But I don't know why you use the
> OUTPUT chain.

So that WAS the correct behavior (well, not the expected one).
 
> > I used only iptables (ipchains and ipfwadm compat isn't even compiled, not
> > even as module)
> 
>         It seems you don't need them anymore :)

Because we use iptables... :) And because ipchains begins to become
"deprecated" (after some opinions).

Radu-Adrian Feurdean
mailto: raf@xxxxxxxx
-------------------------------------------------------------------
"If the night is silent enough you can hear a Windows NT rebooting"



<Prev in Thread] Current Thread [Next in Thread>