Re: [lvs-users] firewall marks + tunneling + persistence = ERR! state

To: " users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] firewall marks + tunneling + persistence = ERR! state
Cc: =?x-unknown?q?Jaroslav_Lib=E1k?= <jarol1@xxxxxxxxx>
From: Con Tassios <ct@xxxxxxxxxxx>
Date: Thu, 29 Nov 2007 22:42:22 +1100 (EST)
On Wed, 29 Nov 2006, Horms wrote:

> On Tue, Nov 28, 2006 at 09:32:09PM +0100, Jaroslav Libák wrote:
> > When i run that I get some connections with ERR! state. Persistence is
> > 600 = 10 minutes, after that these connections dissappear. Without
> > persistence there are no such connections. If I don't use firewall
> > marks then they aren't there either. If I don't use firewall marks,
> > then there are "NONE" connections which from what I have read LVS uses
> > to handle persistence. These "connections" resemble my ERR!
> > connections in this sence. After they dissappear client can be routed
> > to a different real server.
> >
> > Could anyone confirm that in this case this ERR! state is harmless?
> > I'm thinking that it might be happening because usage of firewall
> > marks was added to LVS later and ipvsadm wasn't updated to handle this
> > properly. Or when using firewall marks and persistence, somebody
> > forgot to change the state of the connection to "NONE" in the C code.
> That does sound a little odd.
> I don't think that is is to do with ipvsadm, as I think that the
> strings come from the kernel. Can you see if the same problem shows
> up when you cat /proc/net/ip_vs_conn ?
> Once these connections get into that state, do they stay in that state
> until they timeout, or do they progress to a different state?
> Could you send some examples of this behaviour?
> I suspect that it is harmless, but I also think it is
> a bug in the the reporting functionality.

I'm seeing similar behaviour as reported in this thread a year ago.  Although
this is using the RHELv5 kernel (2.6.18-53.el5), the only patches being applied
by Red Hat to ipvs code is a very minor one to ip_vs_sync.c.  Persistency
appears to be working correctly as far as I can tell.

# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
FWM  1 rr persistent 600
  ->              Route   5      1          0
  ->              Route   5      0          0

(Packets with destination address port 8000 marked with fwm 1)

# ipvsadm -Lcn
IPVS connection entries
pro expire state       source             virtual            destination
IP  09:56  ERR!

# cat /proc/net/ip_vs_conn
Pro FromIP   FPrt ToIP     TPrt DestIP   DPrt State       Expires
IP  88BA0132 0000 00000001 0000 88BA01B9 0000 ERR!            595
TCP 88BA0132 8BB5 88BA011D 1F40 88BA01B9 1F40 ESTABLISHED     895

<Prev in Thread] Current Thread [Next in Thread>
  • Re: [lvs-users] firewall marks + tunneling + persistence = ERR! state, Con Tassios <=