LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] "connection refused" when persistence enabled

To: Julian Anastasov <ja@xxxxxx>, lvs-users@xxxxxxxxxxxxxxxxxxxxxx, Deon van der Merwe <dvdm@xxxxxxxxxxxx>
Subject: Re: [lvs-users] "connection refused" when persistence enabled
From: Simon Horman <horms@xxxxxxxxxxxx>
Date: Tue, 8 Dec 2009 12:15:39 +1100
On Mon, Dec 07, 2009 at 07:45:22PM +1100, Simon Horman wrote:
> On Mon, Dec 07, 2009 at 10:37:32AM +0200, Julian Anastasov wrote:
> > 
> >     Hello,
> > 
> > On Mon, 30 Nov 2009, Simon Horman wrote:
> > 
> > > >         looking at
> > > > 
> > > > kernel: IPVS: template lookup/in IP 160.124.109.65:0->0.0.0.0:0 not hit
> > > > 
> > > >         daddr=0.0.0.0 shows that fwmark is not provided at the
> > > > right place, so the user just needs to upgrade or to apply
> > > > the patch from May, 2009:
> > > > 
> > > > http://marc.info/?t=124050022700001&r=1&w=2
> > > 
> > > Hi Julian,
> > > 
> > > I've been able to reproduce the problem that Deon reported
> > > with 2.6.30, which includes the change above, so I think
> > > that it is a separate problem.
> > 
> >     It is still not clear to me whether Deon needed just 2.6.30
> > or your latest change was needed. I don't see how ipvsadm 1.25
> > can provide virtual port without -t/-u options. What is your
> > ipvsadm command line that reproduces this svc port problem in
> > 2.6.30? May be you have both -f and -t/-u ?

Hi Julian,

here are the results of some tests that I ran today.
They seem to indicate that this problem is real.

2.6.32
======

Test 1
------

# iptables -t mangle -F
# iptables -t mangle -A PREROUTING -p tcp --dport 10000 -j MARK --set-mark 1
# ipvsadm -C
# ipvsadm -A -f 1
# ipvsadm -a -f 1 -r localhost:10000

$ telnet 172.17.60.197 1000
Trying 172.17.60.197...
Ok

Test 2
------

# iptables -t mangle -F
# iptables -t mangle -A PREROUTING -p tcp --dport 10000 -j MARK --set-mark 1
# ipvsadm -C
# ipvsadm -A -f 1 -p 600
# ipvsadm -a -f 1 -r localhost:10000

telnet 172.17.60.197 10000
Trying 172.17.60.197...
telnet: Unable to connect to remote host: Connection refused

2.6.32 + the following debug patch
==================================

--- linux-2.6.orig/net/netfilter/ipvs/ip_vs_core.c      2009-12-08 
09:57:41.000000000 +0900
+++ linux-2.6/net/netfilter/ipvs/ip_vs_core.c   2009-12-08 10:02:35.000000000 
+0900
@@ -295,8 +295,11 @@ ip_vs_sched_persist(struct ip_vs_service
                         * If it is not persistent port zero, return NULL,
                         * otherwise create a connection template.
                         */
-                       if (svc->port)
+                       if (svc->port) {
+                               IP_VS_DBG(6, "p-schedule: non-zero svc port: "
+                                         "0x%x\n", svc->port);
                                return NULL;
+                       }
 
                        dest = svc->scheduler->schedule(svc, skb);
                        if (dest == NULL) {

Test 1
------

# iptables -t mangle -F
# iptables -t mangle -A PREROUTING -p tcp --dport 10000 -j MARK --set-mark 1
# ipvsadm -C
# ipvsadm -A -f 1
# ipvsadm -a -f 1 -r localhost:10000

$ telnet 172.17.60.197 1000
Trying 172.17.60.197...
Ok

Test 2
------

# iptables -t mangle -F
# iptables -t mangle -A PREROUTING -p tcp --dport 10000 -j MARK --set-mark 1
# ipvsadm -C
# ipvsadm -A -f 1 -p 600
# ipvsadm -a -f 1 -r localhost:10000

telnet 172.17.60.197 10000
Trying 172.17.60.197...
telnet: Unable to connect to remote host: Connection refused

In dmsg:
[ 1134.971085] IPVS: p-schedule: src 172.17.60.192:37337 dest 
172.17.60.197:10000 mnet 172.17.60.192
[ 1134.971089] IPVS: p-schedule: non-zero svc port: 0x8801


2.6.32 + my proposed fix
========================

Test 1
------

# iptables -t mangle -F
# iptables -t mangle -A PREROUTING -p tcp --dport 10000 -j MARK --set-mark 1
# ipvsadm -C
# ipvsadm -A -f 1
# ipvsadm -a -f 1 -r localhost:10000

$ telnet 172.17.60.197 1000
Trying 172.17.60.197...
Ok

Test 2
------

# iptables -t mangle -F
# iptables -t mangle -A PREROUTING -p tcp --dport 10000 -j MARK --set-mark 1
# ipvsadm -C
# ipvsadm -A -f 1 -p 600
# ipvsadm -a -f 1 -r localhost:10000

telnet 172.17.60.197 10000
Trying 172.17.60.197...
Ok


_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>