LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] ipvs, keepalived, squid don't work together

To: horms@xxxxxxxxxxxx, lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Re: [lvs-users] ipvs, keepalived, squid don't work together
From: Igor Velichkevich <igor.velichkevich@xxxxxxxxx>
Date: Tue, 2 Mar 2010 01:20:21 +0200
2010/2/16 Simon Horman <horms@xxxxxxxxxxxx>

>  On Sun, Feb 14, 2010 at 04:27:50PM +0200, Igor Velichkevich wrote:
> > Hello, Lvs-users.
> >
> > For balancing two ISP to local network I use keepalived, ipvs, squid and
> > patch
> http://archive.linuxvirtualserver.org/html/lvs-users/2005-06/msg00102.html
> > I adopted it for kernel 2.6.31 (may be not right).
> > All settings below.
> >
> > Local hosts use 192.168.100.100 as default gateway.
> > But Internet works only via 192.168.100.1.
>
> Hi,
>
> I'm not sure that I understand your setup correctly, but I think
> that what you are trying to do is transparently proxy port 80
> to the local port 3128 which in turn is load balanced.
>
> Is that correct?
>
> If so, I suspect that the problem is that netfilter is actuing
> on the PREROUTING chain but that is too late for LVS, which
> effectively hangs off the INPUT chain.
>
> I have two suggestions:
>
> 1) Use a fwmark virtual service that covers all port 80 connections or;
>
> 2) Move LVS to the PREROUTING chain
>
> http://archive.linuxvirtualserver.org/html/lvs-devel/2008-04/msg00026.html
>
>
 Hello, Simon Horman, Lvs-users.

Tanks for your answer.
You understood my setup correctly.

I applied patch from 2).
But it doesn't help me.
Vrrp works fine.
Balancing doesnt work at all.

I try to use my balancing machine as gateway on two hosts and only one
channel is used.

when I type:

 ipvsadm -L

I see :

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.100.100:3128 rr persistent 600
-> 192.168.100.2:3128 Masq 1 0 0
-> 192.168.100.1:3128 Masq 1 0 0

This is my diff http://paste.ubuntu.com/386614/    and below:

diff /cygdrive/c/Temp/ipvs/ip_vs_core.c ./ipvs/ip_vs_core.c
521,540d520
<
< /*
< * It is hooked before NF_IP_PRI_NAT_SRC at the NF_INET_POST_ROUTING
< * chain, and is used for VS/NAT.
< * It detects packets for VS/NAT connections and sends the packets
< * immediately. This can avoid that iptable_nat mangles the packets
< * for VS/NAT.
< */
< static unsigned int ip_vs_post_routing(unsigned int hooknum,
< struct sk_buff *skb,
< const struct net_device *in,
< const struct net_device *out,
< int (*okfn)(struct sk_buff *))
< {
< if (!skb->ipvs_property)
< return NF_ACCEPT;
< /* The packet was sent from IPVS, exit this chain */
< return NF_STOP;
< }
<
892,907c872
<
< /* For policy routing, packets originating from this
< * machine itself may be routed differently to packets
< * passing through. We want this packet to be routed as
< * if it came from this machine itself. So re-compute
< * the routing information.
< */
< #ifdef CONFIG_IP_VS_IPV6
< if (af == AF_INET6) {
< if (ip6_route_me_harder(skb) != 0)
< goto drop;
< } else
< #endif
< if (ip_route_me_harder(skb, RTN_LOCAL) != 0)
< goto drop;
<
---
>
1265,1273c1230
< /*
< * Big tappo: only PACKET_HOST, including loopback for local client
< * Don't handle local packets on IPv6 for now
< */
< if (unlikely(skb->pkt_type != PACKET_HOST)) {
< IP_VS_DBG_BUF(12, "packet type=%d proto=%d daddr=%s ignored\n",
< skb->pkt_type,
< iph.protocol,
< IP_VS_DBG_ADDR(af, &iph.daddr));
---
> if (skb->ipvs_property)
1275d1231
< }
1410,1412c1366,1368
< .pf = PF_INET,
< .hooknum = NF_INET_LOCAL_IN,
< .priority = 100,
---
> .pf = PF_INET,
> .hooknum = NF_INET_POST_ROUTING,
> .priority = NF_IP_PRI_LAST,
1418,1420c1374,1376
< .pf = PF_INET,
< .hooknum = NF_INET_FORWARD,
< .priority = 100,
---
> .pf = PF_INET,
> .hooknum = NF_INET_PRE_ROUTING,
> .priority = NF_IP_PRI_FIRST + 1,
1421a1378,1385
> /* After packet filtering, change source only for VS/NAT */
> {
> .hook = ip_vs_out,
> .owner = THIS_MODULE,
> .pf = PF_INET,
> .hooknum = NF_INET_LOCAL_OUT,
> .priority = NF_IP_PRI_FIRST + 1,
> },
1427,1429c1391,1393
< .pf = PF_INET,
< .hooknum = NF_INET_FORWARD,
< .priority = 99,
---
> .pf = PF_INET,
> .hooknum = NF_INET_PRE_ROUTING,
> .priority = NF_IP_PRI_FIRST,
1431c1395,1396
< /* Before the netfilter connection tracking, exit from POST_ROUTING */
---
> /* After packet filtering (but before ip_vs_out_icmp), catch icmp
> destined for 0.0.0.0/0, which is for incoming IPVS connections */
1433c1398
< .hook = ip_vs_post_routing,
---
> .hook = ip_vs_forward_icmp,
1435,1437c1400,1402
< .pf = PF_INET,
< .hooknum = NF_INET_POST_ROUTING,
< .priority = NF_IP_PRI_NAT_SRC-1,
---
> .pf = PF_INET,
> .hooknum = NF_INET_LOCAL_OUT,
> .priority = NF_IP_PRI_FIRST,
1467,1474d1431
< /* Before the netfilter connection tracking, exit from POST_ROUTING */
< {
< .hook = ip_vs_post_routing,
< .owner = THIS_MODULE,
< .pf = PF_INET6,
< .hooknum = NF_INET_POST_ROUTING,
< .priority = NF_IP6_PRI_NAT_SRC-1,
< },
diff /cygdrive/c/Temp/ipvs/ip_vs_ctl.c ./ipvs/ip_vs_ctl.c
771c771
< | IP_VS_CONN_F_LOCALNODE;
---
> | IP_VS_CONN_F_MASQ;
773c773
< } else
---
> }
775c775
< if (inet_addr_type(&init_net, udest->addr.ip) == RTN_LOCAL) {
---
> if (inet_addr_type(&init_net, udest->addr.ip) == RTN_LOCAL) {
777,778c777,778
< | IP_VS_CONN_F_LOCALNODE;
< }
---
> | IP_VS_CONN_F_MASQ;
> }
diff /cygdrive/c/Temp/ipvs/ip_vs_xmit.c ./ipvs/ip_vs_xmit.c
198,199c198,199
< NF_HOOK(pf, NF_INET_LOCAL_OUT, (skb), NULL, \
< (rt)->u.dst.dev, dst_output); \
---
> NF_HOOK_THRESH(pf, NF_INET_POST_ROUTING, (skb), NULL, \
> (rt)->u.dst.dev, dst_output, NF_IP_PRI_LAST); \
393,394c393,394
< dst_release(skb->dst);
< skb->dst = &rt->u.dst;
---
> //dst_release(skb->dst);
> //skb->dst = &rt->u.dst;
411c411
< IP_VS_XMIT(PF_INET, skb, rt);
---
> //IP_VS_XMIT(PF_INET, skb, rt);
414c414,415
< return NF_STOLEN;
---
> //return NF_STOLEN;
> return NF_ACCEPT;
470,471c471,472
< dst_release(skb->dst);
< skb->dst = &rt->u.dst;
---
> //dst_release(skb->dst);
> //skb->dst = &rt->u.dst;
487c488
< IP_VS_XMIT(PF_INET6, skb, rt);
---
> //IP_VS_XMIT(PF_INET6, skb, rt);
490c491,492
< return NF_STOLEN;
---
> //return NF_STOLEN;
> return NF_ACCEPT;

-- 
С уважением,
Величкевич Игорь
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
<Prev in Thread] Current Thread [Next in Thread>
  • Re: [lvs-users] ipvs, keepalived, squid don't work together, Igor Velichkevich <=