|
hi people
I made some modifications on the lvs specific kernel code, which now
leads into kernel oops. Could someone give me some pointers about how to
find the bug? I am not very familiar with the kernel code, so maybe i
missed some simple tricks which routined people know and me not.
Basically i altered the lvs code in order to make it catch packets
within the PREROUTING chain instead of the INPUT chain. My setup works,
but sometimes i have a kernel oops.
I think somewhere it lacks some sort of spinlock, but i not really know
where to begin in order to find where it must be inserted.
------------------
More detailed description:
My setup:
Kernel is RHEL 2.6.55.0.2.EL.
I have 2 LVS directors (master, backup), which at the same time are real
servers running squid. They are configured as LVS-GW, the real servers
have ip addresses on a different subnet, than the VIP. The backup has
the correct route for both subnets and a default gateway pointing to the
master.
I use keepalived which configures LVS in order to have the correct rules
configured on the master and have no rules on the backup whenever the
master is up.
Connections to port 80 from behind the master going to "the outside"
should be transparently intercepted, balanced by lvs and passed to the
respective squid, which does the rest.
With the standard LVS this setup is not possible, because of 2 causes:
1). I must mark packets within the PREROUTING chain in the mangle table
in order to pass them to LVS, but LVS intercepts only packets coming in
into the INPUT chain, but which forwarded packets will never pass.
2). When I managed it to intercept the packets with LVS, both
realservers needs to DNAT the packets in order to redirect them to
squid, which runs on port 8080.
But packets which come in on Local cannot be NAT'ed because LVS sends
them directly to the wire.
I solved those problems this way:
1.)
--- linux-2.6.9/net/ipv4/ipvs/ip_vs_core.c.orig 2007-07-30
20:40:31.000000000 +0200
+++ linux-2.6.9/net/ipv4/ipvs/ip_vs_core.c 2007-07-30
20:40:37.000000000 +0200
@@ -1095,7 +1095,7 @@
.hook = ip_vs_in,
.owner = THIS_MODULE,
.pf = PF_INET,
- .hooknum = NF_IP_LOCAL_IN,
+ .hooknum = NF_IP_PRE_ROUTING,
.priority = 100,
};
2.) (patch attached)
the patch causes incoming packets which should go to Local to retransmit
through the netfilter hooks starting on NF_IP_PRI_MANGLE, instead of
transmit them directly with ip_vs_null_xmit.
This way i can remove the mark within the mangle table in order to pass
it through LVS twice and then simply DNAT it.
(please ask if you like to have the detailed iptables/ipvsadm rules.)
The setup works.
But sometimes i have a kernel oops (Fatal exception in interrupt,
ip_rcv, ip_rcv_finish is involved).
I tried to narrow down the problem, by removing patch nr 2, but the
problem still exists.
So the problem must be with the 1st patch. But what could cause this. I
simply let LVS catch packets within PREROUTING chain instead INPUT
chain. That seems not too different to me.
I think somewhere it lacks some sort of spinlock, but i not really know
where to begin in order to find where it must be inserted.
Would be very happy if someone could point me in the right direction.
peter
--
:: e n d i a n
:: open source - open minds
:: peter warasin
:: http://www.endian.com :: peter at endian.com
|
