|
This is a resend, only now I'm a list member, so you'll get it. :-)
-----Original Message-----
From: Jake Garver [mailto:garver@xxxxxxxxxxxx]
Sent: Tuesday, August 07, 2001 4:18 PM
To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: Problems with LVS-DR/FWMARK and director as gateway
Well, I've beat me head against this for a while and I'm finally going to
stop and ask for directions. Thanks in advance for any advise you have.
Summary: I'm trying to set up LVS-DR using fwmarks, but the catch is that my
director is also the real servers' default gateway. When I try to access
the virtual service, my traffic gets marked, but is then routed out the
director's default gateway. LVS doesn't seem to be grabbing the packet and
sending it off to a real server.
Details: I'm running LVS-0.8.1 on linux-2.4.6.
We have an existing firewall/router between our servers, internal clients,
and the internet. I'm trying to add LVS to this box and start load
balancing over our existing servers. I chose LVS-DR instead of LVS-NAT just
for efficiency and because I wasn't affected by any of DR's downsides.
Originally, I tried DR with the VIP on the director's lo:0, but ran into the
martian problem. I then read about fwmark. As I understand it, fwmark
promises to work around the martion problem and provide greater flexibility
when grouping services. So I configured LVS for fwmark and got stuck. A
pretty ASCII diagram of my architecture for you:
Internet
|
|
------------
- Firewall -
- Router - ------ Clients
- Director -
------------
|
|
-------
- RS1 -
-------
To test, I set up a single real server (RS1), running POP on port 110. When
I telnet from the client to the VIP:110, I get "no route to host". After
some inspection, I found that iptables was marking my packets, but they were
being routed to the internet. Since the VIP is in my IP space, the first
hop on the internet routed the packets back to me. So, my router played
ping pong with the packet 30 times until it gave up and reported no route to
host.
Here's my iptables config:
iptables -t mangle -N lvs
iptables -t mangle -A PREROUTING -i ${ETH_CLIENT} -j lvs
iptables -t mangle -A PREROUTING -i ${ETH_INTERNET} -j lvs
iptables -t mangle -A lvs -p tcp -d 216.28.214.225/32 --dport 110 -j
MARK --set-mark 1
Here's my lvsadm config:
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 1 rr
-> 10.2.0.3:0 Route 1 0 0
10.2.0.3 is the IP address of RS1.
216.28.214.225 is the VIP and is configured on lo:0 of the RS1.
Here's an abridged version of the route table on the
router/firewall/director:
10.2.0.1 * 255.255.255.255 UH 0 0 0 eth3
216.28.208.1 * 255.255.255.255 UH 0 0 0 eth0
216.28.208.0 * 255.255.255.192 U 0 0 0 eth0
10.2.0.0 * 255.255.0.0 U 0 0 0 eth3
127.0.0.0 * 255.0.0.0 U 0 0 0 lo
default 216.28.214.253 0.0.0.0 UG 0 0 0 eth1
eth1 is connnected to the internet, eth3 to the real server network, and
eth0 to the clients.
I have been digging through the FAQ and code and I'm left even more confused
by two "facts":
1. fwmark does not require the VIP to be configured on an interface on the
director (but its still on the real servers).
2. LVS hooks into netfilter at LOCAL_IN.
These two facts seem to contradict. If LVS is to see the packet, it must be
routed to the local machine. How can it be routed to the local machine if
the local machine doesn't contain the VIP?
Where am I mistaken?
Thanks in advance for your help,
Jake Garver
|
