LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Port translation and FWMARK

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Port translation and FWMARK
From: nick garratt <nick-lvs@xxxxxxxxxxxxxx>
Date: Thu, 25 Mar 2004 11:45:22 +0200
Hi

I'm experiencing issues with port translation using LVS-NAT and FWMARK:

iptables -t mangle -A PREROUTING -d VIP -p tcp -m tcp --syn --dport 1237:1239 -j MARK --set-mark 1238

ipvsadm -A -f 1238 -s wlc -p 900
ipvsadm -a -f 1238 -r 192.168.20.1:1237 -m -w 5 # daemon instance 1
ipvsadm -a -f 1238 -r 192.168.20.1:1238 -m -w 5 # daemon instance 2


What I am trying to achieve is the following:

we have a custom written SMPP service that accepts two connection (transmitter and receiver) from a client. We have run into problems with maximum threads per process and large numbers of binds. As an interim measure we are considering running multiple instances of the daemon on the same server. Its is imperative that a user's two binds are routed to the same daemon instance. The user may connect to a port range so as to allow them to specify different receiver and transmitter ports according to their whim or the peculiarities of their client software but the daemon instance will handle both connections on the same port.

The intention is to group the VIP port range using FWMARK as we do with many other services and load balance them across the RIP service ports ensuring that:

userIP:56789 -> VIP:1237 -> RIP:n
userIP:56790 -> VIP:1238 -> RIP:n

where n is the same port guaranteed by persistence.


Problem: FWMARK and LVS-NAT port translation does not seem to work at all. what actually happens is:

userIP:56789 -> VIP:1237 -> RIP:1237
userIP:56790 -> VIP:1238 -> RIP:1238

which splits the binds across daemon instances.


Another approach to the problem is to configure multiple virtual interfaces on my real server, get the daemon instances to bind to specific IPs/same port ranges and handle as per normal i.e. no port translation:

iptables -t mangle -A PREROUTING -d VIP -p tcp -m tcp --syn --dport 1237:1239 -j MARK --set-mark 1238

ipvsadm -A -f 1238 -s wlc -p 900
ipvsadm -a -f 1238 -r 192.168.20.11:0 -m -w 5 # daemon instance 1 listening on 1237 - 1239 ipvsadm -a -f 1238 -r 192.168.20.12:0 -m -w 5 # daemon instance 2 listening on 1237 - 1239

However I would prefer to keep down the number of IPs I need to failover.


Any suggestions ?

Nick
<Prev in Thread] Current Thread [Next in Thread>