|
On Thu, Mar 25, 2004 at 11:45:22AM +0200, nick garratt wrote:
> Hi
>
> I'm experiencing issues with port translation using LVS-NAT and FWMARK:
>
> iptables -t mangle -A PREROUTING -d VIP -p tcp -m tcp --syn --dport
> 1237:1239 -j MARK --set-mark 1238
>
> ipvsadm -A -f 1238 -s wlc -p 900
> ipvsadm -a -f 1238 -r 192.168.20.1:1237 -m -w 5 # daemon instance 1
> ipvsadm -a -f 1238 -r 192.168.20.1:1238 -m -w 5 # daemon instance 2
>
>
> What I am trying to achieve is the following:
>
> we have a custom written SMPP service that accepts two connection
> (transmitter and receiver) from a client. We have run into problems
> with maximum threads per process and large numbers of binds. As an
> interim measure we are considering running multiple instances of the
> daemon on the same server. Its is imperative that a user's two binds
> are routed to the same daemon instance. The user may connect to a
> port range so as to allow them to specify different receiver and
> transmitter ports according to their whim or the peculiarities of
> their client software but the daemon instance will handle both
> connections on the same port.
>
> The intention is to group the VIP port range using FWMARK as we do
> with many other services and load balance them across the RIP service
> ports ensuring that:
>
> userIP:56789 -> VIP:1237 -> RIP:n
> userIP:56790 -> VIP:1238 -> RIP:n
>
> where n is the same port guaranteed by persistence.
>
>
> Problem: FWMARK and LVS-NAT port translation does not seem to work at
> all. what actually happens is:
>
> userIP:56789 -> VIP:1237 -> RIP:1237
> userIP:56790 -> VIP:1238 -> RIP:1238
>
> which splits the binds across daemon instances.
Yes, port translation does not work with fwmarks, because there is no
way for LVS to tell what the port translation should be. In a fwmark
service the virtual service does not have a port (or address for that
matter). So it can't know that it is accepting packets for, say port
1237, and then use the real server entry to translate that to port 1237
(not much of a translaton) or 1238 (or anything else). It has to just
assume that the port will be unchanged.
It would be possible to modify LVS to allow this kind of translation
to take place, but it isn't immediately obviously how this would
be configured.
> Another approach to the problem is to configure multiple virtual
> interfaces on my real server, get the daemon instances to bind to
> specific IPs/same port ranges and handle as per normal i.e. no port
> translation:
>
> iptables -t mangle -A PREROUTING -d VIP -p tcp -m tcp --syn --dport
> 1237:1239 -j MARK --set-mark 1238
>
> ipvsadm -A -f 1238 -s wlc -p 900
> ipvsadm -a -f 1238 -r 192.168.20.11:0 -m -w 5 # daemon instance 1
> listening on 1237 - 1239
> ipvsadm -a -f 1238 -r 192.168.20.12:0 -m -w 5 # daemon instance 2
> listening on 1237 - 1239
>
> However I would prefer to keep down the number of IPs I need to failover.
I would suggest doing this. You shouldn't need to failover
the IP addresses of your real servers anyway. Just use something like
ldirectord to monitor their availability and manipulate the LVS
table accordingly.
--
Horms
|
