LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

[lvs-users] Outgoing TCP from real servers using VIP as the source in DR

To: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Subject: [lvs-users] Outgoing TCP from real servers using VIP as the source in DR balancing mode
From: Dmitry Akindinov <dimak@xxxxxxxxxxx>
Date: Fri, 06 Jan 2012 11:15:49 +0400
Hello,

We have met the following problem with "reverse balancing". There is a 
set of servers (let's say 10), and a load-balancer (for which we would 
like to use Linux with the ipvs module). The balancer works in the 
Direct Response (DR) mode, so all packets coming to the "standard" ports 
(such as 25, 80, 110, 143) are redirected to running "real-severs" by 
changing the packet destination MAC, but leaving the IP packet itself 
intact.

Each real server has a lo:x virtual interface with the VIP address 
configured. This interface does not answer any arp request, so all 
packets with VIP as IP destination hit the load balancer first.
Each real server has its own "gray" address 10.10.10.x - used for 
pinging and used to retrieve a MAC to redirect the incoming packets to.

It is the standard DR setup, repeated here just for the clarity.

Now, these servers have to make outgoing TCP connections, too. And the 
application requirement is that these outgoing connections are made 
using the same VIP as the source address.

The main idea is to implement the "Direct Client Request" - something as 
the DR method, but inverted.

a) each server has its own unique port range assigned (16000-16999 for 
the server #0, 17000-17999 for the server #1, etc.), and all outgoing 
connections are made using an available port from that range.

b) TCP response packets will go to the VIP address, to the port that was 
specified as the source port on the initiating server.
These packets will hit the load balancer first, and we need it to relay 
them - WITHOUT modification - to the proper real server, in the same way 
it does when it implements the DR method for incoming connections. It 
must  redirect all packets coming to VIP address, port=16000-16999 to 
the server #0, port 17000-17999 to the server #1, etc.

c) in order to implement this, we tried to play with the iptables TPROXY 
module, but to no avail: the server RIP opens an outgoing TCP connection 
to some server X, port RPORT and we want to redirect all packets coming 
FROM RPORT to our server RIP, w/o modifying the IP packet:

*mangle
:PREROUTING ACCEPT [1969:219531]
:INPUT ACCEPT [1777:151627]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [882:697245]
:POSTROUTING ACCEPT [882:697245]
-A PREROUTING -p tcp -m tcp --sport RPORT -j TPROXY --on-port 0 --on-ip 
RIP --tproxy-mark 0x0/0x0
COMMIT

No packet is being relayed, and it seems like TPROXY works for 
redirecting traffic to an internal socket only.

Is there any way to redirect a packet without modification to a 
different server (substituting the taget MAC),
using just the iptables module? Or is there a way to manage LVS director 
connection tables for this purpose: directing the tcp connection 
response packets to the proper real server?

-- 
Best regards,
Dmitry Akindinov

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>