|
Hi
We are trying to use LVS as a virtual load balancer around a transparent
http proxy, and are wondering if it is possible to use persistence and
destination hashing together
(from our tests and as suggested in how-tos, persistence is CIP based, and
therefore may not be so).
To explain our setup a little
- We have a pool of VMs running a transparent http proxy
- We have setup an LVS service on either side of this proxy with
- A 'request' path service that will schedule requests
into the proxy, using src based hashing
- A 'response' path service that will schedule the http
responses through the proxy, using dst based hashing.
- The http proxy maintains state around the 'session' (set of urls
requested by client ip), so we wish to direct clients to the same proxy
instance.
Src hashing allows to achieve this, and dst hashing on the response
path ensure TCP connections get established correctly.
An issue arises when we try to add new instances of the proxy to the pool.
The hashing changes, which breaks the statefulness (user request may go
to different server).
To that extent, we added persistence, which worked for requests.
However, persistence on the response path is sending , perhaps as
expected, TCP packets to the 'wrong' proxy instance in a lot of cases.
This is because the persistence logic is using the web server IP address
(the CIP on the response path).
An example of the problem for us :
Using 2 client ips 172.29.0.12, 172.29.0.11; real servers (http proxies)
192.168.90.58, 192.168.90.59; web server 192.168.10.17.
IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler
Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn FWM 1 sh
persistent 60
-> 192.168.90.58:0 Route 1 0 0
-> 192.168.90.59:0 Route 1 0 0 FWM 3 dh
persistent 60
-> 192.168.90.58:0 Route 1 0 0
-> 192.168.90.59:0 Route 1 0 0
FWM1 represents the request path; FMW 2 the response path.
IPVS connection entries
pro expire state
source virtual
destination pe name pe_data
A) TCP 00:58 ESTABLISHED 172.29.0.12:45659
192.168.10.17:80 192.168.90.58:80
B) IP 00:58 NONE 192.168.10.17:0
0.0.0.3:0 192.168.90.59:0
C) TCP 01:55 FIN_WAIT 172.29.0.11:50919
192.168.10.17:80 192.168.90.59:80
D) IP 00:55 NONE 172.29.0.11:0
0.0.0.1:0 192.168.90.59:0
E) TCP 00:59 SYN_RECV 192.168.10.17:80
172.29.0.12:14038 192.168.90.59:14038
F) TCP 01:55 FIN_WAIT 192.168.10.17:80
172.29.0.11:42443 192.168.90.59:42443
G) IP 00:58 NONE 172.29.0.12:0
0.0.0.1:0 192.168.90.58:0
In the example above, C and F represent a successful proxied http request, C
the request from client to proxy, F the response from web server to proxy.
The request C/F was made first and establishes a persistent connections from
client 172.29.0.11 ->192.168.90.59 proxy and for web server 192.168.10.17 to
proxy 192.168.90.59.
All well.
We subsequently make a request A) from client 172.29.0.11, src hashing places
this correctly on proxy 192.168.90.59.
The proxy then requests to web server, and the response is shown in E) -
persistence directs the response to proxy 192.168.90.58, meaning a TCP
connection 192.168.90.59 <-> 192.168.10.17 cannot be established.
Obviously re-engineering of the proxy instances to share state would be the
ideal solution, as then persistence would not be required.
But we are wondering if there is any means to combine destination hashing
and persistence successfully ? As currently persistence is enforcing
scheduling based on src IP, even when dh is specified.
Thanks
David
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
