Re: Persistance and LVS

To: " users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Persistance and LVS
From: Joseph Mack <mack.joseph@xxxxxxx>
Date: Thu, 29 Apr 2004 14:22:04 -0400
pb wrote:
> Joe,

hmm. I've made a bit of a blunder here. What I said
earlier is true but doesn't apply here.

<cough> Version 2

A (real)server for any port (say port 80), can accept connections
from 32bits of IPs and 16 bits of ports = 2^48 = 10^16 connections.
ie any realserver can simultaneously accept connections from
all the ports on all the computers in IP space (provided it
has enough memory to store all the connection info)


What I said earlier today applies in the case of a single
client connecting to a single port on a (real)server.
The client can only make 64k connections at any one time.

We don't care about the client. We only care about the 
server. If there are clients spread over enough of IP
space, then the server can have a very large number of

I've made this same blunder before and Julian came up 
and straightened me out. I thought I'd put it into the
HOWTO and remembering that ports were a hazardous area
for me, I had a quick look to see if there was anything
on port limitations, but I didn't see anything to worry
about and so posted my earlier statements, which are wrong. 
Sorry about this.

I have run out of ports before on test clients when running
polygraph, so it can be a problem at least in torture testing.

> Thanks for all this info, however, we handle 1 million
> messages a day, and 20,000+ webmail users, thus
> 125,000 messages per hour send/recv in 8 hour work
> day.  

for 4 realservers that's 40k messages/hr. I don't know
how many tcp connections are required for a message transfer,
but let's say it's 1. You have 15mins persistence,
so 10k connections will be in existance at any one time.

For memory for the ipvs hash table:
At 128bytes/connection, that's 1.28M of memory for the
ipvsadm hash table. You have quite a margin with memory.

For disk and network I/O:

Let's say the average e-mail is 10kB. Each realserver
is processing (10k messages/(15*60) secs) * 10kB = 0.1MBytes/sec.
Your disks and network also have large margins of safety.

> The
> error just a couple random people are having with
> WEBMAIL is "invalid session ID" as though they lost
> their connection to the real server (actually a
> "message director") they were on.  But I don't know if
> it is the "message directors" fault, or LVS. 

have no idea, but I don't see any heavy load here. 

Are the clients timing out after 15mins and attempting to 
continue their session? WOuldn't the app/client know 
that the session has been closed and to go through
the whole login procedure again? I don't know much
about your app I'm sorry.

> We have
> 4 of these "message directors" which are real servers
> behind the LVS.

> So is it your suggestion that I have no need to
> increase beyond 15 minutes the persistance for HTTP
> and HTTPS services ? 

nothing here addresses the issue of persistence timeout.
This is determined by how long you allow the client
to be disconnected before you propagate to all realservers,
the state changes in the realserver that occured in the last 


Joseph Mack PhD, High Performance Computing & Scientific Visualization
SAIC, Supporting the EPA Research Triangle Park, NC 919-541-0007
Federal Contact - John B. Smith 919-541-1087 - smith.johnb@xxxxxxx
<Prev in Thread] Current Thread [Next in Thread>