LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Persistance and LVS

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Persistance and LVS
From: Horms <horms@xxxxxxxxxxxx>
Date: Fri, 7 May 2004 11:13:44 +0900
On Thu, May 06, 2004 at 10:39:50AM -0400, Joseph Mack wrote:
> Horms wrote:
> > 
> 
> > 
> > LVS-NAT in 2.4 (and I assume 2.6) does not modify the source port
> > so there is no possibility of collision.
> 
> OK. For regular NAT (not LVS-NAT) and your NAT router has one ethernet
> connection to the outside world. Let's say we have clients connecting to
> the outside world from both the NAT box and from boxes on the NAT'ed 
> network inside. Isn't it possible that a client on the NAT box and a
> client on one of the NAT'ed boxes will call from the same port?

Sure, it is possible that the port will be the same. But the
source IP address will not. NAT will change the source IP address
to that of the NAT interface (VIP or DIP in LVS termonology), 
whereas LVS-NAT leaves it as the CIP. 

> > A persistance template is just like a connection entry.
> > It uses the same data structure. And is stored in
> > the same hash table. The only difference is that the source
> > port is set to 0 so that it can be identified as a persistance
> > template. 
> 
> Let's say I make VIP:https persistent. There will be an entry in
> the hash table for VIP:https and another entry for VIP:0 in the
> persistence template. 

No. When a connection fro an enduser with CIP1 comes in then a
persistance template will be created for CIP:0. A Connection entry will
also be made for CIP:ephemeral_source_port.

> How does ipvsadm know to make only VIP:https persistent?

I am not sure that I understand what you are getting at here.
When you configure a virtual service using ipvsadm you can mark 
it as persistant. This sets a flag in the kernel for the virtual service
that is checked by the LVS scheduler so it knows whether to treat
the connection as persistant or not.

> > This means that it will never match a hash-table lookup
> > for a connection entry. And a connection entry will never
> > match a lookup for a persistance template which is made in
> > the scheduling code.
> 
> I don't know enough about what's going on here to know whether this
> would or would not be true. Can you explain more?

All the connection entries and persistance templates are stored in a
hash table. To retrieve an entry from the hash table, first the hash key
is generated (how is not particularly relevant here) and then that
bucket is searched for the matching entry. A match checks various values
including the source port.  As the following property is true, a
persistance template can never be confused with a connection entry:

   Persistance Template: Source Port = 0
   Connection Entry:     0 > Source Port > 2^16-1

In other words. If you are looking for a persistance template then
your search will always be for something with Source Port = 0.
But if you are looking for a connection entry you will always
be looking for something with Source Port != 0.

Thus there is no ambiguity, despite both types of entries using
the same data structure and being stored in the same hash table.

> > The purpose of a persistance template is, in a nutshell, to
> > effect persistance. When a connection is started for a persistant
> > virtual service, the persistance template is looked up. If it
> > exists then it is used - that is the connection will be forwarded
> > to the same real server as the previous corresponding connection.
> > Otherwise the connection is scheduled, just like a connection for
> > a non-persistant virtual service, and the persistance template is
> > created.
> 
> I presume this has something to do with my question about how ipvsadm
> makes only VIP:https persistent.

I presume so too :-)

Does this help:

   New Connection Comes In
   Look up Virtual Service
   Is Virtual Service Persistant?
      No -> Call Scheduler to allocate a Real Server for the Connection
            Use Result from Scheduler to create Connection Entry
      Yes -> Persistance Template Exists
             No -> Call Scheduler to allocate a Real Server for the Connection
                   Use Result from Scheduler to create Persistance Template.
             Yes -> Nothing Special to Do
             Use Persistance Template to create Connection Entry
   Forward Packet using Connection Entry

> > Like connection entries, persistance templates have timeouts.
> > Actually, again, it is handled by the same code. The only difference
> > is that for persistance templates, the timeout is set by the
> > persistance timeout configured using ipvsadm. Whereas for connection
> > entries the timeout depends on the connection's state.
> 
> my knowledge of persistence templates is sorely lacking.

They aren't really that exciting. Just think of them as special
connection entries. Special entries effect which real server
subsequent connections from a given end user are allocated to.
Rather than effecting which server packets for a current connection
are sent to.

Timeouts just handle how long a connection entry or persistance template
are valid for. Else they would live in the kernel forever.

-- 
Horms
<Prev in Thread] Current Thread [Next in Thread>