Re: Linux Director Reliability

[Peter Nash <peter.nash@xxxxxxxxxxxxxxxxx>]

Hi Horms,

> If anyone is able to volunteer this information either
> on the list or to me privately then I would be most grateful.
> Please include some rough information on your system:
> What architecture, which kernel, which version of LVS,
> approximately how many nodes, and how busy it is.

We've been using LVS-NAT for just over 1 year.  The
configuration consists of two directors with three real servers and ten
VIPs. It's a relatively lightly loaded site serving HTTP with and without
SSL for a mixture of micro-sites using wlc and interactive database applications
using wlc with persistence.  The real servers run IBM Lotus Domino
with database clustering and we use persistence to overcome any database
replication latency.  The system was implemented primarily for resilience
and the ability to carry out scheduled updates and maintenance without
service interruption and it has been extremely successful in meeting those
goals.

The directors are single cpu Celeron 800Mhz with 128Mb
RAM running RedHat 8.0 with IPVS 1.0.9 and ipvsadm 1.21 (identical machines).
 We use Heartbeat and Ldirectord (currently using the Ultramonkey
2.0.1 packages) for director and real server failover.  We tend to
stay up to date with RedHat patches so the kernels are currently 2.4.20-20.8.
 We now have to re-compile with the IPVS patch since RedHat stopped
including it in their standard distro a while back.  We have also
used kernel packages from the UltraMonkey site with IPVS already patched.

The real servers are dual CPU P4 Xeon with 1Gb RAM
running RedHat 8.0 (were running 7.3 until one month ago).

Connection rates peak at about 10 Active and 70 Inactive
for each VIP, i.e. about 100 Active and 700 Inactive peak overall although
the different VIPs have varying usage profiles throughout the day and don't
all peak at the same time.

We've had only one LVS problem, in January 2003, when
the active director just stopped serving anything on the VIPs.  Heartbeat
was still running so the director didn't failover of course.  It was
in working hours so we were on the case in a couple of minutes as soon
as our monitoring system sounded the alert but as we were unable to see
any obvious problem we shut down Heartbeat to force a failover to the backup
director (which was fine).  That cleared the LVS table on the director
of course so after that we had no way to try and investigate the problem
and I can't offer any insight as to what went wrong.  After a reboot
all was well and we've had no problem since.

Overall I am really very impressed with the system
and it absolutely meets our goals for high availability and easy of maintenance.

Regards,

Peter.