LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Recommendations for a large implementation

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Recommendations for a large implementation
From: Todd Lyons <tlyons@xxxxxxxxxx>
Date: Thu, 23 Feb 2006 14:21:26 -0800
On Tue, Feb 21, 2006 at 03:44:43PM -0800, Clint Byrum wrote:

>- What kind of hardware requirements would you, the community suggest
>for this solution? We were looking at using HP DL-360 G1's with a single
>PIII-900Mhz CPU, 1GB RAM.

Assuming this is running only LVS, it should be more than adequate.
Data from ours is at the end of this message.  Our system is smaller
than yours, but it should give you a good measuring stick.

>- Ok this is highly subjective, I know.. but how does LVS fare against
>the BigIP-1500. Anybody replaced one with the other?

My understanding (unbacked by experience) is that a BigIP will
loadbalance at line rate, whereas a Linux box will typically find some
happy medium at a point less than line rate.

As for system construction:
inbound
gateway
   |      _____           ---------
   |  |--| lb1 |\    |---| rproxy1 |---| 
   |  |   -----  \   |    ---------    |      ------
    \_|     |     ---|                 |     /       \
      |   _____  /   |    ---------    |----|   web   |
      |--| lb2 |/    |---| rproxy2 |---|    | servers |
          -----      |    ---------         |         |
                     |     Lots of           \       /
                     |    mod_proxy           -------
                     |    rewriting
                     |    ---------
                     |---| rproxy3 |
                     |    ---------
                     |               
                     |    ---------
outbound             |---| webmail |
gateway              |    ---------
   |                 |             
   |                 |    ---------
   |                 |---| imap    |
    \________________|    ---------
                     |             
                     |    ---------
                     |---| smtp    |
                     |    ---------
                     |             
                     |    ---------
                     |---| ldap    |
                          ---------

Load balanced machines:
  rproxy1     2
  rproxy2     3
  rproxy3     2
  webmail     2
  imap        2
  smtp        2
  ldap        2
  -------------
  Total      15

All measures are Mb/s (Megabits/second).

The biggest traffic by far are the 3 machines in group rproxy2 that are
averaging around 10-20 Mb/s each (peaks around 30 Mb/s).  The 2 machines
in group rproxy1 are only doing around 4-8 Mb/s each.  The traffic to
the load balancer (whichever one is active) is only about 2-3 Mb/s
continuous.  This is just inbound traffic since outbound traffic goes
out through a different gateway.

Now for system utilization:
miniip root # uptime
 13:43:18 up 98 days, 10:10,  1 user,  load average: 0.04, 0.03, 0.00
miniip root # free
             total       used       free     shared    buffers
cached
Mem:       2072284     589920    1482364          0     242520
150180
-/+ buffers/cache:     197220    1875064
Swap:       923728          0     923728
miniip root # cat /proc/cpuinfo 
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 8
model name      : Pentium III (Coppermine)
stepping        : 1
cpu MHz         : 696.744
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 mmx fxsr sse
bogomips        : 1376.25

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 8
model name      : Pentium III (Coppermine)
stepping        : 1
cpu MHz         : 696.744
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 mmx fxsr sse
bogomips        : 1388.54

Note how little memory is actually used.
-- 
Regards...              Todd
we're off on the usual strange tangents.  next will be whether
it is ethical to walk in your neighbor's open house if they're
running ipv6:-).                                  --Randy Bush
Linux kernel 2.6.12-15mdksmp   2 users,  load average: 0.05, 0.05, 0.04

<Prev in Thread] Current Thread [Next in Thread>