LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [LVS - NAT] alternatives

To: <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [LVS - NAT] alternatives
From: "Don Hinshaw" <dwh@xxxxxxxxxxxxxxxxx>
Date: Tue, 11 Sep 2001 04:26:29 -0000
Roberto Nibali <ratz@xxxxxx> said:

> Hello Don,

> > All of the major commercial load balancers (Radware WSD, F5 BigIP, Alteon 
180
> > series and Cisco LocalDirector) are NAT boxes. They work just fine. They 
also
> 
> Cisco knows the triangulation more which can be looked at as the LVS_DR
> method.

It is similar, according to:
http://www.cisco.com/warp/public/cc/pd/si/11000/prodlit/cscnt_wi.htm

"Cisco CSS 11000 series switches use NAT peering to direct requests to the 
best site with the requested content based on URL or file type, geographic 
proximity, and server/network loads, avoiding the limitations of DNS-based 
site selection and the overhead HTTP redirect.

If a Cisco CSS 11000 series switch receives a request for content and the 
cache or server hosting that content is down or busy, the switch uses 
information communicated among its peers (that is, other Cisco switches) at 
other points of presence (POPs) to locate the best site and server. When the 
best destination site is determined, the request is directed to that site and 
the source IP address is translated to the "new" IP address where the content 
resides. 

Cisco NAT peering preserves the original source IP address from the client so 
that the new site can send the response directly back to the client. NAT 
peering acts as "triangulation protocol," allowing the response to be 
delivered directly to the user over the shortest Internet path."
 
So the triangulation is a function of "NAT peering", which is a Cisco 
proprietary function. Also, the switch is making decisions based on metrics, 
such as server/network loads and geographic locations. The NAT peering is 
transferring the request to another site, which is behaviour that is more 
like a combination of LVS_DR and LVS_TUN, but it's only activating that 
behaviour when the metrics indicate the need for it.

> > all have optional gigabit interfaces. I worked at a company that was 
handling
> > 30mil pages/day (~4 megabytes/sec average outbound traffic) on 10 Sun 
220Rs
> > behind Alteons. NAT was definately -not- an issue.
> 
> Because NAT is handled in the RISC processors. OTOH, try NAT and sticky bit
> set on the Alteon ACEdirector3 and watch huge latencies in server takeout
> and packet dropping. The Cisco had a bug where approximately 10% of all 
packets
> weren't NAT under high traffic load (I cannot confirm this on the gigabit
> ones).

Heh, I don't have experience with all the products from each vendor. I don't 
doubt that any of them can be broken under the right circumstances or loads.
  
> > As far as I'm concerned, DR and TUN are nice toys, but not very useful. 
They
> > don't offer signicant advantages over NAT (in most applications) and they
> > require too much special handling to manage. I can see TUN -seeming to- 
have
> > an advantage in being able to balance across different datacenters, but
> > that's largely offset by the single point of failure. I.e., if the 
datacenter
> > where the directors reside goes offline, all the clusters distributed to
> > other datacenters won't be getting any traffic anyway, unless you change 
the
> > DNS to point to another director in the other datacenter. In which case, 
the
> > F5 3DNS, which does metrics, would be a better solution than tunneling.
> 
> Agreed, but always with the deployed network architecture in mind.

Definately, every solution requires analysis before making decisions.
  
> > DR also seems to offer an advantage, but the packets are coming in 
through a
> > router and switch. The return packets may be bypassing the director, but 
they
> > are almost certainly going back out through the same switch and router. 
The
> > difference added by having the outbound packet headers re-written by the
> > director isn't enough to justify the added complexity of the system.
> 
> I intend to disagree on that one considering the fact that you can use
> one load balancer cluster to load balance for example 24 zones with 24
> NICs when using LVS_DR. This you cannot do with LVS_NAT because of interrupt
> latencies caused by interefering network drivers trying to jump into
> the network BH for NAT. For high performance and cheap & reliable solutions
> for datacenters, you don't have the choice.

I don't understand what you mean. What does this mean, "24 zone with 24 NICs"?
  
> > The only real reason to even use DR is if the number of packets in/out
> > exceeds what a gigabit connection can handle. But at what point does that
> 
> Or my case described above.

I need an explanation before I can think about this.
 
> > happen? I've personally seen a cluster of ~140 real-servers behind an F5
> > BigIP with gigabit interfaces that was handling > 20 megabytes/sec 
sustained
> > outbound traffic, most of which was small packets. I don't know how many
> > pages/day this was, but I can probably safely assume that it was about 5
> > times the system that I used to manage, so I can guess that it was 
somewhere
> > around 150 million pages/day. This was with NAT, and with a lot of special
> > packet handling scripts due to the complex multi-tier configuration of the
> > cluster, and yet the site was very fast. NAT was not an problem.
> 
> Thanks for this report. Can you tell me how long a request usually would
> be sustained? I assume you have dynamically generated pages. So if the
> db queries take very long, you might have a problem with NAT before having
> one with DR :)

I don't know how long the requests were sustained. The cluster was managed by 
a close friend of mine and the .com that owned it has gone under and the 
cluster was dismantled and auctioned off. Yes, they were dynamic (mostly). I 
don't think the db queries were long, the db servers were Sequent boxes (8 or 
10 of them as I recall) with 8 p3/500s and Oracle.

> > I think that the HOWTO does a good job of pointing out the advantages of 
DR
> > and TUN, but it tends to give people the impression that they are superior
> > and that NAT is inferior, which it really isn't. DR and TUN are handy -if 
you
> > need them-. Do you need them? Probably not.
> 
> Good point for the howto. Indeed, with nowadays Internet technology and
> computer technology we should not give the reader the impression that as
> soon as he wants to load balance a moderately surfed site he needs to 
> choose LVS_DR.

Exactly. DR and TUN are valuable, but you have to have a VERY busy site, or 
other special circumstances to really need them.
 
> Thank you,
> Roberto Nibali, ratz

-=dwh=-

________________________________________________________________
http://www.OpenRecording.com For musicians by musicians.
Now with free Web-Based email too!



<Prev in Thread] Current Thread [Next in Thread>