LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: observation with ldirectord and using fwmark

To: Ted Pavlic <tpavlic@xxxxxxxxxxx>
Subject: Re: observation with ldirectord and using fwmark
Cc: lvs-users@xxxxxxxxxxxxxxxxxxxxxx
From: Jeremy Hansen <jeremy@xxxxxxxxxxxx>
Date: Fri, 15 Sep 2000 21:09:08 -0400 (EDT)
Absolutely!  This is very cool.

Now...the position I'm in...I work for a fairly open company, but they
still want numbers unfortunately and stats in comparison to other
products, in my case BigIP's don't seem to be available.

Even if the bigIP has slightly better performance, the cost difference is
large enough that performance hits are tolerable, but I still need a way
to come up with some number.  BigIP's are like $15,000 a piece, 1U's
running LVS is about $1200 a piece.

The problem I've run into is the lack of free, good benchmarking
utilities.

What I envision is something that runs on multiple real client machines
and not only get like http information, but also information from each
one of the real servers and client machine.

So, if you see performance drop, what is the reason, LVS bottleneck, real
server bottleneck, network bandwidth bottleneck.  I haven't found a good
all in one tool to give this type of information.

Even if I do win people over with lvs and we drop those bigIP, an
important factor is the ability to determine how many real servers are
needed for a given site or application.  The company I work for turns out
a lot of other companies and each one is unique in its application, so
this is something that seems hard to determine.

Maybe these aren't really completely lvs related issues, but I would like
to use lvs and I need to proper ammunition to make it widely accepted.

Thanks for any suggestions.

-jeremy

> I know you've solved your problem with checkport, but I wanted to add
> another note about the versatility of using fwmark.
> 
> Using fwmark, you can setup something which used to be a big desire in LVS,
> persistence by port groups.
> 
> For example... Say you were serving HTTP and HTTPS. In this case, you would
> probably want calls to one HTTP server to end up hitting the same HTTPS
> server. This way session information and such would be accessable no matter
> how the end-user was accessing the website.
> 
> Say you also wanted all forms of FTP to work... You would need persistence
> there, but not necessarily the same persistence as HTTP/HTTPS.
> 
> And other protocols do not need to be persistent.
> 
> Back in the olden days before fwmark, to do any of this you would have to
> make ALL ports persistent. You couldn't simply say "Group 80 and 443
> together and make them persistent and then make 21, 20, AND 1024:65535
> persistent." If one service went down, you would have to bring down ALL
> services. Some sort of persistence by port groups would allow you to only
> need to take down whatever went down and the affected server could still
> serve other services.
> 
> FWMARK allows you to do this by way of setting up multiple FWMARKs.
> 
> That is -- you can use ipchains to say that:
> 
> HTTP,HTTPS --> FWMARK1
> 
> FTP --> FWMARK2
> 
> SMTP --> FWMARK3
> 
> POP --> FWMARK4
> 
> Then in LVS, setup:
> 
> FWMARK1 --> WLC Persistent 600
> 
> FWMARK2 --> WLC Persistent 300
> 
> FWMARK3 --> WLC
> 
> FWMARK4 --> WLC
> 
> And if FTP went down, all you'd have to do is stop scheduling FTP rather
> than stop scheduling EVERYTHING.
> 
> Also note that FWMARK makes setting up MASS VIPs really easy (of course
> because of recent ARIN policy changes, this probably won't be done much more
> anymore). That is, if you wanted to load balance 1000 VIPs, it might be easy
> to setup one single rule in ipchains to cover them all, where it would be
> 1000 rules for EACH real server in ipvsadm.
> 
> Just a few reasons why I think the FWMARK support in LVS makes LVS a lot
> more powerful.
> 
> It makes me think that if there was a utility already out there that could
> sit on a director and figure out where name-based packets were going it
> might be able to mark each name-based host with a different FWMARK and pass
> that right back to LVS... Then LVS wouldn't have to worry about handling
> name-based stuff ITSELF. Of course the name-based challenge is even more
> challenging considering how much data needs to be looked at to figure out if
> a TCP stream is a name-based HTTP session going to specific name X.... but
> that's a completely other argument... Just food for thought.
> 
> All the best --
> Ted
> 
> ----- Original Message -----
> From: "Jeremy Hansen" <jeremy@xxxxxxxxxxxx>
> To: <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
> Sent: Thursday, September 14, 2000 3:19 PM
> Subject: observation with ldirectord and using fwmark
> 
> 
> >
> > Now that Horms fixed ldirectord to work with fwmark, I've made an
> > observation and I want to see if my concern is valid.
> >
> > The cool thing about using fwmark for me is the fact that you can do
> > something like this:
> >
> > -A input -s 0.0.0.0/0.0.0.0 -d 10.2.9.5/255.255.255.255 80:80 -p 6 -m 1
> > -A input -s 0.0.0.0/0.0.0.0 -d 10.2.9.5/255.255.255.255 443:443 -p 6 -m 1
> > -A input -s 0.0.0.0/0.0.0.0 -d 10.2.9.5/255.255.255.255 21:21 -p 6 -m 1
> >
> > I can associate a bunch of different but specific ports to a single fwmark
> > and then this allows me to place a single rule in ipvsadm:
> >
> > ipvsadm -A -f 1 10.2.9.5:0
> > ipvsadm -a -f 1  -r 10.2.9.10:0
> > ipvsadm -a -f 1  -r 10.2.9.11:0
> >
> > IP Virtual Server version 0.9.15 (size=8192)
> > Prot LocalAddress:Port Scheduler Flags
> >   -> RemoteAddress:Port          Forward Weight ActiveConn InActConn
> > FWM  1 wlc
> >   -> 10.2.9.11:0                 Route   1      0          0
> >   -> 10.2.9.10:0                 Route   1      0          0
> >
> > So this is great, cause if I have ftp, https, http all associate with
> > fwmark 1 for a group of services that will definitely be balances accross
> > all real server, then this to me makes thing real simple.  One rule, three
> > different services, clean and nice.
> >
> > But the problem with this is that ldirectord then seem to get
> > confused.  Something like this in my config:
> >
> > virtual=1
> >         real=10.2.9.10:0 gate
> >         real=10.2.9.11:0 gate
> >         fallback=127.0.0.1:80
> >         service=none
> >         scheduler=rr
> >         #persistent=600
> >         protocol=fwm
> >         checktype=connect
> >
> > just causes ldirectord to fail and use the fallback server and I'm
> > assuming this is because the port 0 is used as the reference as to what
> > port to use to connect and test.
> >
> > So, is there any way around this?
> >
> > Using something like real=10.2.9.10:80 makes it work fine of course, but
> > won't this screw up my ability to fwmark multiple ports?
> >
> > Thanks
> > -jeremy
> >
> > eholes.org * jeremy@xxxxxxxxxx
> > -----------------------------------------
> > eholes have feelings too...
> >
> >
> >
> >
> 
> 

eholes.org * jeremy@xxxxxxxxxx
-----------------------------------------
eholes have feelings too...



<Prev in Thread] Current Thread [Next in Thread>