|
Question: If running a load balancer tester, say the one from IXIA to
issue connections to 100 powerful web servers, would all the parameters
in Julian's description need to be changed, or it should not be a problem
for having many many connections from a single tester?
At 09:44 AM 5/14/00 +0300, Julian Anastasov wrote:
> Hello Dan,
>
>On Sat, 13 May 2000, Dan wrote:
>
> > Hi Julian:
> >
> > I 100%, completely, fully understand the difference between LVS masquerading
> > and Linux masquerading. My point is that LVS does not live in a vacuum.
> > Other system parameters affect it - if one is not aware of these side
> > effects than an LVS box (not just the few snippets of code that implement
> > LVS but the LVS system as a *whole*) can easily be a bottleneck. That is not
> > an opinion - that is a fact. I thought it would be helpful to share my
> > experience with others who may run into the same problem under heavy load
> > rather than letting them figure it out for themselves. I sort've thought
> > that was the point in having a mailing list. Perhaps it would be better in
> > the future if I just keep my experience to myself and let others blindly
> > rediscover what I spent so many hours working out.
>
> My first goal was to understand where the problem
>come from. Of course, the details a meaningful because it is
>possible MASQ to have bugs. This is the reason I asked for
>details. And I still can't understand with what kind of
>connections the problem happens. My opinion is "it is very
>hard to reach a 4096 per-host limit with so small number of
>4746 entries". You are the one who first switched to private
>discussion, remember? My recommendation is: don't perform
>masq-to-host tests with the masquerading. Of course, the
>ports are limited. For me, the main limit in the
>masquerading (lets tell it to the users too) is the limit
>for the number of connections per protocol which is now
>40960 . The limit is increased by changing PORT_MASQ_MUL.
>The other parameter that can be tuned is IP_MASQ_TAB_SIZE to
>lookup the masq tables faster. 256 is not enough. LVS uses
>4096 by default for its table. And as a last step the
>reserved masq port range: PORT_MASQ_BEGIN and PORT_MASQ_END.
>But only when there are many connections from the internal
>hosts to one external service.
>
> I like the details :) Can you tell me what is the
>reported message when a limit is reached but in situation
>without your masq-to-remote_host floods. In normal client
>load. In your first message you report for 36214 free ports
>but induced from tests. This is the host-to-host masquerade
>limit. Once you said that you have 10000-11000 connections.
>What is the reported "free ports" value when the message is
>displayed in normal client load? Are these connections
>established or are connections reported by looking in the
>/proc file system, i.e. in any state? You know, there are
>many connections in FIN_WAIT state. It is possible the
>40960 limit to be reached. It is recommended the
>/proc/sys/net/ipv4/ip_always_defrag value to be inspected.
>But in your configuration it is very difficult because this
>number includes not only the number of the MASQ connections
>but also the number of LVS connections (active and
>inactive). And only the MASQ connections are limited.
>
> Well, You know, the main goal is to tune the MASQ
>for such situations, for all LVS users. And if possible to
>solve any related problems in the masquerade. Because the
>problem exists even without applying the LVS patch. Right?
>Later we can include some changes in the LVS patch related
>to the masquerade limitations. This is the reason I think
>the details are useful. By this way everyone can decide if
>this is a possible problem for his setup. And may be to
>switch to direct route mode because the client requests and
>the proxy requests can be separated. For example to use a
>LVS director to balance the client requests to the proxy
>servers but the real servers to use other gateway(s) for
>their FTP/HTTP traffic. Of course, there are so many things
>that must be considered. At least, it is not very easy.
>
> My wish is the LVS web site to include many variants
>for setups. Even now there are many examples but may be not
>enough.
>
> And please change your statement to "Masquerading is
>a bottleneck", now, when all we know the difference :)
>
> I just searched the archives for examples. You can
>look in this interesting link:
>
>http://wwwcache.ja.net/JanetService/PilotService-announcement.html
>
>
>Regards
>
>--
>Julian Anastasov <uli@xxxxxxxxxxxxxxxxxxxxxx>
>
>
|
