|
Hello,
Thanks a lot for your reply and advice.
On Mon, 3 Jul 2006 19:25:15 +0900 (JST)
Horms <horms@xxxxxxxxxxxx> wrote:
> In article <004001c69ddf$6c357b70$0200a8c0@PCGTR1B> you wrote:
> > Hello,
> >
> > I almost found out the reason of the limit (about 28230)
> > of ActConn + InActConn.
> > The default of /proc/sys/net/ipv4/ip_local_port_range is "32768 61000".
> > In short, 61000 - 32768 = 28232.
> > The number of client of our test environment is one.
>
> As Ratz mention, this value does not affect LVS directly. However,
> it might affect your clients. In particular if you only have one,
> it may only be able to generate 61000 - 32768 connections.
>
> >
> > The hash key of ip_vs_conn_tab (connection table) is based on
> > protocol, s_addr(caddr), s_port(cport), d_addr(vaddr), and d_port(vport).
> > So I think that the max limit of hash key produced by hash function
> > is 28232(default) for one client to same virtual server.
> > Therefore, I think the limit of ActConn + InActConn for every client
> > at a high load exists and the number of hash key for ip_vs_conn_tab
> > from same client to same virtual server (to a realserver) is full.
>
> Please remember that the hash key is just that, and that the hash
> buckets can have multiple entries. Certainly if s_addr, d_addr and
> d_port are constant, which I believe they are in your test, then the
> number of keys will be bound by s_port (which is probably bound by the
> limit discussed above). However for a more usual case, s_addr at least
> varies and this gives a much larger bound of unique hash keys.
I also think your idea is correct in a usual case.
>
> That said, the real bound on the hash keys is likely to be
> CONFIG_IP_VS_TAB_BITS, which by default is 12, giving 2^12 = 2048
> hash keys. But again, the buckets can have multiple entries.
I had understood what you explained.
I also had checked that IP_VS_CONN_TAB_SIZE of hash table is related to
CONFIG_IP_VS_TAB_BITS.
In short, IP_VS_CONN_TAB_SIZE is (1 << IP_VS_CONN_TAB_BITS) in ip_vs.h,
namely IP_VS_CONN_TAB_SIZE is 2^13 = 4096.
And it is used as an initial value by "ip_vs_conn_tab = vmalloc(
IP_VS_CONN_TAB_SIZE*sizeof(struct list_head));" in ip_vs_conn.c .
Then the hash entries of ip_vs_conn_tab can increase If necessary.
However as far as a hash key of ip_vs_conn_tab is based on
s_addr and s_port, I think the max limit of hash key for same client
exists and the limit is related to ip_port_local_range of the client.
But I think bringing the max limit isn't responsibility of the client.
Because the client has only to reuse a port number of tcp connection
without setting HTTP Keep Alive or using many many simultaneous
connections.
I'm anxious whether I can inform you of my thought because of
my poor English.
>
> > So I think that strange behavior at a high load was occured by
> > the above reason.
> > In short, the cause of the load balancing trouble at a high load is mainly
> > related to ip_vs_conn table managed by hash key based on the above elements
> > and the limit of port range of a client
> >
> > But I think that this specification of ip_vs is no problem in real
> > environment.
>
> I suspect you have found a limitation in the number of requests
> that a single client can generate, or more to the point the
> number of requests from a single source IP address. Could you try
> with a few more clients and/or source IP addresses?
I'm sorry, I can't try a test with a few more clients right now
because of my poor test environment.
However I think that load balancing problem (strange behavior)
doesn't improve even if I try with a few more clients.
>
> --
> Horms
> H: http://www.vergenet.net/~horms/ W: http://www.valinux.co.jp/en/
>
Thanks.
Best regards,
--
Hideaki kondo
|
