LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: No buffer space available

To: Roberto Nibali <ratz@xxxxxxxxxxxx>
Subject: Re: No buffer space available
Cc: lvs-users@xxxxxxxxxxxxxxxxxxxxxx, Jeremy Kusnetz <JKusnetz@xxxxxxxx>
From: Julian Anastasov <ja@xxxxxx>
Date: Tue, 1 Oct 2002 00:06:13 +0000 (GMT)
        Hello,

On Mon, 30 Sep 2002, Roberto Nibali wrote:

> Unfortunately this doesn't happen to me, but maybe Alex Kramakov would
> be willing to test it.

        OK, I'll try to create patch, may be tomorrow.

> > gc_thresh2 > 2688 (more than hosts on the LAN), 4096?
> > gc_thresh3 > gc_thresh2
>
> I need to check back with the code to find out which means what. Hey,
> what about extending LVS to have a total count of active RIPs and to
> check if gc_tresh* is too low an raise it accordingly? This would be
> nice, not?

        Good idea for Joe, he can add it to configure-lvs but
of course any other users can's see it. What to say, people
running such big system should tune the kernel :) If there are
no recommendations for kernel settings may be we can write
such document? :)

As for gc_thresh*:

- if entries are more than gc_thresh2 => run GC on each 5sec

- gc_thresh3: max number of entries

- I'm not sure gc_thresh1 is used in 2.4, may be only in Linux 2.2

> > As for /proc/sys/net/ipv4/route:
> >
> > max_size should be increased enough not to lead to failed
> > routes.
>
> Yup, forget about that one. But normally this is high enough, not? Also

        Yes, it is calculated based on the installed RAM.

> it is truely complex to understand what influence each value in
> /proc/sys/net/ipv4/{neigh|route}/* have to the system.

        No influence from other parameters if you are hit from
high load (many hosts contacting you). Sometimes the garbage
collection algorithm (GC) can not reduce the total cache entries
below the max_size and we see the "dst cache overflow"
message when the load is high.

        Ops, there is no such message in the logs, so may
be only the neighbour cache should be tuned.

> > As for the free memory problems, whether ip_dst_cache in /proc/slabinfo
> > shows big number for current/max conns (1st/2nd column)?
>
> As you can see from various postings from him, it doesn't seem to be a
> problem.

        Yes, when the memory is full bad things happen. I remember
the 2.2 system behave very badly when some page order is exhausted
and the network receives ENOBUFS/ENOMEM while at the same time
the buffer cache shows big number of pages, vary bad. I didn't
tried the 2.4 with the same load, I hope the 2.4+ kernels
handle this properly.

> Best regards,
> Roberto Nibali, ratz

Regards

--
Julian Anastasov <ja@xxxxxx>



<Prev in Thread] Current Thread [Next in Thread>