Re: Fwd: LVS on local node

To: Simon Horman <horms@xxxxxxxxxxxx>
Subject: Re: Fwd: LVS on local node
Cc: Franchoze Eric <franchoze@xxxxxxxxx>, wensong@xxxxxxxxxxxx, lvs-devel@xxxxxxxxxxxxxxx, netdev@xxxxxxxxxxxxxxx, netfilter-devel@xxxxxxxxxxxxxxx
From: Eric Dumazet <eric.dumazet@xxxxxxxxx>
Date: Thu, 22 Jul 2010 14:59:43 +0200
Le jeudi 22 juillet 2010 à 21:24 +0900, Simon Horman a écrit :
> On Thu, Jul 22, 2010 at 08:56:51AM +0200, Eric Dumazet wrote:
> [snip]
> > lvs seems not very SMP friendly and a bit complex.
> I'd be interested to hear some thoughts on
> how the SMP aspect of that statement could
> be improved.

Hi Simon

I am not familiar with LVS code, so I am probably wrong, but it seems it
could be changed a bit.

Some rwlocks might become spinlocks (faster than rwlocks)

__ip_vs_securetcp_lock for example is always used with
This can be a regular spinlock without even knowing the code.

Some lookups could use RCU to avoid cache line misses, and to be able to
use spinlocks for the write side.

It would be good to have a bench setup with the case of 16 legacy
daemons, and check how many new connections per second can be
established, in an LVS setup and an iptables based one.

With 2.6.35 and RPS, a REDIRECT based solution can chose the target port
without taking any lock (not counting conntrack internal costs of
course), each cpu accessing local memory only.

# No need is eth0 is a multiqueue NIC
echo ffff >/sys/class/net/eth0/queues/rx-0/rps_cpus

for c in `seq 0 15`
  iptables -t nat -A PREROUTING -p tcp --dport 80 -m cpu --cpu $c -j
REDIRECT --to-port $((1000 + $c))

To unsubscribe from this list: send the line "unsubscribe lvs-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at

<Prev in Thread] Current Thread [Next in Thread>