Re: [lvs-users] question about load balancing smtp

To: " users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] question about load balancing smtp
From: "L.S. Keijser" <leon@xxxxxxxx>
Date: Mon, 26 Apr 2010 06:41:11 +0200
On Sun, 2010-04-25 at 20:03 +0200, Michiel van Es wrote:
> I want to load balance smtp with LVS because I heard that LVS is capable 
> of showing the source ip to the real servers.
> Is that correct?


> I tried the following howto:
> And use CentOS 5.4 .
> ut I can not get a listening port 25 on my server.

Your server is ... ???

> Some configs:
> -bash-3.2# ip addr sh eth0

on what host are you issuing this command? I assume the director, but if
you're going to paste output, please specify where it's coming from.

> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast 
> qlen 1000
> link/ether 00:16:3e:4b:df:69 brd ff:ff:ff:ff:ff:ff
> inet brd scope global eth0
> inet brd scope global secondary eth0:1
> inet6 fe80::216:3eff:fe4b:df69/64 scope link
> valid_lft forever preferred_lft forever
> my /etc/ha.d/ldirectord.conf:
> checktimeout=30
> checkinterval=2
> autoreload=yes
> logfile="/var/log/ldirectord.log"
> quiescent=no
> virtual=
> fallback=

A fallback server should provide the same service as the ones your
trying to load-balance. Unless your local SMTP server runs on port 80,
that should probably read '' ;)

> real= gate
> real= gate
> service=smtp
> persistent=100
> scheduler=lblc
> protocol=tcp
> checktype=negotiate
> In /var/log/ldirectord.log:
> [Sun Apr 25 19:12:59 2010|ldirectord|15310] Invoking ldirectord invoked 
> as: /usr/sbin/ldirectord start
> [Sun Apr 25 19:12:59 2010|ldirectord|15310] Starting Linux Director 
> v1.186-ha-2.1.3 as daemon
> [Sun Apr 25 19:12:59 2010|ldirectord|15313] Added virtual server: 
> [Sun Apr 25 19:12:59 2010|ldirectord|15313] Added fallback server: 
> ( (Weight set to 1)
> [Sun Apr 25 19:12:59 2010|ldirectord|15313] Added real server: 
> ( (Weight set to 1)
> [Sun Apr 25 19:12:59 2010|ldirectord|15313] Deleted fallback server: 
> mapped from (
> [Sun Apr 25 19:12:59 2010|ldirectord|15313] Added real server: 
> ( (Weight set to 1)
> Should I restart something?
> Why isn't anything listening on port 25?

LVS isn't a service, but an advanced router. It won't listen on a port;
it'll only forward packets to another host/port.

> I want to use the direct routing method because of the 2 real servers 
> being able to connect direct ot the internet.
> The real servers have the following setup:
> Cluster Nodes Configurations (Apache Real Web Servers Configuration)
> On both web servers http1 and http2, apache should be running having a 
> common serving file (for purpose of get checked by ldirectord).
> yum install httpd -y
> echo "webserverisworking" > /var/www/html/check.txt
> service httpd start
> chkconfig httpd on

Why? You are configuring them for SMTP. There's no need to install a
webserver then. And the 'check.txt' file (or any files) is only
necessary when using the 'checktype=negotiate' type, not the default tcp
connect check.

> Now, Create a loopback interface on each web server, so it doesn’t 
> communicate with your network gateway/router directly.

That's not the point of a loopback device. The point is so that your
realserver can accept a packet destined for the VIP, forwarded by the

The realserver (in a LVS-DR type setup) should always be able to connect
to the gateway directly. Only in LVS-NAT mode, packets that went from
client -> director -> realserver should return via the director.

> vi /etc/sysconfig/network-scripts/ifcfg-lo:0

You shouldn't use RH's network scripts as they will send out an ARP
broadcast when coming up, messing up the table on other hosts.

> It must look like this:
> DEVICE=lo:0
> ONBOOT=yes
> NAME=loopback
> vi /etc/sysctl.conf
> It must look like this:
> net.ipv4.conf.all.arp_ignore = 1
> net.ipv4.conf.eth0.arp_ignore = 1
> net.ipv4.conf.all.arp_announce = 2
> net.ipv4.conf.eth0.arp_announce = 2
> sysctl -p
> ifup lo:0

A better way of configuring the loopback device would be to place these
lines in your /etc/rc.local :

# solve the ARP problem
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce

# Bring up loopback interface
ifconfig lo:0 broadcast netmask 0xffffffff

It looks like your realservers are in a different subnet than your
director. This way, LVS-DR won't work as it requires that all hosts
should be able to arp eachother (read: are on the same subnet, without a
router in between).

You say that you choose LVS-DR because the realservers should be able to
connect to the internet directly. Unless you also mean that they should
be able to accept (SMTP) connections directly, without the director in
between, you could just as well try LVS-NAT. Your realservers would
still be able to connect to the internet.

Also, a better howto would be found here

It's a big read, but you'll have a better understanding of LVS, instead
of following the howtoforge one. I'm not saying the latter is wrong (i
didn't read it thoroughly) but it doesn't seem to explain why certain
things work the way they do.

Good luck,


Please read the documentation before posting - it's available at: mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to
<Prev in Thread] Current Thread [Next in Thread>