|
On Wed, 21 Dec 2005, Jan Abraham wrote:
How do you get the director to load balance requests from
the realservers to the database machines? A different VIP on
the director?
Okay, lets go into the details, i'll explain it on a stripped down version of
our setup:
DB realservers uses IP 10.0.5.1-.5
DBVIPs are 10.0.4.1 and .2
Directors are 10.0.1.51 and .52
dblb1 ~ # uname -srip
Linux 2.6.12-gentoo-r6 Intel(R) Pentium(R) 4 CPU 2.40GHz GenuineIntel
dblb1 ~ # ip addr
1: ethInt: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether do:nt:te:ll:it:ha brd ff:ff:ff:ff:ff:ff
inet 10.0.1.51/20 brd 10.255.255.255 scope global ethInt
inet 10.0.4.1/32 scope global ethInt
inet 10.0.4.2/32 scope global ethInt
2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
dblb1 ~ # ipvsadm -L --sort
IP Virtual Server version 1.2.1 (size=1048576)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP dbv1:mysql wrr
-> db2:mysql Route 100 0 0
-> db3:mysql Route 100 0 0
TCP dbv2:mysql wrr
-> db4:mysql Route 100 0 0
-> db5:mysql Route 100 0 0
(db1 is mysql replication master and is accessed directly via its rip,
replicating tables to db2-5; db2+3 serves for longlasting selects, db4+5
serves fast selects , thus two different vips are needed)
db2 ~ # uname -srip
Linux 2.6.14-gentoo-r2 Intel(R) Xeon(TM) CPU 3.06GHz GenuineIntel
db2 ~ # ip addr
1: ethInt: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether do:nt:te:ll:it:ha brd ff:ff:ff:ff:ff:ff
inet 10.0.5.2/20 brd 10.0.15.255 scope global ethInt
2: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
inet 10.0.4.1/32 brd 10.0.4.1 scope global lo
inet 10.0.4.2/32 brd 10.0.4.2 scope global lo
db2 ~ # arptables -L -n
Chain INPUT (policy ACCEPT)
Chain OUTPUT (policy ACCEPT)
-j DROP -s 10.0.4.1
-j DROP -s 10.0.4.2
why do you drop these (just curious, not related to your
problem)?
Chain FORWARD (policy ACCEPT)
(You may notice that both virtual IPs are set up on all realservers. This is
to be easily able to switch a realserver from dbv1 to dbv2; i don't think it
can be a cause for the strange behaviour).
there's a lot of detail here. Are you using a different VIP
for the database than for the web front end (I assume yes)?
The tcp syn packets went from the webserver through the
ipvs machine, the mac adresses are replaced there and the
packet arrived at one of the database servers (it's
visible in tcpdump at the database server and looks
correct).
ie has the correct IP etc?
Everything is setup the way it should and works for 99.9% of the connects. I
think it's more likely a problem on the realserver, but I have absolutely no
idea...
this is a new one.
Umph. Any clues?
Since the packets arrive at the realserver, I expect it's
not an LVS problem, I would then look for crazy things.
Trying to debug something that works 99.9% of the time is
not going to be fun, unless you can work out a way of
screening out the 99.9%, Is it only one realserver, both?
Can you replace the realserver hardware, software (different
kernel say)? Replace the NICs (different brand altogether).
I have a Dell machine here whose NIC (3Com I believe)
gets lots of RX errors under 2.6.13 (SuSE or RedHat) but
works OK under w2k. I tried about 4 different cards before I
found one that works under Linux (it was an intel 100).
All these NICs work fine on other hardware.
Joe
--
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!
|
