|
> > Many thanks to Julian, Wensong, and Pablo. I'll give a new
> kernel/lvs
> > combo a shot with the 'hidden' patch and corrected
> > ifconfig'ing. After
> > I figure it all out, I'll try to leave some newbie
> instructions behind
> > for the next Red Hat 7.2 LVS-TUN guy that comes along. :-)
>
> Many thanks again. I went ahead and upgraded to 2.4.17, and used the
> "hidden" patch -- my LVS-DR cluster now works gorgeously. As
> I said, I
> would post my notes
Okay, this is the last time I post on this, I promise. :-) My trigger
finger was a little quick on the last mail I sent out, so this one
includes *all* the instructions (more or less) the way I did it to setup
a LVS-DR on Red Hat 7.2.
I hope that it may be of use to someone sometime. Next project:
automatic failover to backup LVS director...
+---------------------------------------------------
| Dan Browning, Sr. Tech Consultant
| Kavod Technologies, 1498 SE Tech Center Pl Ste 170
| Vancouver, WA 98683 <dan.browning@xxxxxxxxx>
+---------------------------------------------------
BOO! We changed Coke again! BLEAH! BLEAH!
mkdir ~/download/piranha
cd ~/download/piranha
wget \
ftp://ftp.linux.org.uk/pub/linux/piranha/7.2/piranha/piranha-0.6.0-15.i3
86.rpm \
ftp://ftp.linux.org.uk/pub/linux/piranha/7.2/ipvsadm/ipvsadm-1.18-8.i386
.rpm \
ftp://ftp.linux.org.uk/pub/linux/piranha/7.2/scsi_reserve/scsi_reserve-0
.7-6.i386.rpm \
-c
rpm -Uvh *.rpm
chkconfig piranha-gui on
service piranha-gui restart
piranha-passwd homelast
# If you will be using two directors (that need to sync seemlessly)
# Setup keyless scp on all the nodes:
ssh-keygen -t rsa
cat .ssh/id_rsa.pub | ssh SERVERNAME 'cat >>~/.ssh/authorized_keys2'
# Helpful Documentation
http://www.linuxvirtualserver.org/Joseph.Mack/mini-HOWTO/LVS-mini-HOWTO-
6.html
http://ha.redhat.com/docs/high-availability/index.html
http://www.linuxvirtualserver.org/docs/arp.html
http://www.linux-vs.org/~julian/hidden.txt
# Enabling IP Encapsulation
# On each real server, establish a tunnel between it and each virtual
server address. For example, these commands establish two tunnels (tunl0
and # # tunl1) to two virtual server addresses...
# To prevent real servers, rather than the active router,
# from intercepting ARP broadcasts, you also need to hide
# tunnels from ARP broadcasts. For example, these commands
# hide tunnels tunl0:
# Insert the ipip module, if not statically compiled into the kernel
already
insmod ipip
# Make the tunl0 device up
ifconfig tunl0 0.0.0.0 up
# Start the hiding interface functionality
echo 1 > /proc/sys/net/ipv4/conf/all/hidden
# Hide all addresses for this tunnel device
echo 1 > /proc/sys/net/ipv4/conf/tunl0/hidden
# Configure a VIP on an alias of tunnel device
ifconfig tunl0:0 1.2.3.4 up
# Testing
lynx --dump http://VIP/test
ab -n 100 -c 10 http://VIP/index.html
Environment: Red Hat 7.2, Piranha 0.6.0-15, RH stock kernel (2.4.7-10)
________
| |
| client |
|________|
CIP=5.6.7.8
|
|
|
__________
| |
| Internet |
|__________|
|
|
|
VIP=1.2.3.4 (eth0:1)
__________
| |
| director |
|__________|
DIP=1.2.3.5 (eth0)
|
|
/---------------------------------\
| | |
| | |
RIP1=1.2.3.10 N/A (yet) N/A (yet)
_____________ _____________ _____________
| | | | | |
| realserver | | realserver | | realserver |
|_____________| |_____________| |_____________|
###############
## DETAILS:
###############
Setup the Director:
Install Piranha, lvsadm
Configure like so:
serial_no = 38
primary = 1.2.3.4
service = lvs
backup = 0.0.0.0
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 18
network = direct
reservation_conflict_action = preempt
debug_level = NONE
virtual http {
active = 1
address = 1.2.3.5 eth1
port = 80
send = "GET / HTTP/1.0\r\n\r\n"
expect = "HTTP"
load_monitor = none
scheduler = wlc
protocol = tcp
timeout = 6
reentry = 15
quiesce_server = 0
server www4real {
address = 1.2.3.10
active = 1
weight = 1
}
}
IP Virtual Server version 0.8.1 (size=65536)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 1.2.3.5:80 rr
-> 1.2.3.10:80 Route 1 0 0
CURRENT LVS PROCESSES
root 1992 0.0 0.0 1604 600 ? S 15:45 0:00 pulse
root 2295 0.0 0.0 1604 600 ? S 15:45 0:00
/usr/sbin/lvs --nofork -c /etc/sysconfig/ha/lvs.cf
root 2299 0.0 0.0 1640 648 ? S 15:45 0:00
/usr/sbin/nanny -c -h 1.2.3.10 -p 80 -s GET / HTTP/1.0\r\n\r\n -
## Notes for recompiling 2.4.17 with ipvs & hidden patches on Red Hat
7.2 ##
## (On both real server and the director)
# Download directory
export D=/tmp/download
mkdir $D
cd $D
#kernel
wget http://www.kernel.org/pub/linux/kernel/v2.4/linux-2.4.17.tar.gz
#hidden patch
wget http://www.linux-vs.org/~julian/hidden-2.4.5-1.diff
#IPVS patch
wget
http://www.linuxvirtualserver.org/software/kernel-2.4/linux-2.4.12-ipvs-
0.8.2.patch.gz
#net filter module - if you want to do just the module instead of the
big kernel patch above.
wget
http://www.linuxvirtualserver.org/software/kernel-2.4/ipvs-0.8.2.tar.gz
#ipvs admin
wget ftp://rpmfind.net/linux/redhatbeta/ha/i386/ipvsadm-1.17-2.i386.rpm
# Unpack new kernel
tar zxvf linux-2.4.17.tar.gz
# Unpack ipvs patch
gunzip linux-2.4.12-ipvs-0.8.2.patch.gz
# Unpack kernel
mv linux /usr/src/linux-2.4.17
cd /usr/src
# Recreate symlink
rm -f linux-2.4
ln -s linux-2.4.17 linux-2.4
ln -s linux-2.4.17 linux
# Apply "hidden" patch
cd linux-2.4.17
patch -p1 < $D/hidden-2.4.5-1.diff
Should see:
###############################
patching file include/linux/inetdevice.h
patching file include/linux/sysctl.h
Hunk #1 succeeded at 334 (offset 9 lines).
patching file net/ipv4/arp.c
Hunk #3 succeeded at 754 (offset -1 lines).
patching file net/ipv4/devinet.c
Hunk #1 succeeded at 756 (offset 20 lines).
Hunk #2 succeeded at 1013 (offset -4 lines).
Hunk #3 succeeded at 1079 (offset 20 lines).
patching file Documentation/filesystems/proc.txt
Hunk #1 succeeded at 1583 (offset 5 lines).
patching file Documentation/networking/ip-sysctl.txt
###############################
# Apply ipvs patch
patch -p1 < $D/linux-2.4.12-ipvs-0.8.2.patch
# ipvsadm 1.18-8, which is newer, is already installed (from piranha
project)
make clean
make mrproper
make menuconfig
make bzImage
make modules
make modules_install
make install #doesn't support GRUB yet. - or can copy the
arch/i386/boot/bzImage file manually
vi /boot/grub/grub.conf:
title 2.4.17_ipvs
root (hd0,0)
kernel /boot/vmlinuz-2.4.17 ro root=/dev/sda1
#now copy the /usr/src/linux-2.4.17 to the next linux box:
tar czf linux-2.4.17-dir.tgz /usr/src/linux-2.4.17/
scp linux-2.4.17-dir.tgz SERVER_TWO:/usr/src
#now unpack in SERVER_TWO
tar zxvf linux-2.4.17-dir.tgz
cd linux-2.4.17
make modules_install
make install
# do grub config again.
title 2.4.17_ipvs
root (hd0,0)
kernel /boot/vmlinuz-2.4.17 ro root=/dev/sda1
# reboot!
|
