|
Someone wrote me for help:
> you give me a step by step from there i.e. all the
> patches, kernel upgrades, iptables commands,
> versions of ipvsadm , piranha, etc. that you ended
> up using to get things working.
Here's the result:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PB'S NUTSHELL HOWTO FOR PIRANHA/LVS/NAT
This represents my amateur experience learning
and installing Piranha/LVS and making it work.
I do not claim this will work for you, but it
might help. Contact the piranha-list@xxxxxxxxxx
if you need help.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Install Redhat 7.2 + all errata + kernel errata
(currently 2.4.9-31). http://www.redhat.com/errata
Download and install the following RPM's from
http://freshmeat.net/releases/70660/
piranha-0.6.0-19.i386.rpm
scsi-reserve-0.7-6.i386.rpm
scsi-reserve-devel-0.7-6.i386.rpm
ipvsadm-1.18-8.i386.rpm
cd to /etc/sysconfig/ha/web and rename docs.html and
create a symbolic link as follows:
ln -s /usr/share/doc/piranha-0.6.0/docs docs.html
so the Piranha GUI documentation link will work.
Make your /etc/sysctl.conf file look like this:
# Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification
net.ipv4.conf.default.rp_filter = 1
# Disables the magic-sysrq key
kernel.sysrq = 0
Put this rc.lvs (which I created) in your /etc/rc.d/
directory and make the rc.local file run it.
~~~~~~~rc.lvs~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/bin/sh
###################################################
## LVS (ip_vs / piranha) Setup Requires this.
###################################################
# Flush previous rules
iptables -t nat -F -v
# Turn on IP Forwarding or set in /etc/sysctl.conf
echo 1 >/proc/sys/net/ipv4/ip_forward
# EXAMPLES Kernel 2.4 should use this.
# modprobe iptable_nat
# iptables -t nat -A POSTROUTING -s n.n.n.n/24 -j
MASQUERADE
modprobe iptable_nat
iptables -v -t nat -A POSTROUTING -s 184.126.10.0/24
-j MASQUERADE
# EXAMPLES Firewall Marks - Only if used in lvs.cf.
# iptables -F -t mangle -v
# iptables -t mangle -A PREROUTING -i eth0 -p tcp -s
0.0.0.0/0 -d 10.11.12/24 --dport 21 -j MARK --set-mark
2 -v
# iptables -t mangle -A PREROUTING -i eth0 -p tcp -s
0.0.0.0/0 -d 10.11.12/24 --dport ftp -j MARK
--set-mark 2 -v
# iptables -t mangle -A PREROUTING -i eth0 -p tcp -s
0.0.0.0/0 -d 10.11.12/24 --dport 10000:20000 -j MARK
--set-mark 2 -v
# iptables -L -t mangle -v
# Run mannually or uncomment. This will also spawn
nanny daemons.
/usr/sbin/lvs --configfile=/etc/sysconfig/ha/lvs.cf
# EXAMPLES Another way to add a service to several
real hosts. wlc rr
# ipvsadm -A -t 207.175.44.110:80 -s rr
# ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.1:80
-m
# ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.2:80
-m
# ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.3:80
-m
# ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.4:80
-m
# ipvsadm -a -t 207.175.44.110:80 -r 192.168.10.5:80
-m
#
# Need this to make smtp port 25 route with masq. Run
after loading lvs.
# smtp masqued routing does not work without this.
ipvsadm -A -t 184.126.13.26:25 -s wlc
ipvsadm -a -t 184.126.13.26:25 -r 184.126.10.37:25 -m
# Start the Piranha-GUI - Do this last.
# Access with http://piranhahostname:3636
/etc/rc.d/init.d/piranha-gui start
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can edit the /etc/sysconfig/ha/lvs.cf file with
the Piranha GUI using URL
http://yourPiranhaHostname:3636 but you need to first
run "piranha-passwd YourPassword". Then the ID is
"piranha" with that password, when you access Piranha
GUI.
Or you can edit your /etc/sysconfig/ha/lvs.cf
manually, but not recommended unless you know what you
are doing.
But it's pretty straight forward. Here's a sample
which uses one LVS server to load balance to one real
server (but you can add more real servers). Note you
can only use NAT routing. You CANNOT use IP TUNNELING,
nor DIRECT ROUTING, as the stock kernel has an ARP
problem. There is a patch, but I cannot help with
that.
My setup layed out herein uses NAT only.
~~~~~~~~~~~~~~lvs.cf sample~~~~~~~~~~~~~~~~~~~~~
serial_no = 89
primary = 184.126.13.25
service = lvs
backup = 0.0.0.0
heartbeat = 1
heartbeat_port = 539
keepalive = 6
deadtime = 18
network = nat
nat_router = 184.126.10.22 eth1:1
nat_nmask = 255.255.255.0
reservation_conflict_action = preempt
debug_level = NONE
virtual piranha_http {
active = 1
address = 184.126.13.26 eth0:1
vip_nmask = 255.255.255.0
port = 80
send = "GET / HTTP/1.0\r\n\r\n"
expect = "HTTP"
load_monitor = uptime
scheduler = wlc
protocol = tcp
timeout = 6
reentry = 15
quiesce_server = 0
server real1.foobar.edu {
address = 184.126.10.56
active = 1
weight = 1
}
}
virtual piranha_smtp {
active = 1
address = 184.126.13.26 eth0:1
vip_nmask = 255.255.255.0
port = 25
send = "GET / HTTP/1.0\r\n\r\n"
expect = "HTTP"
load_monitor = uptime
scheduler = wlc
protocol = tcp
timeout = 6
reentry = 15
quiesce_server = 0
server real1.foobar.edu {
address = 184.126.10.37
active = 1
weight = 1
}
}
virtual piranha_https {
active = 1
address = 184.126.13.26 eth0:1
vip_nmask = 255.255.255.0
port = 443
send = "GET / HTTP/1.0\r\n\r\n"
expect = "HTTP"
load_monitor = uptime
scheduler = wlc
protocol = tcp
timeout = 6
reentry = 15
quiesce_server = 0
server real1.foobar.edu {
address = 184.126.10.27
active = 1
weight = 1
}
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In the lvs.cf above take note of the Piranha/LVS
server's eth0 / eth0:1 which is the public network and
eth1 / eth1:1 for the private network side. eth0 is
the host address, eth0:1 is the VIRTUAL address (which
you put in your DNS for this host); eth1 is a host
address and eth1:1 is the NAT ROUTER address for
private network side. The gateway for each eth card is
the one your LAN/WAN Admin gives you.
On the private network are REAL SERVERS 1, 2, etc.
(only 1 is in the lvs.cf above). Each real server has
eth0 as host address, eth0:1, eth0:2 etc. for each
service, in my case for http 80, smtp 25 and https
443.
(I DON'T KNOW FOR SURE IF THIS IS ABSOLUTELY REQUIRED,
so many virtual ethernets, one per port, but is only
way I got it to work so far.)
The GATEWAY for the REAL SERVERS must be the IP
ADDRESS OF THE Piranha/LVS SERVER'S NAT ROUTER
address.
You can setup the ethernets and gateways on RH7.2 in
the /etc/sysconfig/network (for default gateway) and
/etc/sysconfig/networking/ and
/etc/sysconfig/network-scripts/ dirs where you will
see scripts like ifcfg-eth0, ifcfg-eth0:1 and so on.
/etc/rc.d/init.d/network restart/stop/start will reset
your network, as well as ifdown eth0:x and ifup
eth0:x.
Check your routing with route -e.
The IP ADDRESSES for ethernet cards for the
Piranha/LVS server on the PRIVATE NETWORK **MUST BE*
the same network as the ethernet cards in the REAL
SERVERS which are on the private network. The IP
ADDRESSES for the eth cards for the Prianha/LVS PUBLIC
SIDE NETWORK should be a differnet network.
Graphically speaking:
PUBLIC NETWORK (ie. 13.0)
|
eth0 host addr public net side
eth0:1 virtual addr public net
[LVS SERVER]
eth1 host addr private side
eth1:1 NAT Router addr private side
|
PRIVATE NETWORK (ie. 10.0)
|
eth0 http 80 service on private net
eth0:1 smtp 25 service "
eth0:2 https 443 service "
[REAL SERVER]
The above setup is reflected in the lvs.cf file.
(But again, I am not sure if each service needs
to have it's own IP address like I am showing, I just
got it to work this way.)
You can access the LVS Howto's in the Piranha GUI for
more on iptables and ipvsadm commands and firewall
marks, but I includes helpful examples in the rc.lvs
above which are both from those howto's and from the
piranha-list red hat help.
FINAL NOTES:
The above setup got my http, smtp and https requests
NAT-routing with masquerade through my Piranha/LVA
server to my one backend Real Server running http and
smtp. You should of course add a 2nd real server to
make this worth while, and thing about a 2nd
Piranha/LVS server with heartbeat failover.
Killing the lvs damon and can be done by first doing a
"ps afx | grep lvs" then "kill -15 <pid for lvs>" and
that will also kill the nanny daemons. Far as I can
tell you need to do then anytime you change the
lvs.cf, and there's no way to make it -SIGHUP, as it
leaves the nanny's out to lunch.
I STILL NEED HELP TOO:
On the real server side, I have not been able to get
an Apache <Virtual-server> Redirect /
https://website.com:443/ as of yet. In other words,
before LVS, I could go to http://website.com and it
would route me to https://website.com:443, but that no
longer works with LVS. Anyone who figures this out,
please tell me the details. peterbaitz@xxxxxxxxxx
Thanks
Peter
__________________________________________________
Do You Yahoo!?
Try FREE Yahoo! Mail - the world's greatest free email!
http://mail.yahoo.com/
|
