|
'Most Excellent' Eric Mehlhaff wrote:
> I am trying to configure a VIP that is an address within the interface
> of eth0, to use direct routing. I configure the VIP as an interface alias
> to the eth0 external interface, and on the 'real' machines, the VIP is an
> alias on the tunnel device, as suggested earlier this week on this mailing
> list. The unexpected behavior I'm seeing is that the linux real servers still
> ARP out that VIP address, even though its on the tunnel interface, even though
> it has ARP turned off.
> Is there any fix for this, its certainly unexpected behavior? I'm
> using lvs0.8.3 patched 2.2.12 kernel on ALL machines involved.
Apply my patch below to 2.2.12 or 2.2.13pre6 and it should solve the problem. I
posted this a few days ago.
> -Eric Mehlhaff
> Cryptic Networking
>
> ----------------------------------------------------------------------
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe, e-mail: lvs-users-unsubscribe@xxxxxxxxxxxxxxxxxxxxxx
> For additional commands, e-mail: lvs-users-help@xxxxxxxxxxxxxxxxxxxxxx
Ok, I solved it...
Suggesting the tunl0 device was very helpful, thanks Ted. It didn't solve it
alone but led me to a solution.
The kernel patch below would not stop the problem with loopback, but it did stop
it with tunl0...
Somehow loopback manages not to really have IFF_NOARP set while tunl0 does
automatically (or some problem that looks similar).
With a tunl0 alias and this patch, if you run arpwatch and also "tail -f
/var/log/messages" you will see the kernel attempt to arp_send the tunl0
interface/ip addresses but the patch catches it and complains.
Oddly, the version of ifconfig I'm using, (net-tools 1.51,ifconfig 1.39
(1999-03-18)) doesn't allow -arp on tunl0 (it has it naturally apparently) but
does allow it on lo, where it doesn't really work (or showup in ifconfig's).
This patch is not permanent but should be applied until the real problem is
found
in loopback's processing in the kernel.
*** 2.2.13pre6.orig/net/ipv4/arp.c Mon Sep 13 00:47:35 1999
--- linux/net/ipv4/arp.c Sun Sep 12 23:38:11 1999
***************
*** 427,432 ****
--- 427,433 ----
struct sk_buff *skb;
struct arphdr *arp;
unsigned char *arp_ptr;
+ struct device *dev_real; /* The real device this ARP is for, if not dev
*/
/*
* No arp on this interface.
***************
*** 434,439 ****
--- 435,459 ----
if (dev->flags&IFF_NOARP)
return;
+
+ /*
+ * If this ARP is for another device that doesn't want to arp,
+ * honor that also.
+ */
+
+ {
+ char cbuf[320];
+ sprintf(cbuf, "%s arp_send: src_ip: %s, dest_ip: %s\n",
+ KERN_INFO, in_ntoa(src_ip), in_ntoa(dest_ip));
+ printk(cbuf);
+ }
+ dev_real = ip_dev_find(src_ip);
+ if (dev_real != NULL && dev_real != dev)
+ printk(KERN_INFO "arp_send is called with another device's address\n");
+ if (dev_real != NULL && dev_real != dev && dev_real->flags&IFF_NOARP) {
+ printk(KERN_INFO "arp_send is called with another devices address that
doesn't want to arp\n");
+ return;
+ }
/*
* Allocate a buffer
See http://www.linuxvirtualserver.org for details.
LinuxDirector scripts I use now, not cleaned up yet, but working (addresses
randomized):
#!/bin/sh
#rc.ipvs_drdirector
# Modified from the example rc.ipvs_tunserver example.
# 13Sep1999 Stephen D. Williams sdw@xxxxxxx
# This version of the file directs 4 IP addresses to local
# httpd and one remote server.
#
echo -n "rc.ipvs_drdirector "
#
#echo "deleting any ipfwadm forwarding rules"
#/sbin/ipfwadm -Ff
#
#network setup
VS_NETWORK="222.133.104.0"
VS_NETMASK="255.255.255.0"
VS_BROADCAST="222.133.104.255"
VIP_ADDRESS="222.133.104.226"
VIP_DEVICE="eth0:10"
VS2_NETWORK="222.133.104.0"
VS2_NETMASK="255.255.255.0"
VS2_BROADCAST="222.133.104.255"
VIP2_ADDRESS="222.133.104.223"
VIP2_DEVICE="eth0:0"
VS3_NETWORK="222.133.104.0"
VS3_NETMASK="255.255.255.0"
VS3_BROADCAST="222.133.104.255"
VIP3_ADDRESS="222.133.104.224"
VIP3_DEVICE="eth0:1"
VS4_NETWORK="222.133.104.0"
VS4_NETMASK="255.255.255.0"
VS4_BROADCAST="222.133.104.255"
VIP4_ADDRESS="222.133.104.225"
VIP4_DEVICE="eth0:11"
#
#standard assignment of IP to outside world on Director
#uncomment if not already part of rc.inet1
#VS_ADDRESS="222.133.104.226
#VS_DEVICE="eth0:10"
#/sbin/ifconfig ${VS_DEVICE} ${VS_ADDRESS} broadcast $VS_BROADCAST up
#route add -net ${VS_NETWORK} netmask ${VS_NETMASK} dev ${VS_DEVICE}
#
#VIP link to servers
/sbin/ifconfig ${VIP_DEVICE} ${VIP_ADDRESS} netmask ${VS_NETMASK} \
broadcast ${VS_BROADCAST} up
route del -host ${VIP_ADDRESS}
route add -host ${VIP_ADDRESS} dev ${VIP_DEVICE}
/sbin/ifconfig ${VIP2_DEVICE} ${VIP2_ADDRESS} netmask ${VS_NETMASK} \
broadcast ${VS_BROADCAST} up
route del -host ${VIP2_ADDRESS}
route add -host ${VIP2_ADDRESS} dev ${VIP2_DEVICE}
/sbin/ifconfig ${VIP3_DEVICE} ${VIP3_ADDRESS} netmask ${VS_NETMASK} \
broadcast ${VS_BROADCAST} up
route del -host ${VIP3_ADDRESS}
route add -host ${VIP3_ADDRESS} dev ${VIP3_DEVICE}
/sbin/ifconfig ${VIP4_DEVICE} ${VIP4_ADDRESS} netmask ${VS_NETMASK} \
broadcast ${VS_BROADCAST} up
route del -host ${VIP4_ADDRESS}
route add -host ${VIP4_ADDRESS} dev ${VIP4_DEVICE}
#
#
/sbin/ipvsadm -L
ipvsadm -D -t ${VIP_ADDRESS}:80
ipvsadm -A -t ${VIP_ADDRESS}:80 -s wlc #rr, wrr, or wlc
ipvsadm -D -t ${VIP2_ADDRESS}:80
ipvsadm -A -t ${VIP2_ADDRESS}:80 -s wlc #rr, wrr, or wlc
ipvsadm -D -t ${VIP3_ADDRESS}:80
ipvsadm -A -t ${VIP3_ADDRESS}:80 -s wlc #rr, wrr, or wlc
ipvsadm -D -t ${VIP4_ADDRESS}:80
ipvsadm -A -t ${VIP4_ADDRESS}:80 -s wlc #rr, wrr, or wlc
#add servers and services
#
#SERVER1="222.133.104.224"
#SERVER1="10.1.1.6"
SERVER1="222.133.104.139"
#http to 222.133.104.226
/sbin/ipvsadm -a -t ${VIP_ADDRESS}:80 -r ${SERVER1} -g -w 1
/sbin/ipvsadm -a -t ${VIP2_ADDRESS}:80 -r ${SERVER1} -g -w 1
/sbin/ipvsadm -a -t ${VIP3_ADDRESS}:80 -r ${SERVER1} -g -w 1
/sbin/ipvsadm -a -t ${VIP4_ADDRESS}:80 -r ${SERVER1} -g -w 1
SERVER2="10.1.1.5"
#http to 192.168.1.3 with weight of 2
#/sbin/ipvsadm -a -t ${VIP_ADDRESS}:80 -r ${SERVER2} -g -w 2
#Localhost
/sbin/ipvsadm -a -t ${VIP_ADDRESS}:80 -r 127.0.0.1 -g -w 1
/sbin/ipvsadm -a -t ${VIP2_ADDRESS}:80 -r 127.0.0.1 -g -w 1
/sbin/ipvsadm -a -t ${VIP3_ADDRESS}:80 -r 127.0.0.1 -g -w 1
/sbin/ipvsadm -a -t ${VIP4_ADDRESS}:80 -r 127.0.0.1 -g -w 1
#
#display current settings on exit
/sbin/ipvsadm
#
#---------rc.ipvs_drdirector----------------------
#!/bin/sh
# Modified from the example rc.ipvs_tunserver example.
# 13Sep1999 Stephen D. Williams sdw@xxxxxxx
echo -n "rc.ipvs_drserver "
# This doesn't autoload with just a reference to tunl0 in ifconfig
# I'm now building this into the kernel, but try to load it here.
modprobe ipip.o 2>/dev/null # don't show complaint as it might already be in.
# lo would work except that we can't get the ARP's off...
# For some reason, with my patch, it works with tunl0.
DEV=tunl0
#lo or tunl0
OPTS=
#up -arp or
# In my case I'm handling 4 IP addresses. Replace this with a loop later.
# network and VIP info
VS_BROADCAST="222.133.104.255"
VIP_NETMASK="255.255.255.255"
VIP_ADDRESS="222.133.104.226"
VIP_DEVICE="${DEV}"
VIP2_ADDRESS="222.133.104.223"
VIP2_DEVICE="${DEV}:1"
VIP3_ADDRESS="222.133.104.224"
VIP3_DEVICE="${DEV}:2"
VIP4_ADDRESS="222.133.104.225"
VIP4_DEVICE="${DEV}:3"
#
#install non-arping tunnel device on Server
eval ifconfig $VIP_DEVICE $VIP_ADDRESS netmask $VIP_NETMASK \
broadcast $VS_BROADCAST $OPTS
route del -host ${VIP_ADDRESS} 2>/dev/null
route add -host ${VIP_ADDRESS} dev ${VIP_DEVICE}
eval ifconfig $VIP2_DEVICE $VIP2_ADDRESS netmask $VIP_NETMASK \
broadcast $VS_BROADCAST $OPTS
route del -host ${VIP2_ADDRESS} 2>/dev/null
route add -host ${VIP2_ADDRESS} dev ${VIP2_DEVICE}
eval ifconfig $VIP3_DEVICE $VIP3_ADDRESS netmask $VIP_NETMASK \
broadcast $VS_BROADCAST $OPTS
route del -host ${VIP3_ADDRESS} 2>/dev/null
route add -host ${VIP3_ADDRESS} dev ${VIP3_DEVICE}
eval ifconfig $VIP4_DEVICE $VIP4_ADDRESS netmask $VIP_NETMASK \
broadcast $VS_BROADCAST $OPTS
route del -host ${VIP4_ADDRESS} 2>/dev/null
route add -host ${VIP4_ADDRESS} dev ${VIP4_DEVICE}
#
#echo "LVS Notice: make sure you have a route to the client(s) "
#uncomment if not already handled by rc.inet1
#CLIENT_NETWORK="192.168.1.0"
#CLIENT_NETMASK="255.255.255.0"
#CLIENT_DEVICE="eth0:1"
#route add -net ${CLIENT_NETWORK} netmask ${CLIENT_NETMASK} dev ${CLIENT_DEVICE}
#
#----------------------rc.ipvs_tunserver-----------------
"Stephen D. Williams" wrote:
> Can I still do direct routing or do I have to actually use tunneling with the
> tun devices?
>
> Ted Pavlic wrote:
>
> > Try installing the tunnel device with a VIP and doing a -noarp on that. I
> > believe there's documentation saying to do this... 2.2.x kernels really
> > added some odd arping.
>
> If you run accross this reference, I'd appreciate a pointer. I thought I read
> everything.
>
> Thanks!
> sdw
>
> >
> >
> > Things work fine for me -- install the VIP aliases not on your loopback
> > device and not on your eth device, but on tunnel devices (and their aliases)
> > (even if you're not doing tunnelling -- you just need a different device...
> > tunl0 or ppp0... something besides lo or eth)
> >
> > All the best --
> > Ted
> >
> > ----- Original Message ----- <elided>
sdw
--
OptimaLogic - Finding Optimal Solutions Web/Crypto/OO/Unix/Comm/Video/DBMS
sdw@xxxxxxx Stephen D. Williams Senior Consultant/Architect http://sdw.st
43392 Wayside Cir,Ashburn,VA 20147-4622 703-724-0118W 703-995-0407Fax 5Jan1999
----------------------------------------------------------------------
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
To unsubscribe, e-mail: lvs-users-unsubscribe@xxxxxxxxxxxxxxxxxxxxxx
For additional commands, e-mail: lvs-users-help@xxxxxxxxxxxxxxxxxxxxxx
|
