|
On Wed, 2 Aug 2006, Joseph T. Duncan wrote:
> Hi,
>
> first off this project is awsome! Its saveing my almost nonexistant budget
> to accomplish an neato feat! Thank you very much for your hard work, and
> if I can offer anything/contribute in any way I will.
>
> I am using LVS to balance 3 windows terminal servers. These servers
> provide a "virtual computer lab" to the students here at the univeristy I
> work for.
>
> so here is the setup(LVS-DR):
>
> _________
> | |
> | clients |
> |_________|
> |
> router/firewall (catalyist 6500) gw ip = 128.193.85.1
> |
> __________ |
> | |---| VIP/RIP = 128.193.85.17, eth1, arps, gig link
> | director | | RIP = 128.193.85.16, eth0, arps, 100meg link,
> |__________|---| used for collecting metircs from realservers
> | sence anything to 85.17 is blackholed
> |
> -------....
> |
> |
> _____________
> | | RIP = 128.193.85.(n+17), eth0, arps, gig link
> | realserverN | VIP = 128.193.85.17, lo, no-arp, (m$ loop back)
> |_____________|
>
>
> -fire wall rules on the cataylist-
> all traffic in 128.193.85.1/24 has no firewall restrictions
> all outbound traffic, allowed out
> 128.193.85.16: ssh,www trafic from outside allowed in
> 128.193.85.(17, 17+N): 3389 (rdp) from outside allowed in
> 128.193.85.(17+N): m$ ports from campus (128.193.*.*) allowed in
>
>
> the director is a Debian (stable) box running a custome 2.6.17.7 kernel
>
> -/etc/ipvsadm.rules-
> # ipvsadm.rules
> -A -t 128.193.85.17:3389 -s wlc -p 360
> -a -t 128.193.85.17:3389 -r 128.193.85.18:3389 -g -w 1
> -a -t 128.193.85.17:3389 -r 128.193.85.19:3389 -g -w 1
> -a -t 128.193.85.17:3389 -r 128.193.85.20:3389 -g -w 1
>
>
> I adjust the weights with a cronjob that checks realservers cpu, memory
> loads via snmp...
>
>
> -----
> So far all of this works great. Here is the issue.
>
> A client connects to 128.193.85.17:3389 and gets ballanced correctly
> however traffic is funny...
>
> for the first packet:
> data -> eth1 -(ipvs ballancer)- eth1 -> real server
>
> subseqent packets would then do the following:
> data -> eth0 -(ipvs ballancer)- eth0 -> real server
>
> clients dont really notice anything, this all happens transperent to
> them.. it just happens that eth0 is willing to accept packets for any
> physical interface on the box... but is not exactly what i wanted to have
> happen.
>
> after pokeing around a bit, did the following:
> (mainly following the directions at this site,
> http://www.linuxjournal.com/article/7291)
>
> set:
>
> /proc/sys/net/ipv4/all/arp_filter = 1
> /proc/sys/net/ipv4/eth0/arp_filter = 1
Yes, This fixes the "data -> eth1 -(ipvs ballancer)" part.
> /proc/sys/net/ipv4/eth1/arp_filter = 1
>
> del original default routes
> add custom default routes with iproute2:
>
> #create routes
> /bin/ip route add default via 128.193.85.1 dev eth1 table 1
> /bin/ip route add default via 128.193.85.1 dev eth0 table 2
>
> #create rules
> /bin/ip rule add from 128.193.85.17/32 table 1 priority 500
> /bin/ip rule add from 128.193.85.16/32 table 2 priority 600
I see... It's been a while but I think this should work.. IF the 'ipvs
packets' get routed like 'normal' packets. Look for recent threads
"LVS-NAT + SNAT is it impossible" and "LVS-NAT and policy routing" I think
the described issues and fixes (might) apply in this case too.
Regards,
Mark.
> #tell kernel to parse new rules
> /bin/ip route flush cache
>
> and now i get the following behavour (closer to what i want)
>
> for the 1 to 1024 packet(s):
> data -> eth1 -(ipvs ballancer)- eth1 -> real server
>
> subseqent packets would then do the following:
> data -> eth1 -(ipvs ballancer)- eth0 -> real server
>
> --
>
> so anyone have ideas on how to keep all trafic entering eth1 to stay and
> exit eth1? Is this a question for some other group? sence its not directly
> related to lvs?
>
> if I swap eth1 and eth0's ip/roles then all of the metric traffic
> (keepalive, and snmp gets to real servers) becomes blackholed, sence reply
> path is .17 and every computer has a .17 address...
>
>
> Thanks for any and all help!
> Joseph
>
> _________________________________________________________________________
> Info: Email:
> Joseph T. Duncan duncan@xxxxxxxxxxxxx
> 109 Kidder hall
> Oregon State Univeristy
> Corvallis Or 97331
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
|
