|
> The test box is using the Windows Media Load Simulator. This just makes a
> lot of connections and streams back data. The average stream only gets up
> to about 35Mbit. At this point, CPU usage on the director is ~20% (which
> would seem to indicate that I should be able to get a lot more out of it).
> CPU on the test box is at about 25% and on the media server at 4%.
>
> The problematic part is that the director begins dropping about 10% of
> externally originated packets at this level of load. I wouldn't say any
> machine involved is stressed here, but pinging the external IP of the
> director gives that huge loss. This noticeably affects say, SSH, on the
> director or TS to the media server. This is constrasted with pinging the
> external IP of the test box, which gives 0% loss.
>
> I would therefore conclude that this is an issue with the
> director, but I'm
> not sure what. My next guess would be to try swapping the VIA NIC for
> another 3com one, but could it really be that bad? I can't see
> it being an
> issue with the cisco switch (test box and director are both connected to
> it); the cisco router (same), or the d-link switch (not involved
> in ping to
> director), so I'm at a loss as to what else to conclude.
>
I got fed up, and trundled my self down the the hosting centre to do some
further testing. It turns out that when plugging a couple of test clients
directly into the switch in front of the director, I can get 90Mbits
sustained load out of it using around 40% CPU on the director, nearly 100%
CPU on both test clients, 5% CPU on the WMS machine, and ~1.5k concurrent
connections. Fantastic!
The issue then, appears to be either the cisco switch or the cable
connecting it; there is nothing else left to test. I did swap the NICs out
for eepros (but the problem still persists when stressing through the cisco
kit). Safe to say, I'm very, very impressed by this!
In case anyone else wants to do the same kind of thing in future, the
LVS+iptables configuration is layed out below. Might be worth adding it to
the howto?
This information is not vastly easy to come across as I can't find a simple
published document which lists what WMS actually does. There is also some
attempt here at WMS9 support, but that's untested and is just based on what
the player tries to do (the player connects more quickly, however, if you
reject rather than drop those connection attempts, which I'm letting the
server do).
Thanks very much for the help,
Mark
# WMS: we want to group TCP 1755 and UDP 1024-500
# Also uses 554/tcp + 554/udp for WMS9.
# You might also want to add port 80 if serving up via http as well.
# To do this, set an fw mark on such connections, and use LVS fwmark
balancing (
# will forward matching IP+fwmark to the same server). Just what we need.
EXT_IP="1.2.3.4"
EXT_IF="eth0"
WMS_MARK="1"
RS1_IP="192.168.1.2"
RS2_IP="192.168.1.3"
# Allow appropriate ports in...
$IPTABLES -A INPUT -i $EXT_IF -p tcp -s 0/0 -d $EXT_IP --dport 1755 -j
ACCEPT
$IPTABLES -A INPUT -i $EXT_IF -p tcp -s 0/0 -d $EXT_IP --dport 554 -j ACCEPT
$IPTABLES -A INPUT -i $EXT_IF -p udp -s 0/0 -d $EXT_IP --dport 554 -j ACCEPT
$IPTABLES -A INPUT -i $EXT_IF -p udp -s 0/0 -d $EXT_IP --dport 1024:5000 -j
ACCEPT
# Group with fwmark...
$IPTABLES -t mangle -A PREROUTING -i $EXT_IF -p tcp -s 0/0 -d
$EXT_IP --dport
1755 -j MARK --set-mark $WMS_MARK
$IPTABLES -t mangle -A PREROUTING -i $EXT_IF -p tcp -s 0/0 -d
$EXT_IP --dport
554 -j MARK --set-mark $WMS_MARK
$IPTABLES -t mangle -A PREROUTING -i $EXT_IF -p udp -s 0/0 -d
$EXT_IP --dport
554 -j MARK --set-mark $WMS_MARK
$IPTABLES -t mangle -A PREROUTING -i $EXT_IF -p udp -s 0/0 -d
$EXT_IP --dport
1024:5000 -j MARK --set-mark $WMS_MARK
# Tell LVS to do the load balancing
$IPVSADM -D -f $WMS_MARK
$IPVSADM -A -f $WMS_MARK -s rr -p 600
$IPVSADM -a -f $WMS_MARK -r $RS1_IP:0 -m
$IPVSADM -a -f $WMS_MARK -r $RS1_2P:0 -m
|
