|
This can be ignored - found the problem, and has nothing to do with LVS. A
very very obscure issue with a deep packet inspection/QoS device in our
network. Way to waste 24 hours *sigh*
> Hey guys,
> We have an existing (2yrs running) setup that hasn't
> recently changed. But are now, fairly suddenly, having problems with our
> IMAP servers experiencing very slow/unusable performance when connecting
via
> the virtual IP.
>
>
> The setup:
> Two "directors" - one active, one standby.
> - Debian Sarge, 2.6.20.1 kernels.
> - Running Ultramonkey
> - Uptimes of:
> director01:~# uptime
> 22:17:17 up 510 days
> director02:~# uptime
> 22:17:20 up 238 days
>
> Two "mailget" servers - both active, direct routing with loopback
addresses,
> running IMAP2 & POP3 services on the same virtual IP.
> - Running Dovecot
>
> All behind a Cisco 6500 switch, running aggregated firewall rules (eg the
> same one rule affects both real IP and virtual IP address).
>
>
> The symptoms/troubleshooting so far:
> ~ When IMAPing via the virtual IP IMAP responsiveness is very slow and
> frequently actions times out.
> ~ When IMAPing directly to the real servers there is no apparent
performance
> issue, with speedy responses from both servers.
> ~ POP3 performance is normal, to same real servers via the same virtual IP
> as our troubled IMAP connections.
> * Tried restarting dovecot on both mailget servers (tyre kick measure)
> * Tried failing over the heartbeat/ultramonkey monitor on director01, so
> that director02 took over the vip's/LVS. There was still
noticeable/sluggish
> performance when using the IMAP virtual IP from this 2nd machine though.
So
> that ruled out my director01 being a problem (was thinking the long uptime
> may have been at play).
> ~ Our webmail server however connects IMAP the same way via the load
> balanced address and appears to have no problems, large'ish mailboxes load
> almost instantly and all actions are responsive. The only difference I
can
> note, is that this server is in the same subnet as the directors/real
> servers.
> ~ IMAP connections represent about 15% of total mail connections, most use
> POP3:
>
> # ipvsadm --list |grep mail
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP mail.myserver.blah:pop3 wlc persistent 600
> -> mailget01.myserver.blah:pop3 Route 1 54 144
> -> mailget02.myserver.blah:pop3 Route 1 59 173
> TCP mail.myserver.blah:imap2 wlc persistent 600
> -> mailget01.myserver.blah:imap Route 1 12 1
> -> mailget02.myserver.blah:imap Route 1 14 16
>
>
> The ldirectord.cf (note, has been the same for 2+ years):
>
> # Global Directives
> checktimeout=3
> checkinterval=15
> autoreload=yes
> quiescent=no
> logfile="/var/log/ldirectord.log"
>
> # Main POP
> virtual=xxx.xxx.xx.80:110
> real=xxx.xxx.xx.81:110 gate
> real=xxx.xxx.xx.82:110 gate
> service=pop
> scheduler=wlc
> login="monitoring@xxxxxxxxxxxxxx"
> passwd="blahblah"
> persistent=600
> protocol=tcp
>
> # Main IMAP
> virtual=xxx.xxx.xx.80:143
> real=xxx.xxx.xx.81:143 gate
> real=xxx.xxx.xx.82:143 gate
> service=imap
> scheduler=wlc
> login="monitoring@xxxxxxxxxxxxxx"
> passwd="blahblah"
> persistent=600
> protocol=tcp
>
>
> Im at a loss of what to try next, with POP3 and other services
(HTTP/HTTPS)
> operating as normal from the same directors, no recent changes in configs,
> and two directors giving the same symptoms - I cant see where the problem
> lies. Any tips on where to start looking ?
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
