Hey guys,
We have an existing (2yrs running) setup that hasn't
recently changed. But are now, fairly suddenly, having problems with our
IMAP servers experiencing very slow/unusable performance when connecting via
the virtual IP.
The setup:
Two "directors" - one active, one standby.
- Debian Sarge, 2.6.20.1 kernels.
- Running Ultramonkey
- Uptimes of:
director01:~# uptime
22:17:17 up 510 days
director02:~# uptime
22:17:20 up 238 days
Two "mailget" servers - both active, direct routing with loopback addresses,
running IMAP2 & POP3 services on the same virtual IP.
- Running Dovecot
All behind a Cisco 6500 switch, running aggregated firewall rules (eg the
same one rule affects both real IP and virtual IP address).
The symptoms/troubleshooting so far:
~ When IMAPing via the virtual IP IMAP responsiveness is very slow and
frequently actions times out.
~ When IMAPing directly to the real servers there is no apparent performance
issue, with speedy responses from both servers.
~ POP3 performance is normal, to same real servers via the same virtual IP
as our troubled IMAP connections.
* Tried restarting dovecot on both mailget servers (tyre kick measure)
* Tried failing over the heartbeat/ultramonkey monitor on director01, so
that director02 took over the vip's/LVS. There was still noticeable/sluggish
performance when using the IMAP virtual IP from this 2nd machine though. So
that ruled out my director01 being a problem (was thinking the long uptime
may have been at play).
~ Our webmail server however connects IMAP the same way via the load
balanced address and appears to have no problems, large'ish mailboxes load
almost instantly and all actions are responsive. The only difference I can
note, is that this server is in the same subnet as the directors/real
servers.
~ IMAP connections represent about 15% of total mail connections, most use
POP3:
# ipvsadm --list |grep mail
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP mail.myserver.blah:pop3 wlc persistent 600
-> mailget01.myserver.blah:pop3 Route 1 54 144
-> mailget02.myserver.blah:pop3 Route 1 59 173
TCP mail.myserver.blah:imap2 wlc persistent 600
-> mailget01.myserver.blah:imap Route 1 12 1
-> mailget02.myserver.blah:imap Route 1 14 16
The ldirectord.cf (note, has been the same for 2+ years):
# Global Directives
checktimeout=3
checkinterval=15
autoreload=yes
quiescent=no
logfile="/var/log/ldirectord.log"
# Main POP
virtual=xxx.xxx.xx.80:110
real=xxx.xxx.xx.81:110 gate
real=xxx.xxx.xx.82:110 gate
service=pop
scheduler=wlc
login="monitoring@xxxxxxxxxxxxxx"
passwd="blahblah"
persistent=600
protocol=tcp
# Main IMAP
virtual=xxx.xxx.xx.80:143
real=xxx.xxx.xx.81:143 gate
real=xxx.xxx.xx.82:143 gate
service=imap
scheduler=wlc
login="monitoring@xxxxxxxxxxxxxx"
passwd="blahblah"
persistent=600
protocol=tcp
Im at a loss of what to try next, with POP3 and other services (HTTP/HTTPS)
operating as normal from the same directors, no recent changes in configs,
and two directors giving the same symptoms - I cant see where the problem
lies. Any tips on where to start looking ?
Regards
__________________________________________
Dean Manners
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|