LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] IMAP slow performance

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] IMAP slow performance
From: David Lang <david.lang@xxxxxxxxxxxxxxxxxx>
Date: Mon, 4 May 2009 08:30:33 -0700 (PDT)
On Mon, 4 May 2009, Dean Manners wrote:

> Hey guys,
>               We have an existing (2yrs running) setup that hasn't
> recently changed.  But are now, fairly suddenly, having problems with our
> IMAP servers experiencing very slow/unusable performance when connecting via
> the virtual IP.
>
>
> The setup:
> Two "directors" - one active, one standby.
> - Debian Sarge, 2.6.20.1 kernels.
> - Running Ultramonkey
> - Uptimes of:
>   director01:~# uptime
>    22:17:17 up 510 days
>   director02:~# uptime
>    22:17:20 up 238 days
>
> Two "mailget" servers - both active, direct routing with loopback addresses,
> running IMAP2 & POP3 services on the same virtual IP.
> - Running Dovecot
>
> All behind a Cisco 6500 switch, running aggregated firewall rules (eg the
> same one rule affects both real IP and virtual IP address).
>
>
> The symptoms/troubleshooting so far:
> ~ When IMAPing via the virtual IP IMAP responsiveness is very slow and
> frequently actions times out.
> ~ When IMAPing directly to the real servers there is no apparent performance
> issue, with speedy responses from both servers.
> ~ POP3 performance is normal, to same real servers via the same virtual IP
> as our troubled IMAP connections.
> * Tried restarting dovecot on both mailget servers (tyre kick measure)
> * Tried failing over the heartbeat/ultramonkey monitor on director01, so
> that director02 took over the vip's/LVS. There was still noticeable/sluggish
> performance when using the IMAP virtual IP from this 2nd machine though.  So
> that ruled out my director01 being a problem (was thinking the long uptime
> may have been at play).
> ~ Our webmail server however connects IMAP the same way via the load
> balanced address and appears to have no problems, large'ish mailboxes load
> almost instantly and all actions are responsive.  The only difference I can
> note, is that this server is in the same subnet as the directors/real
> servers.
> ~ IMAP connections represent about 15% of total mail connections, most use
> POP3:

how slow are you seeing?

my first reaction for something that works going to one IP on a box, but not to 
another on the same box/network/NIC is to look at name resolution. any chance 
that reverse DNS isn't setup for one IP?

David Lang

> # ipvsadm --list |grep mail
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
>  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
> TCP  mail.myserver.blah:pop3 wlc persistent 600
>  -> mailget01.myserver.blah:pop3 Route   1      54         144
>  -> mailget02.myserver.blah:pop3 Route   1      59         173
> TCP  mail.myserver.blah:imap2 wlc persistent 600
>  -> mailget01.myserver.blah:imap Route   1      12         1
>  -> mailget02.myserver.blah:imap Route   1      14         16
>
>
> The ldirectord.cf (note, has been the same for 2+ years):
>
>   # Global Directives
>   checktimeout=3
>   checkinterval=15
>   autoreload=yes
>   quiescent=no
>   logfile="/var/log/ldirectord.log"
>
>   # Main POP
>   virtual=xxx.xxx.xx.80:110
>          real=xxx.xxx.xx.81:110 gate
>          real=xxx.xxx.xx.82:110 gate
>          service=pop
>          scheduler=wlc
>          login="monitoring@xxxxxxxxxxxxxx"
>          passwd="blahblah"
>          persistent=600
>          protocol=tcp
>
>   # Main IMAP
>   virtual=xxx.xxx.xx.80:143
>          real=xxx.xxx.xx.81:143 gate
>          real=xxx.xxx.xx.82:143 gate
>          service=imap
>          scheduler=wlc
>          login="monitoring@xxxxxxxxxxxxxx"
>          passwd="blahblah"
>          persistent=600
>          protocol=tcp
>
>
> Im at a loss of what to try next, with POP3 and other services (HTTP/HTTPS)
> operating as normal from the same directors, no recent changes in configs,
> and two directors giving the same symptoms - I cant see where the problem
> lies.  Any tips on where to start looking ?
>
>
> Regards
> __________________________________________
> Dean Manners
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>