Re: Linux Cluster

To: " users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Linux Cluster
Cc: aaaa@xxxxxxxxxxxx
Cc: wesong@xxxxxxxxxxxx
From: Matthew Crocker <matthew@xxxxxxxxxxx>
Date: Tue, 20 May 2003 20:15:01 -0400

On Tuesday, May 20, 2003, at 06:42 PM, Alexandra Alvarado wrote:

Alexandra Alvarado wrote:


I'm Alexandra Alvarado from Ecuador, I need to implement a cluster for mail server, like this:

2 computers active - active for smtp server
2 computers active - active for pop3 server
2 computers (NAS) active - failover for storage

The idea is to have the information of the mails (/var/spool/mail and /var/spool/mqueue) in the
nas servers both with the same information (online replication).

Do not use mbox mailbox format. mbox does not work well over NFS with multiple hosts writing to the same file. You should use Maildir format. Just about every SMTP server can handle Maildir.

In the 2 Pop3 server I want to mount that NAS file system for pop3 read the mail information
and if one NAS is down the other comes up inmediatly.

Until now I'm using keepalived with lvs in proxy server and firewall but I'dont know how to
implement the mail server service. Can you help me please?

I'm running 4 mail servers. All mail servers have SMTP,POP3,IMAP, SMTPS,IMAP-SSL,POP3-SSL running on them. I'm using qmail, qmail-ldap, and Courier-IMAP.

I have a Network Appliance NetFiler F720 200gig NFS server for my Maildir storage
I have 3 OpenLDAP servers setup with 1 master and 2 slaves.
The Mail servers only talk with the slaves. All mailAddress information is held in the LDAP directory.

You need to centralize the storage of mail using NAS and centralized the storage of account information using either LDAP or MySQL

How do you plan on failing over the NAS?

NFS with soft mounts should handle it pretty well. Setup 2 Linux boxes one as an active, one as a passive NFS server. Both connected via Fiber Channel to a chunk of drives. The passive machine *must* have a way to STOMITH (Shoot The Other Machine In The Head) the active server if it crashes. You do not want to have both machines mounting the same drive space at the same time. Very bad things will happen....


Setup an EXT3 filesystem on the FC drives. mount it on the active Linux box, export it over NFS with a virtual IP address. If the active server fails you need to remove power from it using a remote power switch. You need to be able to guarantee that it won't come back to life and start writing to the filesystem again. Clean the FC filesystem. remount it and export it over NFS on the same virtual IP addresses. keepalived can handle the VIP stuff with VRRP. I think it can also launch an external script during a failover to handle the shooting, cleaning, mounting, exporting of the filesystem. EXT3 cleans pretty quickly.

The SMTP/POP3 servers will be very un-happy to see their NFS server disappear so you will need to recover quickly. Processes will probably pile up in 'D'isk wait status on all of the machines. load will go through the roof. After the NFS server comes back online the hung processes should recover and finish up.

Adaptec makes a very nice 12 disk rack mounted RAID controller that has U160 SCSI going to the disks and 1 gig FiberChannel going to the servers. Redundant power, Redundant RAID controllers, Redundant FC loops going to each server. It is called the DuraStor 7320S. Plan on about $10kUS + drives for this type of system.

Network Appliance make an amazing box with complete High Availability failover of clustered data. You can expect to pay $200KUS for a complete clustered solution with 300GB usable storage. Fully redundant. Pretty much shoot a shotgun at it and not go down or lose data.

EXT3 running on Logical Volume Manager (LVM) can handle journalling and snapshots

Making the servers/services redundant is easy. Making your NAS/SAN redundant is expensive.

I'm looking into the Adaptec external RAID controller/drive array setup with 2 Linux boxes for my NAS. I've been running my NetApp for 3 years and have *NEVER* had it crash. It really is an amazing box. The problem is it is only one box and I don't have $200k to make it a cluster. I think I can do a pretty good job for about $20k with the Adaptec box, a bunch of Seagate drives and a couple Linux boxes.

You could also look into distributed filesystems like GFC, Coda ... but I don't feel confident enough in them to handle production data just yet.


Alexandra Alvarado

_______________________________________________ mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to

<Prev in Thread] Current Thread [Next in Thread>
  • Linux Cluster, Alexandra Alvarado
    • Re: Linux Cluster, Matthew Crocker <=