LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Jabber Scalability and LVS

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Jabber Scalability and LVS
From: Clint Byrum <cbyrum@xxxxxxxxxxx>
Date: Wed, 02 Feb 2005 16:12:05 -0800
On Wed, 2005-02-02 at 15:25 -0800, Jon Phillips wrote:
> > Since any port is only able to make 2^16 connections and you want
> 2^20 simulateous
> > connections, then you'll need 2^4 realservers in your LVS to make
> this number of 
> > connections. So far so good.
> 

AFAIK, Jabber maintains a persistent connection, so you don't even need
persistence tracking on your LVS machine. The client connects to
serverip:jabberport from clientip:randomport, and that pretty much stays
constant until they logout or get disconnected. I would dispute that he
would need 2^4 servers to handle the port needs. He might need that many
because of the resources used (RAM, CPU, OS internal Scalability), but
the ports won't be the limiting factor, as the TCP/IP stack is going to
see each connection as unique because it will be

myip:myport:theirip:theirport

Since most ips (even large offices) won't connect more than 1 time per
jabber user, this should not be a problem.

> Cool...
> 
> > The next problem is what happens when the client writes (sends a
> message) to the 
> > jabber server which in this case would be a bunch of LVS
> realservers. In a normal
> > jabber setup (I expect) the client's message will be sent from
> server_IP:jabber_port
> > to the receiver who also has a connection to server_IP:jabber_port.
> In the case
> > of a bunch of realservers, the receiver will likely be connected to
> another realserver
> > and the realserver you're connected to won't have a connection to
> the receiver.
> 
> Exactly...we are limiting users of our jabber service to our jabber
> servers, 
> however, users will be spread throughout the world. So thus, they
> would connect 
> to the jabber server (a collection of LVS realservers) through the
> load server, 
> right? The jabberd2 server then has a db in mysql that maintains
> information 
> about who is logged in, and then all the users. Where would this data
> exist? 
> Would this be synced with the other servers? Or, would a large RAID be
> needed to 
> store the MYSQL db for all the LVS realservers to read and write to?
> This is 
> where my knowledge gets a little hazy of the subject?

MySQL 4.1 has a clustering component that provides a RAM-only read-write
multi-master database capability. Basically you'd run mysqld, and a
thing called "ndbd" on each jabber server, and they could all see the
same tables and data in a transaction safe manner. Its in RAM, so it is
expensive to run, but its pretty fast and its the only Free product that
provides reliable multi-master capabilities. It is somewhat limited when
compared to other engines inside MySQL, like MyISAM or InnoDB. You
should take a look at it on mysql.com.

http://www.mysql.com/products/cluster/

If you don't use that, you'll have to rely on a single mysql server, and
some sort of failover.

-- 
Clint Byrum <cbyrum@xxxxxxxxxxx>


<Prev in Thread] Current Thread [Next in Thread>