LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Jabber Scalability and LVS

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Jabber Scalability and LVS
From: Joseph Mack <mack.joseph@xxxxxxx>
Date: Wed, 02 Feb 2005 20:19:05 -0500
Jon Phillips wrote:
> 

> > The next problem is what happens when the client writes (sends a message) 
> > to the
> > jabber server which in this case would be a bunch of LVS realservers. In a 
> > normal
> > jabber setup (I expect) the client's message will be sent from 
> > server_IP:jabber_port
> > to the receiver who also has a connection to server_IP:jabber_port. In the 
> > case
> > of a bunch of realservers, the receiver will likely be connected to another 
> > realserver
> > and the realserver you're connected to won't have a connection to the 
> > receiver.
> 
> Exactly...we are limiting users of our jabber service to our jabber servers,
> however, users will be spread throughout the world. So thus, they would 
> connect
> to the jabber server (a collection of LVS realservers) through the load 
> server,
> right? 

yes, but remember the LVS director is just a router with slightly unusual rules.
It just assigns the connection to the next realserver in its list. After that
the client is connected to one of the jabber realservers and doesn't know
that the director is there.

> The jabberd2 server then has a db in mysql that maintains information
> about who is logged in, and then all the users. Where would this data exist?

As Clint says, you can replicate this information between realservers with 
mysql.
Whether it would work as you'd hope isn't known. You may have to talk to the 
jabber
developers to make sure the info can be replicated. jabberd on realserver_2 may
not be happy if new entries appear on its database storage on disk being written
by replication from jabberd on realserver_1, without it being told first (ie 
it's
memory cached tables being updated first). If this could be got to work then 
user_1 would connect at realserver_1 and after replication, realservers_2..n 
would know that user_1 was connected. 

> Would this be synced with the other servers? Or, would a large RAID be needed 
> to
> store the MYSQL db for all the LVS realservers to read and write to? 

this could be a first step to make life a little easier for you, so you don't 
have
to climb all hills at once. There are problems with this approach (see the HOWTO
on databases). Usually a dbd thinks it owns the disk and will not be happy if a 
dbd
on another realserver is writting independantly to the same stored database. 
This is a different problem to a single dbd running on a single machine making 
multiple writes to the same record when the single dbd can order the writes 
according to the rules of db'ing. So the problem is making sure that the dbds
on each machine are cooperating. This is not an easy problem.

Assuming this is all solved, the next problem is that the tcpip connections
are split. user_1 is connected to realserver_1 and sends a message to user_2 who
just happens to be on realserver_2. The jabberd on realserver_1 expects to send 
a packet to user_2 connected to realserver_1 (remember in an LVS, the 
realservers 
don't know that any other realservers exist). The jabberd on realserver_1 knows 
(because of our assumption above) that user_2 is on realserver_2 and will have 
to 
pass the message to jabberd on realserver_2, ie the two jabberds will have 
to cooperate. This will have to be written into jabberd. LVS only acts to
assign the connection from each user to a realserver - after that LVS has
no role in the multiserver jabber setup. 

LVS is designed to run in a read only environment (eg webpages). If any writing
requires passing information to other realservers, then this has to be done
outside LVS. I would expect for jabber that the jabberd will have to be 
rewritten
so that they can cooperate. dhcpd does this.

Doesn't irc have a layered part to its protocol so that messages can be passed
to users on remote servers? If so why doesn't this work for you?

Joe

-- 
Joseph Mack PhD, High Performance Computing & Scientific Visualization
LMIT, Supporting the EPA Research Triangle Park, NC 919-541-0007
Federal Contact - John B. Smith 919-541-1087 - smith.johnb@xxxxxxx

<Prev in Thread] Current Thread [Next in Thread>