|
Michael Sparks wrote:
>
> Hi,
>
> Some of the emerging redirection switches on the
> market support something known as Level 7 redirection
> which essentially allows the redirector to look at the
> start of the TCP data stream, by spoofing the initial
> connection and making load balancing based on what it
> sees there. (Apologies if I'm doing the equivalent of
> "teaching your grandma to suck eggs", but at least this way
> there's less mis-understanding of what I'm getting at/after)
>
> For example if we have X proxy-cache servers, we could
> spoof an HTTP connection, grab the requested URL, and
> hash it to one of those X servers. If it was possible
> to look inside individual UDP packets as well, then we
> would be able to route ICP (inter cache prototol) packets
> in the same way. The result is a cluster that looks like
> a single server to clients.
>
Do you mean that these X proxy-cache servers are not
equivalent, and they are statically partitioned to fetch
different objects? for example, the proxy server 1 is
to cache European URLs, and the proxy server 2 is to
cache Asian URLs, then there is a need to parse the
packets to grab the requested URL. Right?
If you want to do this, I think Apache's mod_rewrite
and mod_proxy can be used to group these X proxy-cache
servers as a single proxy server. Since the overhead
of dispatching requests in application level is high,
its scalability may not be very good, the load balancer
might be a bottleneck when there are 4 proxy servers
or more.
The other way is to copy data packet to userspace to
grap the request if the request is in a single UDP packet,
the userspace program select a server based on the request
and pass it back to the kernel.
> For generic HTTP servers this could also allow the server
> to farm cgi-requests to individual machines, and the normal
> requests to the rest. (eg allowing you to buy a dual/quad
> processor to handle soley cgi-requests, but cheap/fast
> servers for the main donkey work.)
>
It is statically partitioned. Not very flexible and scalable.
> We've spoken to a number of commercial vendors in the past
> who are developing these things but they've always failed
> to come up with the goods, mainly citing incompatibility
> between our needs and their designs :-/
>
> Any ideas how complicated this would be to add to your
> system?
>
If these proxy-cache servers are identical(they are the same
of all kind of URL requests), I have a good solution to
use LVS to build a high-performance proxy server.
request |-<--fetch objects directly
| |-----Squid 1---->reply users directly
|->LinuxDirector ---|
(VS-TUN/VS-DR) |-----Squid 2
| ...
|-----Squid i
Since VS-TUN/VS-DR is on client-to-server half connection,
squid servers can fetch object directly from the Internet
and return objects directly to users. The overheading of
forwarding request is low, scalabilty is very good.
The ICP is used to query among these Squid servers. In order
to avoid the mulitcasting storm, we can add one more NIC in
each squid server for ICP query, we can call it multicast
channel.
Wensong
> Michael.
> --
|
