|
Gary Smith wrote:
>> All,
>>
>>
>> Lets say hypothetically I have a director with two vips. The vips
>> represent different services, different areas of responsibility, etc.
(snipped explanation of two realservers communicating via their VIPs)
> Use different subnets for the different classes of real servers.
>
> Ex:
>
> Data rail: 10.0.1.0/24
> Web rail: 10.0.2.0/24
>
> Data server:
> * IP 10.0.1.2/24
>
> Web server:
> * IP 10.0.2.2/24
>
> Director:
> * IP 10.0.1.1/24
> * IP 10.0.2.1/24
> * VIP 10.0.3.10/24 Data
> * VIP 10.0.3.11/24 Web (or the public IP if ipvs is the firewall as
> well)
>
> ipvsadm -A t 10.0.3.10:3306 -s wlc
> ipvsadm -a t 10.0.3.10:3306 -r 10.0.1.2:3306 -m -we 100
> ipvsadm -A t 10.0.3.11:80 -s wlc
> ipvsadm -a t 10.0.3.11:80 -r 10.0.2.2:80 -m -we 100
>
>
> no need to nat/snat at this point.
>
>
>
>> So, does my question make sense? I would like realservers for one vip
>> to make connections to the vip of another virtual server on the same
>> director. Anyone know how?
>>
Ok - so I've had a look at this solution, and tried something like
this. It may be that the solution could be made to work but it depends
on having a different internal 'routing' vip per set of servers. For
one or three sets this would probably be ok.
I'm trying to develop a commercial offering based on lvs in which
different customers are assigned one or more realservers behind the
director. I can't predict ahead of time what customers will buy, and I
can't renumber on a whim (though to some extent the private IPs behind
the NAT are somewhat hidden...). The realservers are actually virtual
servers on fairly beefy hardware and I can end up with many many
(virtual) realservers and many 10s of vips.
It turns out the performance is acceptable for such a solution according
to the load tests we've done, but this vip-to-vip communication is
somewhat of a deal-breaker for us. I've got a few ugly workarounds, but
nothing I'd like to put real services on. Since I dont have direct
control over the customer's use of the services, and the customers may
want to communicate with each other and may not even be aware that
whoever they're sending email to, etc are on another one of our
realservers they rightly expect that they should just be able to reach
the IP of someone else on the internet...
The problem appears to be outbound LVS routing. It doesn't follow the
normal iptables paths as far as I can tell.
Let me see if I can give you some more detail:
I have two real servers A and B. A has a VIP of 200.0.0.2 and a private
IP of 10.0.0.2, B has 200.0.0.3 and 10.0.0.3. The director has a
private vip of 10.0.0.1 used for outbound routing.
If A tries to connect to B, packets arrive at the directory with a
source IP of 10.0.0.2 and a dest IP of 200.0.0.3. LVS changes that
destination to 10.0.0.3. At this point using symmetric dnat/snat rules,
then iptables fires the snat/postrouting rules and changes the source
IP, but LVS doesn't dot this. The packet is then dropped back out on
the wire on the private interface (vlan in this case...). B gets a
packet addressed to 10.0.0.3 from 10.0.0.2 (which should be 200.0.0.2),
responds, and A sees the response hit the wire because its on the same
lan, the director does nothing with the packet because it thinks it
doesn't need to.
At any rate, I've tried many many configurations and iptables rulesets
to try and get this to work right. One thing that might affect this is
that I'm using one physical lan and a couple of vlans for this
configuration. If I used two physical interfaces (one for public, one
for private) perhaps this would work.
In short, do you have any other suggestions I might try?
Fred
This email message is intended for the use of the person to whom it has been
sent, and may contain information that is confidential or legally protected. If
you are not the intended recipient or have received this message in error, you
are not authorized to copy, distribute, or otherwise use this message or its
attachments. Please notify the sender immediately by return e-mail and
permanently delete this message and any attachments. Verio, Inc. makes no
warranty that this email is error or virus free. Thank you.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
