|
Hello,
On Thu, 12 Apr 2007, Rudd, Michael wrote:
> After some more digging it appears this is related to the OPS or One
> Packet Scheduling feature. With the OPS feature turned off, the source
> IP address is correctly SNATed to my VIP. With the OPS feature on and
> working correctly(which we need for our UDP service), the source IP
> address isn't correctly SNATed.
OPS is implemented for setups where there is no reply for the original
packet. DNS and RADIUS have reply, so they need something different
which is not done by OPS:
- every original packet should pass scheduling step
- reply (or replies) should go back properly, the hash connection
should keep the needed information (VIP, VPORT)
So, what you need is something different. OPS now works in this way:
- schedule original packet but don't hash the connection
- when reply packet is received it can not find connection and the
reply packet is treated as non-IPVS packet (real server receives ICMP
or packets goes further). That is why such replies don't have proper
VIP:VPORT if passed to output device.
> Is anybody aware of the code for this? I assume its related to not
> looking up the connection in the hash table anymore with OPS thus not
> SNATing. Maybe an iptables rule coudl fix this possibly?
You can use rule but may be you will not get the right VPORT every
time. Not sure why you need OPS. It was created when someone needed
to generate many requests from same CIP:CPORT with the assumption that
there are no replies. Only when many connections come from same CIP:CPORT
in the configured UDP time period the connection reusing does not
allow scheduling to be used for every packet. That is why OPS was needed
to schedule many packets (coming before expiration) from same
CIP:CPORT->VIP:VPORT to different real servers.
May be what you need from OPS is impossible: when OPS is not used
if reply is delayed IPVS will wait until the configured UDP timeout is
expired but this value can be different from the timeout your clients is
using. Difference in miliseconds can be fatal. What can happen is that a
different request from same CPORT will go to the same real server as long
as the UDP timeout is not expired. There can be different situations:
- clients can retransmit on some timeout (DNS, RADIUS)
- nobody is instructed how many requests should be passed (and the same
number of replies if such application mode is used) before removing
NAT connection explicitly before expiration to allow next request to be
scheduled to different real server.
So, the main problem is that it is not easy to balance single
CIP to many real servers if there are replies that can be delayed or
when requests can be retransmitted. There is no way IPVS to know when
to forget one connection to allow scheduling for the next packet from
same CIP:CPORT. So, if the client expects replies then OPS should not
be used. Instead, short UDP timeout should be used and one should be
ready single CIP:CPORT to be scheduled to same real server even if
many distinct (from application point of view) requests are sent
from same socket.
> Update has anyone seen this? I am basically seeing using LVS-NAT that
> the return packets are not being SNATed with the LVS directors VIP but
> have a source IP address of the realservers. I saw this work in the 2.4
> kernel but havent been able to make it work in 2.6. Any clues?
>
> Thanks
> Mike
Regards
--
Julian Anastasov <ja@xxxxxx>
|
