|
Hello,
On 2012-01-07 02:02, Julian Anastasov wrote:
>
> Hello,
>
> On Fri, 6 Jan 2012, Dmitry Akindinov wrote:
>
>>> What about using 10 match rules in mangle/PREROUTING
>>> to mark every port range. Add 10 fwmark-based virtual servers
>>> containing single real server again in DR mode. There is a
>>> problem if this download traffic overloads the single
>>> IPVS box.
>>
>> We tried exactly that on a CentOS 6 install. It LOOKS like IPVS alreays
>> builds
>> a 'session' to relay tcp traffic, even if there is only 1 "real" server in
>> the
>> "virtual server set". And it LOOKS like it builds sessions only when it
>> receives the initial SYN packet. In our case, no SYN packet goes via the IPVS
>> server, the first packet for a new connection is SYN ACK. It looks like IPVS
>> does not build a session for it, and drops it and all other packets relayed
>> to
>> it due to lack of session.
>>
>> IF IPVS could work in a stateless mode, dispatching connection using any
>> algorithm based on packet source/destination (we do not care which one, as we
>> would have only one real server for each of these virtual servers) - then it
>> would work.
>
> For IPVS tcp_conn_schedule schedules TCP packets only
> when SYN is present, no matter the ACK flag. Can you check with
> IPVS debugging what exactly happens? May be the problem is netfilter
> stateful inspection where the synack combination results in
> sIV (invalid) packet state.
Let me describe the situation again. One of the servers in the farm,
server X, as a unique port range (16000-16999, for example), assigned to it.
That server starts an outgoing TCP connection using the VIP address as
the source, for that VIP address is configured locally as an IP alias of
the loopback interface.
This means that the initial packet - i.e. the SYN packet goes from the
server X to the remote target directly, bypassing an IPVS system. The
IPVS system will never see a SYN packet for an OUTGOING connection
initiated by one of the servers in the farm.
The response packets for such a connection would go to the VIP address,
and thus to the IPVS system. What we need to do there is to detect these
packets (pretty easy to do with iptables: all packets to a VIP, to any
port in the 16000-16999 range), and then forward these packets w/o any
modification to the server X MAC.
Our understanding is that the IPVS balancer can operate only in the
stateful mode, and that the state table is modified (a "session is
built") only when a SYN packet is processed (we do not know this, it's
just our understanding, but it looks like you are confirming it, right?).
In our case no SYN packet for an OUTGOING connection would ever be seen
by an IPVS system, so it does not build a session, so IPVS cannot be
used for "outoging respose balancing/dispatching".
Our question, thus, is - is it possible to configure IPVS to do
stateless load-balancing, given that there is only one real server to
"balance" to?
If not, then the question is - are there any plans to add stateless
load-balancing to IPVS?
>> Alternatively, if a Virtual server configured with just one real server would
>> redirect ALL packets to it, without building sessions - that would work, too.
>>
>>> Another option is to use ip rules with fwmark
>>> matching to forward the traffic. New kernels support the
>>> option to override the priority 0 for table local.
>>
>> How that can be configured?
>
> # Reduce priority for traffic to local routes
> ip rule add pref 10 lookup local
> ip rule del pref 0
We cannot make it work on the CentOS 6.0 system (2.6.32-71.el6.x86_64
kernel).
What are the kernels that have the ability to remove the rule #0?
Also, these rules need to be modified when, for example, a new server
is added to the farm. It may cause disruption to already established
connections if the rules have to be removed and then added one-by-one.
Managing all these things via IPVS would be a much cleaner way.
> # Add higher priority rules, all with same pref 1
> # Forward traffic for VIP:range to RIPs:
> ip rule add pref 1 from 0/0 iif<EXTIF> fwmark 1 lookup 1
> ip route add default via RIP1 dev<RIP_IF> src<DIP> table 1
> ip rule add pref 1 from 0/0 iif<EXTIF> fwmark 2 lookup 2
> ip route add default via RIP2 dev<RIP_IF> src<DIP> table 2
> ...
> ip rule add pref 1 from 0/0 iif<EXTIF> fwmark 10 lookup 10
> ip route add default via RIP10 dev<RIP_IF> src<DIP> table 10
>
> But the problem with stateful inspection is here too.
Why? AFAIU, ip rules are completely stateless and they do not care
what packet to forward, am I wrong?
>>> You can change the priority and to put your fwmark
>>> rules before the rule for table local. Add 10 routing
>>> tables containing single route to every real server.
>>> Not sure what happens with related ICMP in this case.
>>> That is why the virtual servers is a better option.
>>
>> Thank you very much!
>>
>> --
>> Best regards,
>> Dmitry Akindinov
>
> Regards
>
> --
> Julian Anastasov<ja@xxxxxx>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
--
Best regards,
Dmitry Akindinov
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
