LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: PMTU-D: remember, your load balancer is broken (fwd)

To: Julian Anastasov <uli@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: PMTU-D: remember, your load balancer is broken (fwd)
Cc: Kyle Sparger <ksparger@xxxxxxxxxxxxxxxxxxxx>, lvs-users@xxxxxxxxxxxxxxxxxxxxxx
From: Wensong Zhang <wensong@xxxxxxxxxxxx>
Date: Fri, 16 Jun 2000 14:21:54 +0800 (CST)

On Thu, 15 Jun 2000, Julian Anastasov wrote:

> 
>       Hello,
> 
> On Thu, 15 Jun 2000, Wensong Zhang wrote:
> 
> > 
> > 
> > On Wed, 14 Jun 2000, Julian Anastasov wrote:
> > 
> > >   Yes, it is not handled. ip_fw_unmasq_icmp is not changed
> > > from LVS. But the problem occurs when external_MTU > internal_MTU
> > > in the Director which is not an usual case for LVS. The other case
> > > when the client has little MTU is handled. The result is:
> > > 
> > > - no problems for clients
> > > - the server works or don't works entirely. I think this
> > > could be visible. So, the problem is that the Director doesn't
> > > generate ICMP to the real servers. But the ICMP messages from
> > > clients are propagated to the real servers.
> > > 
> > 
> > For external_MTU < internal_MTU, LVS can handle this, because the
> > ICMP_FRAG_NEEDED message from clients or intermediate routers can be
> > forwarded to the right real server.
> 
>       Yes,  we  correctly forward  these ICMP  messages in
> both directions.
> 
> ext_MTU  < int_MTU is  detected only when  we forward packet
> from  NAT-ed real server to the  client.  It is handled from
> ip_forward.   We use ip_forward only  in VS/NAT. I agree, no
> problem  here. No packet mangling  before icmp_send, no need
> to restore packet.
> 
> > 
> > For external_MTU > internal_MTU, VS/DR cannot handle it, because we don't
> > do MTU checking before calling ip_send(); VS/NAT and VS/TUN currently can
> 
>       Yes,  in ip_vs_dr_xmit() we  need just to  add a mtu
> check  and to  reply with  icmp_send().  The  current status
> is: we don't generate ICMP messages.
> 
> > handle it. In VS/NAT, the mangled packet will go through ip_forward, which
> > has MTU checking. In VS/TUN, ip_vs_tunnel_xmit has MTU checking. Right?
> 
>       Right, VS/TUN is perfect.
> 
>       For  VS/NAT demasq  is broken.  We  must restore the
> original   packet  in  ip_fw_unmasq_icmp  when  called  from
> icmp_send  (called from ip_forward after the packet mangling
> and  after 2nd ip_route_input). (I hope nobody complains the

I am sorry that I forgot to restore the ip header of the mangled packet
before icmp_send, we have to hook ip_fw_unmasq_icmp to restore packet. You
are right.

> 576 data bytes are not restored).  There is no such thing in
> 2.3,  the packet restoring is a  big pain and it was removed
> but  that  doesn't  means  2.3  looks  correct  when sending
> ICMP_FRAG_NEEDED  from ip_forward(). We are  not sure if the
> packet  is  not already  changed  from the  PRE_ROUTING? The
> result:  rewritten iph->daddr (internal address) is returned
> in  the ICMP reply.  Is my interpretation  correct? I didn't
> tested it yet.
> 

Yeah, I agree with you. The netfilter probably have problems in calling
icmp_send for already-mangled packets. I think it is a need to restore ip
header of the packet before calling icmp_send.

>       For the masq direction it is handled from ip_forward
> before packet mangling. No problem here.
> 
> > 
> > BTW, I think it rarely happens. In VS/DR, the load balancer and real
> > servers are connected by an uninterrrupted physical network, such as
> > ethernet. The default MTU is 1500, which usually large than client MTU.
> 
>       Yes,  strange  setup  for director  :)  The external
> link with greater mtu than the internal :)
> 
> > 
> > >   Of course, this must be corrected in next versions.
> > > 
> > 
> > Sure!
> 
>       I think we have to make two changes:
> 
> - ip_vs_dr_xmit(): check mtu and reply
> 
> - ip_fw_unmasq_icmp(): we must put only our out_get() check,
> there  is  no  reason  for the  in_get()  check  (nothing is
> changed before ip_forward in the masq direction).
> 

Yup, we must do the two changes. However, for external_MTU < internal_MTU
in VS/NAT, the ICMP_FRAG_NEEDED will be sent before mangling the packets,
so there is no need to do in_get() check, but I am not sure whether other
ICMP messages will be sent to the real server after the packet is mangled.

Anyway, I don't think that it hurts if we add in_get checking in
ip_fw_unmasq_masq for VS/NAT.

Thanks,

Wensong


> > 
> > Thanks,
> > 
> > Wensong
> 
> 
> Regards
> 
> --
> Julian Anastasov <uli@xxxxxxxxxxxxxxxxxxxxxx>
> 
> 
> 



<Prev in Thread] Current Thread [Next in Thread>