LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

[lvs-users] LVS/DR Packet Loss Problem

To: LVS Maling List <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: [lvs-users] LVS/DR Packet Loss Problem
From: btinsley <btinsley@xxxxxxxxx>
Date: Mon, 24 Jan 2011 09:17:11 -0600
Hi list,

Has anyone experienced packet loss issues on real servers in an LVS/DR 
configuration? I don't believe this is an LVS issue per se, but I am 
curious if anyone else has experienced the same problem.

What I am seeing is packets are being forwarded from the director to the 
real server, but once they arrive at the real server there is packets 
loss such that it totally breaks communication with the client (at the 
TCP level). It seems to be fairly specific to loosing delayed ACKs. I 
see this reflected in the /proc/net/netstat statistics on the client 
(DelayedACKLost and TCPDSACKOldSent) and in a side-by-side comparison of 
tcpdump captures on the real server's receiving interface (bond1) and 
transmitting interface (bond0). The dummy0 interface is configured with 
the VIP on the real server and I have tried using the loopback interface 
with the same result. We have been using this same configuration for 
several years, so I am certain that there are no incorrect sysctl values 
(e.g., arp_announce, arp_ignore, etc) and there is no interference from 
netfilter.

This was first noticed on the SLES11 SP1 kernel, which is based on 
2.6.32. In the process of gathering information to open a case with the 
vendor, I found that the same problem occurrs on the kernel.org 
2.6.32.28 version. Further testing revealed that the issue was 
reproducible on all kernel.org versions (2.6.[32-36]) up to stable 
2.6.37, when it no longer happens. It also works on the SLES11 SP0 
kernel, which is 2.6.27 based.  I have looked through the git changelogs 
for the kernel across all versions and there do no seem to be any 
changes that would indicate why this is happening, but I don't claim to 
be an expert here. I have also searched the LVS mailing list archives in 
some depth and found nothing.

If anyone has had a similar experience, I would appreciate any 
information you may be able to share.


Thanks!!!



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>