|
On Thu, Nov 24, 2011 at 02:10:43PM +1300, Roger Littin wrote:
> Hi Simon,
>
> I didn't want to hijack the other thread so I have started a new one.
Good thinking.
> The issue I was seeing was with tcp packets to media servers behind LVS.
> When the media streams are going out to the clients, everything is fine
> as the packets going through LVS are tiny.
>
> When I try to send incoming streams through LVS, I start seeing issues
> sometimes where an icmp is sent back to the client stating the packet is
> too large to forward to the real server. This causes the incoming stream
> to get further and further behind.
Are the ICMP sent by the linux-director (machine running IPVS)?
If you run tcpdump on the interface on the linux-director that
receives packets from clients does it report larger than MTU packets?
> Originally, on all the servers, I had mtu set to 1500 and tcpdump would
> be showing some packets that were larger than this and would be rejected.
> I increased the mtu to 9000 but then tcpdump simply reported that the
> same clients were sending packets larger than 9000 bytes.
>
> The last tests I did was on a server that I no longer have access to so I
> cannot be sure of the settings then.
>
> My current servers are running Centos 5.7 2.6.18-274.7.1.el5 and have GRO
> off. LRO is not listed.
That most likely means LRO isn't supported, so shouldn't be a problem.
Though it may be a case that ethtool is a bit old. Would it be possible
to try again with a newer ethtool?
> [root@laxlb01 ~]# ethtool -k eth0
> Offload parameters for eth0:
> Cannot get device udp large send offload settings: Operation not supported
> rx-checksumming: on
> tx-checksumming: on
> scatter-gather: on
> tcp segmentation offload: on
> udp fragmentation offload: off
> generic segmentation offload: off
> generic-receive-offload: off
>
> I see that tcp segmentation offload is set to on. Will this cause the same
> effect?
TCP segmentation offload is on the transmit side and I don't believe
it has an adverse affect on IPVS.
The trouble with LRO and GRO is that it the aggregation occurs before IPVS
receives the SKB and IPVS then tries to transmit the over-sized SKB.
The solution for GRO was to re-segment the SKB before transmit.
A solution for LRO would likely use similar logic.
> I would really like to get to the bottom of what is causing this problem
> so that I can run the incoming streams through the LVS. At the moment,
> we are balancing these with DNS which is not ideal.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
