|
Hello,
On Thu, 24 Aug 2017, Owain Jones wrote:
> I've tried "watch ipvsadm -Lnc" as it got stuck and everything looks normal.
> An "ESTABLISHED" TCP connection from the router to port 21 of the virtual and
> then routed to the destination of port 21 on the FTP server. And the same
> thing with another TCP connection but with a port in my passive port range.
> What I'd expect to see.
>
> I did try using tshark on the director, filtering for the VIP and ports 20000
> to 21000 (my passive port range). When it gets stuck, what I'm seeing is:
>
> 192.68.0.1 [router IP] -> 192.168.0.99 [VIP] TCP 85 [TCP Retransmission] ->
> 20103 [ FIN, PSH, ACK ]
>
> And this is repeating over and over.
>
> So what I'm interpreting this as being is the client signalling to the VIP
> that it wants to close the connection (FIN) and, being TCP, there should be an
> ACKnowledgement from the server that it received this. But as that ACK ain't
> coming, then it's repeatedly retransmitting the passive port "close
> connection" to a server that isn't responding.
>
> And, looking on the FTP server, when it's stuck, I'm seeing this repeating
> over and over:
>
> 192.168.0.99 (VIP) -> 192.168.0.1 (router) TCP 85 [TCP Retransmission] 20197
> -> 51311 [ FIN, PSH, ACK ]
>
> So the server is getting it and trying to respond, but it's not getting back
> to the client.
>
> Hmm, on the realserver, I've got a dummy interface with the VIP and, to combat
> the ARP problem, have used the following rules in my /etc/network/interfaces:
>
> ---- 8< ----
>
> auto dummy0
> iface dummy0 inet static
> address 192.168.0.99
> netmask 255.255.255.0
> pre-up arptables -F
> pre-up arptables -A INPUT -d 192.168.0.99 -j DROP
> pre-up arptables -A OUTPUT -s 192.168.0.99 -j mangle
> --mangle-ip-s [external IP address]
>
> ---- >8 ----
>
> So that any ARP requests for the VIP are ignored by the realserver and if it
> ARPs the VIP itself then it's changed to the external IP address to redirect
> it to the right entry point into the cluster.
>
> I preferred to use "arptables" to explicitly control the ARP problem. It just
> felt less messy to do it that way to me.
>
> Looking at these results, I think LVS is actually delivering the close
> connection to the FTP server but the response is not getting back to the
> client. So my configuration of "dummy0" that carries the VIP is possibly wrong
> somehow?
ARP filtering settings should be ok. They should not be a
reason connection to stop every time just before the end of file.
May be you can check for checksum errors, the -v in tcpdump -lnnnv
should report if real server receives corrupted packets from
real server. Sometimes bad-csum is reported for transmitted
packets, eg. when their checksum is to-be-calculated by chip.
This should be ignored. So, just make sure the packets received in
director (by client) and real server (by director) have valid csum.
You can also check if there is conntrack entry in
the real server, its state. I don't expect transfer to stop
just because there is a FIN bit, the retransmitted packets
still have the chance to fill the gaps.
Regards
--
Julian Anastasov <ja@xxxxxx>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
