|
Hello!
I try to do load-balance the service which is listening both TCP and UDP
using same port number *1111*. There are set of real servers, each of them
host few application instances listening on different ports (1111/udp,
1116/udp, 1115/tcp, 1117/tcp)
I used fwmark to group connection by protocol, i.e. 1111/udp have *fwmark 1*,
1111/tcp have *fwmark 2.*
LVS-NAT mode is in use.
What I see is when the client connection coming to 1111/*TCP* the LVS try
to pass it to RTI:1111/*TCP *while the RIP have TCP listeners on ports 1115
and 1117 and LVS was configured accordingly.
Here is test connection info:
# ipvsadm -L -n -c
IPVS connection entries
pro expire state source virtual destination
*TCP* 00:55 SYN_RECV 113.25.79.34:38204 115.5.6.138*:1111 *10.1.2.13
*:1111* *<<<< It should be RIP:1115 or RIP:1117 HERE!*
IP 00:49 NONE 113.25.79.34:0 0.0.0.2:0 10.1.2.13:0
TCP 01:00 SYN_RECV 113.25.79.34:38206 115.5.6.138:1111 10.1.2.13:1111
Below are saved firewall and ipvs configurations:
# Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018
*mangle
:PREROUTING ACCEPT [9507:869411]
:INPUT ACCEPT [7637:700414]
:FORWARD ACCEPT [70:10945]
:OUTPUT ACCEPT [8781:1045717]
:POSTROUTING ACCEPT [8851:1056662]
COMMIT
# Completed on Tue Dec 11 17:02:29 2018
# Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [82:11111]
:OUTPUT ACCEPT [16923:2797363]
-A INPUT -p icmp -m comment --comment "000 accept all icmp" -j ACCEPT
-A INPUT -i lo -m comment --comment "001 accept all to lo interface" -j
ACCEPT
-A INPUT -d 127.0.0.0/8 ! -i lo -m comment --comment "002 reject local
traffic not on loopback interface" -j REJECT --reject-with
icmp-port-unreachable
-A INPUT -m comment --comment "003 accept related established rules" -m
state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m multiport --dports 22 -m comment --comment "100 accept
ssh connection" -m state --state NEW -j ACCEPT
-A INPUT -d 224.0.0.0/8 -p vrrp -m comment --comment "104 Accept VRRP
proto" -j ACCEPT
-A INPUT -m comment --comment "999 drop all" -j REJECT --reject-with
icmp-port-unreachable
COMMIT
# Completed on Tue Dec 11 17:02:29 2018
# Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018
*nat
:PREROUTING ACCEPT [42:2928]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [4:360]
:POSTROUTING ACCEPT [4:360]
-A PREROUTING -p tcp -m multiport --dports 1111 -j MARK --set-xmark
0x2/0xffffffff
-A PREROUTING -p *udp* -m multiport --dports 1111 -m comment --comment "101
MARK udp 1111 traffic from everywhere" -j MARK --set-xmark *0x1*/0xffffffff
-A PREROUTING -p *tcp* -m multiport --dports 1111 -m comment --comment "101
MARK tcp 1111 traffic from everywhere" -j MARK --set-xmark *0x2*/0xffffffff
COMMIT
# Completed on Tue Dec 11 17:02:29 2018
# Generated by iptables-save v1.4.21 on Tue Dec 11 17:02:29 2018
*raw
:PREROUTING ACCEPT [17569:3914462]
:OUTPUT ACCEPT [16925:2798776]
COMMIT
# ipvsadm -S
-A -f 1 -s wrr -p 50
-a -f 1 -r 10.1.2.12:1111 -m -w 2
-a -f 1 -r 10.1.2.12:1116 -m -w 2
-a -f 1 -r 10.1.2.13:1111 -m -w 2
-a -f 1 -r 10.1.2.13:1116 -m -w 2
-a -f 1 -r 10.1.2.14:1111 -m -w 2
-a -f 1 -r 10.1.2.14:1116 -m -w 2
-A -f 2 -s wrr -p 50
-a -f 2 -r 10.1.2.12:1115 -m -w 2
-a -f 2 -r 10.1.2.12:1117 -m -w 2
-a -f 2 -r 10.1.2.13:1115 -m -w 2
-a -f 2 -r 10.1.2.13:1117 -m -w 2
-a -f 2 -r 10.1.2.14:1115 -m -w 2
-a -f 2 -r 10.1.2.14:1117 -m -w 2
# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 1 wrr persistent 50
-> 10.1.2.12:1111 Masq 2 0 0
-> 10.1.2.12:1116 Masq 2 0 0
-> 10.1.2.13:1111 Masq 2 0 0
-> 10.1.2.13:1116 Masq 2 0 0
-> 10.1.2.14:1111 Masq 2 0 0
-> 10.1.2.14:1116 Masq 2 0 0
FWM 2 wrr persistent 50
-> 10.1.2.12:1115 Masq 2 0 0
-> 10.1.2.12:1117 Masq 2 0 0
-> 10.1.2.13:1115 Masq 2 0 0
-> 10.1.2.13:1117 Masq 2 0 1
-> 10.1.2.14:1115 Masq 2 0 0
-> 10.1.2.14:1117 Masq 2 0 0
OS is CentOS 7 with kernel *3.10*
Thank you.
--
Bogdan Rudas
Director of IT Europe
Exadel Inc.
http://www.exadel.com/
E-mail: brudas@xxxxxxxxxx
Skype ID: bogdan.rudas
--
CONFIDENTIALITY
NOTICE: This email and files attached to it are
confidential. If you
are not the intended recipient you are hereby notified
that using,
copying, distributing or taking any action in reliance on the
contents of this information is strictly prohibited. If you have
received
this email in error please notify the sender and delete this
email.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
