LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] Port mapping with LVS-DR using fwmark

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Port mapping with LVS-DR using fwmark
From: Malcolm Turnbull <malcolm@xxxxxxxxxxxxxxxx>
Date: Fri, 17 Jan 2014 19:17:41 +0000
Jacoby,

iptables will work on a different physical server, but does not work
on the director node this is due to the way that LVS interacts with
netfilter.

More discussion here....
http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.rewrite_ports.html

If you need port re-direction you could use HAProxy instead (but its
not transparent).



On 17 January 2014 18:54, Jacoby Hickerson <hickersonjl@xxxxxxxxx> wrote:
> Thanks Malcolm for the response.  That is how it is setup, the real server
> is the same as the director node for one of the nodes.  Even if connecting
> to only the primary node while all others are offline it does not work.
>
> I noticed that if I use xinetd forwarding it works, but that's not what I
> want to use.  I wonder why iptables wouldn't be able to work, if there is a
> method using iptables that'd be great.
>
> Jacoby
>
>
> On Thu, Jan 16, 2014 at 11:41 PM, Malcolm Turnbull <malcolm@xxxxxxxxxxxxxxxx
>> wrote:
>
>> Jacoby,
>>
>> You could put the iptables rules on each real server instead? (which
>> would do the same trick.)
>> LVS is on the INPUT chain so its very hard to use iptables rules like
>> this on the director node.
>> Their may be a way, I just don't know of it.
>>
>>
>>
>> On 17 January 2014 01:27, Jacoby Hickerson <hickersonjl@xxxxxxxxx> wrote:
>> > I've searched Google and this mailing list but haven't quite seen the
>> same
>> > configuration and/or setup as mine.
>> >
>> > The ldirectord documentation states that port mapping on the same server
>> > where the director resides is not possible other than masq, however it
>> says
>> > "non-fwmark".  My setup is using fwmark, however, when trying to port map
>> > from port 80 to another port, the client connection hangs.  Here are the
>> > exact details of my setup:
>> >
>> > The VIP is on the same box as the director and RIP 172.17.0.16.  This
>> setup
>> > works fine when no port mapping is being done, but I need to move the
>> port
>> > to something higher than 1024.
>> >
>> > virtual=172.17.0.24:80
>> > real=172.17.0.16:50000 gate 100
>> > real=172.17.0.17:50000 gate 100
>> > service=http
>> > scheduler=rr
>> > protocol=tcp
>> >   checktype=connect
>> >   fwmark=100
>> >
>> > iptables:
>> > iptables -t mangle -A PREROUTING -d 172.17.0.24/32 ! -i lo -p tcp -m tcp
>> > --dport 80 -j MARK --set-xmark 0x64/0xffffffff
>> > iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT
>> > --to-ports 50000
>> > iptables -t nat -A OUTPUT -o lo -p tcp -m tcp --dport 80 -j REDIRECT
>> > --to-ports 50000
>> >
>> > Issue:
>> > curl -v 'http://172.17.0.24'
>> > * About to connect() to 172.17.0.24 port 80 (#0)
>> > *   Trying 172.17.0.24...
>> >
>> > 00:41:44.503581 IP 172.17.0.2.46099 > 172.17.0.24.80: Flags [S], seq
>> > 1066084928, win 14600, options [mss 1460,sackOK,TS val 2520815062 ecr
>> > 0,nop,wscale 7], length 0
>> > 00:41:44.503581 IP 172.17.0.2.46099 > 172.17.0.24.80: Flags [S], seq
>> > 1066084928, win 14600, options [mss 1460,sackOK,TS val 2520815062 ecr
>> > 0,nop,wscale 7], length 0
>> > 00:41:44.503658 IP 172.17.0.16.50000 > 172.17.0.2.46099: Flags [S.], seq
>> > 824291086, ack 1066084929, win 14480, options [mss 1460,sackOK,TS val
>> > 9521949 ecr 2520815062,nop,wscale 7], length 0
>> > 00:41:44.503663 IP 172.17.0.16.50000 > 172.17.0.2.46099: Flags [S.], seq
>> > 824291086, ack 1066084929, win 14480, options [mss 1460,sackOK,TS val
>> > 9521949 ecr 2520815062,nop,wscale 7], length 0
>> >
>> > So the problem I'm having is that the source ip is not being translated
>> by
>> > iptables but sent via lvs as the RIP.  Is there a kernel option, iptables
>> > option or ipvsadm option that would allow it to change it back to the
>> VIP?
>> >
>> > Any help would be very appreciated!
>> >
>> > Jacoby
>> > _______________________________________________
>> > Please read the documentation before posting - it's available at:
>> > http://www.linuxvirtualserver.org/
>> >
>> > LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
>> > Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
>> > or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>>
>>
>>
>> --
>> Regards,
>>
>> Malcolm Turnbull.
>>
>> Loadbalancer.org Ltd.
>> Phone: +44 (0)870 443 8779
>> http://www.loadbalancer.org/
>>
>> _______________________________________________
>> Please read the documentation before posting - it's available at:
>> http://www.linuxvirtualserver.org/
>>
>> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
>> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
>> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
> Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users



-- 
Regards,

Malcolm Turnbull.

Loadbalancer.org Ltd.
Phone: +44 (0)870 443 8779
http://www.loadbalancer.org/

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>