|
not much of an lvs issue again, but hey, you guys are smart.
i've got a machine that i'm using as a mail server and name server behind
lvs using dr. that machine also needs to get to the outside world, but it
doesn't have a real ip, so it goes through a separate masq server for
that. all works well, but not now that i'm trying to make it an ntp
server also. here's the setup:
jfk: lvs
lga: masq
phl: real server in question.
- - - - -
jfk: 192.168.1.2 on eth1. a few real ips on eth2 for incoming service
requests which get directed to misc real servers. gateway 64.211.224.161
via eth2:
[root@jfk /root]# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
64.211.224.166 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
192.168.1.2 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
64.211.224.165 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.162 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.163 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
192.168.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.160 0.0.0.0 255.255.255.240 U 0 0 0 eth2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 64.211.224.161 0.0.0.0 UG 0 0 0 eth2
[root@jfk /root]# /sbin/ipchains -L -n ; /sbin/ipvsadm -L -n
Chain input (policy ACCEPT):
Chain forward (policy DENY):
Chain output (policy ACCEPT):
IP Virtual Server version 1.0.0-beta1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
UDP 64.211.224.162:53 lc
-> 192.168.1.11:53 Route 1 0 0
UDP 64.211.224.163:53 lc
-> 192.168.1.12:53 Route 1 0 0
TCP 64.211.224.166:22 lc
-> 192.168.1.21:22 Route 1 1 0
TCP 64.211.224.166:25 lc
-> 192.168.1.21:25 Route 1 0 0
TCP 64.211.224.165:80 lc persistent 360
-> 192.168.1.101:80 Route 1 0 0
-> 192.168.1.102:80 Route 1 0 0
[root@jfk /root]# cat /etc/sysctl.conf
# Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS)
net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
kernel.sysrq = 1
# -tcl.
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth1.send_redirects = 0
net.ipv4.conf.eth2.send_redirects = 0
#
- - - - -
lga: 192.168.1.1 on eth1. a few real ips on eth2 for outgoing
masquerading. gateway 64.211.224.161 on eth2.
[root@lga /root]# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
64.211.224.167 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.164 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
192.168.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.160 0.0.0.0 255.255.255.240 U 0 0 0 eth2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 64.211.224.161 0.0.0.0 UG 0 0 0 eth2
[root@lga /root]# /sbin/ipchains -L -n ; /sbin/ipvsadm -L -n
Chain input (policy ACCEPT):
Chain forward (policy DENY):
target prot opt source destination ports
MASQ all ------ 192.168.1.41 0.0.0.0/0 n/a
MASQ all ------ 192.168.1.21 0.0.0.0/0 n/a
Chain output (policy ACCEPT):
[root@lga /root]# cat /etc/sysctl.conf
# Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS)
net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
kernel.sysrq = 1
# -tcl.
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth1.send_redirects = 0
net.ipv4.conf.eth2.send_redirects = 0
#
- - - - -
phl: 192.168.1.21 on eth0. other 192.168.1 addresses on eth0 also. lots
of other ips on other interfaces for internal network stuff. no gateway
in the routing table -- that's handled with iproute2 so that requests via
lvs go back through the 64.211.224.161 gateway directly, but connections
originating on phl go through 192.168.1.1 for masquerading. this config
works for incoming dns and mail requests, and works for outgoing traffic
such as nameservice requests, outgoing mail, pings, and other normal
outgoing traffic.
[root@phl /root]# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.2.10 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
192.168.2.13 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
192.168.1.21 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.3.21 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.162 0.0.0.0 255.255.255.255 UH 0 0 0 lo
64.211.224.163 0.0.0.0 255.255.255.255 UH 0 0 0 lo
192.168.2.14 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
192.168.1.11 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.1.10 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.3.10 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
192.168.1.13 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.3.13 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
192.168.2.21 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
192.168.1.12 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.1.14 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.3.14 0.0.0.0 255.255.255.255 UH 0 0 0 eth2
64.211.224.160 0.0.0.0 255.255.255.240 U 0 0 0 eth0
192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
[root@phl /root]# /sbin/ip rule show
0: from all lookup local
32766: from all lookup main
32767: from all lookup 253
33000: from 192.168.1.0/24 lookup 100
34000: from all lookup 200
[root@phl /root]# /sbin/ip route show table 100
default via 192.168.1.1 dev eth0
[root@phl /root]# /sbin/ip route show table 200
default via 64.211.224.161 dev eth0
[root@phl /root]# cat /etc/sysctl.conf
# Disables packet forwarding
net.ipv4.ip_forward = 1
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Disables automatic defragmentation (needed for masquerading, LVS)
net.ipv4.ip_always_defrag = 0
# Disables the magic-sysrq key
kernel.sysrq = 1
# -tcl.
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.eth0.send_redirects = 0
net.ipv4.conf.all.hidden = 1
net.ipv4.conf.lo.hidden = 1
#
- - - - -
so that's the setup. here's what happens when i try, for example, an
ntpdate:
[root@phl /root]# /usr/sbin/ntpdate ntp.nasa.gov
15 Nov 18:10:25 ntpdate[1256]: no server suitable for synchronization found
tcpdump on phl:
18:10:21.231494 lo > 127.0.0.1.1025 > 127.0.0.1.domain: 14459+ A?
ntp.nasa.gov. (30)
18:10:21.231494 lo < 127.0.0.1.1025 > 127.0.0.1.domain: 14459+ A?
ntp.nasa.gov. (30)
18:10:21.231666 lo > 127.0.0.1.domain > 127.0.0.1.1025: 14459 2/0/0 CNAME
nsipo.nasa.gov., A 143.232.55.13 (66)
18:10:21.231666 lo < 127.0.0.1.domain > 127.0.0.1.1025: 14459 2/0/0 CNAME
nsipo.nasa.gov., A 143.232.55.13 (66)
18:10:21.331314 eth0 > 192.168.1.21.ntp > 143.232.55.13.ntp: v3 client strat 0
poll 4 prec -6
18:10:22.331286 eth0 > 192.168.1.21.ntp > 143.232.55.13.ntp: v3 client strat 0
poll 4 prec -6
18:10:23.331282 eth0 > 192.168.1.21.ntp > 143.232.55.13.ntp: v3 client strat 0
poll 4 prec -6
18:10:24.331281 eth0 > 192.168.1.21.ntp > 143.232.55.13.ntp: v3 client strat 0
poll 4 prec -6
tcpdump on lga and jfk reveal no traffic.
it should be getting to lga at least tho:
[root@phl /root]# /sbin/ip route get from 192.168.1.21 to 143.232.55.13
143.232.55.13 from 192.168.1.21 via 192.168.1.1 dev eth0
cache mtu 1500 rtt 375ms
but it's not.
i tried an ntpdate from another machine inside my network here on a
machine with no advanced routing--just masquerading through 192.168.1.1
(that 192.168.1.41 ip if you refer to lga's ipchains table), and it works
as expected:
[root@ith /root]# /usr/sbin/ntpdate ntp.nasa.gov
15 Nov 18:13:51 ntpdate[6907]: adjust time server 143.232.55.13 offset 0.071391
sec
any ideas on what i may be missing here?
-tcl.
|
