LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Destination unreachable

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Destination unreachable
From: Rainer Anschober <arm@xxxxxxx>
Date: Fri, 12 Sep 2003 08:04:00 +0200
Hi,

at first, thank you very much for your help.

Below my original config file for keepalived. At the moment, i cannot
use both realserver, because one of them has now the ip 80.240.228.100
and is in production modus. Currently, all tests i make with one server
and this config:

[root@lb1 tcpdumps]# ipvsadm -Ln
IP Virtual Server version 1.0.10 (size=32768)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  80.240.228.70:80 sh persistent 300
  -> 172.31.2.171:80              Masq    1      0          0
TCP  80.240.228.74:80 sh persistent 300
  -> 172.31.2.171:80              Masq    1      0          0
TCP  80.240.228.72:80 sh persistent 300
  -> 172.31.2.72:80               Masq    1      0          0
TCP  80.240.228.101:80 sh persistent 300
  -> 172.31.2.171:80              Masq    1      1          2
TCP  80.240.228.72:22 sh persistent 300
  -> 172.31.2.72:22               Masq    1      0          0
TCP  80.240.228.72:21 sh persistent 300
  -> 172.31.2.72:21               Masq    1      0          0
TCP  80.240.228.72:443 sh persistent 300
TCP  80.240.228.74:443 sh persistent 300
  -> 172.31.2.171:443             Masq    1      0          0
TCP  80.240.228.70:443 sh persistent 300
  -> 172.31.2.171:443             Masq    1      0          0

I made this config with ipvsadm himself and not with keepalived, because
i would disable the vrrp. And at the moment, it seems, that the error
will occur 10 times under befor.
Are there configerrors for the vrrpd in my configfile?

Here an cut-out from my tcpdump, where the error occurs:

23:10:37.020778 212.152.215.168.1588 > 80.240.228.101.http: . ack 37781
win 8576 (DF)
23:10:37.021024 80.240.228.101.http > 212.152.215.168.1588: .
45821:46357(536) ack 2304 win 9648 (DF)
23:10:37.076160 80.240.228.101.http > 62.47.21.159.1904: P 1:536(535)
ack 586 win 6435 (DF)
23:10:37.135271 212.152.215.168.1587 > 80.240.228.101.http: . ack 45180
win 8576 (DF)
23:10:37.259744 212.119.130.205.1252 > 80.240.228.101.http: S
222598881:222598881(0) win 16384 <mss 1460,nop,nop,sackOK> (DF)
23:10:37.259770 80.240.228.101 > 212.119.130.205: icmp: 80.240.228.101
tcp port http unreachable [tos 0xc0]
23:10:37.303779 62.47.21.159.1904 > 80.240.228.101.http: . ack 536 win
16529 (DF)
23:10:37.402666 212.152.215.168.1587 > 80.240.228.101.http: . ack 46252
win 8576 (DF)
23:10:37.492264 80.240.228.28 > 224.0.0.18: VRRPv2-advertise 28: vrid=17
prio=150 intvl=1 [tos 0xc0]
23:10:37.670230 212.152.215.168.1587 > 80.240.228.101.http: . ack 47324
win 8576 (DF)
23:10:37.925209 212.152.215.168.1587 > 80.240.228.101.http: . ack 47860
win 8576 (DF)
23:10:38.040296 212.152.215.168.1587 > 80.240.228.101.http: . ack 48932
win 8576 (DF)
23:10:38.133066 212.152.215.168.1587 > 80.240.228.101.http: P
2096:2378(282) ack 49011 win 8497 (DF)
23:10:38.133288 80.240.228.101.http > 212.152.215.168.1587: . ack 2378
win 8576 (DF)
23:10:38.135259 80.240.228.101.http > 212.152.215.168.1587: .
49011:49547(536) ack 2378 win 8576 (DF)
23:10:38.135307 80.240.228.101.http > 212.152.215.168.1587: .
49547:50083(536) ack 2378 win 8576 (DF)
23:10:38.135355 80.240.228.101.http > 212.152.215.168.1587: .
50083:50619(536) ack 2378 win 8576 (DF)
23:10:38.135405 80.240.228.101.http > 212.152.215.168.1587: .
50619:51155(536) ack 2378 win 8576 (DF)
23:10:38.135455 80.240.228.101.http > 212.152.215.168.1587: .
51155:51691(536) ack 2378 win 8576 (DF)
23:10:38.135500 80.240.228.101.http > 212.152.215.168.1587: P
51691:52193(502) ack 2378 win 8576 (DF)
23:10:38.315307 212.152.215.168.1588 > 80.240.228.101.http: . ack 38317
win 8576 (DF)
23:10:38.315606 80.240.228.101.http > 212.152.215.168.1588: .
46357:46893(536) ack 2304 win 9648 (DF)
23:10:38.460267 212.152.215.168.1588 > 80.240.228.101.http: . ack 39389
win 8576 (DF)
23:10:38.460534 80.240.228.101.http > 212.152.215.168.1588: P
46893:47429(536) ack 2304 win 9648 (DF)
23:10:38.460583 80.240.228.101.http > 212.152.215.168.1588: .
47429:47965(536) ack 2304 win 9648 (DF)
23:10:38.492214 80.240.228.28 > 224.0.0.18: VRRPv2-advertise 28: vrid=17
prio=150 intvl=1 [tos 0xc0]
23:10:38.715332 212.152.215.168.1588 > 80.240.228.101.http: . ack 39925
win 8576 (DF)
23:10:38.715573 80.240.228.101.http > 212.152.215.168.1588: .
47965:48501(536) ack 2304 win 9648 (DF)
23:10:38.757088 802.1d config c06f.00:0a:8a:7a:11:c0.8003 root
806f.00:0a:f4:a2:ef:40 pathcost 3023 age 2 max 20 hello 2 fdelay 15
23:10:38.840007 212.152.215.168.1588 > 80.240.228.101.http: . ack 40997
win 8576 (DF)


Are there kernelparameters which i can/must modify?
Currently i am seeking an other NIC like 3COM, because i am not sure
about intel cards working correctly.

Here my Values under /proc/sys/net/ipv4/vs
*** am_droprate ***
10
*** amemthresh ***
1024
*** cache_bypass ***
0
*** debug_level ***
1
*** drop_entry ***
0
*** drop_packet ***
0
*** expire_nodest_conn ***
0
*** lblc_expiration ***
86400
*** lblcr_expiration ***
86400
*** nat_icmp_send ***
0
*** secure_tcp ***
0
*** sync_threshold ***
3
*** timeout_close ***
10
*** timeout_closewait ***
60
*** timeout_established ***
480
*** timeout_finwait ***
60
*** timeout_icmp ***
60
*** timeout_lastack ***
30
*** timeout_listen ***
120
*** timeout_synack ***
100
*** timeout_synrecv ***
10
*** timeout_synsent ***
60
*** timeout_timewait ***
60
*** timeout_udp ***
180


My original keepalived configfile:

global_defs {
  lvs_id DMZ_WEB
}

vrrp_sync_group WEB {
  group {
    WEB_OUT
    WEB_INT
  }
}

vrrp_instance WEB_OUT {
  state MASTER
  interface eth0
  lvs_sync_daemon_inteface eth0
  virtual_router_id 20
  priority 150
  advert_int 1
  authentication {
    auth_type PASS
    auth_pass qkwkekr
  }
  virtual_ipaddress {
    80.240.228.70
    80.240.228.72
    80.240.228.73
    80.240.228.100
    80.240.228.101
  }
}

vrrp_instance WEB_INT {
  state MASTER
  interface eth1
  lvs_sync_daemon_inteface eth1
  virtual_router_id 30
  priority 150
  advert_int 1
  authentication {
    auth_type PASS
    auth_pass qlwlelr
  }
  virtual_ipaddress {
    172.31.2.250
  }
}

virtual_server 80.240.228.70 80 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.171 80 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
  real_server 172.31.2.172 80 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
}

virtual_server 80.240.228.70 443 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.171 443 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
  real_server 172.31.2.172 443 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
}

virtual_server 80.240.228.72 21 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.72 21 {
    weight 1
    TCP_CHECK {
      connect_timeout 10
    }
  }
}

virtual_server 80.240.228.72 22 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.72 22 {
    TCP_CHECK {
      connect_timeout 10
    }
  }
}

virtual_server 80.240.228.72 80 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.72 80 {
    TCP_CHECK {
      connect_timeout 10
    }
  }
}

virtual_server 80.240.228.72 443 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.72 443 {
    TCP_CHECK {
      connect_timeout 10
    }
  }
}

virtual_server 80.240.228.100 80 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.171 80 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
  real_server 172.31.2.172 80 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
}

virtual_server 80.240.228.100 443 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.171 443 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
  real_server 172.31.2.172 443 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
}

virtual_server 80.240.228.101 80 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.181 80 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
  real_server 172.31.2.182 80 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
}

virtual_server 80.240.228.101 443 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.181 443 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
  real_server 172.31.2.182 443 {
    weight 1
    HTTP_GET {
      url {
        path /cgi-bin/check_web.pl
        digest a2299f35097ad6794a9983b39e182f15
      }
      connect_port 80
      connect_timeout 60
      nb_get_retry 2
      delay_before_retry 10
    }
  }
}

virtual_server 80.240.228.73 80 {
  delay_loop 3
  lb_algo sh
  lb_kind NAT
  persistence_timeout 300
  protocol TCP

  real_server 172.31.2.73 80 {
    TCP_CHECK {
      connect_timeout 10
    }
  }
}



<Prev in Thread] Current Thread [Next in Thread>