LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: Ldirectord does not load all real servers

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Ldirectord does not load all real servers
From: Horms <horms@xxxxxxxxxxxx>
Date: Thu, 27 May 2004 14:34:54 +0900
On Thu, May 27, 2004 at 01:30:24PM +0900, Horms wrote:
> On Thu, May 27, 2004 at 12:56:52PM +0900, Horms wrote:
> > On Thu, May 27, 2004 at 11:47:03AM +1000, Jonathan Trott wrote:
> > 
> > [snip]
> > 
> > > >Are there any known problems with 1.87 I should be wary of? Is there  
> > > >another version with the fallback fix but without this new bug? Can  
> > > >anyone supply a patch so that I can fix this problem in 1.87?  
> > > >Unfortunately I don't have the time to come to grips with the  
> > > >ldirectord perl code :(
> > > >Thanks,
> > 
> > > I have tried other versions of the code in CVS and the last one that  
> > > worked for me was 1.86. After that I don't get all my pools containing  
> > > real servers. I will test 1.86 and see if the fallback problem has been  
> > > solved.
> > > Who do I report bugs to regarding the 1.87 problem? BTW, the latest  
> > > stable, 1.77.2.2 has the same problem as 1.87.

[snip]

Hi,

I think that I have a fix for this. Basically the internal state
kept for a real server, to determin if it is up or down was a bit
to simplistic. This has been the root of a number of problems over
the past while. The patch below introduces more sophisticated tracking
of the internal state of real servers. This should resolve your problem.
And hopefully not reintroduce other related problems that were resolve
in the past.

Feedback is more than welcome. 

I have put the change into the HEAD branch of linux-ha CVS as
version 1.88. It should also replace 1.77.2.2 in the STABLE_1_2
branck (this means the next 1.2.X release of heartbeat) but I have
not merged the change there yet. 

Please test this.

The patch is below. It should apply against 1.87 and 1.77.2.2.
If you don't want to patch these versions, information on how to 
get ldirectord from CVS can be found on
http://www.vergenet.net/linux/ldirectord/

-- 
Horms

Index: ldirectord
===================================================================
RCS file: /home/cvs/linux-ha/linux-ha/ldirectord/ldirectord,v
retrieving revision 1.87
diff -u -r1.87 ldirectord
--- ldirectord  7 May 2004 08:53:33 -0000       1.87
+++ ldirectord  27 May 2004 05:26:43 -0000
@@ -1265,7 +1265,6 @@
                        } else {
                                 $$r{wght} = "1";
                        }
-                       $$r{status} = -1;
                        if (defined $$r{request} && defined $$r{receive}) {
                                my $uri = $$r{request};
                                $uri =~ s/^\///g;
@@ -1405,14 +1404,10 @@
                        for $nr (@$nreal) {
                                if (exists($$or{"$$nr{server}:$$nr{port}"})) {
                                        &system_wrapper("$IPVSADM -e 
$$nv{proto} " .  &get_virtual($nv) .  " -r $$nr{server}:$$nr{port} $$nr{forw} 
-w $$nr{wght}");
-                                       $$nv{status}++;
-                                       $$nr{status} = 1;
+                                       _status_up($nv, $nr);
                                        &ld_log("Changed real server: 
$$nr{server}:$$nr{port} ($$nv{status} x " . &get_virtual($nv) . ")\n");
                                        delete($$or{"$$nr{server}:$$nr{port}"});
                                }
-                               else {
-                                       $$nr{status} = 0;
-                               }
                        }
                        # remove remaining entries for real servers
                        for my $k (keys %$or) {
@@ -1423,9 +1418,6 @@
                } else {
                        # no such service, create a new one
                        &system_wrapper("$IPVSADM -A $$nv{flags}");
-                       for $nr (@$nreal) {
-                               $$nr{status} = -1;
-                       }
                        &ld_log("Added virtual server: " . &get_virtual($nv));
                }
 
@@ -1459,10 +1451,9 @@
        foreach my $v (@VIRTUAL) {
                my $real = $$v{real};
                foreach my $r (@$real) {
-                       if ($$r{status}>0) {
+                       if (defined $$r{virtual_status}) {
                                &system_wrapper("$IPVSADM -d $$v{proto} " . 
&get_virtual($v) . " -r $$r{server}:$$r{port}");
-                               $$r{status} = 0;
-                               $$v{status}--;
+                               _status_down($v, $r);
                                &ld_log("Removed real server: 
$$r{server}:$$r{port} ($$v{status} x " . &get_virtual($v) );
                        }
                }
@@ -2001,10 +1992,6 @@
 
        my ($real, $virtual, $virt);
 
-        # Return if the real server is already in the desired state
-       return if ($$r{status}==1 and $state=~/up/i);
-       return if ($$r{status}==0 and $state=~/down/i);
-
         # Find the real server in @REAL
        foreach $real (@REAL) {
                if($real->{"real"} eq get_real_id_str($r, $v)) {
@@ -2181,6 +2168,55 @@
 }
 
 
+# Set the status of a server as up
+# Should onlu be called from _service_up or _ld_start
+
+sub _status_up
+{
+       my ($v, $r) = (@_);
+
+       my $virtual_id = get_virtual_id_str($v);
+
+       if (defined ($r->{virtual_status}) and
+                       $r->{virtual_status}->{"$virtual_id"}) {
+               return undef;
+       }
+       
+       if (!defined ($r->{virtual_status})) {
+               $r->{virtual_status} = {};
+       }
+       $r->{virtual_status}->{"$virtual_id"} = 1;
+       $v->{status}++;
+
+       return 1;
+}
+
+# Set the status of a server as down
+# Should onlu be called from _service_down or _ld_stop
+
+sub _status_down
+{
+       my ($v, $r) = (@_);
+
+       my $virtual_id = get_virtual_id_str($v);
+
+       if (! defined ($r->{virtual_status}) or
+                       ! $r->{virtual_status}->{"$virtual_id"}) {
+               return undef;
+       }
+       
+       delete $r->{virtual_status}->{"$virtual_id"};
+       $v->{status}--;
+       if (! %{$r->{virtual_status}}) {
+               $r->{virtual_status} = undef;
+       }
+
+       return 1;
+}
+
+
+
+
 # _service_up
 # Bring a real service up if it is down
 # Should be called by service_set only
@@ -2197,13 +2233,13 @@
 {
        my ($v, $r) = (@_);
 
-       if ($$r{status}==0 or $$r{status}==-1) {
-                &_restore_service($v, $r->{server} . ":" . $r->{port}, 
-                                  $r->{forw}, $r->{wght}, "real");
-               $$v{status}++;
-               $$r{status} = 1;
-               &fallback_off($v);
+       if (! _status_up($v, $r)) {
+               return;
        }
+
+        &_restore_service($v, $r->{server} . ":" . $r->{port}, 
+                                  $r->{forw}, $r->{wght}, "real");
+       &fallback_off($v);
 }
 
 
@@ -2223,15 +2259,13 @@
 {
        my ($v, $r) = @_;
 
-       if ($$r{status}==1 or $$r{status}==-1) {
-                &_remove_service($v, $r->{server} . ":" . $r->{port}, 
-                                  $r->{forw}, "real");
-               if ($$r{status}==1) {
-                       $$v{status}--;
-               }
-               $$r{status} = 0;
-               &fallback_on($v);
+       if (! _status_down($v, $r)) {
+               return;
        }
+
+        &_remove_service($v, $r->{server} . ":" . $r->{port}, 
+                          $r->{forw}, "real");
+       &fallback_on($v);
 }
 
 
<Prev in Thread] Current Thread [Next in Thread>