LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

Re: [lvs-users] Invalid operation. Possibly wrong module version, addre

To: "LinuxVirtualServer.org users mailing list." <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: Re: [lvs-users] Invalid operation. Possibly wrong module version, address not unicast, ...
Cc: Simon Horman <horms@xxxxxxxxxxxx>
From: Nigel Kukard <nkukard@xxxxxxxx>
Date: Tue, 09 Feb 2010 07:38:29 +0000
>>>>>> Apon upgrading to 2.6.29.6 from 2.6.22 and recompiling ipvsadm 1.25 to
>>>>>> get ipv6 support I'm getting the following error:
>>>>>>
>>>>>> Invalid operation.  Possibly wrong module version, address not unicast, 
>>>>>> ...
>>>>>>
>>>>>> when running ...
>>>>>>
>>>>>> ipvsadm -ln -t SERVICE_HERE
>>>>>>
>>>>>> There is absolutely nothing else that changed apart from the kernel and
>>>>>> recompile of 1.25.
>>>>>>
>>>>>> Everything seems to work fine,  ipvsadm -ln  works fine, setting up
>>>>>> services works fine as does all functionality except the above.
>>>>>>
>>>>>> Can anyone shed some light?
>>>>>>     
>>>>>>       
>>>>>>           
>>>>>>             
>>>>> Hi Nigel,
>>>>>
>>>>> I'm not having any luck reproducing this problem.
>>>>> Is sane output produced when you run "ipvsadm -ln" ?
>>>>> Could you be more specific about what SERVICE_HERE is,
>>>>> and if it includes a host name if it resolves to an ipv4
>>>>> or ipv6 address, or both?
>>>>>
>>>>> Also, is the 2.6.29.6 vanilla, or has it had some patches added?
>>>>>     
>>>>>         
>>>>>           
>>>> 2.6.29.6 vanilla
>>>>
>>>> The problem above was due to some CFLAGS being set in the environment,
>>>> namely  "-O2 --march=pentium-mmx". Without the env being set, there are
>>>> no more errors. I did a total rm -rf sources and recompile.
>>>>
>>>> Which brings me to another problem now... my ipvsadm -ln is not
>>>> displaying the current number of connections. I've copied the same
>>>> binary over to 2.6.29.5 and its working. Using the binary on 2.6.29.6
>>>> seems to just give zero's. Also statement in the original post was
>>>> incorrect, -ln did work, but also didn't display the number of connections.
>>>>
>>>> # ipvsadm -v
>>>> ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1)
>>>>
>>>> # ipvsadm -ln -t a.b.c.d:25
>>>> Prot LocalAddress:Port Scheduler Flags
>>>>   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
>>>> TCP  a.b.c.d:25 wlc
>>>>   -> 10.0.200.131:25              Masq    240    0          75       
>>>>   -> 10.0.200.141:25              Masq    239    0          69       
>>>>   -> 10.0.200.151:25              Masq    238    0          69       
>>>>   -> 10.0.200.161:25              Masq    239    0          67
>>>>
>>>> # ipvsadm -ln -t a.b.c.d:25 --stats
>>>> Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes
>>>> OutBytes
>>>>   -> RemoteAddress:Port
>>>> TCP  a.b.c.d:25                 1516    37220    32114 31690896  1836300
>>>>   -> 10.0.200.131:25                   404    13002    10258 10433390  
>>>> 556410
>>>>   -> 10.0.200.141:25                   360     7119     6485  5986697  
>>>> 392303
>>>>   -> 10.0.200.151:25                   390     9374     8454  8919481  
>>>> 485851
>>>>   -> 10.0.200.161:25                   362     7726     6918  6351400  
>>>> 401884
>>>>
>>>> # ipvsadm -ln -t a.b.c.d:25 --rate
>>>> Prot LocalAddress:Port                 CPS    InPPS   OutPPS    InBPS  
>>>> OutBPS
>>>>   -> RemoteAddress:Port
>>>> TCP  a.b.c.d:25                    6       96       92    69714     5619
>>>>   -> 10.0.200.131:25                     1       14       16    
>>>> 3177     1108
>>>>   -> 10.0.200.141:25                     2       38       32   
>>>> 30077     1911
>>>>   -> 10.0.200.151:25                     1       21       21   
>>>> 14418     1274
>>>>   -> 10.0.200.161:25                     1       24       23   
>>>> 22041     1325
>>>>
>>>> # ipvsadm -ln -c | grep ESTABLISHED | wc -l   
>>>> 65
>>>>   
>>>>       
>>>>         
>>> Ok ... I found the problem.
>>>
>>> * My scripts overrode CFLAGS= which is why it worked before, -D for NL
>>> was being forcefully removed.
>>> * I fixed this a few weeks ago, when upgrading .. same time I upgraded
>>> my kernel, appears that libnl is now used.
>>> * Below ...
>>>
>>> make HAVE_NL=0 POPT_LIB="-lpopt"   <= that works fine, I get the correct
>>> listing below...
>>> TCP  a.b.c.d:25 wlc
>>>   -> 10.0.200.131:25              Masq    238    14         47       
>>>   -> 10.0.200.141:25              Masq    241    14         36       
>>>   -> 10.0.200.151:25              Masq    242    10         109      
>>>   -> 10.0.200.161:25              Masq    242    13         90 
>>>
>>> make POPT_LIB="-lpopt" <= that gives wrong details for ActiveConn, below ...
>>> TCP  a.b.c.d:25 wlc
>>>   -> 10.0.200.131:25              Masq    240    0          31       
>>>   -> 10.0.200.141:25              Masq    240    0          34       
>>>   -> 10.0.200.151:25              Masq    241    0          94       
>>>   -> 10.0.200.161:25              Masq    238    0          87       
>>>
>>> Same results on all kernel versions & servers I've tried.
>>>
>>> My problem is that using HAVE_NL=0 seems not to work with v6 support...
>>> /tmp/ipvsadm.no-nl -A -t [fc00::10]:25 -s rr
>>> Operation not supported with specified address family
>>>     
>>>       
>> That bit is expected, the NL support was added specifically
>> because the non-NL support couldn't be expanded to handle ipv6.
>> So as you need ipv6 you also need NL. And curiously that seems
>> to result in the connection count being bogus.
>>
>> Would it be possible for you to test this against a newer kernel - say
>> 2.6.32 or 2.6.33-rc4. It would be good to know if the problem is
>> still present.
>>     
> I can with no doubt confirm the problem is still present in 2.6.33-rc5.
> I now have a test network setup especially for this.

I've tested this out now on quite a number of systems with different
kernel versions and at least to me its 100% reproduceable.

Simon, you had any luck your side finding what the problem may be?

I've also noticed an issue with a 64bit kernel and 32bit compiled
userspace, not sure if this is related.

Regards
Nigel

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>