|
On 02/09/10 09:39, Simon Horman wrote:
> On Tue, Feb 09, 2010 at 07:38:29AM +0000, Nigel Kukard wrote:
>
>>
>>>>>>>> Apon upgrading to 2.6.29.6 from 2.6.22 and recompiling ipvsadm 1.25 to
>>>>>>>> get ipv6 support I'm getting the following error:
>>>>>>>>
>>>>>>>> Invalid operation. Possibly wrong module version, address not
>>>>>>>> unicast, ...
>>>>>>>>
>>>>>>>> when running ...
>>>>>>>>
>>>>>>>> ipvsadm -ln -t SERVICE_HERE
>>>>>>>>
>>>>>>>> There is absolutely nothing else that changed apart from the kernel and
>>>>>>>> recompile of 1.25.
>>>>>>>>
>>>>>>>> Everything seems to work fine, ipvsadm -ln works fine, setting up
>>>>>>>> services works fine as does all functionality except the above.
>>>>>>>>
>>>>>>>> Can anyone shed some light?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Hi Nigel,
>>>>>>>
>>>>>>> I'm not having any luck reproducing this problem.
>>>>>>> Is sane output produced when you run "ipvsadm -ln" ?
>>>>>>> Could you be more specific about what SERVICE_HERE is,
>>>>>>> and if it includes a host name if it resolves to an ipv4
>>>>>>> or ipv6 address, or both?
>>>>>>>
>>>>>>> Also, is the 2.6.29.6 vanilla, or has it had some patches added?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> 2.6.29.6 vanilla
>>>>>>
>>>>>> The problem above was due to some CFLAGS being set in the environment,
>>>>>> namely "-O2 --march=pentium-mmx". Without the env being set, there are
>>>>>> no more errors. I did a total rm -rf sources and recompile.
>>>>>>
>>>>>> Which brings me to another problem now... my ipvsadm -ln is not
>>>>>> displaying the current number of connections. I've copied the same
>>>>>> binary over to 2.6.29.5 and its working. Using the binary on 2.6.29.6
>>>>>> seems to just give zero's. Also statement in the original post was
>>>>>> incorrect, -ln did work, but also didn't display the number of
>>>>>> connections.
>>>>>>
>>>>>> # ipvsadm -v
>>>>>> ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1)
>>>>>>
>>>>>> # ipvsadm -ln -t a.b.c.d:25
>>>>>> Prot LocalAddress:Port Scheduler Flags
>>>>>> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
>>>>>> TCP a.b.c.d:25 wlc
>>>>>> -> 10.0.200.131:25 Masq 240 0 75
>>>>>> -> 10.0.200.141:25 Masq 239 0 69
>>>>>> -> 10.0.200.151:25 Masq 238 0 69
>>>>>> -> 10.0.200.161:25 Masq 239 0 67
>>>>>>
>>>>>> # ipvsadm -ln -t a.b.c.d:25 --stats
>>>>>> Prot LocalAddress:Port Conns InPkts OutPkts InBytes
>>>>>> OutBytes
>>>>>> -> RemoteAddress:Port
>>>>>> TCP a.b.c.d:25 1516 37220 32114 31690896 1836300
>>>>>> -> 10.0.200.131:25 404 13002 10258 10433390
>>>>>> 556410
>>>>>> -> 10.0.200.141:25 360 7119 6485 5986697
>>>>>> 392303
>>>>>> -> 10.0.200.151:25 390 9374 8454 8919481
>>>>>> 485851
>>>>>> -> 10.0.200.161:25 362 7726 6918 6351400
>>>>>> 401884
>>>>>>
>>>>>> # ipvsadm -ln -t a.b.c.d:25 --rate
>>>>>> Prot LocalAddress:Port CPS InPPS OutPPS InBPS
>>>>>> OutBPS
>>>>>> -> RemoteAddress:Port
>>>>>> TCP a.b.c.d:25 6 96 92 69714 5619
>>>>>> -> 10.0.200.131:25 1 14 16
>>>>>> 3177 1108
>>>>>> -> 10.0.200.141:25 2 38 32
>>>>>> 30077 1911
>>>>>> -> 10.0.200.151:25 1 21 21
>>>>>> 14418 1274
>>>>>> -> 10.0.200.161:25 1 24 23
>>>>>> 22041 1325
>>>>>>
>>>>>> # ipvsadm -ln -c | grep ESTABLISHED | wc -l
>>>>>> 65
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Ok ... I found the problem.
>>>>>
>>>>> * My scripts overrode CFLAGS= which is why it worked before, -D for NL
>>>>> was being forcefully removed.
>>>>> * I fixed this a few weeks ago, when upgrading .. same time I upgraded
>>>>> my kernel, appears that libnl is now used.
>>>>> * Below ...
>>>>>
>>>>> make HAVE_NL=0 POPT_LIB="-lpopt" <= that works fine, I get the correct
>>>>> listing below...
>>>>> TCP a.b.c.d:25 wlc
>>>>> -> 10.0.200.131:25 Masq 238 14 47
>>>>> -> 10.0.200.141:25 Masq 241 14 36
>>>>> -> 10.0.200.151:25 Masq 242 10 109
>>>>> -> 10.0.200.161:25 Masq 242 13 90
>>>>>
>>>>> make POPT_LIB="-lpopt" <= that gives wrong details for ActiveConn, below
>>>>> ...
>>>>> TCP a.b.c.d:25 wlc
>>>>> -> 10.0.200.131:25 Masq 240 0 31
>>>>> -> 10.0.200.141:25 Masq 240 0 34
>>>>> -> 10.0.200.151:25 Masq 241 0 94
>>>>> -> 10.0.200.161:25 Masq 238 0 87
>>>>>
>>>>> Same results on all kernel versions & servers I've tried.
>>>>>
>>>>> My problem is that using HAVE_NL=0 seems not to work with v6 support...
>>>>> /tmp/ipvsadm.no-nl -A -t [fc00::10]:25 -s rr
>>>>> Operation not supported with specified address family
>>>>>
>>>>>
>>>>>
>>>> That bit is expected, the NL support was added specifically
>>>> because the non-NL support couldn't be expanded to handle ipv6.
>>>> So as you need ipv6 you also need NL. And curiously that seems
>>>> to result in the connection count being bogus.
>>>>
>>>> Would it be possible for you to test this against a newer kernel - say
>>>> 2.6.32 or 2.6.33-rc4. It would be good to know if the problem is
>>>> still present.
>>>>
>>>>
>>> I can with no doubt confirm the problem is still present in 2.6.33-rc5.
>>> I now have a test network setup especially for this.
>>>
>> I've tested this out now on quite a number of systems with different
>> kernel versions and at least to me its 100% reproduceable.
>>
>> Simon, you had any luck your side finding what the problem may be?
>>
> Sorry, no. I've been super-busy of late.
>
>
>> I've also noticed an issue with a 64bit kernel and 32bit compiled
>> userspace, not sure if this is related.
>>
> Interesting, could you give some more details?
>
Lets first get the NL issue fixed, then I'll do more testing and post my
findings :)
Is there anything else you want me to try from my side which would make
finding the problem easier?
-N
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
|
