LVS
lvs-users
Google
 
Web LinuxVirtualServer.org

[lvs-users] LVS ixgbe rx_queue

To: <lvs-users@xxxxxxxxxxxxxxxxxxxxxx>
Subject: [lvs-users] LVS ixgbe rx_queue
From: "Aleksey Chudov" <aleksey@xxxxx>
Date: Mon, 21 Nov 2011 10:26:42 +0200
Hello!

 

I have LVS system with two Six-Core CPUs and 2x 10G Intel NICs. Intel ixgbe
driver creates 24 tx/rx queues.

While testing with iperf tool traffic was distributing among 24 RX queues.

But with LVS traffic distributed only among 16 RX queues.

 

How to direct traffic to all 24 rx queues?

 

HW configuration:

HP ProLiant DL360 G7, 2x Intel Six-Core E5645, 16GB RAM, 2x Intel
Corporation 82599EB 10-Gigabit NICs

Linux Kernel 2.6.38.8

 

Some ethtool output

 

# ethtool -i eth0

driver: ixgbe

version: 3.2.9-k2

firmware-version: 0.9-3

bus-info: 0000:09:00.0

 

# ethtool -k eth0

Offload parameters for eth0:

rx-checksumming: on

tx-checksumming: on

scatter-gather: on

tcp-segmentation-offload: on

udp-fragmentation-offload: off

generic-segmentation-offload: on

generic-receive-offload: off

large-receive-offload: off

rx-vlan-offload: on

tx-vlan-offload: on

ntuple-filters: off

receive-hashing: off

 

# ethtool -S eth0

NIC statistics:

     rx_packets: 2353149

     tx_packets: 2710579

     rx_bytes: 651217098

     tx_bytes: 659738990

     rx_pkts_nic: 2353149

     tx_pkts_nic: 2710579

     rx_bytes_nic: 660629694

     tx_bytes_nic: 670597164

     lsc_int: 2

     tx_busy: 0

     non_eop_descs: 0

     rx_errors: 0

     tx_errors: 0

     rx_dropped: 0

     tx_dropped: 0

     multicast: 99

     broadcast: 1281

     rx_no_buffer_count: 0

     collisions: 0

     rx_over_errors: 0

     rx_crc_errors: 0

     rx_frame_errors: 0

     hw_rsc_aggregated: 0

     hw_rsc_flushed: 0

     fdir_match: 291115

     fdir_miss: 2031557

     rx_fifo_errors: 0

     rx_missed_errors: 0

     tx_aborted_errors: 0

     tx_carrier_errors: 0

     tx_fifo_errors: 0

     tx_heartbeat_errors: 0

     tx_timeout_count: 0

     tx_restart_queue: 20

     rx_long_length_errors: 0

     rx_short_length_errors: 0

     tx_flow_control_xon: 0

     rx_flow_control_xon: 0

     tx_flow_control_xoff: 0

     rx_flow_control_xoff: 0

     rx_csum_offload_errors: 49

     alloc_rx_page_failed: 0

     alloc_rx_buff_failed: 0

     rx_no_dma_resources: 0

     tx_queue_0_packets: 165043

     tx_queue_0_bytes: 38401840

     tx_queue_1_packets: 162906

     tx_queue_1_bytes: 38138855

     tx_queue_2_packets: 168570

     tx_queue_2_bytes: 41790265

     tx_queue_3_packets: 167731

     tx_queue_3_bytes: 40535275

     tx_queue_4_packets: 168591

     tx_queue_4_bytes: 42697032

     tx_queue_5_packets: 170165

     tx_queue_5_bytes: 43270191

     tx_queue_6_packets: 173063

     tx_queue_6_bytes: 40746228

     tx_queue_7_packets: 171329

     tx_queue_7_bytes: 42974539

     tx_queue_8_packets: 171401

     tx_queue_8_bytes: 40370468

     tx_queue_9_packets: 174085

     tx_queue_9_bytes: 43069158

     tx_queue_10_packets: 165402

     tx_queue_10_bytes: 40150477

     tx_queue_11_packets: 165939

     tx_queue_11_bytes: 38720432

     tx_queue_12_packets: 174055

     tx_queue_12_bytes: 42915187

     tx_queue_13_packets: 169259

     tx_queue_13_bytes: 40416664

     tx_queue_14_packets: 167273

     tx_queue_14_bytes: 39214682

     tx_queue_15_packets: 175688

     tx_queue_15_bytes: 46319151

     tx_queue_16_packets: 2

     tx_queue_16_bytes: 248

     tx_queue_17_packets: 28

     tx_queue_17_bytes: 2222

     tx_queue_18_packets: 5

     tx_queue_18_bytes: 620

     tx_queue_19_packets: 5

     tx_queue_19_bytes: 620

     tx_queue_20_packets: 5

     tx_queue_20_bytes: 620

     tx_queue_21_packets: 1

     tx_queue_21_bytes: 124

     tx_queue_22_packets: 0

     tx_queue_22_bytes: 0

     tx_queue_23_packets: 33

     tx_queue_23_bytes: 4092

     rx_queue_0_packets: 143415

     rx_queue_0_bytes: 37851657

     rx_queue_1_packets: 139924

     rx_queue_1_bytes: 39637036

     rx_queue_2_packets: 147665

     rx_queue_2_bytes: 41246049

     rx_queue_3_packets: 147788

     rx_queue_3_bytes: 40565392

     rx_queue_4_packets: 146184

     rx_queue_4_bytes: 41169068

     rx_queue_5_packets: 149493

     rx_queue_5_bytes: 42221674

     rx_queue_6_packets: 143145

     rx_queue_6_bytes: 37153935

     rx_queue_7_packets: 148494

     rx_queue_7_bytes: 40966091

     rx_queue_8_packets: 148678

     rx_queue_8_bytes: 39129607

     rx_queue_9_packets: 146972

     rx_queue_9_bytes: 39589930

     rx_queue_10_packets: 146489

     rx_queue_10_bytes: 42729748

     rx_queue_11_packets: 142334

     rx_queue_11_bytes: 37601609

     rx_queue_12_packets: 150304

     rx_queue_12_bytes: 42118352

     rx_queue_13_packets: 153071

     rx_queue_13_bytes: 43629946

     rx_queue_14_packets: 142691

     rx_queue_14_bytes: 37838601

     rx_queue_15_packets: 156491

     rx_queue_15_bytes: 47767637

     rx_queue_16_packets: 0

     rx_queue_16_bytes: 0

     rx_queue_17_packets: 11

     rx_queue_17_bytes: 766

     rx_queue_18_packets: 0

     rx_queue_18_bytes: 0

     rx_queue_19_packets: 0

     rx_queue_19_bytes: 0

     rx_queue_20_packets: 0

     rx_queue_20_bytes: 0

     rx_queue_21_packets: 0

     rx_queue_21_bytes: 0

     rx_queue_22_packets: 0

     rx_queue_22_bytes: 0

     rx_queue_23_packets: 0

     rx_queue_23_bytes: 0

 

 

 

Best regards,

Aleksey

 

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@xxxxxxxxxxxxxxxxxxxxxx
Send requests to lvs-users-request@xxxxxxxxxxxxxxxxxxxxxx
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

<Prev in Thread] Current Thread [Next in Thread>