Anyway to increase the RX/TX queues when reserve NIC?

[Chengyi Qu]

Hi,

I’m finding a way to increase the RX/TX queue number as I increased the CPU cores on the reservation. No matter what I reserve the nodes, when I try to check/increase the number of RX/TX queue number of one NIC, I get error like this:

$ethtool -L ens7 rx 32 tx 32

Cannot set device channel parameters: Invalid argument

Once I checked the default settings from the node, I got:

$ethtool -l ens7

Channel parameters for ens7:
Pre-set maximums:
RX: 0
TX: 0
Other: 0
Combined: 5
Current hardware settings:
RX: 0
TX: 0
Other: 0
Combined: 5

I want to figure out if there is a way to default set the number as the same as the CPU cores I reserved. If get confused, please check this link as a reference: https://developers.redhat.com/articles/2021/11/05/improve-udp-performance-rhel-85#availability_in_red_hat_enterprise_linux_8_5

The purpose of doing this is that when I test the UDP packet transmission performance over nodes, I noticed that it is (a lot) slower than TCP. I wonder if there is a way to increase that performance. Maybe TSO GRO is a good way to test, however, currently, I cannot set the channels to a larger number.

Anyway, if you guys get good performance on testing UDP (like can reach 100Gpps), please let me know your settings, thank you so much!

Chengyi

[Paul Ruth]

I’m not sure about how to use GRO.  There is nothing in particular we are doing to prevent you from using it.   You have full access to the PCI NIC and full control over the operating system.  It might just be that you need to install more tools or update a kernel.  I played with it a bit but didn’t get any further than you.  Have you made GRO work on machines outside of FABRIC?

UDP can be tricky. Have you tried the suggestions here?  https://fasterdata.es.net/host-tuning/linux/udp-tuning

 

[Author]

Posts