2019-11-19 Set Jumbo Frames on Bridges (1)

Overview

I am trying to find out why the interconnect between the nodes of the REDFERN cluster does not support jumbo frames as described in issue #47 (“Ping tests show connectivity across interconnect but large packets are dropped”).

References

• Linux man pages online
  • cd
  • ethtool
  • grep
  • ip
  • uname
• My Oracle Support
  • Oracle VM: Change MTU Value Failed with 'Invalid argument' Running Kernel UEK5 on VM (Doc ID 2478446.1)

Procedure

I am trying to find why the configuration settings are not being applied.

Find the NIC Configuration Files

Run the following command to change to the directory where the NIC configurations files are kept:

cd /etc/sysconfig/network-scripts

Verify MTU Configuration Settings

Run the following command to find what MTU settings are defined for the NICs:

grep -i mtu= ifcfg*

The expected output is:

ifcfg-eth0:MTU="9000" ifcfg-eth1:MTU="9000" ifcfg-eth1:1:MTU="9000" ifcfg-eth1:2:MTU="9000" ifcfg-eth1:3:MTU="9000" ifcfg-eth2:MTU="9000" ifcfg-xenbr0:MTU=9000 ifcfg-xenbr1:MTU="9000" ifcfg-xenbr2:MTU="9000" ifcfg-xenbr3:MTU="9000"

All bridges (xenbr*) have the MTU set to 9000.

Find Actual NIC Settings

Run the following command to see what the actual NIC settings are:

ip link show

The expected output is:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast master xenbr0 state UP qlen 1000 link/ether 44:37:e6:a7:34:9a brd ff:ff:ff:ff:ff:ff 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP qlen 1000 link/ether 00:1b:21:d9:a1:ae brd ff:ff:ff:ff:ff:ff 4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000 link/ether 00:1b:21:d9:a1:af brd ff:ff:ff:ff:ff:ff 5: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UP link/ether 44:37:e6:a7:34:9a brd ff:ff:ff:ff:ff:ff 6: xenbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether 82:84:44:75:b6:bf brd ff:ff:ff:ff:ff:ff 7: xenbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether 8a:ce:1e:a5:7a:d3 brd ff:ff:ff:ff:ff:ff 8: xenbr3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether 86:6f:7c:ea:a3:b2 brd ff:ff:ff:ff:ff:ff

Only the first bridge (xenbr0) has the MTU set to 9000. All other bridges have MTU set to 1500 (the default).

Attempt to Set MTU Manually

Run the following command to see if I set the MTU directly on the second bridge (xenbr1):

ip link set xenbr1 mtu 9000

The expected output is:

RTNETLINK answers: Invalid argument

See “Oracle VM: Change MTU Value Failed with 'Invalid argument' Running Kernel UEK5 on VM (Doc ID 2478446.1)” for a possible hit.

Check Linux Version

Run the following command to find the version of the kernel:

uname -a

The expected output is:

Linux victoria 4.1.12-94.6.4.el6uek.x86_64 #2 SMP Tue Aug 15 14:24:27 PDT 2017 x86_64 x86_64 x86_64 GNU/Linux

The kernel (4.1.12-94) is old enough to be affected by this bug.

Find the NIC Setting

Run the following command to see if the flag (scatter-gather) is not set:

ethtool -k eth1 | grep sca

The expected output is:

scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed]

No fix is applicable because the flag is set on.