Access denied. You must login to view this page.
29
Jan 2010

Enabling jumbo frames in VMware ESX 4

comment icon9 comment(s) |

This topic has been around for awhile but, since we provide EqualLogic iSCSI SANs as part of our product offering, I thought I would write about it.  Jumbo frames are units that contain 9,000 bytes of data instead of the usual 1500, or an MTU of 9000 instead of 1500.  When using the ESX software iSCSI initiator, a network adapter becomes the transport device for your iSCSI traffic.  When creating a virtual switch using the vSphere client, you cannot change the default MTU value of 1500, nor can you even view the values.  These changes must be made through the Service Console.

First, you must create the vSwitch and change the MTU to 9000.  For this example, vSwitch9 will be the name you would replace with your own.

esxcfg-vswitch -a vSwitch9

Then, set the MTU of the vSwitch.

esxcfg-vswitch -m 9000 vSwitch9

esxcfg-vswitch -l will list all the vSwitches so you can confirm your changes and settings.

Switch Name    Num Ports   Used Ports  Configured Ports  MTU     Uplinks
vSwitch9             64                  1            64                 9000

As you can see, the Uplinks field is blank, so you must add a physical NIC to the virtual switch, but first we'll do a few other things.  iSCSI access is controlled by a VMkernel interface and assigned to a port group on the vSwitch.

To create the portgroup:

esxcfg-vswitch -A <port group name> vSwitch9

Then create the VMkernel interface:

esxcfg-vmknic -a -i <your iSCSI initiator IP address> -n <subnet mask> -m 9000 <port group name from the above step>

If you named the port group "iSCSI" and your IP is 172.16.0.1/24, the command would like this:

esxcfg-vmknic -a -i 172.16.0.1 -n 255.255.255.0 -m 9000 iSCSI

esxcfg-vmknic -l will confirm your settings.

The last step is to add a physical NIC to the vSwitch.  This can be done via the GUI, but we are having fun here, so let's do it with commands.

esxcfg-vswitch -L <physical NIC, such as vmnic1> vSwitch9

You're done!  You can now refresh the Networking portion of the Configuration tab in the vSphere Client to see the new virtual switch components.  Keep in mind that you must enable jumbo frames on your physical switch for this to work.

 

 

Subscribe to our Networks

IW on Facebook

Comments

Thanks - Very Simple

Thanks for the info, it was very helpful.
"esxcfg-cfg -l will list all the vSwitches so you can confirm your changes and settings." This did not work for me but "esxcfg-vSwitch -l" did.

Thanks again

Very good explanation...

Very good explanation... thank you! I followed your instructions to enable Jumbo Frames on existing vswitches and vmkernel ports. I just wanted to point out that if the vmkernel ports (in my case they're used for iSCSI & VMotion) already exist, they can be modified. You don't have to delete and recreate them. Use the command:

esxcfg-vmknic -m 9000 "VMkernel iSCSI 2"

Use quotes if name has spaces, otherwise don't.

Thanks again

I did exactly what you told

I did exactly what you told and now I can see that the vSwitch is configured to MTU 9000 but I still can't get my two clients to deliver packet larger then 1500 bytes between them (they are both set to MTU 9000 as well). Of course they are both on the same vm network with this switch.

Are you sure that ESX support Jumbo frames? when you actually deliver packets with larger the 1500 bytes? let's say: ping -s 6000 x.x.x.x ?
6000 is just and example.

Thanks

Hi Dani, thanks for your

Hi Dani, thanks for your feedback. Jumbo frames are absolutely supported on ESX. Jumbo frames are intended for iSCSI traffic from your host to your storage array via a VMkernel connection, not for traffic from host-to-host. Normal network data traffic does not have the overhead of iSCSI packet encapsulation so jumbo frames are not needed. I am able to successfully use vmkping -s 6000 my.iscsi.target.ip with no packet loss.

Thank you for your fast

Thank you for your fast response.
I solved the problem, I was talking about regular network packets (tcp/ip) and not iSCASI traffic.
Only when I configures my network cards as VMXNET 3 and put both my hosts on the same vSwitch (with MTU 9000 as you explained) I manged to create host to host traffic with packets larger then 1500 bytes.

10x

check out setting

You have to use :
ping -s 8000 -d x.x.x.x (don't forget the -d parameter)
If you get an valid reply jumbo frames are working.
If you get back "sendto() failed (Message too long)" something is still wrong.
Check
- esxcfg-vswitch -l
- esxcfg-vmknic -l
on esx side.

  • Check MTU Size on physical switches
  • Check mtu size on the host you want to reach with jumbo frames.

Thank you!

Thanks very much for this Kelly :)

It will really help people to

It will really help people to understand of how to enable jumbo frames in VMware ESX 4 and with such a detailed explanation I don’t think that there is any chance of getting confused about how to go about it!!I was planning to use the vCloud Datacenter, before i read the post. Right from explaining how to create the vSwitch and changing the MTU to 9000 to creating the VMkernel interface and confirming your settings, this is really going to make working this out to be some fun!!

Enabling jumbo frames in VMware ESX 4

I am new to this, and can't figure out how are were to "type " the command in?