Connecting the world…

port-channel

Port-channel configuration for VMWare

I received some e-mails from people asking for configuration examples for Cisco switch in conjunction with VMWare servers. That is why I post the configuration (I normally use) beneath. This configuration enables a 802.1Q trunk connection between the switch and the VMWare server. This configuration requires the VMWare server to use VLAN tagging. The Port-channel consist two physical GigabitEthernet interfaces.

Configuration Example:

port-channel load-balancing src-dst-ip
!
interface Port-channel1
description 802.1Q to VMWare
switchport trunk encapsulation dot1q
switchport nonegotiate
switchport mode trunk
spanning-tree portfast trunk
!
interface GigabitEthernet1/0/1
description Member Po1
switchport trunk encapsulation dot1q
switchport nonegotiate
switchport mode trunk
no cdp enable
channel-group 1 mode on
spanning-tree portfast trunk
!
interface GigabitEthernet2/0/1
description Member Po1
switchport trunk encapsulation dot1q
switchport nonegotiate
switchport mode trunk
no cdp enable
channel-group 1 mode on
spanning-tree portfast trunk

CDP is the Cisco propriatery Cisco Discovery Protocol. CDP can be usefull when trying to discover attached network components. VMWare supports CDP, so it could be enabled on the interfaces. The usage of CDP can help to see which switch port connects who which NIC on the ESX server.

Port-channel Cisco vs. VMware ESX

I have had different discussions with different customers about the load-balancing algorithms between a Cisco switch, configured with a port-channel and a VMware ESX server using multiple NICs. Our VMware consultants always choose Route based on IP hashes as load-balancing algorithm. This means that load-balancing happens on layer 3 of the OSI model (source-destination-IP).

In my opinion, the switch should be configured the same way. Depending on the model switch, you can have different default load-balancing algoritmhs. For example, the Cisco Catalyst 3750 uses src-mac load-balancing and the Cisco Catalyst 6500 use src-dst-ip load-balancing. You can check the configured load-balancing algorithm with the following command:

show etherchannel load-balancing

If you would like you change the load-balancing algorithm you can use the global configuration command:

port-channel load-balancing <option>

Be aware that this is a global configuration command, so it affects all the configured port-channels on the switch.

To check the load-balancing between the different NICs, you should have a tool to look at real-time bandwidth statistics. I normally use the tool SNMP Traffic Grapher to monitor the different switch ports. On the ESX console you can check the load-balancing with the commands:

  1. esxtop [enter]
  2. s2 (schedule interval of 2 seconds) [enter]
  3. n [network]

The load should be spread fairly even across the different switch ports en vmnics.