LACP - How to Configure Network Bonding in Linux

LACP – How to Configure Network Bonding in Linux

Tags: bonding, CentOS, cisco, Debian, gentoo, ios, lacp, linux, networking, redhat

(Update: Feb 5, 2010 – I even more recently obtained a Cisco IOS switch and have included the configuration bits for IOS below.)

I recently obtained a Dell PowerConnect 5224 Gigabit switch which has the ability to combine multiple twisted-pair or fiber Ethernet links into one fault-tolerant and load balanced logical link. It also appears that its configuration syntax is very similar to that of a Cisco switch. In Linux this is called bonding, in switches its commonly referred to as a port channel. Either way, Its using the LACP (802.3ad) Protocol behind the scenes.

Configuring the switch for LACP bonding

Cisco IOS switch LACP configuration

Enabling LACP across two ports in IOS is pretty straightforward. The first thing to do is associate the ports with the channel-group. This is good to do early so that when you apply switchport parameters to the Port-channel interface it automagically applies them to the GigabigEthernet interfaces.

Here are the relevant portions of my running configuration.

interface Port-channel2
 description LACP Channel for mk2
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 1,2
 switchport mode trunk
 spanning-tree portfast trunk
!
interface GigabitEthernet1/0/23
 description mk2 eth0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 2 mode active
!
interface GigabitEthernet1/0/24
 description mk2 eth1
 switchport trunk encapsulation dot1q
 switchport mode trunk
 channel-group 2 mode active
!

Dell PowerConnect switch LACP configuration

The Dell switch configuration is surprisingly easy. A port-channel is automatically created when the Linux host brings up it’s bond interface(s). Just figure out which ports you want to use for your bond and enable LACP on them. I used ports 1/23 and 1/24 (ports 23 & 24 on switch 1).

Vty-0#config
Vty-0(config)#interface ethernet 1/23
Vty-0(config-if)#lacp
Vty-0(config-if)#exit
Vty-0(config)#interface ethernet 1/24
Vty-0(config-if)#lacp
Vty-0(config-if)#exit

‘show run’ now indicates that the selected ports are LACP enabled.

Vty-0#show run
building running-config, please wait.....
...
!
interface ethernet 1/23
 switchport allowed vlan add 1 untagged
 switchport native vlan 1
 lacp
!
interface ethernet 1/24
 switchport allowed vlan add 1 untagged
 switchport native vlan 1
 lacp
!
...

At this point your port-channel will be down. Don’t worry, it will automagically come up when the Linux host brings up the bond interface. You can verify that its down by issuing the following:

Vty-0#show interfaces status port-channel 1
% Trunk 1 does not exist.

Note: This assumes you have no pre-existing port-channels, if you do have other port-channels configured you should iterate the port-channel number to be one more than the number of already defined port-channels.

Configuring the Linux host for LACP bonding:

There are a few places where you define the parameters of the bond. The kernel module defies the protocol, frequency and other attributes of the low-level bond channel configuration. The command ifenslave will create a bond device and allow you to manage the Ethernet devices within it (add/remove,etc.). Finally the network address configuration is handled by ifconfig, consistent with most other network interfaces in Linux. Luckily most of this is taken care of automatically by the networking init scripts.

Linux Kernel Module Configuration

LACP is referred to in linux as bonding mode 4, so we need to inform the kernel module to use this bonding mode. We’ll also pass it a few other parameters like the frequency of which to scan for changes in status.

Add the following to your module config file, in gentoo this is /etc/modules.autoload.d/kernel-2.6. This will pass the following options to the kernel module the next time it is inserted.

Red Hat and CentOS Kernel Module Configuration

#/etc/modprobe.conf
 
alias bond0 bonding
options bond0 miimon=100 mode=4 lacp_rate=1

RHEL 6 and CentOS 6 Kernel Module Configuration

#/etc/modprobe.d/bonding.conf
 
#Deprecated syntax
#alias bond0 bonding  
 
#Updated syntax
alias netdev-bond0 bonding
options bond0 miimon=100 mode=4 lacp_rate=1

Debian Kernel Module Configuration

# /etc/modules: kernel modules to load at boot time.
 
bonding mode=4 miimon=100 lacp_rate=1

Ubuntu Kernel Module Configuration

# /etc/modprobe.d/bonding.conf
 
bonding mode=4 miimon=100 lacp_rate=1

Gentoo Kernel Module Setup

#/etc/modules.autoload.d/kernel-2.6
 
bonding miimon=100 mode=4 lacp_rate=1

Linux Network Configuration

Red Hat and CentOS Network Setup

#/etc/sysconfig/network-scripts/ifcfg-eth0
 
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes

#/etc/sysconfig/network-scripts/ifcfg-eth1
 
DEVICE=eth1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes

#/etc/sysconfig/network-scripts/ifcfg-bond0
 
DEVICE=bond0
IPADDR=10.0.0.80
NETMASK=255.255.255.0
BROADCAST=10.0.0.255
GATEWAY=10.0.0.1
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

Debian / Ubuntu Network Setup

#/etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
 
auto bond0
iface bond0 inet static
	address 10.0.0.80
	gateway 10.0.0.1
	broadcast 10.0.0.255
	netmask 255.255.255.0
	up /sbin/ifenslave bond0 eth1 eth2
	down /sbin/ifenslave -d bond0 eth0 eth1

*Note* This is dependant upon the ifenslave package, to install run the following:

apt-get install ifenslave

Ubuntu 10.04 or newer support an updated interfaces(5) syntax

#/etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
 
auto eth0
    iface eth0 inet manual
    bond-master bond0
 
auto eth1
     iface eth1 inet manual
     bond-master bond0
 
auto bond0
     iface bond0 inet static
     address 10.0.0.80
     gateway 10.0.0.1
     netmask 255.255.255.0
 
 
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate 4
bond-slaves none

Gentoo LACP bonding Setup

#/etc/conf.d/net
 
config_eth0=( "null" )
config_eth1=( "null" )
 
slaves_bond0="eth0 eth1"
 
config_bond0=( "10.0.0.80/24" )

We also need to create a symlink in /etc/init.d for the new bond0 interface and turn off eth0 as it is controlled by the bond now. The following will disable eth0 and enable bond0 on boot.

  cd /etc/init.d
  ln -s net.lo net.bond0
  rc-update del eth0 default
  rc-update add bond0 default

Now you can bring up the bond interface.

  /etc/init.d/net.bond0 start

Checking the Status of the bonded LACP interface

You can check the status of your bond now from within Linux by using the /proc and /sys interfaces into the Linux bond driver.

$ cat /proc/net/bonding/bond0  
 
Ethernet Channel Bonding Driver: v3.1.1 (September 26, 2006)
 
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
 
802.3ad info
LACP rate: fast
Active Aggregator Info:
	Aggregator ID: 1
	Number of ports: 2
	Actor Key: 17
	Partner Key: 1
	Partner Mac Address: 00:77:54:71:a8:6f
 
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:99:97:60:9d:48
Aggregator ID: 1
 
Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:00:85:60:9d:49
Aggregator ID: 1

You can check the bond from the switch.

Cisco IOS

Switch#show interfaces Port-channel 2
Port-channel2 is up, line protocol is up (connected)
  Hardware is EtherChannel, address is 001b.0dbf.ba17 (bia 001b.0dbf.ba17)
  Description: LACP Channel for mk2
  MTU 1500 bytes, BW 2000000 Kbit, DLY 10 usec, 
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Full-duplex, 1000Mb/s, link type is auto, media type is unknown
  input flow-control is off, output flow-control is unsupported 
  Members in this channel: Gi1/0/23 Gi1/0/24 
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 1d23h, output 00:00:01, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 5000 bits/sec, 7 packets/sec
     1060041 packets input, 193406916 bytes, 0 no buffer
     Received 18241 broadcasts (0 multicast)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 11873 multicast, 0 pause input
     0 input packets with dribble condition detected
     3181997 packets output, 2735804051 bytes, 0 underruns
     0 output errors, 0 collisions, 1 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out

Dell PowerConnect:

Vty-0#show interfaces status port-channel 1
Information of Trunk 1
 Basic information: 
  Port type: 1000t
  Mac address: 00-30-F1-71-A8-82
 Configuration: 
  Name: 
  Port admin: Up
  Speed-duplex: Auto
  Capabilities: 10half, 10full, 100half, 100full, 1000full, 
  Flow control: Disabled
 Current status: 
  Created by: Lacp
  Link status: Up
  Port operation status: Up
  Operation speed-duplex: 1000full
  Flow control type: None
  Member Ports: Eth1/23, Eth1/24,

That’s is all the configuration work that I needed to perform. I hope it saves you time. I spent a while digging through dell’s site and the linux kernel docs to find the right combination of options. Please let me know if you had troubles with these directions or if you have questions. keith (at) backdrift.org

Additional docs on linux bonding driver https://www.kernel.org/doc/Documentation/networking/bonding.txt