How to configure network bonding in Linux
Tags: bonding, CentOS, cisco, gentoo, ios, lacp, linux, networking, redhat
(Update: Feb 5, 2010 – I even more recently obtained a Cisco IOS switch and have included the configuration bits for IOS below.)
I recently obtained a Dell PowerConnect 5224 Gigabit switch which has the ability to combine multiple twisted-pair or fiber Ethernet links into one fault-tolerant and load balanced logical link. It also appears that its configuration syntax is very similar to that of a Cisco switch. In Linux this is called bonding, in switches its commonly referred to as a port channel. Either way, Its using the LACP (802.1ad) Protocol behind the scenes.
Configuring the switch for LACP bonding
Cisco IOS switch LACP configuration
Enabling LACP across two ports in IOS is pretty straightforward. The first thing to do is associate the ports with the channel-group. This is good to do early so that when you apply switchport parameters to the Port-channel interface it automagically applies them to the GigabigEthernet interfaces.
Here are the relevant portions of my running configuration.
interface Port-channel2 description LACP Channel for mk2 switchport trunk encapsulation dot1q switchport trunk allowed vlan 1,2 switchport mode trunk spanning-tree portfast trunk ! interface GigabitEthernet1/0/23 description mk2 eth0 switchport trunk encapsulation dot1q switchport mode trunk channel-group 2 mode active ! interface GigabitEthernet1/0/24 description mk2 eth1 switchport trunk encapsulation dot1q switchport mode trunk channel-group 2 mode active !
Dell PowerConnect switch LACP configuration
The Dell switch configuration is surprisingly easy. A port-channel is automatically created when the Linux host brings up it’s bond interface(s). Just figure out which ports you want to use for your bond and enable LACP on them. I used ports 1/23 and 1/24 (ports 23 & 24 on switch 1).
Vty-0#config Vty-0(config)#interface ethernet 1/23 Vty-0(config-if)#lacp Vty-0(config-if)#exit Vty-0(config)#interface ethernet 1/24 Vty-0(config-if)#lacp Vty-0(config-if)#exit
’show run’ now indicates that the selected ports are LACP enabled.
Vty-0#show run building running-config, please wait..... ... ! interface ethernet 1/23 switchport allowed vlan add 1 untagged switchport native vlan 1 lacp ! interface ethernet 1/24 switchport allowed vlan add 1 untagged switchport native vlan 1 lacp ! ...
At this point your port-channel will be down. Don’t worry, it will automagically come up when the Linux host brings up the bond interface. You can verify that its down by issuing the following:
Vty-0#show interfaces status port-channel 1 % Trunk 1 does not exist.
Note: This assumes you have no pre-existing port-channels, if you do have other port-channels configured you should iterate the port-channel number to be one more than the number of already defined port-channels.
Configuring the Linux host for LACP bonding:
There are a few places where you define the parameters of the bond. The kernel module defies the protocol, frequency and other attributes of the low-level bond channel configuration. The command ifenslave will create a bond device and allow you to manage the Ethernet devices within it (add/remove,etc.). Finally the network address configuration is handled by ifconfig, consistent with most other network interfaces in Linux. Luckily most of this is taken care of automatically by the networking init scripts.
Linux Kernel Module Configuration
LACP is referred to in linux as bonding mode 4, so we need to inform the kernel module to use this bonding mode. We’ll also pass it a few other parameters like the frequency of which to scan for changes in status.
Add the following to your module config file, in gentoo this is /etc/modules.autoload.d/kernel-2.6. This will pass the following options to the kernel module the next time it is inserted.
Red Hat and CentOS Kernel Module Configuration
#/etc/modprobe.conf alias bond0 bonding options bond0 miimon=100 mode=4 lacp_rate=1
Gentoo Kernel Module Setup
#/etc/modules.autoload.d/kernel-2.6 bonding miimon=100 mode=4 lacp_rate=1
Linux Network Configuration
Red Hat and CentOS Network Setup
#/etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 ONBOOT=yes BOOTPROTO=none USERCTL=no MASTER=bond0 SLAVE=yes
#/etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE=eth1 ONBOOT=yes BOOTPROTO=none USERCTL=no MASTER=bond0 SLAVE=yes
#/etc/sysconfig/network-scripts/ifcfg-bond0 DEVICE=bond0 IPADDR=10.0.0.80 NETMASK=255.255.255.0 BROADCAST=10.0.0.255 GATEWAY=10.0.0.1 ONBOOT=yes BOOTPROTO=none USERCTL=no
Gentoo Network Setup
#/etc/conf.d/net config_eth0=( "null" ) config_eth1=( "null" ) slaves_bond0="eth0 eth1" config_bond0=( "10.0.0.80/24" )
We also need to create a symlink in /etc/init.d for the new bond0 interface and turn off eth0 as it is controlled by the bond now. The following will disable eth0 and enable bond0 on boot.
cd /etc/init.d ln -s net.lo net.bond0 rc-update del eth0 default rc-update add bond0 default
Now you can bring up the bond interface.
/etc/init.d/net.bond0 start
Checking the Status of the bonded interface
You can check the status of your bond now from within Linux by using the /proc and /sys interfaces into the Linux bond driver.
$ cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.1.1 (September 26, 2006) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: fast Active Aggregator Info: Aggregator ID: 1 Number of ports: 2 Actor Key: 17 Partner Key: 1 Partner Mac Address: 00:77:54:71:a8:6f Slave Interface: eth0 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:99:97:60:9d:48 Aggregator ID: 1 Slave Interface: eth1 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:00:85:60:9d:49 Aggregator ID: 1
You can check the bond from the switch.
Cisco IOS
Switch#show interfaces Port-channel 2
Port-channel2 is up, line protocol is up (connected)
Hardware is EtherChannel, address is 001b.0dbf.ba17 (bia 001b.0dbf.ba17)
Description: LACP Channel for mk2
MTU 1500 bytes, BW 2000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Full-duplex, 1000Mb/s, link type is auto, media type is unknown
input flow-control is off, output flow-control is unsupported
Members in this channel: Gi1/0/23 Gi1/0/24
ARP type: ARPA, ARP Timeout 04:00:00
Last input 1d23h, output 00:00:01, output hang never
Last clearing of "show interface" counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 5000 bits/sec, 7 packets/sec
1060041 packets input, 193406916 bytes, 0 no buffer
Received 18241 broadcasts (0 multicast)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 11873 multicast, 0 pause input
0 input packets with dribble condition detected
3181997 packets output, 2735804051 bytes, 0 underruns
0 output errors, 0 collisions, 1 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 PAUSE output
0 output buffer failures, 0 output buffers swapped outDell PowerConnect:
Vty-0#show interfaces status port-channel 1 Information of Trunk 1 Basic information: Port type: 1000t Mac address: 00-30-F1-71-A8-82 Configuration: Name: Port admin: Up Speed-duplex: Auto Capabilities: 10half, 10full, 100half, 100full, 1000full, Flow control: Disabled Current status: Created by: Lacp Link status: Up Port operation status: Up Operation speed-duplex: 1000full Flow control type: None Member Ports: Eth1/23, Eth1/24,
That’s is all the configuration work that I needed to perform, I hope it saves you time (I spent a while digging through dell’s site and the linux kernel docs to find the right combination of options). Please, let me know if you had troubles with these directions or if you have questions. keith (at) backdrift.org
July 2nd, 2010 at 7:27 am
Hello,
there is a problem in the file modprobe.conf, you write “options bond0..” but it is “options bonding…”
thanks for this howto