Table of Contents
See Section D.3.3, “Joel Knight's license on the CARP article” for the license of this chapter.
CARP is the Common Address Redundancy Protocol. Its primary purpose is to allow multiple hosts on the same network segment to share an IP address. CARP is a secure, free alternative to the Virtual Router Redundancy Protocol and the Hot Standby Router Protocol.
CARP works by allowing a group of hosts on the same network segment to share an IP address. This group of hosts is referred to as a "redundancy group". The redundancy group is assigned an IP address that is shared amongst the group members. Within the group, one host is designated the "master" and the rest as "backups". The master host is the one that currently "holds" the shared IP; it responds to any traffic or ARP requests directed towards it. Each host may belong to more than one redundancy group at a time.
One common use for CARP is to create a group of redundant firewalls. The virtual IP that is assigned to the redundancy group is configured on client machines as the default gateway. In the event that the master firewall suffers a failure or is taken offline, the IP will move to one of the backup firewalls and service will continue unaffected.
While highly redundant and fault-tolerant hardware minimizes the need for CARP, it doesn't erase it. There's no hardware fault tolerance that's capable of helping if someone knocks out a power cord, or if your system administrator types reboot in the wrong window. CARP also makes it easier to make the patch and reboot cycle transparent to users, and easier to test a software or hardware upgrade--if it doesn't work, you can fall back to your spare until fixed.
There are, however, situations in which CARP won't help. CARP's design does require that the members of a group be on the same physical subnet with a static IP address, although with the introduction of the carpdev directive, there is no more need for IP addresses on the physical interfaces. Similarly, services that require a constant connection to the server (such as SSH or IRC) will not be transparently transferred to the other system--though in this case, CARP can help with minimizing downtime. CARP by itself does not synchronize data between applications, for example, manually duplicating data between boxes with rsync, or whatever is appropriate for your application.
CARP supports both IPv4 and IPv6.
The master host in the group sends regular advertisements to the local network so that the backup hosts know it's still alive. If the backup hosts don't hear an advertisement from the master for a set period of time, then one of them will take over the duties of master (whichever backup host has the lowest configured advbase and advskew values). It's possible for multiple CARP groups to exist on the same network segment. CARP advertisements contain the Virtual Host ID which allows group members to identify which redundancy group the advertisement belongs to.
In order to prevent a malicious user on the network segment from spoofing CARP advertisements, each group can be configured with a password. Each CARP packet sent to the group is then protected by an SHA1 HMAC.
Each redundancy group is represented by a carp(4) virtual network interface. As such, CARP is configured using ifconfig(8) The follow options are available:
carpNThe name of the carp(4) virtual interface where N is a integer that represents the interface's number (e.g. carp0).
vhidThe Virtual Host ID. This is a unique number that is used to identify the redundancy group to other nodes on the network. Acceptable values are from 1 to 255. This allows for multiple redundancy groups to exist on the same network.
passwordThe authentication password to use when talking to other CARP-enabled hosts in this redundancy group. This must be the same on all members of the redundancy group.
carpdevThis optional parameter specifies the physical network interface that belongs to this redundancy group. By default, CARP will try to determine which interface to use by looking for a physical interface that is in the same subnet as the ipaddress and mask combination given to the carp(4) interface.
advbaseThis optional parameter specifies how often, in seconds, to advertise that we're a member of the redundancy group. The default is 1 second. Acceptable values are from 1 to 255.
advskewThis optional parameter specifies how much to skew the advbase when sending CARP advertisements. By manipulating advbase, the master CARP host can be chosen. The higher the number, the less preferred the host will be when choosing a master. The default is 0. Acceptable values are from 1 to 254.
stateForce a carp(4) interface into a certain state. Valid states are init, backup, and master
ipaddressThis is the shared IP address assigned to the redundancy group. This address does not have to be in the same subnet as the IP address on the physical interface (if present). This address needs to be the same on all hosts in the group, however.
maskThe subnet mask of the shared IP.
Further CARP behaviour can be controlled via sysctl(8)
net.inet.carp.allowAccept incoming CARP packets or not. Default is 1 (yes).
net.inet.carp.preemptAllow hosts within a redundancy group that have a better advbase and advskew to preempt the master. In addition, this option also enables failing over all interfaces in the event that one interface goes down. If one physical CARP-enabled interface goes down, CARP will change advskew to 240 on all other CARP-enabled interfaces, in essence, failing itself over. This option is 0 (disabled) by default.
net.inet.carp.logLog bad CARP packets. Default is 0 (disabled).
net.inet.carp.arpbalanceLoad balance traffic across multiple redundancy group hosts. Default is 0 (disabled). See carp(4) for more information.
CARP support is not enabled by default.
To use carp(4) you need a kernel with support for the
carp pseudo-device. Make sure the following
line is in your kernel configuration file:
pseudo-device carp # CARP
After configuring the carp
pseudo-device in your kernel configuration, you must recompile your
kernel and reboot to enable carp(4) support.
An example CARP configuration:
#sysctl -w net.inet.carp.allow=1#ifconfig carp0 create#ifconfig carp0 vhid 1 pass lanpasswd \ carpdev em0 advskew 100 10.0.0.1 255.255.255.0
This sets up the following:
Enables receipt of CARP packets (this is the default setting)
Creates a carp(4) interface.
Configures carp0 for virtual host #1, enables a password(lanpasswd), sets em0 as the interface belonging to the group, and makes this host a backup due to the advskew of 100 (assuming of course that the master is set up with an advskew less than 100). The shared IP assigned to this group is 10.0.0.1/255.255.255.0.
Running ifconfig on carp0 shows the status of the interface:
#ifconfig carp0carp0: flags=8802<UP,BROADCAST,SIMPLEX,MULTICAST> mtu 1500 carp: BACKUP carpdev em0 vhid 1 advbase 1 advskew 100 inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255
The following example creates a cluster of two highly-available, redundant firewalls. The following diagram presents what we're trying to achieve:
+----| WAN/Internet |----+
| |
em1| |em1
+-----+ +-----+
| fw1 | | fw2 |
+-----+ +-----+
em0| |em0
| |
---+-------Shared LAN-------+---
Both firewalls are connected to the LAN on em0 and to a WAN/Internet connection on em1. IP addresses are as follows:
Firewall 1 (fw1) em0: 172.16.0.1
Firewall 1 (fw1) em1: 192.0.2.1
Firewall 2 (fw2) em0: 172.16.0.2
Firewall 2 (fw2) em1: 192.0.2.2
The IP addresses we wish to share between the redundancy groups:
WAN/Internet Shared IP: 192.0.2.100
LAN Shared IP: 172.16.0.100
The network policy is that Firewall 1 (fw1) will be the preferred master.
The following configuration is for Firewall 1 (fw1):
#Enable preemption and group interface failover#sysctl -w net.inet.carp.preempt=1
#Configure CARP on the LAN side#ifconfig carp0 create#ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \ 172.16.0.100 255.255.255.0
#Configure CARP on the WAN side#ifconfig carp1 create#ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \ 192.0.2.100 255.255.255.0
As mentioned before, our policy is for Firewall 1 to be the
preferred master. When configuring Firewall 2 we make the
advskew a higher value since it's less
preferred to be the master.
The following configuration is for Firewall 2 (fw2):
#Enable preemption and group interface failover#sysctl -w net.inet.carp.preempt=1
#Configure CARP on the LAN side#ifconfig carp0 create#ifconfig carp0 vhid 1 pass lanpasswd carpdev em0 \ advskew 128 172.16.0.100 255.255.255.0
#Configure CARP on the WAN side#ifconfig carp1 create#ifconfig carp1 vhid 2 pass wanpasswd carpdev em1 \ advskew 128 192.0.2.100 255.255.255.0
There can be times when it's necessary to failover or demote the master node on purpose. Examples include taking the master node down for maintenance or when troubleshooting a problem. The objective here is to gracefully fail over traffic to one of the backup hosts so that users do not notice any impact.
To failover, shut down the carp(4) interface on the master node. This will cause the master to advertise itself with an "infinite" advbase and advskew. The backup host(s) will see this and immediately take over the role of master.
#ifconfig carp0 down