[ILUG] TOTD: multiple gateway routing using iptables and iproute

[Stephen Shirley]

Ok,
	Here's the setup. I have 2 broadband connections here, and i want 
traffic to default to one, but the traffic from specific hosts to 
default to the other. In normal circumstances, this isn't possible; in 
order to route stuff to the net, you hae to set a default route, and 
having multiple default routes doesn't make any sense, the first one 
will always get used. So, what do you do? First, you add a rule to the 
PREROUTING chain of the mangle table, like this:

iptables -t mangle -A PREROUTING -s 10.0.0.45 -j MARK --set-mark 1

where 10.0.0.45 is the client i want to use the 2nd connection. Now 
we've marked the traffic (only internal to the kernel, the packet itself 
isn't modified), next, do something based on that mark.

ip rule add fwmark 1 table 1

This tells the kernel that any packets with the mark no. 1 should be 
routed using table 1 instead of the default routing table (didn't know 
there were multiple routing tables? Neither did i till i started poking 
at this -). Table 1 in this case should be exactly the same as the 
default routing table, but with a different default route. For example:

root at fluff:~(0)# ip route
195.218.116.25 dev ppp0  scope link
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.1
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.107
default via 195.218.116.25 dev ppp0

root at fluff:~(0)# ip route ls table 1
195.218.116.25 dev ppp0  scope link
10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.1
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.107
default via 192.168.1.1 dev eth1

Compare the last line of each. There's one last gotcha to be overcome. 
It's an option called route filtering, and it's set by default in the 
kernel. Basically, what route filtering means is, if traffic arrives on 
an interface over which a reply wouldn't normally be routed, the kernel 
will drop it. However, this fscks up things for the above setup, so 
you'll need to echo 0 into the appropiate interfaces for your net 
connections:

echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/ppp0/rp_filter

in my case. Make this permenant by editing /etc/sysctl.conf.

Right, that's about it. I have some basic scripts written to monitor 
both connections, they're fairly straightforward, you just need to make 
sure you set the source addr of ping to the interface you want to test, 
and ensure that the kernel will route the traffic out the right link -)

Steve