To follow myself up, I've more or less cobbled together a solution for the
scenario below.
The problem was that traffic was not getting balanced across multiple
uplinks, only a single path was getting installed in the route cache as
the traffic was being seen as a single flow. Traffic also needed to be
NATed on egress so that each uplink used its own local endpoint as the
source address (to prevent traffic getting RPFed upstream).
Getting around this involved re-inventing the wheel with four corners:
- iptables and the statistic extension to mark packets with a
fwmark 1..n where n is the number of uplinks.
- Set up n routing tables, with default routes that have n
next hops, shuffled to balance traffic.
- Adding policy routes to distribute traffic over these tables
based on the fwmark assigned above.
- Finally, Stateless NAT (a fairly recent addition to the
kernel and iproute2 package) to rewrite the source address
to that of the uplink's local endpoint.
All very ugly, messy, and inelegant, but it works - as long as
the traffic we're messing with only originates from a single source
address.
It all would have been much easier if the uplinks in question supported
MPPP, but the mobile operators stateside appear to have forgotten it
exists...
-Ronan
On Mon, 1 Sep 2008, Ronan Mullally wrote:
> A back to school question for everybody ;-)
>> I've got a scenario as shown below:
>> +-----------+ +-----------+
> | linux src |=========== Internet -------| linux dst |
> +-----------+ +-----------+
>> There are two (or more) paths from the src box to the Internet. It is
> sending traffic to the dst box, and wants to balance UDP traffic across
> these multiple links. Traffic in the other direction is not an issue.
>> Sane options like Multilink PPP are not avaialble. Less sane options like
> running IPSEC/VPN tunnels are not available. The option I'm left with is
> to load balance outbound traffic across the links. The insane twist is
> that these packets will need to be source NATed on egress, but let's
> ignore that for the moment.
>> I'm trying to use the equalize flag to the ip route command. I can set up
> a multipath route okay:
>> # ip route add 172.16.0.0/24 equalize nexthop dev dummy1 nexthop dev dummy2
>> # ip route show
> 172.16.0.0/24 equalize
> nexthop dev dummy1 weight 1
> nexthop dev dummy2 weight 1
>> And would expect to see a ping to a host in the above range to send
> packets via each link in turn. It doesn't. It sends all traffic via the
> first path, so the routing decision is getting cached rather than
> re-calculated per packet.
>> The man page for 'ip' indicates that the kernel needs to be patched to
> support equalisation. I was hoping that CONFIG_IP_ROUTE_MULTIPATH would
> be sufficient, but it's not. I've found the patch I think the man page
> refers to at http://trash.net/~kaber/equalize/equalize_2.4.18.patch but
> it doesn't apply cleanly against a recent 2.6 kernel.
>> Am I missing something obvious, or can anybody think of a more elegant
> solution? The only other alternative I can think of (apart from trying
> to re-write the patch for 2.6) is the eql driver in the kernel, but that
> looks well and truly dead (circa 1995) and I don't think it'll work in a
> reverse-path-filtering environment.
>>> -Ronan
>>
Maintained by the ILUG website team. The aim of Linux.ie is to
support and help commercial and private users of Linux in Ireland. You can
display ILUG news in your own webpages, read backend
information to find out how. Networking services kindly provided by HEAnet, server kindly donated by
Dell. Linux is a trademark of Linus Torvalds,
used with permission. No penguins were harmed in the production or maintenance
of this highly praised website. Looking for the
Indian Linux Users' Group? Try here. If you've read all this and aren't a lawyer: you should be!
