How to: (Linux router utilizing multiple uplink providers)

Utilizing Multiple DSL Gateways to get combined into Single Gateway

Multiple DSL connection often exist in corporate scenario due to higher gap in prices between DSL plan and Leased Line plans and we also find people struggling with connectivity issues due to downlink with any of the DSL connection, while other connections of them might be working due to failover shifting etc. After struggling with the same issue for a while, few internet resources helped us to devise a solution for the same. A Linux based Software router utilizing all the connections at the same time. Before the implementation details, let’s put a FAQ over here.

Que: Why we need a Linux based router, when already our DSLs are acting as hardware based router from our service provider?

Answer: There are multiple reasons for the same:

1. Connection Reliability & Failover handling: You might be having multiple connections to internet, but still might be struggling due to downlink with any of the uplink as switching mechanism from one uplink is sure going to cumbersome in any environment. Automatic failover handling makes it extremely attractive solution for any demanding environment. Somewhere it scores higher over a dedicated leased due to its failover handling.
   
2. Firewall/ Proxy: The same solution could provide us a mechanism to utilize our uplink connection as no more transparent to our inner network (as each request with pass through Linux router) henceforth could control the way traffic get utilized. It would be a central gateway for outgoing request.
   
3. Better throughput & Load balancing: Theoretically and practically, if one has say three modems of strength 512 Kbps each, then this solution doesn’t provide the combined strength of 1.5 Mbps, but one could attain a better throughput for multiple connections like torrents and multiple applications utilizing and enjoy connections through each of the DSL at the same time.
   

Hardware/ Software requirements:

Minimum requirements might vary, but here I am providing the requirements, which I felt as somewhat standard.

• Processor: P4 2.4 GHz or more.
  
• RAM:
  512 MB or more.
  

There are additional benefit of engaging two or more Ethernet card over the same machine, say two; one handling the requests and the other one communicating with DSLs.

For security reasons, one may wish to isolate the DSLs from the inner network and in that case, you will need a separate Switch having only DSLs and this router connected to the same and no more connection to rest of the network. Rest of the network will be connected to other Ethernet card configured in preferably different subnet.

Step-By-Step Procedure :-

Here I am presenting the way, we did the things in most simplest and layman way, without going into far technical details. Sure there might be better or alternate or customized ways suiting your own environment. One may always tweak over any solution.

Here are the steps:

1. Machine: Get a machine ready with Either of Latest Fedora, RHEL or latest of CentOS with standard installation covering all the important packages.
2. IP Forwarding: The OS need to has IP forwarding enabled. Its disabled by default. For checking the same, you may type the following:
   

   cat /proc/sys/net/ipv4/ip_forward

   If output is 1, then nothing to do and if output is 0, then it needs to be ON.
   

   For permanently putting IP Forwarding as ON, you need to change the value of net.ipv4.ip_forward to 1 from 0 in the file

   /etc/sysctl.conf. The changes could take affect by either a reboot or by the command

   sysctl –p /etc/sysctl.conf
   

3. Configure IPs for both the Ethernet cards. I am keeping mine in the same subnet. Keeping all the IPs in same subnet.

   Say my DSL IPs are [MY_DSL1_IP], [MY_DSL2_IP], [MY_DSL3_IP] and two IPs of this router are [Ethernet1_IP] and [Ethernet2_IP]. I am keeping my subnet as 172.16.x.x/16

4. In next, you need to create multiple profiles of one Ethernet card (take eth1) to communicate with each of DSL (in our case three different profiles with three different IPs).

   Make copies of file ifcfg-eth1 as ifcfg-eth1:0, ifcfg-eth1:1 and ifcfg-eth1:2. [Location /etc/sysconfig/network-scripts]

   cp ifcfg-eth1 ifcfg-eth1:0
   cp ifcfg-eth1 ifcfg-eth1:1
   cp ifcfg-eth1 ifcfg-eth1:2

   Make changes in files as given below:
   

   Content of ifcfg-eth1
   

   # Accton Technology Corporation SMC2-1211TX
   

   DEVICE=eth1
   ONBOOT=yes
   BOOTPROTO=none
   HWADDR=00:10:b5:fd:7f:e7
   NETMASK=255.255.0.0
   IPADDR=[Ethernet2_IP]

   TYPE=Ethernet
   USERCTL=no
   IPV6INIT=no
   PEERDNS=yes
   

   Content of ifcfg-eth1:0
   

   # Accton Technology Corporation SMC2-1211TX
   

   DEVICE=eth1:0
   ONBOOT=yes
   BOOTPROTO=none
   HWADDR=00:10:b5:fd:7f:e7
   NETMASK=255.255.0.0
   IPADDR=[Profile0_IP]

   TYPE=Ethernet
   USERCTL=no
   IPV6INIT=no
   PEERDNS=yes
   

   Content of ifcfg-eth1:1
   

   # Accton Technology Corporation SMC2-1211TX
   

   DEVICE=eth1:1
   ONBOOT=yes
   BOOTPROTO=none
   HWADDR=00:10:b5:fd:7f:e7
   NETMASK=255.255.0.0
   IPADDR=[Profile1_IP]
   TYPE=Ethernet
   USERCTL=no
   IPV6INIT=no
   PEERDNS=yes
   

   Content of ifcfg-eth1:2
   

   # Accton Technology Corporation SMC2-1211TX
   

   DEVICE=eth1:2
   ONBOOT=yes
   BOOTPROTO=none
   HWADDR=00:10:b5:fd:7f:e7
   NETMASK=255.255.0.0
   IPADDR=[Profile2_IP]

   TYPE=Ethernet
   USERCTL=no
   IPV6INIT=no
   PEERDNS=yes
   

5. Add the entries in your rt_tables file for your DSLs
   

   Just do vi /etc/iproute2/rt_tables and put enties, so that the file contents will look like something as given below:
   

1 dsl1
2 dsl2
3 dsl3

#
# reserved values
#
255    local
254    main
253    default
0    unspec
#
# local
#
#1    inr.ruhep

Now most important step, place the routes in the rc.local file. Just do vi /etc/rc.d/rc.local and put enties, so that the file contents will look like something as given below:

#!/bin/sh

#

# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don’t
# want to do the full Sys V style init stuff.
touch /var/lock/subsys/local
/sbin/ip route add 172.16.0.0/16 dev eth1:0 src [Profile0_IP] table dsl1
/sbin/ip route add 172.16.0.0/16 dev eth1:1 src [Profile1_IP] table dsl2
/sbin/ip route add 172.16.0.0/16 dev eth1:2 src [Profile2_IP] table dsl3
/sbin/ip route add default via [MY_DSL1_IP] table dsl1
/sbin/ip route add default via [MY_DSL2_IP] table dsl2
/sbin/ip route add default via [MY_DSL3_IP] table dsl3
/sbin/ip rule add from [MY_DSL1_IP] table dsl1
/sbin/ip rule add from [MY_DSL2_IP] table dsl2
/sbin/ip rule add from [MY_DSL3_IP] table dsl3
/sbin/ip route add default scope global nexthop via [MY_DSL1_IP] dev eth1:0 weight 1 nexthop via [MY_DSL2_IP] dev eth1:1 weight 1 nexthop via [MY_DSL3_IP] dev eth1:2 weight 1
/usr/bin/nohup usr/local/bin/gwping >>/var/log/gwping.log 2>&1

As about the last line and gwping, then it will come in later step.

1. The next part is to place a script, which will take care of failover shifting and logging. What it basically do is to ping some public website through each of DSL, determining that when any of DSL went down and switching routing rules accordingly.
   

   Here is our customized script saved as /usr/local/bin/gwping:
   

   GWCMD="/sbin/ip route replace default scope global"
   

   # Conventionally 0 indicates success in this script.
   

   # Time between checks in seconds
   

   SLEEPTIME=10
   
   

   #IP Address or domain name to ping. The script relies on the domain being
   

   #pingable and always available
   

   TESTIP=http://www.yahoo.com
   
   

   #Ping timeout in seconds
   

   TIMEOUT=2
   

   # External interfaces
   

   EXTIF1=eth1:0
   

   EXTIF2=eth1:1
   

   EXTIF3=eth1:2
   

   #IP address of external interfaces. This is not the gateway address.
   

   IP1=[Profile0_IP]
   

   IP2=[Profile1_IP]
   

   IP3=[Profile2_IP]
   

   #Gateway IP addresses. This is the first (hop) gateway, could be your router IP
   

   #address if it has been configured as the gateway
   

   GW1=[MY_DSL1_IP]
   

   GW2=[MY_DSL2_IP]
   

   GW3=[MY_DSL3_IP]
   

   # Relative weights of routes. Keep this to a low integer value.
   

   W1=1
   

   W2=1    
   

   W3=1
   

   # Broadband providers name; use your own names here.
   

   NAME1=dsl1
   

   NAME2=ds12
   

   NAME3=dsl3
   

   #No of repeats of success or failure before changing status of connection
   

   SUCCESSREPEATCOUNT=4
   

   FAILUREREPEATCOUNT=1
   

   # Do not change anything below this line
   

   # Last link status indicates the macro status of the link we determined. This is down initially to force routing change upfront. Don’t change these values.
   

   LLS1=1
   

   LLS2=1
   

   LLS3=1
   

   # Last ping status. Don’t change these values.
   

   LPS1=1
   

   LPS2=1
   

   LPS3=1
   

   # Current ping status. Don’t change these values.
   

   CPS1=1
   

   CPS2=1
   

   CPS3=1
   

   # Change link status indicates that the link needs to be changed. Don’t change these values.
   

   CLS1=1
   

   CLS2=1
   

   CLS3=1
   

   # Count of repeated up status or down status. Don’t change these values.
   

   COUNT1=0
   

   COUNT2=0
   

   COUNT3=0
   

   while : ; do
   

   /bin/ping -W $TIMEOUT -I $IP1 -c 1 $TESTIP > /dev/null 2>&1
   

   RETVAL=$?
   

   if [ $RETVAL -ne 0 ]; then
   

   echo $NAME1 Down
   

   CPS1=1
   

   else
   

   CPS1=0
   

   fi
   

   if [ $LPS1 -ne $CPS1 ]; then
   

   echo Ping status changed for $NAME1 from $LPS1 to $CPS1
   

   COUNT1=1
   

   else
   

   if [ $LPS1 -ne $LLS1 ]; then
   

   COUNT1=`expr $COUNT1 + 1`
   

   fi
   

   fi
   

   if [[ $COUNT1 -ge $SUCCESSREPEATCOUNT || ($LLS1 -eq 0 && $COUNT1 -ge $FAILUREREPEATCOUNT) ]]; then
   

   echo Uptime status will be changed for $NAME1 from $LLS1
   

   CLS1=0
   

   COUNT1=0
   

   if [ $LLS1 -eq 1 ]; then
   

   LLS1=0
   

   else
   

   LLS1=1
   

   fi
   

   else
   

   CLS1=1
   

   fi
   

   LPS1=$CPS1
   

   /bin/ping -W $TIMEOUT -I $IP2 -c 1 $TESTIP > /dev/null 2>&1
   

   RETVAL=$?
   

   if [ $RETVAL -ne 0 ]; then
   

   echo $NAME2 Down
   

   CPS2=1
   

   else
   

   CPS2=0
   

   fi
   

   if [ $LPS2 -ne $CPS2 ]; then
   

   echo Ping status changed for $NAME2 from $LPS2 to $CPS2
   

   COUNT2=1
   

   else
   

   if [ $LPS2 -ne $LLS2 ]; then
   

   COUNT2=`expr $COUNT2 + 1`
   

   fi
   

   fi
   

   if [[ $COUNT2 -ge $SUCCESSREPEATCOUNT || ($LLS2 -eq 0 && $COUNT2 -ge $FAILUREREPEATCOUNT) ]]; then
   

   echo Uptime status will be changed for $NAME2 from $LLS2
   

   CLS2=0
   

   COUNT2=0
   

   if [ $LLS2 -eq 1 ]; then
   

   LLS2=0
   

   else
   

   LLS2=1
   

   fi    
   

   else
   

   CLS2=1
   

   fi
   

   LPS2=$CPS2
   

   /bin/ping -W $TIMEOUT -I $IP3 -c 1 $TESTIP > /dev/null 2>&1
   

   RETVAL=$?
   

   if [ $RETVAL -ne 0 ]; then
   

   echo $NAME3 Down
   

   CPS3=1
   

   else
   

   CPS3=0
   

   fi
   

   if [ $LPS3 -ne $CPS3 ]; then
   

   echo Ping status changed for $NAME3 from $LPS3 to $CPS3
   

   COUNT3=1
   

   else
   

   if [ $LPS3 -ne $LLS3 ]; then
   

   COUNT3=`expr $COUNT3 + 1`
   

   fi
   

   fi
   

   if [[ $COUNT3 -ge $SUCCESSREPEATCOUNT || ($LLS3 -eq 0 && $COUNT3 -ge $FAILUREREPEATCOUNT) ]]; then
   

   echo Uptime status will be changed for $NAME3 from $LLS3
   

   CLS3=0
   

   COUNT3=0
   

   if [ $LLS3 -eq 1 ]; then
   

   LLS3=0
   

   else
   

   LLS3=1
   

   fi
   

   else
   

   CLS3=1
   

   fi
   

   LPS3=$CPS3
   

   # Changeover Logic here
   

   if [[ $CLS1 -eq 0 || $CLS2 -eq 0 || $CLS3 -eq 0 ]]; then
   

   if [ $LLS1 -eq 0 ]; then
   

   echo Adding $NAME1
   

   GWCMD="$GWCMD nexthop via $GW1 dev $EXTIF1 weight 1"
   

   fi
   

   if [ $LLS2 -eq 0 ]; then
   

   echo Adding $NAME2
   

   GWCMD="$GWCMD nexthop via $GW2 dev $EXTIF2 weight 1"
   

   fi
   

   if [ $LLS3 -eq 0 ]; then
   

   echo Adding $NAME3
   

   GWCMD="$GWCMD nexthop via $GW3 dev $EXTIF3 weight 1"
   

   fi
   

   echo "Change execute now \n $GWCMD"
   

   $GWCMD
   

   GWCMD="/sbin/ip route replace default scope global"
   

   fi
   

   sleep $SLEEPTIME
   

   done

2. The last place is to make DNS entries. Just do a vi /etc/resolv.conf and type the following:

   nameserver 208.67.222.222

   I am using OpenDNS IP as DNS here. One may wish to have any other reliable one.

3. Oops!! forgot one more thing, do a chmod 777 /var/log/gwping.log

   Now, you just need to reboot the same machine and then use [Ethernet1_IP] as Gateway in your production machines/ Proxy/ Firewall and here you go.

Waiting for your comments. Soon will post the HowTo, if one wishes to get the same thing done without putting one additional ethernet card in action means with one card only.

PDF Version