Source based routing: Demystifying Linux routing

Introduction

Laymen explanation

If you have worked on the data path, you might have requirement about data to be sent to specific destination address (server for example). You may want to send some data based on destination address. In some case, you may want to decide that data from specific source should be handled by specific interface. For example, you may want that specific user data request should use VPN tunnel. In this case, you want to tell that any data from 172.10.0.10/32 should be processed by interface vpn0.

This document helps to achieve this

Technical explanation

Most of us know about routing based on destination address of packet. BGP, OSPF, RIP works in this way. Linux provides source based routing facility. In this way, user to create policy to redirect traffic based on source address to an interface. This interface can be a normal interface or a VPN interface or any other type.

Example of source based routing

ip rule show example

# ip rule show

0: from all lookup local

32765: from 1.1.1.1 lookup 100

32766: from all lookup main

32767: from all lookup default

example of source routing configuration

# ip route show table 100

default via 1.1.1.1 dev eth0

1.1.1.0/24 dev eth0 proto kernel scope link src 1.1.1.1

How to add your own routing table

Decide priority

Decide on the priority. Please note that priority 0 is for local and 32766 for main. Priority decides the routing policy ordering. So, priority decision is important

'ip rule add' command adds rules in the order in which it is executed

For example,

Ordering of new routing table

/# ip rule show

0: from all lookup local

32766: from all lookup main

32767: from all lookup default

/# echo "1 admin" >> /etc/iproute2/rt_tables

/# ip rule add from 19.86.100.176/24 table admin

/# ip rule show

0: from all lookup local

32765: from 19.86.100.176/24 lookup admin

32766: from all lookup main

32767: from all lookup default

Configure route and apply

Refer https://www.thegeekstuff.com/2014/08/add-route-ip-command/ for the example.

Useful command example to configure source based routing table

echo "1 admin" >> /etc/iproute2/rt_tables

ip route add 172.16.40.0/24 dev eth0 src 172.16.40.7 table admin

ip route add default via 172.16.40.7 dev eth0 table admin

ip rule add from 19.86.100.176/24 table admin

ip rule add from 172.16.40.0/24 table admin

Example output of ip rule

# ip rule show

0: from all lookup local

32764: from 172.16.40.0/24 lookup admin

32765: from 19.86.100.176/24 lookup admin

32766: from all lookup main

32767: from all lookup default

How to remove your routing table

ip rule del helps in this

Example for removing ip route info

# ip rule show

0: from all lookup local

32764: from 19.86.100.176/24 lookup 101

32765: from 172.16.40.7 lookup 100

32766: from all lookup main

32767: from all lookup default

# ip rule del from 19.86.100.176/24 table 101

# ip rule show

0: from all lookup local

32765: from 172.16.40.7 lookup 100

32766: from all lookup main

32767: from all lookup default

Understanding ip rule show

Default rules injected by Linux

Three rules will be present by default

- Table local with priority 0 which keeps route info for local and broadcast addresses
- Table main which has least priority and it contains all rules which are not policy based
- Table default which is not in use. It is for anything which doesn't fall to any routing rule

1. Priority: 0, Selector: match anything, Action: lookup routing

table local (ID 255). The local table is a special routing table containing high priority control routes for local and broadcast addresses. Rule 0 is special. It cannot be deleted or overridden. 2. Priority: 32766, Selector: match anything, Action: lookup routing table main (ID 254). The main table is the normal routing table containing all non-policy routes. This rule may be deleted and/or overridden with other ones by the administrator. 3. Priority: 32767, Selector: match anything, Action: lookup routing table default (ID 253). The default table is empty. It is reserved for some post-processing if no previous default rules selected the packet. This rule may also be deleted.

Programming tools

Python 'iproute2' library helps to manage it (pip3 install iproute2)

Example output from python library

root@09b7be1098b1:/# curl https://raw.githubusercontent.com/krdpk17/networking-tools/master/iprules_tool.py >iprules_tool.py

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

100 2889 100 2889 0 0 1124 0 0:00:02 0:00:02 --:--:-- 1125

root@09b7be1098b1:/# python3 iprules_tool.py

({'family': 2, 'dst_len': 0, 'src_len': 0, 'tos': 0, 'table': 255, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 255, 'FRA_SUPPRESS_PREFIXLEN': 4294967295}, 'header': {'length': 44, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1929, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 32, 'src_len': 0, 'tos': 0, 'table': 100, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 100, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32765, 'FRA_DST': '10.106.175.101'}, 'header': {'length': 60, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1929, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 0, 'src_len': 0, 'tos': 0, 'table': 254, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 254, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32766}, 'header': {'length': 52, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1929, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 0, 'src_len': 0, 'tos': 0, 'table': 253, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 253, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32767}, 'header': {'length': 52, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1929, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'})

root@09b7be1098b1:/# ip rules show

Object "rules" is unknown, try "ip help".

root@09b7be1098b1:/# ip rule show

0: from all lookup local

32765: from all to 10.106.175.101 lookup 100

32766: from all lookup main

32767: from all lookup default

root@09b7be1098b1:/#

root@09b7be1098b1:/# ip rule add from 1.1.1.1 table 101

root@09b7be1098b1:/# ip rule show

0: from all lookup local

32764: from 1.1.1.1 lookup 101

32765: from all to 10.106.175.101 lookup 100

32766: from all lookup main

32767: from all lookup default

root@09b7be1098b1:/# python3 iprules_tool.py

({'family': 2, 'dst_len': 0, 'src_len': 0, 'tos': 0, 'table': 255, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 255, 'FRA_SUPPRESS_PREFIXLEN': 4294967295}, 'header': {'length': 44, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1937, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 0, 'src_len': 32, 'tos': 0, 'table': 101, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 101, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32764, 'FRA_SRC': '1.1.1.1'}, 'header': {'length': 60, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1937, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 32, 'src_len': 0, 'tos': 0, 'table': 100, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 100, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32765, 'FRA_DST': '10.106.175.101'}, 'header': {'length': 60, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1937, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 0, 'src_len': 0, 'tos': 0, 'table': 254, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 254, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32766}, 'header': {'length': 52, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1937, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'}, {'family': 2, 'dst_len': 0, 'src_len': 0, 'tos': 0, 'table': 253, 'res1': 0, 'res2': 0, 'action': 1, 'flags': 0, 'attrs': {'FRA_TABLE': 253, 'FRA_SUPPRESS_PREFIXLEN': 4294967295, 'FRA_PRIORITY': 32767}, 'header': {'length': 52, 'type': 32, 'flags': 2, 'sequence_number': 255, 'pid': 1937, 'error': None, 'target': 'localhost', 'stats': (0, 0, 0)}, 'event': 'RTM_NEWRULE'})

root@09b7be1098b1:/#

Ref:https://opendev.org/openstack/neutron/raw/commit/57edfee5da12ed32f625de6120c344ce69b46619/neutron/privileged/agent/linux/ip_lib.py

Simple example to add ip rule

root@09b7be1098b1:/# ip rule show

0: from all lookup local

32766: from all lookup main

32767: from all lookup default

root@09b7be1098b1:/# cat simple.py

import pyroute2

iproute = pyroute2.IPRoute()

iproute.rule('add', 14, 32004, src='10.20.30.40')

root@09b7be1098b1:/# python3 simple.py

root@09b7be1098b1:/# ip rule show

0: from all lookup local

32004: from 10.20.30.40 lookup 14

32766: from all lookup main

32767: from all lookup default

root@09b7be1098b1:/#

Ref: https://pyroute2.org/pyroute2-0.3.4/iproute.html

Reference

https://community.ui.com/questions/Policy-Based-Routing-specific-gateway-for-source-address-problem-with-IPSec-VTI-/bc2199ff-4ead-4e71-a117-ef3618abbe46

http://man7.org/linux/man-pages/man8/ip-rule.8.html

https://unix.stackexchange.com/questions/160115/what-do-the-numbers-mean-in-ip-rule-show-command

Page updated

Google Sites

Report abuse