DMVPN with dual ISPs

DMVPN with dual ISPs

This article demonstrates DMVPN with 2 ISPs where the Hub has dual ISP connections. Also, each Spoke router is connected to a separate ISP.

Configuration

PE11, PE12, PE21 and PE22 routers have a very simple and similar VRF configuration. There is an eBGP connection between PE and CE routers. A sample configuration on PE11 follows.

VRF Configuration on PE11 router

mpls label protocol ldp

mpls ldp router-id Loopback 0 force

ip vrf CUST

 rd 1:1

 route-target both 1:1

interface Loopback 0

 ip address 1.1.1.1 255.255.255.255

 ip ospf 1 area 0

interface Serial 0/0

 ip vrf forwarding CUST

 ip address 172.11.1.2 255.255.255.0

router bgp 100

 neighbor 2.2.2.2 remote-as 100

 neighbor 2.2.2.2 update-source Loopback 0

 address-family vpnv4

 neighbor 2.2.2.2 activate

 exit-address-family

 address-family ipv4 vrf CUST

 neighbor 172.11.1.1 remote-as 65001

 neighbor 172.11.1.1 activate

 redistribute connected

 exit-address-family

PE100 and PE200 routers act as Route-Reflectors (RR) for their respective ASes. To exchange VPNv4 routes, they run MP-eBGP connection across the two physical connections- one preferred over the other achieved by setting higher local-preference value for prefixes learnt via one connection.

PE100 Configuration

interface Loopback 0

 ip address 2.2.2.2 255.255.255.255

 ip ospf 1 area 0

router ospf 1

 passive-interface FastEthernet1/0

 passive-interface FastEthernet2/0

 network 10.100.1.0 0.0.0.255 area 0

 network 10.100.2.0 0.0.0.255 area 0

router bgp 100

 no bgp route-target default filter                ! Accept all VPNv4 routes

 neighbor 1.1.1.1 remote-as 100                    ! PE11 router

 neighbor 1.1.1.1 update-source Loopback 0

 neighbor 3.3.3.3 remote-as 100                    ! PE12 router

 neighbor 3.3.3.3 update-source Loopback 0

 neighbor 10.100.1.2 remote-as 200

 neighbor 10.100.1.2 send-label                    ! eBGP connection with PE200 router. The "send-label" keyword allows the router to send label with BGP VPNv4 prefixes

 neighbor 10.100.2.2 remote-as 200

 neighbor 10.100.2.2 send-label

 address-family vpnv4

 neighbor 1.1.1.1 activate

 neighbor 1.1.1.1 route-reflector-client

 neighbor 1.1.1.1 next-hop-self

 neighbor 3.3.3.3 activate

 neighbor 3.3.3.3 route-reflector-client

 neighbor 3.3.3.3 next-hop-self

 neighbor 10.100.1.2 activate

 neighbor 10.100.1.2 route-map LOCAL_PREF in    ! Set Local-Preference to 150 for all incoming VPNv4 prefixes from this neighbor

 neighbor 10.100.2.2 activate

route-map LOCAL_PREF permit 10

 set local-preference 150

Configuration on PE200 router is very similar to PE100 router.

PE200 Configuration

interface Loopback 0

 ip address 5.5.5.5 255.255.255.255

 ip ospf 2 area 0

router ospf 2

 passive-interface FastEthernet1/0

 passive-interface FastEthernet2/0

 network 10.100.1.0 0.0.0.255 area 0

 network 10.100.2.0 0.0.0.255 area 0

router bgp 200

 no bgp route-target default filter

 neighbor 4.4.4.4 remote-as 200                ! PE21 router

 neighbor 4.4.4.4 update-source Loopback 0

 neighbor 6.6.6.6 remote-as 200                ! PE22 router

 neighbor 6.6.6.6 update-source Loopback 0

 neighbor 10.100.1.1 remote-as 100             ! eBGP connection to PE100

 neighbor 10.100.1.1 send-label

 neighbor 10.100.2.1 remote-as 100

 neighbor 10.100.2.1 send-label

 address-family vpnv4

 neighbor 4.4.4.4 activate

 neighbor 4.4.4.4 route-reflector-client

 neighbor 4.4.4.4 next-hop-self

 neighbor 6.6.6.6 activate

 neighbor 6.6.6.6 route-reflector-client

 neighbor 6.6.6.6 next-hop-self

 neighbor 10.100.1.1 activate

 neighbor 10.100.1.1 route-map LOCAL_PREF in

 neighbor 10.100.2.1 activate

route-map LOCAL_PREF permit 10

 set local-preference 150

The following output shows the VPNv4 prefixes learnt by PE100 and PE200 routers. Note the prefixes with higher local-preference value are preferred.

VPNv4 Prefixes on PE100 and PE200

PE100# show ip bgp vpnv4 all summary

BGP router identifier 2.2.2.2, local AS number 100

BGP table version is 13, main routing table version 13

4 network entries using 560 bytes of memory

6 path entries using 408 bytes of memory

4/2 BGP path/bestpath attribute entries using 496 bytes of memory

1 BGP AS-PATH entries using 24 bytes of memory

1 BGP extended community entries using 24 bytes of memory

0 BGP route-map cache entries using 0 bytes of memory

0 BGP filter-list cache entries using 0 bytes of memory

Bitfield cache entries: current 1 (at peak 3) using 32 bytes of memory

BGP using 1544 total bytes of memory

BGP activity 4/0 prefixes, 10/4 paths, scan interval 15 secs

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd

1.1.1.1         4   100      41      47       13    0    0 00:36:36        1

3.3.3.3         4   100      41      47       13    0    0 00:35:42        1

10.100.1.2      4   200      51      52       13    0    0 00:15:47        2

10.100.2.2      4   200      40      41       13    0    0 00:15:43        2

PE100# show ip bgp vpnv4 all

BGP table version is 13, local router ID is 2.2.2.2

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

              r RIB-failure, S Stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 1:1

*>i172.11.1.0/24    1.1.1.1                  0    100      0 ?

*  172.12.1.0/24    10.100.2.2                             0 200 ?

*>                  10.100.1.2                    150      0 200 ?

*>i172.22.1.0/24    3.3.3.3                  0    100      0 ?

*  172.33.1.0/24    10.100.2.2                             0 200 ?

*>                  10.100.1.2                    150      0 200 ?

PE200# show ip bgp vpnv4 all

BGP table version is 13, local router ID is 5.5.5.5

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

              r RIB-failure, S Stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 1:1

*  172.11.1.0/24    10.100.2.1                             0 100 ?

*>                  10.100.1.1                    150      0 100 ?

*>i172.12.1.0/24    4.4.4.4                  0    100      0 ?

*  172.22.1.0/24    10.100.2.1                             0 100 ?

*>                  10.100.1.1                    150      0 100 ?

*>i172.33.1.0/24    6.6.6.6                  0    100      0 ?

DMVPN Hub and Spoke Configuration

Since the Hub router has 2 connections to the ISP, two tunnel interfaces are created on each Hub and Spoke routers. However, only one tunnel is operational at any time.

DMVPN Hub Configuration

crypto isakmp policy 10

 encryption 3des

 authentication pre-share

 hash md5

 group 2

crypto isakmp key 0 cisco address 0.0.0.0 0.0.0.0

crypto ipsec transform-set TS esp-3des esp-md5-hmac

 mode transport

crypto ipsec profile DMVPN

 set transform-set TS

interface Tunnel10                        ! Primary Tunnel

 ip address 192.168.100.1 255.255.255.0

 no ip next-hop-self eigrp 1020

 ip nhrp map multicast dynamic

 ip nhrp network-id 123

 no ip split-horizon eigrp 1020

 delay 5

 tunnel source Serial1/0

 tunnel mode gre multipoint

 tunnel key 123

 tunnel protection ipsec profile DMVPN shared

interface Tunnel20                        ! Backup Tunnel

 ip address 192.168.200.1 255.255.255.0

 no ip next-hop-self eigrp 1020

 ip nhrp map multicast dynamic

 ip nhrp network-id 456

 no ip split-horizon eigrp 1020

 delay 10

 tunnel source Serial1/1

 tunnel mode gre multipoint

 tunnel key 456

 tunnel protection ipsec profile DMVPN shared

router eigrp 1020

 no auto-summary

 network 192.168.0.0

 network 192.168.100.0

 network 192.168.200.0

router bgp 65001

 neighbor 172.11.1.2 remote-as 100

 neighbor 172.11.1.2 filter-list 10 out

 neighbor 172.12.1.2 remote-as 200

 neighbor 172.12.1.2 filter-list 10 out

ip as-path access-list 10 permit ^$        ! Allows only local routes to be advertised

Now, since the Spoke routers have a single connection to the ISP, they share the same interface for both tunnels.

DMVPN Configuration on Spoke1 router

crypto isakmp policy 10

 encr 3des

 hash md5

 authentication pre-share

 group 2

crypto isakmp key cisco address 0.0.0.0 0.0.0.0

crypto ipsec transform-set TS esp-3des esp-md5-hmac

 mode transport

crypto ipsec profile DMVPN

 set transform-set TS

interface Tunnel10

 ip address 192.168.100.2 255.255.255.0

 ip nhrp map 192.168.100.1 172.11.1.1

 ip nhrp map multicast 172.11.1.1

 ip nhrp network-id 123

 ip nhrp nhs 192.168.100.1

 delay 5

 tunnel source Serial1/0

 tunnel mode gre multipoint

 tunnel key 123

 tunnel protection ipsec profile DMVPN shared

interface Tunnel20

 ip address 192.168.200.2 255.255.255.0

 ip nhrp map 192.168.200.1 172.12.1.1

 ip nhrp map multicast 172.12.1.1

 ip nhrp network-id 456

 ip nhrp nhs 192.168.200.1

 delay 10

 tunnel source Serial1/0

 tunnel mode gre multipoint

 tunnel key 456

 tunnel protection ipsec profile DMVPN shared

router bgp 65002

 neighbor 172.22.1.2 remote-as 100

DMVPN Configuration on Spoke2 router

crypto isakmp policy 10

 encryption 3des

 hash md5

 authentication pre-share

 group 2

crypto isakmp key cisco address 0.0.0.0 0.0.0.0

crypto ipsec transform-set TS esp-3des esp-md5-hmac

 mode transport

crypto ipsec profile DMVPN

 set transform-set TS

interface Tunnel10

 ip address 192.168.100.3 255.255.255.0

 ip nhrp map 192.168.100.1 172.11.1.1

 ip nhrp map multicast 172.11.1.1

 ip nhrp network-id 123

 ip nhrp nhs 192.168.100.1

 delay 5

 tunnel source Serial1/0

 tunnel mode gre multipoint

 tunnel key 123

 tunnel protection ipsec profile DMVPN shared

interface Tunnel20

 ip address 192.168.200.3 255.255.255.0

 ip nhrp map 192.168.200.1 172.12.1.1

 ip nhrp map multicast 172.12.1.1

 ip nhrp network-id 456

 ip nhrp nhs 192.168.200.1

 delay 10

 tunnel source Serial1/0

 tunnel mode gre multipoint

 tunnel key 456

 tunnel protection ipsec profile DMVPN shared

router bgp 65003

 neighbor 172.33.1.2 remote-as 200

The major issue with this setup is that if the primary connection to ISP at Hub site goes down, the tunnel interface (tunnel 10) goes down as well. EIGRP adjacency with Spoke routers fall. However, the Spoke routers still have primary tunnel interface as functional. Hence, a tracking mechanism is required on Spoke routers to track the reachability of the source address of primary tunnel interface. IP SLA with tracking can do the job. Further, this reachability information can be used to administer the functionality of tunnel interfaces on Spoke routers. For example, if primary tunnel interface goes down on Hub, the Spoke routers shut down their primary tunnel interface and bring the secondary tunnel up. This can be done using Embedded Event Manager EEM. Before any IP SLA configuration on Spoke routers, ip sla responder command is required on the Hub router. The configuration is as below:

IP SLA on Hub

ip sla responder

IP SLA + Tracking + EEM on Spoke1 router

track 1 ip sla 1 reachability

 delay down 180 up 180

ip sla 1

 icmp-echo 172.11.1.1 source-interface Serial1/0

 timeout 300

 threshold 300

 frequency 3

ip sla schedule 1 life forever start-time now

event manager applet UP_TUNNEL_10

 event track 1 state up

 action 1.0 cli command "enable"

 action 1.1 cli command "config t"

 action 1.2 cli command "interface Tunnel20"

 action 1.3 cli command "shutdown"

 action 1.4 cli command "interface Tunnel10"

 action 1.5 cli command "no shutdown"

 action 1.6 cli command "do clear crypto isakmp"

event manager applet SHUT_TUNNEL_10

 event track 1 state down

 action 1.0 cli command "enable"

 action 1.1 cli command "config t"

 action 1.2 cli command "interface Tunnel10"

 action 1.3 cli command "shutdown"

 action 1.4 cli command "interface Tunnel20"

 action 1.5 cli command "no shutdown"

 action 1.6 cli command "do clear crypto isakmp"

IP SLA + Tracking + EEM on Spoke2 router

track 1 ip sla 1 reachability

 delay down 180 up 180

ip sla 1

 icmp-echo 172.12.1.1 source-interface Serial1/0

 timeout 300

 threshold 300

 frequency 3

ip sla schedule 1 life forever start-time now

event manager applet UP_TUNNEL_10

 event track 1 state up

 action 1.0 cli command "enable"

 action 1.1 cli command "config t"

 action 1.2 cli command "interface Tunnel20"

 action 1.3 cli command "shutdown"

 action 1.4 cli command "interface Tunnel10"

 action 1.5 cli command "no shutdown"

 action 1.6 cli command "do clear crypto isakmp"

event manager applet SHUT_TUNNEL_10

 event track 1 state down

 action 1.0 cli command "enable"

 action 1.1 cli command "config t"

 action 1.2 cli command "interface Tunnel10"

 action 1.3 cli command "shutdown"

 action 1.4 cli command "interface Tunnel20"

 action 1.5 cli command "no shutdown"

 action 1.6 cli command "do clear crypto isakmp"

Verification

Since the primary tunnel interface is UP, all the Spoke routers register their NHRP mappings with the Hub router over primary tunnel interface 10. Also, all internal prefixes are learnt over primary tunnel as EIGRP adjacency is established over that interface.

Hub# show ip nhrp

192.168.100.2/32 via 192.168.100.2

   Tunnel10 created 00:29:51, expire 01:30:28

   Type: dynamic, Flags: unique registered

   NBMA address: 172.22.1.1

192.168.100.3/32 via 192.168.100.3

   Tunnel10 created 00:29:48, expire 01:31:00

   Type: dynamic, Flags: unique registered used

   NBMA address: 172.33.1.1

Hub# show ip route eigrp

D    192.168.1.0/24 [90/25603840] via 192.168.100.2, 00:10:34, Tunnel10

D    192.168.2.0/24 [90/25603840] via 192.168.100.3, 00:10:34, Tunnel10

Spoke1# show ip route eigrp

D    192.168.0.0/24 [90/25603840] via 192.168.100.1, 00:14:19, Tunnel10

D    192.168.2.0/24 [90/25605120] via 192.168.100.3, 00:11:08, Tunnel10

Spoke2# show ip route eigrp

D    192.168.0.0/24 [90/25603840] via 192.168.100.1, 00:11:34, Tunnel10

D    192.168.1.0/24 [90/25605120] via 192.168.100.2, 00:11:34, Tunnel10

When the source address of primary tunnel interface on the Hub router is unreachable (data-link between Hub and PE11 is down), the tracking object discovers that the Hub is not reachable over Tunnel 10. The Spoke routers shut down Tunnel 10 and bring up Tunnel 20. The Hub router receives NHRP mapping over Tunnel 20. EIGRP adjacency is formed over this interface and routes are exchanged.

Spoke1#

*Jul 15 15:42:45.734: %TRACKING-5-STATE: 1 ip sla 1 reachability Up->Down

*Jul 15 15:42:45.982: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is OFF

*Jul 15 15:42:46.434: %SYS-5-CONFIG_I: Configured from console by vty0

*Jul 15 15:42:47.898: %LINK-5-CHANGED: Interface Tunnel10, changed state to administratively down

*Jul 15 15:42:48.186: %LINK-3-UPDOWN: Interface Tunnel20, changed state to up

*Jul 15 15:42:48.286: %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON

*Jul 15 15:42:48.910: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel10, changed state to down

*Jul 15 15:42:49.194: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel20, changed state to up

*Jul 15 15:42:52.614: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1020: Neighbor 192.168.200.1 (Tunnel20) is up: new adjacency

Spoke1# show track

Track 1

  IP SLA 1 reachability

  Reachability is Down

    99 changes, last change 00:00:33

  Delay up 180 secs, down 180 secs

  Latest operation return code: Timeout

  Tracked by:

    EEM applet UP_TUNNEL_10

    EEM applet SHUT_TUNNEL_10

Hub# show ip nhrp

192.168.200.2/32 via 192.168.200.2

   Tunnel20 created 00:03:58, expire 01:56:01

   Type: dynamic, Flags: unique registered

   NBMA address: 172.22.1.1

192.168.200.3/32 via 192.168.200.3

   Tunnel20 created 00:06:54, expire 01:53:05

   Type: dynamic, Flags: unique registered

   NBMA address: 172.33.1.1

During Spoke-to-Spoke traffic forwarding, dynamic tunnel is formed via Tunnel 20.

Dynamic Spoke-to-Spoke tunnel

Spoke1# ping 192.168.2.1 source fa 0/0

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.2.1, timeout is 2 seconds:

Packet sent with a source address of 192.168.1.1

!!!!!

Success rate is 100 percent (5/5), round-trip min/avg/max = 388/864/1440 ms

Spoke1# show ip nhrp

192.168.200.1/32 via 192.168.200.1

   Tunnel20 created 00:04:41, never expire

   Type: static, Flags: used

   NBMA address: 172.12.1.1

192.168.200.2/32 via 192.168.200.2

   Tunnel20 created 00:00:05, expire 01:59:54

   Type: dynamic, Flags: router unique local

   NBMA address: 172.22.1.1

    (no-socket)

192.168.200.3/32 via 192.168.200.3

   Tunnel20 created 00:00:08, expire 01:59:56

   Type: dynamic, Flags: router

   NBMA address: 172.33.1.1

Page updated

Google Sites

Report abuse