GDOI-based DMVPN

GDOI-based DMVPN

In traditional DMVPN, if Spoke-to-Spoke direct connectivity is enabled (i.e DMVPN Phase 2), a permanent IPSec tunnel is maintained between the Spoke and the Hub, and a dynamic IPSec tunnel is created on demand between the Spokes. When a Spoke router wishes to send traffic to another Spoke router, it sends a NHRP Resolution Request message to the DMVPN Hub for the destination Spoke router. Once the mapping is received, the Spoke router will initiate a dynamic IPSec tunnel with the destination Spoke router. Until this dynamic IPSec tunnel is created, traffic flows through the Hub. If traffic needs to come in reverse direction, the destination Spoke router follows the same process towards the originating Spoke router.

When a dynamic IPSec tunnel is created for Spoke-to-Spoke connectivity, it introduces a slight delay caused by IPSec negotiation which can lead to poor performance for certain real-time applications like VOIP. GDOI eliminates this delay.

With GDOI, the DMVPN Hub and Spokes are the Group Members (GMs). Group Keys and security policies are distributed to GMs by the Key Server (KS)- a separate Cisco IOS router. This eliminates the point-to-point IPSec session between the GMs. Once NHRP resolution is completed, the Spokes can communicate using the same key. This reduces delay between Spoke-to-Spoke communication by eliminating creation of dynamic IPSec tunnels between them.

Sample Scenario

Key Server (KS) Configuration

All GMs register with the KS and the KS is responsible for distributing keys and security policies. Since all traffic must flow through mGRE tunnel, the interesting traffic can be categorized as GRE traffic from any DMVPN router to other DMVPN router. The configuration of KS is as below:

KS Configuration

crypto key generate rsa general-keys label GDOI_KEYS modulus 1024 exportable

crypto isakmp policy 10

 authentication pre-share

!---- Manually configuring pre-shared keys for all GMs

crypto isakmp key 0 cisco address 2.2.2.2       ! Hub router

crypto isakmp key 0 cisco address 11.11.11.11   ! Spoke1 router

crypto isakmp key 0 cisco address 12.12.12.12   ! Spoke2 router

crypto isakmp key 0 cisco address 13.13.13.13   ! Spoke3 router

crypto ipsec transform-set GDOI_TS edp-3des esp-md5-hmac

crypto ipsec profile GDOI_PROFILE

 set transform-set GDOI_TS

crypto gdoi group GDOI_GROUP

 identity number 123

 server local

  rekey transport unicast

  rekey authentication mypubkey rsa GDOI_KEYS

  rekey retransmit 60 number 2

  sa ipsec 1

  profile GDOI_PROFILE

  match address ipv4 ACL

  replay time window-size 5

  address ipv4 1.1.1.1    ! Use this address to send rekey packets

ip access-list extended ACL

 permit gre any any       ! Encrypt all GRE packets

GM (Hub and Spoke) Tunnel and GDOI Configuration

The Hub and Spoke routers must be configured with a mGRE tunnel for DMVPN.

Tunnel Configuration on Hub

interface Tunnel 10

 ip address 192.168.100.1 255.255.255.0

 tunnel mode gre multipoint

 tunnel source serial 2/0

 tunnel key 123

 ip nhrp map multicast dynamic

 ip nhrp holdtime 240

 ip nhrp network-id 123

 no ip next-hop-self eigrp 10    ! Disable next-hop overwrite

 no ip split-horizon eigrp 10    ! Allow routes to be re-advertised to Spoke routers

router eigrp 10

 ! routing instance for DMVPN

 no auto-summary

 network 192.168.100.0

 network 192.168.1.0

The following shows configuration on Spoke1 router. A similar configuration applies to other Spoke routers.

Tunnel Configuration on Spoke1

interface Tunnel 10

 ip address 192.168.100.2 255.255.255.0

 tunnel mode gre multipoint

 tunnel source serial 1/0

 tunnel key 123

 ip nhrp nhs 192.168.100.1

ip nhrp map 192.168.100.1 172.17.1.1

 ip nhrp map multicast 172.17.1.1

 ip nhrp holdtime 240

 ip nhrp network-id 123

router eigrp 10

 no auto-summary

 network 192.168.100.0

 network 192.168.2.0

Then GDOI configuration on Hub and Spoke routers is as follows:

GDOI Configuration

crypto isakmp policy 10

 authentication pre-share

crypto isakmp key 0 cisco address 1.1.1.1

crypto gdoi group GDOI_GM

 identity number 123

 server address ipv4 1.1.1.1            ! Address of KS

crypto map DMVPN local-address Loopback0

crypto map DMVPN 10 gdoi

 set group GDOI_GM

interface serial 1/0

 ....

 crypto map DMVPN

Verification

First all routers (Hub and Spokes) register with the KS and download the policies configured on the KS. All routers complete IKE Phase 1 with the KS and start GDOI protocol by sending Register messages to the KS. In the end, the KS sends the Key Download Payload which contains the KEK and TEK keys. The below output shows successful IKE Phase 1 with all routers, but no IKE Phase 2 on KS.

KS# show crypto isakmp sa

IPv4 Crypto ISAKMP SA

dst             src             state          conn-id status

1.1.1.1         13.13.13.13     GDOI_IDLE         1004 ACTIVE

1.1.1.1         11.11.11.11     GDOI_IDLE         1002 ACTIVE

1.1.1.1         12.12.12.12     GDOI_IDLE         1003 ACTIVE

1.1.1.1         2.2.2.2         GDOI_IDLE         1001 ACTIVE

IPv6 Crypto ISAKMP SA

KS# show crypto ipsec sa

No SAs found

The following output shows GDOI policy downloaded by GM Spoke1 router. The output will be similar for all GMs.

GDOI Policy on Spoke1

Spoke1# show crypto gdoi

GROUP INFORMATION

    Group Name               : GDOI_GM

    Group Identity           : 123

    Rekeys received          : 1

    IPSec SA Direction       : Both

    Active Group Server      : 1.1.1.1

    Group Server list        : 1.1.1.1

    GM Reregisters in        : 2287 secs

    Rekey Received(hh:mm:ss) : 00:19:59

    Rekeys received

         Cumulative          : 1

         After registration  : 1

    Rekey Acks sent          : 1

 ACL Downloaded From KS 1.1.1.1:

   access-list  permit gre any any

KEK POLICY:

    Rekey Transport Type     : Unicast

    Lifetime (secs)          : 86399

    Encrypt Algorithm        : 3DES

    Key Size                 : 192

    Sig Hash Algorithm       : HMAC_AUTH_SHA

    Sig Key Length (bits)    : 1024

TEK POLICY for the current KS-Policy ACEs Downloaded:

  Serial1/0:

    IPsec SA:

        spi: 0x42CE20E(70050318)

        transform: esp-3des esp-md5-hmac

        sa timing:remaining key lifetime (sec): (2400)

        Anti-Replay(Time Based) : 5 sec interval

All GMs have a single IKE session with KS. Notice that the remote endpoint peer is 0.0.0.0 i.e. Spoke1 is open for any other GM.

IKE Session on Spoke1

Spoke1# show crypto isakmp sa

IPv4 Crypto ISAKMP SA

dst             src             state          conn-id status

1.1.1.1         11.11.11.11     GDOI_IDLE         1001 ACTIVE

11.11.11.11     1.1.1.1         GDOI_REKEY        1002 ACTIVE

11.11.11.11     1.1.1.1         GDOI_REKEY        1003 ACTIVE

IPv6 Crypto ISAKMP SA

Spoke1# show crypto ipsec sa

     PFS (Y/N): N, DH group: none

interface: Serial1/0

    Crypto map tag: DMVPN, local addr 11.11.11.11

   protected vrf: (none)

   local  ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/47/0)

   remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/47/0)

   current_peer 0.0.0.0 port 848

     PERMIT, flags={origin_is_acl,}

    #pkts encaps: 708, #pkts encrypt: 708, #pkts digest: 708

    #pkts decaps: 777, #pkts decrypt: 777, #pkts verify: 777

    #pkts compressed: 0, #pkts decompressed: 0

    #pkts not compressed: 0, #pkts compr. failed: 0

    #pkts not decompressed: 0, #pkts decompress failed: 0

    #send errors 0, #recv errors 0

     local crypto endpt.: 11.11.11.11, remote crypto endpt.: 0.0.0.0

     path mtu 1500, ip mtu 1500, ip mtu idb Serial1/0

     current outbound spi: 0x42CE20E(70050318)

     inbound esp sas:

      spi: 0x42CE20E(70050318)

        transform: esp-3des esp-md5-hmac ,

        in use settings ={Tunnel, }

        conn id: 5, flow_id: SW:5, sibling_flags 80000040, crypto map: DMVPN

        sa timing: remaining key lifetime (sec): (2359)

        IV size: 8 bytes

        replay detection support: Y  replay window size: 5

        Status: ACTIVE

     inbound ah sas:

     inbound pcp sas:

     outbound esp sas:

      spi: 0x42CE20E(70050318)

        transform: esp-3des esp-md5-hmac ,

        in use settings ={Tunnel, }

        conn id: 6, flow_id: SW:6, sibling_flags 80000040, crypto map: DMVPN

        sa timing: remaining key lifetime (sec): (2359)

        IV size: 8 bytes

        replay detection support: Y  replay window size: 5

        Status: ACTIVE

     outbound ah sas:

     outbound pcp sas:

Then all Spoke routers then register with DMVPN_Hub router using NHRP Registration Request messages. The Hub router learns the Tunnel IP Address-to-NBMA address mapping dynamically. These messages are GRE encapsulated and hence encrypted by GDOI policies.

Hub learns NHRP mappings dynamically

DMVPN_Hub# show ip nhrp

192.168.100.2/32 via 192.168.100.2

   Tunnel10 created 00:04:35, expire 00:03:34

   Type: dynamic, Flags: unique registered used

   NBMA address: 172.27.1.1

192.168.100.3/32 via 192.168.100.3

   Tunnel10 created 00:01:30, expire 00:03:03

   Type: dynamic, Flags: unique registered

   NBMA address: 172.37.1.1

192.168.100.4/32 via 192.168.100.4

   Tunnel10 created 00:04:35, expire 00:03:15

   Type: dynamic, Flags: unique registered

   NBMA address: 172.47.1.1

This forms EIGRP adjacency over the tunnel interface and routes are exchanged between DMVPN routers. Note that these EIGRP packets are also encapsulated in GRE as they travel over tunnel interface, and hence encapsulated by ESP. Notice the next-hop remains unchanged when these routes are advertised to Spoke routers.

EIGRP Routes learnt by Hub router via Tunnel interface

DMVPN_Hub# show ip route eigrp

D    192.168.4.0/24 [90/26882560] via 192.168.100.4, 00:04:16, Tunnel10

D    192.168.2.0/24 [90/26882560] via 192.168.100.2, 00:04:07, Tunnel10

D    192.168.3.0/24 [90/26882560] via 192.168.100.3, 00:00:40, Tunnel10

Spoke3 to Spoke1 Communication

Before Spoke3 sends traffic to Spoke1 router, there is only a static NHRP mapping to the Hub router (the same is true for Spoke1 router as well).

Static NHRP mapping on Spoke3 router

Spoke3# show ip nhrp

192.168.100.1/32 via 192.168.100.1

   Tunnel10 created 00:15:47, never expire

   Type: static, Flags: used

   NBMA address: 172.17.1.1

When Spoke3 router sends traffic to the network behind (192.168.2.0/24) Spoke1 router sourced from the network behind it (192.168.4.0/24), it looks up its routing table and finds the next-hop as Spoke2 router. However, it does not have NHRP mapping for Spoke1 router. Hence it devices to send a Resolution Request to the NHS and meanwhile still send the packets to Spoke1 through the Hub.

debug NHRP

Spoke3# debug nhrp

NHRP protocol debugging is on

Spoke3#

Spoke3# ping 192.168.2.1 source fastethernet 0/0

Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 192.168.2.1, timeout is 2 seconds:

Packet sent with a source address of 192.168.4.1

*Jul  1 00:02:32.959: NHRP: Attempting to send packet via DEST 192.168.100.1

*Jul  1 00:02:32.959: NHRP: NHRP successfully resolved 192.168.100.1 to NBMA 172.17.1.1

*Jul  1 00:02:32.963: NHRP: Encapsulation succeeded.  Tunnel IP addr 172.17.1.1

*Jul  1 00:02:32.963: NHRP: Send Registration Request via Tunnel10 vrf 0, packet size: 92

*Jul  1 00:02:32.963: NHRP: 120 bytes out Tunnel10

*Jul  1 00:02:33.199: NHRP: Receive Registration Reply via Tunnel10 vrf 0, packet size: 112

*Jul  1 00:02:33.203: NHRP: netid_in = 0, to_us = 1

*Jul  1 00:02:34.279: NHRP: MACADDR: if_in null netid-in 0 if_out Tunnel10 netid-out 123

*Jul  1 00:02:34.279: NHRP: Sending packet to NHS 192.168.100.1 on Tunnel10

*Jul  1 00:02:34.279: NHRP: NHRP successfully resolved 192.168.100.1 to NBMA 172.17.1.1

*Jul  1 00:02:34.283: NHRP: Checking for delayed event /192.168.100.2 on list (Tunnel10).

*Jul  1 00:02:34.283: NHRP: No node found.

*Jul  1 00:02:34.287: NHRP: Adding Tunnel Endpoints (VPN: 192.168.10!0.2, NBMA: 172.17.1.1)

*Jul  1 00:02:34.295: NHRP: Enqueued NHRP Resolution Request for destination: 192.168.100.2

*Jul  1 00:02:34.307: NHRP: Checking for delayed event /192.168.100.2 on list (Tunnel10).

*Jul  1 00:02:34.307: NHRP: No node found.

*Jul  1 00:02:34.311: NHRP: Sending NHRP Resolution Request for dest: 192.168.100.2 to NHS: 192.168.100.1 using our src: 192.168.100.4

*Jul  1 00:02:34.311: NHRP: Attempting to send packet via DEST 192.168.100.1

*Jul  1 00:02:34.311: NHRP: NHRP successfully resolved 192.168.100.1 to NBMA 172.17.1.1

*Jul  1 00:02:34.315: NHRP: Encapsulation succeeded.  Tunnel IP addr 172.17.1.1

*Jul  1 00:02:34.315: NHRP: Send Resolution Request via Tunnel10 vrf 0, packet size: 72

*Jul  1 00:02:34.319: NHRP: 100 bytes out Tunnel10

*Jul  1 00:02:34.951: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.17.1.1

*Jul  1 00:02:35.219: NHRP: Receive Resolution Request via Tunnel10 vrf 0, packet size: 92

*Jul  1 00:02:35.219: NHRP: netid_in = 123, to_us = 1

*Jul  1 00:02:35.223: NHRP: nhrp_rtlookup yielded Tunnel10

*Jul  1 00:02:35.223: NHRP: request was to us, responding with ouraddress

*Jul  1 00:02:35.223: NHR!!!P: Checking for delayed event 192.168.100.2/192.168.100.4 on list (Tunnel10).

*Jul  1 00:02:35.227: NHRP: No node found.

*Jul  1 00:02:35.227: NHRP: No need to delay processing of resolution event nbma src:172.47.1.1 nbma dst:172.27.1.1

*Jul  1 00:02:35.231: NHRP: Attempting to send packet via DEST 192.168.100.2

*Jul  1 00:02:35.231: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.17.1.1

*Jul  1 00:02:35.235: NHRP: Encapsulation succeeded.  Tunnel IP addr 172.17.1.1

*Jul  1 00:02:35.235: NHRP: Send Resolution Reply via Tunnel10 vrf 0, packet size: 120

*Jul  1 00:02:35.239: NHRP: 148 bytes out Tunnel10

Once Spoke3 router receives Resolution Reply message back from NHS with NHRP mapping for Spoke1 router, it attempts to send the packet directly to Spoke1 router rather than through Hub router.

*Jul  1 00:02:35.247: NHRP: Receive Resolution Reply via Tunnel10 vrf 0, packet size: 140

*Jul  1 00:02:35.251: NHRP: netid_in = 0, to_us = 1

*Jul  1 00:02:35.251: NHRP: Checking for delayed event /192.168.100.2 on list (Tunnel10).

*Jul  1 00:02:35.251: NHRP: No node found.

*Jul  1 00:02:35.255: NHRP: No need to delay processing of resolution event nbma src:172.47.1.1 nbma dst:172.27.1.1

*Jul  1 00:02:35.255: NHRP: Adding Tunnel Endpoints (VPN: 192.168.100.2, NBMA: 172.27.1.1)

*Jul  1 00:02:35.655: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.27.1.1

*Jul  1 00:02:36.123: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.27.1.1!

Success rate is 100 percent (5/5), round-trip min/avg/max = 280/483/700 ms

Spoke3#

*Jul  1 00:02:36.403: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.27.1.1

Spoke3# show ip nhrp

192.168.100.1/32 via 192.168.100.1

   Tunnel10 created 00:18:06, never expire

   Type: static, Flags: used

   NBMA address: 172.17.1.1

192.168.100.2/32 via 192.168.100.2

   Tunnel10 created 00:00:43, expire 00:03:17

   Type: dynamic, Flags: router

   NBMA address: 172.27.1.1

192.168.100.4/32 via 192.168.100.4

   Tunnel10 created 00:00:42, expire 00:03:17

   Type: dynamic, Flags: router unique local

   NBMA address: 172.47.1.1

    (no-socket)

Note that all NHRP communication and data-traffic are encrypted using GDOI policies. Also, no IPSec tunnels are created between Spoke3 and Spoke1 routers.