GDOI-based DMVPN
GDOI-based DMVPN
In traditional DMVPN, if Spoke-to-Spoke direct connectivity is enabled (i.e DMVPN Phase 2), a permanent IPSec tunnel is maintained between the Spoke and the Hub, and a dynamic IPSec tunnel is created on demand between the Spokes. When a Spoke router wishes to send traffic to another Spoke router, it sends a NHRP Resolution Request message to the DMVPN Hub for the destination Spoke router. Once the mapping is received, the Spoke router will initiate a dynamic IPSec tunnel with the destination Spoke router. Until this dynamic IPSec tunnel is created, traffic flows through the Hub. If traffic needs to come in reverse direction, the destination Spoke router follows the same process towards the originating Spoke router.
When a dynamic IPSec tunnel is created for Spoke-to-Spoke connectivity, it introduces a slight delay caused by IPSec negotiation which can lead to poor performance for certain real-time applications like VOIP. GDOI eliminates this delay.
With GDOI, the DMVPN Hub and Spokes are the Group Members (GMs). Group Keys and security policies are distributed to GMs by the Key Server (KS)- a separate Cisco IOS router. This eliminates the point-to-point IPSec session between the GMs. Once NHRP resolution is completed, the Spokes can communicate using the same key. This reduces delay between Spoke-to-Spoke communication by eliminating creation of dynamic IPSec tunnels between them.
Sample Scenario
Key Server (KS) Configuration
All GMs register with the KS and the KS is responsible for distributing keys and security policies. Since all traffic must flow through mGRE tunnel, the interesting traffic can be categorized as GRE traffic from any DMVPN router to other DMVPN router. The configuration of KS is as below:
KS Configuration
crypto key generate rsa general-keys label GDOI_KEYS modulus 1024 exportable
!
crypto isakmp policy 10
authentication pre-share
!
!---- Manually configuring pre-shared keys for all GMs
!
crypto isakmp key 0 cisco address 2.2.2.2 ! Hub router
crypto isakmp key 0 cisco address 11.11.11.11 ! Spoke1 router
crypto isakmp key 0 cisco address 12.12.12.12 ! Spoke2 router
crypto isakmp key 0 cisco address 13.13.13.13 ! Spoke3 router
!
crypto ipsec transform-set GDOI_TS edp-3des esp-md5-hmac
!
crypto ipsec profile GDOI_PROFILE
set transform-set GDOI_TS
!
crypto gdoi group GDOI_GROUP
identity number 123
server local
rekey transport unicast
rekey authentication mypubkey rsa GDOI_KEYS
rekey retransmit 60 number 2
!
sa ipsec 1
profile GDOI_PROFILE
match address ipv4 ACL
replay time window-size 5
!
address ipv4 1.1.1.1 ! Use this address to send rekey packets
!
!
ip access-list extended ACL
permit gre any any ! Encrypt all GRE packets
!
GM (Hub and Spoke) Tunnel and GDOI Configuration
The Hub and Spoke routers must be configured with a mGRE tunnel for DMVPN.
Tunnel Configuration on Hub
interface Tunnel 10
ip address 192.168.100.1 255.255.255.0
tunnel mode gre multipoint
tunnel source serial 2/0
tunnel key 123
ip nhrp map multicast dynamic
ip nhrp holdtime 240
ip nhrp network-id 123
no ip next-hop-self eigrp 10 ! Disable next-hop overwrite
no ip split-horizon eigrp 10 ! Allow routes to be re-advertised to Spoke routers
!
router eigrp 10
! routing instance for DMVPN
no auto-summary
network 192.168.100.0
network 192.168.1.0
!
The following shows configuration on Spoke1 router. A similar configuration applies to other Spoke routers.
Tunnel Configuration on Spoke1
interface Tunnel 10
ip address 192.168.100.2 255.255.255.0
tunnel mode gre multipoint
tunnel source serial 1/0
tunnel key 123
ip nhrp nhs 192.168.100.1
ip nhrp map 192.168.100.1
172.17.1.1
ip nhrp map multicast 172.17.1.1
ip nhrp holdtime 240
ip nhrp network-id 123
!
router eigrp 10
no auto-summary
network 192.168.100.0
network 192.168.2.0
!
Then GDOI configuration on Hub and Spoke routers is as follows:
GDOI Configuration
crypto isakmp policy 10
authentication pre-share
!
crypto isakmp key 0 cisco address 1.1.1.1
!
crypto gdoi group GDOI_GM
identity number 123
server address ipv4 1.1.1.1 ! Address of KS
!
crypto map DMVPN local-address Loopback0
crypto map DMVPN 10 gdoi
set group GDOI_GM
!
interface serial 1/0
....
crypto map DMVPN
!
Verification
First all routers (Hub and Spokes) register with the KS and download the policies configured on the KS. All routers complete IKE Phase 1 with the KS and start GDOI protocol by sending Register messages to the KS. In the end, the KS sends the Key Download Payload which contains the KEK and TEK keys. The below output shows successful IKE Phase 1 with all routers, but no IKE Phase 2 on KS.
KS# show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
1.1.1.1 13.13.13.13 GDOI_IDLE 1004 ACTIVE
1.1.1.1 11.11.11.11 GDOI_IDLE 1002 ACTIVE
1.1.1.1 12.12.12.12 GDOI_IDLE 1003 ACTIVE
1.1.1.1 2.2.2.2 GDOI_IDLE 1001 ACTIVE
IPv6 Crypto ISAKMP SA
KS# show crypto ipsec sa
No SAs found
The following output shows GDOI policy downloaded by GM Spoke1 router. The output will be similar for all GMs.
GDOI Policy on Spoke1
Spoke1# show crypto gdoi
GROUP INFORMATION
Group Name : GDOI_GM
Group Identity : 123
Rekeys received : 1
IPSec SA Direction : Both
Active Group Server : 1.1.1.1
Group Server list : 1.1.1.1
GM Reregisters in : 2287 secs
Rekey Received(hh:mm:ss) : 00:19:59
Rekeys received
Cumulative : 1
After registration : 1
Rekey Acks sent : 1
ACL Downloaded From KS 1.1.1.1:
access-list permit gre any any
KEK POLICY:
Rekey Transport Type : Unicast
Lifetime (secs) : 86399
Encrypt Algorithm : 3DES
Key Size : 192
Sig Hash Algorithm : HMAC_AUTH_SHA
Sig Key Length (bits) : 1024
TEK POLICY for the current KS-Policy ACEs Downloaded:
Serial1/0:
IPsec SA:
spi: 0x42CE20E(70050318)
transform: esp-3des esp-md5-hmac
sa timing:remaining key lifetime (sec): (2400)
Anti-Replay(Time Based) : 5 sec interval
All GMs have a single IKE session with KS. Notice that the remote endpoint peer is 0.0.0.0 i.e. Spoke1 is open for any other GM.
IKE Session on Spoke1
Spoke1# show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst src state conn-id status
1.1.1.1 11.11.11.11 GDOI_IDLE 1001 ACTIVE
11.11.11.11 1.1.1.1 GDOI_REKEY 1002 ACTIVE
11.11.11.11 1.1.1.1 GDOI_REKEY 1003 ACTIVE
IPv6 Crypto ISAKMP SA
Spoke1# show crypto ipsec sa
PFS (Y/N): N, DH group: none
interface: Serial1/0
Crypto map tag: DMVPN, local addr 11.11.11.11
protected vrf: (none)
local ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/47/0)
remote ident (addr/mask/prot/port): (0.0.0.0/0.0.0.0/47/0)
current_peer 0.0.0.0 port 848
PERMIT, flags={origin_is_acl,}
#pkts encaps: 708, #pkts encrypt: 708, #pkts digest: 708
#pkts decaps: 777, #pkts decrypt: 777, #pkts verify: 777
#pkts compressed: 0, #pkts decompressed: 0
#pkts not compressed: 0, #pkts compr. failed: 0
#pkts not decompressed: 0, #pkts decompress failed: 0
#send errors 0, #recv errors 0
local crypto endpt.: 11.11.11.11, remote crypto endpt.: 0.0.0.0
path mtu 1500, ip mtu 1500, ip mtu idb Serial1/0
current outbound spi: 0x42CE20E(70050318)
inbound esp sas:
spi: 0x42CE20E(70050318)
transform: esp-3des esp-md5-hmac ,
in use settings ={Tunnel, }
conn id: 5, flow_id: SW:5, sibling_flags 80000040, crypto map: DMVPN
sa timing: remaining key lifetime (sec): (2359)
IV size: 8 bytes
replay detection support: Y replay window size: 5
Status: ACTIVE
inbound ah sas:
inbound pcp sas:
outbound esp sas:
spi: 0x42CE20E(70050318)
transform: esp-3des esp-md5-hmac ,
in use settings ={Tunnel, }
conn id: 6, flow_id: SW:6, sibling_flags 80000040, crypto map: DMVPN
sa timing: remaining key lifetime (sec): (2359)
IV size: 8 bytes
replay detection support: Y replay window size: 5
Status: ACTIVE
outbound ah sas:
outbound pcp sas:
Then all Spoke routers then register with DMVPN_Hub router using NHRP Registration Request messages. The Hub router learns the Tunnel IP Address-to-NBMA address mapping dynamically. These messages are GRE encapsulated and hence encrypted by GDOI policies.
Hub learns NHRP mappings dynamically
DMVPN_Hub# show ip nhrp
192.168.100.2/32 via 192.168.100.2
Tunnel10 created 00:04:35, expire 00:03:34
Type: dynamic, Flags: unique registered used
NBMA address: 172.27.1.1
192.168.100.3/32 via 192.168.100.3
Tunnel10 created 00:01:30, expire 00:03:03
Type: dynamic, Flags: unique registered
NBMA address: 172.37.1.1
192.168.100.4/32 via 192.168.100.4
Tunnel10 created 00:04:35, expire 00:03:15
Type: dynamic, Flags: unique registered
NBMA address: 172.47.1.1
This forms EIGRP adjacency over the tunnel interface and routes are exchanged between DMVPN routers. Note that these EIGRP packets are also encapsulated in GRE as they travel over tunnel interface, and hence encapsulated by ESP. Notice the next-hop remains unchanged when these routes are advertised to Spoke routers.
EIGRP Routes learnt by Hub router via Tunnel interface
DMVPN_Hub# show ip route eigrp
D 192.168.4.0/24 [90/26882560] via 192.168.100.4, 00:04:16, Tunnel10
D 192.168.2.0/24 [90/26882560] via 192.168.100.2, 00:04:07, Tunnel10
D 192.168.3.0/24 [90/26882560] via 192.168.100.3, 00:00:40, Tunnel10
Spoke3 to Spoke1 Communication
Before Spoke3 sends traffic to Spoke1 router, there is only a static NHRP mapping to the Hub router (the same is true for Spoke1 router as well).
Static NHRP mapping on Spoke3 router
Spoke3# show ip nhrp
192.168.100.1/32 via 192.168.100.1
Tunnel10 created 00:15:47, never expire
Type: static, Flags: used
NBMA address: 172.17.1.1
When Spoke3 router sends traffic to the network behind (192.168.2.0/24) Spoke1 router sourced from the network behind it (192.168.4.0/24), it looks up its routing table and finds the next-hop as Spoke2 router. However, it does not have NHRP mapping for Spoke1 router. Hence it devices to send a Resolution Request to the NHS and meanwhile still send the packets to Spoke1 through the Hub.
debug NHRP
Spoke3# debug nhrp
NHRP protocol debugging is on
Spoke3#
Spoke3# ping 192.168.2.1 source fastethernet 0/0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.2.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.4.1
*Jul 1 00:02:32.959: NHRP: Attempting to send packet via DEST 192.168.100.1
*Jul 1 00:02:32.959: NHRP: NHRP successfully resolved 192.168.100.1 to NBMA 172.17.1.1
*Jul 1 00:02:32.963: NHRP: Encapsulation succeeded. Tunnel IP addr 172.17.1.1
*Jul 1 00:02:32.963: NHRP: Send Registration Request via Tunnel10 vrf 0, packet size: 92
*Jul 1 00:02:32.963: NHRP: 120 bytes out Tunnel10
*Jul 1 00:02:33.199: NHRP: Receive Registration Reply via Tunnel10 vrf 0, packet size: 112
*Jul 1 00:02:33.203: NHRP: netid_in = 0, to_us = 1
*Jul 1 00:02:34.279: NHRP: MACADDR: if_in null netid-in 0 if_out Tunnel10 netid-out 123
*Jul 1 00:02:34.279: NHRP: Sending packet to NHS 192.168.100.1 on Tunnel10
*Jul 1 00:02:34.279: NHRP: NHRP successfully resolved 192.168.100.1 to NBMA 172.17.1.1
*Jul 1 00:02:34.283: NHRP: Checking for delayed event /192.168.100.2 on list (Tunnel10).
*Jul 1 00:02:34.283: NHRP: No node found.
*Jul 1 00:02:34.287: NHRP: Adding Tunnel Endpoints (VPN: 192.168.10!0.2, NBMA: 172.17.1.1)
*Jul 1 00:02:34.295: NHRP: Enqueued NHRP Resolution Request for destination: 192.168.100.2
*Jul 1 00:02:34.307: NHRP: Checking for delayed event /192.168.100.2 on list (Tunnel10).
*Jul 1 00:02:34.307: NHRP: No node found.
*Jul 1 00:02:34.311: NHRP: Sending NHRP Resolution Request for dest: 192.168.100.2 to NHS: 192.168.100.1 using our src: 192.168.100.4
*Jul 1 00:02:34.311: NHRP: Attempting to send packet via DEST 192.168.100.1
*Jul 1 00:02:34.311: NHRP: NHRP successfully resolved 192.168.100.1 to NBMA 172.17.1.1
*Jul 1 00:02:34.315: NHRP: Encapsulation succeeded. Tunnel IP addr 172.17.1.1
*Jul 1 00:02:34.315: NHRP: Send Resolution Request via Tunnel10 vrf 0, packet size: 72
*Jul 1 00:02:34.319: NHRP: 100 bytes out Tunnel10
*Jul 1 00:02:34.951: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.17.1.1
*Jul 1 00:02:35.219: NHRP: Receive Resolution Request via Tunnel10 vrf 0, packet size: 92
*Jul 1 00:02:35.219: NHRP: netid_in = 123, to_us = 1
*Jul 1 00:02:35.223: NHRP: nhrp_rtlookup yielded Tunnel10
*Jul 1 00:02:35.223: NHRP: request was to us, responding with ouraddress
*Jul 1 00:02:35.223: NHR!!!P: Checking for delayed event 192.168.100.2/192.168.100.4 on list (Tunnel10).
*Jul 1 00:02:35.227: NHRP: No node found.
*Jul 1 00:02:35.227: NHRP: No need to delay processing of resolution event nbma src:172.47.1.1 nbma dst:172.27.1.1
*Jul 1 00:02:35.231: NHRP: Attempting to send packet via DEST 192.168.100.2
*Jul 1 00:02:35.231: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.17.1.1
*Jul 1 00:02:35.235: NHRP: Encapsulation succeeded. Tunnel IP addr 172.17.1.1
*Jul 1 00:02:35.235: NHRP: Send Resolution Reply via Tunnel10 vrf 0, packet size: 120
*Jul 1 00:02:35.239: NHRP: 148 bytes out Tunnel10
Once Spoke3 router receives Resolution Reply message back from NHS with NHRP mapping for Spoke1 router, it attempts to send the packet directly to Spoke1 router rather than through Hub router.
*Jul 1 00:02:35.247: NHRP: Receive Resolution Reply via Tunnel10 vrf 0, packet size: 140
*Jul 1 00:02:35.251: NHRP: netid_in = 0, to_us = 1
*Jul 1 00:02:35.251: NHRP: Checking for delayed event /192.168.100.2 on list (Tunnel10).
*Jul 1 00:02:35.251: NHRP: No node found.
*Jul 1 00:02:35.255: NHRP: No need to delay processing of resolution event nbma src:172.47.1.1 nbma dst:172.27.1.1
*Jul 1 00:02:35.255: NHRP: Adding Tunnel Endpoints (VPN: 192.168.100.2, NBMA: 172.27.1.1)
*Jul 1 00:02:35.655: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.27.1.1
*Jul 1 00:02:36.123: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.27.1.1!
Success rate is 100 percent (5/5), round-trip min/avg/max = 280/483/700 ms
Spoke3#
*Jul 1 00:02:36.403: NHRP: NHRP successfully resolved 192.168.100.2 to NBMA 172.27.1.1
Spoke3# show ip nhrp
192.168.100.1/32 via 192.168.100.1
Tunnel10 created 00:18:06, never expire
Type: static, Flags: used
NBMA address: 172.17.1.1
192.168.100.2/32 via 192.168.100.2
Tunnel10 created 00:00:43, expire 00:03:17
Type: dynamic, Flags: router
NBMA address: 172.27.1.1
192.168.100.4/32 via 192.168.100.4
Tunnel10 created 00:00:42, expire 00:03:17
Type: dynamic, Flags: router unique local
NBMA address: 172.47.1.1
(no-socket)
Note that all NHRP communication and data-traffic are encrypted using GDOI policies. Also, no IPSec tunnels are created between Spoke3 and Spoke1 routers.