CCIE:Sec practice lab – Day 2-5 – VRF-Aware GETVPN

Day 2, 3, 4 and now 5. and I am getting my arse kicked! It’s all about GETVPN. Bearing in mind this will only be the second/third time I have tried it, and this time, it’s VRF based (known as VRF-aware GETVPN). One VRF will go to one GETVPN server, the other VRF to the other, and these will share the routes they get down to the NYC server, which should then pass them where needed.

This part of the topology is actually a little more complex than I had originally envisaged, and if you don’t mind me switching from present to past tense, I spent a good few hours trying to get this to work but have made some errors in either design or understanding.

Here is the issue. GETVPN-Client needs to have two different VRFs. This is the easy part. The hard part is that the routes to the tunnel destination are outside of any VRF. Options here are:

  1. Set up the main interface (Gi0/0) in a third VRF and import/export the RT’s into and out of this.
  2. Set up a tunnel VRF as well as the VRF forwarding, using different VRFs.
  3. Set up VLANs – this will mean a change to the IP addressing scheme

Number one has issues in that we cannot change the topology to suit our needs. Certainly in a lab exam, I cannot change the topology to support and fix a wrong solution. This goes for number three as well. So, I tried out number 2, and although I thought I had cracked it, I have ended up reaching out for help.

The nuts and bolts are in place in terms of connectivity, and it’s now day 5 and I need to move on (one way or another), so we’ll crack on and set up the switch (to give access to the WWW and ACS boxes):

Switch(config)#vlan 3,4
Switch(config-vlan)#exit
Switch(config)#int gi0/1
Switch(config-if)#swi mo acc
Switch(config-if)#swi acc vl 3
Switch(config-if)#int gi0/2
Switch(config-if)#swi mo acc
Switch(config-if)#swi acc vl 4
Switch(config-if)#exi
Switch(config)#int gi0/0
Switch(config-if)#swi tru enc dot
Switch(config-if)#swi mo tru
Switch(config-if)#

For GDOI to work, we need to permit a bunch of stuff through the firewalls:

ASA9(config)# sh run | i access-list
access-list Inside->Outside extended permit icmp host 10.1.26.1 host 10.1.2.254 
access-list Inside->Outside extended permit udp host 10.1.26.1 host 10.1.2.254 eq 848 
access-list Inside->Outside extended permit udp host 10.1.26.1 host 10.1.2.254 eq isakmp 
access-list Inside->Outside extended permit udp host 10.1.26.1 host 10.1.2.254 eq 4500 
access-list Inside->Outside extended permit esp host 10.1.26.1 host 10.1.2.254 
access-list Outside->Inside extended permit icmp host 10.1.2.254 host 10.1.26.1 
access-list Outside->Inside extended permit udp host 10.1.2.254 host 10.1.26.1 eq 848 
access-list Outside->Inside extended permit udp host 10.1.2.254 host 10.1.26.1 eq isakmp 
access-list Outside->Inside extended permit udp host 10.1.2.254 host 10.1.26.1 eq 4500 
access-list Outside->Inside extended permit esp host 10.1.2.254 host 10.1.26.1 
threat-detection statistics access-list
ASA9(config)#

ASAv6(config)# sh run | i access-list
access-list Outside->Inside extended permit icmp host 10.1.2.1 host 10.1.2.254 
access-list Outside->Inside extended permit udp host 10.1.2.254 eq 848 host 10.1.9.103 eq 848 
access-list Outside->Inside extended permit udp host 10.1.9.103 eq 848 host 10.1.2.254 eq 848 
access-list Outside->Inside extended permit udp host 10.1.9.103 host 10.1.2.254 eq 848 
access-list Outside->Inside extended permit icmp host 10.1.9.103 host 10.1.2.254 
access-list Outside->Inside extended permit esp host 10.1.9.103 host 10.1.2.254 
access-list Outside->Inside extended permit udp host 10.1.9.103 host 10.1.2.254 eq 4500 
access-list Outside->Inside extended permit udp host 10.1.9.103 host 10.1.2.254 eq isakmp 
threat-detection statistics access-list
ASAv6(config)#

I do have more rules than needed, but the plan is to then prune back at the end of the lab (keep it neat and so on).

Originally I rushed in trying to get things working with virtual templates, VRFs and so on, but have now reverted back to a single layer, with no VRFs. It’s working and the client has registered. Here are the (relevant) configs, the VRFs are still present, but not being used. We have a sort-of VRF-Aware GETVPN setup:

GETVPN-Client#sh run
!
ip vrf FVRF
 rd 100:100
!
ip vrf RED
 rd 103:103
!
ip vrf WHITE
 rd 104:104
!
crypto keyring RED-G1  
  pre-shared-key address 10.1.9.103 key CCIE
!
crypto isakmp policy 10
 encr 3des
 authentication pre-share
 group 2
 lifetime 3600
!
crypto isakmp policy 20
 encr aes 256
 authentication pre-share
 group 2
crypto isakmp key CCIE address 10.1.9.103     
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 10
crypto isakmp profile GET-ISAK-Profile-RED
   keyring RED-G1
   match identity address 10.1.9.103 255.255.255.255 
!
!
crypto ipsec transform-set GET-TS esp-3des esp-sha-hmac 
 mode tunnel
!
crypto ipsec profile GET-IPS-Profile-RED
 set group G1-RED
 set transform-set GET-TS 
 set isakmp-profile GET-ISAK-Profile-RED
!
!
crypto gdoi group G1-RED
 identity number 103
 server address ipv4 10.1.9.103
 client registration interface GigabitEthernet0/0
!
!
crypto map G1-RED isakmp-profile GET-ISAK-Profile-RED
crypto map G1-RED 103 gdoi 
 set group G1-RED
!
interface GigabitEthernet0/0
 ip address 10.1.2.254 255.255.255.0
 crypto map G1-RED
!
ip route 0.0.0.0 0.0.0.0 10.1.2.1
end

GETVPN-S1#sh run
!
crypto keyring RED-Keyring  
  pre-shared-key address 10.1.2.254 key CCIE
!
crypto ikev2 profile GET-IKEv2-Profile
 match address local interface GigabitEthernet0/1
 match identity remote address 10.1.2.254 255.255.255.255 
 authentication remote pre-share
 authentication local pre-share
!
!
!
crypto ipsec transform-set GET-TS esp-3des esp-sha-hmac 
 mode tunnel
!
crypto ipsec profile GET-Profile
!
crypto gdoi group G1-RED
 identity number 103
 server local
  rekey algorithm aes 128
  rekey retransmit 10 number 3
  rekey authentication mypubkey rsa GET-RSA-Key
  rekey transport unicast
  registration interface GigabitEthernet0/1
  sa ipsec 103
   profile GET-Profile
   match address ipv4 103
   replay counter window-size 64
   no tag
  address ipv4 10.1.26.1
!
ip route 0.0.0.0 0.0.0.0 10.1.26.200
!
access-list 103 permit tcp any any

The NAT rules are set up on the ASA:

ASA9(config)# object network obj-GETVPN-S1
ASA9(config-network-object)# host 10.1.26.1
ASA9(config-network-object)# nat (inside,outside) 1 source static obj-GETVPN-S1 obj-GETVPN-S1-external 
ERROR: obj-GETVPN-S1-external doesn't match an existing object or object-group
ASA9(config)# object network obj-GETVPN-S1-external                                     
ASA9(config-network-object)# host 10.1.9.103
ASA9(config-network-object)# exit
ASA9(config)# object network obj-GETVPN-S1                                              
ASA9(config-network-object)# nat (inside,outside) 1 source static obj-GETVPN-S1 obj-GETVPN-S1-external 
ASA9(config)#

The client happily registers with this configuration:

GETVPN-Client#sh cry gdoi
GROUP INFORMATION

    Group Name               : G1-RED
    Group Identity           : 103
    Group Type               : GDOI (ISAKMP)
    Crypto Path              : ipv4
    Key Management Path      : ipv4
    Rekeys received          : 0
    IPSec SA Direction       : Both

     Group Server list       : 10.1.9.103
                               
Group Member Information For Group G1-RED:
    IPSec SA Direction       : Both
    ACL Received From KS     : gdoi_group_G1-RED_temp_acl

    Group member             : 10.1.2.254      vrf: None
       Local addr/port       : 10.1.2.254/848
       Remote addr/port      : 10.1.9.103/848
       fvrf/ivrf             : None/None
       Version               : 1.0.17
       Registration status   : Registered
       Registered with       : 10.1.9.103
       Re-registers in       : 2222 sec
       Succeeded registration: 1
       Attempted registration: 5
       Last rekey from       : 0.0.0.0
       Last rekey seq num    : 0
       Unicast rekey received: 0
       Rekey ACKs sent       : 0
       Rekey Received        : never
       DP Error Monitoring   : OFF
       IPSEC init reg executed    : 0
       IPSEC init reg postponed   : 0
       Active TEK Number     : 1
       SA Track (OID/status) : disabled

       allowable rekey cipher: any
       allowable rekey hash  : any
       allowable transformtag: any ESP

    Rekeys cumulative
       Total received        : 0
       After latest register : 0
       Rekey Acks sents      : 0

 ACL Downloaded From KS 10.1.9.103:
   access-list   permit tcp any any

KEK POLICY:
    Rekey Transport Type     : Unicast
    Lifetime (secs)          : 85231
    Encrypt Algorithm        : AES
    Key Size                 : 128     
    Sig Hash Algorithm       : HMAC_AUTH_SHA
    Sig Key Length (bits)    : 1296    

TEK POLICY for the current KS-Policy ACEs Downloaded:
  GigabitEthernet0/0:
    IPsec SA:
        spi: 0x80E62B18(2162567960)
        KGS: Disabled
        transform: esp-aes esp-sha-hmac 
        sa timing:remaining key lifetime (sec): (2433)
        Anti-Replay(Counter Based) : 64
        tag method : disabled
        alg key size: 16 (bytes)
        sig key size: 20 (bytes)
        encaps: ENCAPS_TUNNEL
          
GETVPN-Client#

Now this is where it gets frustrating fun. I have proved that GDOI *should* work. Now, GETVPN-Client needs to be switched about, so that the Gi0/0 interface is in VRF FVRF (Front-VRF) and the G1-RED (and then G2-WHITE) are connecting from different VRFs.

I’ll start by moving everything to FVRF and confirm that it still works:

GETVPN-Client(config)#interface GigabitEthernet0/0
GETVPN-Client(config-if)# ip vrf for FVRF
% Interface GigabitEthernet0/0 IPv4 disabled and address(es) removed due to enabling VRF FVRF
GETVPN-Client(config-if)# ip address 10.1.2.254 255.255.255.0
GETVPN-Client(config-if)# crypto map G1-RED
GETVPN-Client(config-if)#exit
GETVPN-Client(config)#ip route vrf FVRF 0.0.0.0 0.0.0.0 10.1.2.1
GETVPN-Client(config)#crypto isakmp profile GET-ISAK-Profile-RED
GETVPN-Client(conf-isa-prof)#   no keyring RED-G1
GETVPN-Client(conf-isa-prof)#no match identity address 10.1.9.103 255.255.255.255   
GETVPN-Client(conf-isa-prof)#no crypto keyring RED-G1 
GETVPN-Client(config)#crypto keyring RED-G1 vrf FVRF
GETVPN-Client(conf-keyring)#  pre-shared-key address 10.1.9.103 key CCIE
GETVPN-Client(conf-keyring)# crypto isakmp profile GET-ISAK-Profile-RED
% A profile is deemed incomplete until it has match identity statements
GETVPN-Client(conf-isa-prof)#   keyring RED-G1
GETVPN-Client(conf-isa-prof)#match identity address 10.1.9.103 255.255.255.255 FVRF 
*Jul 19 11:23:22.674: %CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(conf-isa-prof)#

This gets registered:

GETVPN-Client(conf-keyring)#do sh cry gdo | i fvrf|status|access-list
       fvrf/ivrf             : FVRF/FVRF
       Registration status   : Registered
       SA Track (OID/status) : disabled
   access-list   permit tcp any any
GETVPN-Client(conf-keyring)#

So far so good. Now, can we change the G1-RED GDOI group to use VRF RED? We’ll need to use the ISAKMP policy to split the G1-RED into using a different interface, and the only option here is to use a virtual-template:

GETVPN-Client(config-if)#crypto isakmp profile GET-ISAK-Profile-RED
GETVPN-Client(conf-isa-prof)#?     
Crypto ISAKMP Profile Commands are:

  accounting        Enable AAA Accounting for IPSec Sessions
  ca                Specify certificate authorities to trust
  client            Specify client configuration settings
  default           Set a command to its defaults
  description       Specify a description of this profile
  exit              Exit from crypto isakmp profile sub mode
  initiate          Initiator property
  isakmp            ISAKMP Authorization command
  keepalive         Set a keepalive interval for use with IOS peers
  keyring           Specify keyring to use
  local-address     Interface to use for local address for this isakmp profile
  match             Match values of peer
  no                Negate a command or set its defaults
  qos-group         Apply a Qos policy class map for this profile
  self-identity     Specify Identity to use
  virtual-template  Specify the virtual-template for dynamic interface creation.
  vrf               Specify the VRF it is related to

GETVPN-Client(conf-isa-prof)#

Here is the virtual-template:

GETVPN-Client(config-if)#do sh run int virtual-templ 3 | b interface
interface Virtual-Template3 type tunnel
 ip vrf forwarding RED
 ip unnumbered GigabitEthernet0/0
 tunnel source GigabitEthernet0/0
 tunnel mode ipsec ipv4
 tunnel destination 10.1.9.103
 tunnel vrf FVRF
 tunnel protection ipsec profile GET-IPS-Profile-RED
end

GETVPN-Client(config-if)#

We specify that we are forwarding for VRF RED, but the tunnel VRF should use FVRF, these are the “ivrf” and “fvrf” shown in the “sh cry gdoi” output above. So, should I be using the IP address from Gi0/0 (as it is in a different VRF)? Well, the template seems happy about this:

GETVPN-Client(conf-isa-prof)#do sh ip int bri | i Virtual
Virtual-Template3          10.1.2.254      YES unset  up                    down    
GETVPN-Client(conf-isa-prof)#

Let’s change the ISAKMP profile and find out if this works:

GETVPN-Client(config-if)#crypto isakmp profile GET-ISAK-Profile-RED
GETVPN-Client(conf-isa-prof)#virtual-template 3
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#int gi 0/0
GETVPN-Client(config-if)#shut
GETVPN-Client(config-if)#
%LINK-5-CHANGED: Interface GigabitEthernet0/0, changed state to administratively down
GETVPN-Client(config-if)#
%LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0, changed state to down
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#no shut
GETVPN-Client(config-if)#
%LINK-3-UPDOWN: Interface GigabitEthernet0/0, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/0, changed state to up
GETVPN-Client(config-if)#

The GDOI tunnel is still up, but I don’t think it’s picked up the change yet:

ETVPN-Client(config-if)#
%GDOI-5-SA_TEK_UPDATED: SA TEK was updated
%GDOI-5-GM_RECV_REKEY: Received Rekey for group G1-RED from 10.1.26.1 to 10.1.2.254 with seq # 1, spi 0xCB99183CA67992C99B4503A3EDCF75C2
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(config-if)#end
GETVPN-Client#clear cry  gdoi 
% The Key Server and Group Member will destroy created and downloaded policies.
% All Group Members are required to re-register.

Are you sure you want to proceed ? [yes/no]: yes
GETVPN-Client#
%GDOI-4-GM_RE_REGISTER: The IPSec SA created for group G1-RED may have expired/been cleared, or didn't go through. Re-register to KS.
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-SA_TEK_UPDATED: SA TEK was updated
%GDOI-5-SA_KEK_UPDATED: SA KEK was updated 0xCB99183CA67992C99B4503A3EDCF75C2
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client#
GETVPN-Client#sh cry gdoi | i status
       Registration status   : Registered
       SA Track (OID/status) : disabled
GETVPN-Client#

It’s not using the ivrf of RED, like I hoped it would. I need to make one more change to the ISAKMP profile:

GETVPN-Client(config)#crypto isakmp profile GET-ISAK-Profile-RED
GETVPN-Client(conf-isa-prof)#vrf RED
% VRF configured in VirtualTemplate will get precedence over IKE Profile VRF
GETVPN-Client(conf-isa-prof)#end
GETVPN-Client#
GETVPN-Client#clear cry  gdoi       
% The Key Server and Group Member will destroy created and downloaded policies.
% All Group Members are required to re-register.

Are you sure you want to proceed ? [yes/no]: yes
GETVPN-Client#
%GDOI-4-GM_RE_REGISTER: The IPSec SA created for group G1-RED may have expired/been cleared, or didn't go through. Re-register to KS.
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-SA_TEK_UPDATED: SA TEK was updated
%GDOI-5-SA_KEK_UPDATED: SA KEK was updated 0xCB99183CA67992C99B4503A3EDCF75C2
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client#

The tunnel is still using FRVF for both the ivrf and the fvrf. I’ll make some changes, one at a time, changing them back if there is no success. Starting with the crypto keyring:

GETVPN-Client(config)#crypto keyring RED-G1-RED vrf RED
GETVPN-Client(conf-keyring)#pre-shared-key address 10.1.9.103 key CCIE
GETVPN-Client(conf-keyring)#exit
GETVPN-Client(config)#crypto isakmp profile GET-ISAK-Profile-RED
GETVPN-Client(conf-isa-prof)#keyring RED-G1-RED
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#do clear cry gd
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#!! NOPE !!
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#no keyring RED-G1
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#do clear cry gd  
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#!! NOPE !!       
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#no keyring RED-G1-RED
GETVPN-Client(conf-isa-prof)#keyring RED-G1       
GETVPN-Client(conf-isa-prof)#
GETVPN-Client(conf-isa-prof)#do clear cry gd      
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(conf-isa-prof)#

It’s not the keyring. Maybe we need to try connecting from a different interface, such as a loopback:

GETVPN-Client(conf-isa-prof)#do sh run int loop 103 | b interface
interface Loopback103
 ip vrf forwarding RED
 ip address 10.1.103.1 255.255.255.0
end

GETVPN-Client(conf-isa-prof)#int loop 103
GETVPN-Client(config-if)#crypto map G1-RED
GETVPN-Client(config)#int gi0/0
GETVPN-Client(config-if)#no crypto map G1-RED
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#int virtual-template 3
GETVPN-Client(config-if)#tun source loop 103
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#do clear cry gdoi
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(config-if)#

Nope.

GETVPN-Client(config-if)#ip unnum loop 103
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#do clear cry gdo
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(config-if)#

Still nope.

GETVPN-Client(config-if)#crypto gdoi group G1-RED                        
GETVPN-Client(config-gkm-group)#client registration interface loop 103
GETVPN-Client(config-gkm-group)#
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.103.1 fvrf RED ivrf RED
GETVPN-Client(config-gkm-group)#
GETVPN-Client(config-gkm-group)#

The first change, but now everything is going through VRF RED, and we are not registering with the GDOI server.

GETVPN-Client(config-gkm-group)#interface Virtual-Template3 type tunnel
GETVPN-Client(config-if)#tunnel source gi0/0
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#do clear cry gdoi
GETVPN-Client(config-if)#
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.103.1 fvrf RED ivrf RED
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#

OK, I can spot an issue with the configuration as it currently is:

GETVPN-Client(config-if)#do sh run int virtual-tem 3
interface Virtual-Template3 type tunnel
 ip vrf forwarding RED
 ip unnumbered Loopback103
 tunnel source GigabitEthernet0/0
 tunnel mode ipsec ipv4
 tunnel destination 10.1.9.103
 tunnel vrf FVRF
 tunnel protection ipsec profile GET-IPS-Profile-RED
end

GETVPN-Client(config-if)#do sh run int loop103
interface Loopback103
 ip vrf forwarding RED
 ip address 10.1.103.1 255.255.255.0
 crypto map G1-RED
end

GETVPN-Client(config-if)#do sh run int gi0/0
interface GigabitEthernet0/0
 ip vrf forwarding FVRF
 ip address 10.1.2.254 255.255.255.0
 duplex auto
 speed auto
 media-type rj45
end

GETVPN-Client(config-if)#

The crypto map is applied to an interface that will not be accepting the traffic (loop103). We are just using this for the IP address, not for taking any actual traffic. Let’s set up the crypto map on the virtual template:

GETVPN-Client(config-if)#interface Virtual-Template3 type tunnel
GETVPN-Client(config-if)#crypto map G1-RED
% NOTE: crypto map is configured on tunnel interface.
        Currently only GDOI crypto map is supported on tunnel interface.

GETVPN-Client(config-if)#do clear cry gdoi
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.103.1 fvrf RED ivrf RED
GETVPN-Client(config-if)#

Still not right.

GETVPN-Client(config-if)#crypto gdoi group G1-RED
GETVPN-Client(config-gkm-group)#client registration interface gi0/0
GETVPN-Client(config-gkm-group)#
%GDOI-5-GM_REGS_COMPL: Registration to KS 10.1.9.103 complete for group G1-RED using address 10.1.2.254 fvrf FVRF ivrf FVRF
%GDOI-5-GM_INSTALL_POLICIES_SUCCESS: SUCCESS: Installation of Reg/Rekey policies from KS 10.1.9.103 for group G1-RED & gm identity 10.1.2.254 fvrf FVRF ivrf FVRF
GETVPN-Client(config-gkm-group)#

After much tooing-and-frowing (including setting up the G2-WHITE GDOI group), I have come to the conclusion that what I am trying to achieve with VRF-Aware GETVPN, is not possible. Therefore need to reconfigure the network.

Screw it. We’ll go down the same route as all the docs I have found us, which is to use VLANs and sub-interfaces. The clock is ticking and I need to get on

LON-1(config)#int gi0/1
LON-1(config-if)#no ip add
LON-1(config-if)#int gi0/1.1
LON-1(config-subif)#encap dot 10
LON-1(config-subif)#ip address 10.1.2.1 255.255.255.0
LON-1(config-subif)#exi
LON-1(config)#int gi0/1.103
LON-1(config-subif)#encap dot 103
LON-1(config-subif)#ip add 10.1.103.1 255.255.255.0
LON-1(config-subif)#exi
LON-1(config)#int gi0/1.104
LON-1(config-subif)#encap dot 104
LON-1(config-subif)#ip add 10.1.104.1 255.255.255.0
LON-1(config-subif)#do sh ip int bri
Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         10.1.1.1        YES NVRAM  up                    up      
GigabitEthernet0/1         unassigned      YES manual up                    up      
GigabitEthernet0/1.1       10.1.2.1        YES manual up                    up      
GigabitEthernet0/1.103     10.1.103.1      YES manual up                    up      
GigabitEthernet0/1.104     10.1.104.1      YES manual up                    up      
Tunnel0                    192.168.1.11    YES manual up                    up      
LON-1(config-subif)#

ASAv6(config)# int gi0/0
ASAv6(config-if)# no nameif outside
ASAv6(config-if)# no bridge-gro 1
ASAv6(config-if)# 
ASAv6(config-if)# int gi0/0.1
ASAv6(config-subif)# vlan 10
ASAv6(config-subif)# bridge-group 1
ASAv6(config-subif)# exi
ASAv6(config)# int gi0/0.103
ASAv6(config-subif)# vlan 301
ASAv6(config-subif)# bridge-group 103
ASAv6(config-subif)# exi
ASAv6(config)# int gi0/0.104
ASAv6(config-subif)# vlan 104
ASAv6(config-subif)# bridge 104
ASAv6(config-subif)# exit
ASAv6(config)# sh int ip bri
Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         unassigned      YES unset  up                    up  
GigabitEthernet0/0.1       10.1.2.200      YES unset  up                    up  
GigabitEthernet0/0.103     unassigned      YES unset  up                    up  
GigabitEthernet0/0.104     unassigned      YES unset  up                    up  
GigabitEthernet0/1         10.1.2.200      YES unset  up                    up  
BVI1                       10.1.2.200      YES manual up                    up  
ASAv6(config)# int bvi 103
ASAv6(config-if)# ip ad 10.1.103.200 255.255.255.0
ASAv6(config-if)# int bvi 104
ASAv6(config-if)# ip ad 10.1.104.200 255.255.255.0
ASAv6(config-if)# int gi0/1
ASAv6(config-if)# no nameif inside
ASAv6(config-if)# no bridg 1
ASAv6(config-if)# int gi0/1.1
ASAv6(config-subif)# vlan 11
ASAv6(config-subif)# bridge 1
ASAv6(config-subif)# int gi0/1.103
ASAv6(config-subif)# vlan 103
ASAv6(config-subif)# bridg 103
ASAv6(config-subif)# int gi0/1.104
ASAv6(config-subif)# vlan 401
ASAv6(config-subif)# bridge 104
ASAv6(config-subif)#  
ASAv6(config-subif)# int GigabitEthernet0/0.1
ASAv6(config-subif)# nameif Outside
INFO: Security level for "Outside" set to 0 by default.
ASAv6(config-subif)# int GigabitEthernet0/1.1
ASAv6(config-subif)# nameif Inside
INFO: Security level for "Inside" set to 100 by default.
ASAv6(config-subif)# int GigabitEthernet0/0.103
ASAv6(config-subif)# nameif Outside-RED
INFO: Security level for "Outside-RED" set to 0 by default.
ASAv6(config-subif)# int GigabitEthernet0/0.104
ASAv6(config-subif)# nameif Outside-WHITE
INFO: Security level for "Outside-WHITE" set to 0 by default.
ASAv6(config-subif)# int GigabitEthernet0/1.103                      
ASAv6(config-subif)# nameif Inside-RED
INFO: Security level for "Inside-RED" set to 0 by default.
ASAv6(config-subif)# sec 100
ASAv6(config-subif)# int GigabitEthernet0/1.104
ASAv6(config-subif)# nameif Inside-WHITE       
INFO: Security level for "Inside-WHITE" set to 0 by default.
ASAv6(config-subif)# sec 100
ASAv6(config-subif)#    

GETVPN-Client(config)#interface GigabitEthernet0/0
GETVPN-Client(config-if)#no ip vrf forwarding FVRF
GETVPN-Client(config-if)#
GETVPN-Client(config-if)#int gi0/0.1
GETVPN-Client(config-subif)#encap dot 11
GETVPN-Client(config-subif)#ip vrf for FVRF
GETVPN-Client(config-subif)#ip address 10.1.2.254 255.255.255.0
GETVPN-Client(config-subif)#
GETVPN-Client(config-subif)#int gi0/0.103
GETVPN-Client(config-subif)#encap dot 103
GETVPN-Client(config-subif)#ip add 10.1.103.254 255.255.255.0
GETVPN-Client(config-subif)#ip vrf for RED
GETVPN-Client(config-subif)#ip add 10.1.103.254 255.255.255.0
% 10.1.103.0 overlaps with Loopback103
GETVPN-Client(config-subif)#
GETVPN-Client(config-subif)#no int lo 103  
GETVPN-Client(config)#int gi0/1.103                    
GETVPN-Client(config-subif)#ip add 10.1.103.254 255.255.255.0
GETVPN-Client(config-subif)#
GETVPN-Client(config-subif)#no int loo 104
GETVPN-Client(config-subif)#int gi 0/0.104
GETVPN-Client(config-subif)#ip vrf for WHITE
GETVPN-Client(config-subif)#encap dot 401
GETVPN-Client(config-subif)#ip add 10.1.104.254 255.255.255.0
GETVPN-Client(config-subif)#

LON-1(config)#router eigrp 1
LON-1(config-router)#network 10.1.103.0 0.0.0.255
LON-1(config-router)#network 10.1.104.0 0.0.0.255
LON-1(config-router)#no redistr static metric 100 10 255 1 1500 route-map vrf-routes        
LON-1(config-router)#

Now, let’s get the GDOI’s working!

GETVPN-Client(config-gkm-group)#crypto gdoi group G1-RED
GETVPN-Client(config-gkm-group)#no client registration interface GigabitEthernet0/0.1
GETVPN-Client(config-gkm-group)#crypto gdoi group G2-WHITE
GETVPN-Client(config-gkm-group)#no client registration interface GigabitEthernet0/0.1
GETVPN-Client(config-gkm-group)#
GETVPN-Client(config-gkm-group)#interface Virtual-Template3 type tunnel
GETVPN-Client(config-if)#int gi0/1.103
GETVPN-Client(config-subif)#crypto map G1-RED
GETVPN-Client(config-subif)#   
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.103.254 fvrf RED ivrf RED
GETVPN-Client(config-subif)#int gi0/1.104
GETVPN-Client(config-subif)#crypto map G2-WHITE                   
GETVPN-Client(config-subif)#
%CRYPTO-5-GM_REGSTER: Start registration to KS 10.1.9.103 for group G1-RED using address 10.1.103.254 fvrf RED ivrf RED
GETVPN-Client(config-subif)#interface Virtual-Template3 type tunnel
GETVPN-Client(config-if)#no tunnel vrf FVRF
GETVPN-Client(config-if)#interface Virtual-Template4 type tunnel                                       
GETVPN-Client(config-if)#no tunnel vrf FVRF                     
GETVPN-Client(config-if)#exit
GETVPN-Client(config)#ip route vrf RED 0.0.0.0 0.0.0.0 10.1.103.1
GETVPN-Client(config)#ip route vrf WHITE 0.0.0.0 0.0.0.0 10.1.104.1 
GETVPN-Client(config)#

ASAv6(config)# access-list FVRF extended permit icmp host 10.1.2.1 host 10.1.2.254
ASAv6(config)# access-group FVRF in int Outside
ASAv6(config)# access-list RED extended permit icmp host 10.1.103.1 host 10.1.103.254
ASAv6(config)# access-group RED in interface Outside-RED
ASAv6(config)# access-list WHITE extended permit icmp host 10.1.104.1 host 10.1.104.254
ASAv6(config)# access-group WHITE in interface Outside-WHITE  
ASAv6(config)# access-list RED extended permit udp host 10.1.9.103 eq 848 host 10.1.103.254 eq 848
ASAv6(config)# access-list RED extended permit udp host 10.1.9.103 host 10.1.103.254 eq 4500
ASAv6(config)# access-list RED extended permit udp host 10.1.9.103 host 10.1.103.254 eq isakmp
ASAv6(config)# access-list RED extended permit icmp host 10.1.9.103 host 10.1.103.254
ASAv6(config)# access-list WHITE extended permit udp host 10.1.9.104 eq 848 host 10.1.104.254 eq 848
ASAv6(config)# access-list WHITE extended permit udp host 10.1.9.104 host 10.1.104.254 eq 4500
ASAv6(config)# access-list WHITE extended permit udp host 10.1.9.104 host 10.1.104.254 eq isakmp
ASAv6(config)# access-list WHITE extended permit icmp host 10.1.9.104 host 10.1.104.254

ASA9(config)# access-list Inside->Outside extended permit udp host 10.1.26.1 host 10.1.103.254 eq 848 
ASA9(config)# access-list Inside->Outside extended permit udp host 10.1.26.1 host 10.1.103.254 eq 4500 
ASA9(config)# access-list Inside->Outside extended permit udp host 10.1.26.1 host 10.1.103.254 eq isakmp 
ASA9(config)# access-list Inside->Outside extended permit icmp host 10.1.26.1 host 10.1.103.254 
ASA9(config)# access-list Inside->Outside extended permit udp host 10.1.11.1 host 10.1.104.254 eq 848 
ASA9(config)# access-list Inside->Outside extended permit udp host 10.1.11.1 host 10.1.104.254 eq 4500 
ASA9(config)# access-list Inside->Outside extended permit udp host 10.1.11.1 host 10.1.104.254 eq isakmp
ASA9(config)# access-list Inside->Outside extended permit icmp host 10.1.11.1 host 10.1.104.254  
ASA9(config)# 

GETVPN-S1#sh run | s crypto
crypto keyring RED-Keyring  
  pre-shared-key address 10.1.2.254 key CCIE
crypto ikev2 profile GET-IKEv2-Profile
 match address local interface GigabitEthernet0/1
 match identity remote address 10.1.2.254 255.255.255.255 
 authentication remote pre-share
 authentication local pre-share
crypto ipsec transform-set GET-TS esp-3des esp-sha-hmac 
 mode tunnel
crypto ipsec profile GET-Profile
crypto gdoi group G1-RED
 identity number 103
 server local
  rekey algorithm aes 128
  rekey retransmit 10 number 3
  rekey authentication mypubkey rsa GET-RSA-Key
  rekey transport unicast
  registration interface GigabitEthernet0/1
  sa ipsec 103
   profile GET-Profile
   match address ipv4 103
   replay counter window-size 64
   no tag
  address ipv4 10.1.26.1
GETVPN-S1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
GETVPN-S1(config)#crypto keyring RED-Keyring
GETVPN-S1(conf-keyring)#no pre-shared-key address 10.1.2.254 key CCIE
GETVPN-S1(conf-keyring)#pre-shared-key address 10.1.103.254 key CCIE
GETVPN-S1(conf-keyring)#crypto ikev2 profile GET-IKEv2-Profile
GETVPN-S1(config-ikev2-profile)#no match identity remote address 10.1.2.254 255.255.255.255      
GETVPN-S1(config-ikev2-profile)#match identity remote address 10.1.103.254 255.255.255.255       
GETVPN-S1(config-ikev2-profile)#

GETVPN-S2#sh run | s crypto
crypto keyring WHITE-Keyring  
  pre-shared-key address 10.1.2.254 key CCIE
crypto ikev2 profile GET-IKEv2-Profile
 match address local interface GigabitEthernet0/0
 match identity remote address 10.1.2.254 255.255.255.255 
 authentication remote pre-share
 authentication local pre-share
crypto ipsec transform-set GET-TS esp-3des esp-sha-hmac 
 mode tunnel
crypto ipsec profile GET-Profile
crypto gdoi group G2-WHITE
 identity number 104
 server local
  rekey algorithm aes 128
  rekey retransmit 10 number 3
  rekey authentication mypubkey rsa GET-RSA-Key
  rekey transport unicast
  registration interface GigabitEthernet0/0
  sa ipsec 104
   profile GET-Profile
   match address ipv4 104
   replay counter window-size 64
   no tag
  address ipv4 10.1.11.1
GETVPN-S2#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
GETVPN-S2(config)#line con 0
GETVPN-S2(config-line)#width 255
GETVPN-S2(config-line)#exit
GETVPN-S2(config)#crypto keyring WHITE-Keyring
GETVPN-S2(conf-keyring)#no pre-shared-key address 10.1.2.254 key CCIE
GETVPN-S2(conf-keyring)#pre-shared-key address 10.1.104.254 key CCIE
GETVPN-S2(conf-keyring)#crypto ikev2 profile GET-IKEv2-Profile
GETVPN-S2(config-ikev2-profile)#no match identity remote address 10.1.2.254 255.255.255.255
GETVPN-S2(config-ikev2-profile)#match identity remote address 10.1.104.254 255.255.255.255
GETVPN-S2(config-ikev2-profile)#

GETVPN-Client(conf-isa-prof)#do sh run | s crypto
crypto keyring RED-G1 vrf FVRF 
  pre-shared-key address 10.1.9.103 key CCIE
crypto keyring RED-G1-RED vrf RED 
  pre-shared-key address 10.1.9.103 key CCIE
crypto keyring WHITE-G2-WHITE vrf WHITE 
  pre-shared-key address 10.1.9.104 key CCIE
crypto isakmp policy 10
 encr 3des
 authentication pre-share
 group 2
 lifetime 3600
crypto isakmp policy 20
 encr aes 256
 authentication pre-share
 group 2
crypto isakmp key CCIE address 10.1.9.103     
crypto isakmp key CCIE address 10.1.9.104     
crypto isakmp invalid-spi-recovery
crypto isakmp keepalive 10
crypto isakmp profile GET-ISAK-Profile-RED
   vrf RED
   keyring RED-G1-RED
   match identity address 10.1.9.103 255.255.255.255 
crypto isakmp profile GET-ISAK-Profile-WHITE
   vrf WHITE
   keyring WHITE-G2-WHITE
   match identity address 10.1.9.104 255.255.255.255 
crypto ipsec transform-set GET-TS esp-3des esp-sha-hmac 
 mode tunnel
crypto ipsec profile GET-IPS-Profile-RED
 set group G1-RED
 set transform-set GET-TS 
 set isakmp-profile GET-ISAK-Profile-RED
crypto ipsec profile GET-IPS-Profile-WHITE
 set group G2-WHITE
 set transform-set GET-TS 
 set isakmp-profile GET-ISAK-Profile-WHITE
crypto gdoi group G1-RED
 identity number 103
 server address ipv4 10.1.9.103
crypto gdoi group G2-WHITE
 identity number 104
 server address ipv4 10.1.9.104
crypto map G1-RED local-address GigabitEthernet0/0.103
crypto map G1-RED isakmp-profile GET-ISAK-Profile-RED
crypto map G1-RED 103 gdoi 
 set group G1-RED
crypto map G2-WHITE local-address GigabitEthernet0/0.104
crypto map G2-WHITE isakmp-profile GET-ISAK-Profile-WHITE
crypto map G2-WHITE 104 gdoi 
 set group G2-WHITE
 crypto map G1-RED
 crypto map G2-WHITE
GETVPN-Client(conf-isa-prof)#

GETVPN-Client(conf-isa-prof)#do sh cry gdo | i Group Name|status
    Group Name               : G1-RED
       Registration status   : Registered
       SA Track (OID/status) : disabled
    Group Name               : G2-WHITE
       Registration status   : Registered
       SA Track (OID/status) : disabled
GETVPN-Client(conf-isa-prof)#

VRF-aware GETVPN is working but I am actually pretty annoyed. I have spent way too long battling against this, probably five days (not constant, I still went to work, cooked food, had family time, took the boys to a party and did family stuff), but it’s five days worth of study that I could have spent studying something else.

Trying to put a positive spin on things, this is actually a good lesson for the lab. If something is taking too long then move on. Either you’ll figure it out later, something else will give you a clue, you’ll fix a prerequisite, or you’ll leave that exercise and get points on something else that you’d otherwise have missed because of taking too long to fix one thing instead of moving on.

So, the things I still don’t get are why it would not work, and whether I am trying to fix something that will never work, or whether it’s an issue with vIOS and this is the reason it’s not working, Maybe the topology and constraints were wrong from the start. Maybe someone else can solve this one, but I need to be strict and move on. With only 72 days to go, I need to spend the time wisely, and not get like a dog with a bone when it comes to an issue.

In the next post, we’ll be extending the routes coming into the GETVPN servers, so that we have access to the WWW and ACS servers, and setting up IKEv1 between NYC and the IKEv1 router.