EVPN Multihoming

The EVPN Multihoming feature utilizes the functionality defined in RFC 7432 (BGP MPLS-based Ethernet VPN) to achieve multihoming between Provider Edge (PE) and Customer Edge (CE) devices.

Information about EVPN Multihoming

BGP MPLS-based EVPN

Ethernet VPN (EVPN) is an evolution of the L2VPN VPLS solution that addresses the following requirements:

  • PE node redundancy with load-balancing based on Layer 2, Layer 3, or Layer 4 flows from CE to PE.

  • Flow-based multi-pathing of traffic from local PE to remote PEs across core and vice-versa.

  • Geographically redundant PE nodes with optimum unicast forwarding.

  • Flexible redundancy grouping, where a PE can be a member of multiple redundancy groups each containing a different set of CEs.

There are three fundamental building blocks for EVPN technology - EVPN Instance (EVI), Ethernet Segment (ES), and EVPN BGP routes and extended communities. For more information, refer to EVPN Building Blocks section.

In BGP MPLS-based EVPN, an EVI is configured for every PE device for each customer associated with the PE device. An example of a customer is the CE device that is attached to the PE device. Each EVI has a unique Route Distinguisher (RD) and one or more Route Targets (RT). The CE device can be a host, a switch or a router.

For any port involved in a multihoming CE configuration, an ESI must be defined and associated with it. In Cisco IOS XE Fuji 16.9.x software release, only type 3 ESI is supported as defined in section 5 of RFC7432. Type 3 ESI consists of PE System MAC address and local discriminator.

In EVPN multihoming, a customer site is connected to multiple PE devices and can have an Ethernet Segment with ESI value greater than one.

RFC7432 defines four new routes and four new extended communities to enable EPVN support. From Cisco IOS XE Fuji 16.9.x Software Release, all four route types are supported.

EVPN Multihoming Topology

The following figure shows a typical deployment involving two CE devices, where each CE device connects to multiple PE devices, to mitigate any single-point failures:

Figure 1. EVPN Multihoming Topology

  • CE1 uses a port channel consisting of links, L11 and L12, to connect to PE1 and PE2, respectively.

  • CE2 uses a port channel consisting of links L21 and L22, to connect to PE1 and PE2, respectively.

  • On PE1 and PE2, ESI-1 is used to identify Ethernet Flow Points (EFPs) corresponding to links from P1, and ESI-2 is used to identify EFPs corresponding to links from P2.


Note

Since CE1 and CE2 are port channels, each port channel can support flow-based load balancing for traffic egress towards PE1 and PE2.


Note

For each PE, ESI is a property associated to a port.

All-Active Multihoming

EVPN Multihoming access gateway enables redundant network connectivity by allowing a CE device to connect to more than one PE device. Disruptions to the network connectivity are prevented by allowing a CE device to be connected to a PE device or several PE devices through multihoming. Ethernet segment is the group of ethernet links through which a CE device is connected to more than one PE devices. The All-Active Link Aggregation Group (LAG) bundle operates as an ethernet segment.

In all-active multihoming scenario, when multihop is configured to the same destination, the access side device load balances traffic on the access side and the PEs load balance traffic to remote PEs on the core side.

Route Types

RFC7432 introduces four new BGP route types (1–4) and communities.

  • In EVPN multihoming scenarios, route types 1 and 4 are advertised to discover other PEs and their redundancy modes.

  • Route type 2 is used for MAC learning. EVPN introduces the concept of BGP MAC routing and uses Multiprotocol-BGP (mBGP) for learning the MAC addresses between the PEs.

Route Type 1 - Ethernet Auto-Discovery Route

The route type value for EAD routes is 0x01. This route is originated when a PE is connected to a CE for which multihoming is configured. Two types of EAD routes are supported in this feature: Per-EVI (EVPN Instance) EAD routes and Per-ES (Ethernet Segment) EAD Routes.

Route Type-1 advertisement is used for achieving split-horizon, fast convergence, and aliasing. EAD-ES and EAD-EVI are used to achieve these functionalities. Fast convergence allows PEs to change the next-hop adjacencies for all MACs associated with an ES and aliasing allows balancing traffic across multiple egress points. Route Type 1 is advertised only if ES is set to a non-zero value, that is, type 1 routes are originated only for sites where multihoming is configured. These routes are sent per-ES and carry the combined set of route targets of all of the EVIs that belong to that ES.

The per-ES EAD route includes the ESI label extended community which indicates if it is an all-active or a single-active configuration. The ESI label extended community also carries the ESI label that is used for split horizon configuration. The per-ES EAD route is also used for fast convergence when failure occurs at the ES on the access side.

The per-EVI EAD and per-ES EAD routes are used for aliasing, and fast convergence and providing the split horizon label, respectively. In a multihoming group, each PE associated with a CE may learn only a subset of MAC addresses on traffic ingress from CE. The MAC addresses learned by the PEs may not overlap with each other. Aliasing is the ability of a PE to signal that it has reachability to an EVPN instance on a given ES, even when the PE has not learned MAC addresses from that EVI or ES. In an all-active multi-homing configuration, a remote PE that receives a MAC advertisement route considers the advertised MAC address to be reachable through all PEs that have advertised reachability to EVI or ES of the MAC address.

Table 1. Per-EVI Ethernet Auto-Discovery Route

Field

Value

Length (Octets)

Route Type

0x01

1

Length

25

1

EVI RD

Type 1 (IPv4Addr) RD unique across all EVIs on the PE.

8

ESI

Ethernet Segment Identifier

10

Ethernet Tag

0 or valid Ethernet Tag

4

Label

Valid MPLS label allocated per [EVI, ESI, EtherTag] tuple

3

EVI RT

Type 0 (2byteAS) route target

8

The route target is specific to the EVI. It can be automatically derived from EVI and AS numbers, or explicitly configured. As in L2VPN and L3VPN, multiple route targets can be configured for an EVPN instance (EVI) and in this case multiple route target extended communities are attached to the per-EVI EAD route.

Following is the header format of the Per-ES EAD route:

Table 2. Per-ES Ethernet Auto-Discovery Route

Field

Value

Length (Octets)

Route Type

0x01

1

Length

25

1

ES RD

Type 1 (IPv4Addr) RD.

8

ESI

Ethernet Segment Identifier

10

Ethernet Tag

0xFFFFFFFF

4

Label

0

3

ESI Label

0x0601:{1 byte single-active bit}:0x0000:{Split-Horizon label}

8

EVI-1 RT

Type 0 (2byteAS) route target for EVI-1

8

EVI-2 RT

Type 0 (2byteAS) route target for EVI-2

8

...

...

...

EVI-n RT

Type 0 (2byteAS) route target for EVI-n

8

One per-ES-EAD route is sourced per Ethernet Segment. Per-ES-EAD route carries the route targets of all EVIs the Ethernet Segment belongs to. If the number of EVI route targets is too large to be carried in one per-ES-EAD route, then multiple routes are advertised. Each route is assigned a different Ethernet Segment Route Distinguisher (ES-RD). The per-EVI-EAD route is used along with the per-ES-EAD route for aliasing and backup path. The per-ES-EAD is also used for fast convergence in case of failure in the Ethernet Segment.

Route Type 2 - MAC and IP Advertisement Route

Type 2 routes are used to advertise MAC addresses and their associated IP addresses. When a PE router learns the MAC address of a CE device that is connected to it locally, or a MAC address of a device behind the CE device, a MAC and IP advertisement route is created.

Following is the header format for the MAC and IP Advertisement Route packet:

Table 3. Header format for the MAC and IP Advertisement Route packet

Field

Value

Length (Octets)

Route Type

0x02

1

Length

Variable

1

EVI RD

Type 1 (IPv4 address) RD unique across all EVIs on the PE.

8

ESI

Ethernet Segment Identifier

10

Ethernet Tag

0 or valid Ethernet Tag

4

MAC Addr Len

48

1

MAC Address

Valid MAC address

6

IP Addr Length

IP address length in bits: 0 or 32 or 128

1

IP Address

Optional IP address

0 or 4 or 16

Label1

Valid downstream assigned label to perform forwarding to CE based on the destination MAC address

3

Label2

Specifies a second label

0-3

EVI RT

Type 0 (2byteAS) route target

8

MAC Mobility

0x0600:{1 byte Sticky bit}:0x00:{4 byte sequence number}

8

  • MAC Address field is populated with the CE address.

  • IP address field is optional with IP Address length set to 0 bits.


    Note

    IP learning is not supported on Cisco ASR 1000 Series Aggregation Services Routers.

  • In the Label field, Per-BD or Per-CE labels can be assigned.

    • Per-BD is used when PE advertises a single label for all MAC addresses learned in a given bridge domain.

    • Per-CE label assigns a separate label to each access port in the bridge domain.

Route Type 3 - Inclusive Multicast Ethernet Tag Route

Type 3 routes are used for transporting Broadcast, Unknown Unicast and Multicast (BUM) traffic to other PE devices across a given EVPN network instance.

The following is the header format for Type 3 routes:

Table 4. Route Type 3 - Inclusive Multicast Ethernet Tag Route Header

Field

Value

Length (Octets)

Route Type

0x03

1

Length

26 or 38

1

EVI RD

Type 1 (IPv4Addr) RD unique across all EVIs on the PE.

8

Ethernet Tag

0 or valid Ethernet Tag

4

IP Addr Length

IP Address Length - 32 bits or 128 bits

1

IP Address

IP Address common for all EVIs (for example, loopback address)

4 or 16

PMSI Tunnel Attr

{1 byte flags = 0}:{1 byte Tunnel Type}:{3 byte label}:{variable length Tunnel Identifier}

Variable

EVI RT

Type 0 (2byteAS) route target

8

The PE devices advertises an Inclusive Multicast Ethernet Tag (IMET) Route for every EVI-Ethernet Tag sequence. The Ethernet Tag is set to 0 for VLAN-based and VLAN-bundling service interfaces. The Ethernet Tag is set to a valid VLAN ID for VLAN-aware bundling service interface.

Type 3 route also carries a Provider Multicast Service Interface (PMSI) Tunnel attribute as specified in RFC 6514 (BGP Encodings and Procedures for MVPNs).

For Ingress Replication, the IMET route is used to advertise the label (in the PMSI Tunnel Attribute) that the other PEs can use to send BUM traffic to the originating PE device.

Route Type 4 - Ethernet Segment Route

Ethernet segment routes are needed in multihomed scenarios to enable the discovery of PE devices connected to the same Ethernet segment. Ethernet segment routes are also used electing the designated forwarder (DF) for BUM traffic to the CE, on a particular Ethernet segment. Once an ESI has been assigned for the Ethernet segment for a multihomed CE, the ESI is advertised to the ES-Import extended community by the PE as BGP route type 4. The PEs where the import community matches with the ESI import community, imports ES route to auto-discover each other.

The route type value for Ethernet Segment Route is 0x04. It is originated only by PEs connected to multihomed CEs. It is imported only by PEs connected to the same Ethernet Segment. This route has the following format:

Table 5. Route Type 4 - Ethernet Segment Route

Field

Value

Length (Octets)

Route Type

0x04

1

Length

23

1

ES RD

Type 1 (IPv4Addr) RD unique across all Ethernet Segments on the PE.

8

ESI

Ethernet Segment Identifier

10

IP Addr Length

IP Address Length - 32 bits or 128 bits

1

IP Address

IP Address of the originating PE

4 or 16

ES-Import RT

0x0602:{high order 6-octet portion of the 9-octect ESI value}

8

Core Isolation

In scenarios where a PE loses connectivity to the core network, either the core-facing interface on the PE goes to DOWN state, or an upstream event results in BGP peering loss. All the BGP routes types 1, 2, 3, and 4 are withdrawn after the timers expire. All other PEs in the same Ethernet segment are alerted and a new DF is elected by the remaining PEs. However, the access side switch or node is not aware of this event since the multihomed access interface on the PE is still in the UP state. This results in traffic being blackholed, since the access side device continues to forward traffic to the PE.

To remedy this scenario, the core isolation solution is implemented in Cisco IOS-XE software. In the event of BGP peering loss on the PE or the core facing interface goes to DOWN state, the multihomed access interfaces on the PE are placed in err-disabled state. There are no configuration changes made on these access interfaces. Since the access port is in DOWN state, the link partner on the access switch is also in DOWN state and the corresponding port-channel, on the switch, detects that this member interface has gone DOWN. Therefore, the switch stops forwarding traffic on this interface and load balances the traffic amongst the remaining member interfaces. Once the BGP peering is restored the error-disabled states are removed from the multi-homed access interfaces.

Prerequisites for EVPN Multihoming

  • EVI and Bridge domains must be in established state with associated MPLS labels.

Restrictions for EVPN Multihoming

  • The number of bridge domains that are supported are 16000.

  • The number of EFPs or service instances that are supported per physical interface are 8000.

  • Stateful Switchover is not supported.

  • IP learning is not supported on Cisco ASR 1000 Series Aggregation Services Routers.

  • Only all-active redundancy mode (2 or 2+ PEs in the same redundancy group sharing the same ESI and all forwarding traffic) is supported.

  • Single-active mode is not supported.

  • Only access-side flow-based load balancing with multihoming LAG ON mode is supported. Any ether-channel signaling (LACP or PAgP) is not supported.

  • MAC mobility and duplication detection is not supported.

  • Per-EVI and per-MAC labeling is not supported. Only per-BD and per-CE labeling is supported.

  • Only type 3 ESI is supported as defined in section 5 of RFC7432. Type 3 ESI consists of PE System MAC address and local discriminator.

  • Port-channel signaling is not supported.

  • The port-channel should be configured in ON mode only.

How to Configure EVPN Multioming

Configuring EVPN Multihoming

Figure 2. All-Active Multihoming Topology

The above figure represents L2VPN All-Active Multihoming network. Use the following steps to configure Multihoming:

Configuring L2VPN EVPN Globally and EVI on IOS-XE Router


enable
configure terminal
 l2vpn evpn
  replication-type ingress		-> Enables ingress replication label
  router-id Loopback0			-> Configures L2VPN EVPN Router-ID
!
 l2vpn evpn instance 10 vlan-based	-> Configures Vlan-based EVI 10
!
 l2vpn evpn instance 20 vlan-bundle	-> Configures Vlan-bundled EVI 20
!
 l2vpn evpn instance 30 vlan-aware	-> Configures Vlan-aware EVI 30

Configuring access interface on PE for EVPN Multi-homing all-active

enable
    configure terminal
        interface Port-channel1
           no ip address
           no negotiation auto
           evpn ethernet-segment 1		                    -> Configures Ethernet Segment ID
              identifier type 3  system-mac abcd.abcd.abc1	   -> Configures system MAC
              redundancy all-active			       -> Configures redundancy mode (all-active/single-active)
             service instance 10 ethernet 		-> Enables service instance 10 under the physical interface
              encapsulation dot1q 10
              !
             service instance 20 ethernet 		-> Enables service instance 20 under the physical interface
               encapsulation dot1q 20-21
               !
             service instance 30 ethernet 		-> Enables service instance 30 under the physical interface
                encapsulation dot1q 30
                !
                !
          interface GigabitEthernet3
             no ip address
             negotiation auto
             isis network point-to-point 
             isis three-way-handshake cisco
             channel-group 1

Configuring Bridge-domain on IOS-XE Router


enable
configure terminal
 bridge-domain 10 
  mac aging-time 30				-> Configures aging time for all MACs learnt under bridge-domain	
  member Port-channel1 service-instance 10 	 Links SI 10 on Port-channel1 with Bridge-domain 10
  member evpn-instance 10			-> Links EVI 10 with Bridge-domain 10
!
bridge-domain 20 
 mac aging-time 30
 member Port-channel1 service-instance 20 	-> Links SI 20 on Port-channel1 with Bridge-domain 20
 member evpn-instance 20 			-> Links EVI 20 with Bridge-domain 20
!
bridge-domain 30 
 mac aging-time 30
 member Port-channel1 service-instance 30 	-> Links SI 30 on Port-channel1 with Bridge-domain 30
 member evpn-instance 30 ethernet-tag 30 	-> Links EVI 30 with Bridge-domain 30

Configuring BGP on Provider Edge 

router bgp 100
 bgp router-id 192.168.1.1
 bgp log-neighbor-changes
 bgp graceful-restart
 neighbor 192.168.1.4 remote-as 100
 neighbor 192.168.1.4 update-source Loopback0
 !
 address-family ipv4
  neighbor 192.168.1.4 activate
 exit-address-family
 !
 address-family l2vpn evpn 			-> Enables L2vpn evpn address family
  neighbor 192.168.1.4 activate
  neighbor 192.168.1.4 send-community both
  neighbor 192.168.1.4 soft-reconfiguration inbound
 exit-address-family
30

Configuring BGP on Core Router or Route Reflector

router bgp 100
 bgp router-id 192.168.1.4
 bgp log-neighbor-changes
 bgp graceful-restart
 neighbor 192.168.1.1 remote-as 100
 neighbor 192.168.1.1 update-source Loopback0
 neighbor 192.168.1.2 remote-as 100
 neighbor 192.168.1.2 update-source Loopback0
 neighbor 192.168.1.3 remote-as 100
 neighbor 192.168.1.3 update-source Loopback0
 neighbor 192.168.1.5 remote-as 100
 neighbor 192.168.1.5 update-source Loopback0
 neighbor 192.168.1.6 remote-as 100
 neighbor 192.168.1.6 update-source Loopback0

 !
 address-family ipv4
  neighbor 192.168.1.1 activate
  neighbor 192.168.1.1 route-reflector-client
  neighbor 192.168.1.2 activate
  neighbor 192.168.1.2 route-reflector-client
  neighbor 192.168.1.3 activate
  neighbor 192.168.1.3 route-reflector-client
  neighbor 192.168.1.5 activate
  neighbor 192.168.1.5 route-reflector-client
  neighbor 192.168.1.6 activate
  neighbor 192.168.1.6 route-reflector-client  
 exit-address-family
 !
 address-family l2vpn evpn  			-> Enables L2vpn evpn address family
  neighbor 192.168.1.1 activate
  neighbor 192.168.1.1 send-community both
  neighbor 192.168.1.1 route-reflector-client
  neighbor 192.168.1.1 soft-reconfiguration inbound
  neighbor 192.168.1.2 activate
  neighbor 192.168.1.2 send-community both
  neighbor 192.168.1.2 route-reflector-client
  neighbor 192.168.1.2 soft-reconfiguration inbound
  neighbor 192.168.1.3 activate
  neighbor 192.168.1.3 send-community both
  neighbor 192.168.1.3 route-reflector-client
  neighbor 192.168.1.3 soft-reconfiguration inbound
  neighbor 192.168.1.5 activate
  neighbor 192.168.1.5 send-community both
  neighbor 192.168.1.5 route-reflector-client
  neighbor 192.168.1.5 soft-reconfiguration inbound
  neighbor 192.168.1.6 activate
  neighbor 192.168.1.6 send-community both
  neighbor 192.168.1.6 route-reflector-client
  neighbor 192.168.1.6 soft-reconfiguration inbound
exit-address-family

Configuration Examples for EVPN Mulltihoming

Verifying EVPN Multihoming

Use the following commands to verify that the bridge domains are in established state and that bridge domain has learnt the local MAC address: 

PE1# show bridge-domain 10 mac dynamic address 
          Port               MAC Address   
          Po1 ServInst 10    000c.2911.6d2a	-> MAC learnt on port-channel 1 for service instance 10

PE1#show bridge-domain 10
Bridge-domain 10 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 30 second(s) 			-> MAC aging timer for bridge-domain
    Port-channel1 service instance 10
    EVPN Instance 10
   AED MAC address    Policy  Tag       Age  Pseudoport
   -   000C.29F8.5078 forward static_r  0    OCE_PTR:0xe8e5dda0
   -   000C.2911.6D2A forward dynamic_c 28   Port-channel1.EFP10 

PE1#show bridge-domain 10
Bridge-domain 10 (2 ports in all)
State: UP                    Mac learning: Enabled
Aging-Timer: 30 second(s)
    Port-channel1 service instance 10
    EVPN Instance 10
   AED MAC address    Policy  Tag       Age  Pseudoport
   -   000C.29F8.5078 forward static_r  0    OCE_PTR:0xe8e5dda0
   -   000C.2911.6D2A forward static_a  0    Port-channel1.EFP10

Note

In the above output, MAC addresses with forward dynamic_c tags are locally learned addresses and MAC addresses with forward static_r tags are remote addresses learned through EVPN.

Use the following command to verify the number and type of EVIs configured on the PE, number of bridge-domains configured, and number of MACs learnt locally and remotely:
PE1#show l2vpn evpn summary 
L2VPN EVPN
  EVPN Instances (excluding point-to-point): 3
    VLAN Aware:   1
    VLAN Based:   1
    VLAN Bundle:  1
  Bridge Domains: 3
  BGP: ASN 100, address-family l2vpn evpn configured
  Router ID: 192.168.1.1
  Label Allocation Mode: Per-BD
  Replication Type: Ingress
  Forwarding State: UP
  MAC Duplication: seconds 180 limit 5
  MAC Addresses: 6
    Local:     3
    Remote:    3
    Duplicate: 0
  IP Duplication: seconds 180 limit 5
  IP Addresses: 0
    Local:     0
    Remote:    0
    Duplicate: 0
  Maximum number of Route Targets per EAD-ES route: 200

Note

In the above output, the remote MAC addresses' next hops are the addresses of the provider edge devices that these MAC addresses are learned from.

Use the following command to verify ethernet-segments attached to the PE:

PE1#show l2vpn evpn ethernet-segment detail 
EVPN Ethernet Segment ID: 03AB.CDAB.CDAB.C100.0001
  Interface:              Po1
  Redundancy mode:        all-active
  DF election wait time:  3 seconds
  Split Horizon label:    16
  State:                  Ready
  Ordinal:                0
  RD:                     192.168.1.1:1
    Export-RTs:           100:10 100:20 100:30 
  Forwarder List:         192.168.1.1 192.168.1.2 192.168.1.3

Use the following command to verify EVPN manager details regarding an EVI:

PE1#show l2vpn evpn evi detail 
EVPN instance:    10 (VLAN Based)		í VLAN based EVI
  RD:             192.168.1.1:10 (auto)			-> RD derived from Loopback0 EVPN Router-ID:EVI number
  Import-RTs:     100:10 
  Export-RTs:     100:10 
  Per-EVI Label:  none
  State:          Established			-> EVI state
  Encapsulation:  mpls
  Bridge Domain:  10
    Ethernet-Tag: 0
    BUM Label:    18
    Per-BD Label: 19
    State:        Established			-> BD state
    Pseudoports:				-> Access interface and DF election status for EVI 10
      Port-channel1 service instance 10 (DF state: PE-to-CE BUM blocked)

EVPN instance:    20 (VLAN Bundle)		-> VLAN bundled EVI
  RD:             192.168.1.1:20 (auto)
  Import-RTs:     100:20 
  Export-RTs:     100:20 
  Per-EVI Label:  none
  State:          Established
  Encapsulation:  mpls
  Bridge Domain:  20
    Ethernet-Tag: 0
    BUM Label:    20
    Per-BD Label: 21
    State:        Established
    Pseudoports:
      Port-channel1 service instance 20 (DF state: PE-to-CE BUM blocked)

EVPN instance:    30 (VLAN Aware)		-> VLAN aware EVI
  RD:             192.168.1.1:30 (auto)
  Import-RTs:     100:30 
  Export-RTs:     100:30 
  Per-EVI Label:  none
  State:          Established
  Encapsulation:  mpls
  Bridge Domain:  30
    Ethernet-Tag: 30
    BUM Label:    22
    Per-BD Label: 23
    State:        Established
    Pseudoports:				-> Elected DF for EVI 30
      Port-channel1 service instance 30 (DF state: forwarding)

Note

Designated Forwarder (DF) is responsible for forwarding Broadcast, Unicast and Multicast (BUM) traffic on an ethernet segment. Route-type 4 is used to carry this information.

Use the following command to verify EVPN manager details for bridge-domain 10:

 PE1#show l2vpn evpn mac bridge-domain 10 detail 
MAC Address:               000c.2911.6d2a
EVPN Instance:             10
Bridge Domain:             10
Ethernet Segment:          03AB.CDAB.CDAB.C100.0001	-> ESI number assigned to the MAC learnt on this EFP
Ethernet Tag ID:           0
Next Hop(s):               Port-channel1 service instance 10	-> MAC learnt locally on port-channel 1
                                   3.3.3.3
Local Address:             0.0.0.0
Label:                     17
Sequence Number:           0
MAC only present:          Yes
MAC Duplication Detection: Timer not running	

MAC Address:               000c.29f8.5078
EVPN Instance:             10
Bridge Domain:             10
Ethernet Segment:          03AB.CDAB.CDAB.C200.0002
Ethernet Tag ID:           0
Next Hop(s):               6.6.6.6
Local Address:             1.1.1.1
Label:                     19
Sequence Number:           0
MAC only present:          Yes
MAC Duplication Detection: Timer not running

Use the following command to verify EVPN manager details EVI 10:

PE1#show l2vpn evpn mac evi 10 detail 
MAC Address:               000c.2911.6d2a
EVPN Instance:             10
Bridge Domain:             10
Ethernet Segment:          03AB.CDAB.CDAB.C100.0001
Ethernet Tag ID:           0
Next Hop(s):               Port-channel1 service instance 10
                                   192.168.1.2
Local Address:             0.0.0.0
Label:                     19
Sequence Number:           0
MAC only present:          Yes
MAC Duplication Detection: Timer not running

MAC Address:               000c.29f8.5078
EVPN Instance:             10
Bridge Domain:             10
Ethernet Segment:          03AB.CDAB.CDAB.C200.0002
Ethernet Tag ID:           0
Next Hop(s):               192.168.1.5
Local Address:             192.168.1.1
Label:                     23
Sequence Number:           0
MAC only present:          Yes
MAC Duplication Detection: Timer not running

Use the following command to verify that the information on BGP routes is sent to Layer 2 RIB :

 PE1#show l2rib producers  
    Producer(ID)     Client ID  Object Type  Admin Dist  Purge Time(sec)    State
    -------------    ---------  -----------  ----------  ---------------  -----------
   L2VPN(  9)          1       Topology          5              120        Converged
     BGP(  5)          0          MAC           20              600        Converged
   L2VPN(  9)          1          MAC           5             1800         Converged
     BGP(  5)          0          EAD           20              600        Converged
   L2VPN(  9)          1          EAD           6              120         Converged
     BGP(  5)          0     IMET_ROUTE         20              600        Converged
   L2VPN(  9)          1     IMET_ROUTE          6              120        Converged
     BGP(  5)          0       MAC-IP           20              600        Converged
   L2VPN(  9)          1       MAC-IP            6             1800        Converged
     BGP(  5)          0     ES_ROUTE           20              600        Converged
   L2VPN(  9)          1       ES_ROUTE          6             1800        Converged

Use the following command to verify Route Type 3 IMET tunnels created for each EVI:

PE1#show l2route evpn imet         
  EVI       ETAG  Prod  Router IP Addr  Type   Label       Tunnel ID
----- ---------- ----- --------------- ----- ------- ---------------
   10          0   BGP     192.168.1.2     6      22     192.168.1.2
   10          0   BGP     192.168.1.3     6      22     192.168.1.3
   10          0   BGP     192.168.1.5     6      22     192.168.1.5
   10          0   BGP     192.168.1.6     6      22     192.168.1.6
   10          0 L2VPN     192.168.1.1     6      18     192.168.1.1
   20          0   BGP     192.168.1.2     6      20     192.168.1.2
   20          0   BGP     192.168.1.3     6      20     192.168.1.3
   20          0   BGP     192.168.1.5     6      20     192.168.1.5
   20          0   BGP     192.168.1.6     6      20     192.168.1.6
   20          0 L2VPN     192.168.1.1     6      20     192.168.1.1
   30         30   BGP     192.168.1.2     6      18     192.168.1.2
   30         30   BGP     192.168.1.3     6      18     192.168.1.3
   30         30   BGP     192.168.1.5     6      18     192.168.1.5
   30         30   BGP     192.168.1.6     6      18     192.168.1.6
   30         30 L2VPN     192.168.1.1     6      22     192.168.1.1

Use the following command to verify EAD-EVI route-type 1 for EVI 10 for BGP:

PE1# show ip bgp l2vpn evpn evi 10 route-type 1
 BGP routing table entry for [1][192.168.1.1:10][03ABCDABCDABC1000001][0]/23, version 109
Paths: (3 available, best #2, table evi_10)
  Flag: 0x8000
  Advertised to update-groups:
     1         
  Refresh Epoch 4
  Local, (received & used), imported path from [1][192.168.1.2:10][03ABCDABCDABC1000001][0]/23 (global)
    192.168.1.2 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, multipath
      Rcvd Label: 19, Local Label: None
      Extended Community: RT:100:10
      Originator: 192.168.1.2, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  Local
    :: (via default) from 0.0.0.0 (192.168.1.1)
      Origin incomplete, localpref 100, weight 32768, valid, sourced, local, multipath, best
      Rcvd Label: None, Local Label: 25
      Extended Community: RT:100:10
      rx pathid: 0, tx pathid: 0x0
  Refresh Epoch 3
  Local, (received & used), imported path from [1][192.168.1.3:10][03ABCDABCDABC1000001][0]/23 (global)
    192.168.1.3 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, multipath(oldest)
      Rcvd Label: 19, Local Label: None
      Extended Community: RT:100:10
      Originator: 192.168.1.3, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0
BGP routing table entry for [1][192.168.1.1:10][03ABCDABCDABC2000002][0]/23, version 61
Paths: (2 available, best #1, table evi_10)
  Not advertised to any peer
  Refresh Epoch 2
  Local, (received & used), imported path from [1][192.168.1.5:10][03ABCDABCDABC2000002][0]/23 (global)
    192.168.1.5 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, multipath, best
      Rcvd Label: 19, Local Label: None
      Extended Community: RT:100:10
      Originator: 192.168.1.5, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0x0
  Refresh Epoch 2
  Local, (received & used), imported path from [1][192.168.1.6:10][03ABCDABCDABC2000002][0]/23 (global)
    192.168.1.6 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, multipath(oldest)
      Rcvd Label: 25, Local Label: None
      Extended Community: RT:100:10
      Originator: 192.168.1.6, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0

Use the following command to verify EAD-ES route-type 1 output for EVI 10 in BGP database:

PE1# show ip bgp l2vpn evpn route-type 1

BGP routing table entry for [1][192.168.1.2:10][03ABCDABCDABC1000001][0]/23, version 2
Paths: (1 available, best #1, table EVPN-BGP-Table)
  Not advertised to any peer
  Refresh Epoch 6
  Local, (received & used)
    192.168.1.2 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Rcvd Label: 23, Local Label: None
      Extended Community: RT:100:10
      Originator: 192.168.1.2, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0x0

Use the following command to verify information regarding the PEs with active ESI configuration:

PE1#sh ip bgp l2vpn evpn route-type 4
BGP routing table entry for [4][192.168.1.1:1][03ABCDABCDABC1000001][32][192.168.1.1]/23, version 99
Paths: (1 available, best #1, table EVPN-BGP-Table)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  Local
    :: (via default) from 0.0.0.0 (192.168.1.1)
      Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best
      Extended Community: EVPN ES-IMPORT:0xABCD:0xABCD:0xABC1
      rx pathid: 0, tx pathid: 0x0
BGP routing table entry for [4][192.168.1.2:1][03ABCDABCDABC1000001][32][192.168.1.2]/23, version 102
Paths: (1 available, best #1, table EVPN-BGP-Table)
  Not advertised to any peer
  Refresh Epoch 5
  Local, (received & used)
    192.168.1.2 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: EVPN ES-IMPORT:0xABCD:0xABCD:0xABC1
      Originator: 192.168.1.2, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0x0
BGP routing table entry for [4][192.168.1.3:1][03ABCDABCDABC1000001][32][192.168.1.3]/23, version 100
Paths: (1 available, best #1, table EVPN-BGP-Table)
  Not advertised to any peer
  Refresh Epoch 5
  Local, (received & used)
    192.168.1.3 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: EVPN ES-IMPORT:0xABCD:0xABCD:0xABC1
      Originator: 192.168.1.3, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0x0
BGP routing table entry for [4][192.168.1.5:2][03ABCDABCDABC2000002][32][192.168.1.5]/23, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  Refresh Epoch 5
  Local, (received-only)
    192.168.1.5 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal
      Extended Community: EVPN ES-IMPORT:0xABCD:0xABCD:0xABC2
      Originator: 192.168.1.5, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0
BGP routing table entry for [4][192.168.1.6:2][03ABCDABCDABC2000002][32][192.168.1.6]/23, version 0
Paths: (1 available, no best path)
  Not advertised to any peer
  Refresh Epoch 5
  Local, (received-only)
    192.168.1.6 (metric 30) (via default) from 192.168.1.4 (192.168.1.4)
      Origin incomplete, metric 0, localpref 100, valid, internal
      Extended Community: EVPN ES-IMPORT:0xABCD:0xABCD:0xABC2
      Originator: 192.168.1.6, Cluster list: 192.168.1.4
      rx pathid: 0, tx pathid: 0

Use the following ether channel state output on tbe CE device:

CE1# show port-channel summary 
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        b - BFD Session Wait
        S - Switched    R - Routed
        U - Up (port-channel)
        p - Up in delay-lacp mode (member)
        M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
1     Po1(SU)     Eth      NONE      Eth1/1(P)    Eth1/2(P)    Eth1/3(P)

Use the following Ether Channel state output on the PE device:

PE1#show etherchannel summary 
Flags:  D - down        P/bndl - bundled in port-channel
        I - stand-alone s/susp - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 1
Number of aggregators:           1

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1       Po1(RU)                  Gi3(P)

RU - L3 port-channel UP State
SU - L2 port-channel UP state
P/bndl -  Bundled
S/susp  - Suspended

Additional References for EVPN Multihoming

Standards and RFCs

Standard

Title

RFC 7432

BGP MPLS-Based Ethernet VPN

Feature Information for EVPN Multihoming

Feature Name

Releases

Feature Information

EVPN Multioming

Cisco IOS XE Fuji 16.9.x

The EVPN Multihoming feature utilizes the BGP MPLS-based Ethernet VPN (EVPN) functionality to achieve Multihoming between a Provider Edge and a Customer Edge device.

The following command was introduced or modified: redundancy all-active