Configuring Ethernet Dataplane Loopback

Ethernet data plane loopback provides a means for remotely testing the throughput of an Ethernet port.

Prerequisites for Ethernet Data Plane Loopback

  • Ethernet loopback sessions are supported only of EFPs (service instances, Ethernet flow points, EVCs).

  • Dot1q tags must be configured while configuring Ethernet loopback sessions on EFPs. However, loopback sessions can be configured using dot1q/QinQ, even if the underlying EFP has the dot1q/QinQ range configured.

  • Internal loopback sessions configured must be within the 1 GB reserved bandwidth.

  • Internal loopback can be launched even when the physical interface port state is down.

Restrictions for Ethernet Data Plane Loopback on Physical Interfaces

  • Data plane loopback on routed port infrastructure is not supported.

  • Etype, src-mac, and llc-oui based loopback traffic filtering is not supported.

  • Port-level QoS is not bypassed.

  • Port shaper cannot be bypassed in facility loopback.

  • Facility and terminal Ethernet data plane loopback (ELB) are not supported on dot1ad nni interface.

  • Internal loopback sessions configured must be within the 1 GB reserved bandwidth for Cisco ASR 900 Series RSP2 Module.

  • A maximum number of 20 facility loopback sessions can be created per system, provided 16 sessions are with Dot1Q and 4 sessions are with Dot1Q and destination MAC address. This scale reduces if SPAN or RSPAN is configured. This scale is supported on the Cisco ASR 900 Series RSP2 module.

  • A maximum number of 12 terminal loopback sessions can be created per system, provided 8 sessions are with Dot1Q and 4 sessions are with Dot1Q and destination MAC address. This scale reduces if RSPAN or SADT is configured. This scale is supported on the Cisco ASR 900 Series RSP2 module.

  • Only one Ethernet loopback (terminal or facility) session can be active on an EFP at any instance.

  • Local SPAN and ELB cannot be enabled on a physical interface at the same time.

  • Loopback sessions cannot be initiated on a port configured with SPAN or RSPAN.

  • Ethernet loopback is not supported on a range of dot1q tags.

  • Ethernet Data Plane Loopback is affected on STP enabled interface.

  • Dynamic addition of rewrite ingress tags with default EFP is not supported.

  • Dynamic changes at EFP and interface level are not supported when Ethernet Data Plane Loopback is active.

  • Egress EFP is not updated for external Ethernet data plane loopback statistics.

  • For internal Ethernet data plane loopback ingress and egress interface statistics are not updated on interface, where internal ELB is enabled.

  • If traffic is more than 650Mbps and if the packet size is less than a frame size of 64, then BFD and OSPF flaps are expected.

  • Starting from Cisco IOS XE Cupertino Release 17.7.1, SADT is not supported over EFP with untagged VLAN.

RSP3 Module

  • Starting from Cisco IOS XE Amsterdam 17.1.x release, the template sr_5_label_push_enable is not supported with Ethernet loopback.

  • Etype, VLAN, COS, src-mac, and llc-oui based loopback traffic filtering is not supported.

  • Port-based ELB is not supported. Ethernet loopback sessions are supported only on EFPs and Trunk EFPs on the RSP3 module.

  • Internal ELB is not supported when the physical interface port state is down.

  • Data filtering of loopback is not enforced for the traffic coming in the opposite direction.

  • Filtering based on specific VLAN is not supported. ELB is applicable to all the VLANs configured in EFP on the RSP3 module.

  • Random unknown IP packets may be looped back with MAC address swap, but RSP3 module does not support IP address swap. If the packet has a destination MAC address and a destination IP address similar to the BDI MAC and IP address, then the packet is punted and is not looped back. The L3 packets that must be routed is also not looped back. Hence, dscp/prec marking with ELB is not supported.

  • Traffic to TEFP, which is not part of BDI or CFM, is looped back based on filters. But traffic to TEFP with VLAN which is part of BDI or CFM, is not looped back on the RSP3 module. This is applicable for both types of Ethernet data plane loopback.

  • All packets with broadcast or multicast destination MAC is not qualified for ELB, hence such packets will not be looped back on the RSP3 module.

  • Three-level HQOS shaper/policer not supported with ELB.

  • Dot1Q filter is not supported.

  • Internal loopback sessions configured must be within the 100 GB reserved recycle bandwidth.

  • MAC-ACL cannot be bypassed in with facility loopback.

  • A maximum number of 20 facility loopback and 12 terminal loopback sessions are supported.

Information on Ethernet Data Plane Loopback

The Ethernet data plane loopback feature provides a means for remotely testing the throughput of an Ethernet port. You can verify the maximum rate of frame transmission with no frame loss. This feature allows for bidirectional or unidirectional throughput measurement, and on-demand/out-of-service (intrusive) operation during service turn-up. This feature supports two types of Ethernet loopback. RSP3 supports the following types of loopback from Cisco IOS XE Everest 16.5.1 release.

  • Facility loopback (external)—Traffic loopback occurs at the Ingress interface. Traffic does not flow into the router for loopback.

  • Terminal loopback (internal)—Traffic loopback occurs at the Egress interface. Traffic loopback occurs after the traffic flows into the router to the other interface.

Prior to Cisco IOS XE Cupertino Release 17.8.1, this feature was only supported on the physical interfaces.

Starting with Cisco IOS XE Cupertino Release 17.8.1, this feature is also supported on the bundle interfaces.

QoS Support for Ethernet Data Plane Loopback

  • Ingress QoS is bypassed in external loopback on service instances.

  • Internal loopback sequence is as follows:

    • Ingress QoS

    • Egress QoS (egress port) (both, shaper and policer are supported).

    • Ingress QoS on ingress port and egress QoS on egress port (both, shaper and policer are supported) on the RSP3 module.

    • Ingress QoS on egress port and egress QoS on ingress port on the RSP3 module.

  • All port-level and EFP-level QoS is applicable for internal Ethernet data plane loopback.

  • For external Ethernet data plane loopback:

    • All port-level and EFP-level QoS is bypassed except for shaper.

    • Port-level shaper cannot be bypassed.

How to Configure Ethernet Data Plane Loopback on Physical Interfaces

Enabling Ethernet Data Plane Loopback on Physical Interfaces

Table 1. Feature History

Feature Name

Release Information

Feature Description

EDPL support on dot1ad

Cisco IOS XE Cupertino 17.8.1

This features enables configuration of Ethernet Data Plane Loopback on interfaces configured with 802.1ad encapsulation. This helps measure the interface throughput handing encapsualted traffic. 

enable
configure terminal
interface gigabitethernet 0/2/1
service instance 1 ethernet
encapsulation dot1ad 101 dot1q 100
bridge-domain 120
ethernet loopback permit external
end

Note
ELB is supported using a MAC filter for UP-MEP session. If you are starting ELB without the MAC filter, the UP-MEP session will go DOWN.

Starting an Ethernet Data Plane Loopback Session on Physical Interfaces


Note
To start a loopback for untagged and default EFPs, dot1q and second-dot1q are not needed. Dot1q is not applicable to start a loopback session on the RSP3 module.

Note
By default the session would be running for 300 seconds unless you explicitly specify and automatically stops after the session time expiry. 

enable
configure terminal
ethernet loopback start local interface gigabitEthernet 0/4/1 service instance 10 external dot1q 10 cos 1 destination mac-address 0000.0000.0001 timeout none
end
This is an intrusive loopback and the packets matched with the service will not be able
to pass through.
Continue? (yes/[no]): yes
Dot1q and COS-based filtering is not supported on the RSP3 module.

enable
configure terminal
ethernet loopback start local interface gigabitEthernet 0/4/1 service instance 10 external destination mac-address 0000.0000.0001 timeout none
end

Stopping an Active Session on Physical Interfaces

Use the ethernet loopback stop command to stop an active session on an interface or to stop all sessions based on the session id. 

Router# ethernet loopback stop local interface gigabitEthernet 0/4/1 id 1

Configuration Examples

Example: Configuring External Loopback on Physical Interfaces

This example shows how to configure external (facility) loopback. 

Router(config)# interface gigabitEthernet 0/4/1
Router(config-if)# service instance 1 ethernet
Router(config-if-srv)# encapsulation dot1ad 100 dot1q 120
Router(config-if-srv)# bridge-domain 120
Router(config-if-srv)# ethernet loopback permit external

This example shows external (facility) loopback on the Gigabit Ethernet 0/4/1 interface: 

interface GigabitEthernet0/4/1
 no ip address
 negotiation auto
 service instance 10 ethernet
  encapsulation dot1q 10
  rewrite ingress tag pop 1 symmetric
  bridge-domain 10
  ethernet loopback permit external ===? For facility loopback
 !
end

This example below shows how to start external (facility) loopback on the router. A warning message is displayed. Type yes to continue. 

Router# ethernet loopback start local interface gigabitEthernet 0/4/1 service instance 10 external dot1q 10 cos 1
 destination mac-address 0000.0000.0001 timeout none

This is an intrusive loopback and the packets matched with the service will not  be able to pass through. 
Continue? (yes/[no]): yes

Note
Dot1q and COS-based filtering is not supported on the RSP3 module.

Example: Configuring Terminal Loopback on Physical Interfaces

This example shows how to configure internal (terminal) loopback. 

Router(config)# interface gigabitEthernet 0/0/0
Router(config-if)# service instance 1 ethernet
Router(config-if-srv)# encapsulation dot1q 120
Router(config-if-srv)# bridge-domain 120
Router(config-if-srv)# ethernet loopback permit internal

This example shows internal (terminal) loopback on Gigabit Ethernet 0/0/0 interface: 

interface TenGigabitEthernet0/0/0
 no ip address
service instance 10 ethernet
  encapsulation dot1q 10
  rewrite ingress tag pop 1 symmetric
  bridge-domain 10
  ethernet loopback permit internal 
 !
end

Verifying Ethernet Data Plane Loopback

Example: Verifying Ethernet Dataplane Loopback on Physical Interfaces

Use the show ethernet loopback {active | permitted} [interface interface number] command.

  • The following example displays the loopback capabilities per interface. The output shows internal (terminal) loopback has been permitted on Ten Gigabit Ethernet 0/0/0 interface and external (facility) loopback has been permitted on Gigabit Ethernet 0/4/1 interface. 
    Router# show ethernet loopback permitted 

--------------------------------------------------------------------------------
Interface                               SrvcInst Direction            
Dot1q/Dot1ad(s)                         Second-Dot1q(s)
--------------------------------------------------------------------------------
Te0/0/0                                 10                             Internal
10
Gi0/4/1                                 10                             External
10
  • This example shows all active sessions on the router. 
    Router# show ethernet loopback active           

============================================================
Loopback Session ID        : 1
Interface                  : GigabitEthernet0/4/1
Service Instance           :10
Direction                  : External
Time out(sec)              : none
Status                     : on
Start time                 : 10:31:09.539 IST Mon Aug 26 2013
Time left                  : N/A

Second-dot1q(s)            :
Source Mac Address         : Any
Destination Mac Address    : 0000.0000.0001
Ether Type                 : Any
Class of service           : 1
Llc-oui                    : Any


Total Active Session(s)    : 1
Total Internal Session(s)  : 0
Total External Session(s)  : 1

  • This example shows how to stop the sessions on the router. 

    Router# ethernet loopback stop local interface GigabitEthernet
0/4/1 id 1

Information on Enhanced Ethernet Data Plane Loopback

Table 2. Feature History

Feature Name

Release

Description

Enhanced Ethernet Data Plane Loopback

 Cisco IOS XE Bengaluru 17.4.1

The Ethernet data plane loopback feature is enhanced to avoid control packets getting dropped. The enhancement supports internal shaper configuration, when terminal ELB session is activated or deactivated to rate the limit the ELB session traffic.

The enhancement is applicable only on internal loopback.

The Ethernet data plane loopback feature is enhanced to avoid control packets getting dropped, besides the ELB traffic drop. If terminal ELB is configured on 1 GB interface, then other priority traffic like BFD is dropped due to congestion.

The enhancement supports internal shaper configuration, when terminal ELB session is activated or deactivated to limit the rate for ELB session traffic.

Restrictions

  • After starting EDPL, even if you remove ELB_SHAPE from EFP, then EDPL continues to run.

  • Do not modify the ELB_SHAPE directly.

    • Use the platform command to make any changes in ELB_SHAPE PM.

    • If you directly modify the ELB_SHAPE PM, then the shaper is applied based on the updated values that are defined under ELB_SHAPE.

  • If you remove ELB_SHAPE PM from the global config accidentally, then you must remove the platform edpl_internal_shaper xxx platform command and reapply.

  • For any change to the EDPL parameters, we recommended you to perform the following:

    1. Deactivate or Stop the EDPL session.

    2. Do the required configuration changes.

    3. Activate or Start the EDPL session.

    Do not dynamically change or delete the EDPL shaper or the interface bandwidth.

  • In an xconnect scenario, when the interface goes down, the xconnect goes down, but EDPL remains active. Hence, you must stop the EDPL session and start back the EDPL session, after the xconnect is UP.

Configuring Ethernet Data Plane Loopback Session

Procedure

Step 1

enable

Example:


Device> enable

Enables privileged EXEC mode.

  • Enter your password if prompted.

Step 2

configure terminal

Example:


Device# configure terminal

Enters global configuration mode.

Step 3

configure EDPL shaper

Example:


Device# platform edpl_internal_shaper  600000000

Enables EDPL shaper.

Step 4

interface GigabitEthernet

Example:


Device(config)# interface GigabitEthernet 0/0/9
Device(config-if)# service instance 1401 ethernet
Device(config-if-srv)# encapsulation dot1q 1401
Device(config-if-srv)# rewrite ingress tag pop 1 symmetric
Device(config-if-srv)# ethernet loopback permit internal
Device(config-if-srv)# end

Configures internal loopback on the interface.

Step 5

ethernet loopback start local

Example:


Device# ethernet loopback start local interface GigabitEthernet 0/0/9 service instance 1401 internal dot1q 1401 timeout none

Starts EDPL loopback session.

Example: Configuring Ethernet Loopback Active

The following example shows the activated and running EDPL session:

Router# show ethernet loopback active 

Load for five secs: 5%/1%; one minute: 6%; five minutes: 6%
Time source is NTP, 18:18:23.680 IST Wed Aug 26 2020

============================================================
Loopback Session ID     : 1
Interface               : GigabitEthernet0/0/9
Service Instance        : 1401
Direction               : Internal
Time out(sec)           : none
Status                  : on
Start time              : 18:17:58.360 IST Wed Aug 26 2020
Time left               : N/A
Dot1q/Dot1ad(s)         : 1401
Second-dot1q(s)         : 
Source Mac Address      : Any
Destination Mac Address : Any
Ether Type              : Any
Class of service        : Any
Llc-oui                 : Any

Total Active Session(s): 1
Total Internal Session(s): 1
Total External Session(s): 0

To deactivate an EDPL session.

Router# ethernet loopback stop local interface gigabitEthernet 0/0/9 id 1

003846: Aug 26 18:40:56.528 IST: %E_DLB-6-DATAPLANE_LOOPBACK_STOP: Ethernet Dataplane Loopback Stop on interface GigabitEthernet0/0/9 service instance 1401 with session id 1

Example: Configuring Ethernet Data Plane Loopback Session

The following example shows the global shaper configuration:


Note
If the interface bandwidth is less than the shaper value, then EDPL does not get started. You can refer to the syslogs. 

Interface bandwidth 400Mb, shaper value 600Mb

platform edpl_internal_shaper  600000000

interface GigabitEthernet0/0/9
 bandwidth 400000

service instance 1401 ethernet
  encapsulation dot1q 1401
  rewrite ingress tag pop 1 symmetric
  xconnect 192.211.92.3 1401 encapsulation mpls
  ethernet loopback permit internal
 !

Router# ethernet loopback start local interface gigabitEthernet 0/0/9 service instance 1401 internal dot1q 1401 timeout none

This is an intrusive loopback and the packets matched with the service will not be able to pass through. Continue? (yes/[no]): yes
QoS Configuration failed !!! Shaper value is greater than interface speed

 Failed to config EDPL shaper, EDPL not activated !!!

Use Cases or Deployment Scenarios

ELB is Supported with MAC Filter for UP-MEP Session

In the following scenario, ELB is supported using a MAC filter for UP-MEP session. If you starting ELB with out MAC filter, the UP-MEP session will go DOWN. 

enable
configure terminal
service instance 800 ethernet 800
encapsulation dot1q 800
service-policy input <NAME>
xconnect 10.0.0.2 880 encapsulation mpls
cfm mep domain <NAME> mpid 200
cos 7
ethernet loopback permit external
ethernet loopback permit internal

Router#ethernet loopback start local interface gi0/0/0 service instance 800 internal dot1q 800 destination mac-address f078.1685.313f timeout none

This is an intrusive loopback and the packets matched with the service will not  be able to pass through. Continue? (yes/[no]): yes


Router#show ethernet cfm maintenance-points remote
--------------------------------------------------------------------------------
MPID  Domain Name                                 MacAddress          IfSt  PtSt
 Lvl  Domain ID                                   Ingress
 RDI  MA Name                                     Type Id             SrvcInst
      EVC Name                                                        Age
      Local MEP Info
--------------------------------------------------------------------------------
220   CCI                                         f078.1685.313f      Up    Up
 0    CCI                                         Gi0/0/0:(10.0.0.2, 880)
 -    800                                         XCON N/A            800
      800                                                             0s
      MPID: 200 Domain: CCI MA: 800

Total Remote MEPs: 1

Support for Ethernet Data Plane Loopback on Bundle Interface

Table 3. Feature History

Feature Name

Release Information

Feature Description

Support for Ethernet Data Plane Loopback on Bundle Interface

 Cisco IOS XE Cupertino 17.8.1

This feature enables ethernet data plane loopback on bundle interfaces. You can also configure the feature when the router is not physically connected and the port is in down state.

This feature is only applicable on internal or terminal loopback in up or down state.

Bundle interface or a link bundle is a group of one or more ports that are aggregated together and treated as a single link. This allows you to group multiple point-to-point links together into one logical link and provide higher bidirectional bandwidth, redundancy, and load balancing between two routers. A virtual interface is assigned to the bundled link. The component links can be dynamically added and deleted from the virtual interface. The virtual interface is treated as a single interface on which you can configure an IP address and other software features used by the link bundle. Packets sent to the link bundle are forwarded to one of the links in the bundle.

Bundle interfaces increase bandwidth availability, because traffic is forwarded over all available members of the bundle. Therefore, traffic can if one of the links within a bundle fails. can without interrupting packet flow. The ethernet dataplane loopback feature configured on bundle interfaces provides a methodology to verify the maximum rate of frame transmission with no frame loss.

Prior to Cisco IOS XE Cupertino 17.8.1, you could only configure ethernet data plane loopback on the physical interfaces.

Starting with Cisco IOS XE Cupertino 17.8.1, you can also configure ethernet data plane loopback feature on the bundle interfaces. But, you can only configure internal or terminal loopback in up or down state.

This feature is only supported on Cisco RSP2 module.

Scenario: Support for Ethernet Dataplane Loopback on Link Down Port

Consider a scenario when you need to configure ethernet dataplane loopback feature before the router is physically connected. Thus, you need to configure the feature when the port link is down. Starting with Cisco IOS XE Cupertino 17.8.1, you can configure the terminal or internal ethernet dataplane loopback feature even when the router is not physically connected and the link is down. But, as this feature is not supported on external or facility loopback, you cannot configure external loopback feature when the port link is down.

A port channel bundles individual interfaces into a group to provide increased bandwidth and redundancy. Previously, the internal ethernet dataplane loopback was not supported on the port channel interface. Starting with Cisco IOS XE Cupertino 17.8.1, you can now configure internal loopback on the port channel interface even when the interface is down.

Restrictions for Ethernet Dataplane Loopback on Bundle Interface

  • The internal ethernet dataplane loopback feature is only available with service instance for port channel interface. It is not available when you configure MPLS or IP or Layer 3 on port channel interface.

  • At least one member link must be added to the port channel interface for ethernet dataplane loopback.

  • External ethernet loopback session on port channel interface is not supported.

  • This feature will only function for traffic flow on first member of the port channel.

  • You cannot configure the feature when the bundle members are in suspended state.

  • The maximum traffic performance of terminal loopback is 1GBPS.

Configure Ethernet Dataplane Loopback Start Session on Bundle Interface

  1. Activate Loopback on the EFP

    To activate terminal loopback on the EFP:

    interface Po1
 no ip address
service instance 10 ethernet
  encapsulation dot1q 10
  rewrite ingress tag pop 1 symmetric
  bridge-domain 10
  ethernet loopback permit internal  === For Terminal Loopback
 !

  2. Start Loopback Session

    To start a terminal loopback session on bundle interface:

    R11#ethernet loopback start local in po1 ser ins 2 inte dot1q 2 destination mac-address 3333.0001.0003 tim non

This is an intrusive loopback and the packets matched with the service will not  be able to pass through. Continue? (yes/[no]): yes

Configure Ethernet Dataplane Loopback Stop Session on Bundle Interface

  1. Stop Loopback Session

    To stop a terminal loopback session on a bundle interface:

    R2#ethernet loopback stop local interface po1 id 1

  2. Deactivate Loopback Session on the EFP

    To stop the terminal loopback session in the EFP:

    interface Po1
 no ip address
 service instance 10 ethernet
  encapsulation dot1q 10
  rewrite ingress tag pop 1 symmetric
  bridge-domain 10
  no ethernet loopback permit internal

Verification of Ethernet Dataplane Loopback Configuration on Bundle Interface

Use the show ethernet loopback active command to display all active sessions on the router. 

R11#show ethernet loopback active
============================================================
Loopback Session ID     : 1
Interface               : Port-channel1
Service Instance        : 2
Direction               : Internal
Time out(sec)           : none
Status                  : on
Start time              : 10:35:16.940 IST Fri Dec 17 2021
Time left               : N/A
Dot1q/Dot1ad(s)         : 2
Second-dot1q(s)         : 
Source Mac Address      : Any
Destination Mac Address : 3333.0001.0003
Ether Type              : Any
Class of service        : Any
Llc-oui                 : Any


Total Active Session(s): 1
Total Internal Session(s): 1
Total External Session(s): 0