Configuring 1G Traffic on 8-port 10 Gigabit Ethernet Interface Module

The 8-port 10 Gigabit Ethernet Interface Module (8X10GE) has eight ports and is supported on the RSP3 module. Prior to Cisco IOS XE Everest 16.5.1, 1G traffic support was provided only with the devices placed in the access layer. Effective Cisco IOS XE Everest 16.5.1, 1G traffic support is provided to devices in the distribution layer. Thus, all the eight port provide support for 1G mode as well as 10G mode.

The configuration of 1G traffic on 8X10GE interface module provides cost-effective solution during migration from 1G mode to 10G mode as a single device supports both the modes.


Note


By default, the 8X10GE inteface module comes up in the 10G mode after reboot.


Restrictions for 1G Mode on 8X10 GE Interface Module

  • SFP+ is not supported on 1G mode, but the physical link with SFP+ in 1G mode comes up.

  • Support of 1G mode on a port and 10G mode on another port in the same interface module is not supported.

  • Precision Time Protocol (PTP) is not supported.

  • Sync-E is not supported. However, Sync-E is supported in over subscription mode on the interface module.

  • Port channel bundling on 1G mode is not supported.

  • Although 1G mode is supported on the interface module, the interface is displayed as "Te0/X/Y" depending on the port numbers for both 1G and 10G modes.

  • 10G mode support on 8X10GE interface module does not change with dual-rate support.

  • Carrier delay configuration of less than 2 seconds is not supported on both 1G and 10G modes for the 8-port 10 Gigabit Ethernet interface module.

Configuring 1G Mode

Defaulting the Interface Module:

enable
hw-module subslot 0/4 default
end

Changing the Mode:

enable
configure terminal
hw-module subslot 0/4 ether-mode 1G
end

Configuring the Ports:

enable
configure terminal
interface te0/4/0
ip address 63.0.0.1 255.0.0.0
end

Verifying 1G Mode Configuration

The transport mode is LAN (1GB/s). The speed and bandwidth are 1000 Mbps and 1000000 Kbit/sec, respectively.

To verify the configuration, use show interface command in privileged EXEC mode:

Router#show interface tengigabitethernet0/4/0 
 
   TenGigabitEthernet0/4/0 is up, line protocol is up 
   Hardware is A900-IMA8Z, address is c8f9.f98d.2024 (bia c8f9.f98d.2024)
   Internet address is 50.0.0.1/8
   MTU 1500 bytes, te0/4/0, DLY 10 usec, 
      reliability 255/255, txload 1/255, rxload 1/255
   Encapsulation ARPA, loopback not set
   Keepalive set (10 sec)
   Full Duplex, 1000Mbps, link type is auto, media type is SX
   output flow-control is off, input flow-control is off
   Transport mode LAN (1Gb/s) 
   ARP type: ARPA, ARP Timeout 04:00:00
   Last input 00:08:24, output 00:08:24, output hang never
   Last clearing of "show interface" counters 00:07:59
   Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
   Queueing strategy: fifo
  …..

To verify the slots configured in 1G mode, use the show running-config | i ether-mode command in privileged EXEC mode:

Router#show running-config | i ether-mode 
hw-module subslot 0/3 ether-mode 1g
hw-module subslot 0/4 ether-mode 1g
hw-module subslot 0/11 ether-mode 1g

To verify the bandwidth and port speed, use the show platform hardware pp active interface all in privileged EXEC mode:

Router#show platform hardware pp active interface all
      Interface manager platform keys
     -------------------------------------------------

     Name: TenGigabitEthernet0/4/7, Asic: 0, hwidx: 9
     lpn: 0, ppn: 9, gid: 9, mac: c8f9.f98d.202b
    InLportId: 0, ELportId: 0, dpidx: 31, l3ID: 25
    port_flags: 0, port_speed: 1000 Mbps, efp_count: 0, destIndex: 9, intType: 1
    etherchnl: 0, efp: 0, bdi: 0, l2PhyIf: 0, l3PhyIf: 1, l3TDM: 0, loopBack: 0
    tunnel: 0, tunneltp: 0, icmp_flags: 0, icmp6_flags: 0
    bandwidth: 1000000, fcid: 0, cid: 0, mpls_tbid: 0, protocols: 4
    v4_netsmask: 8, v4_tableid: 8, v6_tableid: 65535, vrf_tbid_dstrm: , snmp_index: 0
   bd_id: 0, encap: 1, ip_mtu: 1500, l2_max_tu: 1500, l2_min_tu: 0
   vrfid: 8, enctype: 0, admin_state: 1, admin_state_oir: 0

   Name: TenGigabitEthernet0/4/6, Asic: 0, hwidx: 10
   lpn: 0, ppn: 10, gid: 10, mac: c8f9.f98d.202a
   InLportId: 0, ELportId: 0, dpidx: 30, l3ID: 24
   port_flags: 0, port_speed: 1000 Mbps, efp_count: 0, destIndex: 10, intType: 1
  etherchnl: 0, efp: 0, bdi: 0, l2PhyIf: 0, l3PhyIf: 1, l3TDM: 0, loopBack: 0
  tunnel: 0, tunneltp: 0, icmp_flags: 0, icmp6_flags: 0
  bandwidth: 1000000, fcid: 0, cid: 0, mpls_tbid: 0, protocols: 4
  v4_netsmask: 8, v4_tableid: 6, v6_tableid: 65535, vrf_tbid_dstrm: , snmp_index: 0
  bd_id: 0, encap: 1, ip_mtu: 1500, l2_max_tu: 1500, l2_min_tu: 0
  vrfid: 6, enctype: 0, admin_state: 1, admin_state_oir: 0

Configuring 10G Mode from 1G Mode

Deafulting the Interface Module:

enable
hw-module subslot 0/4 default
end

Changing the Mode:

enable
configure terminal
hw-module subslot 0/4 ether-mode 10G
end

Note


The default is 10G mode.


Configuring the Ports:

enable
configure terminal
interface te0/4/0
ip address 63.0.0.1 255.0.0.0
end

Verifying 10G Mode Configuration

To verify the configuration, use show interface command in privileged EXEC mode:
Router#show interface tengigabitethernet0/4/0  
   TenGigabitEthernet0/4/0 is up, line protocol is up 
   Hardware is A900-IMA8Z, address is c8f9.f98d.2024 (bia c8f9.f98d.2024)
   Internet address is 50.0.0.1/8
   MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec, 
      reliability 255/255, txload 1/255, rxload 1/255
   Encapsulation ARPA, loopback not set
   Keepalive set (10 sec)
   Full Duplex, 10000Mbps, link type is auto, media type is SX
   output flow-control is off, input flow-control is off
   Transport mode LAN 
   ARP type: ARPA, ARP Timeout 04:00:00
   Last input 00:08:24, output 00:08:24, output hang never
   Last clearing of "show interface" counters 00:07:59
   Input queue: 0/375/0/0 (size/max/drops/flushes); Total output drops: 0
   Queueing strategy: fifo
  …..

Note


For 10G mode, the hw-module subslot 0/x ether-mode 10G command is not displayed when you use show running-config command.


To verify the bandwidth and port speed, use the show platform hardware pp active interface all in privileged EXEC mode:

Router#show platform hardware pp active interface all
      Interface manager platform keys
     -------------------------------------------------

     Name: TenGigabitEthernet0/4/7, Asic: 0, hwidx: 9
     lpn: 0, ppn: 9, gid: 9, mac: c8f9.f98d.202b
    InLportId: 0, ELportId: 0, dpidx: 31, l3ID: 25
    port_flags: 0, port_speed: 10000 Mbps, efp_count: 0, destIndex: 9, intType: 1
    etherchnl: 0, efp: 0, bdi: 0, l2PhyIf: 0, l3PhyIf: 1, l3TDM: 0, loopBack: 0
    tunnel: 0, tunneltp: 0, icmp_flags: 0, icmp6_flags: 0
    bandwidth: 10000000, fcid: 0, cid: 0, mpls_tbid: 0, protocols: 4
    v4_netsmask: 8, v4_tableid: 8, v6_tableid: 65535, vrf_tbid_dstrm: , snmp_index: 0
   bd_id: 0, encap: 1, ip_mtu: 1500, l2_max_tu: 1500, l2_min_tu: 0
   vrfid: 8, enctype: 0, admin_state: 1, admin_state_oir: 0

   Name: TenGigabitEthernet0/4/6, Asic: 0, hwidx: 10
   lpn: 0, ppn: 10, gid: 10, mac: c8f9.f98d.202a
   InLportId: 0, ELportId: 0, dpidx: 30, l3ID: 24
   port_flags: 0, port_speed: 10000 Mbps, efp_count: 0, destIndex: 10, intType: 1
  etherchnl: 0, efp: 0, bdi: 0, l2PhyIf: 0, l3PhyIf: 1, l3TDM: 0, loopBack: 0
  tunnel: 0, tunneltp: 0, icmp_flags: 0, icmp6_flags: 0
  bandwidth: 10000000, fcid: 0, cid: 0, mpls_tbid: 0, protocols: 4
  v4_netsmask: 8, v4_tableid: 6, v6_tableid: 65535, vrf_tbid_dstrm: , snmp_index: 0
  bd_id: 0, encap: 1, ip_mtu: 1500, l2_max_tu: 1500, l2_min_tu: 0
  vrfid: 6, enctype: 0, admin_state: 1, admin_state_oir: 0

Overview of Over Subscription and Partial Port Modes on the 8-port 10 Gigabit Ethernet Interface Module

The 8-port 10 Gigbait Ethernet interface module (8X10GE) reqiures eight backplane XFI lines to the ASIC to operate efficiently. The chassis has different backplance capcity or bandwidth on each of its subslot. The 8X10GE interface module could only be used in sublsots that offered the eight XFI backplance lines. The following table shows the slots that 8X10GE interface module support without over subscription mode:

Slot No

Slot 0

Slot 1

Slot 2

Slot 3

Slot 4

Slot 5

Slot 6

Slot 7

Slot 8

Slot 9

Slot 10

Slot 11

Slot 12

Slot 13

Slot 14

Slot 15

8X10GE

No

No

No

Yes

Yes

No

No

Yes

Yes

No

No

Yes

Yes

No

No

No


Note


The router supports the 8X10GE interface module individually on the above slots, and offer eight XFI/SFI lines. But as a combination of slots to support 400G bandwidth, only five slots are supported for the 8X10GE interface module. With over subscription or partial mode enabled on the router six slots are available to support the bandwidth.


Over subscription mode enables the operation of the 8X10GE interface module in all subslots with a lesser backplane capacity. Hence, with over subscription mode enabled, all the front plane ports of the interface module are able to receive and transmit traffic.

Partial port mode is used to free the used serializer/deserializer (SerDes) lines to accommodate interface modules that support over subscription in those slots that may utilize the shared SerDes. The advantage of this mode is that the Channelized Network Interface Scheduler (CNIS) of ASIC, a limited resource, is not utilized, as compared to the over subscription mode.

Both these modes aid in increasing the nmber of interface modules in the maximum number of subslots on the chassis.

Over Subscription Mode

Over subscription mode is introduced to support population of maximum number of interface modules on the chassis.

The 8X10GE interface module requires eight backplane XFI lines to operate, where each front plane port fully utilizes a backplane XFI line. Hence, it operates with an overall bandwidth of 80Gbps. When over subscription is enabled, a group of front plane ports are channelized onto a single backplane XFI line, which reduces the bandwidth based on the number of ports multiplexed onto the backplane XFI line.

When the 8X10GE interface module is in over subscribed mode, all the eight front plane ports are functional.

2:1 — Two front plane ports are multiplexed onto one backplane XFI. The overall bandwidth of the interface module is 40Gbps.

Partial Port Mode

Partial port mode is also introduced to support maximum number of interface modules on the chassis.

This mode, unlike over subscription mode does not multiplex the front plane port, but blocks some front plane ports to free up the backplane XFI lines used by them.

Partial Port mode has one variant:

4 port mode — Only four front plane ports are enabled. Each port uses one backplane XFI line. Hence each port supports 10Gbps data rate, and the interface module supports 40Gbps datarate.

Prerequisites for Over Subscription Mode on the 8-port 10 Gigabit Ethernet Interface Module

  • FPGA must be upgraded to version 0.22. Use the upgrade hw-module subslot 0/x fpd bundled reload command to upgrade manually, before configuring over-subscription mode.

Restrictions for Over Subscription Mode 8-port 10 Gigabit Ethernet Interface Module

The following restrictions are applicable for the over subscription mode on the 8-port 10 Gigabit Ethernet Interface Module (A900-IMA8Z) on the ASR 907 Router:

  • Traffic prioritization is supported, but policing is not supported.

  • PTP over over subscription mode is not supported.

  • Dynamic over subscription mode change does not work. Reload the router after any mode change.

Supported Features and Constraints

Following are the supported features and constraints for configuring over subscription and partial port mode on the 8X10 GE interface module.

Table 1. Over Subscription Mode and Partial Port Mode Support Features and Constraints

8X10 GE Over Subscription Mode

4 X10 G Partial Port Mode

Supported Platforms

ASR 907 RSP3-400

ASR 907 RSP3-400

FPGA Mode

Supported only with XFI passthrough mode

Minimum version 0.22

Supported on both XFI passthrough and port expansion mode

Subslots

Supported on only selected subslots

Supported on only selected subslots

Mode Enablement

Activated on router reload

Activated on router reload

Backplane SerDes Selection

Static; Cannot define backplane SerDes

Static; Cannot define backplane SerDes

Dual Rate Support (1G / 10G)

Not supported on 1G mode in Cisco IOS XE Fuji 16.9.1.

Not supported on 1G mode in Cisco IOS XE Fuji 16.9.1.

LAN/WAN/OTN Support

10G Eth (LAN) mode is supported in Cisco IOS XE Fuji 16.9.1.

Supports LAN/WAN/OTN modes

Supported Subslots

The table shows the subslots of the different over subscription modes and also provides information about the SerDes line from the ASIC (multiplexed) to the frontplane ports on the chassis:

Table 2. Supported Subslots and SerDes Lines used by the 8X10GE Interface Module with Over Subscription Modes

Mode

Supported Slots

SerDes Lines Used

Enabled Ports

2:1 over subscription mode

3, 4

2, 3 6, 7

All ports

11, 12

0, 1,2, 3

4 Port Mode (Partial Port mode)

3,4

2,3,6,7

0,1,4,5


Note


Serializer/Deserializer (SerDes) is not released when dependant slot interface modules are in shutdown unpowered state.


FPGA Operating Mode

The FPGA operates in the following modes. The FPGA operating modes are selected by configuration.

  • Port Expansion Mode — Allows port expansion on QSGMII based interface module such as the 8X1G interface module or 8x1G+10G combo interface module. The FPGA consumes the port expansion quad on ASIC.

  • XFI Passthrough Mode — Supports XFI passthrough for enabling new XFI lines in certain slots of the chassis.


Note


System reload is required after changing the FPGA mode .



Note


Over subscription on the 8X10GE interface module is supported only with the XFI Passthrough mode.


The license feature service-offlload enable command is used to change the FPGA mode to the XFI Passsthrough mode.

The default setting of this command is the no form of the command. The default FPGA operation mode is XLAUI-QSGMII Port expansion mode.

Maximum Slot Population of the 8-port 10 Gigabit Ethernet Interface Module

Over subscription and partial port mode is implemented to free up the shared SerDes lines to other interface modules, and to also populate the 8X10GE interface modules in maximum possible slots with an optimum bandwidth support.


Note


A total of six 8x10GE interface modules are populated on the ASR 907 chassis with the RSP3-400 module.


The following table shows the modes selected on each subslot, and the CNIS utilized in that subslot in order to realise the maximum slot population of 8X10GE interface module.
Table 3. Maximum Slot Population of the 8X10 GE Interface Module

Subslot

8X10 GE Interface Module Mode

Port Numbers

SerDes Numbers

ASIC No.

CNIS Used

4

4X10G Partial Port

0

27

ASIC-1

0

1

26

4

15

5

14

8

8X10G Fully Subscibed Mode

0

7

ASIC-1

0

1

6

2

5

3

4

4

3

5

2

6

1

7

0

12

4X10G Partial Port

4

11

ASIC-1

0

5

10

6

9

7

8

3

4X10G Partial Port

0

27

ASIC-0

0

1

26

4

15

5

14

7

8X10G Fully Subscribed

0

7

ASIC-0

0

1

6

2

5

3

4

4

3

5

2

6

1

7

0

11

4X10G Partial Port

4

11

ASIC-0

0

5

10

6

9

7

8

Configuring Over Subscription and Partial Mode

Use the platform hw-module configuration to configure the mode on the chassis.

  • Example: Configuring over subscription mode

    Router(config)#platform hw-module configuration
    
    Router(conf-plat-hw-conf)# hw-module 0/12 A900-IMA8Z mode 8x10G-2:1-OS
    
  • Example: Confguring parital port mode

    Example: Router(config)#platform hw-module configuration
    Router(conf-plat-hw-conf)# hw-module 0/3 A900-IMA8Z mode 4-ports-only
    

Persistent Bandwidth for A900-IMA8Z

Table 4. Feature History

Feature Name

Release Information

Description

Persistent Bandwidth for 8-port 10 Gigabit Ethernet Interface module (A900-IMA8Z)

Cisco IOS XE Cupertino 17.9.1

This feature persistently retains the configured bandwidth value of the interface for 8-port 10 Gigabit Ethernet Interface module (A900-IMA8Z) across triggers such as interface shut or no-shut, IM reload, Stateful Switchover (SSO), and so on.

This feature is only supported on Cisco RSP3 module.

This feature is only supported on NCS 4206 and NCS 4216 routers.

Interface bandwidth sets and communicates bandwidth value for an interface to higher-level protocols such as OSPFv2 and OSPFv3. Starting with Cisco IOS XE Cupertino Release 17.9.1, when you configure interface bandwidth value for 8-port 10 Gigabit Ethernet Interface module (A900-IMA8Z) and perform triggers such as interface shut or no-shut, IM reload, and Stateful Switchover (SSO), the bandwidth value for the interface is persistently retained. Prior to this release, the bandwidth value would reset to the default value for any trigger.

Configure Bandwidth on Physical Interfaces

To configure bandwidth on the physical interfaces:

! 
interface TenGigabitEthernet0/4/6 
bandwidth 2000
 ip address 1.1.11.1 255.255.255.224
no shut
!

Verify Bandwidth Configuration

Use the show interface command to display statistics for the network interfaces.

Router#show interface Te0/4/6
TenGigabitEthernet0/4/6 is up, line protocol is up 
  Hardware is A900-IMA8Z, address is 00af.1f5a.5ac1 (bia 00af.1f5a.5a94)
  MTU 1500 bytes, BW 2000 Kbit/sec, DLY 10 usec, 
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full Duplex, 10000Mbps, link type is auto, media type is 10GBase-SR
  output flow-control is unsupported, input flow-control is on
  Transport mode LAN