Cisco MDS 9000 Series IP Services Configuration Guide, Release 9.x

Traffic Management Services

In-band options are compliant with and use the RFC 2625 standards. An NMS host running the IP protocol over an Fibre Channel interface can access the switch using the IPFC functionality. If the NMS does not have a Fibre Channel HBA, in-band management can still be performed using one of the switches as an access point to the fabric as shown in Figure 5-1.

Figure 5-1 Management Access to Switches

Management Interface Configuration

The management interface on the switch allows multiple simultaneous Telnet or SNMP sessions. You can remotely configure the switch through the management interface, but first you must configure IP version 4 (IPv4) parameters (IP address, subnet mask) or an IP version 6 (IPv6) address and prefix length so that the switch is reachable. For information on configuring IPv6 addresses, see Chapter8, “Configuring IPv6 for Gigabit Ethernet Interfaces”

On director class switches, a single IP address is used to manage the switch. The active supervisor module's management (mgmt0) interface uses this IP address. The mgmt0 interface on the standby supervisor module remains in an inactive state and cannot be accessed until a switchover happens. After a switchover, the mgmt0 interface on the standby supervisor module becomes active and assumes the same IP address as the previously active supervisor module.

Note The port on the Ethernet switch to which the MDS management interface is connected should be configured as a host port (also known as access port) instead of a switch port. Spanning tree configuration for that port (on the Ethernet switch) should disabled. This helps avoid the delay in the MDS management port coming up due to delay from Ethernet spanning tree processing that the Ethernet switch would run if enabled. For Cisco Ethernet switches, use either the switchport host command in Cisco IOS is or the set port host command in the Catalyst OS. Refer to the configuration guide for your Ethernet switch.

Note Before you begin to configure the management interface manually, obtain the switch’s IP address and IP subnet mask. Also make sure the console cable is connected to the console port.

To configure the mgmt0 Ethernet interface for IPv4, follow these steps:

To configure the mgmt0 Ethernet interface for IPv6, follow these steps:

To configure the mgmt0 Ethernet interface using Device Manager for IPv6, follow these steps:

Step 1 Select Interface > Mgmt > Mgmt0.

Step 2 Enter the description.

Step 3 Select the administrative state of the interface.

Step 4 Check the CDP check box to enable CDP.

Step 5 Enter the IP address mask.

Step 6 Click Apply to apply the changes.

Default Gateway

You can configure a default gateway IPv4 address on your Cisco MDS 9000 Family switch.

This section includes the following topics:

• Configuring the Default Gateway
• Verifying the Default Gateway Configuration

The default gateway IPv4 address should be configured along with the IPv4 static routing attributes (IP default network, destination prefix, and destination mask, and next hop address). If you configure the static route IP forwarding and the default-network details, these IPv4 addresses will be used regardless of the default-gateway being enabled or disabled.

The default gateway IPv4 address should be configured along with the IPv4 static routing attributes commands (IP default network, destination prefix, and destination mask, and next hop address).

Tip If you configure the static route IP forwarding and the default-network details, these IPv4 addresses will be used regardless of the default-gateway being enabled or disabled. If these IP addresses are configured but not available, the switch will fall back to using the default gateway IP address, if you have configured it. Be sure to configure IP addresses for all entries in the switch.

Use the ip default-gateway command to configure the IP address for a switch’s default gateway and the show ip route command to verify that the IPv4 address for the default gateway is configured.

Configuring the Default Gateway

To configure an IP route using Fabric Manager, follow these steps:

Step 1 Select Switches > Interfaces > Management, and select IP in the Physical Attributes pane.

Step 2 Click the Route tab in the information pane.

You see the IP route window showing the switch name, destination, mask, gateway, metric, interface, and active status of each IP route as shown in Figure 5-2.

To configure the default gateway, follow these steps:

Figure 5-2 IP Route For Multiple Switches

Step 3 Click the Create Row icon to add a new IP route.

You see the dialog box shown in Figure 5-3.

Figure 5-3 User-Defined Command Dialog Box

Step 4 Complete the fields in this window.

• Enter the switch name in the Switch field.
• Configure a static route, by entering the destination network ID and subnet mask in the Routedest and Mask fields.
• Configure a default gateway by entering the IP address of the seed switch in the Gateway field.
• Set the Metric and Interface fields.

Note With Cisco NX-OS Release 4.2(1) and later, CPP interfaces also are available for selection when you create a new IP route.

Step 5 Click the Create icon.

To configure an IP route or identify the default gateway using Device Manager, follow these steps:

Step 1 Choose IP > Routes.

You see the IP Routes window.

Step 2 Create a new IP route or identify the default gateway on a switch by clicking Create.

You see the dialog box shown in Figure 5-4.

Figure 5-4 User-Defined Command Dialog Box

Step 3 Complete the fields in this window.

• Enter the switch name in the Switch field.
• Configure a static route, by entering the destination network ID and subnet mask in the Routedest and Mask fields.
• Configure a default gateway by entering the IP address of the seed switch in the Gateway field.
• Set the Metric and Interface fields.

Note With Cisco NX-OS Release 4.2(1) and later, CPP interfaces also are available for selection when you create a new IP route.

If you choose the CPP interface, the switch uses the input CPP-assigned IP address and mask to generate the IP route prefix.

Step 4 Click Create to add the IP route.

The new IP route is created as shown in Figure 5-5.

Figure 5-5 IP Route Window

Note You cannot delete the switch-generated IP route for the CPP interface. If you try to delete the IP route for the CPP interface, SNMP displays this error message:
ip: route type not supported.

Verifying the Default Gateway Configuration

Use the show ip route command to verify the default gateway configuration.

IPv4 Default Network Configuration

If you assign the IPv4 default network address, the switch considers routes to that network as the last resort. If the IPv4 default network address is not available, the switch uses the IPv4 default gateway address. For every network configured with the IPv4 default network address, the switch flags that route as a candidate default route, if the route is available.

If you configure the static route IP forwarding and the default network details, these IPv4 addresses will be used regardless of the default gateway being enabled or disabled. If these IPv4 addresses are configured and not available, the switch will fall back to using the default gateway IPv4 address, if you have configured it. Be sure to configure IPv4 addresses for all entries in the switch if you are using IPv4.

When the Ethernet interface is configured, the switch should point to the gateway router for the IP network. The host accesses the gateway using a gateway switch. This gateway switch is configured as the default gateway. The other switches in the fabric that are connected to the same VSAN as the gateway switch can also be connected through the gateway switch. Every interface connected to this VSAN should be configured with the VSAN IPv4 address of the gateway switch as shown in Figure 5-6.

Figure 5-6 Overlay VSAN Functionality

In Figure 5-1, switch A has the IPv4 address 1.12.11.1, switch B has the IPv4 address 1.12.11.2, switch C has the IPv4 address 1.12.11.3, and switch D has the IPv4 address 1.12.11.4. Switch A is the gateway switch with the Ethernet connection. The NMS uses the IPv4 address 1.1.1.10 to connect to the gateway switch. Frames forwarded to any switch in the overlaid VSAN 1 are routed through the gateway switch. Configuring the gateway switch’s IPv4 address (1.12.11.1) in the other switches enable the gateway switch to forward the frame to the intended destination. Similarly, if a non-gateway switch in the VSAN forwards a frame to the Ethernet, the frame is routed through the gateway switch.

When forwarding is disabled (default), IP frames are not sent from one interface to another. In these cases, the software performs local IP routing between two switches using the in-band option for Fibre Channel traffic and the mgmt0 option for Ethernet traffic.

When a VSAN is created, a VSAN interface is not created automatically. You need to specifically create the interface.

To configure default networks using IPv4 addresses, follow these steps:

IP over Fibre Channel

IP over Fibre Channel (IPFC) provides IP forwarding on in-band switch management over a Fibre Channel interface (rather than out-of-band using the Gigabit Ethernet mgmt 0 interface). You can be use IPFC to specify that IP frames can be transported over Fibre Channel using encapsulation techniques. IP frames are encapsulated into Fibre Channel frames so NMS information can cross the Fibre Channel network without using an overlay Ethernet network.

Once the VSAN interface is created, you can specify the IP address for that VSAN. You can assign an IPv4 address or an IPv6 address.

Note See the Chapter 8, “Configuring IPv6 for Gigabit Ethernet Interfaces” for information about configuring IPv6 on the Cisco MDS 9000 Family switches.

This topic includes the following sections:

• IPFC Configuration
• Configuring an IPv4 Address in a VSAN
• Verifying the VSAN Interface Configuration
• Enabling IPv4 Routing
• Verifying the IPv4 Routing Configuration
• IPFC Configuration Example

IPFC Configuration

Follow this procedure to configure IPFC:

1. Create the VSAN to use for in-band management, if necessary.

2. Configure an IPv4 address and subnet mask for the VSAN interface.

3. Enable IPv4 routing.

4. Verify connectivity.

Configuring an IPv4 Address in a VSAN

To create a VSAN interface and configure an IPv4 address for that interface, follow these steps:

Verifying the VSAN Interface Configuration

Use the show interface vsan command to verify the configuration of the VSAN interface.

Note You can see the output for this command only if you have previously configured a VSAN interface.

Enabling IPv4 Routing

By default, the IPv4 routing feature is disabled in all switches.

To enable the IPv4 routing feature, follow these steps:

Verifying the IPv4 Routing Configuration

Use the show ip routing command to verify the IPv4 routing configuration.

IPFC Configuration Example

This section describe an example configuration for IPFC. Figure 5-7 shows an example network.

The example network has the following links:

• Switch_1 is connected to the main network by the mgmt 0 interface and to the fabric by an ISL.
• Switch_2 and Switch_3 are connected to the fabric by an ISL but are not connected to the main network.

Figure 5-7 IPFC Example Network

The following steps show how to configure Switch_1 in the example network in Figure 5-7:

Step 1 Create the VSAN interface and enter interface configuration submode:

Step 2 Configure the IP address and subnet mask:

Step 3 Enable the VSAN interface and exit interface configuration submode:

Step 4 Enable IPv4 routing:

Step 5 Display the routes:

The following steps show how to configure Switch_2 in the example network in Figure 5-7:

Step 1 Enable the mgmt 0 interface:

Note Configure this switch using the console connection.

Step 2 Create the VSAN interface and enter interface configuration:

Step 3 Configure the IP address and subnet mask:

Step 4 Enable the VSAN interface and exit interface configuration submode:

Step 5 Enable IPv4 routing:

Step 6 Display the routes:

Step 7 Verify the connectivity to Switch_1:

The following steps show how to configure Switch_3 in the example network in Figure 5-7:

Step 1 Enable the mgmt 0 interface:

Note Configure this switch using the console connection.

Step 2 Configure the IP address and subnet mask:

Step 3 Enable the VSAN interface and exit interface configuration submode:

Step 4 Enable IPv4 routing:

Step 5 Display the routes:

Step 6 Verify the connectivity to Switch_1:

IPv4 Static Routes

If your network configuration does not need an external router, you can configure IPv4 static routing on your MDS switch.

Note For information about IPv6 static routing, see the Chapter8, “Configuring IPv6 for Gigabit Ethernet Interfaces”

This section includes the following topics:

• Overview of IPv4 Static Routes
• Configuring IPv4 Static Routes
• Verifying IPv4 Static Route Information
• Displaying and Clearing ARPs

Overview of IPv4 Static Routes

Static routing is a mechanism to configure IPv4 routes on the switch. You can configure more than one static route.

If a VSAN has multiple exit points, configure static routes to direct traffic to the appropriate gateway switch. IPv4 routing is disabled by default on any gateway switch between the out-of-band management interface and the default VSAN, or between directly connected VSANs.

Configuring IPv4 Static Routes

To configure an IPv4 static route, follow these steps:

Verifying IPv4 Static Route Information

Use the show ip route command to verifying the IPv4 static route configuration:

Use the show ip route command to verifying the active and connected IPv4 static route:

Example 5-1 Displaying the IP Routing Status

Displaying and Clearing ARPs

Address Resolution Protocol (ARP) entries in Cisco MDS 9000 Family switches can be displayed, deleted, or cleared. The ARP feature is enabled on all switches.

• Use the show arp command to display the ARP table.

• Use the no arp command in configuration mode to remove an ARP entry from the ARP table.

• Use the clear arp command to delete all entries from the ARP table. The ARP table is empty by default:

Overlay VSANs

This section describes overlay VSANs and how to configure them.

This section includes the following topics:

Overlay VSANs

VSANs enable deployment of larger SANs by overlaying multiple logical SANs, each running its own instance of fabric services, on a single large physical network. This partitioning of fabric services reduces network instability by containing fabric reconfiguration and error conditions within an individual VSAN. VSANs also provide the same isolation between individual VSANs as physically separated SANs. Traffic cannot cross VSAN boundaries and devices may not reside in more than one VSAN. Because each VSAN runs separate instances of fabric services, each VSAN has its own zone server and can be zoned in exactly the same way as SANs without VSAN capability.

Configuring Overlay VSANs

To configure an overlay VSAN, follow these steps:

Step 1 Add the VSAN to the VSAN database on all switches in the fabric.

Step 2 Create a VSAN interface for the VSAN on all switches in the fabric. Any VSAN interface belonging to the VSAN has an IP address in the same subnet. Create a route to the IPFC cloud on the IP side.

Step 3 Configure a default route on every switch in the Fibre Channel fabric pointing to the switch that provides NMS access.

Step 4 Configure the default gateway (route) and the IPv4 address on switches that point to the NMS as shown in Figure 5-8:

Figure 5-8 Overlay VSAN Configuration Example

Note To configure the management interface displayed in Figure 5-8, set the default gateway to an IPv4 address on the Ethernet network.

The following procedure configures an overlay VSAN in one switch. This procedure must be repeated for each switch in the fabric.

To configure an overlay VSAN in one switch (using the example in Figure 5-8), follow these steps:

To configure the NMS station displayed in Figure 5-8, follow this step:

Configuring Multiple VSANs

More than one VSAN can be used to segment the management network in multiple subnets. An active interface must be present on the switch for the VSAN interface to be enabled.

To configure multiple VSANs, follow these steps:

Step 1 Add the VSAN to the VSAN database on any switch in the fabric.

Step 2 Create a VSAN interface for the appropriate VSAN on any switch in the fabric.

Step 3 Assign an IP address on every VSAN interface on the same subnet as the corresponding VSAN.

Step 4 Define the multiple static routes on the Fibre Channel switches and the IP cloud as shown in Figure 5-9:

Figure 5-9 Multiple VSAN Configuration Example

To configure an overlay VSAN (using the example in Figure 5-9), follow these steps:

Virtual Router Redundancy Protocol

Cisco MDS 9000 Family switches are compliant with RFC 2338 standards for Virtual Router Redundancy Protocol (VRRP) features. This section provides details on the VRRP feature.

This section includes the following topics:

• Overview of VRRP
• Configuring VRRP

Overview of VRRP

Cisco MDS 9000 Family switches are compliant with RFC 2338 standards for Virtual Router Redundancy Protocol (VRRP) features. VRRP provides a redundant alternative path to the gateway switch, which has connectivity to the NMS.

Note VRRP is not supported on Cisco MDS 24/10 port SAN Extension Module.

VRRP has the following characteristics and advantages:

• VRRP is a restartable application.
• When a VRRP master fails, the VRRP backup takes over within three times the advertisement time.
• VRRP over Ethernet, VRRP over VSAN, and Fibre Channel functions are implemented as defined in RFC 2338 and the draft-ietf-vrrp-ipv6 specification.
• A virtual router is mapped to each VSAN and Ethernet interface with its unique virtual router IP, virtual router MAC, and VR ID.
• VR IDs can be reused in multiple VSANs with different virtual router IP mapping.
• Both IPv4 and IPv6 is supported.
• The management interface (mgmt 0) supports only one virtual router group. All other interfaces each support up to seven virtual router groups, including both IPv4 and IPv6 combined. Up to 255 virtual router groups can be assigned in each VSAN.
• VRRP security provides three options, including no authentication, simple text authentication, and MD5 authentication.

Note If you are using IPv6, you must either configure an IPv6 address on the interface or enable IPv6 on the interface. For more information about IPv6, see Chapter8, “Configuring IPv6 for Gigabit Ethernet Interfaces”

In Figure 5-10, switch A is the VRRP master and switch B is the VRRP backup switch. Both switches have an IP address to VRRP mapping configured. The other switches set switch A as the default gateway. If switch A fails, the other switches do not have to change the routing configurations as switch B automatically becomes the master and takes over the function of a gateway.

Figure 5-10 VRRP Functionality

In Figure 5-11, the fabric example has two virtual router groups (VR1 and VR 2) because a virtual router cannot span across different types of interfaces. In both switch 1 and switch 2, the Ethernet interface is in VR 1 and the FC interface is in VR 2. Each virtual router is uniquely identified by the VSAN interface and the VR ID.

Figure 5-11 Redundant Gateway

Configuring VRRP

This section describes how to configure VRRP and includes the following topics:

• Adding and Deleting a Virtual Router
• Virtual Router Initiation
• Adding Virtual Router IP Addresses
• Setting the Priority for the Virtual Router
• Setting the Time Interval for Advertisement Packets
• Configuring or Enabling Priority Preemption
• Setting Virtual Router Authentication
• Tracking Interface Priority
• Displaying IPv4 VRRP Information
• Displaying IPv6 VRRP Information
• Displaying VRRP Statistics
• Clearing VRRP Statistics

Adding and Deleting a Virtual Router

All VRRP configurations should be replicated across switches in a fabric that runs VRRP.

Note The total number of VRRP groups that you can configure on a Gigabit Ethernet port, including main interfaces and subinterfaces, cannot exceed seven. This limitation applies to both IPv4 and IPv6 groups.

To create or remove a VR for IPv4, follow these steps:

To create or remove a VR for IPv6, follow these steps:

Virtual Router Initiation

By default, a virtual router is always disabled. VRRP can be configured only if this state is enabled. Be sure to configure at least one IP address, either IPv4 or IPv6, before attempting to enable a VR.

To enable or disable a virtual router configure for IPv4, follow these steps:

To enable or disable a virtual router configured for IPv6, follow these steps:

Adding Virtual Router IP Addresses

One virtual router IP address can be configured for a virtual router. If the configured IP address is the same as the interface IP address, this switch automatically owns the IP address. You can configure either an IPv4 address or an IPv6 address.

According to the VRRP specification, the master VRRP router drops the packets addressed to the virtual router's IP address because the virtual router is only intended as a next-hop router to forward packets. In MDS switches however, some applications require that packets addressed to virtual router's IP address be accepted and delivered to them. By using the secondary option to the virtual router IPv4 address, the VRRP router will accept these packets when it is the master.

To manage IP addresses for virtual routers from Device Manager, follow these steps:

Step 1 Choose IP > VRRP. You see the Operations tab of the VRRP dialog box.

Step 2 Click the IP Addresses tab on the VRRP dialog box.

Step 3 To create a new VRRP entry, click Create. You see the Create VRRP IP Addresses window.

Step 4 Complete the fields in this window to create a new VRRP IP address, and click OK or Apply.

To configure an IPv4 address for a virtual router, follow these steps:

To configure an IPv6 address for a virtual router, follow these steps:

Setting the Priority for the Virtual Router

The valid range to assign a virtual router priority is 1 to 254 with 1 being the lowest priority and 254 being the highest priority. The default value is 100 for switches with secondary IP addresses and 255 for switches with the primary IP address.

To set the priority for a virtual router using IPv4, follow these steps:

To set the priority for a virtual router using IPv6, follow these steps:

Setting the Time Interval for Advertisement Packets

The valid time range for an advertisement packet on an interface using IPv4 is between 1 and 41seconds. The default value is 1 (one) second. If the switch has the primary IP address, this time must be specified.

To set the time interval for advertisement packets for a virtual router using IPv4, follow these steps:

To set the time interval for advertisement packets for a virtual router using IPv6, follow these steps:

Configuring or Enabling Priority Preemption

You can enable a higher-priority backup virtual router to preempt the lower-priority master virtual router.

Note If the virtual IP address is also the IP address for the interface, then preemption is implicitly applied.

To enable or disable preempting when using IPv4, follow these steps:

To enable or disable preempting when using IPv6, follow these steps:

Setting Virtual Router Authentication

VRRP security provides three options, including simple text authentication, MD5 authentication, and no authentication:

• Simple text authentication uses a unique, 1 to 8 character password that is used by all switches participating in the same virtual router. This password should be different from other security passwords.
• MD5 authentication uses a unique, 16 character key that is shared by all switches participating in the same virtual router. This secret key is shared by all switches in the same virtual router.
• No authentication is the default option.

You can configure the key using the authentication option in the VRRP submode and distribute it using the configuration file. The security parameter index (SPI) settings assigned in this option should be unique for each VSAN.

Note All VRRP configurations must be duplicated.

Note VRRP router authentication does not apply to IPv6.

To set an authentication option for a virtual router, follow these steps:

Tracking Interface Priority

Interface state tracking changes the priority of the virtual router based on the state of another interface in the switch. When the tracked interface is down, the priority of the virtual router is restored to the interface state tracking value. When the tracked interface is up, the priority reverts to the priority value for the virtual router (see the “Setting the Priority for the Virtual Router” section). You can track the state of either a specified VSAN interface or the management interface (mgmt 0). The interface state tracking feature is disabled by default.

Note For interface state tracking to function, you must enable preemption on the interface. See the “Configuring or Enabling Priority Preemption” section.

To track the interface priority for a virtual router using IPv4, follow these steps:

To track the interface priority for a virtual router using IPv6, follow these steps:

Displaying IPv4 VRRP Information

Use the show vrrp vr command to display configured IPv4 VRRP information (see Examples 5-2 to 5-4 ).

Example 5-2 Displaying IPv4 VRRP Configured Information

Example 5-3 Displaying IPv4 VRRP Status Information

Example 5-4 Displaying IPv4 VRRP Statistics

Displaying IPv6 VRRP Information

Note The Cisco MDS 9250i switch, Cisco MDS 9148S switch, and the Cisco MDS 9396S switch do not support VRRP IPV6 feature.

Use the show vrrp ipv6 vr command to display configured IPv6 VRRP information (see Example 5-5 through Example 5-8).

Example 5-5 Displaying IPv6 VRRP Information

Example 5-6 Displaying IPv6 VRRP Interface Configuration Information

Example 5-7 Displaying IPv6 VRRP Interface Status Information

Example 5-8 Displaying IPv6 VRRP Statistics

Displaying VRRP Statistics

Use the show vrrp statistics command to display configured IPv6 VRRP information (see Example 5-9).

Example 5-9 Displaying VRRP Cumulative Statistics

Clearing VRRP Statistics

Use the clear vrrp statistics command to clear all the VRRP statistics for all interfaces on the switch (see Example 5-10).

Example 5-10 Clearing VRRP Statistics

Use the clear vrrp vr command to clear both the IPv4 and IPv6 VRRP statistics for a specified interface (see Example 5-11).

Example 5-11 Clearing VRRP Statistics on a Specified Interface

Use the clear vrrp ipv4 command to clear all the statistics for the specified IPv4 virtual router (see Example 5-12).

Example 5-12 Clearing VRRP IPv4 Statistics on a Specified Interface

Use the clear vrrp ipv6 command to clear all the statistics for the specified IPv6 virtual router (see Example 5-13).

Example 5-13 Clearing VRRP IPv6 Statistics on a Specified Interface

DNS Configuration

The DNS client on the switch communicates with the DNS server to perform the IP address-name server correspondence.

The DNS server may be dropped after two attempts because of one of the following reasons:

• The IP address or the switch name is wrongly configured.
• The DNS server is not reachable because external reasons (reasons beyond our control).

Note When accessing a Telnet host, if the DNS server is not reachable (for any reason) the switch login prompt may take a longer time to appear. If so, verify that the DNS server is accurately configured and reachable.

To configure a DNS server, follow these steps:

Displaying DNS Host Information

Use the show hosts command to display the DNS configuration (see Example 5-14).

Example 5-14 Displaying Configured Host Details

Default domain is cisco.com

Domain list: ucsc.edu harvard.edu yale.edu stanford.edu

Name/address lookup uses domain service

Default Settings for Distributed Name Server Features

Table 5-1 lists the default settings for DNS features.

Table 5-2 lists the default settings for VRRP features.