Catalyst 6500 Release 12.2SX Software Configuration Guide

NetFlow Overview

The NetFlow feature collects traffic statistics about the packets that flow through the switch and stores the statistics in the NetFlow table. The NetFlow table on the route processor (RP) captures statistics for flows routed in software and the NetFlow table on the PFC (and on each DFC) captures statistics for flows routed in hardware.

Several features use the NetFlow table. Features such as network address translation (NAT) use NetFlow to modify the forwarding result; other features (such as QoS microflow policing) use the statistics from the NetFlow table to apply QoS policies. The NetFlow Data Export (NDE) feature provides the ability to export the statistics to an external device (called a NetFlow collector).

In PFC3A mode, NetFlow collects statistics only for routed traffic. With other PFCs, you can configure NetFlow to collect statistics for both routed and bridged traffic.

Collecting and exporting a large volume of statistics can significantly impact switch processor (SP) and route processor (RP) CPU usage, so NetFlow provides configuration options to control the volume of statistics. These options include the following:

• NetFlow flow masks determine the granularity of the flows to be measured. Very specific flow masks generate a large number of NetFlow table entries and a large volume of statistics to export. Less specific flow masks aggregate the traffic statistics into fewer NetFlow table entries and generate a lower volume of statistics.
• Per-interface NetFlow allows you to enable or disable NetFlow data collection on Layer 3 interfaces.
• NetFlow Flow Sampling exports data for a subset of traffic in a flow, which can greatly reduce the volume of statistics exported. NetFlow Flow Sampling does not reduce the volume of statistics collected.
• NetFlow aggregation merges the collected statistics prior to export. Aggregation reduces the volume of records exported, but does not reduce the volume of statistics collected. NetFlow aggregation increases SP CPU utilization and reduces the data available at the collector. NetFlow aggregation uses NetFlow version 8.

NetFlow defines three configurable timers to identify stale flows that can be deleted from the table. NetFlow deletes the stale entries to clear table space for new entries.

NetFlow on the PFC

The NetFlow table on the PFC captures statistics for flows routed in hardware. A flow is a unidirectional stream of packets between a source and a destination. The flow mask specifies the fields in the incoming packet that NetFlow uses to match (or create) a NetFlow table entry.

All flow masks include the ingress interface in their definition. Therefore, NetFlow always collects statistics on a per-interface basis. You can also enable or disable NetFlow per-interface.

The PFC supports the following flow masks:

• interface-source—A less-specific flow mask. Statistics for all ingress flows on an interface from each source IP address aggregate into one entry.
• interface-destination—A less-specific flow mask. Statistics for all ingress flows on an interface to each destination IP address aggregate into one entry.
• interface-destination-source—A more-specific flow mask. Statistics for all ingress flows on an interface between the same source IP address and destination IP address aggregate into one entry.
• interface-full—The most-specific flow mask. The PFC creates and maintains a separate table entry for each IP flow on an interface. An interface-full entry includes the source IP address, destination IP address, protocol, and protocol ports.

The flow mask determines the granularity of the statistics gathered, which controls the size of the NetFlow table. The less-specific flow masks result in fewer entries in the NetFlow table and the most-specific flow masks result in the most NetFlow entries.

For example, if the flow mask is set to interface-source, the NetFlow table contains one entry per source IP address. (Assume that NetFlow is enabled on only one interface). The statistics for all flows from each source are accumulated in the one entry. However, if the flow mask is configured as interface-full, the NetFlow table contains one entry per full flow. Many entries may exist per source IP address, so the NetFlow table can become very large. See the “NetFlow Restrictions” section for information about NetFlow table capacity.

NetFlow on the RP

The NetFlow feature on the RP captures statistics for flows routed in software.

For additional information about configuring NetFlow on the RP, see the Cisco IOS NetFlow Configuration Guide, Release 12.2SX.

NetFlow Features

NetFlow supports the following features:

• Per Interface NetFlow
• NetFlow Aggregation
• NetFlow for Multicast IP

Per Interface NetFlow

Cisco IOS Release 12.2(33)SXH and later releases support per-interface NetFlow, which enables PFC NetFlow data collection on a per-interface basis. With releases earlier than Release 12.2(33)SXH, NetFlow on the PFC could be only be enabled and disabled globally.

When you upgrade to a software release that supports the per-interface NetFlow feature, the system automatically enables per-interface NetFlow and configures the ip flow ingress command on every Layer 3 interface. This one-time action takes place on the first reload after the upgrade and maintains backward compatibility with the global NetFlow enable command. After the reload, you can configure the no ip flow ingress command on Layer 3 interfaces to selectively disable PFC and RP NetFlow data collection.

The per-interface NetFlow feature only applies to IPv4 unicast flows on Layer 3 interfaces. Flows for non-IPv4 protocols (such as IPv6 and MPLS) are not controlled by this feature.

NetFlow Aggregation

NetFlow supports aggregation for packets forwarded in hardware (PFC) or software (RP). See the Cisco IOS NetFlow Configuration Guide, Release 12.2SX for information about these features:

• NetFlow aggregation schemes
• Configuring NetFlow aggregation
• ToS-based router aggregation, which is supported by NetFlow on the RP

NetFlow for Multicast IP

NetFlow is supported for multicast IP packets forwarded in hardware (PFC) or software (RP).

NetFlow multicast provides ingress accounting and egress accounting. With ingress accounting, NetFlow creates one flow per source and includes information about how many packet replications occur. With egress accounting, NetFlow creates one flow for each outgoing interface.

Optionally, NetFlow multicast keeps statistics for multicast packets that fail the reverse path fail (RPF) check.

Note Disabling the mls netflow command globally will cause non-RPF multicast traffic to be dropped in software, as new non-RPF Netflow entries will not be created.

General NetFlow Restrictions

• The CEF table (rather than the NetFlow table) implements Layer 3 switching in hardware.
• Except in PFC3A mode, NetFlow supports bridged IP traffic. PFC3A mode does not support NetFlow bridged IP traffic.
• NetFlow supports multicast IP traffic.
• Supervisor Engine 720 and earlier hardware do not support egress Netflow accounting for unicast traffic. It is supported only for multicast traffic. However, Supervisor Engine 2T supports ingress and egress Netflow accounting for unicast traffic.
• In PFC3A mode, NAT requires a non-interface-full flow mask for fragmented packets because the PFC cannot provide hardware acceleration for them. Fragmented NAT traffic is sent to the RP to be processed in software, which requires additional fragment ACE entries in the ACL TCAM.

Other PFC3 modes support the mls ip nat netflow-frag-l4-zero command, which removes the specific flow mask requirement and resolves NetFlow mask conflicts between NDE and NAT features. With the mls ip nat netflow-frag-l4-zero command configured, the PFC clears the initial fragmented packet L4 information before it gets NAT is applied.

• By default, NAT overload processes initial fragments in software on the RP because NAT for subsequent fragments depends on the Layer 4 information in the first fragment. To ensure that initial fragments do not get switched in hardware, two ACL entries that require a flowmask different from the one for NAT NetFlow are added to the NAT inside interface. Initial fragments hit one of the fragment ACL entries on the NAT inside interface and because it uses a different flowmask, they do not hit the NetFlow shortcut and so are not hardware switched. The two additional ACL entries added to the NAT inside interface could lead to a merge blowup if a big ACL is configured on the NAT inside interface.

Except in PFC3A mode, you can configure the mls ip nat netflow-frag-l4-zero command to zero out the Layer 4 port information from the NetFlow lookup key for fragmented packets, which are then correctly sent to the RP for processing. In Layer 4 zero mode, fragmented packets (including initial fragments) do not match the NetFlow entries that have non-zero Layer 4 port information. In this mode, the 2 additional fragment entries for NAT are not required.

This can alleviate possible merge failures if a big ACL is configured on the NAT inside interface, and avoids flowmask conflicts between NAT and other features like NDE that arise due to the NAT requirement for a non-interface-full flowmask for fragment entries.

Note In this mode, fragmented packets are not counted correctly if NDE uses the full or interface-full flowmask. Similarly, initial fragments are not counted against the correct bucket with microflow policing that uses the full-flow mask.

• No statistics are available for flows that are switched when the NetFlow table is full.
• If the NetFlow table utilization exceeds the recommended utilization levels, there is an increased probability that there will be insufficient room to store statistics. Table 63-2 lists the recommended maximum utilization levels.

Flow Mask Conflicts

Several features use the NetFlow table. Table 63-3 lists the flow mask requirements for each feature.

NetFlow data export, hardware accelerated NetFlow features, and microflow policers are the three categories of features that request flowmasks. All of them can be configured at the same time, but depending on the flowmask that each requests, the configuration might be allowed or not. Typically, only one of the hardware accelerated NetFlow features is configured on an interface. If multiple hardware accelerated NetFlow features are configured, some of them might not be hardware accelerated if the flow is subject to more than one of these features.

Each of these features request a flowmask. The final flowmask depends on the other features.

Example Feature Configurations

Note ● “Min” indicates that the flowmask requirement is flexible: a more granular flowmask will also work. For example, “interface-source min” indicates that interface-source-destination can also be used.

• “Exact” indicates that the flowmask requirement is not flexible.
• You can use the information in Table 63-3 to check for conflicts between features.

NAT “Non-interface Full” would conflict with NDE “Interface Full”, but NAT “Interface Full” would not conflict with NDE “Interface Full”:

NAT “Interface Full” would not conflict with NDE “Interface Full”:

NAT “Interface Full” would not conflict with NDE “Interface Source”:

WCCP would not conflict with NDE “Interface Destination Source”:

WCCP would not conflict with NDE “Interface Full”:

NDE “Interface Full” would conflict with microflow policing “Destination”:

NDE “Interface Full” would not conflict with microflow policing “Interface Full”:

WCCP, NAT “Interface Full”, and NDE “Interface Full” would not conflict:

Configuring NetFlow on the PFC

These sections describe how to configure NetFlow statistics collection on the PFC:

• NetFlow PFC Commands Summary
• Enabling NetFlow on the PFC
• Setting the Minimum IP MLS Flow Mask
• Configuring the MLS Aging Time
• Displaying PFC NetFlow Information

NetFlow PFC Commands Summary

Table 63-4 shows a summary of the NetFlow commands available on the PFC.

Enabling NetFlow on the PFC

To enable NetFlow statistics collection globally on the PFC, perform this task:

This example shows how to disable NetFlow statistics collection on the PFC (the default setting is enabled):

Setting the Minimum IP MLS Flow Mask

You can set the minimum specificity of the flow mask for the NetFlow table on the PFC. The actual flow mask may be more specific than the level configured in the mls flow command, if other configured features need a more specific flow mask (see the “Flow Mask Conflicts” section).

To set the minimum IPv4 flow mask, perform this task:

This example shows how to set the minimum flow mask:

To display the IP MLS flow mask configuration, perform this task:

This example shows how to display the MLS flow mask configuration:

Configuring the MLS Aging Time

The MLS aging time (default 300 seconds) applies to all NetFlow table entries. You can configure the normal aging time in the range of 32 to 4092 seconds. Flows can age as much as 4 seconds sooner or later than the configured interval. On average, flows age within 2 seconds of the configured value.

Other events might cause MLS entries to be purged, such as routing changes or a change in link state.

Note If the number of MLS entries exceeds the recommended utilization (see the “NetFlow Restrictions” section), only adjacency statistics might be available for some flows.

To keep the NetFlow table size below the recommended utilization, enable the following parameters when using the mls aging command:

• normal—Configures an inactivity timer. If no packets are received on a flow within the duration of the timer, the flow entry is deleted from the table.
• fast aging—Configures an efficient process to age out entries created for flows that only switch a few packets, and then are never used again. The fast aging parameter uses the time keyword value to check if at least the threshold keyword value of packets have been switched for each flow. If a flow has not switched the threshold number of packets during the time interval, then the entry is aged out.
• long—Configures entries for deletion that have been active for the specified value even if the entry is still in use. Long aging is used to prevent counter wraparound, which can cause inaccurate statistics.

A typical table entry that is removed by fast aging is the entry for flows to and from a Domain Name Server (DNS) or TFTP server.

If you need to enable MLS fast aging time, initially set the value to 128 seconds. If the size of the NetFlow table continues to grow over the recommended utilization, decrease the setting until the table size stays below the recommended utilization. If the table continues to grow over the recommended utilization, decrease the normal MLS aging time.

To configure the MLS aging time, perform this task:

This example displays how to configure the MLS aging time:

To display the MLS aging-time configuration, perform this task:

This example shows how to display the MLS aging-time configuration:

Configuring Exclude ACL-deny

By default, NetFlow table entries are created for ACL-denied flows. These flows can cause the NetFlow table to overflow. With Release 12.2(33)SXH and later releases, to exclude ACL-denied flows from the NetFlow table, perform this task:

This example shows how to exclude ACL-denied flows from the NetFlow table:

Displaying PFC NetFlow Information

To display information about NetFlow on the PFC, perform this task:

Configuring NetFlow Features

NetFlow features generally apply to packets forwarded in hardware (PFC) and software (RP). For the features to apply to PFC, you need to enable NetFlow on the PFC.

These sections describe how to configure NetFlow features:

• Configuring NetFlow on Layer 3 Interfaces
• Enabling NetFlow for Ingress-Bridged IP Traffic
• Configuring NetFlow Aggregation
• Configuring NetFlow for Multicast IP Traffic

Configuring NetFlow on Layer 3 Interfaces

The per-interface NDE feature allows you to enable or disable NetFlow collection on a per-interface basis for packets forwarded in hardware (PFC) or software (RP). This feature is automatically enabled in Release 12.2(33)SXH and later releases.

To enable or disable NetFlow for a Layer 3 interface, perform this task:

When you upgrade for the first time to a software image that supports per-interface NetFlow on the PFC, the system automatically configures each Layer 3 interface to enable NetFlow (this ensures backward compatibility with the global mls netflow command). This one-time action occurs during the first system restart after the upgrade. After this action, you can configure Layer 3 interfaces to disable or enable NetFlow data collection.

Enabling NetFlow for Ingress-Bridged IP Traffic

Except in PFC3A mode, NetFlow supports ingress-bridged IP traffic. PFC3A mode does not support NetFlow for bridged IP traffic.

Note ● When you enable NetFlow for ingress-bridged IP traffic, the statistics are available to the NetFlow Flow Sampling feature (see the “NetFlow Overview” section).

• To enable NetFlow for bridged IP traffic on a VLAN, you must create a corresponding VLAN interface and enter the no shutdown command. The no shutdown command can be followed, if necessary, by the shutdown command.
• For Layer 3 VLANs, enabling NetFlow for ingress-bridged IP traffic also enables NetFlow for Layer 3 flows on the specified VLANs.
• The exported bridged flows will have ingress and egress VLAN information and not the physical port information.

To enable NetFlow for ingress-bridged IP traffic in VLANs, perform this task:

This example shows how to enable NetFlow for ingress-bridged IP traffic in VLAN 200:

Configuring NetFlow Aggregation

To configure NetFlow aggregation, use the procedures in the Cisco IOS NetFlow Configuration Guide, Release 12.2SX.

Note ● When you configure NetFlow aggregation, it is configured automatically for packets forwarded in hardware (PFC) or software (RP).

• The PFC and DFCs do not support NetFlow ToS-based router aggregation.

To display NetFlow Aggregation information for the PFC or DFCs, perform this task:

Note The PFC and DFCs do not support NetFlow ToS-based router Aggregation.

This example shows how to display the NetFlow Aggregation cache information:

This example shows how to display the NetFlow Aggregation flow mask information:

Configuring NetFlow for Multicast IP Traffic

To configure NetFlow for multicast IP traffic, perform this task:

For additional information about configuring NetFlow for multicast traffic, see the “ Configuring NetFlow Multicast Accounting ” section of the Cisco IOS NetFlow Configuration Guide, Release 12.2SX.

The “Configuring NetFlow Multicast Accounting” section specifies as a prerequisite that you need to configure multicast fast switching or multicast distributed fast switching (MDFS), but this prerequisite does not apply when configuring NetFlow multicast support with 12.2SX releases.

Note The Configuring NetFlow Multicast Accounting document describes new configuration commands for Cisco IOS Release 12.2(4) and newer releases. The 12.2SX releases support the new commands.

http://www.cisco.com/en/US/products/hw/switches/ps708/tsd_products_support_series_home.html

Participate in the Technical Documentation Ideas forum