Chapter 4. NetFlow for Cybersecurity

The chapter covers the following exam topics:

Introduction to NetFlow

NetFlow versions

Configuring NetFlow

Understanding IPFIX

NetFlow for cybersecurity and incident response

NetFlow analysis tools

This chapter starts with an introduction to NetFlow and then covers details about all the different NetFlow versions. In this chapter, you will learn how to configure basic NetFlow in a Cisco device. You will also learn about the industry standard IPFIX as well as how NetFlow is used for cybersecurity and incident response. This chapter also covers examples of commercial and open source NetFlow analysis tools.

“Do I Know This Already?” Quiz

The “Do I Know This Already?” quiz helps you identify your strengths and deficiencies in this chapter’s topics. The 10-question quiz, derived from the major sections in the “Foundation Topics” portion of the chapter, helps you determine how to spend your limited study time. Table 4-1 outlines the major topics discussed in this chapter and the “Do I Know This Already?” quiz questions that correspond to those topics.

Table 4-1 “Do I Know This Already?” Foundation Topics Section-to-Question Mapping

Foundation Topics Section	Questions Covered in This Section
Introduction to NetFlow	1–3
NetFlow Versions	4–5
IPFIX	6
NetFlow for Cybersecurity and Incident Response	7–8
NetFlow Analysis Tools	9–10

1. Which of the following are some common uses of NetFlow? (Choose three.)

a. To see what is actually happening across the entire network

b. To identify DoS attacks

c. To quickly identify compromised endpoints and network infrastructure devices

d. To perform network scans to detect vulnerabilities

2. Flexible NetFlow, Cisco’s next-generation NetFlow, can track a wide range of Layer 2, IPv4, and IPv6 flow information. Which of the following are examples of that information? (Choose four.)

a. Source and destination IPv4 or IPv6 addresses

b. Source and destination ports

c. Packet and byte counts

d. Flow timestamps

e. Usernames

f. Application ID

3. NetFlow supports different types of cache. Which of the following are the NetFlow cache types? (Choose three.)

a. Normal

b. Flexible

c. Immediate

d. Permanent

4. IPFIX is a flow standard based on what version of NetFlow?

a. Version 1

b. Version 5

c. Version 7

d. Version 9

5. What is one of the benefits of NetFlow templates?

a. Templates make flow records more organized and better structured.

b. Templates provide a vendor-neutral support for companies that create applications that provide collector or analysis capabilities for NetFlow so that they are not required to reinvent their product each time a new NetFlow feature is added.

c. Templates provide a faster way of processing NetFlow records.

d. Templates can be used to detect zero-day attacks faster because they provide support for indicators of compromise.

6. What protocol is used by IPFIX for packet transport?

a. SNMP

b. HTTPS

c. SCTP

d. TLS

7. NetFlow is a great tool for anomaly and DDoS detection. Before implementing these detection capabilities, you should perform which of the following tasks?

a. Enable NetFlow in more than two interfaces.

b. Enable BGP for route redirection.

c. Develop a traffic baseline.

d. Enable anti-spoofing protection.

8. Many network telemetry sources can also be correlated with NetFlow when responding to security incidents and performing network forensics. Which of the following are examples of other telemetry sources that can be correlated with NetFlow? (Choose two.)

a. Dynamic Host Configuration Protocol (DHCP) logs

b. VPN logs

c. Core dumps

d. Process utilization and hardware inventory logs

9. Which of the following are examples of open source tools that can be used for NetFlow analysis? (Choose three.)

a. SiLK

b. Elasticsearch, Logstash, Kibana (ELK)

c. Lancope

d. Graylog

10. Which of the following are components of the Cisco Lancope StealthWatch solution?

a. StealthWatch Management Console

b. FlowCollector

c. FlowConnector

d. ISE Connector

Foundation Topics

Introduction to NetFlow

NetFlow is a Cisco technology that provides comprehensive visibility into all network traffic that traverses a Cisco-supported device. Cisco invented NetFlow and is the leader in IP traffic flow technology. NetFlow was initially created for billing and accounting of network traffic and to measure other IP traffic characteristics such as bandwidth utilization and application performance. NetFlow has also been used as a network-capacity planning tool and to monitor network availability. NetFlow is used by many cybersecurity professionals as a network security tool because its reporting capabilities provide nonrepudiation, anomaly detection, and investigative capabilities. As network traffic traverses a NetFlow-enabled device, the device collects traffic flow information and provides a network administrator or security professional with detailed information about such flows.

NetFlow provides detailed network telemetry that allows the administrator to do the following:

See what is actually happening across the entire network.

Identify DoS attacks.

Quickly identify compromised endpoints and network infrastructure devices.

Monitor network usage of employees, contractors, or partners.

Obtain network telemetry during security incident response and forensics.

Detect firewall misconfigurations and inappropriate access to corporate resources.

NetFlow supports both IP version 4 (IPv4) and IP version 6 (IPv6), and it plays a crucial role in the following:

Network planning

Network security

Network troubleshooting

Traffic engineering

---

TIP

Do not confuse the feature in Cisco IOS software called IP Accounting with NetFlow. IP Accounting is a great Cisco IOS tool, but it is not as robust or as well known as NetFlow.

---

What Is a Flow in NetFlow?

A flow is a unidirectional series of packets between a given source and destination. In a flow, the same source and destination IP addresses, source and destination ports, and IP protocol are shared. This is often referred to as the 5-tuple. Figure 4-1 shows an example of a flow between a client and a server.

A laptop at the left named “Client (Source)” with IP Address “192.168.10.1” is connected to a router at the center. The router is connected to a network cloud at the right named “example.org (Destination)” with IP Address “93.184.216.34.” Above the connection between the source and destination, a dashed line labeled “HTTP” is displayed. The router is connected to a server at the bottom named “NetFlow Collector.” Parallel to the connection, a dashed downward arrow labeled “Flow Record” is shown.

Figure 4-1 Basic NetFlow Example

In Figure 4-1, the client (source) establishes a connection to the server (destination). When the traffic traverses the router (configured for NetFlow), it generates a flow record. At the very minimum, the 5-tuple is used to identify the flow in the NetFlow database of flows kept on the device. This database is often called the NetFlow cache.

Table 4-2 shows the 5-tuple for the basic flow represented in Figure 4-1.

Table 4-2 NetFlow 5-Tuple

Flow Record Field	Value
Source IP address	192.168.10.1
Destination IP address	93.184.216.34
Source port	17238
Destination port	80
Protocol	TCP

Depending on the version of NetFlow, the router can also gather additional information, such as type of service (ToS) byte, differentiated services code point (DSCP), the device’s input interface, TCP flags, byte counters, and start and end times.

Flexible NetFlow, Cisco’s next-generation NetFlow, can track a wide range of Layer 2, IPv4, and IPv6 flow information, such as the following:

Source and destination MAC addresses

Source and destination IPv4 or IPv6 addresses

Source and destination ports

ToS

DSCP

Packet and byte counts

Flow timestamps

Input and output interface numbers

TCP flags and encapsulated protocol (TCP/UDP) and individual TCP flags

Sections of a packet for deep packet inspection

All fields in an IPv4 header, including IP-ID and TTL

All fields in an IPv6 header, including Flow Label and Option Header

Routing information, such as next-hop address, source autonomous system number (ASN), destination ASN, source prefix mask, destination prefix mask, Border Gateway Protocol (BGP) next hop, and BGP policy accounting traffic index

NetFlow protocol data units (PDUs), also referred to as flow records, are generated and sent to a NetFlow collector after the flow concludes or expires (times out).

The NetFlow Cache

There are three types of NetFlow cache:

Normal cache: This is the default cache type in many infrastructure devices enabled with NetFlow and Flexible NetFlow. The entries in the flow cache are removed (aged out) based on the configured timeout active seconds and timeout inactive seconds settings.

Immediate cache:

Flow accounts for a single packet

Desirable for real-time traffic monitoring and distributed DoS (DDoS) detection

Used when only very small flows are expected (for example, sampling)

---

NOTE

The immediate cache may result in a large amount of export data.

---

Permanent cache:

Used to track a set of flows without expiring the flows from the cache.

The entire cache is periodically exported (update timer).

The cache is a configurable value.

After the cache is full, new flows will not be monitored.

Uses update counters rather than delta counters.

Many people often confuse a flow with a session. All traffic in a flow is unidirectional; however, when the client establishes the HTTP connection (session) to the server and accesses a web page, this represents two separate flows. The first flow is the traffic from the client to the server, and the other is the return flow from the server to the client.

NetFlow Versions

There are several versions of NetFlow. Table 4-3 lists all versions of NetFlow and provides a brief description of the features supported.

Table 4-3 NetFlow Versions

NetFlow Version	Description
Version 1 (v1)	(Obsolete.) The first implementation of NetFlow. NetFlow v1 was limited to IPv4 without IP network masks and autonomous system numbers (ASNs).
Version 2 (v2)	Never released.
Version 3 (v3)	Never released.
Version 4 (v4)	Never released.
Version 5 (v5)	Popular NetFlow version on many routers from different vendors. Limited to IPv4 flows.
Version 6 (v6)	(Obsolete.) No longer supported by Cisco.
Version 7 (v7)	(Obsolete.) Like version 5, with a source router field.
Version 8 (v8)	(Obsolete.) Several aggregation forms, but only for information that is already present in v5 records.
Version 9 (v9)	Template based, available (as of 2009) on some recent routers. Mostly used to report flows such as IPv6, Multiprotocol Label Switching (MPLS), and even plain IPv4 with Border Gateway Protocol (BGP) next hop.
IPFIX	IPFIX is an IETF standard based on NetFlow v9 with several extensions.

Table 4-4 lists the NetFlow v1 flow header format, and Table 4-5 lists the attributes of the NetFlow v1 flow record format.

Table 4-4 NetFlow v1 Flow Header Format

Bytes	Contents	Description
0–1	version	NetFlow export format version number
2–3	count	Number of flows exported in this packet (1–24)
4–7	sys_uptime	Current time in milliseconds since the export device booted
8–11	unix_secs	Current count of seconds since 0000 UTC 1970
12–16	unix_nsecs	Residual nanoseconds since 0000 UTC 1970

Table 4-5 NetFlow v1 Flow Record Format

Bytes	Contents	Description
0–3	srcaddr	Source IP address
4–7	dstaddr	Destination IP address
8–11	nexthop	IP address of next-hop router
12–13	input	SNMP index of input interface
14–15	output	SNMP index of output interface
16–19	dPkts	Packets in the flow
20–23	dOctets	Total number of Layer 3 bytes in the packets of the flow
24–27	first	SysUptime at start of flow
28–31	last	SysUptime at the time the last packet of the flow was received
32–33	srcport	TCP/UDP source port number or equivalent
34–35	dstport	TCP/UDP destination port number or equivalent
36–37	pad1	Unused (0) bytes
38	prot	IP protocol type (for example, TCP = 6; UDP = 17)
39	tos	IP type of service (ToS)
40	flags	Cumulative OR of TCP flags
41-48	pad2	Unused (0) bytes

Table 4-6 lists the NetFlow v5 flow header format, and Table 4-7 lists the attributes of the NetFlow v5 flow record format.

Table 4-6 NetFlow v5 Flow Header Format

Bytes	Contents	Description
0–1	version	NetFlow export format version number.
2–3	count	Number of flows exported in this packet (1–30).
4–7	sys_uptime	Current time in milliseconds since the export device booted.
8–11	unix_secs	Current count of seconds since 0000 UTC 1970.
12–15	unix_nsecs	Residual nanoseconds since 0000 UTC 1970.
16-19	flow_sequence	Sequence counter of total flows seen.
20	engine_type	Type of flow-switching engine.
21	engine_id	Slot number of the flow-switching engine.
22–23	sampling_interval	First 2 bits hold the sampling mode; remaining 14 bits hold value of sampling interval.

Table 4-7 NetFlow v5 Flow Record Format

Bytes	Contents	Description
0–3	srcaddr	Source IP address
4–7	dstaddr	Destination IP address
8–11	nexthop	IP address of next-hop router
12–13	input	Simple Network Management Protocol (SNMP) index of input interface
14–15	output	SNMP index of output interface
16–19	dPkts	Packets in the flow
20–23	dOctets	Total number of Layer 3 bytes in the packets of the flow
24–27	first	SysUptime at start of flow
28–31	last	SysUptime at the time the last packet of the flow was received
32–33	srcport	TCP/UDP source port number or equivalent
34–35	dstport	TCP/UDP destination port number or equivalent
36	pad1	Unused (0) bytes
37	tcp_flags	Cumulative OR of TCP flags
38	prot	IP protocol type (for example, TCP = 6; UDP = 17)
39	tos	IP type of service (ToS)
40–41	src_as	Autonomous system number (ASN) of the source, either origin or peer
42–43	dst_as	ASN of the destination, either origin or peer
44	src_mask	Source address prefix mask bits
45	dst_mask	Destination address prefix mask bits
46–47	pad2	Unused (0) bytes

Table 4-8 lists the NetFlow v7 flow header format, and Table 4-9 lists the attributes of the NetFlow v7 flow record format.

Table 4-8 NetFlow v7 Flow Header Format

Bytes	Contents	Description
0–1	version	NetFlow export format version number
2–3	count	Number of flows exported in this packet (1–30)
4–7	sys_uptime	Current time in milliseconds since the export device booted
8–11	unix_secs	Current count of seconds since 0000 UTC 1970
12–15	unix_nsecs	Residual nanoseconds since 0000 UTC 1970
16–19	flow_sequence	Sequence counter of total flows seen
20–23	reserved	Unused (0) bytes

Table 4-9 NetFlow v7 Flow Record Format

Bytes	Contents	Description
0–3	srcaddr	Source IP address.
4–7	dstaddr	Destination IP address.
8–11	nexthop	IP address of next-hop router.
12–13	input	SNMP index of input interface.
14–15	output	SNMP index of output interface.
16–19	dPkts	Packets in the flow.
20–23	dOctets	Total number of Layer 3 bytes in the packets of the flow.
24–27	first	SysUptime at start of flow.
28–31	last	SysUptime at the time the last packet of the flow was received.
32–33	srcport	TCP/UDP source port number or equivalent.
34–35	dstport	TCP/UDP destination port number or equivalent.
36	pad1	Unused (0) bytes.
37	tcp_flags	Cumulative OR of TCP flags.
38	prot	IP protocol type (for example, TCP = 6; UDP = 17).
39	tos	IP type of service (ToS).
40–41	src_as	ASN of the source, either origin or peer.
42–43	dst_as	ASN of the destination, either origin or peer.
44	src_mask	Source address prefix mask bits.
45	dst_mask	Destination address prefix mask bits.
46–47	flags	Flags indicating, among other things, what flows are invalid.
48–51	router_sc	IP address of the router that is bypassed by the Catalyst 5000 series switch. (This is the same address the router uses when it sends NetFlow export packets. This IP address is propagated to all switches bypassing the router through the Fibre Channel Protocol [FCP].)

The most popular version of NetFlow is Version 9. The NetFlow v9 format is template based. Templates provide a flexible design to the record format. This feature allows for future enhancements to NetFlow services without requiring fundamental changes to the underlying flow record format.

The following are the benefits of using NetFlow templates:

They provide vendor-neutral support for companies that create applications that provide collector or analysis capabilities for NetFlow so that they are not required to reinvent their product each time a new NetFlow feature is added.

New features can be added to NetFlow more quickly, without breaking current implementations and with backward compatibility.

The NetFlow v9 record format consists of a packet header followed by at least one or more template or data FlowSets. A template FlowSet provides a description of the fields that will be present in future data FlowSets. These data FlowSets may occur later within the same export packet or in subsequent export packets. Figure 4-2 shows a basic illustration of the NetFlow v9 export packet.

In the box, there are 7 sections that read: Packet Header, Template FlowSet, Data FlowSet, Data FlowSet, ........, Template FlowSet and Data FlowSet.

Figure 4-2 NetFlow v9 Export Packet

Figure 4-3 shows a more detailed illustration of the NetFlow v9 export packet and the relationship between each field and its attributes.

In the box, there are 4 sections. The first section reads: HEADER. In the second section titled “Template FlowSet,” there are 2 narrow boxes. The first box reads: Template Record Template ID #1 (Specific Field Types and Lengths). The second box reads: Template Record Template ID #2 (Specific Field Types and Lengths). In the next section titled “Data FlowSet FlowSet ID #1” and labeled “Template 1,” there are 2 boxes. Both the boxes read: Data Record (Field Values). In the last section titled “Data FlowSet FlowSet ID #2” and labeled “Template 2,” there is one box that reads: Data Record (Field Values). The following text are in bold: Template ID #1, FlowSet ID #1, and Template 1. The following text are in different color: Template ID #2, FlowSet ID #2, and Template 2. The following are connected by double headed arrows that bend downwards from the box, run parallel to the box and then bend upwards to the box: first box of second section to first box of third section, first box of second section to second box of second section, second box of second section to the box in last section.

Figure 4-3 NetFlow v9 Export Packet Details

The format of the NetFlow v9 packet header is very similar to its predecessors and is illustrated in Figure 4-4.

In the box, there are 6 layers. In the first layer, there are 32 fields that are filled with numbers from 0 to 32. In the second layer, there are 2 fields: Version and Count. The next layers read: System Uptime, UNIX Seconds, Package Sequence, and Source ID.

Figure 4-4 NetFlow v9 Packet Header Format

Table 4-10 lists the NetFlow v9 packet header field descriptions.

Table 4-10 NetFlow v9 Packet Header Field Descriptions

Field Name	Value
Version	The version of NetFlow records exported in this packet. The hexadecimal value 0x0009 represents NetFlow v9.
Count	Number of FlowSet records (both template and data) contained within the export packet.
System Uptime	Time in milliseconds since the device started.
UNIX Seconds	Seconds since 0000 coordinated universal time (UTC) 1970.
Sequence Number	Incremental sequence counter of all export packets sent by this export device; this value is cumulative, and it can be used to identify whether any export packets have been missed. Note: This is a change from the NetFlow v5 and v8 headers, where this number represented “total flows.”
Source ID	A 32-bit value that is used to ensure uniqueness for all flows exported from a particular NetFlow-enabled device. The Source ID field is the equivalent of the Engine Type and Engine ID fields found in the NetFlow v5 and v8 headers. The format of this field is vendor specific. In Cisco’s implementation, the first 2 bytes are reserved for future expansion and will always be 0. Byte 3 provides uniqueness with respect to the routing engine on the exporting device. Byte 4 provides uniqueness with respect to the particular line card or Versatile Interface Processor on the exporting device. NetFlow collectors should use the combination of the source IP address plus the Source ID field to associate an incoming NetFlow export packet with a unique instance of NetFlow on a particular device.

As previously mentioned, templates are one of the main benefits of NetFlow v9 because they provide flexibility to allow a NetFlow collector or display application to process NetFlow data without necessarily knowing the format of the data in advance.

Figure 4-5 shows the format of the NetFlow v9 template FlowSet.

In the box, there are 21 layers. In the first layer, there are 16 fields that are filled with numbers from 0 to 15. The next layers read: Flowset_Id = 0, Length, Template_Id, Field_Count, Field_1_Type, Field_1_Length, Field_2_Type, Field_2_Length, Field_3_Type, Field_3_Length, …, Field_N_Type, Field_N_Length, Template_Id, Field_Count, Field_1_Type, Field_1_Length, …, Field_N_Type, and Field_N_Length.

Figure 4-5 NetFlow v9 Template FlowSet Format

Table 4-11 lists the NetFlow v9 template FlowSet field descriptions.

Table 4-11 NetFlow v9 Template FlowSet Field Descriptions

Field	Description
flowset_id	The flowset_id field is used to distinguish template records from data records. A template record always has a flowset_id in the range of 0 to 255. Currently, the template record that describes flow fields has a flowset_id of 0, and the template record that describes option fields (described later) has a flowset_id of 1. A data record always has a nonzero flowset_id greater than 255.
length	Length refers to the total length of this FlowSet. Because an individual template FlowSet may contain multiple template IDs (as illustrated earlier), the length value should be used to determine the position of the next FlowSet record, which could be either a template or a data FlowSet. Length is expressed in type/length/value (TLV) format, meaning that the value includes the bytes used for the flowset_id and the length of bytes themselves, in addition to the combined lengths of all template records included in this FlowSet.
template_id	As a router generates different template FlowSets to match the type of NetFlow data it will be exporting, each template is given a unique ID. This uniqueness is local to the router that generated the template_id. Templates that define data record formats begin numbering at 256 because 0 through 255 are reserved for FlowSet IDs.
field_count	This field gives the number of fields in this template record. Because a template FlowSet may contain multiple template records, this field allows the parser to determine the end of the current template record and the start of the next.
field_type	This numeric value represents the type of the field. The possible values of the field type are vendor specific. Cisco-supplied values are consistent across all platforms that support NetFlow v9. At the time of the initial release of the NetFlow v9 code (and after any subsequent changes that could add new field-type definitions), Cisco provided a file that defines the known field types and their lengths. Table 4-12 details the currently defined field types.
field_length	This number gives the length of the field_type field, in bytes.

Table 4-12 lists the NetFlow v9 field type definitions.

Table 4-12 NetFlow v9 Field Type Definitions

Field Type	Value	Length (bytes)	Description
IN_BYTES	1	N (default is 4)	Incoming counter with length N × 8 bits for number of bytes associated with an IP flow.
IN_PKTS	2	N (default is 4)	Incoming counter with length N × 8 bits for the number of packets associated with an IP flow.
FLOWS	3	N	Number of flows that were aggregated; default for N is 4.
PROTOCOL	4	1	IP protocol byte.
SRC_TOS	5	1	Type of service byte setting when entering incoming interface.
TCP_FLAGS	6	1	Cumulative of all the TCP flags seen for this flow.
L4_SRC_PORT	7	2	TCP/UDP source port number (for example, FTP, Telnet, or equivalent).
IPV4_SRC_ADDR	8	4	IPv4 source address.
SRC_MASK	9	1	The number of contiguous bits in the source address subnet mask (that is, the submask in slash notation).
INPUT_SNMP	10	N	Input interface index; default for N is 2, but higher values could be used.
L4_DST_PORT	11	2	TCP/UDP destination port number (for example, FTP, Telnet, or equivalent).
IPV4_DST_ADDR	12	4	IPv4 destination address.
DST_MASK	13	1	The number of contiguous bits in the destination address subnet mask (that is, the submask in slash notation).
OUTPUT_SNMP	14	N	Output interface index; default for N is 2, but higher values could be used.
IPV4_NEXT_HOP	15	4	IPv4 address of next-hop router.
SRC_AS	16	N (default is 2)	Source BGP ASN, where N could be 2 or 4.
DST_AS	17	N (default is 2)	Destination BGP ASN, where N could be 2 or 4.
BGP_IPV4_NEXT_HOP	18	4	Next-hop router’s IP in the BGP domain.
MUL_DST_PKTS	19	N (default is 4)	IP multicast outgoing packet counter with length N × 8 bits for packets associated with the IP flow.
MUL_DST_BYTES	20	N (default is 4)	IP multicast outgoing byte counter with length N × 8 bits for bytes associated with the IP flow.
LAST_SWITCHED	21	4	System uptime at which the last packet of this flow was switched.
FIRST_SWITCHED	22	4	System uptime at which the first packet of this flow was switched.
OUT_BYTES	23	N (default is 4)	Outgoing counter with length N × 8 bits for the number of bytes associated with an IP flow.
OUT_PKTS	24	N (default is 4)	Outgoing counter with length N × 8 bits for the number of packets associated with an IP flow.
MIN_PKT_LNGTH	25	2	Minimum IP packet length on incoming packets of the flow.
MAX_PKT_LNGTH	26	2	Maximum IP packet length on incoming packets of the flow.
IPV6_SRC_ADDR	27	16	IPv6 source address.
IPV6_DST_ADDR	28	16	IPv6 destination address.
IPV6_SRC_MASK	29	1	Length of the IPv6 source mask in contiguous bits.
IPV6_DST_MASK	30	1	Length of the IPv6 destination mask in contiguous bits.
IPV6_FLOW_LABEL	31	3	IPv6 flow label as per RFC 2460 definition.
ICMP_TYPE	32	2	Internet Control Message Protocol (ICMP) packet type; reported as ((ICMP Type * 256) + ICMP code).
MUL_IGMP_TYPE	33	1	Internet Group Management Protocol (IGMP) packet type.
SAMPLING_INTERVAL	34	4	When using sampled NetFlow, this is the rate at which packets are sampled. For example, a value of 100 indicates that 1 of every 100 packets is sampled.
SAMPLING_ALGORITHM	35	1	The type of algorithm used for sampled NetFlow: 0x01 = deterministic sampling, 0x02 = random sampling.
FLOW_ACTIVE_TIMEOUT	36	2	Timeout value (in seconds) for active flow entries in the NetFlow cache.
FLOW_INACTIVE_TIMEOUT	37	2	Timeout value (in seconds) for inactive flow entries in the NetFlow cache.
ENGINE_TYPE	38	1	Type of flow switching engine: RP = 0, VIP/Linecard = 1.
ENGINE_ID	39	1	ID number of the flow switching engine.
TOTAL_BYTES_EXP	40	N (default is 4)	Counter with length N × 8 bits for the number of bytes exported by the observation domain.
TOTAL_PKTS_EXP	41	N (default is 4)	Counter with length N × 8 bits for the number of packets exported by the observation domain.
TOTAL_FLOWS_EXP	42	N (default is 4)	Counter with length N × 8 bits for the number of flows exported by the observation domain.
Vendor proprietary	43	N/A	N/A
IPV4_SRC_PREFIX	44	4	IPv4 source address prefix (specific for Catalyst architecture).
IPV4_DST_PREFIX	45	4	IPv4 destination address prefix (specific for Catalyst architecture).
MPLS_TOP_LABEL_TYPE	46	1	MPLS top label type: 0x00 = UNKNOWN, 0x01 = TE-MIDPT, 0x02 = ATOM, 0x03 = VPN, 0x04 = BGP, 0x05 = LDP.
MPLS_TOP_LABEL_IP_ADDR	47	4	Forwarding Equivalent Class corresponding to the MPLS top label.
FLOW_SAMPLER_ID	48	1	Identifier shown in show flow-sampler.
FLOW_SAMPLER_MODE	49	1	The type of algorithm used for sampling data: 0x02 = random sampling. Used in connection with FLOW_SAMPLER_MODE.
FLOW_SAMPLER_RANDOM_INTERVAL	50	4	Packet interval at which to sample. Used in connection with FLOW_SAMPLER_MODE.
Vendor proprietary	51	N/A	N/A
MIN_TTL	52	1	Minimum Time to Live (TTL) on incoming packets of the flow.
MAX_TTL	53	1	Maximum TTL on incoming packets of the flow.
IPV4_IDENT	54	2	The IP v4 identification field.
DST_TOS	55	1	Type of service byte setting when exiting outgoing interface.
IN_SRC_MAC	56	6	Incoming source MAC address.
OUT_DST_MAC	57	6	Outgoing destination MAC address.
SRC_VLAN	58	2	Virtual LAN identifier associated with ingress interface.
DST_VLAN	59	2	Virtual LAN identifier associated with egress interface.
IP_PROTOCOL_VERSION	60	1	Internet Protocol version set to 4 for IPv4, set to 6 for IPv6. If not present in the template, Version 4 is assumed.
DIRECTION	61	1	Flow direction: 0 = ingress flow, 1 = egress flow.
IPV6_NEXT_HOP	62	16	IPv6 address of the next-hop router.
BPG_IPV6_NEXT_HOP	63	16	Next-hop router in the BGP domain.
IPV6_OPTION_HEADERS	64	4	Bit-encoded field identifying IPv6 option headers found in the flow.
Vendor proprietary	65	N/A	N/A
Vendor proprietary	66	N/A	N/A
Vendor proprietary	67	N/A	N/A
Vendor proprietary	68	N/A	N/A
Vendor proprietary	69	N/A	N/A
MPLS_LABEL_1	70	3	MPLS label at position 1 in the stack.
MPLS_LABEL_2	71	3	MPLS label at position 2 in the stack.
MPLS_LABEL_3	72	3	MPLS label at position 3 in the stack.
MPLS_LABEL_4	73	3	MPLS label at position 4 in the stack.
MPLS_LABEL_5	74	3	MPLS label at position 5 in the stack.
MPLS_LABEL_6	75	3	MPLS label at position 6 in the stack.
MPLS_LABEL_7	76	3	MPLS label at position 7 in the stack.
MPLS_LABEL_8	77	3	MPLS label at position 8 in the stack.
MPLS_LABEL_9	78	3	MPLS label at position 9 in the stack.
MPLS_LABEL_10	79	3	MPLS label at position 10 in the stack.
IN_DST_MAC	80	6	Incoming destination MAC address.
OUT_SRC_MAC	81	6	Outgoing source MAC address.
IF_NAME	82	N (default specified in template)	Shortened interface name (for example, FE1/0).
IF_DESC	83	N (default specified in template)	Full interface name (for example, FastEthernet 1/0).
SAMPLER_NAME	84	N (default specified in template)	Name of the flow sampler.
IN_PERMANENT_BYTES	85	N (default is 4)	Running byte counter for a permanent flow.
IN_PERMANENT_PKTS	86	N (default is 4)	Running packet counter for a permanent flow.
Vendor proprietary	87	N/A	N/A
FRAGMENT_OFFSET	88	2	The fragment-offset value from fragmented IP packets.
FORWARDING STATUS	89	1	Forwarding status is encoded on 1 byte, with the 2 left bits giving the status and the 6 remaining bits giving the reason code.
MPLS PAL RD	90	8 (array)	MPLS PAL route distinguisher.
MPLS PREFIX LEN	91	1	Number of consecutive bits in the MPLS prefix length.
SRC TRAFFIC INDEX	92	4	BGP policy accounting source traffic index.
DST TRAFFIC INDEX	93	4	BGP policy accounting destination traffic index.
APPLICATION DESCRIPTION	94	N	Description of the application.
APPLICATION TAG	95	1+n	Eight bits of engine ID, followed by n bits of classification.
APPLICATION NAME	96	N	Application name associated with a classification.
Not used	97	N/A	N/A
postipDiffServCodePoint	98	1	The value of a differentiated services code point (DSCP) encoded in the Differentiated Services field, after modification.
replication factor	99	4	Multicast replication factor.
Deprecated	100	N	Deprecated.
Not used	101	N/A	N/A
layer2packetSectionOffset	102		Layer 2 packet section offset.
layer2packetSectionSize	103		Layer 2 packet section size.
layer2packetSectionData	104		Layer 2 packet section data.
Reserved for future use	105 thru 127	N/A	N/A

Figure 4-6 shows the NetFlow v9 template FlowSet format.

In the box, there are 22 layers. In the first layer, there are 16 fields that are filled with numbers from 0 to 15. The next layers read: FlowSet ID = Template ID, Length, Record 1 - Field 1 Value, Record 1 - Field 2 Value, Record 1 - Field 3 Value, Record 1 – Field 4 Value, ., ., ., Record 1 – Field N Value, Record 2 - Field 1 Value, Record 2 - Field 2 Value, Record 2 - Field 3 Value, ., ., ., Record 2 – Field N Value, ., ., ., and Padding.

Figure 4-6 NetFlow v9 Template FlowSet Format

Table 4-13 lists the NetFlow v9 data FlowSet definitions.

Table 4-13 NetFlow v9 Data FlowSet Definitions

Field	Description
flowset_id	A FlowSet ID precedes each group of records within a NetFlow v9 data FlowSet. The FlowSet ID maps to a (previously received) template_id. The collector and display applications should use the flowset_id to map the appropriate type and length to any field values that follow.
length	This field gives the length of the data FlowSet. Length is expressed in TLV format, meaning that the value includes the bytes used for the flowset_id and the length bytes themselves, as well as the combined lengths of any included data records.
record_N through field_M	The remainder of the v9 data FlowSet is a collection of field values. The type and length of the fields have been previously defined in the template record referenced by the flowset_id/template_id.
padding	Padding should be inserted to align the end of the FlowSet on a 32-bit boundary. Pay attention that the length field will include those padding bits.

IPFIX is modeled after NetFlow v9. This is why many of these NetFlow v9 concepts and fields are very similar to IPFIX. Just like IPFIX, NetFlow v9 has the concept of options templates used to supply metadata about the NetFlow process itself. Figure 4-7 illustrates the format of the options template.

In the box, there are 15 layers. In the first layer, there are 16 fields that are filled with numbers from 0 to 15. The next layers read: FlowSet_Id = 1, Length, Template_Id, Option_Scope_Length, Option_Length, Scope_Field_1_Type, Scope_Field_1_Length, ..., Scope_Field_N_Length, Option_Field_1_Type, Option_Field_1_Length, .…, Option_Field_N_Length, and Padding.

Figure 4-7 NetFlow v9 Options Template Format

Table 4-14 lists the NetFlow v9 data options template definitions.

Table 4-14 NetFlow v9 Data Options Template Definitions

Field	Description
flowset_id = 1	The flowset_id is used to distinguish template records from data records. A template record always has a flowset_id of 1. A data record always has a nonzero flowset_id that is greater than 255.
length	This field gives the total length of this FlowSet. Because an individual template FlowSet may contain multiple template IDs, the length value should be used to determine the position of the next FlowSet record, which could be either a template or a data FlowSet. Length is expressed in TLV format, meaning that the value includes the bytes used for the flowset_id and the length of bytes themselves, as well as the combined lengths of all template records included in this FlowSet.
template_id	As a router generates different template FlowSets to match the type of NetFlow data it will be exporting, each template is given a unique ID. This uniqueness is local to the router that generated the template_id. The template_id is greater than 255. Template IDs less than 255 are reserved.
option_scope_length	This field gives the length in bytes of any scope fields contained in this options template.
options_length	This field gives the length (in bytes) of any Options field definitions contained in this options template.
scope_field_N_type	This field gives the relevant portion of the NetFlow process to which the options record refers. Currently defined values follow: 0x0001 = system 0x0002 = interface 0x0003 = line card 0x0004 = NetFlow cache 0x0005 = template For instance, Random Sampled NetFlow can be implemented on a per-interface basis. So, if the options record were reporting on how sampling is configured, the scope for the report would be 0x0002 (interface).
scope_field_N_length	This field gives the length (in bytes) of the Scope field, as it would appear in an options record.
option_field_N_type	This numeric value represents the type of the field that appears in the options record. Possible values are detailed in template FlowSet format.
option_field_N_length	This number is the length (in bytes) of the field, as it would appear in an options record.
padding	Padding is inserted to align the end of the FlowSet on a 32-bit boundary.

Cisco Flexible NetFlow

Flexible NetFlow provides enhanced optimization of the network infrastructure, reduces costs, and improves capacity planning and security detection beyond other flow-based technologies available today. Flexible NetFlow supports IPv6 and Network-Based Application Recognition (NBAR) 2 for IPv6 starting in Cisco IOS Software Version 15.2(1)T. It also supports IPv6 transition techniques (IPv6 inside IPv4). Flexible NetFlow can detect the following tunneling technologies that give full IPv6 connectivity for IPv6-capable hosts that are on the IPv4 Internet but that have no direct native connection to an IPv6 network:

Teredo

Intra-Site Automatic Tunnel Addressing Protocol (ISATAP)

6to4

6rd

Flexible NetFlow classification inside Teredo, ISATAP, 6to4, and 6rd was introduced in Cisco IOS Software Version 15.2(2)T. Export over IPv6 was introduced in Cisco IOS Software Version 15.2(2)T, Cisco IOS XE 3.7.0S, and Cisco Nexus Software Version 4.2.1.

Flexible NetFlow tracks different applications simultaneously. For instance, security monitoring, traffic analysis, and billing can be tracked separately, and the information customized per application.

Flexible NetFlow allows the network administrator or security professional to create multiple flow caches or information databases to track. Conventionally, NetFlow has a single cache and all applications use the same cache information. Flexible NetFlow supports the collection of specific security information in one flow cache and traffic analysis in another. Subsequently, each NetFlow cache serves a different purpose. For instance, multicast and security information can be tracked separately and the results sent to two different collectors. Figure 4-8 shows the Flexible NetFlow model and how three different monitors are used. Monitor 1 exports Flexible NetFlow data to “Exporter 1.” Monitor 2 exports Flexible NetFlow data to “Exporter 2,” and Monitor 3 exports Flexible NetFlow data to “Exporter 1” and “Exporter 3.”

In the diagram, the Interface box is shown at the top. Interface is branched into 3 components: Monitor 1, Monitor 2, and Monitor 3. Monitor 1 is branched into 2 components: Exporter 1 and Record 1. Monitor 2 is branched into: Exporter 2 and Record 2. Monitor 3 is split into: Exporter 1, Exporter 3, and Record 3.

Figure 4-8 The Flexible NetFlow Model

The following are the Flexible NetFlow components:

Records

Flow monitors

Flow exporters

Flow samplers

In Flexible NetFlow, the administrator can specify what to track, resulting in fewer flows. This helps to scale in busy networks and use fewer resources that are already taxed by other features and services.

Flexible NetFlow Records

Flexible NetFlow records are a combination of key and non-key fields. In Flexible NetFlow, records are appointed to flow monitors to define the cache that is used for storing flow data. There are seven default attributes in the IP packet identity, or “key fields,” for a flow and for a device to determine whether the packet information is unique or similar to other packets sent over the network. Fields such as TCP flags, subnet masks, packets, and number of bytes are “non-key fields.” However, they are often collected and exported in NetFlow or in IPFIX.

Flexible NetFlow Key Fields

There are several Flexible NetFlow key fields in each packet that is forwarded within a NetFlow-enabled device. The device looks for a set of IP packet attributes for the flow and determines whether the packet information is unique or similar to other packets. In Flexible NetFlow, key fields are configurable, which enables the administrator to conduct a more granular traffic analysis.

Table 4-15 lists the key fields related to the actual flow, device interface, and Layer 2 services.

Table 4-15 Flexible NetFlow Key Fields Related to Flow, Interface, and Layer 2

	Flow	Interface	Layer 2
Fields	Sampler ID Direction Class ID	Input Output	Source VLAN Destination VLAN Dot1q priority Source MAC address Destination MAC address

Table 4-16 lists the IPv4- and IPv6-related key fields.

Table 4-16 Flexible NetFlow IPv4 and IPv6 Key Fields

	IPv4	IPv6
Fields	IP (Source or Destination) Prefix (Source or Destination) Mask (Source or Destination) Minimum-Mask (Source or Destination) Protocol Fragmentation Flags Fragmentation Offset Identification Header Length Total Length Payload Size Packet Section (Header) Packet Section (Payload) Time to Live (TTL) Options bitmap Version Precedence DSCP TOS	IP (Source or Destination) Prefix (Source or Destination) Mask (Source or Destination) Minimum-Mask (Source or Destination) Protocol Traffic Class Flow Label Option Header Header Length Payload Length Payload Size Packet Section (Header) Packet Section (Payload) DSCP Extension Headers Hop-Limit Length Next-header Version

Table 4-17 lists the Layer 3 routing protocol–related key fields.

Table 4-17 Flexible NetFlow Layer 3 Routing Protocol Key Fields

	Routing
Fields	Source or Destination AS Peer AS Traffic Index Forwarding Status Input VRF Name IGP Next Hop BGP Next Hop

Table 4-18 lists the transport-related key fields.

Table 4-18 Flexible NetFlow Transport Key Fields

	Transport
Fields	Destination Port Source Port ICMP Code ICMP Type IGMP Type (IPv4 only) TCP ACK Number TCP Header Length TCP Sequence Number TCP Window-Size TCP Source Port TCP Destination Port TCP Urgent Pointer

Only the Application ID is a Layer 3 routing protocol key field.

Table 4-19 lists the multicast-related key fields.

Table 4-19 Flexible NetFlow Multicast Key Fields

	Multicast
Fields	Replication Factor (IPv4 only) RPF Check Drop (IPv4 only) Is-Multicast

Flexible NetFlow Non-Key Fields

There are several non-key Flexible NetFlow fields. Table 4-20 lists the non-key fields that are related to counters, such as byte counts, number of packets, and more. A network administrator can use non-key fields for different purposes. For instance, the number of packets and amount of data (bytes) can be used for capacity planning and also to identify denial-of-service (DoS) attacks as well as other anomalies in the network.

Table 4-20 Flexible NetFlow Counters Non-Key Fields

	Counters
Fields	Bytes Bytes Long Bytes Square Sum Bytes Square Sum Long Packets Packets Long Bytes Replicated Bytes Replicated Long Packets Replicated Packets Replicated Long

Table 4-21 lists the timestamp-related non-key fields.

Table 4-21 Flexible NetFlow Timestamp Non-Key Fields

	Timestamp
Fields	sysUpTime First Packet sysUpTime First Packet Absolute First Packet Absolute Last Packet

Table 4-22 lists the IPv4-only non-key fields.

Table 4-22 Flexible NetFlow IPv4-Only Non-Key Fields

	IPv4 Only
Fields	Total Length Minimum Total Length Maximum TTL Minimum TTL Maximum

Table 4-23 lists the IPv4 and IPv6 non-key fields.

Table 4-23 Flexible NetFlow IPv4 and IPv6 Non-Key Fields

	IPv4 and IPv6
Fields	Total Length Minimum Total Length Maximum

NetFlow Predefined Records

Flexible NetFlow includes several predefined records that can help an administrator and security professional start deploying NetFlow within their organization. Alternatively, they can create their own customized records for more granular analysis. As Cisco evolves Flexible NetFlow, many popular user-defined flow records could be made available as predefined records to make them easier to implement.

The predefined records guarantee backward compatibility with legacy NetFlow collectors. Predefined records have a unique blend of key and non-key fields that allows the network administrator and security professional to monitor different types of traffic in their environment without any customization.

---

NOTE

Flexible NetFlow predefined records that are based on the aggregation cache schemes in legacy NetFlow do not perform aggregation. Alternatively, the predefined records track each flow separately.

---

User-Defined Records

As the name indicates, Flexible NetFlow gives the network administrator and security professional the flexibility to create their own records (user-defined records) by specifying key and non-key fields to customize the data collection. The values in non-key fields are added to flows to provide additional information about the traffic in the flows. A change in the value of a non-key field does not create a new flow. In most cases, the values for non-key fields are taken from only the first packet in the flow. Flexible NetFlow enables you to capture counter values such as the number of bytes and packets in a flow as non-key fields.

Flexible NetFlow adds a new NetFlow v9 export format field type for the header and packet section types. A device configured for Flexible NetFlow communicates to the collector the configured section sizes in the corresponding NetFlow v9 export template fields.

Flow Monitors

In Flexible NetFlow, flow monitors are applied to the network device interfaces to perform network traffic monitoring. Flow data is collected from the network traffic and added to the flow monitor cache during the monitoring process based on the key and non-key fields in the flow record.

Flow Exporters

The entities that export the data in the flow monitor cache to a remote system are called flow exporters. Flow exporters are configured as separate entities and are assigned to flow monitors. An administrator can create several flow exporters and assign them to one or more flow monitors. A flow exporter includes the destination address of the reporting server, the type of transport (UDP or SCTP), and the export format corresponding of the NetFlow version or IPFIX.

---

NOTE

You can configure up to eight flow exporters per flow monitor in Cisco IOS-XE software.

---

Flow Samplers

Flow samplers are created as separate components in a router’s configuration. Flow samplers are used to reduce the load on the device that is running Flexible NetFlow by limiting the number of packets that are selected for analysis.

Flow sampling exchanges monitoring accuracy for router performance. When you apply a sampler to a flow monitor, the overhead load on the router due to running the flow monitor is reduced because the number of packets that the flow monitor must analyze is reduced. The reduction in the number of packets that are analyzed by the flow monitor causes a corresponding reduction in the accuracy of the information stored in the flow monitor’s cache.

Flexible NetFlow Configuration

The following sections provide step-by-step configuration guidance on how to enable and configure Flexible NetFlow in a Cisco IOS device. Figure 4-9 shows the configuration steps in a sequential graphical representation.

In the flow diagram, the first step is “Configure a Flow Record.” The second step is “Configure a Flow Monitor.” The third step is “Configure a Flow Exporter for the Flow Monitor.” The final step is “Apply the Flow Monitor to an Interface.”

Figure 4-9 Flexible NetFlow Configuration Steps

The configuration steps, which are described in detail in the corresponding sections, are as follows:

Step 1. Configure a flow record

Step 2. Configure a flow monitor

Step 3. Configure a flow exporter for the flow monitor

Step 4. Apply the flow monitor to an interface

The topology shown in Figure 4-10 is used in the following examples.

In the diagram, an Internet cloud at the top with IP Address/ Subnet Mask “209.165.200.224/29” is connected to a port labeled “.225” of a router “NY-ASR1004” at its bottom. Port labeled “.233” of the router with IP Address/ Subnet Mask “209.165.200.232/29” is connected to port labeled “.234” of another router at its bottom. The router is connected to a server at its bottom with IP Address “10.10.10.123.” The routers and the server are enclosed within a network cloud labeled “Corporate Headquarters (New York).”

Figure 4-10 Flexible NetFlow Configuration Example Topology

This figure shows a Cisco ASR 1004 at the New York headquarters that is configured for Flexible NetFlow. The outside network is 209.165.200.224/29, and the inside network is 209.165.200.232/29.

Configure a Flow Record

The following are the steps required to configure a customized flow record.

---

NOTE

There are hundreds of possible ways to configure customized flow records. The following steps can be followed to create one of the possible variations. You can create a customized flow record depending on your organization’s requirements.

---

Step 1. Log in to your router and enter into enable mode with the enable command:

NY-ASR1004>enable

Step 2. Enter into configuration mode with the configure terminal command:

Click here to view code image

NY-ASR1004#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.

Step 3. Create a flow record with the flow record command. In this example, the record name is NY-ASR-FLOW-RECORD-1. After you’ve entered the flow record command, the router enters flow record configuration mode. You can also use the flow record command to edit an existing flow record:

Click here to view code image

NY-ASR1004(config)#flow record NY-ASR-FLOW-RECORD-1

Step 4. (Optional.) Enter a description for the new flow record:

Click here to view code image

NY-ASR1004(config-flow-record)#description FLOW RECORD 1 for basic traf
fic analysis

Step 5. Configure a key field for the flow record using the match command. In this example, the IPv4 destination address is configured as a key field for the record:

Click here to view code image

NY-ASR1004(config-flow-record)#match ipv4 destination address

The output of the match ? command shows all the primary options for the key field categories that you learned earlier in this chapter:

Click here to view code image

NY-ASR1004(config-flow-record)#match ?
  application  Application fields
  flow         Flow identifying fields
  interface    Interface fields
  ipv4         IPv4 fields
  ipv6         IPv6 fields
  routing      Routing attributes
  transport    Transport layer fields

Step 6. Configure a non-key field with the collect command. In this example, the input interface is configured as a non-key field for the record:

Click here to view code image

NY-ASR1004(config-flow-record)#collect interface input

The output of the collect ? command shows all the options for the non-key field categories that you learned earlier in this chapter:

Click here to view code image

NY-ASR1004(config-flow-record)#collect ?
  application  Application fields
  counter      Counter fields
  flow         Flow identifying fields
  interface    Interface fields
  ipv4         IPv4 fields
  ipv6         IPv6 fields
  routing      Routing attributes
  timestamp    Timestamp fields
  transport    Transport layer fields

Step 7. Exit configuration mode with the end command and return to privileged EXEC mode:

Click here to view code image

NY-ASR1004(config-flow-record)#end

---

NOTE

You can configure Flexible NetFlow to support Network-Based Application Recognition (NBAR) with the match application name command under Flexible NetFlow Flow Record configuration mode.

---

You can use the show flow record command to show the status and fields for the flow record. If multiple flow records are configured in the router, you can use the show flow record name command to show the output of a specific flow record, as shown in Example 4-1.

Click here to view code image

Example 4-1 Output of the show flow record Command

NY-ASR1004#/>show flow record NY-ASR-FLOW-RECORD-1/>
flow record NY-ASR-FLOW-RECORD-1:
  Description:        Used for basic traffic analysis
  No. of users:       0
  Total field space:  8 bytes
  Fields:
    match ipv4 destination address
    collect interface input

Use the show running-config flow record command to show the flow record configuration in the running configuration, as shown in Example 4-2.

Click here to view code image

Example 4-2 Output of the show running-config flow record Command

NY-ASR1004#show running-config flow record
Current configuration:
!
flow record NY-ASR-FLOW-RECORD-1
 description Used for basic traffic analysis
 match ipv4 destination address
 collect interface input
!

Configuring a Flow Monitor for IPv4 or IPv6

The following are the steps required to configure a flow monitor for IPv4 or IPv6 implementations. In the following examples, a flow monitor is configured for the previously configured flow record.

Step 1. Log in to your router and enter into enable mode with the enable command:

NY-ASR1004>enable

Step 2. Enter into configuration mode with the configure terminal command:

Click here to view code image

NY-ASR1004#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.

Step 3 Create a flow monitor with the flow monitor command. In this example, the flow monitor is called NY-ASR-FLOW-MON-1:

Click here to view code image

NY-ASR1004(config)#flow monitor NY-ASR-FLOW-MON-1

Step 4. (Optional.) Enter a description for the new flow monitor:

Click here to view code image

NY-ASR1004(config-flow-monitor)#description monitor for IPv4 traffic in
NY

Step 5. Identify the record for the flow monitor:

Click here to view code image

NY-ASR1004(config-flow-monitor)#record netflow NY-ASR-FLOW-RECORD-1

In the following example, the record ? command is used to see all the flow monitor record options:

Click here to view code image

NY-ASR1004(config-flow-monitor)#record ?
  NY-ASR-FLOW-RECORD-1  Used for basic traffic analysis
  netflow               Traditional NetFlow collection schemes
  netflow-original      Traditional IPv4 input NetFlow with origin ASs

Step 6. Exit configuration mode with the end command and return to privileged EXEC mode:

NY-ASR1004(config-flow-record)#end

You can use the show flow monitor command to show the status and configured parameters for the flow monitor, as shown in Example 4-3.

Click here to view code image

Example 4-3 Output of the show flow monitor Command

NY-ASR1004#/>show flow monitor/>
Flow Monitor NY-ASR-FLOW-MON-1:
  Description:       monitor for IPv4 traffic in NY
  Flow Record:       NY-ASR-FLOW-RECORD-1
  Cache:
    Type:              normal (Platform cache)
    Status:            not allocated
    Size:              200000 entries
    Inactive Timeout:  15 secs
    Active Timeout:    1800 secs
    Update Timeout:    1800 secs

Use the show running-config flow monitor command to display the flow monitor configuration in the running configuration, as shown in Example 4-4.

Click here to view code image

Example 4-4 Output of the show running-config flow monitor Command

NY-ASR1004#/>show running-config flow monitor/>
Current configuration:
!
flow monitor NY-ASR-FLOW-MON-1
 description monitor for IPv4 traffic in NY
 record NY-ASR-FLOW-RECORD-1
 cache entries 200000

Configuring a Flow Exporter for the Flow Monitor

Complete the following steps to configure a flow exporter for the flow monitor in order to export the data that is collected by NetFlow to a remote system for further analysis and storage. This is an optional step. IPv4 and IPv6 are supported for flow exporters.

---

NOTE

Flow exporters use UDP as the transport protocol and use the NetFlow v9 export format. Each flow exporter supports only one destination. If you want to export the data to multiple destinations, you must configure multiple flow exporters and assign them to the flow monitor.

---

Step 1. Log in to the router and enter into enable and configuration modes, as you learned in previous steps.

Step 2. Create a flow exporter with the flow exporter command. In this example, the exporter name is NY-EXPORTER-1:

Click here to view code image

NY-ASR1004(config)#flow exporter NY-EXPORTER-1

Step 3. (Optional.) Enter a description for the exporter:

Click here to view code image

NY-ASR1004(config-flow-exporter)#description exports to New York Collector

Step 4. Configure the export protocol using the export-protocol command. In this example, NetFlow v9 is used. You can also configure legacy NetFlow v5 with the netflow-v5 keyword or IPFIX with the ipfix keyword. IPFIX support was added in Cisco IOS Software Release 15.2(4)M and Cisco IOS XE Release 3.7S:

Click here to view code image

NY-ASR1004(config-flow-exporter)#export-protocol netflow-v9

Step 5. Enter the IP address of the destination host with the destination command. In this example, the destination host is 10.10.10.123:

Click here to view code image

NY-ASR1004(config-flow-exporter)#destination 10.10.10.123

Step 6. You can configure the UDP port used by the flow exporter with the transport udp command. The default is UDP port 9995.

Step 7. Exit the Flexible NetFlow flow monitor configuration mode with the exit command and specify the name of the exporter in the flow monitor:

Click here to view code image

NY-ASR1004(config)#flow monitor NY-ASR-FLOW-MON-1
NY-ASR1004(config-flow-monitor)#exporter NY-EXPORTER-1

You can use the show flow exporter command to view the configured options for the Flexible NetFlow exporter, as demonstrated in Example 4-5.

Click here to view code image

Example 4-5 Output of the show flow exporter Command

NY-ASR1004#/>show flow exporter/>
Flow Exporter NY-EXPORTER-1:
  Description:              exports to New York Collector
  Export protocol:          NetFlow Version 9
  Transport Configuration:
    Destination IP address: 10.10.10.123
    Source IP address:      209.165.200.225
    Transport Protocol:     UDP
    Destination Port:       9995
    Source Port:            55939
    DSCP:                   0x0
    TTL:                    255
    Output Features:        Used

You can use the show running-config flow exporter command to view the flow exporter configuration in the command line interface (CLI), as demonstrated in Example 4-6.

Click here to view code image

Example 4-6 Output of the show running-config flow exporter Command

NY-ASR1004# />show running-config flow exporter/>
Current configuration:
!
flow exporter NY-EXPORTER-1
 description exports to New York Collector
 destination 10.10.10.123

You can use the show flow monitor name NY-ASR-FLOW-MON-1 cache format record command to display the status and flow data in the NetFlow cache for the flow monitor, as demonstrated in Example 4-7.

Click here to view code image

Example 4-7 Output of the show flow monitor name NY-ASR-FLOW-MON-1 cache format record Command

NY-ASR1004#/>show flow monitor name NY-ASR-FLOW-MON-1 cache format record/>
  Cache type:                                 Normal (Platform cache)
  Cache size:                               200000
  Current entries:                            4
  High Watermark:                             4
  Flows added:                              132
  Flows aged:                                42
    - Active timeout   (  3600 secs)          3
    - Inactive timeout (    15 secs)         94
    - Event aged                              0
    - Watermark aged                          0
    - Emergency aged                          0
IPV4 DESTINATION ADDRESS:  10.10.20.5
ipv4 source address:       10.10.10.42
trns source port:          25
trns destination port:     25
counter bytes:             34320
counter packets:           1112
IPV4 DESTINATION ADDRESS:  10.10.1.2
ipv4 source address:       10.10.10.2
trns source port:          20
trns destination port:     20
counter bytes:             3914221
counter packets:           5124
IPV4 DESTINATION ADDRESS:  10.10.10.200
ipv4 source address:       10.20.10.6
trns source port:          32
trns destination port:     3073
counter bytes:             82723
counter packets:           8232

Applying a Flow Monitor to an Interface

A flow monitor must be applied to at least one interface. To apply the flow monitor to an interface, use the ip flow monitor name input command in interface configuration mode, as demonstrated in Example 4-8.

Click here to view code image

Example 4-8 Applying the Flow Monitor to an Interface

NY-ASR1004(config)#/>interface FastEthernet0/1/1/>
NY-ASR1004(config-if)#/>ip flow monitor NY-ASR-FLOW-MON-1 input/>

In Example 4-8, the flow monitor NY-ASR-FLOW-MON-1 is applied to interface FastEthernet0/1/1. Example 4-9 shows the complete configuration.

Click here to view code image

Example 4-9 Flexible NetFlow Configuration

flow record NY-ASR-FLOW-RECORD-1
 description used for basic traffic analysis
 match ipv4 destination address
 collect interface input
!
!
flow exporter NY-EXPORTER-1
 description exports to New York Collector
 destination 10.10.10.123
!
!
flow monitor NY-ASR-FLOW-MON-1
 description monitor for IPv4 traffic in NY
 record NY-ASR-FLOW-RECORD-1
 exporter NY-EXPORTER-1
 cache entries 200000
!
interface FastEthernet0/1/1
 ip address 209.165.200.233 255.255.255.248
 ip flow monitor NY-ASR-FLOW-MON-1 input

IPFIX

The Internet Protocol Flow Information Export (IPFIX) is a network flow standard led by the Internet Engineering Task Force (IETF). IPFIX was created to develop a common, universal standard of export for flow information from routers, switches, firewalls, and other infrastructure devices. IPFIX defines how flow information should be formatted and transferred from an exporter to a collector. IPFIX is documented in RFC 7011 through RFC 7015 as well as RFC 5103. Cisco NetFlow Version 9 is the basis and main point of reference for IPFIX. IPFIX changes some of the terminologies of NetFlow, but in essence they are the same principles of NetFlow v9.

---

NOTE

The different NetFlow versions, as well as each of the components, packet types, and other detailed information, are covered in Chapter 2, “Forensics.”

---

IPFIX defines different elements that are placed into 12 groups according to their applicability:

1. Identifiers

2. Metering and exporting process configuration

3. Metering and exporting process statistics

4. IP header fields

5. Transport header fields

6. Sub-IP header fields

7. Derived packet properties

8. Min/Max flow properties

9. Flow timestamps

10. Per-flow counters

11. Miscellaneous flow properties

12. Padding

IPFIX is considered to be a push protocol. Each IPFIX-enabled device regularly sends IPFIX messages to configured collectors (receivers) without any interaction by the receiver. The sender controls most of the orchestration of the IPFIX data messages. IPFIX introduces the concept of templates, which make up these flow data messages to the receiver. IPFIX also allows the sender to use user-defined data types in its messages. IPFIX prefers the Stream Control Transmission Protocol (SCTP) as its transport layer protocol; however, it also supports the use of the Transmission Control Protocol (TCP) or User Datagram Protocol (UDP) messages.

Traditional Cisco NetFlow records are usually exported via UDP messages. The IP address of the NetFlow collector and the destination UDP port must be configured on the sending device. The NetFlow standard (RFC 3954) does not specify a specific NetFlow listening port. The standard or most common UDP port used by NetFlow is UDP port 2055, but other ports, such as 9555, 9995, 9025, and 9026, can also be used. UDP port 4739 is the default port used by IPFIX.

IPFIX Architecture

IPFIX uses the following architecture terminology:

Metering process (MP): Generates flow records from packets at an observation point. It timestamps, samples, and classifies flows. The MP also maintains flows in an internal data structure and passes complete flow information to an exporting process (EP).

Exporting process (EP): Sends flow records via IPFIX from one or more MPs to one or more collecting processes (CPs).

Collecting process (CP): Receives records via IPFIX from one or more EPs.

IPFIX Mediators

IPFIX introduces the concept of mediators. Mediators collect, transform, and re-export IPFIX streams to one or more collectors. Their main purpose is to allow federation of IPFIX messages. Mediators include an intermediate process (ImP) that allows for the following:

For NetFlow data to be kept anonymously

For NetFlow data to be aggregated

Filtering of NetFlow data

Proxying of web traffic

IP translation

IPFIX Templates

An IPFIX template describes the structure of flow data records within a dataset. Templates are identified by a template ID, which corresponds to set ID in the set header of the dataset. Templates are composed of “information element (IE) and length” pairs. IEs provide field type information for each template.

A standard information model covers nearly all common flow collection use cases, such as the following:

The traditional 5-tuple (source IP address, destination IP address, source port, destination port, and IP protocol)

Packet treatment such as IP next-hop IPv4 addresses, BGP destination ASN, and others

Timestamps to nanosecond resolution

IPv4, IPv6, ICMP, UDP, and TCP header fields

Sub-IP header fields such as source MAC address and wireless local area network (WLAN) service set identifier (SSID)

Various counters (packet delta counts, total connection counts, top talkers, and so on)

Flow metadata information such as ingress and egress interfaces, flow direction, and virtual routing and forwarding (VRF) information

---

NOTE

There are numerous others defined at the Internet Assigned Numbers Authority (IANA) website: http://www.iana.org/assignments/ipfix/ipfix.xhtml.

---

Option Templates

Option templates are a different type of IPFIX template used to define records referred to as “options” that are associated with a specified scope. A scope may define an entity in the IPFIX architecture, including the exporting process, other templates, or a property of a collection of flows. Flow records describe flows, and option records define things other than flows, such as the following:

Information about the collection infrastructure

Metadata about flows or a set of flows

Other properties of a set of flows

Introduction to the Stream Control Transmission Protocol (SCTP)

IPFIX uses SCTP, which provides a packet transport service designed to support several features beyond TCP or UDP capabilities. These features include the following:

Packet streams

Partial reliability (PR) extension

Unordered delivery of packets or records

Transport layer multihoming

Many refer to SCTP as a simpler state machine (compared to the features provided by TCP) with an “a la carte” selection of features. PR-SCTP provides a reliable transport with a mechanism to skip packet retransmissions. It allows for multiple applications with different reliability requirements to run on the same flow association. In other words, it combines the best effort reliability of UDP while still providing TCP-like congestion control. SCTP ensures that IPFIX templates are sent reliably by improving end-to-end delay. RFC 6526 introduces additional features such as per-template drop counting with partial reliability and fast template reuse.

NetFlow and IPFIX Comparison

IPFIX was derived from NetFlow v9. The IPFIX standard specifications (RFC 5101 and RFC 5102) were co-authored by Benoit Clais, who also co-authored the NetFlow v9 RFC (RFC 3954).

IPFIX introduces several extensions, the most popular of which are the Information Element identifiers. These identifiers are compatible with the field types used by NetFlow v9 that you learned about in the previous sections of this chapter.

There are several similar concepts between NetFlow v9 and IPFIX. The first identifier in NetFlow v9 is called IN_BYTES, and in IPFIX is called octetDeltaCount.

As you learned earlier in this chapter, NetFlow v9 has 127 field types. IPFIX defines 238, many of which are the same as the ones defined in NetFlow v9. IPFIX allows a vendor ID to be specified, whereby the vendor can stick proprietary information into NetFlow.

The Cisco Flexible NetFlow IPFIX Export Format feature allows a NetFlow-enabled device to export packets using the IPFIX export protocol.

NetFlow for Cybersecurity and Incident Response

NetFlow is a tremendous security tool. It provides anomaly detection and investigative capabilities that can be helpful in incident response. The Cisco Cyber Threat Defense (CTD) solution uses NetFlow as the primary security visibility tool. Complete visibility is one of the key requirements when identifying and classifying security threats.

The first step in the process of preparing your network and staff to successfully identify security threats is achieving complete network visibility. You cannot protect against or mitigate what you cannot view/detect. You can achieve this level of network visibility through existing features on network devices you already have and on devices whose potential you do not even realize. In addition, you should create strategic network diagrams to clearly illustrate your packet flows as well as where, within the network, you may enable security mechanisms to identify, classify, and mitigate the threat. Remember that network security is a constant war. When defending against the enemy, you must know your own territory and implement defense mechanisms in place. Your goal should be to eliminate any network blind spots.

NetFlow as an Anomaly Detection Tool

You can use NetFlow as an anomaly detection tool. Anomaly-based analysis keeps track of network traffic that diverges from “normal” behavioral patterns. You must define what is considered to be normal behavior. You can use anomaly-based detection to mitigate DDoS attacks and zero-day outbreaks. DDoS attacks are often used maliciously to consume the resources of your hosts and network that would otherwise be used to serve legitimate users. The goal with these types of attacks is to overwhelm the victim network resources, or a system’s resources such as CPU and memory. In most cases, this is done by sending numerous IP packets or forged requests.

Particularly dangerous is when an attacker builds up a more powerful attack with a more sophisticated and effective method of compromising multiple hosts and installing small attack daemons. This is what many call zombies, or bot hosts/nets. Subsequently, an attacker can launch a coordinated attack from thousands of zombies onto a single victim. This daemon typically contains both the code for sourcing a variety of attacks and some basic communications infrastructure to allow for remote control.

Typically, an anomaly-detection system monitors network traffic and alerts and then reacts to any sudden increase in traffic and any other anomalies.

NetFlow, along with other mechanisms such as syslog and SNMP, can be enabled within your infrastructure to provide the necessary data used for identifying and classifying threats and anomalies. Before implementing these anomaly-detection capabilities, you should perform traffic analysis to gain an understanding of general traffic rates and patterns. In anomaly detection, learning is generally performed over a significant interval, including both the peaks and valleys of network activity.

Incident Response and Network Security Forensics

NetFlow is often compared to a phone bill. When police want to investigate criminals, for instance, they often collect and investigate their phone records. NetFlow provides information about all network activity that can be very useful for incident response and network forensics. This information can help you discover indicators of compromise (IOC).

The following six-step methodology on security incident handling has been adopted by many organizations, including service providers, enterprises, and government organizations:

Step 1. Preparation

Step 2. Identification

Step 3. Containment

Step 4. Eradication

Step 5. Recovery

Step 6. Lessons learned

NetFlow plays a crucial role in the preparation and identification phases. Information collected in NetFlow records can be used as part of identifying, categorizing, and scoping suspected incidents as part of the identification. NetFlow data also provides great benefits for attack traceback and attribution. In addition, NetFlow provides visibility into what is getting into your network and what information is being exfiltrated out of your network.

Figure 4-11 shows an example of how a botnet is performing a DDoS attack against the corporate network, while at the same time communicating with an internal host in the call center. NetFlow in this case can be used as an anomaly-detection tool for the DDoS attack and also as a forensics tool to potentially find other IOCs of more sophisticated attacks that may be carried out incognito.

At the top left of the diagram, a group of zombie computers enclosed within a network cloud named “Botnet” is shown. The Botnet is connected to an Internet cloud at its right. The Internet cloud is connected to 2 NetFlow-Enabled Routers at its bottom-left and bottom-right, which are connected to each other. From the Botnet, 5 dashed arrows point to the NetFlow-Enabled Router at the left. Each router is connected to a Firewall Appliance at its bottom, which is connected to each other. Each Firewall Appliance is connected to a Route/Switch Processor at its bottom, which is connected to each other. The Route/Switch Processor at the left is connected to 2 switches at its left. Both switches are connected to a network cloud with 12 laptops named “Call Center” at its left. From the Botnet, a dashed arrow runs via the NetFlow-Enabled Router, Firewall Appliance, and Route/Switch Processor at the left and then points to the Call Center. The Route/Switch Processor at the right is connected to 2 switches at its right. Both switches are connected to a network cloud with 12 laptops at its right named “Engineering.” Each Route/Switch Processor is connected to a Nexus 7000 switch at its bottom, which are connected to each other. Below each Nexus 7000 Switch, another Nexus 7000 Switch is present. All 4 Nexus 7000 Switches are interconnected. Each Nexus 7000 switch at the bottom is connected to a Cisco UCS C-Series server at its bottom, which are connected to each other. Each Cisco UCS C-Series server is connected to 2 Cisco UCS 5108 Chassis servers at its bottom.

Figure 4-11 Detecting What Is Getting Into Your Network

Figure 4-12 shows how a “stepping-stone” attack is carried out in the corporate network. A compromised host in the engineering department is extraditing large amounts of sensitive data to an attacker in the Internet from a server in the data center.

You can also use NetFlow in combination with DNS records to help you detect suspicious and malicious traffic, such as the following:

Suspicious requests to .gov, .mil, and .edu sites when you do not even do business with any of those entities

Large amounts of traffic leaving the organization late at night to suspicious sites

Traffic to embargoed countries that should not have any business partners or transactions

Suspicious virtual private network (VPN) requests and VPN traffic

Requests and transactions to sites without any content

Pornography sites or any other sites that violate corporate policy

Illegal file-sharing sites

At the top right of the diagram, an Attacker is shown. The Attacker is connected to an Internet cloud at its left. The Internet cloud is connected to 2 NetFlow-Enabled Routers at its bottom-left and bottom-right, which are connected to each other. Each router is connected to a Firewall Appliance at its bottom, which is connected to each other. Each Firewall Appliance is connected to a Route/Switch Processor at its bottom, which is connected to each other. The Route/Switch Processor at the left is connected to 2 switches at its left. Both switches are connected to a network cloud with 12 laptops named “Call Center” at its left. The Route/Switch Processor at the right is connected to 2 switches at its right. Both switches are connected to a network cloud with 12 laptops at its right named “Engineering.” From the Attacker, a dashed arrow labeled “2” runs via the NetFlow-Enabled Router, Firewall Appliance, and Route/Switch Processor at the right and then points to the Engineering cloud. Each Route/Switch Processor is connected to a Nexus 7000 switch at its bottom, which are connected to each other. Below each Nexus 7000 Switch, another Nexus 7000 Switch is present. All 4 Nexus 7000 Switches are interconnected. Each Nexus 7000 switch at the bottom is connected to a Cisco UCS C-Series server at its bottom, which is connected to each other. Each Cisco UCS C-Series server is connected to 2 Cisco UCS 5108 Chassis servers at its bottom. From the Chassis server at the right end, a dashed arrow labeled “1” points to the Engineering cloud.

Figure 4-12 Detecting What Is Getting Out of Your Network

Syslog and packet captures are also often used in network forensics; however, an area where these traditional network forensics tools fall short is in coverage. For instance, it is very difficult to deploy hundreds of sniffers (packet-capture devices) in the network of large organizations. In addition, the cost will be extremely high. When a security incident or breach is detected, the incident responders need answers fast! They do not have time to go over terabytes of packet captures, and they can definitely not analyze every computer on the network to find the root cause, miscreant, and source of the breach. You can use NetFlow to obtain a high-level view of what is happening in the network, and then the incident responder can perform a deep-dive investigation with packet captures and other tools later in the investigation. Sniffers then can be deployed as needed in key locations where suspicious activity is suspected. The beauty of NetFlow is that you can deploy it anywhere you have a supported router, switch, or Cisco ASA; alternatively, you can use Cisco NetFlow Generation Appliance (NGA). After the Lancope acquisition, Cisco also sells the StealthWatch FlowSensor product, which is a physical or virtual appliance that can generate NetFlow data when legacy Cisco network infrastructure components are not capable of producing line-rate, unsampled NetFlow data.

NetFlow can fill in some of the gaps and challenges regarding the collection of packet captures everywhere in the network. It is easier to store large amounts of NetFlow data because it is only a transactional record. Therefore, administrators can keep a longer history of events that occurred on their networks. Historical records can prove very valuable when investigating a breach. Network transactions can show you where an initial infection came from, what command-and-control channel was initiated by the malware, what other computers on the internal network were accessed by that infected host, and whether other hosts in the network reached out to the same attacker or command-and-control system.

The logging facility on Cisco IOS routers, switches, Cisco ASA, and other infrastructure devices allows you to save syslog messages locally or to a remote host. By default, routers send logging messages to a logging process. The logging process controls the delivery of logging messages to various destinations, such as the logging buffer, terminal lines, a syslog server, or a monitoring event correlation system such as Cisco Prime Infrastructure or Splunk. You can set the severity level of the messages to control the type of messages displayed, in addition to a timestamp to successfully track the reported information. Every security professional and incident responder knows how important it is to have good logs. There is no better way to find out what was happening in a router, switch, or firewall at the time an attack occurred. However, like all things, syslog has limitations. You have to enable the collection of logs from each endpoint; so in many environments, syslog coverage is incomplete, and after a computer has been compromised, it is not possible to trust the logs coming from that device anymore. Syslog is extremely important, but it cannot tell you everything. Many network telemetry sources can also be correlated with NetFlow while responding to security incidents and performing network forensics, including the following:

Dynamic Host Configuration Protocol (DHCP) logs

VPN logs

Network address translation (NAT) information

802.1x authentication logs

Server logs (syslog)

Web proxy logs

Spam filters from email security appliances such as the Cisco Email Security Appliance (ESA)

DNS server logs

Active directory logs

Table 4-24 lists the different event types, their source, and their respective events that can be combined with NetFlow while you’re responding to security incidents and performing network forensics.

Table 4-24 Network Telemetry Type, Sources, and Events

Event Type	Source	Events
Attribution	DHCP server	IP assignments to machine MAC addresses
	VPN server	IP assignments to users VPN source addresses
	NAT gateway	NAT/PAT logs
	802.1x authentication logs DNS logs	IP assignment to user MAC addresses Known malware domains
System activity	Syslog server	Authentication and authorization events Services starting and stopping Configuration changes Security events
Web proxy logs	Web proxies, such as Cisco Web Security (CWS) and Web Security Appliance (WSA)	Web malware downloads Command-and-control check-ins
Spam filter logs	Spam filter, such as Cisco Email Security Appliance (ESA)	Malicious URLs Malicious attachments
Firewall logs	Network firewall, such as Cisco ASA	Accepted/denied connections
Web server logs	Web servers	Access logs Error logs

---

TIP

It is extremely important that your syslog and other messages are timestamped with the correct date and time. This is why the use of Network Time Protocol (NTP) is strongly recommended.

---

Network forensics can be an intimidating topic for many security professionals. Everyone knows that forensic investigation may entail many other sources of information from end hosts, servers, and any affected systems. Each forensics team needs to have awareness of many different areas, such as the following:

Having thorough knowledge of assets, risks, impact, and likelihood of events.

Practicing incident response policies and procedures in mock events and collecting NetFlow on a regular basis to analyze what is happening in the network.

Awareness of current vulnerabilities and threats.

Understanding evidence handling and chain of custody. (Even NetFlow events can be used as evidence.)

Enacting mitigation based on evidence collected.

Knowing the documentation requirements for evidence, depending on your country and local laws.

Understanding the analysis process during and after the incident.

Having a framework for communications, both within the team and external to the team.

Using NetFlow for Data Leak Detection and Prevention

Many network administrators, security professionals, and business leaders struggle in the effort to prevent data loss within their organizations. The ability to identify anomalous behavior in data flows is crucial to detect and prevent data loss. The application of analytics to data collected via NetFlow can aid security professionals in detecting anomalous amounts of data leaving the organization and abnormal traffic patterns inside of the organization.

Using NetFlow along with identity management systems, you can detect who initiated the data transfer, the hosts (IP addresses) involved, the amount of data transferred, and the services used. In addition, you can measure how long the communications lasted as well as the frequency of the same connection attempts.

---

TIP

Often, tuning is necessary because certain traffic behavior could cause false positives. For instance, your organization may be legitimately sharing large amounts of data or streaming training videos to business partners and customers. In addition, analytics software that examines baseline behavior may be able to detect typical file transfers and incorporate them into existing baselines.

---

In the following scenario, a large retail organization is the victim of a major breach where attackers stole more than 100,000 credit card numbers. The retailer headquarters is in New York, NY and has two large secondary offices in Raleigh, North Carolina, and San Juan, Puerto Rico. This retailer also has more than 1000 stores in the United States and Canada. Figure 4-13 illustrates these offices and stores.

In the diagram, an internet cloud is shown which encloses 3 Headquarters, which are interconnected. New York, NY Headquarters at the top of the network cloud is connected to 10 Small business units below the network cloud. Raleigh, NC Headquarters at the left of the network cloud is connected to the first 3 Small business units. San Juan, PR Headquarters at the right of the network cloud is connected to the last 7 Small business units. Below each Small business unit, another Small business unit is shown. All connections are made by dashed lines.

Figure 4-13 Retailer High-Level Network Topology

The breach was not detected for several months after the attackers had already penetrated the network. The retailer had firewalls and intrusion prevention devices, but those were not enough to detect or mitigate the attack. The attack was thought to be an inside job, because the malware that was extracting the credit card numbers was very sophisticated and tailored to such an organization. The breach was detected only because law enforcement contacted the victimized retailer, telling them that thousands of fraudulent credit card transactions had been detected on credit cards that were last legitimately used at their stores.

After the organization started their incident response and forensics investigation, they decided to deploy NetFlow in routers at the edge of the data center. The topology in Figure 4-14 illustrates the network at the New York headquarters and the two routers that were configured with NetFlow.

At the top right of the diagram, a box is shown enclosing a router connected to San Juan Headquarters at its right. At the top left of the diagram, a box is shown enclosing a router connected to Raleigh Headquarters at its left. Both the routers are connected to an Internet cloud between them. The Internet cloud is connected to 2 NetFlow-Enabled Routers at its bottom-left and bottom-right, which are connected to each other. Each NetFlow-Enabled Router is connected to a Firewall at its bottom, which are connected to each other. Within a box named “Internet Edge,” the Firewalls and NetFlow-Enabled Routers are enclosed. Each Firewall is connected to a Route/Switch Processor at its bottom, which are connected to each other within a box named “Distribution.” The Route/Switch Processor at the left is connected to 2 switches enclosed in a box named “Access Layer” at its left. At the left of the switches, 3 boxes named “Call Center,” “Training,” and “Finance” enclosing 3 laptops each are placed one below the other. Each Route/Switch Processor is connected to a Nexus 7000 switch at its bottom, which are connected to each other. Below each Nexus 7000 Switch, another Nexus 7000 Switch is present. All 4 Nexus 7000 Switches are interconnected and enclosed in a box named “Core.” Each Nexus 7000 switch at the bottom is connected to a Cisco UCS C-Series server at its bottom, which are connected to each other. Both servers are enclosed in a box named “Data Center.” Each server is connected to a NetFlow-Enabled Router at its bottom, which is enclosed within a box labeled “NetFlow-Enabled Routers.” Below each Netflow-Enabled Router, a DC Core N9500 switch is present. The DC Core N9500 switches and NetFlow-Enabled Routers are connected and interconnected. At the bottom-left and bottom-right of the DC Core switches, 2 boxes are shown connected by a dotted line. At the top of the boxes, 2 connected DC Core switches labeled “DC Aggregation” at the left and “N9500” at the right are shown. DC Aggregation switches are connected to 4 switches below it labeled “UCS FI.” Both the DC switches above the boxes are connected to all the DC switches within the 2 boxes. At the bottom of each of the UCS FI switches, 2 Cisco UCS C-Series servers are placed one below the other. In a dashed box, the 2 switches and 4 servers at the left are enclosed. In another dashed box, the 2 switches and 4 servers at the right are enclosed. Below the Internet cloud, all the icons are enclosed within a box named “New York.”

Figure 4-14 New York Headquarters NetFlow Routers

The data center has numerous servers that are dedicated for credit card processing applications (software), as illustrated in Figure 4-15.

At the top of the diagram, a box labeled “NetFlow-Enabled Routers” is shown enclosing 2 connected NetFlow-Enabled Routers on either side. From the top of the routers, an open ended cable is shown. Below each router, a DC Core N9500 switch is present. The DC Core N9500 switches and NetFlow-Enabled Routers are connected and interconnected. At the bottom-left and bottom-right of the DC Core switches, 2 boxes named “Credit Card Processing Applications” are shown connected by a dotted line. At the top of the boxes, 2 connected DC Core switches labeled “DC Aggregation” at the left and “N9500” at the right are shown. DC Aggregation switches are connected to 4 switches below it. Both the DC switches above the boxes are connected to all the DC switches within the 2 boxes. At the bottom of each of the 4 switches below DC Aggregation, 2 Cisco UCS C-Series servers are placed one below the other. In a dashed box, the 2 switches and 4 servers at the left are enclosed. In another dashed box, the 2 switches and 4 servers at the right are enclosed.

Figure 4-15 Credit Card Processing Servers

After deploying NetFlow in their data center edge routers, the retailer observed that numerous DNS requests were being sent from the credit card processing servers to DNS servers outside of the country (United States). The most interesting fact was that such DNS servers were in embargoed countries where the retailer previously had never transacted any business. In addition, most of these DNS requests were being sent during off-hours (mostly around 2:00 to 4:00 a.m. local time).

The retailer was able to inspect NetFlow traffic and detect the country where the credit card information was sent by using the MaxMind Geolocation database. MaxMind provides IP intelligence to thousands of companies to locate their Internet visitors and perform analytics. MaxMind has a service called minFraud, which helps businesses prevent fraudulent online transactions and reduce manual review.

---

NOTE

You can obtain more information about MaxMind at https://www.maxmind.com. You can also get access to their database files and open source utilities at https://github.com/maxmind.

---

The attackers were sending stolen credit card data over DNS using tunneling. DNS is a protocol that enables systems to resolve domain names (for example, example.com) into IP addresses (for example, 93.184.216.34). DNS is not intended for a command channel or even tunneling. However, attackers have developed software that enables tunneling over DNS. Because traditionally DNS it is not designed for data transfer, it is less inspected in terms of security monitoring. Undetected DNS tunneling (otherwise known as DNS exfiltration) represents a significant risk to any organization.

In this case, the credit cards were base64 encoded and sent over DNS requests (tunneling) to cybercriminals abroad. Attackers nowadays use different DNS record types and encoding methods to exfiltrate data from victims’ systems and networks. The following are some examples of encoding methods:

Base64 encoding

Binary (8-bit) encoding

NetBIOS encoding

Hex encoding

Several utilities have been created to perform DNS tunneling (for the good and also the bad). The following are a few examples:

DeNiSe: A Python tool for tunneling TCP over DNS.

dns2tcp: Written by Olivier Dembour and Nicolas Collignon in C, dns2tcp supports KEY and TXT request types.

DNScapy: Created by Pierre Bienaime, this Python-based Scapy tool for packet generation even supports SSH tunneling over DNS, including a SOCKS proxy.

DNScat or DNScat-P: This Java-based tool created by Tadeusz Pietraszek supports bidirectional communication through DNS.

DNScat (DNScat-B): Written by Ron Bowes, this tool runs on Linux, Mac OS X, and Windows. DNScat encodes DNS requests in NetBIOS or hex encoding.

Heyoka: This tool, written in C, supports bidirectional tunneling for data exfiltration.

Iodine: Written by Bjorn Andersson and Erik Ekman in C, Iodine runs on Linux, Mac OS X, and Windows, and can even be ported to Android.

Nameserver Transfer Protocol (NSTX): Creates IP tunnels using DNS.

OzymanDNS: Written in Perl by Dan Kaminsky, this tool is used to set up an SSH tunnel over DNS or for file transfer. The requests are base32 encoded, and responses are base64-encoded TXT records.

psudp: Developed by Kenton Born, this tool injects data into existing DNS requests by modifying the IP/UDP lengths.

Feederbot and Moto: This malware using DNS has been used by attackers to steal sensitive information from many organizations.

---

NOTE

Some of these tools were not created with the intent to steal data, but cybercriminals have used them for their own purposes.

---

The retailer’s network security personnel were able to perform detailed analysis of the techniques used by the attackers to steal this information and discovered the types of malware and vulnerabilities being exploited in systems in the data center. Network telemetry tools such as NetFlow are invaluable when trying to understand what is happening (good and bad) in the network, and it is a crucial tool for incident response and network forensics.

Retailers or any organizations that process credit cards or electronic payments are often under regulation from the Payment Card Industry Data Security Standard (PCI DSS). PCI DSS was created to encourage and maintain cardholder data security and expedite the consistent use of data security methodologies. This standard enforces a baseline of technical and operational requirements. PCI DSS applies to the following:

Banks

“Brick-and-mortar” retailers and online retailers

Merchants

Processors

Acquirers

Issuers

Service providers or any other organizations that store, process, or transmit cardholder data (CHD) and/or sensitive authentication data (SAD)

The PCI DSS defines general requirements for the following:

Building and maintaining secure networks

Protecting cardholder data

Enforcing vulnerability management and patching programs

Implementing adequate access control measures

Consistently monitoring networks and their systems

Guaranteeing the maintenance of information security policies

As you can see from this list, adequate monitoring of systems is an underlying and fundamental requirement. NetFlow, intrusion prevention systems, and others are often used to maintain this required visibility into what is happening in the network. Additional details about PCI DSS are covered in Chapter 7, “Compliance Frameworks.”

Many threat actors are seeking to steal intellectual property from many organizations and individuals. According to the Merriam-Webster dictionary, intellectual property is “something (such as an idea, invention, or process) that derives from the work of the mind or intellect; an application, right, or registration relating to this.” Intellectual property (and other forms of expression) is often protected by patent, trademark, and copyright (in addition to state and federal laws). In today’s world, espionage (both cyber and corporate) is a huge business. Many types of attackers (for example, corporations, cybercriminals, and nation states) are after information with independent economic value, such as the following:

Blueprints

Chemical formulas

Research and development documents

Marketing strategies

Manufacturing processes

Source code

Songs

Books

Documentation guides

In 1996, to maintain the health and competitiveness of the U.S. economy, the United States Congress passed the Economic Espionage Act to protect trade secrets from bad actors.

NetFlow Analysis Tools

There are many different commercial and open source NetFlow analysis tools in the industry. The following two sections cover several examples of commercial and open source NetFlow analysis tools.

Commercial NetFlow Analysis Tools

Table 4-25 lists the most popular commercial NetFlow monitoring and analysis software packages in the industry today.

Table 4-25 Examples of Commercial NetFlow Monitoring and Analysis Software

Commercial Software	Description	Website
ManageEngine NetFlow Analyzer	A web-based bandwidth monitoring tool.	http://manageengine.adventnet.com/products/netflow
NetUsage	Tool for network traffic monitoring, capacity planning, business justification, and cost control.	http://www.netusage.net
Caligare	Traffic monitoring and network anomalies detection.	http://www.caligare.com/
Evident Software Evident Analyze	Tool for billing and traffic analysis.	http://www.evidentsoftware.com/products/anlz_functions.aspx
Fluke Networks	Traffic analysis, NetFlow collection, and low-cost Windows-based NetFlow product.	http://www.flukenetworks.com/
Hewlett Packard NetFlow Insight	Traffic analysis and NetFlow collection using HP Insight Network Performance Monitoring.	http://www.openview.hp.com/products/ovpi_net/
IBM NetFlow Aurora	NetFlow traffic profiling tool commercially available as Tivoli Netcool Performance Flow Analyzer (TNPFA).	http://www.zurich.ibm.com/aurora
IdeaData NetFlow Auditor	Tool used for network troubleshooting, security monitoring, and baseline trending.	http://www.netflowauditor.com
InfoVista 5View NetFlow	NetFlow monitoring tool.	http://www.infovista.com/products/NetFlow-Monitoring-Network-Traffic-Analysis
Cisco Lancope StealthWatch	Traffic analysis, NetFlow collection, and security monitoring tool suite part of Cisco’s Cyber Threat Defense Solution.	http://lancope.com
Paessler PRTG	Network monitoring tool suite.	http://www.paessler.com
Plixer International Scrutinizer	Plixer offers free and commercial NetFlow reporting software. Scrutinizer is an incident response and network monitoring suite of tools.	http://www.plixer.com
SolarWinds NetFlow Traffic Analyzer	NetFlow traffic analyzer and performance management tool.	http://www.solarwinds.com/netflow-traffic-analyzer.aspx

Two of the most popular commercial products are Lancope’s StealthWatch solution and Plixer Scrutinizer, as described in greater detail in the sections that follow.

Cisco’s Lancope StealthWatch Solution

Cisco acquired Lancope, a company who produced the StealthWatch solution, which is a key component of the Cisco Cyber Threat Defense (CTD) solution. One of the key benefits of Lancope’s StealthWatch is its capability to scale in large enterprises. It also provides integration with the Cisco Identity Services Engine (ISE) for user identity information. Cisco ISE is a security policy management and control system that you can use for access control and security compliance for wired, wireless, and virtual private network (VPN) connections.

One other major benefit of Lancope’s StealthWatch is its graphical interface, which includes great visualizations of network traffic, customized summary reports, and integrated security and network intelligence for drill-down analysis.

Figure 4-16 shows a screenshot of Lancope’s StealthWatch Management Console (SMC).

In the left pane of the screenshot, SMC under Enterprise section is chosen. In the content pane, Network Visibility Report tab is selected from the tabs shown. At the top of the Network Visibility Report, the domain and host group names are displayed. Below that, Network Map button is chosen. In the dropdown menu at the top right, “Traffic” is chosen. At the center of the report, a world map with “North America,” “South America,” “Europe,” “Africa,” “Asia,” and “Australia” regions labeled with boxes are shown. The island at the bottom right of Australia is labeled “Others.” All the boxes are connected to a box at the bottom that reads: Corporate Network. The line connecting Others to Corporate Network is colored and labeled “51 bps.” At the top left, the map controls to control the zooming of the map is shown. At the bottom right, a minimized version of the graph is shown.

Figure 4-16 SMC Web Management Application

Figure 4-17 shows a report of the top applications observed in the network. You can drill down into each application and host to get more detailed information about what is happening in the network.

In the left pane of the screenshot, SMC under Enterprise section is chosen. In the content pane, Network Visibility Report tab is selected from the tabs shown. At the top of the Network Visibility Report, the domain name is displayed. Below that, host group name is displayed as “Inside Hosts.” Below that, Network Summary button is chosen. At the top left of the report, a line graph displays the traffic to other inside hosts and outside hosts for that particular day. At the bottom left, a line graph displays Application Traffic within groups for that particular day. The groups include DNS (Unclassified), HTTP (Unclassified), ICMP, routing, etc. At the top right, a pie chart displays the top 25 applications outbound for the last 1 day. In the bottom right section titled “Top Conversations last 1 hour,” a table lists 51 records. The column headers are: Host, Host Groups, Peer, Peer Host Groups, Port, etc. In all the graphs, instruction is given to guide the user to click on an icon with green rightward arrow to expand the view and edit filters.

Figure 4-17 Network Summary Report

Lancope used to have a security research initiative that tracked emerging threat information from around the world, called the StealthWatch Labs Intelligence Center (SLIC). Nowadays, it’s integrated with the Cisco Talos security research team.

Figure 4-18 illustrates the major components of Lancope’s StealthWatch solution.

At the bottom of the diagram, a box named “SYSLOG, SNMP” encloses 2 routers, 4 servers, 2 Firewall Appliances, and 2 switches. An upward arrow labeled “StealthWatch FlowReplicator” from the box points to a server at its top. A leftward arrow from the server points to another server with a dull color labeled “StealthWatch FlowCollector” at its left. A rightward arrow from a box named “NetFlow-Enabled Devices” enclosing a NetFlow-Enabled Router, a Firewall Appliance, and a switch points to StealthWatch FlowCollector. An upward arrow from StealthWatch FlowCollector points to a server colored black at its top named “SMC (Primary).” SMC (Primary) is connected to another dark colored server named “SMC (Failover)” at its right via dashed lines. Pointing to the right of the arrow connecting the SMC and StealthWatch FlowCollector, a dashed leftward arrow and a dashed double-headed arrow are shown. The double-headed arrow points to a box that encloses a Cisco ISE and a server named “StealthWatch IDentity.” The leftward arrow points to a network cloud named “Cisco Talos.”

Figure 4-18 The Lancope’s StealthWatch Solution Components

The following are the primary components of the Lancope StealthWatch solution shown in Figure 4-18:

StealthWatch Management Console: Provides centralized management, configuration, and reporting of the other StealthWatch components. It can be deployed in a physical server or a virtual machine (VM). The StealthWatch Management Console provides high-availability features (failover), as shown in Figure 4-18.

FlowCollector: A physical or virtual appliance that collects NetFlow data from infrastructure devices.

FlowSensor: A physical or virtual appliance that can generate NetFlow data when legacy Cisco network infrastructure components are not capable of producing line-rate, unsampled NetFlow data. Alternatively, the Cisco NetFlow Generator Appliance (NGA) can be used.

FlowReplicator: A physical appliance used to forward NetFlow data as a single data stream to other devices.

StealthWatch IDentity: Provides user identity monitoring capabilities. Administrators can search on usernames to obtain a specific user network activity. Identity data can be obtained from the StealthWatch IDentity appliance or through integration with the Cisco ISE.

---

NOTE

Lancope StealthWatch also supports usernames within NetFlow records from Cisco ASA appliances.

---

Lancope’s StealthWatch solution supports a feature called network address translation (NAT) stitching. NAT stitching uses data from network devices to combine NAT information from inside a firewall (or a NAT device) with information from outside the firewall (or a NAT device) to identify which IP addresses and users are part of a specific flow. A great feature of the StealthWatch solution is its ability to perform “NetFlow deduplication.” This feature allows you to deploy several NetFlow collectors within your organization without worrying about double or triple counting the traffic.

Plixer’s Scrutinizer

Plixer’s Scrutinizer is another commercial NetFlow monitoring and analysis software package that has gone through interoperability tests by Cisco. Scrutinizer is used for incident response and network monitoring. Just like several components of Lancope’s StealthWatch solution, Scrutinizer is available as a physical or virtual appliance. Plixer also sells two other products that provide additional network visibility: FlowPro and Flow Replicator.

FlowPro is an appliance that can be deployed in a specific area of the corporate network to perform deep packet inspection (DPI) by combining NetFlow/IPFIX data. Plixer’s Flow Replicator allows several sources of network device and server log data to be replicated to different destinations. Flow Replicator can also be configured as a syslog-to-IPFIX gateway. It converts syslog messages and forwards them on inside IPFIX datagrams.

Open Source NetFlow Monitoring and Analysis Software Packages

The number of open source NetFlow monitoring and analysis software packages is on the rise. You can use these open source tools to successfully identify security threats within your network.

Table 4-26 lists the most popular open source NetFlow monitoring and analysis software packages.

Table 4-26 Examples of Open Source NetFlow Monitoring and Analysis Software

Open Source Software	Description	Website
cflowd	Traffic flow analysis tool provided by the Center for Applied Internet Data Analysis.	http://www.caida.org/tools/measurement/cflowd
flowtools	Tool set created by Mark Fullmer for collecting and working with NetFlow data.	http://www.splintered.net/sw/flow-tools
flowviewer	FlowViewer is a web-based interface to flow tools and SiLK.	http://sourceforge.net/projects/flowviewer
flowd	Small-packaged NetFlow collector.	http://www.mindrot.org/projects/flowd
IPFlow	NetFlow collector developed by Christophe Fillot of the University of Technology of Compiegne, France.	http://www.ipflow.utc.fr
NFdump	NetFlow analysis toolkit under the BSD license.	http://nfdump.sourceforge.net
NfSen	Web interface for NFdump.	http://sourceforge.net/projects/nfsen
Stager	Provides visualizations for NFdump.	https://trac.uninett.no/stager
Panoptis	NetFlow tool for detecting denial-of-service attacks. Development is fairly limited.	http://panoptis.sourceforge.net
Plixer’s Scrutinizer NetFlow Analyzer	Scrutinizer NetFlow Analyzer is a free version of Plixer’s Scrutinizer.	http://www.plixer.com/Support/free-tools.xhtml
SiLK	System for Internet-Level Knowledge (SiLK) is a NetFlow collector and analysis tool developed by the Carnegie Mellon University’s CERT Network Situational Awareness Team (CERT NetSA).	https://tools.netsa.cert.org/silk
iSiLK	iSiLK is a graphical front end for the SiLK toolkit.	http://tools.netsa.cert.org/isilk
Elasticsearch, Logstash, and Kibana (ELK)	A distributed, scalable, open source, Big Data analytics platform.	https://www.elastic.co
Graylog	A scalable open source log visualization solution.	https://www.graylog.org

NFdump

NFdump is a set of Linux-based tools that support NetFlow Versions 5, 7, and 9. You can download NFdump from http://nfdump.sourceforge.net and install it from source. Alternatively, you can easily install NFdump in multiple Linux distributions such as Ubuntu using sudo apt-get install nfdump.

Table 4-27 lists all the components of the NFdump toolkit.

Table 4-27 NFdump Components

Component	Description
nfcapd	The NetFlow capture daemon. A separate nfcapd process needs to be launched for each NetFlow stream.
nfdump	Reads the NetFlow data from the files stored by nfcapd. The output and syntax are very similar to the Linux-based packet-capture tool tcpdump.
nfprofile	Filters the NetFlow data recorded by nfcapd and stores the filtered data into files for later use. The filters are referred to as profiles.
nfreplay	Replays NetFlow data.
nfclean.pl	Pearl sample script to clean up historical NetFlow data.
ft2nfdump	Converts flow tools data from files or from standard input into nfdump format.

The command to capture the NetFlow data is nfcapd. All processed NetFlow records are stored in one or more binary files. These binary files are read by nfdump and can be displayed in plaintext to standard output (stdout) or written to another file. Example 4-10 demonstrates how the nfcapd command is used to capture and store NetFlow data in a directory called netflow. The server is configured to listen to port 9996 for NetFlow communication.

Click here to view code image

Example 4-10 Using the nfcapd Command

omar@server1:~$ />nfcapd -w -D -l netflow -p 9996/>
omar@server1:~$ />cd netflow/>
omar@server1:~/netflow$ />ls -l/>
total 544
-rw-r--r-- 1 omar omar 20772 Jun 18 00:45 nfcapd.201506180040
-rw-r--r-- 1 omar omar 94916 Jun 18 00:50 nfcapd.201506180045
-rw-r--r-- 1 omar omar 84108 Jun 18 00:55 nfcapd.201506180050
-rw-r--r-- 1 omar omar 78564 Jun 18 01:00 nfcapd.201506180055
-rw-r--r-- 1 omar omar 106732 Jun 18 01:05 nfcapd.201506180100
-rw-r--r-- 1 omar omar 73692 Jun 18 01:10 nfcapd.201506180105
-rw-r--r-- 1 omar omar 76996 Jun 18 01:15 nfcapd.201506180110
-rw-r--r-- 1 omar omar    276 Jun 18 01:15 nfcapd.current

Flows are read either from a single file or from a sequence of files. In Example 4-10, a series of files were created by the nfcapd daemon. Example 4-11 shows the command options of the nfcapd daemon command.

Click here to view code image

Example 4-11 nfcapd Daemon Command Options

omar@ server1:~$ />nfcapd -h/>
usage nfcapd [options]
-h             this text you see right here
-u userid      Change user to username
-g groupid    Change group to groupname
-w             Sync file rotation with next 5min (default) interval
-t interval    set the interval to rotate nfcapd files
-b host        bind socket to host/IP addr
-j mcastgroup Join multicast group <mcastgroup>
-p portnum    listen on port portnum
-l basdir      set the output directory. (no default)
-S subdir      Sub directory format. see nfcapd(1) for format
-I Ident       set the ident string for stat file. (default ‘none’)
-H             Add port histogram data to flow file.(default ‘no’)
-n Ident,IP,logdir Add this flow source - multiple streams
-P pidfile     set the PID file
-R IP[/port]   Repeat incoming packets to IP address/port
-s rate        set default sampling rate (default 1)
-x process     launch process after a new file becomes available
-z             Compress flows in output file.
-B bufflen     Set socket buffer to bufflen bytes
-e             Expire data at each cycle.
-D             Fork to background
-E             Print extended format of netflow data. for debugging purpose only.
-T             Include extension tags in records.
-4             Listen on IPv4 (default).
-6             Listen on IPv6.
-V             Print version and exit.

Example 4-12 demonstrates how to use the nfdump command to process and analyze all files that were created by nfcapd in the netflow directory.

Click here to view code image

Example 4-12 Processing and Displaying the nfcapd Files with nfdump

omar@server1::~$ />nfdump -R netflow -o extended -s srcip -s ip/flows/>
Top 10 Src IP Addr ordered by flows:
Date first seen          Duration Proto       Src IP Addr    Flows(%)     Packets(%)       Bytes(%)         pps      bps   bpp
2017-01-2222:35:10.805     2.353 any       192.168.1.140     1582(19.5)        0(-nan)       0(-nan)        0        0     0
2017-01-2222:35:10.829     2.380 any       192.168.1.130      875(10.8)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.404 any       192.168.1.168      807( 9.9)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:11.219     1.839 any       192.168.1.142      679( 8.4)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.258 any       192.168.1.156      665( 8.2)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.297 any       192.168.1.205      562( 6.9)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.404 any        192.168.1.89      450( 5.5)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:11.050     1.989 any      10.248.91.231      248( 3.1)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:11.633     1.342 any       192.168.1.149      234( 2.9)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:11.040     2.118 any       192.168.1.157      213( 2.6)        0(-nan)        0(-nan)        0        0     0

Top 10 IP Addr ordered by flows:
Date first seen          Duration Proto           IP Addr    Flows(%)     Packets(%)       Bytes(%)         pps      bps   bpp
2017-01-2222:35:10.805     2.353 any       192.168.1.140     1582(19.5)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.353 any             10.8.8.8     1188(14.6)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.297 any         192.168.1.1     1041(12.8)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.829     2.380 any       192.168.1.130      875(10.8)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.404 any       192.168.1.168      807( 9.9)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:11.219     1.839 any       192.168.1.142      679( 8.4)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.258 any       192.168.1.156      665( 8.2)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.297 any       192.168.1.205      562( 6.9)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.825     2.277 any        10.190.38.99      467( 5.8)        0(-nan)        0(-nan)        0        0     0
2017-01-2222:35:10.805     2.404 any        192.168.1.89      450( 5.5)        0(-nan)        0(-nan)        0        0     0

Summary: total flows: 8115, total bytes: 0, total packets: 0, avg bps: 0, avg pps: 0, avg bpp: 0
Time window: 2017-01-2222:35:10 - 2017-01-2222:35:13
Total flows processed: 8115, Blocks skipped: 0, Bytes read: 457128
Sys: 0.009s flows/second: 829924.3   Wall: 0.008s flows/second: 967222.9

In Example 4-12, you can see the top talkers (top hosts that are sending the most traffic in the network). You can refer to the nfdump man pages for details about usage of the nfdump command (using the man nfdump command). Example 4-13 shows an excerpt of the output of the nfdump man pages showing several examples of the nfdump command usage.

Click here to view code image

Example 4-13 nfdump Man Pages Excerpt

EXAMPLES
       nfdump -r /and/dir/nfcapd.201107110845 -c 100 ‘proto tcp and ( src ip 172.16.17.18 or
dst ip 172.16.17.19 )’ Dumps the first 100 netflow records which match the given filter:
       nfdump -r /and/dir/nfcapd.201107110845 -B Map matching flows as bin-directional single flow.
       nfdump -R /and/dir/nfcapd.201107110845:nfcapd.200407110945 ‘host 192.168.1.2’ Dumps all netflow records of host 192.168.1.2 from July 11 08:45 - 09:45
       nfdump -M /to/and/dir1:dir2 -R nfcapd.200407110845:nfcapd.200407110945 -s record -n 20 Generates the Top 20 statistics from 08:45 to 09:45 from 3 sources
       nfdump -r /and/dir/nfcapd.201107110845 -s record -n 20 -o extended Generates the Top 20 statistics, extended output format
       nfdump -r /and/dir/nfcapd.201107110845 -s record -n 20 ‘in if 5 and bps > 10k’ Generates the Top 20 statistics from flows coming from interface 5
       nfdump -r /and/dir/nfcapd.201107110845 ‘inet6 and proto tcp and ( src port > 1024
and dst port 80 ) Dumps all port 80 IPv6 connections to any web server.

NOTES
       Generating the statistics for data files of a few hundred MB is no problem. However be careful if you want to create statistics of several GB of data. This may consume a lot of memory and can take a while. Flow anonymization has moved into nfanon.

SEE ALSO
       nfcapd(1), nfanon(1), nfprofile(1), nfreplay(1)

NfSen

NfSen is the graphical web-based frontend for NFdump. You can download and obtain more information about NFSen at http://nfsen.sourceforge.net.

SiLK

The SiLK analysis suite is a very popular open source command-line “Swiss army knife” developed by the Computer Emergency Response Team (CERT) at Carnegie Mellon University. Administrators and security professionals combine these tools in various ways to perform detailed NetFlow analysis. SiLK includes numerous tools and plug-ins.

The SiLK packing system includes several applications (daemons) that collect NetFlow data and translate them into a more space-efficient format. SiLK stores these records into service-specific binary flat files for use by the analysis suite. Files are organized in a time-based directory hierarchy.

Elasticsearch, Logstash, and Kibana Stack

Elasticsearch ELK stack is a very powerful open source analytics platform. ELK stands for Elasticsearch, Logstash, and Kibana.

Elasticsearch is the name of a distributed search and analytics engine, but it is also the name of the company founded by the folks behind Elasticsearch and Apache Lucene. Elasticsearch is built on top of Apache Lucene, which is a high-performance search and information retrieval library written in Java. Elasticsearch is a schema-free, full-text search engine with multilanguage support. It provides support for geolocation, suggestive search, auto-completion, and search snippets.

Logstash offers centralized log aggregation of many types, such as network infrastructure device logs, server logs, and also Netflow. Logstash is written in JRuby and runs in a Java virtual machine (JVM). It has a very simple message-based architecture. Logstash has a single agent that is configured to perform different functions in combination with the other ELK components. The following are the four major components in the Logstash ecosystem:

The shipper: Sends events to Logstash. Typically, remote agents will only run this component.

The broker and indexer: Receive and index the events.

The search and storage: Allow you to search and store events.

The web interface: A web-based interface called Kibana.

Logstash is very scalable because servers running Logstash can run one or more of these aforementioned components independently.

Kibana is an analytics and visualization platform architected for Elasticsearch. It provides real-time summary and charting of streaming data, with the ability to share and embed dashboards.

Marvel and Shield are two additional components that can be integrated with ELK:

Marvel: Provides monitoring of an Elasticsearch deployment. It uses Kibana to visualize the data. It provides a detailed explanation of things that are happening within the ELK deployment that are very useful for troubleshooting and additional analysis. You can obtain information about Marvel at http://www.elasticsearch.org/overview/marvel.

Shield: Provides security features to ELK such as role-based access control, authentication, IP filtering, encryption of ELK data, and audit logging. Shield is not free, and it requires a license. You can obtain more information about Shield at http://www.elasticsearch.org/overview/shield.

Elasticsearch also provides integration with Big Data platforms such as Hadoop.

---

NOTE

Refer to the Elasticsearch documentation at https://www.elastic.co/guide/index.xhtml for more information.

---

---

TIP

You can review examples and provide your own at Omar’s NetFlow GitHub repository at https://github.com/santosomar/netflow.

---

Exam Preparation Tasks

Review All Key Topics

Review the most important topics in the chapter, noted with the Key Topic icon in the outer margin of the page. Table 4-28 lists these key topics and the page numbers on which each is found.

Table 4-28 Key Topics

Key Topic Element	Description	Page
Summary	What is a flow?	78
List	NetFlow versions	81
List	NetFlow fields	87
Summary	What is IPFIX?	110
Summary	Comparing NetFlow and IPFIX	113
Summary	How can NetFlow be used for anomaly detection?	113
Summary	How can NetFlow be used for incident response?	114
Summary	Using NetFlow for Data Leak Detection and Prevention	119
List	Commercial NetFlow analysis tools	125
List	Open source NetFlow analysis tools	129

Print a copy of Appendix B, “Memory Tables,” (found on the book website), or at least the section for this chapter, and complete the tables and lists from memory. Appendix C, “Memory Tables Answer Key,” also on the website, includes completed tables and lists to check your work.

Define Key Terms

Define the following key terms from this chapter and check your answers in the glossary:

IPFIX

flow collector

Stream Control Transmission Protocol (SCTP)

Q&A

The answers to these questions appear in Appendix A, “Answers to the ‘Do I Know This Already’ Quizzes and Q&A.” For more practice with exam format questions, use the exam engine on the website.

1. Using NetFlow along with identity management systems, an administrator can detect which of the following? (Select all that apply.)

a. Who initiated the data transfer

b. The hosts (IP addresses) involved

c. Who configured NetFlow

d. Which RADIUS server has an active NetFlow connection

2. Network forensics can be an intimidating topic for many security professionals. Everyone knows that forensic investigation may entail many other sources of information, including end hosts, servers, and any affected systems. Each forensics team needs to have awareness of many different areas, such as which of the following? (Select all that apply.)

a. Assets, risks, impacts, and the likelihood of events

b. Incident response policies and procedures in mock events as well as NetFlow to analyze what is happening in the network

c. The current budget

d. Evidence handling and chain of custody (even NetFlow events can be used as evidence)

3. What are some telemetry sources that are good for attribution? (Select all that apply.)

a. DHCP server logs

b. VPN server logs

c. 802.1x authentication logs

d. IP route table

4. What are some of the necessary steps in order to configure Flexible NetFlow in a Cisco IOS or Cisco IOS-XE device? (Select all that apply.)

a. Configure a flow record.

b. Configure a flow monitor.

c. Configure a neighbor.

d. Apply a crypto map to an interface.

5. It is extremely important that your syslog and other messages are timestamped with the correct date and time. The use of which of the following protocols is strongly recommended?

a. SNMP

b. BGP

c. TNP

d. NTP

6. Which of the following is not an example of a Flexible NetFlow component?

a. Flow records

b. Flow monitors

c. Flow NTP

d. Flow samplers

7. Which of the following is not a component of the 5-tuple of a flow in NetFlow?

a. Source IP address

b. Destination IP address

c. Gateway

d. Source port

e. Destination port

8. Which of the following is not true about the NetFlow immediate cache?

a. It is the default cache used in many NetFlow implementations.

b. The flow accounts for a single packet.

c. It is desirable for real-time traffic monitoring and DDoS detection.

d. It is used when only very small flows are expected (NetFlow sampling).

9. Flexible NetFlow can track a wide range of Layer 2, IPv4, and IPv6 flow information, except which of the following?

a. Source and destination MAC addresses

b. ToS

c. DSCP

d. Encryption security association serial numbers

10. Which of the following statements is true about Flexible NetFlow?

a. It is supported in IPv6 and IPv4, but only when IPv6 tunnels are used.

b. It supports IPv4, but not IPv6.

c. It supports encryption of NetFlow data to a collector.

d. It uses the concept of templates.