Network Flow Analysis by Lucas, Michael W. -- Read -- Imperial Library of Trantor

Index

Network Flow Analysis

ACKNOWLEDGMENTS INTRODUCTION

Network Administration and Network Management Network Management Tools

MRTG, Cricket, and Cacti RTG Nagios and Big Brother CiscoWorks, OpenView, and More

Enough Griping: What's the Solution? Flow-Tools and Its Prerequisites Flows and This Book

1. FLOW FUNDAMENTALS

What Is a Flow? Flow System Architecture The History of Network Flow

NetFlow Versions

NetFlow Version 1 NetFlow Version 5 NetFlow Version 7 NetFlow Version 8 NetFlow Version 9

NetFlow Competition The Latest Standards

Flows in the Real World

ICMP Flows UDP Flows TCP Flows Other Protocols

Flow Export and Timeouts Packet-Sampled Flows

2. COLLECTORS AND SENSORS

Collector Considerations

Operating System System Resources

Sensor Considerations

Location

Internet Border Ethernet Core

From Remote Facilities From Private Network Segments/DMZs

Implementing the Collector Installing Flow-tools

Installing from Packages Installing from Source

Running flow-capture

Starting flow-capture at Boot

How Many Collectors? Collector Log Files Collector Troubleshooting Configuring Hardware Flow Sensors

Cisco Routers Cisco Switches Juniper Routers

Configuring Software Flow Sensors

Setting Up Sensor Server Hardware Network Setup Sensor Server Setup Running the Sensor on the Collector

The Sensor: softflowd

Running softflowd Watching softflowd

Viewing Tracked Flows Viewing Flow Statistics

3. VIEWING FLOWS

Using flow-print

Printing Protocol and Port Names Common Protocol and Port Number Assignments Viewing Flow Record Header Information with -p Printing to a Wide Terminal

Setting flow-print Formats with -f

Showing Interfaces and Ports in Hex with Format -f 0 Two Lines with Times, Flags, and Hex Ports Using -f 1 Printing BGP Information Wide-Screen Display IP Accounting Format

TCP Control Bits and Flow Records ICMP Types and Codes and Flow Records

Types and Codes in ICMP Flows and ICMP Details

4. FILTERING FLOWS

Filter Fundamentals

Common Primitives Creating a Simple Filter with Conditions and Primitives Using Your Filter

Useful Primitives

Protocol, Port, and Control Bit Primitives

IP Protocol Primitives Port Number Primitives TCP Control Bit Primitives ICMP Type and Code Primitives

IP Address and Subnet Primitives

IP Addresses Subnet Primitives

Time, Counter, and Double Primitives

Comparison Operators in Primitives Time Primitives Counter Primitives Double Primitives

Interface and BGP Primitives

Identifying Interface Numbers Using SNMP Interface Number Primitive Autonomous System Primitives

Filter Match Statements

Protocols, Ports, and Control Bits

Network Protocol Filters Source or Destination Port Filters TCP Control Bit Filters ICMP Type and Code Filters

Addresses and Subnets Filtering by Sensor or Exporter Time Filters Clipping Levels

Octets, Packets, and Duration Filters Packets or Bits per Second Filters

BGP and Routing Filters

Autonomous System Number Filters Next-Hop Address Filters Interface Filters

Using Multiple Filters Logical Operators in Filter Definitions

Logical "or" Filter Inversion

Filters and Variables

Using Variable-Driven Filters Defining Your Own Variable-Driven Filters Creating Your Own Variables

5. REPORTING AND FOLLOW-UP ANALYSIS

Default Report

Timing and Totals Packet Size Distribution Packets per Flow Octets in Each Flow Flow Time Distribution

Modifying the Default Report

Using Variables: Report Type Using Variables: SORT

Analyzing Individual Flows from Reports Other Report Customizations

Choosing Fields Displaying Headers, Hostnames, and Percentages Presenting Reports in HTML

Useful Report Types

IP Address Reports

Highest Data Exchange: ip-address Flows by Recipient: ip-destination-address Most Connected Source: ip-source-address-destination-count Most Connected Destination: ip-destination-address-source-count

Network Protocol and Port Reports

Ports Used: ip-port Flow Origination: ip-source-port Flow Termination: ip-destination-port Individual Connections: ip-source/destination-port Network Protocols: ip-protocol

Traffic Size Reports

Packet Size: packet-size Bytes per Flow: octets Packets per Flow: packets

Traffic Speed Reports

Counting Packets: pps Traffic at a Given Time: linear-interpolated-flows-octets-packets

Routing, Interfaces, and Next Hops

Interfaces and Flow Data The First Interface: input-interface The Last Interface: output-interface The Throughput Matrix: input/output-interface The Next Address: ip-next-hop-address Where Traffic Comes from and How It Gets There: ip-source-address/output-interface Where Traffic Goes, and How It Gets There: ip-destination-address/input-interface Other Address and Interface Reports

Reporting Sensor Output BGP Reports

Using AS Information Traffic's Network of Origin: source-as Destination Network: destination-as BGP Reports and Friendly Names

Customizing Reports

Custom Report: Reset-Only Flows

Report Format and Output Removing Columns Applying Filters to Reports Combining stat-reports and stat-definitions

More Report Customizations

Reversing Sampling Filters in stat-report Statements Reporting by BGP Routing

Customizing Report Appearance

flow-rptfmt Options Dump CSV to a File Using Time to Direct Output Set Sorting Order Cropping Output Other Output Options Alternate Configuration Files

6. PERL, FLOWSCAN, AND CFLOW.PM

Installing Cflow.pm

Testing Cflow.pm Install from Operating System Package Install from Source Installing from Source with a Big Hammer

flowdumper and Full Flow Information FlowScan and CUFlow FlowScan Prerequisites Installing FlowScan and CUFlow

FlowScan User, Group, and Data Directories FlowScan Startup Script Configuring FlowScan Configuring CUFlow: CUFlow.cf

Subnet Network OutputDir Scoreboard AggregateScore Router Service Protocol AS

Rotation Programs and flow-capture Running FlowScan FlowScan File Handling Displaying CUFlow Graphs

Flow Record Splitting and CUFlow

Splitting Flows Scripting Flow Record Splitting Filtered CUFlow and Directory Setup

Using Cflow.pm

A Sample Cflow.pm Script Cflow.pm Variables Other Cflow.pm Exports Acting on Every File Return Value Verbose Mode

7. FLOWVIEWER

FlowTracker and FlowGrapher vs. CUFlow FlowViewer Security Installing FlowViewer

Prerequisites FlowViewer Installation Process

Configuring FlowViewer

Directories and Site Paths Website Setup Devices and Exporters

One Collector per Sensor One Collector for All Sensors

Troubleshooting the FlowViewer Suite

Using FlowViewer

Filtering Flows with FlowViewer

Device Next Hop IP Start and End Date and Time TOS Field, TCP Flag, and Protocol Source and Dest IP Source and Dest Interface Source and Dest Port and AS

Reporting Parameters

Include Flow If Sort Field, Resolve Addresses, and Oct Conv, and Sampling Multip Pie Charts Cutoffs

Printed Reports Statistics Reports

FlowGrapher

FlowGrapher Settings

Detail Lines Graph Width Sample Time Graph Type

FlowGrapher Output

FlowTracker

FlowTracker Processes FlowTracker Settings

Tracking Set Label Tracking Type Sampling Multiplier Alert Threshold Alert Frequency Alert Destination General Comment

Viewing Trackers Group Trackers

Interface Names and FlowViewer

8. AD HOC FLOW VISUALIZATION

gnuplot 101

Starting gnuplot gnuplot Configuration Files

Time-Series Example: Bandwidth

Total Bandwidth Report

Filtering Flows for Total Traffic The Target Graph The First Graph: Missing the Target Changing How the Graph Is Drawn Clipping Levels Printing Graphs to Files Save Your Work!

Unidirectional Bandwidth Reports

Filtering Flows for Unidirectional Traffic Creating a Unidirectional Graph

Combined Inbound/Outbound Traffic

Preparing the Data Files Displaying Two Graphs Simultaneously

Automating Graph Production Comparison Graphs

Data Normalizing Time Scale

9. EDGES AND ANALYSIS

NetFlow v9

Installing flowd Configuring flowd Converting flowd Data to Flow-tools

sFlow

Configuring sFlow Export with sflowenable Convert sFlow to NetFlow

Problem Solving with Flow Data

Finding Busted Software

Broken Connection Filters Checking for Resets Checking for Failed Connections

Identifying Worms Traffic to Illegal Addresses Traffic to Nonexistent Hosts

Afterword

UPDATES

← Prev
Back
Next →

← Prev
Back
Next →