Network Security Through Data Analysis · Building Situational Awareness by Collins, Michael -- Read -- Imperial Library of Trantor

Index

Network Security Through Data Analysis Preface

Audience Contents of This Book Conventions Used in This Book Using Code Examples Safari® Books Online How to Contact Us Acknowledgements

I. Data

1. Sensors and Detectors: An Introduction

Vantages: How Sensor Placement Affects Data Collection Domains: Determining Data That Can Be Collected Actions: What a Sensor Does with Data Conclusion

2. Network Sensors

Network Layering and Its Impact on Instrumentation

Network Layers and Vantage Network Layers and Addressing

Packet Data

Packet and Frame Formats Rolling Buffers Limiting the Data Captured from Each Packet Filtering Specific Types of Packets What If It’s Not Ethernet?

NetFlow

NetFlow v5 Formats and Fields

“Flow and Stuff:” NetFlow v9 and IPFIX

NetFlow Generation and Collection

Further Reading

3. Host and Service Sensors: Logging Traffic at the Source

Accessing and Manipulating Logfiles The Contents of Logfiles

The Characteristics of a Good Log Message Existing Logfiles and How to Manipulate Them

Representative Logfile Formats

HTTP: CLF and ELF SMTP Microsoft Exchange: Message Tracking Logs

Logfile Transport: Transfers, Syslog, and Message Queues

Transfer and Logfile Rotation Syslog

Further Reading

4. Data Storage for Analysis: Relational Databases, Big Data, and Other Options

Log Data and the CRUD Paradigm

Creating a Well-Organized Flat File System: Lessons from SiLK

A Brief Introduction to NoSQL Systems What Storage Approach to Use

Storage Hierarchy, Query Times, and Aging

II. Tools

5. The SiLK Suite

What Is SiLK and How Does It Work? Acquiring and Installing SiLK

The Datafiles

Choosing and Formatting Output Field Manipulation: rwcut Basic Field Manipulation: rwfilter

Ports and Protocols Size IP Addresses Time TCP Options Helper Options Miscellaneous Filtering Options and Some Hacks

rwfileinfo and Provenance Combining Information Flows: rwcount rwset and IP Sets rwuniq rwbag Advanced SiLK Facilities

pmaps

Collecting SiLK Data

YAF rwptoflow rwtuc

Further Reading

6. An Introduction to R for Security Analysts

Installation and Setup Basics of the Language

The R Prompt R Variables Writing Functions Conditionals and Iteration

Using the R Workspace Data Frames Visualization

Visualization Commands Parameters to Visualization Annotating a Visualization Exporting Visualization

Analysis: Statistical Hypothesis Testing

Hypothesis Testing Testing Data

Further Reading

7. Classification and Event Tools: IDS, AV, and SEM

How an IDS Works

Basic Vocabulary Classifier Failure Rates: Understanding the Base-Rate Fallacy Applying Classification

Improving IDS Performance

Enhancing IDS Detection Enhancing IDS Response Prefetching Data

Further Reading

8. Reference and Lookup: Tools for Figuring Out Who Someone Is

MAC and Hardware Addresses IP Addressing

IPv4 Addresses, Their Structure, and Significant Addresses IPv6 Addresses, Their Structure and Significant Addresses Checking Connectivity: Using ping to Connect to an Address Tracerouting IP Intelligence: Geolocation and Demographics

DNS

DNS Name Structure Forward DNS Querying Using dig The DNS Reverse Lookup Using whois to Find Ownership

Additional Reference Tools

DNSBLs

9. More Tools

Visualization

Graphviz

Communications and Probing

netcat nmap Scapy

Packet Inspection and Reference

Wireshark GeoIP The NVD, Malware Sites, and the C*Es Search Engines, Mailing Lists, and People

Further Reading

III. Analytics

10. Exploratory Data Analysis and Visualization

The Goal of EDA: Applying Analysis EDA Workflow Variables and Visualization Univariate Visualization: Histograms, QQ Plots, Boxplots, and Rank Plots

Histograms Bar Plots (Not Pie Charts) The Quantile-Quantile (QQ) Plot The Five-Number Summary and the Boxplot Generating a Boxplot

Bivariate Description

Scatterplots Contingency Tables

Multivariate Visualization

Operationalizing Security Visualization

Rule one: bound and partition your visualization to manage disruptions Rule two: label anomalies Rule three: use trendlines, distinguish artifacts from observations Rule four: be consistent across plots Rule five: annotate with contextual information Rule six: avoid flash in favor of expressiveness Rule seven: when performing long jobs, give the user some status feedback

Further Reading

11. On Fumbling

Attack Models Fumbling: Misconfiguration, Automation, and Scanning

Lookup Failures Automation Scanning

Identifying Fumbling

TCP Fumbling: The State Machine

Network maps Unidirectional flow filtering

ICMP Messages and Fumbling Identifying UDP Fumbling

Fumbling at the Service Level

HTTP Fumbling SMTP Fumbling

Analyzing Fumbling

Building Fumbling Alarms Forensic Analysis of Fumbling Engineering a Network to Take Advantage of Fumbling

Further Reading

12. Volume and Time Analysis

The Workday and Its Impact on Network Traffic Volume Beaconing File Transfers/Raiding Locality

DDoS, Flash Crowds, and Resource Exhaustion DDoS and Routing Infrastructure

Applying Volume and Locality Analysis

Data Selection Using Volume as an Alarm Using Beaconing as an Alarm Using Locality as an Alarm Engineering Solutions

Further Reading

13. Graph Analysis

Graph Attributes: What Is a Graph? Labeling, Weight, and Paths Components and Connectivity Clustering Coefficient Analyzing Graphs

Using Component Analysis as an Alarm Using Centrality Analysis for Forensics Using Breadth-First Searches Forensically Using Centrality Analysis for Engineering

Further Reading

14. Application Identification

Mechanisms for Application Identification

Port Number Application Identification by Banner Grabbing Application Identification by Behavior Application Identification by Subsidiary Site

Application Banners: Identifying and Classifying

Non-Web Banners Web Client Banners: The User-Agent String

Further Reading

15. Network Mapping

Creating an Initial Network Inventory and Map

Creating an Inventory: Data, Coverage, and Files Phase I: The First Three Questions

The Default Network

Phase II: Examining the IP Space

Identifying Asymmetric Traffic Identifying Dark Space Finding Network Appliances

Phase III: Identifying Blind and Confusing Traffic

Identifying NATs Identifying Proxies Identifying VPN Traffic

Phase IV: Identifying Clients and Servers

Identifying Servers

Identifying Sensing and Blocking Infrastructure

Updating the Inventory: Toward Continuous Audit Further Reading

Index Colophon Copyright

← Prev
Back
Next →

← Prev
Back
Next →