Practical Malware Analysis · The Hands-On Guide to Dissecting Malicious Software by Sikorski, Michael -- Read -- Imperial Library of Trantor

Index

Practical Malware Analysis: The Hands-On Guide to Dissecting Malicious Software Praise for Practical Malware Analysis Warning About the Authors

About the Technical Reviewer About the Contributing Authors

Foreword Acknowledgments

Individual Thanks

Introduction

What Is Malware Analysis? Prerequisites Practical, Hands-On Learning What’s in the Book?

1. Malware Analysis Primer

The Goals of Malware Analysis Malware Analysis Techniques

Basic Static Analysis Basic Dynamic Analysis Advanced Static Analysis Advanced Dynamic Analysis

Types of Malware General Rules for Malware Analysis

I. Basic Analysis

2. Basic Static Techniques

Antivirus Scanning: A Useful First Step Hashing: A Fingerprint for Malware Finding Strings Packed and Obfuscated Malware

Packing Files Detecting Packers with PEiD

Portable Executable File Format Linked Libraries and Functions

Static, Runtime, and Dynamic Linking Exploring Dynamically Linked Functions with Dependency Walker Imported Functions Exported Functions

Static Analysis in Practice

PotentialKeylogger.exe: An Unpacked Executable PackedProgram.exe: A Dead End

The PE File Headers and Sections

Examining PE Files with PEview Viewing the Resource Section with Resource Hacker Using Other PE File Tools PE Header Summary

Conclusion Labs

Lab 1-1

Questions

Lab 1-2

Questions

Lab 1-3

Questions

Lab 1-4

Questions

3. Malware Analysis in Virtual Machines

The Structure of a Virtual Machine Creating Your Malware Analysis Machine

Configuring VMware

Disconnecting the Network Setting Up Host-Only Networking Using Multiple Virtual Machines

Using Your Malware Analysis Machine

Connecting Malware to the Internet Connecting and Disconnecting Peripheral Devices Taking Snapshots Transferring Files from a Virtual Machine

The Risks of Using VMware for Malware Analysis Record/Replay: Running Your Computer in Reverse Conclusion

4. Basic Dynamic Analysis

Sandboxes: The Quick-and-Dirty Approach

Using a Malware Sandbox Sandbox Drawbacks

Running Malware Monitoring with Process Monitor

The Procmon Display Filtering in Procmon

Viewing Processes with Process Explorer

The Process Explorer Display Using the Verify Option Comparing Strings Using Dependency Walker Analyzing Malicious Documents

Comparing Registry Snapshots with Regshot Faking a Network

Using ApateDNS Monitoring with Netcat

Packet Sniffing with Wireshark Using INetSim Basic Dynamic Tools in Practice Conclusion Labs

Lab 3-1

Questions

Lab 3-2

Questions

Lab 3-3

Questions

Lab 3-4

Questions

II. Advanced Static Analysis

5. A Crash Course in x86 Disassembly

Levels of Abstraction Reverse-Engineering The x86 Architecture

Main Memory Instructions Opcodes and Endianness Operands Registers

General Registers Flags EIP, the Instruction Pointer

Simple Instructions

Arithmetic NOP

The Stack

Function Calls Stack Layout

Conditionals Branching Rep Instructions C Main Method and Offsets More Information: Intel x86 Architecture Manuals

Conclusion

6. IDA Pro

Loading an Executable The IDA Pro Interface

Disassembly Window Modes

Graph Mode Text Mode

Useful Windows for Analysis Returning to the Default View Navigating IDA Pro

Using Links and Cross-References Exploring Your History Navigation Band Jump to Location

Searching

Using Cross-References

Code Cross-References Data Cross-References

Analyzing Functions Using Graphing Options Enhancing Disassembly

Renaming Locations Comments Formatting Operands Using Named Constants Redefining Code and Data

Extending IDA with Plug-ins

Using IDC Scripts Using IDAPython Using Commercial Plug-ins

Conclusion Labs

Lab 5-1

Questions

7. Recognizing C Code Constructs in Assembly

Global vs. Local Variables Disassembling Arithmetic Operations Recognizing if Statements

Analyzing Functions Graphically with IDA Pro Recognizing Nested if Statements

Recognizing Loops

Finding for Loops Finding while Loops

Understanding Function Call Conventions

cdecl stdcall fastcall Push vs. Move

Analyzing switch Statements

If Style Jump Table

Disassembling Arrays Identifying Structs Analyzing Linked List Traversal Conclusion Labs

Lab 6-1

Questions

Lab 6-2

Questions

Lab 6-3

Questions

Lab 6-4

Questions

8. Analyzing Malicious Windows Programs

The Windows API

Types and Hungarian Notation Handles File System Functions Special Files

Shared Files Files Accessible via Namespaces Alternate Data Streams

The Windows Registry

Registry Root Keys Regedit Programs that Run Automatically Common Registry Functions Analyzing Registry Code in Practice Registry Scripting with .reg Files

Networking APIs

Berkeley Compatible Sockets The Server and Client Sides of Networking The WinINet API

Following Running Malware

DLLs

How Malware Authors Use DLLs Basic DLL Structure

Processes

Creating a New Process

Threads

Thread Context Creating a Thread

Interprocess Coordination with Mutexes Services The Component Object Model

CLSIDs, IIDs, and the Use of COM Objects COM Server Malware

Exceptions: When Things Go Wrong

Kernel vs. User Mode The Native API Conclusion Labs

Lab 7-1

Questions

Lab 7-2

Questions

Lab 7-3

Questions

III. Advanced Dynamic Analysis

9. Debugging

Source-Level vs. Assembly-Level Debuggers Kernel vs. User-Mode Debugging Using a Debugger

Single-Stepping Stepping-Over vs. Stepping-Into Pausing Execution with Breakpoints

Software Execution Breakpoints Hardware Execution Breakpoints Conditional Breakpoints

Exceptions

First- and Second-Chance Exceptions Common Exceptions

Modifying Execution with a Debugger Modifying Program Execution in Practice Conclusion

10. OllyDbg

Loading Malware

Opening an Executable Attaching to a Running Process

The OllyDbg Interface Memory Map

Rebasing

Base Addresses Absolute vs. Relative Addresses

Viewing Threads and Stacks Executing Code Breakpoints

Software Breakpoints Conditional Breakpoints Hardware Breakpoints Memory Breakpoints

Loading DLLs Tracing

Standard Back Trace Call Stack Run Trace Tracing Poison Ivy

Exception Handling Patching Analyzing Shellcode Assistance Features Plug-ins

OllyDump Hide Debugger Command Line Bookmarks

Scriptable Debugging Conclusion Labs

Lab 9-1

Questions

Lab 9-2

Questions

Lab 9-3

Questions

11. Kernel Debugging with WinDbg

Drivers and Kernel Code Setting Up Kernel Debugging Using WinDbg

Reading from Memory Using Arithmetic Operators Setting Breakpoints Listing Modules

Microsoft Symbols

Searching for Symbols Viewing Structure Information Configuring Windows Symbols

Kernel Debugging in Practice

Looking at the User-Space Code Looking at the Kernel-Mode Code Finding Driver Objects

Rootkits

Rootkit Analysis in Practice Interrupts

Loading Drivers Kernel Issues for Windows Vista, Windows 7, and x64 Versions Conclusion Labs

Lab 10-1

Questions

Lab 10-2

Questions

Lab 10-3

Questions

IV. Malware Functionality

12. Malware Behavior

Downloaders and Launchers Backdoors

Reverse Shell

Netcat Reverse Shells Windows Reverse Shells

RATs Botnets RATs and Botnets Compared

Credential Stealers

GINA Interception Hash Dumping Keystroke Logging

Kernel-Based Keyloggers User-Space Keyloggers Identifying Keyloggers in Strings Listings

Persistence Mechanisms

The Windows Registry

AppInit_DLLs Winlogon Notify SvcHost DLLs

Trojanized System Binaries DLL Load-Order Hijacking

Privilege Escalation

Using SeDebugPrivilege

Covering Its Tracks—User-Mode Rootkits

IAT Hooking Inline Hooking

Conclusion Labs

Lab 11-1

Questions

Lab 11-2

Questions

Lab 11-3

Questions

13. Covert Malware Launching

Launchers Process Injection

DLL Injection Direct Injection

Process Replacement Hook Injection

Local and Remote Hooks Keyloggers Using Hooks Using SetWindowsHookEx Thread Targeting

Detours APC Injection

APC Injection from User Space APC Injection from Kernel Space

Conclusion Labs

Lab 12-1

Questions

Lab 12-2

Questions

Lab 12-3

Questions

Lab 12-4

Questions

14. Data Encoding

The Goal of Analyzing Encoding Algorithms Simple Ciphers

Caesar Cipher XOR

Brute-Forcing XOR Encoding Brute-Forcing Many Files NULL-Preserving Single-Byte XOR Encoding Identifying XOR Loops in IDA Pro

Other Simple Encoding Schemes Base64

Transforming Data to Base64 Identifying and Decoding Base64

Common Cryptographic Algorithms

Recognizing Strings and Imports Searching for Cryptographic Constants

Using FindCrypt2 Using Krypto ANALyzer

Searching for High-Entropy Content

Custom Encoding

Identifying Custom Encoding Advantages of Custom Encoding to the Attacker

Decoding

Self-Decoding Manual Programming of Decoding Functions Using Instrumentation for Generic Decryption

Conclusion Labs

Lab 13-1

Questions

Lab 13-2

Questions

Lab 13-3

Questions

15. Malware-Focused Network Signatures

Network Countermeasures

Observing the Malware in Its Natural Habitat Indications of Malicious Activity OPSEC = Operations Security

Safely Investigate an Attacker Online

Indirection Tactics Getting IP Address and Domain Information

Content-Based Network Countermeasures

Intrusion Detection with Snort Taking a Deeper Look

Combining Dynamic and Static Analysis Techniques

The Danger of Overanalysis Hiding in Plain Sight

Attackers Mimic Existing Protocols Attackers Use Existing Infrastructure Leveraging Client-Initiated Beaconing

Understanding Surrounding Code Finding the Networking Code Knowing the Sources of Network Content Hard-Coded Data vs. Ephemeral Data Identifying and Leveraging the Encoding Steps Creating a Signature Analyze the Parsing Routines Targeting Multiple Elements

Understanding the Attacker’s Perspective Conclusion Labs

Lab 14-1

Questions

Lab 14-2

Questions

Lab 14-3

Questions

V. Anti-Reverse-Engineering

16. Anti-Disassembly

Understanding Anti-Disassembly Defeating Disassembly Algorithms

Linear Disassembly Flow-Oriented Disassembly

Anti-Disassembly Techniques

Jump Instructions with the Same Target A Jump Instruction with a Constant Condition Impossible Disassembly NOP-ing Out Instructions with IDA Pro

Obscuring Flow Control

The Function Pointer Problem Adding Missing Code Cross-References in IDA Pro Return Pointer Abuse Misusing Structured Exception Handlers

Thwarting Stack-Frame Analysis Conclusion Labs

Lab 15-1

Questions

Lab 15-2

Questions

Lab 15-3

Questions

17. Anti-Debugging

Windows Debugger Detection

Using the Windows API Manually Checking Structures

Checking the BeingDebugged Flag Checking the ProcessHeap Flag Checking NTGlobalFlag

Checking for System Residue

Identifying Debugger Behavior

INT Scanning Performing Code Checksums Timing Checks

Using the rdtsc Instruction Using QueryPerformanceCounter and GetTickCount

Interfering with Debugger Functionality

Using TLS Callbacks Using Exceptions Inserting Interrupts

Inserting INT 3 Inserting INT 2D Inserting ICE

Debugger Vulnerabilities

PE Header Vulnerabilities The OutputDebugString Vulnerability

Conclusion Labs

Lab 16-1

Questions

Lab 16-2

Questions

Lab 16-3

Questions

18. Anti-Virtual Machine Techniques

VMware Artifacts

Bypassing VMware Artifact Searching Checking for Memory Artifacts

Vulnerable Instructions

Using the Red Pill Anti-VM Technique Using the No Pill Technique Querying the I/O Communication Port Using the str Instruction Anti-VM x86 Instructions Highlighting Anti-VM in IDA Pro Using ScoopyNG

Tweaking Settings Escaping the Virtual Machine Conclusion Labs

Lab 17-1

Questions

Lab 17-2

Questions

Lab 17-3

Questions

19. Packers and Unpacking

Packer Anatomy

The Unpacking Stub Loading the Executable Resolving Imports The Tail Jump Unpacking Illustrated

Identifying Packed Programs

Indicators of a Packed Program Entropy Calculation

Unpacking Options Automated Unpacking Manual Unpacking

Rebuilding the Import Table with Import Reconstructor Finding the OEP

Using Automated Tools to Find the OEP Finding the OEP Manually

Repairing the Import Table Manually

Tips and Tricks for Common Packers

UPX PECompact ASPack Petite WinUpack Themida

Analyzing Without Fully Unpacking Packed DLLs Conclusion Labs

VI. Special Topics

20. Shellcode Analysis

Loading Shellcode for Analysis Position-Independent Code Identifying Execution Location

Using call/pop Using fnstenv

Manual Symbol Resolution

Finding kernel32.dll in Memory Parsing PE Export Data Using Hashed Exported Names

A Full Hello World Example Shellcode Encodings NOP Sleds Finding Shellcode Conclusion Labs

Lab 19-1

Questions

Lab 19-2

Questions

Lab 19-3

Questions

21. C++ Analysis

Object-Oriented Programming

The this Pointer Overloading and Mangling Inheritance and Function Overriding

Virtual vs. Nonvirtual Functions

Use of Vtables Recognizing a Vtable

Creating and Destroying Objects Conclusion Labs

Lab 20-1

Questions

Lab 20-2

Questions

Lab 20-3

Questions

22. 64-Bit Malware

Why 64-Bit Malware? Differences in x64 Architecture

Differences in the x64 Calling Convention and Stack Usage

Leaf and Nonleaf Functions Prologue and Epilogue 64-Bit Code

64-Bit Exception Handling

Windows 32-Bit on Windows 64-Bit 64-Bit Hints at Malware Functionality Conclusion Labs

Lab 21-1

Questions

Lab 21-2

Questions

A. Important Windows Functions B. Tools for Malware Analysis C. Solutions to Labs

Lab 1-1 Solutions