Windows Internals · 5th Edition by Solomon, David -- Read -- Imperial Library of Trantor

Index

Windows Internals, Fifth Edition

Foreword Acknowledgments Introduction

Structure of the Book History of the Book Fifth Edition Changes Hands-On Experiments Topics Not Covered A Warning and a Caveat Find Additional Content Online Support

From the Authors From Microsoft Press Questions and Comments

1. Concepts and Tools

Windows Operating System Versions Foundation Concepts and Terms

Windows API Services, Functions, and Routines Processes, Threads, and Jobs

Virtual Memory

Kernel Mode vs. User Mode Terminal Services and Multiple Sessions Objects and Handles

Security

Registry Unicode

Digging into Windows Internals

Reliability and Performance Monitor Kernel Debugging Symbols for Kernel Debugging Debugging Tools for Windows LiveKd Tool Windows Software Development Kit Windows Driver Kit Sysinternals Tools

Conclusion

2. System Architecture

Requirements and Design Goals Operating System Model Architecture Overview

Portability Symmetric Multiprocessing Scalability

Differences Between Client and Server Versions

Checked Build

Key System Components Environment Subsystems and Subsystem DLLs

Windows Subsystem POSIX Subsystem Ntdll.dll Executive Kernel Kernel Objects Kernel Processor Control Region and Control Block (KPCR and KPRCB) Hardware Support Hardware Abstraction Layer Device Drivers Windows Driver Model (WDM) Windows Driver Foundation System Processes Idle Process Interrupts and DPCs System Process and System Threads Session Manager (Smss) Winlogon, LogonUI, LSASS, and Userinit Service Control Manager (SCM)

Conclusion

3. System Mechanisms

Trap Dispatching

Interrupt Dispatching Hardware Interrupt Processing x86 Interrupt Controllers x64 Interrupt Controllers IA64 Interrupt Controllers Software Interrupt Request Levels (IRQLs) Software Interrupts Exception Dispatching Unhandled Exceptions Windows Error Reporting

System Service Dispatching

32-Bit System Service Dispatching 64-Bit System Service Dispatching Kernel-Mode System Service Dispatching Service Descriptor Tables

Object Manager

Executive Objects Object Structure Object Headers and Bodies Type Objects Object Methods Object Handles and the Process Handle Table Object Security Object Retention Resource Accounting Object Names Session Namespace Object Filtering

Synchronization

High-IRQL Synchronization Interlocked Operations Spinlocks Queued Spinlocks Instack Queued Spinlocks Executive Interlocked Operations Low-IRQL Synchronization Kernel Dispatcher Objects Keyed Events Fast Mutexes and Guarded Mutexes Executive Resources Pushlocks Critical Sections Condition Variables Slim Reader Writer Locks Run Once Initialization

System Worker Threads Windows Global Flags Advanced Local Procedure Calls (ALPCs) Kernel Event Tracing Wow64

Wow64 Process Address Space Layout System Calls Exception Dispatching User Callbacks File System Redirection Registry Redirection and Reflection I/O Control Requests 16-Bit Installer Applications Printing Restrictions

User-Mode Debugging

Kernel Support Native Support Windows Subsystem Support

Image Loader

Early Process Initialization Loaded Module Database Import Parsing Post Import Process Initialization

Hypervisor (Hyper-V)

Partitions Root Partition Root Partition Operating System VM Service and Worker Processes Virtualization Service Providers VM Infrastructure Driver and Hypervisor API Library Hypervisor Child Partitions Virtualization Service Clients Enlightenments Hardware Emulation and Support Emulated Devices Synthetic Devices Virtual Processors Memory Virtualization Intercepts

Kernel Transaction Manager Hotpatch Support Kernel Patch Protection Code Integrity Conclusion

4. Management Mechanisms

The Registry

Viewing and Changing the Registry Registry Usage Registry Data Types Registry Logical Structure

HKEY_CURRENT_USER HKEY_USERS HKEY_CLASSES_ROOT HKEY_LOCAL_MACHINE HKEY_CURRENT_CONFIG HKEY_PERFORMANCE_DATA

Transactional Registry (TxR) Monitoring Registry Activity

Process Monitor Internals Process Monitor Troubleshooting Techniques Logging Activity in Unprivileged Accounts or During Logon/Logoff

Registry Internals

Hives Hive Size Limits Hive Structure Cell Maps The Registry Namespace and Operation Stable Storage Registry Filtering Registry Optimizations

Services

Service Applications

Service Accounts The Local System Account The Network Service Account The Local Service Account Running Services in Alternate Accounts Running with Least Privilege Service Isolation Interactive Services and Session 0 Isolation

The Service Control Manager Service Startup Startup Errors Accepting the Boot and Last Known Good Service Failures Service Shutdown Shared Service Processes Service Tags Service Control Programs

Windows Management Instrumentation

WMI Architecture

Providers The Common Information Model and the Managed Object Format Language

The WMI Namespace

Class Association WMI Implementation WMI Security

Windows Diagnostic Infrastructure

WDI Instrumentation Diagnostic Policy Service Diagnostic Functionality

Conclusion

5. Processes, Threads, and Jobs

Process Internals

Data Structures Kernel Variables Performance Counters Relevant Functions

Protected Processes

Flow of CreateProcess Stage 1: Converting and Validating Parameters and Flags Stage 2: Opening the Image to Be Executed Stage 3: Creating the Windows Executive Process Object (PspAllocateProcess)

Stage 3A: Setting Up the EPROCESS Block Stage 3B: Creating the Initial Process Address Space Stage 3C: Creating the Kernel Process Block Stage 3D: Concluding the Setup of the Process Address Space Stage 3E: Setting Up the PEB Stage 3F: Completing the Setup of the Executive Process Object (PspInsertProcess)

Stage 4: Creating the Initial Thread and Its Stack and Context Stage 5: Performing Windows Subsystem–Specific Post-Initialization Stage 6: Starting Execution of the Initial Thread Stage 7: Performing Process Initialization in the Context of the New Process

Thread Internals

Data Structures Kernel Variables Performance Counters Relevant Functions Birth of a Thread

Examining Thread Activity

Limitations on Protected Process Threads

Worker Factories (Thread Pools) Thread Scheduling

Overview of Windows Scheduling Priority Levels Windows Scheduling APIs Relevant Tools Real-Time Priorities Thread States

Dispatcher Database

Quantum

Quantum Accounting Controlling the Quantum Quantum Boosting Quantum Settings Registry Value

Scheduling Scenarios

Voluntary Switch Preemption Quantum End Termination

Context Switching Idle Thread Priority Boosts

Priority Boosting after I/O Completion Boosts After Waiting for Events and Semaphores Boosts During Waiting on Executive Resources Priority Boosts for Foreground Threads After Waits Priority Boosts After GUI Threads Wake Up Priority Boosts for CPU Starvation Priority Boosts for MultiMedia Applications and Games (MMCSS)

Multiprocessor Systems

Hyperthreaded and Multicore Systems NUMA Systems Affinity Ideal and Last Processor Dynamic Processor Addition and Replacement

Multiprocessor Thread-Scheduling Algorithms

Choosing a Processor for a Thread When There Are Idle Processors Choosing a Processor for a Thread When There Are No Idle Processors Selecting a Thread to Run on a Specific CPU

CPU Rate Limits

Job Objects Conclusion

6. Security

Security Ratings

Trusted Computer System Evaluation Criteria The Common Criteria

Security System Components Protecting Objects

Access Checks

Security Identifiers (SIDs) Integrity Levels Tokens Impersonation Restricted Tokens Filtered Admin Token

Security Descriptors and Access Control

ACL Assignment Determining Access

Account Rights and Privileges

Account Rights Privileges Super Privileges

Security Auditing Logon

Winlogon Initialization User Logon Steps

User Account Control

Virtualization

File Vir tualization Registry Virtualization

Elevation

Running with Administrator Rights Requesting Administrative Rights

Software Restriction Policies Conclusion

7. I/O System

I/O System Components

The I/O Manager Typical I/O Processing

Device Drivers

Types of Device Drivers

WDM Drivers Layered Drivers

Structure of a Driver Driver Objects and Device Objects Opening Devices

I/O Processing

Types of I/O

Synchronous and Asynchronous I/O Fast I/O Mapped File I/O and File Caching Scatter/Gather I/O I/O Request Packets IRP Stack Locations IRP Buffer Management

I/O Request to a Single-Layered Driver

Servicing an Interrupt Completing an I/O Request Synchronization

I/O Requests to Layered Drivers

Thread Agnostic I/O

I/O Cancellation

User-Initiated I/O Cancellation I/O Cancellation for Thread Termination

I/O Completion Ports

The IoCompletion Object Using Completion Ports I/O Completion Port Operation

I/O Prioritization

I/O Priorities Bandwidth Reservation (Scheduled File I/O)

Driver Verifier

Kernel-Mode Driver Framework (KMDF)

Structure and Operation of a KMDF Driver KMDF Data Model KMDF I/O Model

User-Mode Driver Framework (UMDF) The Plug and Play (PnP) Manager

Level of Plug and Play Support Driver Support for Plug and Play Driver Loading, Initialization, and Installation

The Start Value Device Enumeration Devnodes Devnode Driver Loading

Driver Installation

The Power Manager

Power Manager Operation Driver Power Operation Driver and Application Control of Device Power

Conclusion

8. Storage Management

Storage Terminology Disk Drivers

Winload Disk Class, Port, and Miniport Drivers

iSCSI Drivers Multipath I/O (MPIO) Drivers

Disk Device Objects Partition Manager

Volume Management

Basic Disks

MBR-Style Partitioning GUID Partition Table Partitioning Basic Disk Volume Manager

Dynamic Disks

The LDM Database LDM and GPT or MBR-Style Partitioning Dynamic Disk Volume Manager

Multipartition Volume Management

Spanned Volumes Striped Volumes Mirrored Volumes RAID-5 Volumes

The Volume Namespace

The Mount Manager Mount Points Volume Mounting

Volume I/O Operations Virtual Disk Service

BitLocker Drive Encryption

BitLocker Architecture Encryption Keys Trusted Platform Module (TPM) BitLocker Boot Process BitLocker Key Recovery Full Volume Encryption Driver BitLocker Management

Volume Shadow Copy Service

Shadow Copies

Clone Shadow Copies Copy-on-Write Shadow Copies

VSS Architecture VSS Operation

Shadow Copy Provider

Uses in Windows

Backup Previous Versions and System Restore Shadow Copies for Shared Folders

Conclusion

9. Memory Management

Introduction to the Memory Manager Memory Manager Components Internal Synchronization Examining Memory Usage Services the Memory Manager Provides

Large and Small Pages

Reserving and Committing Pages Locking Memory Allocation Granularity

Shared Memory and Mapped Files Protecting Memory No Execute Page Protection Software Data Execution Prevention Copy-on-Write Address Windowing Extensions

Kernel-Mode Heaps (System Memory Pools)

Pool Sizes Monitoring Pool Usage

Look-Aside Lists Heap Manager

Types of Heaps Heap Manager Structure Heap Synchronization The Low Fragmentation Heap Heap Security Features Heap Debugging Features Pageheap

Virtual Address Space Layouts

x86 Address Space Layouts x86 System Address Space Layout x86 Session Space System Page Table Entries 64-Bit Address Space Layouts 64-Bit Virtual Addressing Limitations Dynamic System Virtual Address Space Management System Virtual Address Space Quotas

User Address Space Layout

Image Randomization Stack Randomization Heap Randomization

Address Translation

x86 Virtual Address Translation Page Directories Page Tables and Page Table Entries Byte Within Page Translation Look-Aside Buffer Physical Address Extension (PAE) IA64 Virtual Address Translation x64 Virtual Address Translation

Page Fault Handling

Invalid PTEs Prototype PTEs In-Paging I/O Collided Page Faults Clustered Page Faults Page Files

Stacks

User Stacks Kernel Stacks DPC Stack

Virtual Address Descriptors

Process VADs Rotate VADs

NUMA Section Objects Driver Verifier Page Frame Number Database

Page List Dynamics Page Priority Modified Page Writer PFN Data Structures

Physical Memory Limits

Windows Client Memory Limits 32-Bit Client Effective Memory Limits

Working Sets

Demand Paging Logical Prefetcher Placement Policy Working Set Management Balance Set Manager and Swapper System Working Set Memory Notification Events

Proactive Memory Management (SuperFetch)

Components Tracing and Logging Scenarios Page Priority and Rebalancing Robust Performance ReadyBoost ReadyDrive

Conclusion

10. Cache Manager

Key Features of the Cache Manager

Single, Centralized System Cache The Memory Manager Cache Coherency Virtual Block Caching Stream-Based Caching Recoverable File System Support

Cache Virtual Memory Management Cache Size

Cache Virtual Size Cache Working Set Size Cache Physical Size

Cache Data Structures

Systemwide Cache Data Structures Per-File Cache Data Structures

File System Interfaces

Copying to and from the Cache Caching with the Mapping and Pinning Interfaces Caching with the Direct Memory Access Interfaces

Fast I/O Read Ahead and Write Behind

Intelligent Read-Ahead Write-Back Caching and Lazy Writing Disabling Lazy Writing for a File Forcing the Cache to Write Through to Disk Flushing Mapped Files Write Throttling System Threads

Conclusion

11. File Systems

Windows File System Formats

CDFS UDF FAT12, FAT16, and FAT32 exFAT NTFS

File System Driver Architecture

Local FSDs Remote FSDs File System Operation

Explicit File I/O Memory Manager's Modified and Mapped Page Writer Cache Manager's Lazy Writer Cache Manager's Read-Ahead Thread Memory Manager's Page Fault Handler

File System Filter Drivers

Process Monitor

Troubleshooting File System Problems

Process Monitor Basic vs. Advanced Modes Process Monitor Troubleshooting Techniques

Common Log File System

Marshalling Log Types Log Layout Log Sequence Numbers Log Blocks Owner Pages Translating Virtual LSNs to Physical LSNs Management Policies

NTFS Design Goals and Features

High-End File System Requirements

Recoverability Security Data Redundancy and Fault Tolerance

Advanced Features of NTFS

Multiple Data Streams Unicode-Based Names General Indexing Facility Dynamic Bad-Cluster Remapping Hard Links Symbolic (Soft) Links and Junctions Compression and Sparse Files Change Logging Per-User Volume Quotas Link Tracking Encryption POSIX Support Defragmentation Dynamic Partitioning

NTFS File System Driver NTFS On-Disk Structure

Volumes Clusters Master File Table File Reference Numbers File Records File Names Resident and Nonresident Attributes Data Compression and Sparse Files

Compressing Sparse Data Compressing Nonsparse Data Sparse Files

The Change Journal File Indexing Object IDs Quota Tracking Consolidated Security Reparse Points Transaction Support

Isolation Transactional APIs Resource Managers On-Disk Implementation Logging Implementation Recovery Implementation

NTFS Recovery Support

Design Metadata Logging

Log File Service Log Record Types

Recovery

Analysis Pass Redo Pass Undo Pass

NTFS Bad-Cluster Recovery Self-Healing

Encrypting File System Security

Encrypting a File for the First Time

Constructing Key Rings Encrypting File Data Encryption Process Summary

The Decryption Process

Decrypted FEK Caching Decrypting File Data

Backing Up Encrypted Files

Conclusion

12. Networking

Windows Networking Architecture

The OSI Reference Model Windows Networking Components

Networking APIs

Windows Sockets

Winsock Client Operation Winsock Server Operation Winsock Extensions Extending Winsock Winsock Implementation

Winsock Kernel (WSK)

WSK Implementation

Remote Procedure Call

RPC Operation RPC Security RPC Implementation

Web Access APIs

WinInet WinHTTP HTTP

Named Pipes and Mailslots

Named Pipe Operation Mailslot Operation Named Pipe and Mailslot Implementation

NetBIOS

NetBIOS Names NetBIOS Operation NetBIOS API Implementation

Other Networking APIs

BITS Peer-to-Peer Infrastructure DCOM Message Queuing UPnP with PnP-X

Multiple Redirector Support

Multiple Provider Router Multiple UNC Provider

Name Resolution

Domain Name System Windows Internet Name Service Peer Name Resolution Protocol

PNRP Resolution and Publication

Location and Topology

Network Location Awareness (NLA) Link-Layer Topology Discovery (LLTD)

Protocol Drivers

Windows Filtering Platform (WFP)

Network Address Translation IP Filtering Internet Protocol Security

NDIS Drivers

Variations on the NDIS Miniport Connection-Oriented NDIS Remote NDIS QoS

Binding Layered Network Services

Remote Access Active Directory Network Load Balancing Distributed File System and DFS Replication

Conclusion

13. Startup and Shutdown

Boot Process

BIOS Preboot The BIOS Boot Sector and Bootmgr The EFI Boot Process Initializing the Kernel and Executive Subsystems Smss, Csrss, and Wininit ReadyBoot Images That Start Automatically

Troubleshooting Boot and Startup Problems

Last Known Good Safe Mode

Driver Loading in Safe Mode Safe-Mode-Aware User Programs Boot Logging in Safe Mode

Windows Recovery Environment (WinRE) Solving Common Boot Problems

MBR Corruption Boot Sector Corruption BCD Misconfiguration System File Corruption System Hive Corruption Post–Splash Screen Crash or Hang

Shutdown Conclusion

14. Crash Dump Analysis

Why Does Windows Crash? The Blue Screen Troubleshooting Crashes Crash Dump Files

Crash Dump Generation

Windows Error Reporting Online Crash Analysis Basic Crash Dump Analysis

Notmyfault Basic Crash Dump Analysis Verbose Analysis

Using Crash Troubleshooting Tools

Buffer Overrun, Memory Corruptions, and Special Pool Code Overwrite and System Code Write Protection

Advanced Crash Dump Analysis

Stack Trashes Hung or Unresponsive Systems When There Is No Crash Dump

Conclusion

Glossary

← Prev
Back
Next →

← Prev
Back
Next →