Design and Implementation of the FreeBSD Operating System, The, 2/e by McKusick, Marshall Kirk -- Read -- Imperial Library of Trantor

Index

About This eBook Title Page Copyright Page Dedication Contents Preface

UNIX-like Systems Berkeley Software Distributions Material Covered in this Book Use by Computer Professionals Use in Courses on Operating Systems Organization Getting BSD Acknowledgments References

About the Authors Part I: Overview

Chapter 1. History and Goals

1.1 History of the UNIX System

Origins Research UNIX AT&T UNIX System III and System V Berkeley Software Distributions UNIX in the World

1.2 BSD and Other Systems

The Influence of the User Community

1.3 The Transition of BSD to Open Source

Networking Release 2 The Lawsuit 4.4BSD 4.4BSD-Lite Release 2

1.4 The FreeBSD Development Model

References

Chapter 2. Design Overview of FreeBSD

2.1 FreeBSD Facilities and the Kernel

The Kernel

2.2 Kernel Organization 2.3 Kernel Services 2.4 Process Management

Signals Process Groups and Sessions

2.5 Security

Process Credentials Privilege Model Discretionary Access Control Capability Model Jail Lightweight Virtualization Mandatory Access Control Event Auditing Cryptography and Random-Number Generators

2.6 Memory Management

BSD Memory-Management Design Decisions Memory Management Inside the Kernel

2.7 I/O System Overview

Descriptors and I/O Descriptor Management Devices Socket IPC Scatter-Gather I/O Multiple Filesystem Support

2.8 Devices 2.9 The Fast Filesystem

Filestores

2.10 The Zettabyte Filesystem 2.11 The Network Filesystem 2.12 Interprocess Communication 2.13 Network-Layer Protocols 2.14 Transport-Layer Protocols 2.15 System Startup and Shutdown

Exercises References

Chapter 3. Kernel Services

3.1 Kernel Organization

System Processes System Entry Run-Time Organization Entry to the Kernel Return from the Kernel

3.2 System Calls

Result Handling Returning from a System Call

3.3 Traps and Interrupts

I/O Device Interrupts Software Interrupts

3.4 Clock Interrupts

Statistics and Process Scheduling Timeouts

3.5 Memory-Management Services 3.6 Timing Services

Real Time External Representation Adjustment of the Time Interval Time

3.7 Resource Services

Process Priorities Resource Utilization Resource Limits Filesystem Quotas

3.8 Kernel Tracing Facilities

System-Call Tracing DTrace Kernel Tracing Exercises References

Part II: Processes

Chapter 4. Process Management

4.1 Introduction to Process Management

Multiprogramming Scheduling

4.2 Process State

The Process Structure The Thread Structure

4.3 Context Switching

Thread State Low-Level Context Switching Voluntary Context Switching Synchronization Mutex Synchronization Mutex Interface Lock Synchronization Deadlock Prevention

4.4 Thread Scheduling

The Low-Level Scheduler Thread Run Queues and Context Switching Timeshare Thread Scheduling Multiprocessor Scheduling Adaptive Idle Traditional Timeshare Thread Scheduling

4.5 Process Creation 4.6 Process Termination 4.7 Signals

Posting of a Signal Delivering a Signal

4.8 Process Groups and Sessions

Process Groups Sessions Job Control

4.9 Process Debugging

Exercises References

Chapter 5. Security

5.1 Operating-System Security 5.2 Security Model

Process Model Discretionary and Mandatory Access Control Trusted Computing Base (TCB) Other Kernel-Security Features

5.3 Process Credentials

The Credential Structure Credential Memory Model Access-Control Checks

5.4 Users and Groups

Setuid and Setgid Binaries

5.5 Privilege Model

Implicit Privilege Explicit Privilege

5.6 Interprocess Access Control

Visibility Signals Scheduling Control Waiting on Process Termination Debugging

5.7 Discretionary Access Control

The Virtual-Filesystem Interface and DAC Object Owners and Groups UNIX Permissions Access Control Lists (ACLs) POSIX.1e Access Control Lists NFSv4 Access Control Lists

5.8 Capsicum Capability Model

Capsicum Application Structure Capability Systems Capabilities Capability Mode

5.9 Jails 5.10 Mandatory Access-Control Framework

Mandatory Policies Guiding Design Principles Architecture of the MAC Framework Framework Startup Policy Registration Framework Entry-Point Design Considerations Policy Entry-Point Considerations Kernel Service Entry-Point Invocation Policy Composition Object Labelling Label Life Cycle and Memory Management Label Synchronization Policy-Agnostic Label Management from Userspace

5.11 Security Event Auditing

Audit Events and Records BSM Audit Records and Audit Trails Kernel-Audit Implementation

5.12 Cryptographic Services

Cryptographic Framework Random-Number Generator

5.13 GELI Full-Disk Encryption

Confidentiality and Integrity Protection Key Management Starting GELI Cryptographic Block Protection I/O Model Limitations Exercises References

Chapter 6. Memory Management

6.1 Terminology

Processes and Memory Paging Replacement Algorithms Working-Set Model Swapping Advantages of Virtual Memory Hardware Requirements for Virtual Memory

6.2 Overview of the FreeBSD Virtual-Memory System

User Address-Space Management

6.3 Kernel Memory Management

Kernel Maps and Submaps Kernel Address-Space Allocation The Slab Allocator The Keg Allocator The Zone Allocator Kernel Malloc Kernel Zone Allocator

6.4 Per-Process Resources

FreeBSD Process Virtual-Address Space Page-Fault Dispatch Mapping to Vm_objects Vm_objects Vm_objects to Pages

6.5 Shared Memory

Mmap Model Shared Mapping Private Mapping Collapsing of Shadow Chains Private Snapshots

6.6 Creation of a New Process

Reserving Kernel Resources Duplication of the User Address Space Creation of a New Process Without Copying

6.7 Execution of a File 6.8 Process Manipulation of Its Address Space

Change of Process Size File Mapping Change of Protection

6.9 Termination of a Process 6.10 The Pager Interface

Vnode Pager Device Pager Physical-Memory Pager Swap Pager

6.11 Paging

Hardware-Cache Design Hardware Memory Management Superpages

6.12 Page Replacement

Paging Parameters The Pageout Daemon Swapping The Swap-In Process

6.13 Portability

The Role of the pmap Module Initialization and Startup Mapping Allocation and Deallocation Change of Access and Wiring Attributes for Mappings Maintenance of Physical Page-Usage Information Initialization of Physical Pages Management of Internal Data Structures Exercises References

Part III: I/O System

Chapter 7. I/O System Overview

7.1 Descriptor Management and Services

Open File Entries Management of Descriptors Asynchronous I/O File-Descriptor Locking Multiplexing I/O on Descriptors Implementation of Select Kqueues and Kevents Movement of Data Inside the Kernel

7.2 Local Interprocess Communication

Semaphores Message Queues Shared Memory

7.3 The Virtual-Filesystem Interface

Contents of a Vnode Vnode Operations Pathname Translation Exported Filesystem Services

7.4 Filesystem-Independent Services

The Name Cache Buffer Management Implementation of Buffer Management

7.5 Stackable Filesystems

Simple Filesystem Layers The Union Filesystem Other Filesystems Exercises References

Chapter 8. Devices

8.1 Device Overview

The PC I/O Architecture The Structure of the FreeBSD Mass Storage I/O Subsystem Device Naming and Access

8.2 I/O Mapping from User to Device

Device Drivers I/O Queueing Interrupt Handling

8.3 Character Devices

Raw Devices and Physical I/O Character-Oriented Devices Entry Points for Character Device Drivers

8.4 Disk Devices

Entry Points for Disk Device Drivers Sorting of Disk I/O Requests Disk Labels

8.5 Network Devices

Entry Points for Network Drivers Configuration and Control Packet Reception Packet Transmission

8.6 Terminal Handling

Terminal-Processing Modes User Interface Process Groups, Sessions, and Terminal Control Terminal Operations Terminal Output (Upper Half) Terminal Output (Lower Half) Terminal Input Closing of Terminal Devices

8.7 The GEOM Layer

Terminology and Topology Rules Changing Topology Operation Topological Flexibility

8.8 The CAM Layer

The Path of a SCSI I/O Request Through the CAM Subsystem ATA Disks

8.9 Device Configuration

Device Identification Autoconfiguration Data Structures Resource Management

8.10 Device Virtualization

Interaction with the Hypervisor Virtio Xen Device Pass-Through Exercises References

Chapter 9. The Fast Filesystem

9.1 Hierarchical Filesystem Management 9.2 Structure of an Inode

Changes to the Inode Format Extended Attributes New Filesystem Capabilities File Flags Dynamic Inodes Inode Management

9.3 Naming

Directories Finding of Names in Directories Pathname Translation Links

9.4 Quotas 9.5 File Locking 9.6 Soft Updates

Update Dependencies in the Filesystem Dependency Structures Bitmap Dependency Tracking Inode Dependency Tracking Direct-Block Dependency Tracking Indirect-Block Dependency Tracking Dependency Tracking for New Indirect Blocks New Directory-Entry Dependency Tracking New Directory Dependency Tracking Directory-Entry Removal-Dependency Tracking File Truncation File and Directory Inode Reclamation Directory-Entry Renaming Dependency Tracking Fsync Requirements for Soft Updates File-Removal Requirements for Soft Updates Soft-Updates Requirements for fsck

9.7 Filesystem Snapshots

Creating a Filesystem Snapshot Maintaining a Filesystem Snapshot Large Filesystem Snapshots Background fsck User-Visible Snapshots Live Dumps

9.8 Journaled Soft Updates

Background and Introduction Compatibility with Other Implementations Journal Format Modifications That Require Journaling Additional Requirements of Journaling The Recovery Process Performance Future Work Tracking File-Removal Dependencies

9.9 The Local Filestore

Overview of the Filestore User I/O to a File

9.10 The Berkeley Fast Filesystem

Organization of the Berkeley Fast Filesystem Boot Blocks Optimization of Storage Utilization Reading and Writing to a File Layout Policies Allocation Mechanisms Block Clustering Extent-Based Allocation Exercises References

Chapter 10. The Zettabyte Filesystem

10.1 Introduction 10.2 ZFS Organization

ZFS Dnode ZFS Block Pointers ZFS objset Structure

10.3 ZFS Structure

The MOS Layer The Object-Set Layer

10.4 ZFS Operation

Writing New Data to Disk Logging RAIDZ Snapshots ZFS Block Allocation Freeing Blocks Deduplication Remote Replication

10.5 ZFS Design Tradeoffs

Exercises References

Chapter 11. The Network Filesystem

11.1 Overview 11.2 Structure and Operation

The FreeBSD NFS Implementation Client–Server Interactions Security Issues Techniques for Improving Performance

11.3 NFS Evolution

Namespace Attributes Access Control Lists Caching, Delegation, and Callbacks Locking Security Crash Recovery Exercises References

Part IV: Interprocess Communication

Chapter 12. Interprocess Communication

12.1 Interprocess-Communication Model

Use of Sockets

12.2 Implementation Structure and Overview 12.3 Memory Management

Mbufs Storage-Management Algorithms Mbuf Utility Routines

12.4 IPC Data Structures

Socket Addresses Locks

12.5 Connection Setup 12.6 Data Transfer

Transmitting Data Receiving Data

12.7 Socket Shutdown 12.8 Network-Communication Protocol Internal Structure

Data Flow Communication Protocols

12.9 Socket-to-Protocol Interface

Protocol User-Request Routines Protocol Control-Output Routine

12.10 Protocol-to-Protocol Interface

pr_output pr_input pr_ctlinput

12.11 Protocol-to-Network Interface

Network Interfaces and Link-Layer Protocols Packet Transmission Packet Reception

12.12 Buffering and Flow Control

Protocol Buffering Policies Queue Limiting

12.13 Network Virtualization

Exercises References

Chapter 13. Network-Layer Protocols

13.1 Internet Protocol Version 4

IPv4 Addresses Broadcast Addresses Internet Multicast Link-Layer Address Resolution

13.2 Internet Control Message Protocols (ICMP) 13.3 Internet Protocol Version 6

IPv6 Addresses IPv6 Packet Formats Changes to the Socket API Autoconfiguration

13.4 Internet Protocols Code Structure

Output Input Forwarding

13.5 Routing

Kernel Routing Tables Routing Lookup Routing Redirects Routing-Table Interface User-Level Routing Policies User-Level Routing Interface: Routing Socket

13.6 Raw Sockets

Control Blocks Input Processing Output Processing

13.7 Security

IPSec Overview Security Protocols Key Management IPSec Implementation

13.8 Packet-Processing Frameworks

Berkeley Packet Filter IP Firewalls IPFW and Dummynet Packet Filter (PF) Netgraph Netmap Exercises References

Chapter 14. Transport-Layer Protocols

14.1 Internet Ports and Associations

Protocol Control Blocks

14.2 User Datagram Protocol (UDP)

Initialization Output Input Control Operations

14.3 Transmission Control Protocol (TCP)

TCP Connection States Sequence Variables

14.4 TCP Algorithms

Timers Estimation of Round-Trip Time Connection Establishment SYN Cache SYN Cookies Connection Shutdown

14.5 TCP Input Processing 14.6 TCP Output Processing

Sending Data Avoidance of the Silly-Window Syndrome Avoidance of Small Packets Delayed Acknowledgments and Window Updates Selective Acknowledgment Retransmit State Slow Start Buffer and Window Sizing Avoidance of Congestion with Slow Start Fast Retransmission Modular Congestion Control The Vegas Algorithm The Cubic Algorithm

14.7 Stream Control Transmission Protocol (SCTP)

Chunks Association Setup Data Transfer Association Shutdown Multihoming and Heartbeats Exercises References

Part V: System Operation

Chapter 15. System Startup and Shutdown

15.1 Firmware and BIOSes 15.2 Boot Loaders

Master Boot Record and Globally Unique Identifier Partition Table The Second-Stage Boot Loader: gptboot The Final-Stage Boot Loader: /boot/loader Boot Loading on Embedded Platforms

15.3 Kernel Boot

Assembly-Language Startup Platform-Specific C-Language Startup Modular Kernel Design Module Initialization Basic Kernel Services Kernel-Thread Initialization Device-Module Initialization Loadable Kernel Modules

15.4 User-Level Initialization

/sbin/init System Startup Scripts /usr/libexec/getty /usr/bin/login

15.5 System Operation

Kernel Configuration System Shutdown and Autoreboot System Debugging Passage of Information To and From the Kernel Exercises References

Glossary Index FreeBSD Kernel Internals on Video Advanced FreeBSD Course on Video FreeBSD Networking from the Bottom Up on Video CSRG Archive CD-ROMs History of UNIX at Berkeley Teaching a Course Using This Book Code Snippets

← Prev
Back
Next →

← Prev
Back
Next →