The Linux Programming Interface by Kerrisk, Michael -- Read -- Imperial Library of Trantor

Index

The Linux Programming Interface

Praise for The Linux Programming Interface Preface

Subject Intended audience Linux and UNIX Usage and organization Example programs Exercises Standards and portability Linux kernel and C library versions Using the programming interface from other languages About the author Acknowledgements Permissions Web site and source code of example programs Feedback

1. History and Standards

A Brief History of UNIX and C

UNIX First through Sixth editions The birth of BSD and System V

A Brief History of Linux

The GNU Project The Linux Kernel

An aside: the BSDs Linux kernel version numbers Ports to other hardware architectures Linux distributions

Standardization

The C Programming Language The First POSIX Standards

POSIX.1 and POSIX.2 FIPS 151-1 and FIPS 151-2

X/Open Company and The Open Group SUSv3 and POSIX.1-2001

POSIX conformance, XSI conformance, and the XSI extension Unspecified and weakly specified LEGACY features

SUSv4 and POSIX.1-2008 UNIX Standards Timeline Implementation Standards Linux, Standards, and the Linux Standard Base

Summary

Further information

2. Fundamental Concepts

The Core Operating System: The Kernel

Tasks performed by the kernel Kernel mode and user mode Process versus kernel views of the system Further information

The Shell Users and Groups

Users Groups Superuser

Single Directory Hierarchy, Directories, Links, and Files

File types Directories and links Symbolic links Filenames Pathnames Current working directory File ownership and permissions

File I/O Model

File descriptors The stdio library

Programs

Filters Command-line arguments

Processes

Process memory layout Process creation and program execution Process ID and parent process ID Process termination and termination status Process user and group identifiers (credentials) Privileged processes Capabilities The init process Daemon processes Environment list Resource limits

Memory Mappings Static and Shared Libraries

Static libraries Shared libraries

Interprocess Communication and Synchronization Signals Threads Process Groups and Shell Job Control Sessions, Controlling Terminals, and Controlling Processes Pseudoterminals Date and Time Client-Server Architecture Realtime The /proc File System Summary

3. System Programming Concepts

System Calls Library Functions The Standard C Library; The GNU C Library (glibc)

Determining the version of glibc on the system

Handling Errors from System Calls and Library Functions

Handling system call errors Handling errors from library functions

Notes on the Example Programs in This Book

Command-Line Options and Arguments Common Functions and Header Files

Common header file Error-diagnostic functions Functions for parsing numeric command-line arguments

Portability Issues

Feature Test Macros

_POSIX_C_SOURCE, _XOPEN_SOURCE, and POSIX.1/SUS Feature test macros in function prototypes and source code examples

System Data Types

Printing system data type values

Miscellaneous Portability Issues

Initializing and using structures Using macros that may not be present on all implementations Variation in required header files across implementations

Summary Exercise

4. File I/O: The Universal I/O Model

Overview Universality of I/O Opening a File: open()

File descriptor number returned by open()

The open() flags Argument Errors from open() The creat() System Call

Reading from a File: read() Writing to a File: write() Closing a File: close() Changing the File Offset: lseek()

File holes Example program

Operations Outside the Universal I/O Model: ioctl() Summary Exercises

5. File I/O: Further Details

Atomicity and Race Conditions

Creating a file exclusively Appending data to a file

File Control Operations: fcntl() Open File Status Flags Relationship Between File Descriptors and Open Files Duplicating File Descriptors File I/O at a Specified Offset: pread() and pwrite() Scatter-Gather I/O: readv() and writev()

Scatter input Gather output Performing scatter-gather I/O at a specified offset

Truncating a File: truncate() and ftruncate() Nonblocking I/O I/O on Large Files

The transitional LFS API The _FILE_OFFSET_BITS macro Passing off_t values to printf()

The /dev/fd Directory Creating Temporary Files Summary Exercises

6. Processes

Processes and Programs Process ID and Parent Process ID Memory Layout of a Process Virtual Memory Management The Stack and Stack Frames Command-Line Arguments (argc, argv) Environment List

Accessing the environment from a program

Modifying the environment Example program

Performing a Nonlocal Goto: setjmp() and long jmp()

Example program Restrictions on the use of setjmp() Abusing longjmp() Problems with optimizing compilers Avoid setjmp() and long jmp() where possible

Summary

Further information

Exercises

7. Memory Allocation

Allocating Memory on the Heap

Adjusting the Program Break: brk() and sbrk() Allocating Memory on the Heap: malloc() and free()

Example program To free() or not to free()?

Implementation of malloc() and free()

Tools and libraries for malloc debugging Controlling and monitoring the malloc package

Other Methods of Allocating Memory on the Heap

Allocating memory with calloc() and realloc() Allocating aligned memory: memalign() and posix_memalign()

Allocating Memory on the Stack: alloca() Summary Exercises

8. Users and Groups

The Password File: /etc/passwd The Shadow Password File: /etc/shadow The Group File: /etc/group Retrieving User and Group Information

Retrieving records from the password file Retrieving records from the group file Example program Scanning all records in the password and group files Retrieving records from the shadow password file

Password Encryption and User Authentication

Example program

Summary Exercises

9. Process Credentials

Real User ID and Real Group ID Effective User ID and Effective Group ID Set-User-ID and Set-Group-ID Programs Saved Set-User-ID and Saved Set-Group-ID File-System User ID and File-System Group ID Supplementary Group IDs Retrieving and Modifying Process Credentials

Retrieving and Modifying Real, Effective, and Saved Set IDs

Retrieving real and effective IDs Modifying effective IDs Modifying real and effective IDs Retrieving real, effective, and saved set IDs Modifying real, effective, and saved set IDs

Retrieving and Modifying File-System IDs Retrieving and Modifying Supplementary Group IDs Summary of Calls for Modifying Process Credentials Example: Displaying Process Credentials

Summary Exercises

10. Time

Calendar Time Time-Conversion Functions

Converting time_t to Printable Form Converting Between time_t and Broken-Down Time Converting Between Broken-Down Time and Printable Form

Converting from broken-down time to printable form Converting from printable form to broken-down time

Timezones

Timezone definitions Specifying the timezone for a program

Locales

Locale definitions Specifying the locale for a program

Updating the System Clock The Software Clock (Jiffies) Process Time

Example program

Summary

Further information

Exercise

11. System Limits and Options

System Limits

Runtime invariant values (possibly indeterminate) Pathname variable values Runtime increasable values Summary of selected SUSv3 limits Determining limits and options from the shell: getconf

Retrieving System Limits (and Options) at Run Time Retrieving File-Related Limits (and Options) at Run Time Indeterminate Limits System Options Summary

Further information

Exercises

12. System and Process Information

The /proc File System

Obtaining Information About a Process: /proc/PID

The /proc/PID/fd directory Threads: the /proc/PID/task directory

System Information Under /proc Accessing /proc Files

Accessing files in /proc/PID Example program

System Identification: uname() Summary

Further information

Exercises

13. File I/O Buffering

Kernel Buffering of File I/O: The Buffer Cache

Effect of buffer size on I/O system call performance

Buffering in the stdio Library

Setting the buffering mode of a stdio stream Flushing a stdio buffer

Controlling Kernel Buffering of File I/O

Synchronized I/O data integrity and synchronized I/O file integrity System calls for controlling kernel buffering of file I/O Making all writes synchronous: O_SYNC Performance impact of O_SYNC The O_DSYNC and O_RSYNC flags

Summary of I/O Buffering Advising the Kernel About I/O Patterns Bypassing the Buffer Cache: Direct I/O

Alignment restrictions for direct I/O Example program

Mixing Library Functions and System Calls for File I/O Summary

Further information

Exercises

14. File Systems

Device Special Files (Devices)

Device IDs

Disks and Partitions

Disk drives Disk partitions

File Systems

The ext2 file system File-system structure

I-nodes

I-nodes and data block pointers in ext2

The Virtual File System (VFS) Journaling File Systems Single Directory Hierarchy and Mount Points Mounting and Unmounting File Systems

Mounting a File System: mount()

Example program

Unmounting a File System: umount() and umount2()

Advanced Mount Features

Mounting a File System at Multiple Mount Points Stacking Multiple Mounts on the Same Mount Point Mount Flags That Are Per-Mount Options Bind Mounts Recursive Bind Mounts

A Virtual Memory File System: tmpfs Obtaining Information About a File System: statvfs() Summary

Further information

Exercise

15. File Attributes

Retrieving File Information: stat()

Device IDs and i-node number File ownership Link count File type and permissions File size, blocks allocated, and optimal I/O block size File timestamps Example program

File Timestamps

Nanosecond timestamps

Changing File Timestamps with utime() and utimes() Changing File Timestamps with utimensat() and futimens()

File Ownership

Ownership of New Files Changing File Ownership: chown(), fchown(), and lchown()

File Permissions

Permissions on Regular Files Permissions on Directories Permission-Checking Algorithm

Permission checking for privileged processes

Checking File Accessibility: access() Set-User-ID, Set-Group-ID, and Sticky Bits The Process File Mode Creation Mask: umask() Changing File Permissions: chmod() and fchmod()

I-node Flags (ext2 Extended File Attributes) Summary Exercises

16. Extended Attributes

Overview

EA namespaces Creating and viewing EAs from the shell

Extended Attribute Implementation Details

Restrictions on user extended attributes Implementation limits

System Calls for Manipulating Extended Attributes

Creating and modifying EAs Retrieving the value of an EA Removing an EA Retrieving the names of all EAs associated with a file Example program

Summary Exercise

17. Access Control Lists

Overview

ACL entries Minimal and extended ACLs

ACL Permission-Checking Algorithm Long and Short Text Forms for ACLs The ACL_MASK Entry and the ACL Group Class The getfacl and setfacl Commands Default ACLs and File Creation ACL Implementation Limits The ACL API

Overview Fetching a file's ACL into memory Retrieving entries from an in-memory ACL Retrieving and modifying attributes in an ACL entry Creating and deleting ACL entries Updating a file's ACL Converting an ACL between in-memory and text form Other functions in the ACL API Example program

Summary

Further information

Exercise

18. Directories and Links

Directories and (Hard) Links Symbolic (Soft) Links

Interpretation of symbolic links by system calls File permissions and ownership for symbolic links

Creating and Removing (Hard) Links: link() and unlink()

An open file is deleted only when all file descriptors are closed

Changing the Name of a File: rename() Working with Symbolic Links: symlink() and readlink() Creating and Removing Directories: mkdir() and rmdir() Removing a File or Directory: remove() Reading Directories: opendir() and readdir()

Directory streams and file descriptors Example program The readdir_r() function

File Tree Walking: nftw()

Example program The nftw() FTW_ACTIONRETVAL flag

The Current Working Directory of a Process

Retrieving the current working directory Changing the current working directory

Operating Relative to a Directory File Descriptor Changing the Root Directory of a Process: chroot() Resolving a Pathname: realpath() Parsing Pathname Strings: dirname() and basename() Summary Exercises

19. Monitoring File Events

Overview The inotify API inotify Events Reading inotify Events

Example program

Queue Limits and /proc Files An Older System for Monitoring File Events: dnotify Summary Exercise

20. Signals: Fundamental Concepts

Concepts and Overview Signal Types and Default Actions Changing Signal Dispositions: signal() Introduction to Signal Handlers Sending Signals: kill() Checking for the Existence of a Process Other Ways of Sending Signals: raise() and killpg() Displaying Signal Descriptions Signal Sets

Example program

The Signal Mask (Blocking Signal Delivery) Pending Signals Signals Are Not Queued Changing Signal Dispositions: sigaction() Waiting for a Signal: pause() Summary

Further information

Exercises

21. Signals: Signal Handlers

Designing Signal Handlers

Signals Are Not Queued (Revisited) Reentrant and Async-Signal-Safe Functions

Reentrant and nonreentrant functions Example program Standard async-signal-safe functions Use of errno inside signal handlers Use of unsafe functions in example programs in this book

Global Variables and the sig_atomic_t Data Type

Other Methods of Terminating a Signal Handler

Performing a Nonlocal Goto from a Signal Handler

Example program

Terminating a Process Abnormally: abort()

Handling a Signal on an Alternate Stack: sigaltstack() The SA_SIGINFO Flag

The siginfo_t structure The ucontext argument

Interruption and Restarting of System Calls

System calls (and library functions) for which SA_RESTART is effective Modifying the SA_RESTART flag for a signal Unhandled stop signals can generate EINTR for some Linux system calls

Summary

Further information

Exercise

22. Signals: Advanced Features

Core Dump Files

Circumstances in which core dump files are not produced Naming the core dump file: /proc/sys/kernel/core_pattern

Special Cases for Delivery, Disposition, and Handling

SIGKILL and SIGSTOP SIGCONT and stop signals Don’t change the disposition of ignored terminal-generated signals

Interruptible and Uninterruptible Process Sleep States Hardware-Generated Signals Synchronous and Asynchronous Signal Generation Timing and Order of Signal Delivery

When is a signal delivered? Order of delivery of multiple unblocked signals

Implementation and Portability of signal()

Some glibc details sigaction() is the preferred API for establishing a signal handler

Realtime Signals

Limits on the number of queued realtime signals Using realtime signals

Sending Realtime Signals Handling Realtime Signals

Waiting for a Signal Using a Mask: sigsuspend()

Example program

Synchronously Waiting for a Signal Fetching Signals via a File Descriptor Interprocess Communication with Signals Earlier Signal APIs (System V and BSD)

The System V signal API The BSD signal API

Summary

Further information

Exercises

23. Timers and Sleeping

Interval Timers

Example program A simpler timer interface: alarm() Interactions between setitimer() and alarm()

Scheduling and Accuracy of Timers

High-resolution timers

Setting Timeouts on Blocking Operations Suspending Execution for a Fixed Interval (Sleeping)

Low-Resolution Sleeping: sleep() High-Resolution Sleeping: nanosleep()

POSIX Clocks

Retrieving the Value of a Clock: clock_gettime() Setting the Value of a Clock: clock_settime() Obtaining the Clock ID of a Specific Process or Thread Improved High-Resolution Sleeping: clock_nanosleep()

POSIX Interval Timers

Creating a Timer: timer_create() Arming and Disarming a Timer: timer_settime() Retrieving the Current Value of a Timer: timer_gettime() Deleting a Timer: timer_delete() Notification via a Signal Timer Overruns Notification via a Thread

Timers That Notify via File Descriptors: The timerfd API

Interactions of timerfd with fork() and exec() Reading from the timerfd file descriptor Example program

Summary

Further information

Exercises

24. Process Creation

Overview of fork(), exit(), wait(), and execve() Creating a New Process: fork()

File Sharing Between Parent and Child Memory Semantics of fork()

Controlling a process’s memory footprint

The vfork() System Call Race Conditions After fork() Avoiding Race Conditions by Synchronizing with Signals Summary

Further information

Exercises

25. Process Termination

Terminating a Process: _exit() and exit() Details of Process Termination Exit Handlers

Registering exit handlers Example program

Interactions Between fork(), stdio Buffers, and _exit() Summary

Further information

Exercise

26. Monitoring Child Processes

Waiting on a Child Process

The wait() System Call The waitpid() System Call The Wait Status Value

Example program

Process Termination from a Signal Handler The waitid() System Call The wait3() and wait4() System Calls

Orphans and Zombies The SIGCHLD Signal

Establishing a Handler for SIGCHLD

Design issues for SIGCHLD handlers Example program

Delivery of SIGCHLD for Stopped Children Ignoring Dead Child Processes

Deviations from SUSv3 in older Linux kernels The sigaction()SA_NOCLDWAIT flag The System V SIGCLD signal

Summary

Further information

Exercises

27. Program Execution

Executing a New Program: execve()

Example program

The exec() Library Functions

The PATH Environment Variable Specifying Program Arguments as a List Passing the Caller’s Environment to the New Program Executing a File Referred to by a Descriptor: fexecve()

Interpreter Scripts

Execution of interpreter scripts Using the script optional-arg Executing scripts with execlp() and execvp()

File Descriptors and exec()

The close-on-exec flag (FD_CLOEXEC)

Signals and exec() Executing a Shell Command: system()

Example program Avoid using system() in set-user-ID and set-group-ID programs

Implementing system()

A simple implementation of system() Treating signals correctly inside system() An improved system() implementation Further details on system()

Summary

Further information

Exercises

28. Process Creation and Program Execution in More Detail

Process Accounting

Enabling and disabling process accounting Process accounting records Example program Process accounting Version 3 file format

The clone() System Call

Example program

The clone() flags Argument

Sharing file descriptor tables: CLONE_FILES Sharing file system-related information: CLONE_FS Sharing signal dispositions: CLONE_SIGHAND Sharing the parent’s virtual memory: CLONE_VM Thread groups: CLONE_THREAD Threading library support: CLONE_PARENT_SETTID, CLONE_CHILD_SETTID, and CLONE_CHILD_CLEARTID Thread-local storage: CLONE_SETTLS Sharing System V semaphore undo values: CLONE_SYSVSEM Per-process mount namespaces: CLONE_NEWNS Making the child’s parent the same as the caller’s: CLONE_PARENT Making the child’s PID the same as the parent’s PID: CLONE_PID (obsolete) Process tracing: CLONE_PTRACE and CLONE_UNTRACED Suspending the parent until the child exits or execs: CLONE_VFORK New clone() flags to support containers Use of clone() flags

Extensions to waitpid() for Cloned Children

Speed of Process Creation Effect of exec() and fork() on Process Attributes Summary

Further information

Exercise

29. Threads: Introduction

Overview Background Details of the Pthreads API

Pthreads data types Threads and errno Return value from Pthreads functions Compiling Pthreads programs

Thread Creation Thread Termination Thread IDs Joining with a Terminated Thread

Example program

Detaching a Thread Thread Attributes Threads Versus Processes Summary

Further information

Exercises

30. Threads: Thread Synchronization

Protecting Accesses to Shared Variables: Mutexes

Statically Allocated Mutexes Locking and Unlocking a Mutex

Example program pthread_mutex_trylock() and pthread_mutex_timedlock()

Performance of Mutexes Mutex Deadlocks Dynamically Initializing a Mutex Mutex Attributes Mutex Types

Signaling Changes of State: Condition Variables

Statically Allocated Condition Variables Signaling and Waiting on Condition Variables

Using a condition variable in the producer-consumer example

Testing a Condition Variable’s Predicate Example Program: Joining Any Terminated Thread Dynamically Allocated Condition Variables

Summary

Further information

Exercises

31. Threads: Thread Safety and Per-Thread Storage

Thread Safety (and Reentrancy Revisited)

Non-thread-safe functions Reentrant and nonreentrant functions

One-Time Initialization Thread-Specific Data

Thread-Specific Data from the Library Function’s Perspective Overview of the Thread-Specific Data API Details of the Thread-Specific Data API Employing the Thread-Specific Data API Thread-Specific Data Implementation Limits

Thread-Local Storage Summary

Further information

Exercises

32. Threads: Thread Cancellation

Canceling a Thread Cancellation State and Type Cancellation Points

Example program

Testing for Thread Cancellation Cleanup Handlers

Example program

Asynchronous Cancelability Summary

Further information

33. Threads: Further Details

Thread Stacks Threads and Signals

How the UNIX Signal Model Maps to Threads Manipulating the Thread Signal Mask Sending a Signal to a Thread Dealing with Asynchronous Signals Sanely

Threads and Process Control

Threads and exec() Threads and fork() Threads and exit()

Thread Implementation Models

Many-to-one (M:1) implementations (user-level threads) One-to-one (1:1) implementations (kernel-level threads) Many-to-many (M:N) implementations (two-level model)

Linux Implementations of POSIX Threads

LinuxThreads

LinuxThreads deviations from specified behavior Other problems with LinuxThreads

NPTL

NPTL standards conformance

Which Threading Implementation?

Discovering the threading implementation Selecting the threading implementation used by a program

Advanced Features of the Pthreads API Summary

Further information

Exercises

34. Process Groups, Sessions, and Job Control

Overview Process Groups

Using setpgid() in a job-control shell Other (obsolete) interfaces for retrieving and modifying process group IDs

Sessions Controlling Terminals and Controlling Processes

Removing a process’s association with the controlling terminal Establishing a controlling terminal on BSD Obtaining a pathname that refers to the controlling terminal: ctermid()

Foreground and Background Process Groups The SIGHUP Signal

Handling of SIGHUP by the Shell SIGHUP and Termination of the Controlling Process

Job Control

Using Job Control Within the Shell Implementing Job Control

The SIGTTIN and SIGTTOU signals Example program: demonstrating the operation of job control

Handling Job-Control Signals

Example program Dealing with ignored job-control and terminal-generated signals

Orphaned Process Groups (and SIGHUP Revisited)

Example program Orphaned process groups and the SIGTSTP, SIGTTIN, and SIGTTOU signals

Summary

Further information

Exercises

35. Process Priorities and Scheduling

Process Priorities (Nice Values)

Effect of the nice value Retrieving and modifying priorities

Overview of Realtime Process Scheduling

POSIX realtime versus hard realtime

The SCHED_RR Policy The SCHED_FIFO Policy The SCHED_BATCH and SCHED_IDLE Policies

Realtime Process Scheduling API

Realtime Priority Ranges Modifying and Retrieving Policies and Priorities

Modifying scheduling policies and priorities Privileges and resource limits affecting changes to scheduling parameters Retrieving scheduling policies and priorities Preventing realtime processes from locking up the system Preventing child processes from inheriting privileged scheduling policies

Relinquishing the CPU The SCHED_RR Time Slice

CPU Affinity Summary

Further information

Exercises

36. Process Resources

Process Resource Usage Process Resource Limits

Example program Unrepresentable limit values

Details of Specific Resource Limits

RLIMIT_AS RLIMIT_CORE RLIMIT_CPU RLIMIT_DATA RLIMIT_FSIZE RLIMIT_MEMLOCK RLIMIT_MSGQUEUE RLIMIT_NICE RLIMIT_NOFILE RLIMIT_NPROC RLIMIT_RSS RLIMIT_RTPRIO RLIMIT_RTTIME RLIMIT_SIGPENDING RLIMIT_STACK

Summary Exercises

37. Daemons

Overview Creating a Daemon Guidelines for Writing Daemons Using SIGHUP to Reinitialize a Daemon Logging Messages and Errors Using syslog

Overview The syslog API Establishing a connection to the system log Logging a message Closing the log Filtering log messages The /etc/syslog.conf File

Summary

Further information

Exercise

38. Writing Secure Privileged Programs

Is a Set-User-ID or Set-Group-ID Program Required? Operate with Least Privilege

Hold privileges only while they are required Drop privileges permanently when they will never again be required General points on changing process credentials

Be Careful When Executing a Program

Drop privileges permanently before execing another program Avoid executing a shell (or other interpreter) with privileges Close all unnecessary file descriptors before an exec()

Avoid Exposing Sensitive Information Confine the Process

Consider using capabilities Consider using a chroot jail

Beware of Signals and Race Conditions Pitfalls When Performing File Operations and File I/O Don't Trust Inputs or the Environment

Don't trust the environment list Handle untrusted user inputs defensively Avoid unreliable assumptions about the process's run-time environment

Beware of Buffer Overruns Beware of Denial-of-Service Attacks Check Return Statuses and Fail Safely Summary

Further information

Exercises

39. Capabilities

Rationale for Capabilities The Linux Capabilities Process and File Capabilities

Process Capabilities File Capabilities Purpose of the Process Permitted and Effective Capability Sets Purpose of the File Permitted and Effective Capability Sets Purpose of the Process and File Inheritable Sets Assigning and Viewing File Capabilities from the Shell

The Modern Capabilities Implementation Transformation of Process Capabilities During exec()

Capability Bounding Set Preserving root Semantics

Effect on Process Capabilities of Changing User IDs Changing Process Capabilities Programmatically

The libcap API Example program

Creating Capabilities-Only Environments

SECBIT_KEEP_CAPS and the prctl() PR_SET_KEEPCAPS operation

Discovering the Capabilities Required by a Program Older Kernels and Systems Without File Capabilities

The CAP_SETPCAP capability The capability bounding set Using capabilities within a program on a system without file capabilities

Summary Exercise

40. Login Accounting

Overview of the utmp and wtmp Files The utmpx API The utmpx Structure Retrieving Information from the utmp and wtmp Files

Example program

Retrieving the Login Name: getlogin() Updating the utmp and wtmp Files for a Login Session

Example program

The lastlog File Summary

Further information

Exercises

41. Fundamentals of Shared Libraries

Object Libraries

An aside: including debugger information when compiling a program

Static Libraries

Creating and maintaining a static library Using a static library

Overview of Shared Libraries Creating and Using Shared Libraries—A First Pass

Creating a Shared Library Position-Independent Code Using a Shared Library

The LD_LIBRARY_PATH environment variable Static linking and dynamic linking contrasted

The Shared Library Soname

Useful Tools for Working with Shared Libraries

The ldd command The objdump and readelf commands The nm command

Shared Library Versions and Naming Conventions

Real names, sonames, and linker names Creating a shared library using standard conventions

Installing Shared Libraries

ldconfig

Compatible Versus Incompatible Libraries Upgrading Shared Libraries Specifying Library Search Directories in an Object File

Using the -rpath linker option when building a shared library The ELF DT_RPATH and DT_RUNPATH entries Using $ORIGIN in rpath

Finding Shared Libraries at Run Time Run-Time Symbol Resolution Using a Static Library Instead of a Shared Library Summary

Further information

Exercise

42. Advanced Features of Shared Libraries

Dynamically Loaded Libraries

Opening a Shared Library: dlopen() Diagnosing Errors: dlerror() Obtaining the Address of a Symbol: dlsym()

Using library pseudohandles with dlsym() Example program

Closing a Shared Library: dlclose() Obtaining Information About Loaded Symbols: dladdr() Accessing Symbols in the Main Program

Controlling Symbol Visibility Linker Version Scripts

Controlling Symbol Visibility with Version Scripts Symbol Versioning

Initialization and Finalization Functions

The _init() and _fini() functions

Preloading Shared Libraries Monitoring the Dynamic Linker: LD_DEBUG Summary

Further information

Exercises

43. Interprocess Communication Overview

A Taxonomy of IPC Facilities Communication Facilities

Data transfer Shared memory

Synchronization Facilities Comparing IPC Facilities

IPC object identification and handles for open objects Functionality Network communication Portability System V IPC design issues Accessibility Persistence Performance

Summary Exercises

44. Pipes and FIFOs

Overview

A pipe is a byte stream Reading from a pipe Pipes are unidirectional Writes of up to PIPE_BUF bytes are guaranteed to be atomic Pipes have a limited capacity

Creating and Using Pipes

Pipes allow communication between related processes Closing unused pipe file descriptors Example program

Pipes as a Method of Process Synchronization Using Pipes to Connect Filters

Example program

Talking to a Shell Command via a Pipe: popen()

Example program

Pipes and stdio Buffering FIFOs

Using FIFOs and tee(1) to create a dual pipeline

A Client-Server Application Using FIFOs

Application overview Server program Client program

Nonblocking I/O

Nonblocking read() and write()

Semantics of read() and write() on Pipes and FIFOs Summary

Further information

Exercises

45. Introduction to System V IPC

API Overview

Creating and opening a System V IPC object IPC object deletion and object persistence

IPC Keys

Generating a unique key with IPC_PRIVATE Generating a unique key with ftok()

Associated Data Structure and Object Permissions IPC Identifiers and Client-Server Applications Algorithm Employed by System V IPC get Calls The ipcs and ipcrm Commands Obtaining a List of All IPC Objects IPC Limits Summary

Further information

Exercises

46. System V Message Queues

Creating or Opening a Message Queue Exchanging Messages

Sending Messages Receiving Messages

Example program

Message Queue Control Operations Message Queue Associated Data Structure Message Queue Limits Displaying All Message Queues on the System Client-Server Programming with Message Queues

Using a single message queue for server and clients Using one message queue per client

A File-Server Application Using Message Queues

Common header file Server program Client program

Disadvantages of System V Message Queues Summary Exercises

47. System V Semaphores

Overview Creating or Opening a Semaphore Set Semaphore Control Operations

Generic control operations Retrieving and initializing semaphore values Retrieving per-semaphore information

Semaphore Associated Data Structure

Monitoring a semaphore set Initializing all semaphores in a set

Semaphore Initialization Semaphore Operations

Example program

Handling of Multiple Blocked Semaphore Operations Semaphore Undo Values

Example of the effect of SEM_UNDO Limitations of SEM_UNDO

Implementing a Binary Semaphores Protocol Semaphore Limits Disadvantages of System V Semaphores Summary

Further information

Exercises

48. System V Shared Memory

Overview Creating or Opening a Shared Memory Segment Using Shared Memory Example: Transferring Data via Shared Memory Location of Shared Memory in Virtual Memory Storing Pointers in Shared Memory Shared Memory Control Operations

Generic control operations Locking and unlocking shared memory

Shared Memory Associated Data Structure Shared Memory Limits Summary

Further information

Exercises

49. Memory Mappings

Overview Creating a Mapping: mmap()

Memory protection in more detail Alignment restrictions specified in standards for offset and addr Example program

Unmapping a Mapped Region: munmap() File Mappings

Private File Mappings Shared File Mappings

Memory-mapped I/O IPC using a shared file mapping Example program

Boundary Cases Memory Protection and File Access Mode Interactions

Synchronizing a Mapped Region: msync() Additional mmap() Flags Anonymous Mappings

MAP_ANONYMOUS and /dev/zero MAP_PRIVATE anonymous mappings MAP_SHARED anonymous mappings Example program

Remapping a Mapped Region: mremap() MAP_NORESERVE and Swap Space Overcommitting

The OOM killer

The MAP_FIXED Flag Nonlinear Mappings: remap_file_pages() Summary

Further information

Exercises

50. Virtual Memory Operations

Changing Memory Protection: mprotect() Memory Locking: mlock() and mlockall()

The RLIMIT_MEMLOCK resource limit Locking and unlocking memory regions Details of the semantics of memory locking Locking and unlocking all of a process’s memory

Determining Memory Residence: mincore() Advising Future Memory Usage Patterns: madvise() Summary Exercises

51. Introduction to POSIX IPC

API Overview

IPC object names Creating or opening an IPC object Closing an IPC object IPC object permissions IPC object deletion and object persistence Listing and removing POSIX IPC objects via the command line Compiling programs that use POSIX IPC on Linux

Comparison of System V IPC and POSIX IPC Summary

52. POSIX Message Queues

Overview Opening, Closing, and Unlinking a Message Queue

Opening a message queue Effect of fork(), exec(), and process termination on message queue descriptors Closing a message queue Removing a message queue

Relationship Between Descriptors and Message Queues Message Queue Attributes

Setting message queue attributes during queue creation Retrieving message queue attributes Modifying message queue attributes

Exchanging Messages

Sending Messages Receiving Messages Sending and Receiving Messages with a Timeout

Message Notification

Receiving Notification via a Signal Receiving Notification via a Thread

Linux-Specific Features

Displaying and deleting message queue objects via the command line Obtaining information about a message queue Using message queues with alternative I/O models

Message Queue Limits Comparison of POSIX and System V Message Queues Summary

Further information

Exercises

53. POSIX Semaphores

Overview Named Semaphores

Opening a Named Semaphore

Example program

Closing a Semaphore Removing a Named Semaphore

Semaphore Operations

Waiting on a Semaphore Posting a Semaphore Retrieving the Current Value of a Semaphore

Example

Unnamed Semaphores

Unnamed versus named semaphores

Initializing an Unnamed Semaphore

Example program

Destroying an Unnamed Semaphore

Comparisons with Other Synchronization Techniques

POSIX semaphores versus System V semaphores POSIX semaphores versus Pthreads mutexes

Semaphore Limits Summary

Further information

Exercises

54. POSIX Shared Memory

Overview Creating Shared Memory Objects

Example program

Using Shared Memory Objects Removing Shared Memory Objects Comparisons Between Shared Memory APIs Summary Exercise

55. File Locking

Overview

Mixing locking and stdio functions Advisory and mandatory locking

File Locking with flock()

Semantics of Lock Inheritance and Release Limitations of flock()

Record Locking with fcntl()

The flock structure The cmd argument Details of lock acquisition and release

Deadlock Example: An Interactive Locking Program Example: A Library of Locking Functions Lock Limits and Performance Semantics of Lock Inheritance and Release Lock Starvation and Priority of Queued Lock Requests

Mandatory Locking

Effect of mandatory locking on file I/O operations Mandatory locking caveats

The /proc/locks File Running Just One Instance of a Program Older Locking Techniques

open(file, O_CREAT | O_EXCL,...) plus unlink(file) link(file, lockfile) plus unlink(lockfile) open(file, O_CREAT | O_TRUNC | O_WRONLY, 0) plus unlink(file)

Summary

Further information

Exercises

56. Sockets: Introduction

Overview

Communication domains Socket types Socket system calls

Creating a Socket: socket() Binding a Socket to an Address: bind() Generic Socket Address Structures: struct sockaddr Stream Sockets

Active and passive sockets

Listening for Incoming Connections: listen() Accepting a Connection: accept() Connecting to a Peer Socket: connect() I/O on Stream Sockets Connection Termination: close()

Datagram Sockets

Exchanging Datagrams: recvfrom() and sendto() Using connect() with Datagram Sockets

Summary

Further information

57. Sockets: UNIX Domain

UNIX Domain Socket Addresses: struct sockaddr_un Stream Sockets in the UNIX Domain Datagram Sockets in the UNIX Domain

Maximum datagram size for UNIX domain datagram sockets Example program

UNIX Domain Socket Permissions Creating a Connected Socket Pair: socketpair() The Linux Abstract Socket Namespace Summary

Further information

Exercises

58. Sockets: Fundamentals of TCP/IP Networks

Internets Networking Protocols and Layers

Encapsulation

The Data-Link Layer The Network Layer: IP

IP transmits datagrams IP is connectionless and unreliable IP may fragment datagrams

IP Addresses

IPv4 addresses IPv6 addresses

The Transport Layer

Port Numbers

Well-known, registered, and privileged ports Ephemeral ports

User Datagram Protocol (UDP)

Selecting a UDP datagram size to avoid IP fragmentation

Transmission Control Protocol (TCP)

Connection establishment Packaging of data in segments Acknowledgements, retransmissions, and timeouts Sequencing Flow control Congestion control: slow-start and congestion-avoidance algorithms

Requests for Comments (RFCs) Summary

Further information

59. Sockets: Internet Domains

Internet Domain Sockets Network Byte Order Data Representation Internet Socket Addresses

IPv4 socket addresses: struct sockaddr_in IPv6 socket addresses: struct sockaddr_in6 The sockaddr_storage structure

Overview of Host and Service Conversion Functions

Converting IPv4 addresses between binary and human-readable forms Converting IPv4 and IPv6 addresses between binary and human-readable forms Converting host and service names to and from binary form (obsolete) Converting host and service names to and from binary form (modern)

The inet_pton() and inet_ntop() Functions Client-Server Example (Datagram Sockets) Domain Name System (DNS)

Recursive and iterative resolution requests Top-level domains

The /etc/services File Protocol-Independent Host and Service Conversion

The getaddrinfo() Function

The hints argument

Freeing addrinfo Lists: freeaddrinfo() Diagnosing Errors: gai_strerror() The getnameinfo() Function

Client-Server Example (Stream Sockets)

Common header file Server program Client program

An Internet Domain Sockets Library Obsolete APIs for Host and Service Conversions

The inet_aton() and inet_ntoa() Functions The gethostbyname() and gethostbyaddr() Functions The getservbyname() and getservbyport() Functions

UNIX Versus Internet Domain Sockets Further Information Summary Exercises

60. Sockets: Server Design

Iterative and Concurrent Servers An Iterative UDP echo Server A Concurrent TCP echo Server Other Concurrent Server Designs

Preforked and prethreaded servers Handling multiple clients from a single process Using server farms

The inetd (Internet Superserver) Daemon

Operation of the inetd daemon The /etc/inetd.conf file Example: invoking a TCP echo service via inetd

Summary

Further information

Exercises

61. Sockets: Advanced Topics

Partial Reads and Writes on Stream Sockets The shutdown() System Call

Example program

Socket-Specific I/O System Calls: recv() and send() The sendfile() System Call

The TCP_CORK socket option

Retrieving Socket Addresses A Closer Look at TCP

Format of a TCP Segment TCP Sequence Numbers and Acknowledgements TCP State Machine and State Transition Diagram TCP Connection Establishment TCP Connection Termination Calling shutdown() on a TCP Socket The TIME_WAIT State

Monitoring Sockets: netstat Using tcpdump to Monitor TCP Traffic Socket Options The SO_REUSEADDR Socket Option Inheritance of Flags and Options Across accept() TCP Versus UDP Advanced Features

Out-of-Band Data The sendmsg() and recvmsg() System Calls Passing File Descriptors Receiving Sender Credentials Sequenced-Packet Sockets SCTP and DCCP Transport-Layer Protocols

Summary

Further information

Exercises

62. Terminals

Overview Retrieving and Modifying Terminal Attributes The stty Command Terminal Special Characters

CR DISCARD EOF EOL and EOL2 ERASE INTR KILL LNEXT NL QUIT REPRINT START and STOP SUSP WERASE Other terminal special characters Example program

Terminal Flags

BRKINT ECHO ECHOCTL ECHOE ECHOK and ECHOKE ICANON IEXTEN IMAXBEL IUTF8 NOFLSH OPOST PARENB, IGNPAR, INPCK, PARMRK, and PARODD Example program

Terminal I/O Modes

Canonical Mode Noncanonical Mode

MIN == 0, TIME == 0 (polling read) MIN > 0, TIME == 0 (blocking read) MIN == 0, TIME > 0 (read with timeout) MIN > 0, TIME > 0 (read with interbyte timeout) Portably modifying and restoring MIN and TIME

Cooked, Cbreak, and Raw Modes

Example: setting raw and cbreak mode

Terminal Line Speed (Bit Rate) Terminal Line Control Terminal Window Size Terminal Identification Summary

Further information

Exercises

63. Alternative I/O Models

Overview

Which technique?

Level-Triggered and Edge-Triggered Notification Employing Nonblocking I/O with Alternative I/O Models

I/O Multiplexing

The select() System Call

File descriptor sets The timeout argument Return value from select() Example program

The poll() System Call

The pollfd array The timeout argument Return value from poll() Example program

When Is a File Descriptor Ready?

Regular files Terminals and pseudoterminals Pipes and FIFOs Sockets

Comparison of select() and poll()

Implementation details API differences Portability Performance

Problems with select() and poll()

Signal-Driven I/O

Example program Establish the signal handler before enabling signal-driven I/O Setting the file descriptor owner

When Is "I/O Possible" Signaled?

Terminals and pseudoterminals Pipes and FIFOs Sockets inotify file descriptors

Refining the Use of Signal-Driven I/O

Handling signal-queue overflow Using signal-driven I/O with multithreaded applications

The epoll API

Creating an epoll Instance: epoll_create() Modifying the epoll Interest List: epoll_ctl()

The max_user_watches limit

Waiting for Events: epoll_wait()

epoll events The EPOLLONESHOT flag Example program

A Closer Look at epoll Semantics Performance of epoll Versus I/O Multiplexing Edge-Triggered Notification

Preventing file-descriptor starvation when using edge-triggered notification

Waiting on Signals and File Descriptors

The pselect() System Call

The ppoll() and epoll_pwait() system calls

The Self-Pipe Trick

Summary

Further information

Exercises

64. Pseudoterminals

Overview

The pseudoterminal master and slave devices How programs use pseudoterminals Applications of pseudoterminals System V (UNIX 98) and BSD pseudoterminals

UNIX 98 Pseudoterminals

Opening an Unused Master: posix_openpt()

Limits on the number of UNIX 98 pseudoterminals

Changing Slave Ownership and Permissions: grantpt() Unlocking the Slave: unlockpt() Obtaining the Name of the Slave: ptsname()

Opening a Master: ptyMasterOpen() Connecting Processes with a Pseudoterminal: ptyFork() Pseudoterminal I/O

Packet mode

Implementing script(1) Terminal Attributes and Window Size BSD Pseudoterminals Summary Exercises

A. Tracing System Calls B. Parsing Command-Line Options

Example program GNU-specific behavior GNU extensions

C. Casting the NULL Pointer D. Kernel Configuration E. Further Sources of Information

Manual pages GNU info documents The GNU C library (glibc) manual Books Source code of existing applications The Linux Documentation Project The GNU project Newsgroups Linux kernel mailing list Web sites The kernel source code

F. Solutions to Selected Exercises

Chapter 5 Chapter 6 Chapter 8 Chapter 9 Chapter 10 Chapter 12 Chapter 13 Chapter 15 Chapter 18 Chapter 20 Chapter 22 Chapter 23 Chapter 24 Chapter 25 Chapter 26 Chapter 27 Chapter 29 Chapter 31 Chapter 33 Chapter 34 Chapter 35 Chapter 36 Chapter 37 Chapter 38 Chapter 44 Chapter 45 Chapter 46 Chapter 47 Chapter 48 Chapter 49 Chapter 50 Chapter 52 Chapter 53 Chapter 55 Chapter 57 Chapter 59 Chapter 60 Chapter 61 Chapter 62 Chapter 63 Chapter 64

Bibliography

G. Updates Index About the Author Colophon

← Prev
Back
Next →

← Prev
Back
Next →