Expert Python Programming, Fourth Edition by Michał Jaworski, Tarek Ziadé -- Read -- Imperial Library of Trantor

Index

Preface

Who this book is for What this book covers To get the most out of this book Get in touch

Current Status of Python

Where are we now and where are we going? What to do with Python 2 Keeping up to date

PEP documents Active communities Other resources

Summary

Modern Python Development Environments

Technical requirements Python's packaging ecosystem

Installing Python packages using pip

Isolating the runtime environment

Application-level isolation versus system-level isolation

Application-level environment isolation

Poetry as a dependency management system

System-level environment isolation

Containerization versus virtualization Virtual environments using Docker

Writing your first Dockerfile Running containers Setting up complex environments Useful Docker and Docker Compose recipes for Python

Virtual development environments using Vagrant

Popular productivity tools

Custom Python shells Using IPython Incorporating shells in your own scripts and programs Interactive debuggers Other productivity tools

Summary

New Things in Python

Technical requirements Recent language additions

Dictionary merge and update operators

Alternative – Dictionary unpacking Alternative – ChainMap from the collections module

Assignment expressions Type-hinting generics Positional-only parameters zoneinfo module graphlib module

Not that new, but still shiny

breakpoint() function Development mode Module-level __getattr__() and __dir__() functions Formatting strings with f-strings Underscores in numeric literals secrets module

What may come in the future?

Union types with the | operator Structural pattern matching

Summary

Python in Comparison with Other Languages

Technical requirements Class model and object-oriented programming

Accessing super-classes Multiple inheritance and Method Resolution Order Class instance initialization Attribute access patterns Descriptors

Real-life example – lazily evaluated attributes

Properties

Dynamic polymorphism

Operator overloading

Dunder methods (language protocols) Comparison to C++

Function and method overloading

Single-dispatch functions

Data classes Functional programming

Lambda functions The map(), filter(), and reduce() functions Partial objects and partial functions Generators Generator expressions Decorators

Enumerations Summary

Interfaces, Patterns, and Modularity

Technical requirements Interfaces

A bit of history: zope.interface Using function annotations and abstract base classes

Using collections.abc

Interfaces through type annotations

Inversion of control and dependency injection

Inversion of control in applications Using dependency injection frameworks

Summary

Concurrency

Technical requirements What is concurrency? Multithreading

What is multithreading? How Python deals with threads When should we use multithreading?

Application responsiveness Multiuser applications Work delegation and background processing

An example of a multithreaded application

Using one thread per item Using a thread pool Using two-way queues Dealing with errors in threads Throttling

Multiprocessing

The built-in multiprocessing module Using process pools Using multiprocessing.dummy as the multithreading interface

Asynchronous programming

Cooperative multitasking and asynchronous I/O Python async and await keywords A practical example of asynchronous programming Integrating non-asynchronous code with async using futures

Executors and futures Using executors in an event loop

Summary

Event-Driven Programming

Technical requirements What exactly is event-driven programming?

Event-driven != asynchronous Event-driven programming in GUIs Event-driven communication

Various styles of event-driven programming

Callback-based style Subject-based style Topic-based style

Event-driven architectures

Event and message queues

Summary

Elements of Metaprogramming

Technical requirements What is metaprogramming? Using decorators to modify function behavior before use

One step deeper: class decorators

Intercepting the class instance creation process Metaclasses

The general syntax Metaclass usage Metaclass pitfalls Using the __init__subclass__() method as an alternative to metaclasses

Code generation

exec, eval, and compile The abstract syntax tree Import hooks Notable examples of code generation in Python

Falcon's compiled router Hy

Summary

Bridging Python with C and C++

Technical requirements C and C++ as the core of Python extensibility Compiling and loading Python C extensions The need to use extensions

Improving performance in critical code sections Integrating existing code written in different languages Integrating third-party dynamic libraries Creating efficient custom datatypes

Writing extensions

Pure C extensions

A closer look at the Python/C API Calling and binding conventions Exception handling Releasing GIL Reference counting

Writing extensions with Cython

Cython as a source-to-source compiler Cython as a language

Downsides of using extensions

Additional complexity Harder debugging

Interfacing with dynamic libraries without extensions

The ctypes module

Loading libraries Calling C functions using ctypes Passing Python functions as C callbacks

CFFI

Summary

Testing and Quality Automation

Technical requirements The principles of test-driven development Writing tests with pytest

Test parameterization pytest's fixtures Using fakes Mocks and the unittest.mock module

Quality automation

Test coverage Style fixers and code linters Static type analysis

Mutation testing Useful testing utilities

Faking realistic data values Faking time values

Summary

Packaging and Distributing Python Code

Technical requirements Packaging and distributing libraries

The anatomy of a Python package

setup.py setup.cfg MANIFEST.in Essential package metadata Trove classifiers

Types of package distributions

sdist distributions bdist and wheel distributions

Registering and publishing packages Package versioning and dependency management

The SemVer standard for semantic versioning CalVer for calendar versioning

Installing your own packages

Installing packages directly from sources Installing packages in editable mode

Namespace packages Package scripts and entry points

Packaging applications and services for the web

The Twelve-Factor App manifesto Leveraging Docker Handling environment variables The role of environment variables in application frameworks

Creating standalone executables

When standalone executables are useful Popular tools

PyInstaller cx_Freeze py2exe and py2app

Security of Python code in executable packages

Summary

Observing Application Behavior and Performance

Technical requirements Capturing errors and logs

Python logging essentials

Logging system components Logging configuration

Good logging practices Distributed logging Capturing errors for later review

Instrumenting code with custom metrics

Using Prometheus

Distributed application tracing

Distributed tracing with Jaeger

Summary

Code Optimization

Technical requirements Common culprits for bad performance

Code complexity

Cyclomatic complexity The big O notation

Excessive resource allocation and leaks Excessive I/O and blocking operations

Code profiling

Profiling CPU usage

Macro-profiling Micro-profiling

Profiling memory usage

Using the objgraph module C code memory leaks

Reducing complexity by choosing appropriate data structures

Searching in a list Using sets Using the collections module

deque defaultdict namedtuple

Leveraging architectural trade-offs

Using heuristics and approximation algorithms Using task queues and delayed processing Using probabilistic data structures Caching

Deterministic caching Non-deterministic caching

Summary

Other Books You May Enjoy Index

← Prev
Back
Next →

← Prev
Back
Next →