Expert Python Programming · 3rd Edition by Jaworski, Michal -- Read -- Imperial Library of Trantor

Index

Title Page Copyright and Credits

Expert Python Programming Third Edition

Dedication About Packt

Why subscribe? Packt.com

Contributors

About the authors About the reviewer Packt is searching for authors like you

Preface

Who this book is for What this book covers To get the most out of this book

Download the example code files Download the color images Conventions used

Get in touch

Reviews

Section 1: Before You Start Current Status of Python

Technical requirements Where are we now and where we are going to? Why and how Python changes Being up-to-date with changes by following PEP documents Python 3 adoption at the time of writing this book The main differences between Python 3 and Python 2

Why should I care? The main syntax differences and common pitfalls

Syntax changes Changes in the standard library Changes in data types and collections and string literals

The popular tools and techniques used for maintaining cross-version compatibility

Not only CPython

Why should I care? Stackless Python Jython IronPython PyPy MicroPython

Useful resources Summary

Modern Python Development Environments

Technical requirements Installing additional Python packages using pip Isolating the runtime environment

Application-level isolation versus system-level isolation

Python's venv

venv versus virtualenv

System-level environment isolation

Virtual development environments using Vagrant Virtual environments using Docker

Containerization versus virtualization Writing your first Dockerfile Running containers Setting up complex environments Useful Docker recipes for Python

Reducing the size of containers Addressing services inside of a Compose environment Communicating between multiple Compose environments

Popular productivity tools

Custom Python shells – ipython, bpython, ptpython, and so on

Setting up the PYTHONSTARTUP environment variable IPython bpython ptpython

Incorporating shells in your own scripts and programs Interactive debuggers

Summary

Section 2: Python Craftsmanship Modern Syntax Elements - Below the Class Level

Technical requirements Python's built-in types

Strings and bytes

Implementation details String concatenation

Constant folding, the peephole optimizer, and the AST optimizer

String formatting with f-strings

Containers

Lists and tuples

Implementation details List comprehensions Other idioms

Dictionaries

Implementation details Weaknesses and alternatives

Sets

Implementation details

Supplemental data types and containers

Specialized data containers from the collections module Symbolic enumeration with the enum module

Advanced syntax

Iterators Generators and yield statements Decorators

General syntax and possible implementations

As a function As a class Parametrizing decorators Introspection preserving decorators

Usage and useful examples

Argument checking Caching Proxy Context provider

Context managers – the with statement

The general syntax and possible implementations

As a class As a function – the contextlib module

Functional-style features of Python

What is functional programming? Lambda functions map(), filter(), and reduce() Partial objects and partial() functions Generator expressions

Function and variable annotations

The general syntax The possible uses Static type checking with mypy

Other syntax elements you may not know of yet

The for ... else ... statement Keyword-only arguments

Summary

Modern Syntax Elements - Above the Class Level

Technical requirements The protocols of the Python language – dunder methods and attributes Reducing boilerplate with data classes Subclassing built-in types MRO and accessing methods from superclasses

Old-style classes and super in Python 2 Understanding Python's Method Resolution Order Super pitfalls

Mixing super and explicit class calls Heterogeneous arguments

Best practices

Advanced attribute access patterns

Descriptors

Real-life example – lazily evaluated attributes

Properties Slots

Summary

Elements of Metaprogramming

Technical requirements What is metaprogramming?

Decorators – a method of metaprogramming Class decorators Using __new__() for overriding the instance creation process Metaclasses

The general syntax New Python 3 syntax for metaclasses Metaclass usage Metaclass pitfalls

Code generation

exec, eval, and compile Abstract syntax tree (AST)

Import hooks

Projects that use code generation patterns

Falcon's compiled router Hy

Summary

Choosing Good Names

Technical requirements PEP 8 and naming best practices

Why and when to follow PEP 8? Beyond PEP 8 – Team-specific style guidelines

Naming styles

Variables

Constants Naming and usage Public and private variables Functions and methods The private controversy Special methods Arguments Properties Classes Modules and packages

The naming guide

Using the has/is prefixes for Boolean elements Using plurals for variables that are collections Using explicit names for dictionaries Avoid generic names and redundancy Avoiding existing names

Best practices for arguments

Building arguments by iterative design Trusting the arguments and your tests Using *args and **kwargs magic arguments carefully

Class names Module and package names Useful tools

Pylint pycodestyle and flake8

Summary

Writing a Package

Technical requirements Creating a package

The confusing state of Python packaging tools

The current landscape of Python packaging thanks to PyPA Tool recommendations

Project configuration

setup.py setup.cfg MANIFEST.in Most important metadata Trove classifiers Common patterns

Automated inclusion of version string from package README file Managing dependencies

The custom setup command Working with packages during development

setup.py install Uninstalling packages setup.py develop or pip -e

Namespace packages

Why is it useful?

PEP 420 - implicit namespace packages Namespace packages in previous Python versions

Uploading a package

PyPI - Python Package Index

Uploading to PyPI - or other package index .pypirc

Source packages versus built packages

sdist bdist and wheels

Standalone executables

When standalone executables useful? Popular tools

PyInstaller cx_Freeze py2exe and py2app

Security of Python code in executable packages

Making decompilation harder

Summary

Deploying the Code

Technical requirements The Twelve-Factor App Various approaches to deployment automation

Using Fabric for deployment automation

Your own package index or index mirror

PyPI mirroring Bundling additional resources with your Python package

Common conventions and practices

The filesystem hierarchy Isolation Using process supervision tools Application code running in user space Using reverse HTTP proxies Reloading processes gracefully

Code instrumentation and monitoring

Logging errors – Sentry/Raven Monitoring system and application metrics Dealing with application logs

Basic low-level log practices Tools for log processing

Summary

Python Extensions in Other Languages

Technical requirements Differentiating between the C and C++ languages

Loading extensions in C or C++

The need to use extensions

Improving the performance in critical code sections Integrating existing code written in different languages Integrating third-party dynamic libraries Creating custom datatypes

Writing extensions

Pure C extensions

A closer look at Python/C API Calling and binding conventions Exception handling Releasing GIL Reference counting

Writing extensions with Cython

Cython as a source-to-source compiler Cython as a language

Challenges with using extensions

Additional complexity Debugging

Interfacing with dynamic libraries without extensions

The ctypes module

Loading libraries Calling C functions using ctypes Passing Python functions as C callbacks

CFFI

Summary

Section 3: Quality over Quantity Managing Code

Technical requirements Working with a version control system

Centralized systems Distributed systems

Distributed strategies

Centralized or distributed? Use Git if you can GitFlow and GitHub Flow

Setting up continuous development processes

Continuous integration

Testing every commit Merge testing through CI Matrix testing

Continuous delivery Continuous deployment Popular tools for continuous integration

Jenkins Buildbot Travis CI GitLab CI

Choosing the right tool and common pitfalls

Problem 1 – Complex build strategies Problem 2 – Long building time Problem 3 – External job definitions Problem 4 – Lack of isolation

Summary

Documenting Your Project

Technical requirements The seven rules of technical writing

Write in two steps Target the readership Use a simple style Limit the scope of information Use realistic code examples Use a light but sufficient approach Use templates

Documentation as code

Using Python docstrings Popular markup languages and styles for documentation

Popular documentation generators for Python libraries

Sphinx

Working with the index pages Registering module helpers Adding index markers Cross-references

MkDocs Documentation building and continuous integration

Documenting web APIs

Documentation as API prototype with API Blueprint Self-documenting APIs with Swagger/OpenAPI

Building a well-organized documentation system

Building documentation portfolio

Design Usage

Recipe Tutorial Module helper

Operations

Your very own documentation portfolio Building a documentation landscape

Producer's layout Consumer's layout

Summary

Test-Driven Development

Technical requirements I don't test

Three simple steps of test-driven development

Preventing software regression Improving code quality Providing the best developer documentation Producing robust code faster

What kind of tests?

Unit tests Acceptance tests Functional tests Integration tests Load and performance testing Code quality testing

Python standard test tools

unittest doctest

I do test

unittest pitfalls unittest alternatives

nose

Test runner Writing tests Writing test fixtures Integration with setuptools and plugin system Wrap-up

py.test

Writing test fixtures Disabling test functions and classes Automated distributed tests Wrap-up

Testing coverage Fakes and mocks

Building a fake Using mocks

Testing environment and dependency compatibility

Dependency matrix testing

Document-driven development

Writing a story

Summary

Section 4: Need for Speed Optimization - Principles and Profiling Techniques

Technical requirements The three rules of optimization

Making it work first Working from the user's point of view Keeping the code readable and maintainable

Optimization strategy

Looking for another culprit Scaling the hardware Writing a speed test

Finding bottlenecks

Profiling CPU usage

Macro-profiling Micro-profiling

Profiling memory usage

How Python deals with memory Profiling memory

objgraph

C code memory leaks

Profiling network usage

Tracing network transactions

Summary

Optimization - Some Powerful Techniques

Technical requirements Defining complexity

Cyclomatic complexity The big O notation

Reducing complexity by choosing proper data structures

Searching in a list

Using sets

Using collections

deque defaultdict namedtuple

Using architectural trade-offs

Using heuristics and approximation algorithms Using task queues and delayed processing Using probabilistic data structures

Caching

Deterministic caching Non-deterministic caching Cache services

Memcached

Summary

Concurrency

Technical requirements Why concurrency? Multithreading

What is multithreading? How Python deals with threads When should we use threading?

Building responsive interfaces Delegating work Multiuser applications An example of a threaded application

Using one thread per item Using a thread pool Using two-way queues Dealing with errors and rate limiting

Multiprocessing

The built-in multiprocessing module

Using process pools Using multiprocessing.dummy as the multithreading interface

Asynchronous programming

Cooperative multitasking and asynchronous I/O Python async and await keywords asyncio in older versions of Python A practical example of asynchronous programming Integrating non-asynchronous code with async using futures

Executors and futures Using executors in an event loop

Summary

Section 5: Technical Architecture Event-Driven and Signal Programming

Technical requirements What exactly is event-driven programming?

Event-driven != asynchronous Event-driven programming in GUIs Event-driven communication

Various styles of event-driven programming

Callback-based style Subject-based style Topic-based style

Event-driven architectures

Event and message queues

Summary

Useful Design Patterns

Technical requirements Creational patterns

Singleton

Structural patterns

Adapter

Interfaces

Using zope.interface Using function annotations and abstract base classes Using collections.abc

Proxy Facade

Behavioral patterns

Observer Visitor Template

Summary

reStructuredText Primer

reStructuredText

Section structure Lists Inline markup Literal block Links

Other Books You May Enjoy

Leave a review - let other readers know what you think

← Prev
Back
Next →

← Prev
Back
Next →