Learning Concurrency in Python by Forbes, Elliot -- Read -- Imperial Library of Trantor

Log In

Or create an account ->

Imperial Library

Home
About
News
Upload
Forum

Help

Login/SignUp

Index

Learning Concurrency in Python Title Page Copyright Learning Concurrency in Python Credits About the Author About the Reviewer www.PacktPub.com Why subscribe? Customer Feedback Table of Contents Preface What this book covers What you need for this book Who this book is for Conventions Reader feedback Downloading the example code Errata Piracy Questions Speed It Up! History of concurrency Threads and multithreading What is a thread? Types of threads What is multithreading? Processes Properties of processes Multiprocessing Event-driven programming Turtle Breaking it down Reactive programming ReactiveX - RxPy Breaking it down GPU programming PyCUDA OpenCL Theano The limitations of Python Jython IronPython Why should we use Python? Concurrent image download Sequential download Breaking it down Concurrent download Breaking it down Improving number crunching with multiprocessing Sequential prime factorization Breaking it down Concurrent prime factorization Breaking it down Summary Parallelize It Understanding concurrency Properties of concurrent systems I/O bottlenecks Understanding parallelism CPU-bound bottlenecks How do they work on a CPU? Single-core CPUs Clock rate Martelli model of scalability Time-sharing - the task scheduler Multi-core processors System architecture styles SISD SIMD MISD MIMD Computer memory architecture styles UMA NUMA Summary Life of a Thread Threads in Python Thread state State flow chart Python example of thread state Breaking it down Different types of threads POSIX threads Windows threads The ways to start a thread Starting a thread Inheriting from the thread class Breaking it down Forking Example Breaking it down Daemonizing a thread Example Breaking it down Handling threads in Python Starting loads of threads Example Breaking it down Slowing down programs using threads Example Breaking it down Getting the total number of active threads Example Breaking it down Getting the current thread Example Breaking it down Main thread Example Breaking it down Enumerating all threads Example Breaking it down Identifying threads Example Breakdown Ending a thread Best practice in stopping threads Example Output Orphan processes How does the operating system handle threads Creating processes versus threads Example Breaking it down Multithreading models One-to-one thread mapping Many-to-one Many-to-many Summary Synchronization between Threads Synchronization between threads The Dining Philosophers Example Output Race conditions Process execution sequence The solution Critical sections Filesystem Life-critical systems Shared resources and data races The join method Breaking it down Putting it together Locks Example Breaking it down RLocks Example Breaking it down Output RLocks versus regular locks Condition Definition Example Our publisher Our subscriber Kicking it off The results Semaphores Class definition Example The TicketSeller class Output Thread race Bounded semaphores Events Example Breaking it down Barriers Example Breaking it down Output Summary Communication between Threads Standard data structures Sets Extending the class Exercise - extending other primitives Decorator Class decorator Lists Queues FIFO queues Example Breaking it down Output LIFO queues Example Breaking it down Output PriorityQueue Example Breakdown Output Queue objects Full/empty queues Example Output The join() function Example Breakdown Output Deque objects Example Breakdown Output Appending elements Example Breaking it down Output Popping elements Example Breaking it down Output Inserting elements Example Breaking it down Output Rotation Example Breaking it down Output Defining your own thread-safe communication structures A web Crawler example Requirements Design Our Crawler class Our starting point Extending the queue object Breaking it down Output Future enhancements Conclusion Exercise - testing your skills Summary Debug and Benchmark Testing strategies Why do we test? Testing concurrent software systems What should we test? Unit tests PyUnit Example Output Expanding our test suite Unit testing concurrent code Integration tests Debugging Make it work as a single thread Pdb An interactive example Catching exceptions in child threads Benchmarking The timeit module Timeit versus time Command-line example Importing timeit into your code Utilizing decorators Timing context manager Output Profiling cProfile Simple profile example The line_profiler tool Kernprof Memory profiling Memory profile graphs Summary Executors and Pools Concurrent futures Executor objects Creating a ThreadPoolExecutor Example Output Context manager Example Output Maps Example Output Shutdown of executor objects Example Output Future objects Methods in future objects The result() method The add_done_callback() method The .running() method The cancel() method The .exception() method The .done() method Unit testing future objects The set_running_or_notify_cancel() method The set_result() method The set_exception() method Cancelling callable Example Output Getting the result Example Output Using as_completed Example Output Setting callbacks Example Output Chaining callbacks Exception classes Example Output ProcessPoolExecutor Creating a ProcessPoolExecutor Example Output Context Manager Example Output Exercise Getting started Improving the speed of computationally bound problems Full code sample Output Improving our crawler The plan New improvements Refactoring our code Storing the results in a CSV file Exercise - capture more info from each page crawl concurrent.futures in Python 2.7 Summary Multiprocessing Working around the GIL Utilizing sub-processes Example Output The life of a process Starting a process using fork Spawning a process Forkserver Daemon processes Example Breaking it down Output Identifying processes using PIDs Example Output Terminating a process Example Getting the current process Subclassing processes Example Output Multiprocessing pools The difference between concurrent.futures.ProcessPoolExecutor and Pool Context manager Example Output Submitting tasks to a process pool Apply Apply_async Map Map_async Imap Imap_unordered Starmap Starmap_async Maxtasksperchild Communication between processes Pipes Anonymous pipes Named pipes Working with pipes Example Handling Exceptions Using pipes Multiprocessing managers Namespaces Example Queues Example Output Listeners and clients Example The Listener class The Client class Output Logging Example Communicating sequential processes PyCSP Processes in PyCSP Output Summary Event-Driven Programming Event-driven programming The event loop Asyncio Getting started Event loops The run_forever() method The run_until_complete() method The stop() method The is_closed() method The close() function Tasks Example The all_tasks(loop=None) method The current_tasks() function The cancel() function Task functions The as_completed(fs, *, loop=None, timeout=None) function The ensure_future(coro_or_future, *, loop=None) function The wrap_future(future, *, loop=None) function The gather(*coroes_or_futures, loop=None, return_exceptions=False) function The wait() function Futures Example Output Coroutines Chaining coroutines Output Transports Protocols Synchronization between coroutines Locks Queues Events and conditions Semaphores and BoundedSemaphores Sub-processes Debugging asyncio programs Debug mode Twisted A simple web server example Gevent Event loops Greenlets Simple example-hostnames Output Monkey patching Summary Reactive Programming Basic reactive programming Maintaining purity ReactiveX, or RX Installing RxPY Observables Creating observers Example Example 2 Breaking it down Output Lambda functions Example Breaking it down On_next, on_completed, and on_error in lambda form Output Operators and chaining Filter example Breaking it down Chained operators The different operators Creating observables Transforming observables Filtering observables Error-handling observables Hot and cold observables Emitting events Example Breaking it down Output Multicasting Example Output Combining observables Zip() example Output The merge_all() operator Output Concurrency Example Output PyFunctional Installation and official docs Simple example Output Streams, transformations, and actions Filtering lists Output Reading/writing SQLite3 Compressed files Parallel execution Summary Using the GPU Basic reactive programming Maintaining purity ReactiveX, or RX Installing RxPY Observables Creating observers Example Example 2 Breaking it down Output Lambda functions Example Breaking it down On_next, on_completed, and on_error in lambda form Output Operators and chaining Filter example Breaking it down Chained operators The different operators Creating observables Transforming observables Filtering observables Error-handling observables Hot and cold observables Emitting events Example Breaking it down Output Multicasting Example Output Combining observables Zip() example Output The merge_all() operator Output Concurrency Example Output PyFunctional Installation and official docs Simple example Output Streams, transformations, and actions Filtering lists Output Reading/writing SQLite3 Compressed files Parallel execution Summary Choosing a Solution Libraries not covered in this book GPU PyGPU Event-driven and reactive libraries Tornado Flask Celery Data science Pandas Matplotlib TensorFlow Designing your systems Requirements Functional requirements Non-functional requirements Design Computationally expensive Event-heavy applications I/O-heavy applications Recommended design books Software Architecture with Python Python: Master the Art of Design Patterns Research Summary

← Prev
Back
Next →

← Prev
Back
Next →

Chief Librarian: Las Zenow <zenow@riseup.net>
Fork the source code from gitlab.

This is a mirror of the Tor onion service:
http://kx5thpx2olielkihfyo4jgjqfb7zx7wxr3sd4xzt26ochei4m6f7tayd.onion