Using multiprocessing.dummy as the multithreading interface

The high-level abstractions from the multiprocessing module, such as the Pool class, are great advantages over the simple tools provided in the threading module. But it does not mean that multiprocessing is always better than multithreading. There are a lot of use cases where threads may be a better solution than processes. This is especially true for situations where low latency and/or high resource efficiency is required.

Still, it does not mean that you need to sacrifice all the useful abstractions from the multiprocessing module whenever you want to use threads instead of processes. There is the multiprocessing.dummy module that replicates the multiprocessing API, but uses multiple threads instead of forking/spawning new processes.

This allows you to reduce the amount of boilerplate in your code and also have a more pluggable code structure. For instance, let's take yet another look at our main() function from the previous examples. We could give the user control over which processing backend to use (processes or threads). We could do that simply by replacing the pool object constructor class, as follows:

from multiprocessing import Pool as ProcessPool 
from multiprocessing.dummy import Pool as ThreadPool 
 
 
def main(use_threads=False): 
    if use_threads: 
        pool_cls = ThreadPool 
    else: 
        pool_cls = ProcessPool 
 
    with pool_cls(POOL_SIZE) as pool: 
        results = pool.map(fetch_rates, BASES) 
 
    for result in results: 
        present_result(*result) 

Let's take a look at asynchronous programming in the next section.