Using one thread per item

Now, it is time for improvement. We don't do a lot of processing in Python, and long execution times are caused by communication with the external service. We send an HTTP request to the remote server, it calculates the answer, and then we wait until the response is transferred back. There is a lot of I/O involved, so multithreading seems like a viable option. We can start all the requests at once in separate threads and then just wait until we receive data from all of them. If the service that we are communicating with is able to process our requests concurrently, we should definitely see a performance improvement.

So, let's start with the easiest approach. Python provides clean and easy to use abstraction over system threads with the threading module. The core of this standard library is the Thread class, which represents a single thread instance. Here is a modified version of the main() function that creates and starts a new thread for every place to geocode and then waits until all the threads finish:

from threading import Thread

def main():
    threads = []
    for base in BASES:
        thread = Thread(target=fetch_rates, args=[base])
        thread.start()
        threads.append(thread)

    while threads:
        threads.pop().join()

It is a quick and dirty solution that approaches the problem in a bit of a frivolous way. And it is not a way to write reliable software that will serve thousands or millions of users. It has some serious issues that we will have to address later. But hey, it works, as we can see from the following code:

$ python3 threads_one_per_item.py
1 CZK =   0.044 USD,   0.039 EUR,   0.167 PLN,   0.375 NOK,     1.0 CZK
1 NOK =   0.117 USD,   0.104 EUR,   0.446 PLN,     1.0 NOK,    2.66 CZK
1 USD =     1.0 USD,   0.887 EUR,     3.8 PLN,    8.53 NOK,    22.7 CZK
1 EUR =    1.13 USD,     1.0 EUR,    4.29 PLN,    9.62 NOK,    25.6 CZK
1 PLN =   0.263 USD,   0.233 EUR,     1.0 PLN,    2.24 NOK,    5.98 CZK
time elapsed: 0.13s

And it is also considerably faster.

So, when we know that threads have a beneficial effect on our application, it is time to use them in a saner way. First, we need to identify the following issues in the preceding code:

• We start a new thread for every parameter. Thread initialization also takes some time, but this minor overhead is not the only problem. Threads also consume other resources, like memory or file descriptors. Our example input has a strictly defined number of items, but what if it did not have a limit? You definitely don't want to run an unbound number of threads that depend on the arbitrary size of data input.
• The fetch_rates() function that's executed in threads calls the built-in print() function, and in practice it is very unlikely that you would want to do that outside of the main application thread. It is mainly due to the way the standard output is buffered in Python. You can experience malformed output when multiple calls to this function interleave between threads. Also, the print() function is considered slow. If used recklessly in multiple threads, it can lead to serialization that will waste all your benefits of multithreading.
• Last but not least, by delegating every function call to a separate thread, we make it extremely hard to control the rate at which our input is processed. Yes, we want to do the job as fast as possible, but very often, external services enforce hard limits on the rate of requests from a single client that they can process. Sometimes, it is reasonable to design the program in a way that enables you to throttle the rate of processing, so your application won't be blacklisted by external APIs for abusing their usage limits.

In the next section, we will see how to use a thread pool.