Python async and await keywords

The async and await keywords are the main building blocks in Python asynchronous programming.

The async keyword, when used before the def statement, defines a new coroutine. The execution of the coroutine function may be suspended and resumed in strictly defined circumstances. Its syntax and behavior are very similar to generators (refer to Chapter 3, Modern Syntax Elements - Below the Class Level). In fact, generators need to be used in the older versions of Python whenever you want to implement coroutines. Here is an example of function declaration that uses the async keyword:

async def async_hello(): 
    print("hello, world!")

Functions defined with the async keyword are special. When called, they do not execute the code inside, but instead return a coroutine object, for example:

>>> async def async_hello():
...     print("hello, world!")
... 
>>> async_hello()
<coroutine object async_hello at 0x1014129e8>

The coroutine object does not do anything until its execution is scheduled in the event loop. The asyncio module is available in order to provide the basic event loop implementation, as well as a lot of other asynchronous utilities, as follows:

>>> import asyncio
>>> async def async_hello():
...     print("hello, world!")
... 
>>> loop = asyncio.get_event_loop()
>>> loop.run_until_complete(async_hello())
hello, world!
>>> loop.close()

Obviously, since we have created only one simple coroutine, there is no concurrency involved in our program. In order to see something really concurrent, we need to create more tasks that will be executed by the event loop.

New tasks can be added to the loop by calling the loop.create_task() method or by providing another object to wait for using the asyncio.wait() function. We will use the latter approach and try to asynchronously print a sequence of numbers that's been generated with the range() function, as follows:

import asyncio 
 
async def print_number(number): 
    print(number) 
 
 
if __name__ == "__main__": 
    loop = asyncio.get_event_loop() 
 
    loop.run_until_complete( 
        asyncio.wait([ 
            print_number(number) 
            for number in range(10) 
        ]) 
    ) 
    loop.close()

The asyncio.wait() function accepts a list of coroutine objects and returns immediately. The result is a generator that yields objects representing future results (so-called futures). As the name suggests, it is used to wait until all of the provided coroutines complete. The reason why it returns a generator instead of a coroutine object is backwards compatibility with previous versions of Python, which will be explained later in the asyncio in the older version of Python section. The result of running this script may be as follows:

$ python asyncprint.py 
0
7
8
3
9
4
1
5
2
6

As we can see, the numbers are not printed in the same order as the ones we created for our coroutines. But this is exactly what we wanted to achieve.

The second important keyword that was added in Python 3.5 was await. It is used to wait for results of coroutines or a future (explained later), and release the control over execution to the event loop. To better understand how it works, we need to review a more complex example of code.

Let's say we want to create the following two coroutines that will perform some simple task in a loop:

Let's start with the following simple implementation that has some concurrency issues that we will later try to improve with the additional await usage:

import time 
import random 
import asyncio 
 
 
async def waiter(name): 
    for _ in range(4): 
        time_to_sleep = random.randint(1, 3) / 4 
        time.sleep(time_to_sleep) 
        print( 
            "{} waited {} seconds" 
            "".format(name, time_to_sleep) 
        ) 
 
 
async def main(): 
    await asyncio.wait([waiter("first"), waiter("second")]) 
 
 
if __name__ == "__main__": 
    loop = asyncio.get_event_loop() 
    loop.run_until_complete(main()) 
    loop.close()

When executed in the Terminal (with time command to measure time), it might give the following output:

$ time python corowait.py 
second waited 0.25 seconds
second waited 0.25 seconds
second waited 0.5 seconds
second waited 0.5 seconds
first waited 0.75 seconds
first waited 0.75 seconds
first waited 0.25 seconds
first waited 0.25 seconds
    
real  0m3.734s
user  0m0.153s
sys   0m0.028s

As we can see, both the coroutines completed their execution, but not in an asynchronous manner. The reason is that they both use the time.sleep() function that is blocking, but not releasing the control to the event loop. This would work better in a multithreaded setup, but we don't want to use threads now. So, how can we fix this?

The answer is to use asyncio.sleep(), which is the asynchronous version of time.sleep(), and await its result using the await keyword. We have already used this statement in the first version of the main() function, but it was only to improve the clarity of the code. It clearly did not make our implementation more concurrent. Let's see the following improved version of the waiter() coroutine that uses await asyncio.sleep():

async def waiter(name): 
    for _ in range(4): 
        time_to_sleep = random.randint(1, 3) / 4 
        await asyncio.sleep(time_to_sleep) 
        print( 
            "{} waited {} seconds" 
            "".format(name, time_to_sleep) 
        )

If we run the updated script, we can see how the output of two functions interleave with each other:

$ time python corowait_improved.py 
second waited 0.25 seconds
first waited 0.25 seconds
second waited 0.25 seconds
first waited 0.5 seconds
first waited 0.25 seconds
second waited 0.75 seconds
first waited 0.25 seconds
second waited 0.5 seconds
    
real  0m1.953s
user  0m0.149s
sys   0m0.026s

The additional advantage of this simple improvement is that the code ran faster. The overall execution time was less than the sum of all sleeping times because coroutines were cooperatively releasing the control.

In the next section, we will take a look at asyncio in older versions of Python.