Chapter 20. Test Fixtures and a Decorator for Explicit Waits

Now that we have a functional authentication system, we want to use it to identify users, and be able to show them all the lists they have created.

To do that, we’re going to have to write FTs that have a logged-in user. Rather than making each test go through the (time-consuming) login email dance, we want to be able to skip that part.

This is about separation of concerns. Functional tests aren’t like unit tests, in that they don’t usually have a single assertion. But, conceptually, they should be testing a single thing. There’s no need for every single FT to test the login/logout mechanisms. If we can figure out a way to “cheat” and skip that part, we’ll spend less time waiting for duplicated test paths.

Tip

Don’t overdo de-duplication in FTs. One of the benefits of an FT is that it can catch strange and unpredictable interactions between different parts of your application.

Note

This chapter has only just been rewritten for the new edition, so let me know via obeythetestinggoat@gmail.com if you spot any problems or have any suggestions for improvement!

Skipping the Login Process by Pre-creating a Session

It’s quite common for a user to return to a site and still have a cookie, which means they are “pre-authenticated”, so this isn’t an unrealistic cheat at all. Here’s how you can set it up:

functional_tests/test_my_lists.py

from django.conf import settings
from django.contrib.auth import BACKEND_SESSION_KEY, SESSION_KEY, get_user_model
from django.contrib.sessions.backends.db import SessionStore
from .base import FunctionalTest
User = get_user_model()


class MyListsTest(FunctionalTest):

    def create_pre_authenticated_session(self, email):
        user = User.objects.create(email=email)
        session = SessionStore()
        session[SESSION_KEY] = user.pk 
        session[BACKEND_SESSION_KEY] = settings.AUTHENTICATION_BACKENDS[0]
        session.save()
        ## to set a cookie we need to first visit the domain.
        ## 404 pages load the quickest!
        self.browser.get(self.live_server_url + "/404_no_such_url/")
        self.browser.add_cookie(dict(
            name=settings.SESSION_COOKIE_NAME,
            value=session.session_key, 
            path='/',
        ))

We create a session object in the database. The session key is the primary key of the user object (which is actually the user’s email address).

We then add a cookie to the browser that matches the session on the server—on our next visit to the site, the server should recognise us as a logged-in user.

Note that, as it is, this will only work because we’re using LiveServerTestCase, so the User and Session objects we create will end up in the same database as the test server. Later we’ll need to modify it so that it works against the database on the staging server too.

Django Sessions: How a User’s Cookies Tell the Server She Is Authenticated

Being an attempt to explain sessions, cookies, and authentication in Django.

Because HTTP is stateless, servers need a way of recognising different clients with every single request. IP addresses can be shared, so the usual solution is to give each client a unique session ID, which it will store in a cookie, and submit with every request. The server will store that ID somewhere (by default, in the database), and then it can recognise each request that comes in as being from a particular client.

If you log in to the site using the dev server, you can actually take a look at your session ID by hand if you like. It’s stored under the key sessionid by default. See Figure 20-1.

Figure 20-1. Examining the session cookie in the Debug toolbar

These session cookies are set for all visitors to a Django site, whether they’re logged in or not.

When we want to recognise a client as being a logged-in and authenticated user, again, rather asking the client to send their username and password with every single request, the server can actually just mark that client’s session as being an authenticated session, and associate it with a user ID in its database.

A session is a dictionary-like data structure, and the user ID is stored under the key given by django.contrib.auth.SESSION_KEY. You can check this out in a ./manage.py shell if you like:

$ python manage.py shell
[...]
In [1]: from django.contrib.sessions.models import Session

# substitute your session id from your browser cookie here
In [2]: session = Session.objects.get(
    session_key="8u0pygdy9blo696g3n4o078ygt6l8y0y"
)

In [3]: print(session.get_decoded())
{'_auth_user_id': 'obeythetestinggoat@gmail.com', '_auth_user_backend':
'accounts.authentication.PasswordlessAuthenticationBackend'}

You can also store any other information you like on a user’s session, as a way of temporarily keeping track of some state. This works for non–logged-in users too. Just use request.session inside any view, and it works as a dict. There’s more information in the Django docs on sessions.

Checking That It Works

To check that it works, it would be good to use some of the code from our previous test. Let’s make a couple of functions called wait_to_be_logged_in and wait_to_be_logged_out. To access them from a different test, we’ll need to pull them up into FunctionalTest. We’ll also tweak them slightly so that they can take an arbitrary email address as a parameter:

functional_tests/base.py (ch18l002)

class FunctionalTest(StaticLiveServerTestCase):
    [...]

    def wait_to_be_logged_in(self, email):
        self.wait_for(
            lambda: self.browser.find_element_by_link_text('Log out')
        )
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertIn(email, navbar.text)


    def wait_to_be_logged_out(self, email):
        self.wait_for(
            lambda: self.browser.find_element_by_name('email')
        )
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertNotIn(email, navbar.text)

Hm, that’s not bad, but I’m not quite happy with the amount of duplication of wait_for stuff in here. Let’s make a note to come back to it, and get these helpers working.

SCRATCHPAD:

• Clean up wait_for stuff in base.py

First we use them in test_login.py:

functional_tests/test_login.py (ch18l003)

    def test_can_get_email_link_to_log_in(self):
        [...]
        # she is logged in!
        self.wait_to_be_logged_in(email=TEST_EMAIL)

        # Now she logs out
        self.browser.find_element_by_link_text('Log out').click()

        # She is logged out
        self.wait_to_be_logged_out(email=TEST_EMAIL)

Just to make sure we haven’t broken anything, we rerun the login test:

$ python manage.py test functional_tests.test_login
[...]
OK

And now we can write a placeholder for the “My Lists” test, to see if our pre-authenticated session creator really does work:

functional_tests/test_my_lists.py (ch18l004)

    def test_logged_in_users_lists_are_saved_as_my_lists(self):
        email = 'edith@example.com'
        self.browser.get(self.live_server_url)
        self.wait_to_be_logged_out(email)

        # Edith is a logged-in user
        self.create_pre_authenticated_session(email)
        self.browser.get(self.live_server_url)
        self.wait_to_be_logged_in(email)

That gets us:

$ python manage.py test functional_tests.test_my_lists
[...]
OK

That’s a good place for a commit:

$ git add functional_tests
$ git commit -m "test_my_lists: precreate sessions, move login checks into base"

JSON Test Fixtures Considered Harmful

When we pre-populate the database with test data, as we’ve done here with the User object and its associated Session object, what we’re doing is setting up a “test fixture”.

Django comes with built-in support for saving database objects as JSON (using the manage.py dumpdata), and automatically loading them in your test runs using the fixtures class attribute on TestCase.

More and more people are starting to say: don’t use JSON fixtures. They’re a nightmare to maintain when your model changes. Plus it’s difficult for the reader to tell which of the many attribute values specified in the JSON are critical for the behaviour under test, and which are just filler. Finally, even if tests start out sharing fixtures, sooner or later one test will want slightly different versions of the data, and you end up copying the whole thing around to keep them isolated, and again it’s hard to tell what’s relevant to the test and what is just happenstance.

It’s usually much more straightforward to just load the data directly using the Django ORM.

Tip

Once you have more than a handful of fields on a model, and/or several related models, even using the ORM can be cumbersome. In this case, there’s a tool that lots of people swear by called factory_boy.

Our Final Explicit Wait Helper: A Wait Decorator

We’ve used decorators a few times in our code so far, but it’s time to learn how they actually work by making one of our own.

First, let’s imagine how we might want our decorator to work. It would be nice to be able to replace all the custom wait/retry/timeout logic in wait_for_row_​in_list_table and the inline self.wait_fors in the wait_to_be_logged_in/out. Something like this would look lovely:

functional_tests/base.py (ch18l005)

    @wait
    def wait_for_row_in_list_table(self, row_text):
        table = self.browser.find_element_by_id('id_list_table')
        rows = table.find_elements_by_tag_name('tr')
        self.assertIn(row_text, [row.text for row in rows])


    @wait
    def wait_to_be_logged_in(self, email):
        self.browser.find_element_by_link_text('Log out')
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertIn(email, navbar.text)


    @wait
    def wait_to_be_logged_out(self, email):
        self.browser.find_element_by_name('email')
        navbar = self.browser.find_element_by_css_selector('.navbar')
        self.assertNotIn(email, navbar.text)

Are you ready to dive in? Although decorators are quite difficult to wrap your head around (I know it took me a long time before I was comfortable with them, and I still have to think about them quite carefully whenever I make one), the nice thing is that we’ve already dipped our toes into functional programming in our self.wait_for helper function. That’s a function that takes another function as an argument, and a decorator is the same. The difference is that the decorator doesn’t actually execute any code itself—it returns a modified version of the function that it was given.

Our decorator wants to return a new function which will keep calling the function it was given, catching our usual exceptions, until a timeout occurs. Here’s a first cut:

functional_tests/base.py (ch18l006)

def wait(fn):  
    def modified_fn():  
        start_time = time.time()
        while True:  
            try:
                return fn()  
            except (AssertionError, WebDriverException) as e:  
                if time.time() - start_time > MAX_WAIT:
                    raise e
                time.sleep(0.5)
    return modified_fn  

A decorator is a way of modifying a function; it takes a function as an argument…

and returns another function as the modified (or “decorated”) version.

Here’s where we create our modified function.

And here’s our familiar loop, which will keep going, catching the usual exceptions, until our timeout expires.

And as always, we call our function and return immediately if there are no exceptions.

That’s almost right, but not quite; try running it?

$ python manage.py test functional_tests.test_my_lists
[...]
    self.wait_to_be_logged_out(email)
TypeError: modified_fn() takes 0 positional arguments but 2 were given

Unlike in self.wait_for, the decorator is being applied to functions that have arguments:

functional_tests/base.py

    @wait
    def wait_to_be_logged_in(self, email):
        self.browser.find_element_by_link_text('Log out')

wait_to_be_logged_in takes self and email as positional arguments. But when it’s decorated, it’s replaced with modified_fn, which takes no arguments. How do we magically make it so our modified_fn can handle the same arguments as whatever fn the decorator gets given has?

The answer is a bit of Python magic, *args and **kwargs, more formally known as “variadic arguments”, apparently (I only just learned that):

functional_tests/base.py (ch18l007)

def wait(fn):
    def modified_fn(*args, **kwargs):  
        start_time = time.time()
        while True:
            try:
                return fn(*args, **kwargs)  
            except (AssertionError, WebDriverException) as e:
                if time.time() - start_time > MAX_WAIT:
                    raise e
                time.sleep(0.5)
    return modified_fn

Using *args and **kwargs, we specify that modified_fn may take any arbitrary positional and keyword arguments.

As we’ve captured them in the function definition, we make sure to pass those same arguments to fn when we actually call it.

One of the fun things this can be used for is to make a decorator that changes the arguments of a function. But we won’t get into that now. The main thing is that our decorator now works:

$ python manage.py test functional_tests.test_my_lists
[...]
OK

And do you know what’s truly satisfying? We can use our wait decorator for our self.wait_for helper as well! Like this:

functional_tests/base.py (ch18l008)

    @wait
    def wait_for(self, fn):
        return fn()

Lovely! Now all our wait/retry logic is encapsulated in a single place, and we have a nice easy way of applying those waits, either inline in our FTs using self.wait_for, or on any helper function using the @wait decorator.

In the next chapter we’ll try to deploy our code to staging, and use the pre-authenticated session fixtures on the server. As we’ll see it’ll help us catch a little bug or two!

Lessons Learned

Decorators are nice

Decorators can be a great way of abstracting out different levels of concerns. They let us write our test assertions without having to think about waits at the same time.

De-duplicate your FTs, with caution

Every single FT doesn’t need to test every single part of your application. In our case, we wanted to avoid going through the full login process for every FT that needs an authenticated user, so we used a test fixture to “cheat” and skip that part. You might find other things you want to skip in your FTs. A word of caution, however: functional tests are there to catch unpredictable interactions between different parts of your application, so be wary of pushing de-duplication to the extreme.

Test fixtures

Test fixtures refers to test data that needs to be set up as a precondition before a test is run—often this means populating the database with some information, but as we’ve seen (with browser cookies), it can involve other types of preconditions.

Avoid JSON fixtures

Django makes it easy to save and restore data from the database in JSON format (and others) using the dumpdata and loaddata management commands. Most people recommend against using these for test fixtures, as they are painful to manage when your database schema changes. Use the ORM, or a tool like factory_boy.