Log In
Or create an account ->
Imperial Library
Home
About
News
Upload
Forum
Help
Login/SignUp
Index
Why I Wrote a Book About Test-Driven Development
Part I. The Basics of TDD and Django
Chapter 1. Getting Django Set Up Using a Functional Test TDD isn’t something that comes naturally. It’s a discipline, like a martial art, and just like in a Kung-Fu movie, you need a bad-tempered and unreasonable master to force you to learn the discipline. Ours is the Testing Goat. Obey the Testing Goat! Do Nothing Until You Have a Test The Testing Goat is the unofficial mascot of TDD in the Python testing community. It probably means different things to different people, but, to me, the Testing Goat is a voice inside my head that keeps me on the True Path of Testing—like one of those little angels or demons that pop up above your shoulder in the cartoons, but with a very niche set of concerns. I hope, with this book, to install the Testing Goat inside your head too. We’ve decided to build a website, even if we’re not quite sure what it’s going to do yet. Normally the first step in web development is getting your web framework installed and configured. Download this, install that, con
Chapter 1. Getting Django Set Up Using a Functional Test
Chapter 2. Extending Our Functional Test Using the unittest Module Let’s adapt our test, which currently checks for the default Django “it worked” page, and check instead for some of the things we want to see on the real front page of our site. Time to reveal what kind of web app we’re building: a to-do lists site! In doing so we’re very much following fashion: a few years ago all web tutorials were about building a blog. Then it was forums and polls; nowadays it’s all to-do lists. The reason is that a to-do list is a really nice example. At its most basic it is very simple indeed—just a list of text strings—so it’s easy to get a “minimum viable” list app up and running. But it can be extended in all sorts of ways—different persistence models, adding deadlines, reminders, sharing with other users, and improving the client-side UI. There’s no reason to be limited to just “to-do” lists either; they could be any kind of lists. But the point is that it should allow me to demonstrate all of
Chapter 2. Extending Our Functional Test Using the unittest Module
Chapter 3. Testing a Simple Home Page with Unit Tests We finished the last chapter with a functional test failing, telling us that it wanted the home page for our site to have “To-Do” in its title. It’s time to start working on our application. Warning: Things Are About to Get Real The first two chapters were intentionally nice and light. From now on, we get into some more meaty coding. Here’s a prediction: at some point, things are going to go wrong. You’re going to see different results from what I say you should see. This is a Good Thing, because it will be a genuine character-building Learning Experience™. One possibility is that I’ve given some ambiguous explanations, and you’ve done something different from what I intended. Step back and have a think about what we’re trying to achieve at this point in the book. Which file are we editing, what do we want the user to be able to do, what are we testing and why? It may be that you’ve edited the wrong file or function, or are running
Chapter 3. Testing a Simple Home Page with Unit Tests
Chapter 4. What Are We Doing with All These Tests? (And, Refactoring) Now that we’ve seen the basics of TDD in action, it’s time to pause and talk about why we’re doing it. I’m imagining several of you, dear readers, have been holding back some seething frustration—perhaps some of you have done a bit of unit testing before, and perhaps some of you are just in a hurry. You’ve been biting back questions like: Aren’t all these tests a bit excessive? Surely some of them are redundant? There’s duplication between the functional tests and the unit tests. I mean, what are you doing importing django.core.urlresolvers in your unit tests? Isn’t that testing Django, i.e., testing third-party code? I thought that was a no-no? Those unit tests seemed way too trivial—testing one line of declaration, and a one-line function that returns a constant! Isn’t that just a waste of time? Shouldn’t we save our tests for more complex things? What about all those tiny changes during the unit-test/code cycle? S
Chapter 4. What Are We Doing with All These Tests? (And, Refactoring)
Chapter 5. Saving User Input: Testing the Database We want to take the to-do item input from the user and send it to the server, so that we can save it somehow and display it back to her later. As I started writing this chapter, I immediately skipped to what I thought was the right design: multiple models for lists and list items, a bunch of different URLs for adding new lists and items, three new view functions, and about half a dozen new unit tests for all of the above. But I stopped myself. Although I was pretty sure I was smart enough to handle all those problems at once, the point of TDD is to allow you to do one thing at a time, when you need to. So I decided to be deliberately short-sighted, and at any given moment only do what was necessary to get the functional tests a little further. It’s a demonstration of how TDD can support an iterative style of development—it may not be the quickest route, but you do get there in the end. There’s a neat side benefit, which is that it allo
Chapter 5. Saving User Input: Testing the Database
Chapter 6. Improving Functional Tests: Ensuring Isolation and Removing Voodoo Sleeps Before we dive in and fix our real problem, let’s take care of a couple of housekeeping items. At the end of the last chapter, we made a note that different test runs were interfering with each other, so we’ll fix that. I’m also not happy with all these time.sleeps peppered through the code, they seem a bit unscientific, so we’ll replace them with something more reliable. SCRATCHPAD: Clean up after FT runs Remove time.sleeps Both of these changes will be moving towards testing “best practices”, making our tests more deterministic and more reliable. Ensuring Test Isolation in Functional Tests We ended the last chapter with a classic testing problem: how to ensure isolation between tests. Each run of our functional tests was leaving list items lying around in the database, and that would interfere with the test results when you next ran the tests. When we run unit tests, the Django test runner automatica
Chapter 6. Improving Functional Tests: Ensuring Isolation and Removing Voodoo Sleeps
Chapter 7. Working Incrementally Now let’s address our real problem, which is that our design only allows for one global list. In this chapter I’ll demonstrate a critical TDD technique: how to adapt existing code using an incremental, step-by-step process which takes you from working state to working state. Testing Goat, not Refactoring Cat. Small Design When Necessary Let’s have a think about how we want support for multiple lists to work. Currently the FT (which is the closest we have to a design document) says this: functional_tests/tests.py # Edith wonders whether the site will remember her list. Then she sees # that the site has generated a unique URL for her -- there is some # explanatory text to that effect. self.fail('Finish the test!') # She visits that URL - her to-do list is still there. # Satisfied, she goes back to sleep But really we want to expand on this, by saying that different users don’t see each other’s lists, and each get their own URL as a way of going back to th
Chapter 7. Working Incrementally
Part II. Web Development Sine Qua Nons
Chapter 8. Prettification: Layout and Styling, and What to Test About It We’re starting to think about releasing the first version of our site, but we’re a bit embarrassed by how ugly it looks at the moment. In this chapter, we’ll cover some of the basics of styling, including integrating an HTML/CSS framework called Bootstrap. We’ll learn how static files work in Django, and what we need to do about testing them. What to Functionally Test About Layout and Style Our site is undeniably a bit unattractive at the moment (Figure 8-1). Note If you spin up your dev server with manage.py runserver, you may run into a database error “table lists_item has no column named list_id”. You need to update your local database to reflect the changes we made in models.py. Use manage.py migrate. If it gives you any grief about IntegrityErrors, just delete1 the database file and try again. We can’t be adding to Python’s reputation for being ugly, so let’s do a tiny bit of polishing. Here’s a few things we
Chapter 8. Prettification: Layout and Styling, and What to Test About It
Chapter 9. Testing Deployment Using a Staging Site Is all fun and game until you are need of put it in production. Devops Borat It’s time to deploy the first version of our site and make it public. They say that if you wait until you feel ready to ship, then you’ve waited too long. Is our site usable? Is it better than nothing? Can we make lists on it? Yes, yes, yes. No, you can’t log in yet. No, you can’t mark tasks as completed. But do we really need any of that stuff? Not really—and you can never be sure what your users are actually going to do with your site once they get their hands on it. We think our users want to use the site for to-do lists, but maybe they actually want to use it to make “top 10 best fly-fishing spots” lists, for which you don’t need any kind of “mark completed” function. We won’t know until we put it out there. In this chapter we’re going to go through and actually deploy our site to a real, live web server. You might be tempted to skip this chapter—there’s l
Chapter 9. Testing Deployment Using a Staging Site
Chapter 10. Getting to a Production-Ready Deployment In this chapter we’ll make some changes to our site to move to a configuration that’s more production-ready. As we make each change, we’ll use the tests to tell us whether things are still working. What’s wrong with our hacky deployment? Well, we can’t use the Django dev server for production, it’s not designed for “real-life” loads. We’ll use something called Gunicorn instead to run our Django code, and we’ll get Nginx to serve our static files. Our settings.py currently has DEBUG=True, and that’s strongly recommended against for production (you don’t want users staring at debug tracebacks of your code when your site errors for example). We’ll also need to set ALLOWED_HOSTS for security. We want our site to start up automatically whenever the server reboots. For that we’ll write a Systemd config file. Finally, hard-coding port 8000 won’t let us run multiple sites on this server, so we’ll switch to using “unix sockets” to communicate
Chapter 10. Getting to a Production-Ready Deployment
Chapter 11. Automating Deployment with Fabric Automate, automate, automate. Cay Horstman Automating deployment is critical for our staging tests to mean anything. By making sure the deployment procedure is repeatable, we give ourselves assurances that everything will go well when we deploy to production. Fabric is a tool which lets you automate commands that you want to run on servers. “fabric3” is the Python 3 fork: $ pip install fabric3 Tip It’s safe to ignore any errors that say “failed building wheel” during the fabric3 installation, as long as it says “Successfully installed…” at the end. The usual setup is to have a file called fabfile.py, which will contain one or more functions that can later be invoked from a command-line tool called fab, like this: fab function_name:host=SERVER_ADDRESS That will call function_name, passing in a connection to the server at SERVER_ADDRESS. There are lots of other options for specifying usernames and passwords, which you can find out about using
Chapter 11. Automating Deployment with Fabric
Chapter 12. Splitting our tests into multiple files, and a generic wait helper The next feature we might like to implement is a little input validation. But as we start writing new tests, we’ll notice that it’s getting hard to find our way around a single functional_tests.py, and tests.py, so we’ll re-organise them into multiple files—a little refactor of our tests, if you will. We’ll also build a generic explicit wait helper. Start on a Validation FT: Preventing Blank Items As our first few users start using the site, we’ve noticed they sometimes make mistakes that mess up their lists, like accidentally submitting blank list items, or accidentally inputting two identical items to a list. Computers are meant to help stop us from making silly mistakes, so let’s see if we can get our site to help. Here’s the outline of an FT: functional_tests/tests.py (ch11l001) def test_cannot_add_empty_list_items(self): # Edith goes to the home page and accidentally tries to submit # an empty list item
Chapter 12. Splitting our tests into multiple files, and a generic wait helper
Chapter 13. Validation at the Database Layer Over the next few chapters we’ll talk about testing and implementing validation of user inputs. In terms of content, there’s going to be quite a lot of material here that’s more about the specifics of Django, and less discussion of TDD philosophy. That doesn’t mean you won’t be learning anything about testing—there are plenty of little testing tidbits in here, but perhaps it’s more about really getting into the swing of things, the rhythm of TDD and how we get work done. Once we get through these three short chapters, I’ve saved a bit of fun with JavaScript (!) for the end of Part II. Then it’s on to Part III, where I promise we’ll get right back into some of the real nitty-gritty discussions in TDD methodology—unit tests vs integrated tests, mocking and more. Stay tuned! But for now, a little validation. Let’s just remind ourselves where our FT is pointing us: $ python3 manage.py test functional_tests.test_list_item_validation [...] =======
Chapter 13. Validation at the Database Layer
Chapter 14. A Simple Form At the end of the last chapter, we were left with the thought that there was too much duplication of code in the validation handling bits of our views. Django encourages you to use form classes to do the work of validating user input, and choosing what error messages to display. Let’s see how that works. As we go through the chapter, we’ll also spend a bit of time tidying up our unit tests, and making sure each of them only tests one thing at a time. Moving Validation Logic into a Form Tip In Django, a complex view is a code smell. Could some of that logic be pushed out to a form? Or to some custom methods on the model class? Or maybe even to a non-Django module that represents your business logic? Forms have several superpowers in Django: They can process user input and validate it for errors. They can be used in templates to render HTML input elements, and error messages too. And, as we’ll see later, some of them can even save data to the database for you. Y
Chapter 14. A Simple Form
Chapter 15. More Advanced Forms Now let’s look at some more advanced forms usage. We’ve helped our users to avoid blank list items, now let’s help them avoid duplicate items. This chapter goes into more intricate details of Django’s form validation, and you have my official permission to skip it if you already know all about customising Django forms, or if you’re reading this book for the TDD rather than for the Django. If you’re still learning Django, there’s good stuff in here. If you want to skip ahead, that’s OK too. Make sure you take a quick look at the aside on developer stupidity, and the recap on testing views at the end. Another FT for Duplicate Items We add a second test method to ItemValidationTest: functional_tests/test_list_item_validation.py (ch13l001) def test_cannot_add_duplicate_items(self): # Edith goes to the home page and starts a new list self.browser.get(self.live_server_url) self.get_item_input_box().send_keys('Buy wellies') self.get_item_input_box().send_keys(K
Chapter 15. More Advanced Forms
Chapter 16. Dipping Our Toes, Very Tentatively, into JavaScript If the Good Lord had wanted us to enjoy ourselves, he wouldn’t have granted us his precious gift of relentless misery. John Calvin (as portrayed in Calvin and the Chipmunks) Our new validation logic is good, but wouldn’t it be nice if the duplicate item error messages disappeared once the user started fixing the problem? Just like our nice HTML5 validation errors do? For that we’d need a teeny-tiny bit of JavaScript. We are utterly spoiled by programming every day in such a joyful language as Python. JavaScript is our punishment. As a web developer though, there’s no way around it. So let’s dip our toes in, very gingerly. Note I’m going to assume you know the basics of JavaScript syntax. If you haven’t read JavaScript: The Good Parts, go and get yourself a copy right away! It’s not a very long book. Starting with an FT Let’s add a new functional test to the ItemValidationTest class: functional_tests/test_list_item_validati
Chapter 16. Dipping Our Toes, Very Tentatively, into JavaScript
Chapter 17. Deploying Our New Code It’s time to deploy our brilliant new validation code to our live servers. This will be a chance to see our automated deploy scripts in action for the second time. Note At this point I want to say a huge thanks to Andrew Godwin and the whole Django team. Up until Django 1.7, I used to have a whole long section, entirely devoted to migrations. Migrations now “just work”, so I was able to drop it altogether. Thanks for all the great work gang! Staging Deploy We start with the staging server: $ git push $ cd deploy_tools $ fab deploy:host=elspeth@superlists-staging.ottg.eu Disconnecting from superlists-staging.ottg.eu... done. Restart Gunicorn: elspeth@server:$ sudo systemctl restart gunicorn-superlists-staging.ottg.eu And run the tests against staging: $ STAGING_SERVER=superlists-staging.ottg.eu python manage.py test functional_tests OK Live Deploy Assuming all is well, we then run our deploy against live: $ fab deploy:host=elspeth@superlists.ottg.eu el
Chapter 17. Deploying Our New Code
Part III. More Advanced Topics in Testing
Chapter 18. User Authentication, Spiking and De-Spiking Our beautiful lists site has been live for a few days, and our users are starting to come back to us with feedback. “We love the site”, they say, “but we keep losing our lists. Manually remembering URLs is hard. It’d be great if it could remember what lists we’d started”. Remember Henry Ford and faster horses. Whenever you hear a user requirement, it’s important to dig a little deeper and think—what is the real requirement here? And how can I make it involve a cool new technology I’ve been wanting to try out? Clearly the requirement here is that people want to have some kind of user account on the site. So, without further ado, let’s dive into authentication. Naturally we’re not going to mess about with remembering passwords ourselves—besides being so ’90s, secure storage of user passwords is a security nightmare we’d rather leave to someone else. We’ll use something fun called passwordless auth instead. (If you insist on storing
Chapter 18. User Authentication, Spiking and De-Spiking
Chapter 19. Using Mocks to Test External Dependencies or Reduce Duplication In this chapter we’ll start testing the parts of our code that send emails. In the FT, you saw that Django gives us a way of retrieving any emails it sends by using the mail.outbox attribute. But in this chapter, I want to demonstrate a very important testing technique called mocking, so for the purpose of these unit tests, we’ll pretend that this nice Django shortcut doesn’t exist. Note Am I telling you not to use Django’s mail.outbox? No; use it, it’s a neat shortcut. But I want to teach mocks because they’re a useful general-purpose tool for unit testing external dependencies. You may not always be using Django! And even if you are, you may not be sending email—any interaction with a third-party API is a good candidate for testing with mocks. Note This chapter has been substantially rewritten for the new edition, so let me know via obeythetestinggoat@gmail.com if you have any suggestions. I’m not quite happy
Chapter 19. Using Mocks to Test External Dependencies or Reduce Duplication
Chapter 20. Test Fixtures and a Decorator for Explicit Waits Now that we have a functional authentication system, we want to use it to identify users, and be able to show them all the lists they have created. To do that, we’re going to have to write FTs that have a logged-in user. Rather than making each test go through the (time-consuming) login email dance, we want to be able to skip that part. This is about separation of concerns. Functional tests aren’t like unit tests, in that they don’t usually have a single assertion. But, conceptually, they should be testing a single thing. There’s no need for every single FT to test the login/logout mechanisms. If we can figure out a way to “cheat” and skip that part, we’ll spend less time waiting for duplicated test paths. Tip Don’t overdo de-duplication in FTs. One of the benefits of an FT is that it can catch strange and unpredictable interactions between different parts of your application. Note This chapter has only just been rewritten fo
Chapter 20. Test Fixtures and a Decorator for Explicit Waits
Chapter 21. Server-Side Debugging Popping a few layers off the stack of things we’re working on: we have nice wait-for helpers, what were we using them for? Oh yes, waiting to be logged in. And why was that? Ah yes, we had just built a way of pre-authenticating a user. The Proof Is in the Pudding: Using Staging to Catch Final Bugs They’re all very well for running the FTs locally, but how would they work against the staging server? Let’s try and deploy our site. Along the way we’ll catch an unexpected bug (that’s what staging is for!), and then we’ll have to figure out a way of managing the database on the test server. $ cd deploy_tools $ fab deploy --host=elspeth@superlists-staging.ottg.eu [...] And restart Gunicorn… elspeth@server:$ sudo systemctl daemon-reload elspeth@server:$ sudo systemctl restart gunicorn-superlists-staging.ottg.eu Here’s what happens when we run the functional tests: $ STAGING_SERVER=superlists-staging.ottg.eu python manage.py test functional_tests =============
Chapter 21. Server-Side Debugging
Chapter 22. Finishing “My Lists”: Outside-In TDD In this chapter I’d like to talk about a technique called “Outside-In” TDD. It’s pretty much what we’ve been doing all along. Our “double-loop” TDD process, in which we write the functional test first and then the unit tests, is already a manifestation of outside-in—we design the system from the outside, and build up our code in layers. Now I’ll make it explicit, and talk about some of the common issues involved. The Alternative: “Inside Out” The alternative to “Outside In” is to work “Inside Out”, which is the way most people intuitively work before they encounter TDD. After coming up with a design, the natural inclination is sometimes to implement it starting with the innermost, lowest-level components first. For example, when faced with our current problem, providing users with a “My Lists” page of saved lists, the temptation is to start by adding an “owner” attribute to the List model object, reasoning that an attribute like this is
Chapter 22. Finishing “My Lists”: Outside-In TDD
Chapter 23. Test Isolation, and “Listening to Your Tests” In the last chapter, we made the decision to leave a unit test failing in the views layer while we proceeded to write more tests and more code at the models layer to get it to pass. We got away with it because our app was simple, but I should stress that, in a more complex application, this would be a dangerous decision. Proceeding to work on lower levels while you’re not sure that the higher levels are really finished or not is a risky strategy. Note I’m grateful to Gary Bernhardt, who took a look at an early draft of the previous chapter, and encouraged me to get into a longer discussion of test isolation. Ensuring isolation between layers does involve more effort (and more of the dreaded mocks!), but it can also help to drive out improved design, as we’ll see in this chapter. Revisiting Our Decision Point: The Views Layer Depends on Unwritten Models Code Let’s revisit the point we were at half-way through the last chapter, wh
Chapter 23. Test Isolation, and “Listening to Your Tests”
Chapter 24. Continuous Integration (CI) As our site grows, it takes longer and longer to run all of our functional tests. If this continues, the danger is that we’re going to stop bothering. Rather than let that happen, we can automate the running of functional tests by setting up a “Continuous Integration” or CI server. That way, in day-to-day development, we can just run the FT that we’re working on at that time, and rely on the CI server to run all the tests automatically, and let us know if we’ve broken anything accidentally. The unit tests should stay fast enough that we can keep running them every few seconds. The CI server of choice these days is called Jenkins. It’s a bit Java, a bit crashy, a bit ugly, but it’s what everyone uses, and it has a great plugin ecosystem, so let’s get it up and running. Note this chapter has not yet been updated for the new edition. Some of the specific problems discussed with Persona in CI may not be relevant, but the basic lessons of the chapter
Chapter 24. Continuous Integration (CI)
Chapter 25. The Token Social Bit, the Page Pattern, and an Exercise for the Reader Are jokes about how “everything has to be social now” slightly old hat? Everything has to be all A/B tested big data get-more-clicks lists of 10 Things This Inspiring Teacher Said That Will Make You Change Your Mind About Blah Blah now…anyway. Lists, be they Inspirational or otherwise, are often better shared. Let’s allow our users to collaborate on their lists with other users. Along the way we’ll improve our FTs by starting to implement something called the Page Object pattern. Then, rather than showing you explicitly what to do, I’m going to let you write your unit tests and application code by yourself. Don’t worry, you won’t be totally on your own! I’ll give an outline of the steps to take, as well as some hints and tips. An FT with Multiple Users, and addCleanup Let’s get started—we’ll need two users for this FT: functional_tests/test_sharing.py (ch22l001) from selenium import webdriver from .base
Chapter 25. The Token Social Bit, the Page Pattern, and an Exercise for the Reader
Chapter 26. Fast Tests, Slow Tests, and Hot Lava The database is Hot Lava! Casey Kinsey Right up until Chapter 23, almost all of the “unit” tests in the book should perhaps have been called integrated tests, because they either rely on the database, or they use the Django test client, which does too much magic with the middleware layers that sit between requests, responses, and view functions. There is an argument that a true unit test should always be isolated, because it’s meant to test a single unit of software. Some TDD veterans say you should strive to write “pure”, isolated unit tests wherever possible, instead of writing integrated tests. It’s one of the ongoing (occasionally heated) debates in the testing community. Being merely a young whippersnapper myself, I’m only part way towards all the subtleties of the argument. But in this chapter, I’d like to try and talk about why people feel strongly about it, and try and give you some idea of when you can get away with muddling thr
Chapter 26. Fast Tests, Slow Tests, and Hot Lava
Sinon and testing the asynchronous part of Ajax
← Prev
Back
Next →
← Prev
Back
Next →