[Test-Driven Development with Python 01] • Test-Driven Development With Python · 2nd Edition by Percival, Harry J.W. -- Read -- Imperial Library of Trantor

Index

Preface

Why I Wrote a Book About Test-Driven Development Aims of This Book Outline Conventions Used in This Book Submitting Errata Using Code Examples O’Reilly Safari Contacting O’Reilly

Prerequisites and Assumptions

Python 3 and Programming How HTML Works Django JavaScript Required Software Installations

Git’s Default Editor, and Other Basic Git Config Installing Firefox and Geckodriver

Setting Up Your Virtualenv

Activating and Deactivating the Virtualenv Installing Django and Selenium Some Error Messages You’re Likely to See When You Inevitably Fail to Activate Your Virtualenv

Companion Video Acknowledgments

Additional Thanks for the Second Edition

I. The Basics of TDD and Django 1. Getting Django Set Up Using a Functional Test

Obey the Testing Goat! Do Nothing Until You Have a Test Getting Django Up and Running Starting a Git Repository

2. Extending Our Functional Test Using the unittest Module

Using a Functional Test to Scope Out a Minimum Viable App The Python Standard Library’s unittest Module Commit

3. Testing a Simple Home Page with Unit Tests

Our First Django App, and Our First Unit Test Unit Tests, and How They Differ from Functional Tests Unit Testing in Django Django’s MVC, URLs, and View Functions At Last! We Actually Write Some Application Code! urls.py Unit Testing a View

The Unit-Test/Code Cycle

4. What Are We Doing with All These Tests? (And, Refactoring)

Programming Is Like Pulling a Bucket of Water Up from a Well Using Selenium to Test User Interactions The “Don’t Test Constants” Rule, and Templates to the Rescue

Refactoring to Use a Template The Django Test Client

On Refactoring A Little More of Our Front Page Recap: The TDD Process

5. Saving User Input: Testing the Database

Wiring Up Our Form to Send a POST Request Processing a POST Request on the Server Passing Python Variables to Be Rendered in the Template Three Strikes and Refactor The Django ORM and Our First Model

Our First Database Migration The Test Gets Surprisingly Far A New Field Means a New Migration

Saving the POST to the Database Redirect After a POST

Better Unit Testing Practice: Each Test Should Test One Thing

Rendering Items in the Template Creating Our Production Database with migrate Recap

6. Improving Functional Tests: Ensuring Isolation and Removing Voodoo Sleeps

Ensuring Test Isolation in Functional Tests

Running Just the Unit Tests

Aside: Upgrading Selenium and Geckodriver On Implicit and Explicit Waits, and Voodoo time.sleeps

7. Working Incrementally

Small Design When Necessary

Not Big Design Up Front YAGNI! REST (ish)

Implementing the New Design Incrementally Using TDD Ensuring We Have a Regression Test Iterating Towards the New Design Taking a First, Self-Contained Step: One New URL

A New URL A New View Function

Green? Refactor Another Small Step: A Separate Template for Viewing Lists A Third Small Step: A URL for Adding List Items

A Test Class for New List Creation A URL and View for New List Creation Removing Now-Redundant Code and Tests A Regression! Pointing Our Forms at the New URL

Biting the Bullet: Adjusting Our Models

A Foreign Key Relationship Adjusting the Rest of the World to Our New Models

Each List Should Have Its Own URL

Capturing Parameters from URLs Adjusting new_list to the New World

The Functional Tests Detect Another Regression One More View to Handle Adding Items to an Existing List

Beware of Greedy Regular Expressions! The Last New URL The Last New View Testing the Response Context Objects Directly

A Final Refactor Using URL includes

II. Web Development Sine Qua Nons 8. Prettification: Layout and Styling, and What to Test About It

What to Functionally Test About Layout and Style Prettification: Using a CSS Framework Django Template Inheritance Integrating Bootstrap

Rows and Columns

Static Files in Django

Switching to StaticLiveServerTestCase

Using Bootstrap Components to Improve the Look of the Site

Jumbotron! Large Inputs Table Styling

Using Our Own CSS What We Glossed Over: collectstatic and Other Static Directories A Few Things That Didn’t Make It

9. Testing Deployment Using a Staging Site

TDD and the Danger Areas of Deployment As Always, Start with a Test Getting a Domain Name Manually Provisioning a Server to Host Our Site

Choosing Where to Host Our Site Spinning Up a Server User Accounts, SSH, and Privileges Installing Nginx Installing Python 3.6 Configuring Domains for Staging and Live Using the FT to Confirm the Domain Works and Nginx Is Running

Deploying Our Code Manually

Adjusting the Database Location Creating a Virtualenv Manually, and Using requirements.txt Simple Nginx Configuration Creating the Database with migrate

Success! Our Hack Deployment Works

10. Getting to a Production-Ready Deployment

Switching to Gunicorn Getting Nginx to Serve Static Files Switching to Using Unix Sockets Switching DEBUG to False and Setting ALLOWED_HOSTS Using Systemd to Make Sure Gunicorn Starts on Boot

Saving Our Changes: Adding Gunicorn to Our requirements.txt

Thinking About Automating

Saving Templates for Our Provisioning Config Files

Saving Our Progress

11. Automating Deployment with Fabric

Breakdown of a Fabric Script for Our Deployment

Creating the Directory Structure Pulling Down Our Source Code with Git Updating settings.py Updating the Virtualenv Migrating the Database If Necessary

Trying It Out

Deploying to Live Nginx and Gunicorn Config Using sed

Git Tag the Release Further Reading

12. Splitting Our Tests into Multiple Files, and a Generic Wait Helper

Start on a Validation FT: Preventing Blank Items

Skipping a Test Splitting Functional Tests Out into Many Files Running a Single Test File

A New Functional Test Tool: A Generic Explicit Wait Helper Finishing Off the FT Refactoring Unit Tests into Several Files

13. Validation at the Database Layer

Model-Layer Validation

The self.assertRaises Context Manager A Django Quirk: Model Save Doesn’t Run Validation

Surfacing Model Validation Errors in the View

Checking That Invalid Input Isn’t Saved to the Database

Django Pattern: Processing POST Requests in the Same View as Renders the Form

Refactor: Transferring the new_item Functionality into view_list Enforcing Model Validation in view_list

Refactor: Removing Hardcoded URLs

The {% url %} Template Tag Using get_absolute_url for Redirects

14. A Simple Form

Moving Validation Logic into a Form

Exploring the Forms API with a Unit Test Switching to a Django ModelForm Testing and Customising Form Validation

Using the Form in Our Views

Using the Form in a View with a GET Request A Big Find and Replace

Using the Form in a View That Takes POST Requests

Adapting the Unit Tests for the new_list View Using the Form in the View Using the Form to Display Errors in the Template

Using the Form in the Other View

A Helper Method for Several Short Tests An Unexpected Benefit: Free Client-Side Validation from HTML5

A Pat on the Back But Have We Wasted a Lot of Time? Using the Form’s Own Save Method

15. More Advanced Forms

Another FT for Duplicate Items

Preventing Duplicates at the Model Layer A Little Digression on Queryset Ordering and String Representations Rewriting the Old Model Test Some Integrity Errors Do Show Up on Save

Experimenting with Duplicate Item Validation at the Views Layer A More Complex Form to Handle Uniqueness Validation Using the Existing List Item Form in the List View Wrapping Up: What We’ve Learned About Testing Django

16. Dipping Our Toes, Very Tentatively, into JavaScript

Starting with an FT Setting Up a Basic JavaScript Test Runner Using jQuery and the Fixtures Div Building a JavaScript Unit Test for Our Desired Functionality Fixtures, Execution Order, and Global State: Key Challenges of JS Testing

console.log for Debug Printing Using an Initialize Function for More Control Over Execution Time

Columbo Says: Onload Boilerplate and Namespacing JavaScript Testing in the TDD Cycle A Few Things That Didn’t Make It

17. Deploying Our New Code

Staging Deploy Live Deploy What to Do If You See a Database Error Wrap-Up: git tag the New Release

III. More Advanced Topics in Testing 18. User Authentication, Spiking, and De-Spiking

Passwordless Auth Exploratory Coding, aka “Spiking”

Starting a Branch for the Spike Frontend Log in UI Sending Emails from Django Using Environment Variables to Avoid Secrets in Source Code Storing Tokens in the Database Custom Authentication Models Finishing the Custom Django Auth

De-spiking

Reverting Our Spiked Code

A Minimal Custom User Model

Tests as Documentation

A Token Model to Link Emails with a Unique ID

19. Using Mocks to Test External Dependencies or Reduce Duplication

Before We Start: Getting the Basic Plumbing In Mocking Manually, aka Monkeypatching The Python Mock Library

Using unittest.patch Getting the FT a Little Further Along Testing the Django Messages Framework Adding Messages to Our HTML Starting on the Login URL Checking That We Send the User a Link with a Token

De-spiking Our Custom Authentication Backend

1 if = 1 More Test The get_user Method Using Our Auth Backend in the Login View

An Alternative Reason to Use Mocks: Reducing Duplication

Using mock.return_value Patching at the Class Level

The Moment of Truth: Will the FT Pass? It Works in Theory! Does It Work in Practice? Finishing Off Our FT, Testing Logout

20. Test Fixtures and a Decorator for Explicit Waits

Skipping the Login Process by Pre-creating a Session

Checking That It Works

Our Final Explicit Wait Helper: A Wait Decorator

21. Server-Side Debugging

The Proof Is in the Pudding: Using Staging to Catch Final Bugs

Setting Up Logging

Setting Secret Environment Variables on the Server Adapting Our FT to Be Able to Test Real Emails via POP3 Managing the Test Database on Staging

A Django Management Command to Create Sessions Getting the FT to Run the Management Command on the Server Using Fabric Directly from Python Recap: Creating Sessions Locally Versus Staging

Locally: Against staging:

Baking In Our Logging Code Wrap-Up

22. Finishing “My Lists”: Outside-In TDD

The Alternative: “Inside-Out” Why Prefer “Outside-In”? The FT for “My Lists” The Outside Layer: Presentation and Templates Moving Down One Layer to View Functions (the Controller) Another Pass, Outside-In

A Quick Restructure of the Template Inheritance Hierarchy Designing Our API Using the Template Moving Down to the Next Layer: What the View Passes to the Template

The Next “Requirement” from the Views Layer: New Lists Should Record Owner

A Decision Point: Whether to Proceed to the Next Layer with a Failing Test

Moving Down to the Model Layer

Final Step: Feeding Through the .name API from the Template

23. Test Isolation, and “Listening to Your Tests”

Revisiting Our Decision Point: The Views Layer Depends on Unwritten Models Code A First Attempt at Using Mocks for Isolation

Using Mock side_effects to Check the Sequence of Events

Listen to Your Tests: Ugly Tests Signal a Need to Refactor Rewriting Our Tests for the View to Be Fully Isolated

Keep the Old Integrated Test Suite Around as a Sanity Check A New Test Suite with Full Isolation Thinking in Terms of Collaborators

Moving Down to the Forms Layer

Keep Listening to Your Tests: Removing ORM Code from Our Application

Finally, Moving Down to the Models Layer

Back to Views

The Moment of Truth (and the Risks of Mocking) Thinking of Interactions Between Layers as “Contracts”

Identifying Implicit Contracts Fixing the Oversight

One More Test Tidy Up: What to Keep from Our Integrated Test Suite

Removing Redundant Code at the Forms Layer Removing the Old Implementation of the View Removing Redundant Code at the Forms Layer

Conclusions: When to Write Isolated Versus Integrated Tests

Let Complexity Be Your Guide Should You Do Both? Onwards!

24. Continuous Integration (CI)

Installing Jenkins Configuring Jenkins

Initial Unlock Suggested Plugins for Now Configuring the Admin User Adding Plugins Telling Jenkins Where to Find Python 3 and Xvfb Finishing Off with HTTPS

Setting Up Our Project First Build! Setting Up a Virtual Display So the FTs Can Run Headless Taking Screenshots If in Doubt, Try Bumping the Timeout! Running Our QUnit JavaScript Tests in Jenkins with PhantomJS

Installing node Adding the Build Steps to Jenkins

More Things to Do with a CI Server

25. The Token Social Bit, the Page Pattern, and an Exercise for the Reader

An FT with Multiple Users, and addCleanup The Page Pattern Extend the FT to a Second User, and the “My Lists” Page An Exercise for the Reader

26. Fast Tests, Slow Tests, and Hot Lava

Thesis: Unit Tests Are Superfast and Good Besides That

Faster Tests Mean Faster Development The Holy Flow State Slow Tests Don’t Get Run as Often, Which Causes Bad Code We’re Fine Now, but Integrated Tests Get Slower Over Time Don’t Take It from Me And Unit Tests Drive Good Design

The Problems with “Pure” Unit Tests

Isolated Tests Can Be Harder to Read and Write Isolated Tests Don’t Automatically Test Integration Unit Tests Seldom Catch Unexpected Bugs Mocky Tests Can Become Closely Tied to Implementation But All These Problems Can Be Overcome

Synthesis: What Do We Want from Our Tests, Anyway?

Correctness Clean, Maintainable Code Productive Workflow Evaluate Your Tests Against the Benefits You Want from Them

Architectural Solutions

Ports and Adapters/Hexagonal/Clean Architecture Functional Core, Imperative Shell

Conclusion