Beautiful Testing by Riley, Tim -- Read -- Imperial Library of Trantor

Introduction XMPP 101 Testing XMPP Protocols Unit Testing Simple Request-Response Protocols Unit Testing Multistage Protocols Testing Session Initialization Automated Interoperability Testing Diamond in the Rough: Testing XML Validity Conclusions References

8. Beautiful Large-Scale Test Automation

Before We Start What Is Large-Scale Test Automation?

The Basics of a Test Automation System A Beautiful System

The First Steps

Test Infrastructure Is Critical Test Collateral

Automated Tests and Test Case Management The Automated Test Lab

Deploying the Test Bed Other Considerations

Test Distribution Failure Analysis Reporting Putting It All Together

9. Beautiful Is Better Than Ugly

The Value of Stability Ensuring Correctness

The Buildbot System Refleak Testing Doc Testing Release Testing Dynamic Analysis Static Analysis

Conclusions

10. Testing a Random Number Generator

What Makes Random Number Generators Subtle to Test? Uniform Random Number Generators Nonuniform Random Number Generators A Progression of Tests

Range Tests Mean Test Variance Test Bucket Test Kolmogorov-Smirnov Test

Conclusions

11. Change-Centric Testing

How to Set Up the Document-Driven, Change-Centric Testing Framework? Change-Centric Testing for Complex Code Development Models

Steps to Change-Centric Testing

Understanding the caller–callee dependency and test case to source file mapping Generating the call graph of caller–callee functions in the executable Example 1 Understanding the change impact Understanding the code coverage and gap analysis Example 2 Example 3

What Have We Learned So Far?

Example 4

Scenario 1 Scenario 2 Scenario 3

Conclusions

12. Software in Use

A Connection to My Work From the Inside Adding Different Perspectives Exploratory, Ad-Hoc, and Scripted Testing Multiuser Testing The Science Lab Simulating Real Use Testing in the Regulated World At the End

13. Software Development Is a Creative Process

Agile Development As Performance Practice, Rehearse, Perform Evaluating the Ineffable Two Critical Tools Software Testing Movements The Beauty of Agile Testing QA Is Not Evil Beauty Is the Nature of This Work References

14. Test-Driven Development: Driving New Standards of Beauty

Beauty As Proportion and Balance Agile: A New Proportion and Balance Test-Driven Development Examples Versus Tests Readable Examples Permanent Requirement Artifacts Testable Designs Tool Support

Author Reader Executors Result consumer Report consumer

Team Collaboration Experience the Beauty of TDD References

15. Beautiful Testing As the Cornerstone of Business Success

The Whole-Team Approach

No Story Is “Done” Until It’s Tested Continuous Improvement Delivering Maximum Business Value Focusing on Testability

Automating Tests

Choosing Tools by Team Consensus Incremental Automation Flipping the Pyramid

Driving Development with Tests

Experimenting Planning Driving Coding with Examples Bumps in the Road

Delivering Value

Everyone Owns Business Problems Incremental Testing and Coding Plan Just Enough Ahead

A Success Story Post Script

16. Peeling the Glass Onion at Socialtext

It’s Not Business…It’s Personal Tester Remains On-Stage; Enter Beauty, Stage Right Come Walk with Me, The Best Is Yet to Be Automated Testing Isn’t Into Socialtext

But…What Do You Make? Software Process at Socialtext What We Actually Do Wikitests

A Balanced Breakfast Approach Regression and Process Improvement The Last Pieces of the Puzzle Acknowledgments

17. Beautiful Testing Is Efficient Testing

SLIME

S: Security L: Languages I: requIrements M: Measurement E: Existing

Scripting Discovering Developer Notes

LOUD

Oracles and Test Data Generation Mindmaps Efficiency Achieved

III. Beautiful Tools

18. Seeding Bugs to Find Bugs: Beautiful Mutation Testing

Assessing Test Suite Quality Watching the Watchmen

Efficient Mutation Testing Selecting Mutation Operators

An AspectJ Example Equivalent Mutants Focusing on Impact The Javalanche Framework Odds and Ends Acknowledgments References

19. Reference Testing As Beautiful Testing

Reference Test Structure Reference Test Extensibility

Asynchronous Tests Printing Tests Invalidation Tests

Building Community

20. Clam Anti-Virus: Testing Open Source with Open Tools

The Clam Anti-Virus Project Testing Methods

Black Box Versus White Box Testing Static Analysis

GCC Clang Static Analyzer Splint Be patient

Memory Checkers

Valgrind Electric Fence and DUMA Mudflap Limitations

Unit Testing Test Scripts Fuzz Testing Collection of Problematic Files Testing the Environment Buildbot Compatibility Testing Performance Testing Testing for False Positive and False Negative Alerts

False positives False negatives

Usability and User Acceptance Testing

Opinion polling Release candidates

Summary Credits

21. Web Application Testing with Windmill

Introduction Overview Writing Tests

Running Tests Debugging Test Firebug Lite LookupNode

The Project Comparison Conclusions References

22. Testing One Million Web Pages

In the Beginning… The Tools Merge and Evolve The Nitty-Gritty Summary Acknowledgments

23. Testing Network Services in Multimachine Scenarios

The Need for an Advanced Testing Tool in eBox Development of ANSTE to Improve the eBox QA Process How eBox Uses ANSTE

Sample ANSTE Tests for eBox

How Other Projects Can Benefit from ANSTE

A. Contributors Index Colophon SPECIAL OFFER: Upgrade this ebook with O’Reilly

← Prev
Back
Next →

← Prev
Back
Next →