Log In
Or create an account ->
Imperial Library
Home
About
News
Upload
Forum
Help
Login/SignUp
Index
CONTENTS
A SCHEMA FOR DOMAIN TESTING: AN OVERVIEW ON ONE PAGE
DEDICATION
PREFACE
Testing Techniques
Domain Testing as a Test Technique
Domain Testing is Not the Only Technique
Who This Book is For
Testers
Instructors and Professors
Programmers?
How to Use This Book
Section 1 Provides an Overview of Domain Testing and Key Technical Concepts
Section 2 Works Through the Domain Testing Schema
Section 3 Provides a Collection of Examples
The Examples are Simple, Increasing in Complexity
We Deliberately Repeat Information
We Write in a Spiral
Don’t Just Read the Examples—Work Through Them Yourself
Notice the References
Other Testing Books and Courses
Books for Practitioners
Books for University Students
Why Write an Entire Book on Domain Testing?
ACKNOWLEDGMENTS
SECTION 1: WHAT IS DOMAIN TESTING?
SECTION 1. PART 1: INTRODUCTION TO DOMAIN TESTING
An Example Test Series
The Example Test Series Again, Using the Schema
1. Characterize the Variable
2. Analyze the Variable and Create Tests
3. Generalize to Multidimensional Variables
A More Detailed Look at Domain Testing
Domains
Partitioning
Equivalence Classes
Selecting Representatives
Automating Domain Tests
Documenting and Exploring
SECTION 1. PART 2. SUMMARIES OF KEY TECHNICAL CONCEPTS
Black-Box Software Testing
What’s Special About Black-Box Testing
Sometimes You will be Stuck with Black-Box Testing
BBST®
Goals of Testing
Domain
Input and Output Domains
Specified Domain
Specification
Variable
Input Variable
Result Variable
Output Variable
Configuration Variable
Stored Data
Variable Types
Record
Data Entry Field
User Interface Control
Dimensions
Primary Versus Secondary Dimensions
Notional Variable
Why do You Need Notional Variables?
Multidimensional Variables
Primary and Secondary Dimensions
Explicit Versus Implicit Multidimensionality
Explicit Dimensions
Implicit Dimensions
N-Tuples
Combinatorial Explosion
Risk
Theory of Error
Similarity of Tests
Test Idea
Risk/Equivalence Table
Power of a Test
Error Revealing
Best Representative
Corner Case
Should All Tests be Powerful?
Oracles
Filters
Testing Past the Filter
An Analogy: How Domain Testing is Like Opinion Polling
Stratified Sampling
Partitioning Compared to Stratifying
Linearizable Variables
Coverage
Structural Coverage
It is Easy to Imagine More Coverage than You Can (or Should) Test
There is More to Coverage than Code Coverage
Coverage Criterion
Coverage-Focused Test Techniques
Try to Domain Test Every Variable
SECTION 2: WORKING THROUGH THE SCHEMA’S TASKS
SECTION 2. PART 1: CHARACTERIZE THE VARIABLE
A. IDENTIFY POTENTIALLY INTERESTING VARIABLES
From Example 3: SunTrust VISA
From Example 5: Undergraduate Grades
From Example 27: OpenOffice Impress Grid Options
Record-Keeping
Exercises: Part 1-A
B. IDENTIFY VARIABLE(S) YOU CAN ANALYZE NOW
From Example 4: Page Setup Options
From Example 6: Tax Table
From Example 23: Frequent Flyer Miles
Exercises: Part 1-B
C. DETERMINE THE PRIMARY DIMENSION OF THE VARIABLE OF INTEREST
From Example 3: SunTrust VISA
From Example 5: Undergraduate Grades
From Example 27: OpenOffice Impress Grid Options
Exercises: Part 1-C
D. DETERMINE TYPE AND SCALE OF THE VARIABLE’S PRIMARY DIMENSION
Data Type
Integers
MaxInt
Fixed Point
Floating Point
The Relevance of Floating Point Math to Domain Testing
Different Types of Floating-Point Numbers
Tolerance Levels, Delta and Machine Epsilon
A Few More Details About Floating Point
Char
Enumerated
Boolean (or Binary)
Array
String
Record
Programmer-Specified Type, Struct or Class
List
Scale
Nominal Scale
Ordinal Scale
Interval Scale
Ratio Scale
From Examples 3, 5 and 8
From Example 24: Notepad Open File Dialog
From Example 25: Moodle Assign User Roles
Exercises: Part 1-D
E. DETERMINE WHETHER YOU CAN ORDER THE VARIABLE’S VALUES
Examples of Variables that Can be Ordered
Character-Based Encoding
How Many
How Big
Timing
Speed
Configuration
Variation Between Things that Should be Equivalent
Membership in the Same Group
Lead to the Same Consequence
Exposure to the Same Risk
From Example 26: Install Printer Software
Some Theorists Reserve Domain Testing for Interval and Ratio Variables
From Example 12: Piecewise Boundary
From Example 10: The Circle
From Example 24: Notepad Open File Dialog
Exercises: Part 1-E
F. DETERMINE WHETHER THIS IS AN INPUT VARIABLE OR A RESULT
From Example 2: ATM Withdrawal
From Example 4: Page Setup Options
From Example 14: Result of Calculations
From Example 23: Frequent-Flyer Miles
Exercises: Part 1-F
G. DETERMINE HOW THE PROGRAM USES THIS VARIABLE
From Example 4: Page Setup Options
From Example 6: Tax Table
From Example 23: Frequent-Flyer Miles
Exercises: Part 1-G
H. DETERMINE WHETHER OTHER VARIABLES ARE RELATED TO THIS ONE
How Can you Identify Potentially-Related Variables?
Value of One Passed to the Other
Constraint Relationships
Causal Effect
Joint Effect
Risk in Common
Competition for a Common Resource
Historical Relationship
From Example 6: Tax Table
From Example 10: The Circle
From Example 26: Install Printer Software
Exercises: Part 1-H
SECTION 2. PART 2: ANALYZE THE VARIABLE AND CREATE TESTS
Practical Considerations in Working With Secondary Dimensions
Creating These Lists
Prioritizing Your Tests
Using Standard Lists for Secondary Dimensions
I. PARTITION THE VARIABLE
Analyzing Ordered Dimensions
From Example 4: Page Setup Options
From Example 6: Tax Table
From Example 19: Sum of Squares
From Example 28: OpenOffice Printing Options
Analyzing Non-Ordered Dimensions
From Example 26: Install Printer Software
From Example 24: Notepad Open File Dialog
Exercises: Part 2-1
J. LAY OUT THE ANALYSIS IN THE CLASSICAL TABLE. IDENTIFY BEST REPRESENTATIVES
From Example 3: SunTrust VISA
The Classical Boundary/Equivalence Table
The Risk/Equivalence Table
From Example 6: Tax Table
The Classical Boundary/Equivalence Table
The Risk/Equivalence Table
From Example 8: Create Table (Columns, Rows)
From Example 9: Create Table (Max Cells)
Exercises: Part 2-J
K. CREATE TESTS FOR THE CONSEQUENCES OF THE DATA ENTERED
Calculate the Amount to Pay an Employee
Add Columns to a Spreadsheet
Solving a System of Linear Equations
From Example 15: Mailing Labels
From Example 4: Page Setup Options
Exercises: Part 2-K
L. IDENTIFY SECONDARY DIMENSIONS. ANALYZE THEM IN THE CLASSICAL WAY
Secondary Dimensions
From Example 1: Integer Input
From Example 3: SunTrust VISA
The Classical Boundary/Equivalence Table
From Example 7: Student Names
Exercises: Part 2-L
M. SUMMARIZE YOUR ANALYSIS WITH A RISK/EQUIVALENCE TABLE
Secondary Dimensions
From Example 1: Integer Input
Secondary Dimensions for Integers
The Risk/Equivalence Table (Integers)
From Example 3: SunTrust VISA
Secondary Dimensions for Fixed-Point Variables
The Risk/Equivalence Table (Fixed Point)
From Example 4: Page Setup Options
Secondary Dimensions for Floating-Point Variables
A Risk/Equivalence Table for a Floating-Point Variable
From Example 7: Student Names
Secondary Dimensions for Strings
The Risk/Equivalence Table (Strings)
Exercises: Part 2-M
SECTION 2: PART 3: GENERALIZE TO MULTIDIMENSIONAL VARIABLES
Multidimensional Design Traditions
N-Tuple Notation
The Combinatorial Explosion
Input Variables
Output Variables
Configuration Variables
Internal Variables
Testing 2 Variables Together
Testing m+n+p+q Variables Together
Designing Combination Tests
(A) What Variables Should You Test Together?
Independent Variables
Non-Independent Variables
(B) What Values of those Variables Should You Test?
(C) How Should You Combine Values of Variables into Tests?
(D) Coverage Criteria
N. ANALYZE INDEPENDENT VARIABLES THAT SHOULD BE TESTED TOGETHER
Testing Independent Variables
Random Combinations
All Singles
All N-Tuples
All Pairs, All Triples, etc.
From Example 30: Configuration Testing
Exercises: Part 3-N
O. ANALYZE VARIABLES THAT HOLD RESULTS
Equivalent Results
Data Flows
From Example 14: Result of Calculations
From Example 17: Joan’s Pay
From Example 20: Sum of Squares (MAX)
Analysis of the xi’s.
Analysis of the Result Variable, ss
Extreme Inputs Don’t Necessarily Yield Extreme Outputs
Exercises: Part 3-O
P. ANALYZE NON-INDEPENDENT VARIABLES. DEAL WITH RELATIONSHIPS AND CONSTRAINTS
Ways Variables Constrain Each Other
Example 8: Create Table (Columns, Rows) (Variables that Multiply to a Maximum)
Example 12: Piecewise Boundary (Linear Relationships)
Example 10: The Circle (Nonlinear Relationships)
(a) The Circle {(x1, x2) | x12 + x22 = 100}
(b) Inside the Circle {(x1, x2) | x12 + x22 < 100}
(c) Outside the Circle {(x1, x2) | x12 + x22 > 100}
(d) The Circle & Points Inside it {(x1, x2) | x12 + x22 ≤ 100}
Example 3: SunTrust VISA (Complex Relationships)
Additional Examples
Selection from a List
Sometimes Related, Sometimes Not
Exercises: Part 3-P
SECTION 2. PART 4: PREPARE FOR ADDITIONAL TESTING
Q. IDENTIFY AND LIST UNANALYZED VARIABLES. Gather Information for Later Analysis
From Example 24: Notepad Open File Dialog
From Example 3: SunTrust VISA
Exercises: Part 4-Q
R. IMAGINE AND DOCUMENT RISKS THAT DON’T MAP TO AN OBVIOUS DIMENSION
From Example 3: SunTrust VISA
From Example 25: Moodle Assign User Roles
From Example 30: Configuration Testing
Exercises: Part 4-R
SECTION 3: WORKED EXAMPLES
SECTION 3. PART 1: EXAMPLES THAT ILLUSTRATE THE SCHEMA
Example 1: Integer Input
Test Idea Catalog for Integers
Generic Risk/Equivalence Table for Integers
Example 2: ATM Withdrawal
Example 3: SunTrust VISA
Test Idea Catalog for Fixed-Point Variables
Generic Risk/Equivalence Table for Fixed-Point Variables
Example 4: Page Setup Options
Test Idea Catalog for Floating-Point Variables
Secondary Dimensions in the Classical Table
A Risk/Equivalence Table for a Floating-Point Variable
Example 5: Undergraduate Grades
Example 6: Tax Table
Showing Multiple Valid Equivalence Classes on the Same Table
Example 7: Student Names
Test Idea Catalog for Strings
Generic Risk/Equivalence Table for Strings
Example 8: Create Table (Columns, Rows)
Example 9: Create Table (Max Cells)
Example 10: The Circle
Choosing On Points and Off Points for Testing Nonlinear Relationships
(a) The Circle {(x1, x2) | x12 + x22 = 100}
(b) Inside the Circle {(x1, x2) | x12 + x22 < 100}
(c) Outside the Circle {(x1, x2) | x12 + x22 > 100}
(d) The Circle & Points Inside it {(x1, x2) | x12 + x22 ≤ 100}
(e) The Circle & Points Outside it {(x1, x2) | x12 + x22 ≥ 100}
Example 11: University Admissions
Tests Based on a Black-Box Model may Differ from Tests Based on the Code
Analysis on the Assumption the Implementation is Equivalent to Case A
Risk/Equivalence Table Showing Multiple Related Variables
Analysis on the Assumption the Implementation is Equivalent to Case B
Example 12: Piecewise Boundary
Choosing On Points and Off Points for Testing Linear Relationships
Additional Notes
Example 13: Sum of Two Integers
Example 14: Result Of Calculations
Example 15: Mailing Labels
Constraint Satisfaction Problems
Why Should YOU Care about Geometric Boundary Problems?
What Should You Do with Problems like This?
Example 16: Unemployment Insurance Benefits
Example 17: Joan’s Pay
Example 18: Volume Discounts
Example 19: Sum Of Squares
Example 20: Sum Of Squares (MAX)
Analysis of Example 20
Example 21: Spreadsheet Sort Table
Case 1: One-Column, Homogeneous Orderable Data
Case 2: One-Column, Homogeneous Orderable Data, with Ties
Case 3: One-Column, Non-Homogeneous Data
Cases 4, 5 and 6: Multiple Columns
Cases 7,8 and 9: Sort by Multiple Columns
Example 22: NextDate
Example 23: Frequent-Flyer Miles
Example 24: Notepad Open File Dialog
A Test Idea Catalog for File Name Strings
Grouping File Names by Type
SECTION 3. PART 2: ADVANCED EXAMPLES
Example 25: Moodle Assign User Roles
Overview
Where Can You Apply Domain Analysis?
Example 26: Install Printer Software
Overview
Where Can You Apply Domain Analysis?
Example 27: OpenOffice Impress Grid Options
Overview
Where Can You Apply Domain Analysis?
Example 28: OpenOffice Printing Options
Overview
Where Can You Apply Domain Analysis?
Example 29: Select And Flip Picture
Overview
Where Can You Apply Domain Analysis?
Example 30: Configuration Testing
Overview
Where Can You Apply Domain Analysis?
A Schema for Domain Testing
AFTERWORD: DOMAIN TESTING AS PART OF A TESTING STRATEGY
Using Domain Testing when You are Just Learning the Program
Simple Domain Testing has Limits and Will Run Out of Steam
As You Learn More about the Program, Use Domain Testing in a Deeper Way
What Makes These Tests Domain Tests?
Use Other Test Techniques Too
Combine Domain Testing with Other Techniques to Increase Their Efficiency and Power
APPENDIX: NOTES FOR INSTRUCTORS
Commercial Courses on Software Testing
University Courses on Software Testing
Designing a Course
Activity-Focused Design
Coverage-Focused Design
Backward Design
We Recommend Assessment-Focused Design
Levels of Knowledge
Appropriate Evaluation
Appropriate Instruction
Transfer of Learning and the Evolution of Our Schema
For more information
REFERENCES
DOMAIN TESTING TABLES
The Classical Boundary/Equivalence Table
The Risk/Equivalence Table
← Prev
Back
Next →
← Prev
Back
Next →