Integrated test design and automation : using the test frame method

Foreword ix

Acknowledgements xi

1 Introduction 1

1.1 What is testing? 1

1.1.1 Why do we test? 2

1.1.2 Facts of life in testing 2

1.1.3 Testing and development 3

1.1.4 Quality of testing 5

1.2 An introduction to TestFrame 6

1.3 The TestFrame model 7

1.3.1 Reusable test products 8

1.3.2 Fitting 8

1.3.3 Structuring 9

1.3.4 Tooling 9

1.4 Phasing with TestFrame 10

1.4.1 TestFrame products 11

1.5 Preparation 12

1.5.1 Risk analysis 13

1.5.2 Test strategy 13

1.5.3 Test plan 14

1.5.4 Test planning 14

1.6 Analysis 14

1.6.1 Clusters 14

1.6.2 Test conditions 15

1.6.3 Action words 15

1.7 Navigation 16

1.7.1 Test tools 16

1.7.2 Navigation script 16

1.7.3 Separating analysis and navigation 16

1.8 Execution 17

1.8.1 Test report 18

1.8.2 Error management 18

1.9 Summary 20

2 Preparation 23

2.1 Introduction 23

2.2 Preliminary study 24

2.2.1 General 25

2.2.2 Organization 26

2.2.3 Test effort 27

2.2.4 Physical test environment 29

2.2.5 Documentation / experts 31

2.3 Risk analysis 32

2.4 Test strategy 51

2.4.1 Organization structure 52

2.4.2 Quality attributes and their relative importance 53

2.4.3 Test types 56

2.4.4 Cluster matrix 57

2.4.5 Cluster cards 58

2.4.6 Tuning 60

2.5 The test plan 60

2.5.1 Test plan versus plan of approach 61

2.5.2 Test plan structure 62

2.5.2.1 Description of the assignment 62

2.5.2.2 Defining the scope of the assignment 63

2.5.2.3 Filling the TestFrame test model 64

2.5.2.4 Specifying the time schedule 65

2.5.2.5 Defining quality assurance 65

2.5.2.6 Describing the test organization 65

2.5.2.7 Defining standards and procedures 66

2.5.2.8 Miscellaneous 67

2.6 Structuring the test environment 67

2.6.1 Step 1: Determine the effect of the test s scope on the test environment 68

2.6.2 Step 2: Draw up an inventory of the future production environment 69

2.6.3 Step 3: Draw up an overview of the required test environment 69

2.6.4 Step 4: Describe the differences between the test environment and the
(future) production environment 71

2.6.5 Step 5: Describe the responsibilities for structuring the test environment 71

2.6.6 Step 6: Describe the responsibilities in maintaining the test environment
during the project 72

2.6.7 Step 7: Structure the test environment and maintain it during the project 72

2.6.8 Step 8: Describe the responsibilities of test environment maintenance
after the test project 73

2.6.9 Step 9: Maintaining the test environment after completion of the test
project 73

2.6.10 Work area 73

2.7 Project file 74

2.7.1 Planning 75

2.7.2 Monitoring and control 76

2.7.3 Standards and procedures within the project 80

2.8 Summary 80

3 Analysis 83

3.1 Introduction 83

3.2 Test set structure 84

3.2.1 Initial database 85

3.2.2 Division into clusters 85

3.2.3 General cluster documentation 86

3.3 Scope 86

3.3.1 Determining the basic information 87

3.3.2 Determining the test s depth of testing 87

3.3.3 Example of defining the scope for a test object 88

3.4 Clusters 95

3.4.1 Division into clusters 96

3.4.2 Recording clusters 97

3.4.3 Cluster overview 97

3.5 Test conditions 98

3.5.1 Creating the right test conditions 99

3.5.2 Recording test conditions 100

3.5.3 Example of test conditions for a test object 101

3.5.4 Another way of drawing up test conditions 104

3.6 Test cases 106

3.6.1 Naming action words 106

3.6.2 Naming arguments for action words 109

3.6.3 Documenting the action words 111

3.6.4 Example of action word documentation for a test object 111

3.6.5 Recording test cases 114

3.6.6 Example of test cases for a test object 115

3.6.7 Drawing up test cases 116

3.6.8 Documentation lines 122

3.6.9 Making optimal use of spreadsheet functionality 122

3.6.10 Argument commands 123

3.7 Test conditions and test techniques 126

3.7.1 Decision table technique 127

3.7.2 The decision table technique s working method 128

3.7.3 Example of a decision table for a t