Exploratory Software Testing

Description

James Whittaker has spent his career in software testing and has left his mark on many aspects of the discipline. He was a pioneer in the field of model-based testing, where his Ph.D. dissertation from the University of Tennessee stands as a standard reference on the subject. His work in fault injection produced the highly acclaimed runtime fault injection tool Holodeck, and he was an early thought leader in security and penetration testing. He is also well regarded as a teacher and presenter, and has won numerous best paper and best presentation awards at international conferences. While a professor at Florida Tech, his teaching of software testing attracted dozens of sponsors from both industry and world governments, and his students were highly sought after for their depth of technical knowledge in testing.

 

Dr. Whittaker is the author of How to Break Software and its series follow-ups How to Break Software Security (with Hugh Thompson) and How to Break Web Software (with Mike Andrews). After ten years as a professor, he joined Microsoft in 2006, and left in 2009 to join Google as the Director of Test Engineering for the Kirkland and Seattle offices. He lives in Woodinville, Washington, and is working toward a day when software just works.

 

Breakthrough software testing techniques to improve quality and customer satisfaction, from top expert James Whittaker

 

Systematically reveals the real causes of today’s software bugs – and powerful new “exploratory” techniques for finding and correcting them.

Beyond conventional “ad hoc” or automated testing: how to consistently discover and eliminate the “show stopper” bugs that derail projects

By one of the testing field’s most respected and highest-profile personalities

Foreword by Alan Page     xv

Preface     xvii

 

Chapter 1    The Case for Software Quality     1

The Magic of Software     1

The Failure of Software     4

Conclusion     9

Exercises     9

 

Chapter 2    The Case for Manual Testing     11

The Origin of Software Bugs     11

Preventing and Detecting Bugs     12

Manual Testing     14

Conclusion     19

Exercises     20

 

Chapter 3    Exploratory Testing in the Small     21

So You Want to Test Software?     21

Testing Is About Varying Things     23

User Input     23

    What You Need to Know About User Input     24

    How to Test User Input     25

State     32

    What You Need to Know About Software State     32

    How to Test Software State     33

Code Paths     35

User Data     36

Environment     36

Conclusion     37

Exercises     38

 

Chapter 4    Exploratory Testing in the Large     39

Exploring Software     39

The Tourist Metaphor     41

“Touring” Tests     43

    Tours of the Business District     45

    Tours Through the Historical District     51

    Tours Through the Entertainment District     52

    Tours Through the Tourist District     55

    Tours Through the Hotel District     58

    Tours Through the Seedy District     60

Putting the Tours to Use     62

Conclusion     63

Exercises     64

 

Chapter 5    Hybrid Exploratory Testing Techniques     65

Scenarios and Exploration     65

Applying Scenario-Based Exploratory Testing     67

Introducing Variation Through Scenario Op

How to Find and Fix the Killer Software Bugs that Evade Conventional Testing

 

In Exploratory Software Testing, renowned software testing expert James Whittaker reveals the real causes of today’s most serious, well-hidden software bugs–and introduces powerful new “exploratory” techniques for finding and correcting them.

 

Drawing on nearly two decades of experience working at the cutting edge of testing with Google, Microsoft, and other top software organizations, Whittaker introduces innovative new processes for manual testing that are repeatable, prescriptive, teachable, and extremely effective. Whittaker defines both in-the-small techniques for individual testers and in-the-large techniques to supercharge test teams. He also introduces a hybrid strategy for injecting exploratory concepts into traditional scripted testing. You’ll learn when to use each, and how to use them all successfully.

 

Concise, entertaining, and actionable, this book introduces robust techniques that have been used extensively by real testers on shipping software, illuminating their actual experiences with these techniques, and the results they’ve achieved. Writing for testers, QA specialists, developers, program managers, and architects alike, Whittaker answers crucial questions such as:

 

•  Why do some bugs remain invisible to automated testing–and how can I uncover them?

•  What techniques will help me consistently discover and eliminate “show stopper” bugs?

•  How do I make manual testing more effective–and less boring and unpleasant?

•  What’s the most effective high-level test strategy for each project?

•  Which inputs should I test when I can’t test them all?

•  Which test cases will provide the best feature coverage?

•  How can I get better results by combining exploratory testing with traditional script or scenario-based testing?

•  How do I reflect feedback from the development process, such as code changes?

 

How to Find and Fix the Killer Software Bugs that Evade Conventional Testing

 

In Exploratory Software Testing, renowned software testing expert James Whittaker reveals the real causes of today’s most serious, well-hidden software bugs–and introduces powerful new “exploratory” techniques for finding and correcting them.

 

Drawing on nearly two decades of experience working at the cutting edge of testing with Google, Microsoft, and other top software organizations, Whittaker introduces innovative new processes for manual testing that are repeatable, prescriptive, teachable, and extremely effective. Whittaker defines both in-the-small techniques for individual testers and in-the-large techniques to supercharge test teams. He also introduces a hybrid strategy for injecting exploratory concepts into traditional scripted testing. You’ll learn when to use each, and how to use them all successfully.

 

Concise, entertaining, and actionable, this book introduces robust techniques that have been used extensively by real testers on shipping software, illuminating their actual experiences with these techniques, and the results they’ve achieved. Writing for testers, QA specialists, developers, program managers, and architects alike, Whittaker answers crucial questions such as:

 

•  Why do some bugs remain invisible to automated testing–and how can I uncover them?

•  What techniques will help me consistently discover and eliminate “show stopper” bugs?

•  How do I make manual testing more effective–and less boring and unpleasant?

•  What’s the most effective high-level test strategy for each project?

•  Which inputs should I test when I can’t test them all?

•  Which test cases will provide the best feature coverage?

•  How can I get better results by combining exploratory testing with traditional script or scenario-based testing?

•  How do I reflect feedback from the development process, such as code changes?