Effectively Employing Exploratory Testing
Exploratory testing (ET), or testing when learning, design and execution are concurrent, is not the random and ineffective approach as once believed. ET is still a new concept and hasn’t been wholly embraced by the industry, but surprisingly it is one of the most widely used approaches in software testing. The problem is that very few people take it seriously and its real value is often misunderstood.
Exploratory testing has a highly situational structure as opposed to a planned set of requirements that is often favored. There are lots of myths (well, I will refer to them as myths) out there justifying that scripted testing (ST) and only scripted testing is the ideal and that exploratory testing is becoming obsolete. I am a firm believer that ET integrated with your existing approaches fits in nicely with the other tests testers perform. When we are faced with software with unknown requirements or software in an unstable condition, exploratory testing is a cost-effective approach and may uncover those vulnerabilities that traditional, pre-planned tests tend to miss.
As companies begin to seek more agile methods of developing software, the acceptance of exploratory testing will begin to increase. Until the acceptance rate of the value is better recognized questions such as when it should be applied and how to choose between ET and ST still need to be warranted. Unlike a scripted testing approach, exploratory testing follows an intuitive plan of observing, evaluating, and teaching. Testers usually have a general test plan in their mind as well as a fairly good understanding of the application and the business domain. ET emphasives flexibility and creativity as testers adapt to interacting test tasks.
It is important to realize that exploratory testing is not against the idea of scripting; in fact the most effective test strategies implement a hybrid ET/ST approach. There are many factors to consider when determining which approach to implement. Factors such as how well the tester understands the operation and complexity of a system, the skill sets of the team, and the degree of acceptable risk are important to assess when deciding which approach will be most conducive.
With the appropriate blend of both ET and ST the chances of successfully spotting most of the risks will be improved. The power of how effective employing exploratory testing will be is really based on the knowledge, experience and intuition of the individual tester and the variability among the test team as a whole.
May 13th, 2006 at 12:26 pm
This is a fair description of ET. A problem with describing ET, and we see the problem here, is that it is so tempting to speak about intuition when describing it– but intuition is an empty idea. Intuition is the word we use when we want to say “ideas popping into my head mysteriously, as if by magic.”
There are many subtle aspects of the ET approach– just as there are many subtle aspects about playing chess or solving real life math problems (not those canned problems you get in school). However, no real life mathematician or chess expert would say that all there is to it is “intuition”. There is a great deal that is trainable and tangible about ET. And I generally don’t find it necessary to tell testers to “be creative” or “use intuition”. Instead, I offer them specific heuristic techniques to generate ideas and make judgments. I train them to reason in a variety of ways. I make them practice, and practice makes them better according to a predictable pattern of mental development. To become a great tester, just as to become a great chess player, means learning how to think in new ways.
Exploratory testing deserves to be taken seriously, not because it is a mysterious process that somehow works well, but because it is a teachable and obsevable process that works well.
– james
May 25th, 2006 at 3:16 pm
[…] My last two blog entries, Effectively Exploring Exploratory Testing and Reckless Test Automation covered two Agile techniques and the controversies surrounding their acceptance in the industry. I am continuing my discussion of practices championed by Agile approaches by further exploring Pair Programming (PP), the extreme of Extreme Programming (XP). […]