Aggressive and Passive Testing


I’ve been thinking about how I *bucket* testing. Here is what I mean. I see testing as aggressive and passive.

 Aggressive testing, to me, is the art of asking the product questions, to think outside of the box (and the text-book) and to try different ways to test the application looking for interesting information (whether they be bugs, issues or curios). I prefer to be an aggressive tester. My mindset is to look for ways in which a product could fail. I see this as our value add as testers. When we find a bug, the bug is reported and resolved in *someway* thereby helping increase the quality of the product.

I believe that, while there is an element of passive testing (and what i mean here is checking), a tester is more beneficial to a project IF they are being aggressive and proactive and looking for potential failures or issues.

Detailed testing scripting can be *aggressive* in some ways but I’ve found that by having a pre-determined course of action, I am more likely to allow confirmation bias to influence the way I work. By exploring (and I mean having some structure – whether it is by session based test management or using high level test conditions/risks/ideas), I have found that I am more likely to be more aggressive in nature and pursue lurking bugs in the code as I have not been constrained by following detailed test steps.

I believe that to be an effective tester ultimately means that we are aware of the context of the project, application and environment, we are in pursuit of information (bugs, issues or curios – curio *term* taken from a discussion on twitter from James Bach and Michael Bolton) and we are feeding this information back into the project thereby helping management make more of an informed go/no go decision.

In one project I worked on there was a significant element of what i call Passive testing which came in the form of *running* a regression suite. This involved executing a test script which quickly devolved in an almost meaningless tick off and check exercise (check the test step – is it correct? – If so, check it off, if not do a superficial investigation (though I can state that no regression bugs were *found* as a result of this *testing*!)

This is bad passive testing which unfortunately is common in my part of the testing world. How many times have I seen disengaged *testers* running scripts that supposedly are meant to add value to the project – to my sceptical mind, all they add is paper.

Now, not all passive testing is as I’ve described. Even when we are aggressive in our approach there may still be elements of passive testing (checking against rules, contracts, laws, configurations, environments or anything that may require checks that help support what we do as engaged testers).

Aggressive and passive testing are NOT mutually exclusive – they are interdependent and intertwined – the issue I have is when the passive (non-engaged) testing is more prevalent than the engaged (by being engaged I mean brain switched on testing).

Unfortunately this is common in bigger organisations where I live. Fortunately, there are pockets of very engaged, context driven testers around that add much more value than the stock standard factory schoolers. I am thankful for that for it means that I’m not a lone voice in this part of the world!

New terminology?

Am i introducing a new set of terminology? No. Will others view testing as aggressive or passive? Maybe not. What I am doing is highlighting how I see testing. There is nothing wrong with that. I am not constrained by a glossary (though they could be useful) rather I am attempting to demonstrate what i mean by testing and how I view the craft that I work in.

Advertisements

Author: bjosman

Principal Consultant at OsmanIT brian.osman@osmanit.com

8 thoughts on “Aggressive and Passive Testing”

  1. Interesting! Is running a test automation test script deemed as passive testing then? If it is then your argument suggests that it conforms to the meaningless tick off and check exercise. Clearly some passive testing, as you mention, is worth while. Just that I think quite a lot of it is worthwhile especially when it comes to regression testing. Like most things in life it’s probably more about getting the balance right between the two types of testing you describe rather than trying to eliminate one of them all together. And I think this should reflect in the type of people you have in a test team. Having some testers that are passive testers and some that are aggressive is likely to be a good thing.
    I like the concept of categorising testing in terms of passive and aggressive though.

    1. @Bill Echlin – “Interesting! Is running a test automation test script deemed as passive testing then?” – Yes and No – the automated test iteslf is passive as it is executing exactly as it is *told to do* – However, the aggressive part is the preparation of the automated test and the analysis after. The tool runs the check – the *how* (passive) but the tester determines the *what* and the *why* (aggressive). The difference in this case between automated and manual/sapient testing is in the feedback loop.

      The shorter/tighter the loop, the more engaged hopefully we are and therefore, more aggressive.

      Thanks for your comment.

  2. Interesting point, Brian. Totally agree with what you’re saying re the two types of testing. There should be a BALANCED combination of the two (passive and agressive), rather than one approach dominating the other.

    To me, passive testing should be the starting point, eg, running a ‘regression test’ bto determine whether the application is stable enough for you to commence aggressive testing.

    The most useful asset and skill a tester can offer, is their THINKING. The challenging and exercising of the ‘what if’ scenarios. Your “aggressive testing” seems to sum this up perfectly.

    If you think of James Whittaker’s exploratory testing tours, passive testing may equate to ‘the money tour’ where you are simply following the ‘tour brochure’ and confirming the things they report are true. As ‘tourists’, we are getting off the tour bus, taking photos to prove we’ve been there, and then hopping back on to go to the next destination.

    Aggressive testing may be more aligned with the “the intellectual tour” or the “FedEx tour”, where we are actively engaged asking the ‘what if’ questions – THINKING about the app, how it may/may not be used in reality.

    As tourists in this scenario, we are the FIRST off the tour bus, we conduct a quick look at the obvious points about the destination, and then we’re off actively investigating the interesting stuff, getting off the beaten track, doing the things that if the bus driver saw you do them, you might get growled at! Digging up the dirt, so to speak.

    That’s where I think as testers, we add value. This is what makes us professionals

    A thought occurred to me about the timing of ‘passive testing’. Passive testing IS important eg, checking legal compliance etc. But wouldn’t it be better to have these sorts of things included in an automated test suite? And even better, if this test suite was run in by the developers before the code was deployed into test? Why don’t we as testers work WITH the developers to identify the ‘standard checks’ we would be doing in a passive testing approach, and have these included in unit tests in the dev environment? These tests passing could be a pre-req for the code being deployed into the test environment.

    We did exactly this on a large project I’ve recently been involved with. Our “standard checks” were incorporated into automated regression test suites which were able to be run across all environments. The suites were run in dev before any deployment into test. The deploy only happened if the suites passed. We then ran them again in test, to ensure no environmental issues existed before we began our ‘aggressive testing’. This approach was also applied to deployments into the UAT and Prod environments.

    As this was a long-term Agile project (4 years!), with fortnightly sprints, you can well imagine the number of times these suites were run, and the value they added. They paid for themselves in no time flat.

    The existance of these suites meant that our focus as testers could be the “aggressive testing / actively engaged thinking” focus which is where we add REAL value. 🙂 Digging up the dirt. 🙂

    Interestingly, these suites were built by the TEST team with support from the devs. And to take Bill Echlin’s point, we had testers on the team who excelled at building these suites, and other testers who excelled at the ‘aggressive testing’ that the existance of these suites enabled. A balance of not only people and skills, but focuses and thinking as well. 🙂

    And isn’t that what project teams should all be about? Collaboration, smart thinking and playing to people’s strengths?

  3. @Sally Lawson – “And isn’t that what project teams should all be about? Collaboration, smart thinking and playing to people’s strengths?” – agreed and yes it is about balance. I’m not saying that passive checks are unnecessary – rather they are in important in the right context (regression??)…

    However, I believe that an aggressive testing mindset outweighs (in my opinion) a consistent check, verify and validate mindset that is unfortunately too common nowadays!

    Thanks taking the time to comment 🙂

  4. Excellent.Very nice post on passive and aggressive testing. It is indeed helpful and informative. For any successful application or product QA & Testing remains one of the crucial aspects that can’t be overlooked.It would great if you share your views on various methodologies for performance testing services.

  5. Thanks for the post. I recently started working for a new company and there’s been a lot of discussion about aggression testing which is a new term to me. So it sounds like to a great extent that aggression testing is basically trying to break a system or software to look for possible issues. What would passive testing be? It sounds like most of the testing I’ve done in the past is aggressive. I’m always trying to break my code in order to make it better.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s