STANZ 2009 Wellington New Zealand

STANZ (Software Testing Australia New Zealand) is the premier Software Testing conference this side of the equator! The conference kick off in Wellington New Zealand with Lee Copeland , James Bach, Karen N Johnson, Julian Harty and Brian Bryson forming the international cast of speakers along with a  host of talented local speakers.

Monday started with a keynote from Lee Copeland from which in outlined the innovations he sees coming. I found him warm, engaging and very humble.

James Bach was next and what impressed me the most was the way he *prowled* the side of the conference room before being introduced and then ran and jumped on stage! I was wondering a whole bunch of “what if’s” then! His talk Becoming a Software Testing Expert was vintage James Bach in which he discussed the plays of Euripides and other Greek tragedians and related them back to software testing. The point from my perspective is that testing is neither purely technical or engineering but that we can learn from all multple areas and disciplines (history, philosophy, pyschology etc). James also discussed his Huh-Really-So heuristic which he uses when someone makes a claim about something. Huh means i don’t understand, please explain what you mean. Really is what other approaches are there, what else could happen, what other tools could we use and So is to dismantle the argument or to determine whether or not the idea is worth pursuing (I hope i got this right! :))

Unfortunately i didn’t get to speak to either Lee or James one on one but i did manage to talk to Karen N Johnson and Julian Harty. Karen’s workshop on test pairing was very interesting but more so the discussion we had (myself, Karen and Sharon Robson) after. Karen also gave a wonderful keynote on story telling which i think as testers, is an area on which we can improved. We may test but how do we say what we see? How do we know who to talk to and how to talk to them?

The last highlight for me from a presentation point of view was Julian Harty’s presentation on security testing which i found extremely interesting. I came away from the talk with the ideas of :-

*Finding a mentor

*Use tools

*Threat modelling

*and continuous learning (including self study or self learning).

I managed to talk to Julian afterwards and what surprised me was that security testing is about 1% of what he does as a tester. However when he did do security testing, he taught himself/found ways to make himself knowledgeable and very effective.

STANZ was a blast! Great speakers, great conference and more importantly great people. I managed to catch up with a host of new/old friends and its was awesome to share STANZ with them!

SDC 2010 – BUSINESS ANALYSIS GETS AGILE!

Following on a from two very successful conferences SDC and STANZ 2009 (in both Wellington and Sydney), SDC 2010 has been annouced with the theme – Business Analysis gets agile. This will no doubt be a fantastic conference! Start planning to attend now!

See SDC 2010 for more details!

Can’t Beat Experience!

It has been awhile since I’ve written and its mainly because i have moved into the “academic” side of testing – delivering software testing courses for Software Education in New Zealand. As a result i have been busy travelling and delivering!

One of the main things that I’ve noticed during the course delivery is  the degree of separation between a junior test analyst and someone with more experience. At the end of the day, it appears to come down to how many more stories or experiences that someone who has been in the *game* longer has to draw on. This lead me to think about a quote from W.Edward Deming:-

“Experience by itself teaches nothing.” This statement emphasizes the need to interpret and apply information against a theory or framework of concepts that is the basis for knowledge about a system. It is considered as a contrast to the old statement, “Experience is the best teacher” (Dr. Deming disagreed with that). To Dr. Deming, knowledge is best taught by a master who explains the overall system through which experience is judged; experience, without understanding the underlying system, is just raw data that can be misinterpreted against a flawed theory of reality. Deming’s view of experience is related to Shewhart’s concept, “Data has no meaning apart from its context” (see Walter A. Shewhart, “Later Work”). – http://en.wikipedia.org/wiki/W._Edwards_Deming

From my perspective, the more “battles” one has been in, the more experiences to draw from (even if one has “only” been in one organisation) and to some extent, the approach of the test analyst (Agile, Automated, Exploratory, Scripted, UAT etc) may not been as relevant as it serves to broaden the range of ones knowledge.

When i first started testing, one of the theories that i held/learnt was that testing breaks software. When i was asked what i do, i would often respond “I test  software by breaking it”.

Over time, associating with different projects and talking to many people, i view testing as Archeology – we sift through the dust and dirt using whatever approach is necessary to uncover the bugs – the software is already broken – we look for clues to find where these may hide!

Therefore, as testers, it is up to us to find our theory – whatever or wherever that may be. Seek to learn, broaden your skills and then apply this theory when getting your hands *dirty* . This will help to broaden our experience and maybe, just maybe, our individual value as a tester!

BTW – I’m currently reading an Introduction to General Systems Thinking by Gerald M Weinberg – so I’m beginning to walk the talk!

Happy learning!

An Expression of Thought – Testing Ideas

Having no way as way, having no limitation as limitationI have become a fulltime trainer working for Software Education in New Zealand ( www.softed.com ) delivering software testing courses. As i’m now sitting more in the Academic space as opposed to the Practioner space, I have been given the opportunity to meet many different people, with different backgrounds, looking to gather new ideas to use in their testing jobs.

Some people won’t do much, if anything with this new knowledge (it’s human nature after all especially when the work pressure comes on) but some will. It is these testers that will hopefully feel inspired to share their thoughts and ideas with us all.

The internet has made us a very small, very connected global community and each thought expressed or shared (particularly in testing) is a thought worth considering. Maybe you have discovered a new idea with regards to testing or maybe even reaffirming an existing idea (and adding your own wrapper around it).

To those testers that i have met or to those people who may be reading this and haven’t yet considered creating a blog, please rethink. Your thoughts are valuable and your ideas are at least worthy of expression and/or comment.

I would to *hear* them – please let me know if you do! 

Happy blogging!

The Power of Two

I am currently watching and listening to colleagues perform Exploratory Testing simultaneously. Instead of one working the keyboard and the other gathering oracles and recording paths, they are testing the application at the same time on different PC’s.

WOW! What a synergy! There is a flood of ideas, debates, discussions, agreements and the beginnings of their conclusions on this particular application.

The idea that Exploratory Testing is a cheap approach to find quick, superficial bugs is completely untrue….I’ve just in the last 30 minutes seen the converse to that argument! I am watching a creative collaboration of minds – coverage obtained – yes (i know that application enough to understand the coverage of functionality) diverse – yes, depth  – yes – Superficial – NO.

I have been involved in Exploratory Test sessions where the creative juices just absolutely flowed – to those that oppose Exploratory Testing with superfluous arguments like ‘its monkey testing with a million monkeys at the keyboard’ – miss the point (maybe its because they want to quantify creativity but can’t …somehow…fit the square peg…into the…round..hole).

The point to Exploratory Testing is that the mind is the key to testing for it is the mind that allows inspiration and ideas to be generated and therefore expressed onto the ‘canvas’. It’s not ‘touchy-feely’ and to suggest otherwise may also suggest that the spark of creativity is missing from that person.

Otherwise, how do you explain music? How do you explain that feeling of ‘being in the zone’? How do you explain the artist that adds the touches to their work of art guided by their inner feelings?

Testing may be part of computer science but that doesn’t mean we need to conform to the discipline like robots. Testing doubles its effectiveness when its couple with intelligent thought processes.

I’ve just seen it!

The Art of Championing Bugs – The Bug Advocacy Course

Well its been awhile since i’ve last had the opportunity to post and there are a couple things that i will comment on in due course. The first of these is the BBST (Blackbox Software Testing) course 200A – Bug Advocacy. This course is part of the Association of Software Testing’s course curriculum (http://www.associationforsoftwaretesting.org/drupal/courses/schedule).

There are a number of positives aspects to the method of delivery and to the content contained within the course. First of all, you (as a student) are connected with software testers around the world (i have ‘met’ testers from Australia, India, New Zealand, Sweden and of course the United States) and learning starts straight away. This is because my testing context in New Zealand may differ from someone in India and will differ from other’s in the US. This is valuable because you are now connected to some real thought leaders and people who have different experiences ground in practicality.

Second is the quality of the instructors – Professor Cem Kaner (a leader in the testing world) and Scott Barber (a guru in the Performance testing sphere) coupled with other quality instructors such as Doug Hoffman, Pat McGee et al (refer to the Association for Software Testing website for the course instructors and then google their names for context). The instructors have *been around* (excuse the term 8) ) and are willingly to share their knowledge and understanding freely. They critique with validlity meaning that what they have to say has substance and credence (i would cite the many examples from the course but that may detract from future opportunites of growth for the next crop of course participants) and allows the student to actually learn.

I can’t do that from a multi choice tickbox with no feedback given.

Thirdly, the questions in the exams/quizzes are designed to be read throughly and applied to the context at hand. I struggled with this. I could say that because i haven’t been to University and received a degree in anything (other than life!) my exam taking skills are outdated …. but that didn’t matter. See, you don’t need to have a degree to be successful in this course – just listening eyes, observant ears (yes that’s exactly what i mean) and a thinking mind. I struggled because i’m a jump in and do person – stepping back and thinking things through come second…

While i didn’t overcome this tendancy i did make progress and we as students got some great instructor led/peer feedback so learning was maximised through collaboration and guidance.

And lastly, working together as teammates in some course exercises (and this may be dependent on the course content) allowed us to utilise other testers thoughts, points of view and experiences together with our own ideas to deliver a stronger, better framed answer to some of the questions we were given.

Learning was therefore continual, learning was shared and learning was amplified. The AST courses are some of the best courses i had ever been on and i highly recommend them (…and they are free!)

Part of my email to Cem Kaner and Scott Barber capture my thoughts thus…

“…I have learnt alot from this course and i feel that i’ve gone better this time around compared to Foundations. Cem, the recent discussion on grading and call of questioning was like a big light bulb going off in my head when i read it….being someone that has not attended University, these ideas were ‘foreign’ to me but refreshingly interesting (i think my mind has ‘expanded’ during these two courses).

Scott, your insights and answers were ones that i learnt alot from and was drawn to (as well as Jeff’s, Dee’s and Anne’s) – you were like a stealth instructor/student…i’m sure that if you were my PM, i would flourish under your guidance! The discussion of Question 5 was gold!

 Bug Advocacy and Foundations – I have learnt more, made more mistakes, kicked myself, got mad at the questions but came away with a feeling of actually learning something and achieving it. I compare this to a certain certification that is now prevelant in the marketplace (well in this marketplace). I sat the course and pass the multi choice questioned exam very, very well….but i don’t remember alot of it (except the V-model which is now ingrained in my head despite the fact that i don’t know if i’ve ever worked in a V-model environment) and I’m not sure if i learnt much.

That certificate for me is, at this stage, my commercial ticket (in this marketplace) but the BBST courses are, for me, where the real growth and learning have come.

Thank you both, thank you Doug and Pat for your time and also all the participants on the bug advocacy course!

The Pursuit of the Unreplicable Bug

ghostbusters.gifI’ve been recently testing a web-based application that produced a very interesting defect. It seemed that in one particular screen, a user (with the right combination key strokes and mouse clicks) could actually enter a supposedly uneditable error message field and enter text! At first i wasn’t able to repeat this behaviour but with the words from a James Bach article ringing in my ears about “…ignoring unreproducible bugs at your peril”, i logged it waiting for the right opportunity to attack it.

I had already spent time looking for this ‘bug’ but figured that i would put it to one side and come back to it with fresh eyes and clearer thoughts. Interestingly enough, the developers caught hold of this bug and attempt to replicate in their dev environments – i was even ‘challenged’ in a joking way that if i couldn’t reproduce the bug within 5 attempts then it didn’t exisit!! Oh, did the competitive urges come out then! (This was done in good spirits – we have a tremendous rapport between developers, testers and BA’s). However, it was another developer that found the key/mouse strokes that generated the bug and we discovered that it was a validation error on that web page!

So what were the lessons learnt?

  1. Exploratory testing found this bug – some may say that discovery was a ‘fluke’ but scripted testing would never have picked this bug up.
  2. Fresh eyes and a clearer head can aid tremendously in trying to replicate a bug (especially one discovered late in the day!)
  3. Having a rapport with developers helps in solving bugs – personal agendas and politics are put to one side for the greater good of the ‘team’
  4. Working alongside developers generally breaks down communication barriers (percieved and physical)
  5. Unreproducable bugs ARE best ignored at ones own peril – in this case finding this bug lead to a tightening of field validation for the application
  6. Bugs are bugs are bugs…testers find them, developers fix them, buisness decide what they want done with them – never give up on trying to replicate bugs that are difficult to reproduce!
  7. Teamwork – i honestly believe the power of many can be greater than the power of one
  8. It’s tremendously satisfying finding a bug that is difficult to find and reproduce – the testing equilivant of a three-pointer!