The Pursuit of the Unreplicable Bug

ghostbusters.gifI’ve been recently testing a web-based application that produced a very interesting defect. It seemed that in one particular screen, a user (with the right combination key strokes and mouse clicks) could actually enter a supposedly uneditable error message field and enter text! At first i wasn’t able to repeat this behaviour but with the words from a James Bach article ringing in my ears about “…ignoring unreproducible bugs at your peril”, i logged it waiting for the right opportunity to attack it.

I had already spent time looking for this ‘bug’ but figured that i would put it to one side and come back to it with fresh eyes and clearer thoughts. Interestingly enough, the developers caught hold of this bug and attempt to replicate in their dev environments – i was even ‘challenged’ in a joking way that if i couldn’t reproduce the bug within 5 attempts then it didn’t exisit!! Oh, did the competitive urges come out then! (This was done in good spirits – we have a tremendous rapport between developers, testers and BA’s). However, it was another developer that found the key/mouse strokes that generated the bug and we discovered that it was a validation error on that web page!

So what were the lessons learnt?

  1. Exploratory testing found this bug – some may say that discovery was a ‘fluke’ but scripted testing would never have picked this bug up.
  2. Fresh eyes and a clearer head can aid tremendously in trying to replicate a bug (especially one discovered late in the day!)
  3. Having a rapport with developers helps in solving bugs – personal agendas and politics are put to one side for the greater good of the ‘team’
  4. Working alongside developers generally breaks down communication barriers (percieved and physical)
  5. Unreproducable bugs ARE best ignored at ones own peril – in this case finding this bug lead to a tightening of field validation for the application
  6. Bugs are bugs are bugs…testers find them, developers fix them, buisness decide what they want done with them – never give up on trying to replicate bugs that are difficult to reproduce!
  7. Teamwork – i honestly believe the power of many can be greater than the power of one
  8. It’s tremendously satisfying finding a bug that is difficult to find and reproduce – the testing equilivant of a three-pointer!

AST and the BBST Foundations Course

astlogo.gifIt has been awhile since my last post and its because I (along with 19 other esteemed test colleagues from around the world) have been ‘attending’ the Association of Software Testing online course – BBST – Foundations – see http://www.associationforsoftwaretesting.org/drupal/courses 

(as well as doing work of course!)

I have ‘met’ testers from Australia, New Zealand. India and the United States and to share in their knowledge has been superb! I have learnt alot and i have been challenged mentally with regards to my view on testing.

The instructors were Scott Barber http://www.perftestplus.com/ and Cem Kaner http://www.kaner.com/ and their knowledge and willingness to help everyone learn was outstanding. I highly recommend this course (actually a series of courses). The following is an email that i wrote to Scott…

Hi Scott,

Thank you very much…it was a privillege to have learnt from the ‘best’ – from the participants and of course our esteemed instructors! Yes it would be fine to post my name on the website. Again, as i’ve explained in my course evaluation – i have sat ISTQB and passed well BUT this means more to me – it was more challenging, stimulating and has me rethinking the way i approach things (either as good reminders or changes to my testing habits). Thank you once again and i hope we all can stay in touch.
Kind Regards
Brian