My experience of the magic of StarWest 2012 – Day 1

October 3 and 4 – Star West 2012 – Disneyland Resort, Anaheim California. What better place to be than at Disneyland to experience the magic of sharing, learning and collaborating with fellow testers.

StarWest 2012 was my first experience of a Star conference and the expectations that I had were definitely surpassed. Yes, I suppose  in some ways, in could be termed a *trade show* for testers but it is also a very good opportunity to hear (and be heard), see (and be seen – networking) with fellow testers who are typically on the other side of the world to me.

The following is a summary of what I discovered at this years StarWest…

Day One – October 3th
The day started with me getting *lost* by following my iPad map and taking the wrong turn. This was a case of error between tablet and user.
Upon arriving and registering, we (delegates) were presented with a free breakfast – free? food? Yup i was there and didn’t turn it down.

As i ventured into the conference room, my immediate thought was that this place was big…and there are a lot of testers here (relative to Australasian conferences I’ve attended). Lee Copeland open the conference and as a nice gesture acknowledged all of the overseas testers in the room.
I park myself down to the side of the main stage – I think I spy Scott Barber, Griffin Jones and Michael Bolton and make a mental note that I must catch up with them.

So I opened the iPad, twitter fired up (and I notice how quick the internet access was) and settled down to two excellent keynotes.

The first talk was by Jonathan Kohl on Tapping into Testing Mobile Applications. This talk got my interest in that I don’t test in that space but I own a phone and the fact that Jonathan made his talk very interesting got me thinking that I may in future be testing devices and apps – so why not pay attention!?  These were some the ideas that I took away (as I captured them and recorded them on twitter)

#starwest.. Move beyond the black box…the black box is dead in mobile testing @jonathan_kohl #testing
#starwest…gamification…reward the best bug video of the week.. @jonathan_kohl … Me: great idea!
#starwest…gamification of work @jonathan_kohl .. Me: send testers on a quest, level up to test paladin level 1…sounds cool!
#starwest … Gamification of work movement… @jonathan_kohl me: Great idea!
#starwest … Social features..now do we harness the time spent on social media…use social media interruptions in your app testing
starwest … Testing mobile apps..get out in the real world..at home, on the move and away from home (mall,weather etc)
#starwest …build something amazing as opposed to strictly following a certain methodology @jonathan_kohl
#starwest…usability testing and how a user uses an app is paramount! @jonathan_kohl ..research usability testing approaches
#starwest …one type of project challenge with web apps…extreme time pressure @jonathan_kohl
#starwest …airline apps are developed on assumption of a strong network signal…at an airport not always true @jonathan_kohl
#starwest @jonathan_kohl …think about whats outside of the device…
#starwest think about testing apps in the grey box space. Black box may not be enough
#starwest..understand users motivation and emotions
#starwest …testers don’t use apps in ideal environments
#starwest deleting apps is an emotional thing. Think about emotions when testing mobile apps
#starwest apps…don’t give people reasons to delete your app #testing

The key points for me were about understanding a users emotions and motivations when testing a mobile device and that testing is not confined to just the *black box*. There is more to testing that the front end.

What also struck a chord with me is when testing an app is the  need to have in our mind the thought of not giving a user a reason to delete the app. It doesn’t take long to install and even less time to delete. Also the reason to delete an app may be magnified if a gripe about the app is spread on social media. If this was done, it may require a hail mary pass to come back from that!

Great talk – crowd now *fired* up!

The second keynote of the day was by Johanna Rothman on Becoming a Kick#$% Test Manager. To start with, Kick#$% here actually means awesome!

Here’s is what I captured via twitter…

#starwest…if there is no problem to solve then you don’t need a meeting @johannarothman
#starwest…if the meeting doesn’t have an agenda you don’t have to go…ask for 24hours notice for an agenda @johannarothman
#starwest…How to say no to multi tasking amongst projects @johannarothman http://t.co/JDLEib6l
#starwest…Build communities of practice…”lunch and learns” great for learning what others are doing/sharing @johannarothman
#starwest.. Forget about solo experts and multitasking…testers assigned to multiple projects..nothing gets done @johannarothman
#starwest…When people start caring about bugs depends on where they are in the products lifecycle @johannarothman
#starwest…An awesome manager has regular one on ones…they are more AWARE of what is going on @johannarothman
#starwest…testers are generalists …help them move into other positions with they want to.. @johannarothman
#starwest…coaching is when your offer options with support..when its needed @johannarothman
#starwest…as a test manager, how do we determine that our people are high worth to you? @johannarothman
#starwest…you need testers that can also understand the solution space. @johannarothman
#starwest….hire smart, high value people. When you start talking about price, you’re not talking about value @johannarothman
#starwest…@johannarothman … We test from the perspective of curiosity not victims….
#starwest…@johannarothman … Next StarWest keynote on becoming a kick@$$ test manager. Kick@$$ = awesome!

I dislike meetings or to be clear, meetings that do nothing and have no direction. They are time wasters and give the illusion of activity so when Johanna reiterated then need for an agenda or not having a meeting if there is nothing to solve, that got a big green tick in my mind. Like a lot of us, we have been in far too many non-productive meetings that generate…nothing.
Great talk – first time I’ve heard Johanna speak and the crowd was fired up again ( I thought I heard chanting like an English Premier League soccer match in the far corner acknowledging Johanna’s keynote – but i digress).

After the keynote, the trade show opened when a million tool vendors (of very similar tools) and consultants dazzled everyone with their wares (and swag – I was too slow for the Atlassian t-shirt but picked up a couple of usb sticks, a mouse pad and a whizzy pen that lit up.) Interestingly enough, the big focus appeared to be tools that tested mobile applications.

So, being the tester that I am, I decided to rock up to a few vendors and ask if they have any tools that work on iSeries.

Mostly I got blank looks.

One vendor attempted to find out more but admitted that they didn’t really know if their tool could – If it did and If I was looking at their tool for iSeries, I would sound them out. Why? They were honest enough to admit that they didn’t know (though did they did try to find out) and that *integrity* counts for something (some other vendors gave me a rehearsed sales pitch without listening to what i was saying – at that point, I just wanted to leave – but pass me the swag first! :))

I quite enjoyed walking the tool vendor hall and it was good to be amongst the *buzz*.

Spotted Michael Bolton and introduced myself and chatted to him. Bumped into Scott Barber and we had a good talk – 2 out of 2 so far…

From 11:00am till 12:00pm I was involved in giving free consulting sessions. Anyone that wanted to talk and ask could (and did). I had some good discussions particularly on the state of test automation in New Zealand (which unsurprisingly, was similar to the US – just a different degree of magnitude). I also managed to get a *free* consult with Doug Hoffman, a real gentleman amongst testers and I came way with some answers to some questions.

After lunch, track sessions began in earnest and there were six streams to choose from which I won’t go into detail here. Suffice to say that I took quite a few notes – I was like a kid in a candy store!

Leaving the conference after day one was a buzz. Ideas floating around my head – walking through downtown Disney with the Disney tunes playing non-stop –  spying Goofy and the Mad Hatter – it all added to the magic of StarWest 2012 (how can you NOT be entranced by having a conference here)!

I’ll talk on day 2 next post…

Advertisement

Agile @ StarWest Software Testing Conference 2012

In my experience, what makes agile so powerful is the  encouragement of rapid, effective communication to achieve, uncover and discover what is wanted, what is being built and what could be going wrong. Practices such as collaboration and co-location can be effective tools for any project regardless of whether your project is agile or not.

I will be at Star West 2012, Anaheim, California next week (1 October 2012 – 5 October 2012) and will be speaking Thursday on how I borrowed some agile practices for a non-agile project and the lessons that I learnt.

If you’re attending Star West, come a long and say hello otherwise I will blog and tweet where I can!

See http://www.sqe.com/StarWest/Concurrent/Default.aspx?Date=10/4/2012#T19

Star West Software Testing Conference 2012

During the first week of October 2012, I will be presenting at Star West at The Disneyland hotel, Anaheim, California on Using Agile techniques to Manage Testing – Even on non-agile projects (http://www.sqe.com/StarWest/Concurrent/Default.aspx?Date=10/4/2012#T19 ).

Its going to be an exciting testing conference and I’m looking forward to meeting fellow testers at such a prestigious event. Already as I scan the speaker list I see testers such as Michael Bolton, Dawn Haynes, Rob Sabourin and so forth who are leaders in our craft and I’m looking forward to meeting them (again) and talking – what else? Testing!

No doubt there are many more here that are not on the speakers list and I’m looking forward to meeting you too. 🙂

See you there!

Whom shall I serve?

Tweet – Mike Talks – KWST#2 – June 2012

Whom shall i serve?

A song, a hymn, or a reminder as to who our customers are? Who do we serve and why is that important?

During KWST#2 (June 2012, Wellington, New Zealand), the discussion about whom  we serve came up.

Mostly, the answers tended to support the obvious conclusion (to me at least) that whom we serve could be:-

Our employer(s)
The project manager
The developers
The business
The test manager
The test team
The project team
Our family

And these are all valid customers/people/organisations/groups that we give service to in some way. But there is one other element that sometimes we don’t consider…

Ourselves

Whom shall I serve? I think first and foremost it is ourselves. We are responsible for our own work, for our own ethics, our output, our own learning, our own interactions with others, our own interactions with other testers and our own interactions with the software testing community.

Sometimes we take a high degree of responsibility for one or some of these things and sometimes we don’t. What may be important is that we come to understand that we also serve ourselves and by seeing ourselves as a customer (if you will) then it allows us to appreciate who we are as a tester, what we can deliver, what skills we have and what we stand for.

Too often I have seen testers wilt in the face of criticism (and scrutiny for that matter) from management attempting to justify testing or test artifacts or activities. Knowing what we stand for gives us a moral ground to argue from. Unfortunately, it doesn’t mean that everything will be *perfect* because we are conscious of our position but at least we know our tipping point.

So how do you deal when reaching your tipping point?

Well, that does depend but some of the ways that I have used have been:-

• Educate those that may be pushing you towards your tipping point – (in my experience, it is typically a manager)

• Listen to those pushing you to your tipping point – (it is possible that we don’t understand their context)
  

• Use your influence and credibility to help educate

• Employ a stealth approach – (one project I was on, the project wanted test cases (with expected and actual results) and use what they saw as structured testing. While we spent time giving them what they wanted, the majority of the issues during test execution came, not from the test cases, but from an undeclared exploratory approach. OUR plan of attack became give the customer what they wanted, educate them along the way and use good exploratory testing to find valuable information *quickly*. The test cases in this instance were our checks, the exploratory test charters, our tests. The stealth here was from discerning the clients context,employing what became a blended approach and not necessarily letting management know that this is what was happening.)

• Leave – (this is most likely the extreme option but sometimes it is more beneficial to/for you to leave a project/employer/organisation than having to adhere to rules that may not make sense. I have done this, it was a challenge but I’m glad I did it.)

So, whom do we serve? Ourselves first (it’s not as selfish as it may seem) and then those mentioned above. Putting ourselves first means that we are taking responsibility for the quality of our own work which means in turn, we are better placed to serve our customers.

Learning from the frustration of test case debates

What is a test case?

The reason I ask is that recently I have been following (and commenting) on a question in the LinkedIn group Software Testing & Quality Assurance  – ” hi guys do u think that creating Test Cases is must? According to me, creating Test Cases is just a waste of time rather we should utilize the time in doing testing. what is your opinion? ”

At first glance I thought it would relatively *easy* and pick apart this question and the ensuing replies. However, after reading through the comments, I immediately felt frustrated. Why?

Upon reflection, I noticed a couple of things.

First, it helps to view the comments from the start. I had missed the fact that there were something like 100 comments before I jumped in. Realising this would’ve help save the frustration because Griffin Jones said it from comment one

@Pradeep – I forecast that this conversation will become confusing because:

a. people will be using unstated assumptions about what is a “Test Case”. Some people’s working definition will exclude other people’s working definition of the term. This “shallow agreement” problem will not become obvious until comment # 78.

And Griffin’s prophecy came to pass.

Which led to *the* problem:

Comments were roughly divided between *a test case is a set of inputs with expected results* group that talked of the test case as a tangible artifact. The second group tended towards seeing the test case as an instance of a test idea and generally speaking this second group were the ones that seem to constantly challenge the assertions of the first group.

And then it dawned on me.

The second group appeared to be aligned with the context driven school of testing and as such realise that were *a lot* of dangerous assertions in the comments made by the first group. For example:

Testcases ensures the tester does not over do his testing and makes sure when and at what stage of his testing he could exit and say that the testing is complete.

If we were to look at the above statement a number of questions spring to mind. First of all, how does a test case ensure that a tester does not over do his testing? What does it mean to overdo testing and if testing is *overdone* what is it compared to be deemed overdone?  If the commenter means ignoring the risk of testing something else or finding information outside of the scope of the test case then overdone has potentially risky consequences for him or her (as they have now have jumped outside of the test case box and may find interesting information…tsk, tsk as now they may not meet their execution test case complete target because now they are THINKING about what they are doing as opposed to just doing *something*.). If the tester became engaged then they would be aware of their coverage and risk model and seek after information that may challenge that model. Notice that the engaged tester does not complete a test just because they have ticked off all of the steps to execute;   otherwise, we end up blindly following a script and we’re checking not testing. This highlights an issue of the commenter viewing a test case as a tangible item when in reality it is an abstraction. It is an idea (or collection of ideas) and *passing* a test case does not guarantee that the idea is finished with. Rather a good tester will most likely extend that idea into many ideas.

Of course we could critically pull apart the rest of the comment and show the fallacy in the statement (such as how does finishing your test cases mean that your testing is complete? It could in some circumstances but I suspect that the commenter meant completing testing – full stop). There are a number of comments like this and they all follow the same theme. We write test cases so that we can cover the requirements and we have repeatable tests so that we can teach others and because the v-model aligns with Saturn and Mercury in the house of Leo – so it must be good!

But I digress…

AND this was  frustrating for me. It seemed that no matter now many times (and in different ways), the second group (lets called them Team CDT) highlighted flaws in the first groups arguments (lets called them Team Factory) then another equally inane comment appears and it made me realise that (to paraphrase James Bach)…

If you are frustrated then it means that something is frustrating!

Realising this then made the rest of the journey…well…more fun.! I realised that I could not wilfully change anyone’s mind except my own. I realised that, regardless of what I shared, others are free to disagree. I realised that no matter how many times I pointed out a fallacy in someone’s argument, it’s up to them if they take heed or not.

AND I realised that I could actually benefit from this and not let the emotion of frustration take hold.

How you say?

By looking for like minded individuals and engaging with them knowing that I’m mostly likely to get a meaningful discussion coming back. By practising pulling apart a comment and challenging someone’s assertions. By applying James Bach’s Huh?Really?So? heuristic and what was initially a frustration quickly became a learning experience.

While it’s galling to see many testers fall into Team Factory, I am hearted to see a number of testers critical of the *status quo* and challenge them (as demonstrated by their replies to team Factory comments). It is through challenging that we grow the craft into something that is stronger, assertive and more critical overall.

KWST#2 – Day 2 – some thoughts

Day 2 had  a different (positive) vibe and in part I think it was due to the solid first time experience reports by Katrina Edgar, Mike Ward and Mike Talks. All three gave reports that really highlighted some of the ethical challenges we face as testers in our day-to-day world.

Some of the thoughts as tweeted on day 2 were…

• Testing based on rituals is NOT testing – [the blind adherence to a test tool or process or syllabus is NOT testing. Testing is a brain engaged activity]
• Testers are there to report the truth, not the convenient truth – [Testing is about presenting the facts as they stand and not manipulating them to suit an agenda (for an example of the *truth* being misused see http://www.theaustralian.com.au/australian-it/states-health-payroll-change-was-adopted-untested/story-e6frgakx-1225888223958]
• If challenging, is your reputation strong enough to withstand any ethical fall out?
• As a tester have to look through the eyes of the people who matter. Those with whom we have a contract.

During discussion, the topic of agency came up and the point of who do we serve as testers (see http://en.wikipedia.org/wiki/Law_of_agency )? This discussion brought a renewed energy into the room with Geoff Horne leading this. The question that was asked was…

Who do we serve?

Geoff stated…

We are engaged by an individual, and by the organisation behind the individual

Which is true to a point and from the perspective of the engagement between tester and *client*. However, I see the question from the perspective of the tester which is…

I serve myself first before any institution, as I’m responsible for my own ethics

In other words, as a tester, I am responsible for the ethics I hold and I carry. I am responsible for making sure that my house is in order first before the needs of the organisation are considered. At this point, the extension of what I consider ethical is extended to who I serve literally (or in Geoff’s case, from the perspective of the engagement) which are the people I work for/with and the organisation at large.  The discussion of agency and other ethics topics can be summed up quite nicely by a tweet by @NZTestSheep (aka Mike Talks)..

The great thing about an event like #KWST2 is how it challenges our models and maps, and we’re still processing it days afterwards

Learnings:

KWST takes a lot of organising and it is the detail that count such as…

• A good venue (space, lighting etc)
• Internet connection ( VERY helpful)
• Appropriate twitter tag
• The RIGHT people to invite [This year revealed some really good thinkers and it will be exciting to be working with them at future KWST conferences]
• It can spawn off-shoots (like David Greenlee’s OZWST)
• Facilitation is king – it takes practice, a firm hand and the ability to know when to let the conversation flow
• Preparation before hand FROM everyone (and reminding everyone know that they are potential *speakers*)

Thank you for all those that attended KWST (see http://hellotestworld.com/ and http://martialtester.wordpress.com/2012/06/18/kwst2-what-a-ride/ and http://martialtester.wordpress.com/2012/06/19/kwst2-happy-snaps/ )

Thank you James Bach for your time in helping build a credible, professional, thinking community of testers down under and thank you software Education for your support in hosting KWST#2!

***EDIT: Much thanks must also go to Oliver Erlewein, Richard Robinson and Aaron Hodder for their drive and passion in prompting thinking, engaged testing especially here in Wellington, New Zealand. ***

KWST#2 – Day 1 – The beginning of real leadership downunder

KWST#2 started off with an absolute air of anticipation.  I arrived early to make sure the venue was all set up and ready for the intrepid KWST’ers!  KWST is an off shoot of LAWST and is unique in that everyone is able, likely and potentially willing to contribute through questioning or by giving an experience report.

 

This year’s theme was Ethical Challenges for Testers and it promised to be an intense conference!

 

This year, 21 test leaders throughout New Zealand Australia were in invited.  There was no magic formula, no *robust estimation tool* used, rather the attendees were invited on what they have done within the testing community (see www.hellotestworld.com or www.martialtester.wordpress.com ) or could achieve.  In attendance were testers from vendors, consultancies, independents and various companies throughout the country. So…what happened?

 

Day One

Once assembled, everyone *checked in* (a process whereby we explained where our head space is at and a chance to off load by announcing any distractions that may bother us that day) and James as content owner explained the process and the use of the coloured cards.  Richard Robinson was lead off facilitator and I was supporting him.  James led the way with his first ER and there were a number of questions (threads) which flowed from that.  As is typical of a peer conference, the first ER tends to *flesh* out a lot of questions around the topic (in this case ethics) and was a good jumping point into KWST.  Jeff Bidwell, Geoff Horne and Andrew Robins also gave experience reports on day one with varying degrees of success (i.e. the ability to give an experience report and defend their position).

 

First of all…

 

• When giving an experience report, the deliverer should ensure that they can back up what they say and defend their statements when necessary. This is a credibility issue because if someone can’t defend their work then they could lose credibility with their peers
• Reputation by attribution was something that James spoke about (and tweeted well by Oliver). Reputation needs to be defended as is your reputation by association.  If for example, you are hired by a consultancy then that consultancy picks up your reputation as a tester and the tester may be tainted (whether it’s good or bad) reputation of that consultancy regardless of work done by the tester.  It’s almost like reputation by diffusion.
• Counting test cases OR understanding and using meaningful metrics was a hotly debated topic.  It was clear that the room was divided on counting test counts being ethical (of which the discussion itself was taken outside of the conference when it was agreed that we were heading down a rat hole).
• When is a tester ethically responsible for what he does? Make a clear distinction if you are a tool-tester or are directly responsible (tweet by Oliver) which followed a comment by James that a tool tester may not be responsible ethically because they are directed and told what to do.  However, IF you are directly responsible for what work you do then yes, you have an ethical responsibility to produce good, meaningful testing for your client.

Day one was like the start of a boxing match or sporting contest in which the *contestants* feel each other out, understand expectations and determine limits.  Some didn’t like or understand the process or power of a peer conference.  This power comes from the CONFERing – discussing, challenging, critiquing and attempting to understand a presenter’s point of view. The power comes from dialogue. The power comes from testers grappling with the assertions made and dissecting them. This is how a presenter’s reputation is won (or lost) at a peer conference.

Day one was an opportunity to confer AND to network. KWST was about test leaders coming together. It’s not about business or certification or testing fallacies. Not all invitees will/have stepped up as leaders within the community BUT a number have….

And the testing community in New Zealand and Australia is the better for it…

 

Next Post – KWST#2 – Day 2***

Attendees for KWST#2

 

 

 

 

 

 

 

 

James Bach (content owner), Richard Robinson (facilitator), Brian Osman (facilitator), Oliver Erlewein, Aaron Hodder, David Greenlees, Mike Talks, Katrina McNicholls, Liz Hutching, Katrina Edgar, Geoff Horne, Andrew Black, Farid Vaswani, Jeff Bidwell, Sheryl Toenders, Chris Stapleton, Donna Chin, Andrew Robins, John Lockhart and Mike Ward.

 

 

 

 

 

Countdown – one week to KWST!

Just over one week to go until KWST – a thought leadership peer CONFERence like no other downunder. For those that may want to follow the twitter feedback, the #tag is #KWST2.

The theme is Ethical Challenges for Testers and the conference starts 15/16 June 2012.

There will be 20 test leaders from throughout New Zealand and Australia and really promises to be a special event – so stay posted!

Much thanks for this conference must go to James Bach for his time and willingness to help found KWST and for Software Education for bringing James out to New Zealand and for providing a place for software testing thought leadership to grow!

 

 

KWST#2 leadership conference is back!

Reposted from http://hellotestworld.com

The second KWST or Kiwi Workshop on Software Testing  with be held on the 15th and 16th of June 2012, Wellington, New Zealand. KWST is modelled on the LAWST style peer conferences and is the only test leadership summit in New Zealand. There are a number of things that make this conference unique:-

• It is an invite only conference – we are looking for industry thought leaders and hence why you will find few, if any, invites from body shop organisations. While they may have some talented testers, these companies tend to pay lip service to thought leadership
• James Bach will again be back as content owner and helping grow the core of professional test leadership in New Zealand
• Some of the brightest, insightful test thinkers down under will be there
• Unlike any other conference held here, this is a CONFERence where ALL participants participate!
• The theme is Ethical challenges faced by testers which is relevant considering the overuse of body shop testing companies and certification within the testing industry

The twitter hash tag will be KWST2 and we will be tweeting all of the great thoughts and ideas that will flow from this conference. See https://bjosman.wordpress.com/2011/06/28/kwst-kiwi-workshop-on-software-testing/ and https://bjosman.wordpress.com/2011/07/08/kwst-kiwi-workshop-of-software-testing-day-2/ for details of last years event.

 

 

The Invasion of the Bodyshoppers – Part II

Here is a list of ideas on overcoming the influence of the bodyshoppers

• Understand your commitments and ethics – usually bodyshoppers pimp out their *testers* for the sake of profit not skill
• Look at the reputation of the tester – I mean the real reputation. Look beyond what some so called bodyshop test practice manager thinks. A real reputation means how the tester is viewed by the community and fellow testers
• Following on from that, what contribution does the tester make to the testing community? Do they blog or do they attend events? On a smaller (but just as important) scale, do they mentor fellow testers?
• Don’t feel the need to conform to *standards* – ISTQB, ITIL – whatever – most organisations ask for them but don’t really do anything with them making these standards irrelevant. The best standard is your own skill and reputation
• Confront the bodyshop consultant that begins to use terms such as best practice – there is no such thing! The bodyshop is spews best practice to make like they have the *answer*. Our job as real testers is to challenge such nonsense
• Bodyshops will say that they support the industry but sponsor nothing events that generate – nothing.  Look for events that are actually worthwhile (KWST in New Zealand or CAST or Rapid Software Intensive in the US for example)
• Challenge those that see testing as a set process to follow – i.e. join the dots testing – that is the worst kind and must be fought vigorously!
• If in a bodyshop, leave when you can – come to the light
• Resist joining the bodyshoppers – consider going out on your own

These are some ideas for now – what other ideas do you have that can help overcome the bodyshoppers?