The Tester’s Job


In my first blog, I talked about Test’s value to the business.   It’s insurance against risk.    For the longest time, my answer to *how* to do this was one of my most precious assests.  If, as a manager, I was faced with a difficult decision, I could use my definition of a Tester’s Job to guide me to the right place.  It enabled me to set a vision and a goal.  Hugely valuable!

Here’s what it was:

A tester’s job is to effectively identify, efficiently execute, and promptly report on the minimum number of test cases need to achieve the customer’s quality bar for the product.

To me, it was brilliant.  It wrapped up all the things we need to do to be successful.   Test Cases, Bug reporting, Automation, Customer Empathy, helping Dev get the results they needed in time, etc.   It’s all there.   Highly efficient.  There was one BIG problem with it.

IT SUCKS!

An employee of mine at the time, Isidro Hegouaburu, said it best (imagine this being said with a rich Argentinian accent):

“Brent,  your definition is really helpful and it tells me what to do, but it depresses me.  I don’t want to do that job.  It’s not sexy!”

Ouch!   That hurt!  What was worse was: he was right!   Customer Delight is an Emotion after all; Emotion is key to quality.   To define a tester job, a champion of quality, so stoically was to demean it.

Ugh.

So I threw it away and began a search to find a better one.   I talked with several leaders and got their opinion over the years.

  • To find bugs
  • To move Quality Upstream
  • To inspect
  • To manage and mitigate risk
  • To make the product better
  • To measure
  • To make developers better
  • To ask questions

I’m sure you have heard these yourself.    I find these mostly unsatisfying.  QA tries to do all of these at times.  These definitions attempt to take one task and make it *the* priority.   I wanted something goal-driven and encapsulated it all (much like my long and painful previous definition did).

Also, most of these came from people who’s experience was mostly Waterfall-based, I wanted my new definition to cover the entire gammut of project execution styles, especially Agile, since many are switching to that.

the Ah Ha Moment

I often refer to an NFL player metaphor when I consider test.    I think of Developers as players and Testers as the armor being worn.

I was admiring Frank Gore, in particular.   I was amazed at his ability to deflect oncoming traffic and to get to his goal.  Then it clicked for me:

A tester’s job is to accelerate the achievement of shippable quality

Since dev puts the quality into the product, our job was to speed them through that potential quagmire and help them get the code (ball) to the goal.  We will use a variety of tactics to get that done (including many of those mentioned in the defintions above)

One team meeting a couple of months ago, I tested this out.   Since what we had scheduled for that week had been taken care of already,  I asked each of my employees to tell me either the most important or the fastest thing each of them should do that week to accelerate the achievement of shippable quality.

Their responses were each fantastic, IMHO.  This definition encouraged them to think of quality and what it would take to ship and what was the current bottleneck for each of them.  My favorite answer that day was the Unit tests from dev were so poor that, if necessary, the QA team should fund writing them and making them easier for dev to write in the future.

I’d love to hear what you think and what you think this definition might be missing.

Merry Christmas!

Advertisements

22 thoughts on “The Tester’s Job

    • Dutifully fixed! (twice) Teach me to write a blog without Word at my side. Thanks, George! Feel free to quote. It’s a form of feedback and I think feedback is essential to forming a quality community about quality. 🙂

  1. To misquote your employee: “Brent, your new definition is really sexy, but it seems unrealistic that testers can really accelerate achivement of /any/ quality level. Now, it really depresses me.”
    Maybe you could illustrate your point with more examples?

    • Kurniliya, thank you for your comment. First off, the employee I’m referring to here is one of the most upbeat people you could ever meet. I think it would be *very* hard for anything to depress him.

      Certainly, if he felt the goals of his job were unrealistic, it just might.

      Unfortunately, your viewpoint is not obvious to me. I’d be happy to give more context. Can you explain why you feel it’s unrealisitic?

      • Assume I did some testing and brought to you a bunch of questions and bug reports. Did this change product quality? (I bet it didn’t.) Did this change our awarness about product quality? (I hope it did.)

        As far as I understand, ‘accelerate the achievement of shippable quality’ thing is all about improving quality. How can one improve something she has no control over?

        If my goal is to improve quality then the most effective way to do this is becoming a programmer myself. Maybe, becoming a manager would do the trick even better.

        I felt that in your example (unit tests) QA team actually assumed responsibilities of dev folks. So, I hoped you could mention a bit more straightforward example.

      • If the limit of your influence is bringing a bunch of questions and bug reports, then that’s indicative of some of the problems. If only the manager or programmer has “control” over achieving shippable quality, then I’d say you’re not part of a team. (I’d also say you’re probably over-estimating the control that the manager and programmer can assert.)

        Take a look at http://manage.techwell.com/articles/weekly/three-amigos and see if it doesn’t spark some ideas on another way of working–one in which the tester CAN “accelerate the achievement of shippable quality.”

      • Kurniliya,
        Thanks for clarifying.
        I agree with the points that George and Scott have made.
        I have a few more to add to the discussion. I interpret your statement to be pointing out that the act of asking questions or filing bugs reports fails to literally change the product code. I agree. Typically, testers put quality into the product with the same frequency that a Hockey helmet puts the puck into the goal (it happens, but it’s rare). Devs are the ones who change the code; for better or worse.

        If I am reading in between the lines correctly, I think you are saying a Tester’s job is to test and then noting that the act of testing *does not* actually do anything with quality. Here I, also, agree.

        BUT I believe this view of a tester’s job is too simple. Certainly, testing is a task we must undertake, but it is but one of many.
        It is not the job of the tester to:
        • Write tests
        • Run tests
        • Write automation
        • Run automation
        • File bugs
        • Achieve x% code coverage
        • Etc

        These are all means to an end. But it’s not a fire and forget activity.

        Our job is to:
        • Understand our customer – it is *their* POV that defines quality, *not* the listing of requirements in a spec
        • Understand the business – What is the minimum viable product and quality level that enables our business goals?
        • Then, armed with this information, help to accelerate the product to the goal.

        In that team, Unit tests were very poorly utilized. I even had arguments with some of the developers as to whether or not they were valuable (though, only with Dev who had never written unit tests). However, all on the test team saw that they had value. Dev was frequently breaking the product and this was slowing down everyone.
        My tester noted this. We needed a unit test process to help us accelerate our engineering. Certainly, if management had come down and stated Dev all needed to write units, it would have helped QA a bunch, but their lack of doing so isn’t a blocker. If it speeds up our ability to deliver quality, test should not be afraid of taking it on. Test will need to get Dev to own this one day (to maximize the speed), but that shouldn’t be a blocker. This particular race is a marathon, not a sprint.

        Another example: in my last team, I banned UI automation. Why? Because my testers were viewing the quality problem as “how do I automate this?”, not “how do I sustainably measure the quality of the product?”. The product UI was constantly changing AND my testers as a whole did not have the skills for creating resilient tests. This was a wicked combination that resulted (pre-ban) in my testers being sucked away daily into fixing automation and not actually testing anything any further. They were playing catch-up, not improving quality. So I banned UI automation, told them to automate the middle tier instead, and focused on teaching some of them the art of testing, which they had forgotten. By doing this, we sped up productivity on the whole team (dev, test, and PM) Test was focused on the right priorities to the customer. That team is, of course, now buried by manual tests, but they are better prepared to tackle them efficiently.

        Hope this clarifies.

      • Brent,

        First of all, thanks for time and thought you put into the answer. I really appreciate it.
        Generalizing, you said that Test should not artificially restrict its role. If everyone agrees that writing unit tests should help, and Test can step up and write them – go and write. It is how things should be, I’m with you here.
        In the 2nd example you mention “the art of testing, which they had forgotten”. This intrigues me. Now I understand that when I asked for more examples, I really wanted to ask for “more examples from the art of testing” that support your tester’s job definition.

        My basic argument is this. Let us assume 2 things:
        1. A tester’s job is to accelerate the achievement of shippable quality.
        2. I want to be as good a tester as possible.
        What is the most effective way to accelerate desired achievement? I say: go and put some quality into product. We agree that Test do it rather rarely. Thus I have no other choice than becoming Dev.

        Here is one more thing that bothers me. There is too much of customer advocacy in your definition of tester’s job. I agree that it is *customers’* perception that defines quality. But customers do not employ testers, business does. Customers do not pay testers, business does. Don’t get me wrong, I do not intend to say: forget about customers altogether, never think about them. I just try to emphasize that interests of business should come first, before customers’ interests. Business and Customer are not of equal importance for tester, Business is more important by an order of magnitude. And “quality” thing is definitely customers’ one.

      • Ilya,
        Again, thanks you so much for the response. I think I see where you are coming from now. There are two statements you make that I want to address:

        1) Does one *have* to become a dev in order to accelerate the achievement of shippable quality?
        2) Brent, don’t you think the Business’ interests come before the Customers?

        #1
        I’ve got a future blog queued up on this very topic, so I’ll be brief here. Instead of talking about Dev and Test, let’s talk about Car Racing. Both the driver and the engine mechanic have the same goal: Win the race. But let’s say the engine mechanic’s job is to “accelerate the achievement of winning the race”. If the mechanic was good enough, he could simply take the wheel and drive. In fact, some drivers are very capable of working on their engines as well as driving the race. This does not mean the engine mechanic doesn’t have a role to master and get better at. He could look at different designs of engines, different fuels, analyze the driver’s style in steering/acceleration/braking/etc. – all to enhance the ability for the driver to get the car to the goal line.
        It is no different in test. We look at the goal line and the engine and we do what’s needed to accelerate the code there.

        #2
        Maybe. In my experience though, the customer’s interests ARE the Business’ interests. The business hires testers in order to minimize risk to the customer. Customer Empathy/understanding/advocacy is, imho, where we differentiate ourselves from Dev. While they focus on high code quality, we focus on high experience quality.

      • George,
        Thanks for the link. Three Amigos is great. Actually, it sparks even more ideas than are relevant to our current discussion. I definitely need to think more about what you write before answering in a contributing way.

        For now, I’d say this. Both you and Brent seem to underestimate the power of “merely” asking questions and providing information. Hence, desire to grant Tester more direct abilities to change quality. Maybe this story could shine a light on power of questioning: http://www.joelonsoftware.com/items/2006/06/16.html and its value for business.

      • Ilya,

        I don’t think I do underestimate the power of asking questions and providing information. If I did, then my friend Michael Bolton would put me right again.

        But those questions and that information don’t have much power unless they influence “the acceleration of achievement of shippable quality.” In some organizations, a question or two to the right person may well do that. In most, not.

        Joel’s story shows that someone with great power in an organization can exert a lot of influence with a few questions (particularly when those questions grounded in detailed knowledge). In many organizations, the testers are not given much power and influence, and their questions and information are often overlooked until it comes time to blame someone for a problem. Great testers who understand social systems can frequently overcome such dysfunctions, but it’s hard.

        My goal with the Three Amigos concept is to make the tester’s activity of questioning and providing information a more explicit part of the way things are done, and to move that activity up in time, so it can provide value throughout the development cycle.

    • Thank you for the links! I very much agree; you and I share a viewpoint here. I really liked your STP summary as well. (I’ve got a post on Metrics queued up)

      I am happy to call Dr. Whittaker a friend, so was lucky enough to get the a synopses of his STAR keynote speech (http://news.yahoo.com/google-james-whittaker-declares-death-tester-152231257.html) before the event. I agree with James that Testers, not Testing, is what is ill-fated.

      I will return the compliment and state that your “10 things…” post is fantastic. All of the things you list are bottlenecks that prevent acceleration. My personal favorite: “Process Weinie-ness” – don’t ever let anyone tell you it’s not a word! 🙂

  2. Hi Brent, thanks for sharing your definition/vision/goal. I like it and only have one small niggle with it. The word ‘Tester’s’. I believe it’s everybody’s job to accelerate the achievement of shippable quality.

    Testers neither decelerate or accelerate the achievement of shippable quality directly, they provide information which can be used to make a decision on whether shippable quality is being achieved, or not.

    • Thank you so much for your feedback! I agree it is everyone’s job to be concerned about quality. Part of what testers do is to provide accurate information, but it’s also our job to convince the business to act on our information.

      see more detail in my response to Ilya. In my experience, Testers have both decelerated and accelerated the achievement of shippable quality.

  3. Not sure why you would have the tester’s perspective to stop at shippable quality. I’d rather say that testing skills aim to make user adoption and benefits realization more achievable and, for lack of a better metaphor, smoother.

    A crux (not sure if it’s the crux) is that you build the right product, and you can build it right, and still not deliver value to the customer because they can’t use/adopt it for some reason. You didn’t communicate well, and they’re doing something else. You didn’t provide a slick “unboxing” experience for first-time users and they were underwhelmed before really getting into it. In the enterprise world, your acceptance testing used the same business people for every test cycle so they got experienced with the product, missing important first-use experience issues. You had a big gap between the user training and when they actually needed to start using the product…

    A/B testing of a new consumer product would benefit from the tester’s perspective. I’m sure I saw Joshua Kerievsky write on Twitter about how using usage data has helped adjust the product post-release, feeding back into the development cycle. Testing.

    I suggest this continuous development and lean startup stuff is going to extend the utility of testing skills into the post-shipped world. it will show up as the need for a test strategy that isn’t for this release, but forever. User’s actual usage as test results on an ongoing basis.

    • I often use the term “frictionless” where you use smoother, if that helps any. I’m 100% with you on A/B testing. Access to real-time customer behaviour information is a relatively new and wonderful phenomenon. It will and should change “testing” forevermore.

  4. Pingback: There’s no Hope in Test « Testastic

  5. Hi Brent!

    Thank you for an inspiring definition! Until now I felt comfortable with the definition I found in “Lesson Learnt in Software Testing” book written by several test-guru:

    “Test is a service role and its mission is to provide information about the products. This information is used by different functions in different ways: sales needs this information in order to know what they may offer to the customers; development needs this information in order to know if they can build more functionalities on the product code; production needs this information in order to be able to handle the product; management needs this information in order to have a basis on which to make decisions.”

    But I was always missing a big goal in this definition. After all, “find information” feels like an important but borrowing task 🙂

    The only one thing I would like to comment in your definition is: wouldn’t it be better to exchange word “testers” with word “test”? From my point of view testers are not the only one who should work with test. There are parts of test activities which we may move to others and cooperate with them, e.g. checking (http://www.developsense.com/blog/2009/08/testing-vs-checking/, from my point of view checking is a test activity) may be done by others especially when Acceptance Test Driven Development is applied. Unit Test, which is traditionally done by developers, also accelerate the achievement of shippable quality if it is done “in the right” way. I’ve worked with developers having good unit tests on place, not only checking every component but also acceptance criteria (or business-facing tests, or confirmation criteria – different names for the same thing – tests you write together with the customers in their domain language). Testing of this code was sexy :), the system we delivered was one of the most stable systems in our productions and enjoyed by our customers.

    To summarize: What do you think about the definition:

    “Test role is to accelerate the achievement of shippable quality”.

    When I, as a tester, will act according to it as I work with test.

    Regards,

    //Liza

    • Hi Liza,

      I adore comments, so thank you for doing so. I’m glad you found value in this definition. I think there is also value in other definition as well.

      If you haven’t already done so, I would encourage you to read Test is a four letter word (https://testastic.wordpress.com/2012/04/30/test-is-a-four-letter-word/)

      One challenge with switching the term from Testerto Testis that, in my beleif, there’s no common definition for Test. Test in your company is likely different than at mine.

      When I originally wrote this article, I was thinking about what guidance to give the folks who’s title is Tester. Another person commented to the effect of “everyone is supposed to do that”.

      I would agree. But, Testers are supposed to specialize in it. Just as, Testers often write code, but development specialize in that skill.

      So in one sense, yes, I would agree it should state “Test Role”. But if you are a Tester fulfilling the Test Role, my message to you is you should be going deep in that skill. This is what you are being paid to master. You should be better at it than Dev and PM.

      Cheers,
      -BJ

  6. Pingback: All your Leftover are belong to Test | Testastic

  7. Pingback: What would a Leader do? | Testastic

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s