Recommended prerequisite reading
If this is your first visit to my blog, then I recommend some pre-reading.
I think of this blog post as the 3rd part of a series. I suggest reading “Test doesn’t understand the customer” first. There I talk about how testers may have a glass ceiling in their careers because they are too far removed from Quality. I then recommend reading TEST is a four letter word. In that post, I bemoan the tendency of the software industry to overload the term TEST and thereby render it meaningless.
Since I wrote that last article, I’ve been in a couple of heated discussions about Quality at work. I was invited to a discussion of Test Leadership who was strategizing over how to best improve Quality. When it occurred to me, there are a few topics where you can get 10 Testers into a room and come out with 11 definitions. Quality is definitely one of these.
Based on these experiences and some alone-time spent pondering, I have come to grips with the idea that Quality has a problem. It is a complex idea that is getting oversimplified. It has occurred to me that Quality may also be a 4 letter word.
By the end of this tongue-in-cheek article, I hope to convince you of this and inspire some to think a bit more deeply on the topic.
Quantity of Quality?
For those who would choose to believe that Quality is, in fact, a 7 letter word, I empathize. Actually, I see this phenomenon quite often in test. Testers are very good at figuring out metrics and claiming that these counts represent Quality.
I started my career firmly believing this as well. Even today, I still gravitate towards roles heavy in Beancounting, but I no longer view them as measuring quality.
- Tests Created
- Tests Automated (or percentage)
- Tests executed (or pass rate)
- Bugs found (or Bugs found per tester)
- Code Coverage percentage
In my experience, metrics of this sort serve to do one thing and one thing only. They seek to modify behavior of the test team. This is due to the Hawthorne Effect. Peoples’ behavior change in accordance to how they are being measured. As an experienced manager, I know the ones I chose above, if used “right”, will make sure more and more tests are being produced, but they tell me nothing of the quality of the code or product.
I offer you these words of advice:
It may be important to your business to count metrics, but if you can count or measure it, it is called “Quantity”, not “Quality”.
So what should we consider when thinking about Quality? First off, Quality is the aggregated subjective point of view of your customers. But how do we evaluate that? What characteristics can we look at/for? I offer you a 4 letter acronym to help, and as an added bonus, each word within it is an F-word. Quality needs all of these things. (characteristics, not F-words)
First thing to consider are the features of the product. Does it do what your customer expects?
Consider: A plain pinewood chair constructed without a seat. While it is clearly recognizable as a chair, it’s going to have a low quality simply because with no seat, you cannot sit in it.
Is the product working correctly? Does the product do things your customer wished it didn’t do?
Consider: The same chair above with a seat, but now the front right leg is too short by 5 inches. While you can sit in it now, if you lean to far forward, the chair will crash taking you with it. It won’t be long before the irritation causes the high maintenance chair to be discarded.
How polished is the final product? Is it desirable? Does it feel special?
Consider: Now think of a plain pinewood chair as well as one made by artisans. Think of one that was hand carved into hardwoods, polished to a sheen, comfortable cushions, etc. It’s a thing of beauty and comfort. It’s quality is higher.
Frame of Reference
Is the context right? Is it relevant? Does it do what the customer wants? This *might* just be the most important characteristic of Quality. In my experience, getting the context wrong can really place you at odds with the customer.
Consider: This time the chair to consider is a toilet. When it is placed in a restroom, it is viewed commonplace enough and no big deal. But… Move it to the center of kitchen, and its context is dramatically out of place. To some, disturbingly so. Wouldn’t call that quality? Now let’s say that same kitchen/toilet combo is part of a domicile located in a remote village in Africa… It suddenly isn’t so bad any more. The quality feels improved.
Consider also: consider also my original chair (the one without a seat), what happens in your mind if I say that chair is actually a picture frame – I place a picture where the seat should be and hang the chair up in the ceiling? By claiming its Art, I’ve changed the chair’s frame of reference and changed how quality will be perceived.
Consider lastly: My post today is titled “Quality is a 4 letter word too”. Given, I started from a rant and I am following up from another article of similar intent, I might argue I have set things up so that the context of this blog title is more important that the correctness of the # of letters in the word. Because of this context change, I argue today’s title is a quality one. (but it’s subjective… you may not agree)
This last item is an important item to consider when in test. It implies there may be times in your career where the context of the feature is such that correctness may not matter that much…
What about Cost?
I was explaining this to one of my mentees the other day and he asked “Isn’t there a 5th? What about cost?” To me, cost is not part of the quality equation. Rather, both Cost and Quality are part of the Value equation. Both Quality and Value are subjective, but basically, Value equals (figuratively) Quality divided by Cost. Cost, imho, isn’t just money, btw, but barrier to entry. A website that takes 15 minutes to render is likely too expensive (time is money) to be valueable in my way of thinking.
It is my hope that I helped give you some other things to consider when/if you are pushing to achieve higher quality. While it is a part of it, quality is not simply the features, or the bugs you find. It is the correct balance of several characteristics that, working together, will delight your customer.
By the way, there is another 4 letter word to describe quality, it’s QWAN. Since this article is getting on the longer side of things, I will save that for my next post.