Quality is a 4 letter word too

Recommended prerequisite reading

If this is your first visit to my blog, then I recommend some pre-reading.

 I think of this blog post as the 3^rd part of a series.  I suggest reading “Test doesn’t understand the customer” first.   There I talk about how testers may have a glass ceiling in their careers because they are too far removed from Quality.   I then recommend reading TEST is a four letter word.   In that post, I bemoan the tendency of the software industry to overload the term TEST and thereby render it meaningless. 

Intro

Since I wrote that last article, I’ve been in a couple of heated discussions about Quality at work.   I was invited to a discussion of Test Leadership who was strategizing over how to best improve Quality.   When it occurred to me, there are a few topics where you can get 10 Testers into a room and come out with 11 definitions.   Quality is definitely one of these.

Based on these experiences and some alone-time spent pondering, I have come to grips with the idea that Quality has a problem.  It is a complex idea that is getting oversimplified.  It has occurred to me that Quality may also be a 4 letter word. 

By the end of this tongue-in-cheek article, I hope to convince you of this and inspire some to think a bit more deeply on the topic.

Quantity of Quality?

For those who would choose to believe that Quality is, in fact, a 7 letter word, I empathize.  Actually, I see this phenomenon quite often in test.  Testers are very good at figuring out metrics and claiming that these counts represent Quality.

They don’t.

 I started my career firmly believing this as well.   Even today, I still gravitate towards roles heavy in Beancounting, but I no longer view them as measuring quality.

Popular Metrics:

•          Tests Created
•          Tests Automated (or percentage)
•          Tests executed (or pass rate)
•          Bugs found (or Bugs found per tester)
•          Code Coverage percentage

In my experience, metrics of this sort serve to do one thing and one thing only.  They seek to modify behavior of the test team.  This is due to the Hawthorne Effect.  Peoples’ behavior change in accordance to how they are being measured.  As an experienced manager, I know the ones I chose above, if used “right”, will make sure more and more tests are being produced, but they tell me nothing of the quality of the code or product.

I offer you these words of advice:

                It may be important to your business to count metrics, but if you can count or measure it, it is called “Quantity”, not “Quality”.

FFFF

So what should we consider when thinking about Quality?  First off, Quality is the aggregated subjective point of view of your customers.  But how do we evaluate that?  What characteristics can we look at/for?   I offer you a 4 letter acronym to help, and as an added bonus, each word within it is an F-word.   Quality needs all of these things.  (characteristics, not F-words)

Features

First thing to consider are the features of the product.  Does it do what your customer expects?

Consider: A plain pinewood chair constructed without a seat.   While it is clearly recognizable as a chair, it’s going to have a low quality simply because with no seat, you cannot sit in it.

Faultlessness

Is the product working correctly?  Does the product do things your customer wished it didn’t do?

Consider: The same chair above with a seat, but now the front right leg is too short by 5 inches.   While you can sit in it now, if you lean to far forward, the chair will crash taking you with it.   It won’t be long before the irritation causes the high maintenance chair to be discarded.

Finish

How polished is the final product?  Is it desirable?  Does it feel special?

Consider: Now think of a plain pinewood chair as well as one made by artisans.  Think of one that was hand carved into hardwoods, polished to a sheen, comfortable cushions, etc.  It’s a thing of beauty and comfort.  It’s quality is higher.

Frame of Reference

Is the context right?  Is it relevant?  Does it do what the customer wants? This *might* just be the most important characteristic of Quality.  In my experience, getting the context wrong can really place you at odds with the customer.

Consider: This time the chair to consider is a toilet.   When it is placed in a restroom, it is viewed commonplace enough and no big deal.  But… Move it to the center of kitchen, and its context is dramatically out of place.  To some, disturbingly so.  Wouldn’t call that quality?  Now let’s say that same kitchen/toilet combo is part of a domicile located in a remote village in Africa… It suddenly isn’t so bad any more.  The quality feels improved.

Consider also: consider also my original chair (the one without a seat), what happens in your mind if I say that chair is actually a picture frame – I place a picture where the seat should be and hang the chair up in the ceiling?  By claiming its Art, I’ve changed the chair’s frame of reference and changed how quality will be perceived.

Consider lastly: My post today is titled “Quality is a 4 letter word too”.  Given, I started from a rant and I am following up from another article of similar intent, I might argue I have set things up so that the context of this blog title is more important that the correctness of the # of letters in the word.  Because of this context change, I argue today’s title is a quality one.  (but it’s subjective…  you may not agree)

This last item is an important item to consider when in test.   It implies there may be times in your career where the context of the feature is such that correctness may not matter that much…

What about Cost?

I was explaining this to one of my mentees the other day and he asked “Isn’t there a 5th? What about cost?”  To me, cost is not part of the quality equation.   Rather, both Cost and Quality are part of the Value equation.    Both Quality and Value are subjective, but basically, Value equals (figuratively) Quality divided by Cost.   Cost, imho, isn’t just money, btw, but barrier to entry.   A website that takes 15 minutes to render is likely too expensive (time is money) to be valueable in my way of thinking.

Closing

It is my hope that I helped give you some other things to consider when/if you are pushing to achieve higher quality.   While it is a part of it, quality is not simply the features, or the bugs you find.   It is the correct balance of several characteristics that, working together, will delight your customer.

By the way, there is another 4 letter word to describe quality, it’s QWAN.  Since this article is getting on the longer side of things, I will save that for my next post.

“Test doesn’t understand the customer”

One thing I learned long ago is: if you want to make a breakthrough, you’ve got to try to look at the problem multiple ways.  Get different perspectives.   This is one reason why I am very fond of mentor relationships.   I currently have 4 mentors and about twice that number of mentees.   Maybe I’ll talk to their importance in some future blog, but needless to say, these folks are invaluable.    Especially, those who did not grow their careers following the same path, I did.

This is true of Mr. R.  He was an executive who had ascended via the Program Management path.  I had asked him to be my mentor because he had a shrewd eye for managing the business of software.  Something I was looking to understand more of and to master.  At the time, my title was Test Manager, which is essentially a middle manager role between the executives and frontline managers.  Mr. R’s role was to manage a Product Business.   This meant engineering, documentation, marketing, etc.

On the first day of our mentorship, I asked his opinion around something I had casually observed at Microsoft.   What I did not expect was how fundamentally his answer would shake my world.

Me:  “…In addition, given your experience, there’s another pattern I’ve observed that I’d love your opinion on.  I’ve noticed that around 15 out of 20 folks in your role got there via Program Management, about 4 via development, and around 1 out of 20 via test.”

Mr. R: “I don’t know if that’s totally accurate, but seems about right.”

Me:  “So why do you think test so rarely gets the role?”

Mr. R: “Oh, That’s easy!  It’s because Test doesn’t understand the customer.”

I remember the impact of those words.  Particularly, “That’s easy”.   Mr. R did not even pause to consider.  He knew that was the correct answer and he knew it confidently.

I have a reputation of not being afraid of any 1:1 debate and what this guy just told me would have been fighting words in just about any other situation.  Test is the damned Customer Defender, after all.   I flipped the Bozo bit.  I thought: “Well, I guess I’m going to have to find someone else to be my mentor for this.  This guy’s clueless.”  Thankfully, the problem solver side of me still didn’t have closure.   “Can’t have him telling everyone Test doesn’t understand the customer.  That’s not good for the Test discipline.  Let’s correct his thinking and go get lunch”, my thoughts continued.

But…

I couldn’t.

Nothing was coming to mind.  What did test, as a whole, do to understand the customer?  I wasn’t suffering Thinker’s Block. I simply was unable to come up with any argument to disprove him.

I thanked Mr. R for his time and said that I needed to think some more on this.

And think I did…   For the next several weeks.

I examined the activities that Testers did.  Test cases, specs, automation, bugs, etc.   Failing to find anything really relevant, I told several peer Test Managers my story and asked them the same question.   It was like I had just told them there was no Santa Claus.  Some fought.  And hard, but eventually they all came to the same conclusion:  Test had wrapped itself up in the Emperor’s Clothes.

So the “Test Thinktank” worked the problem.  We discovered there were 3 places where Test had a touchpoint with the customer:

1. Indirectly through PM’s feature requirements
2. Vaguely through trolling newsgroups.  Though, this usually resulted in aggregation of complaints.  Ie. Understanding what it takes to stop irritating the customer.
3. Actually interacting with the customer

Points #2 and #3, we noticed happened rarely.   Ironically, Testers were often too busy Testing.   Testers who were especially good at interacting with the customer were often considered misplaced and quickly moved to the PM organization.  Testers who weren’t good at it would be uninvited from future events.  This would be due to the natural tester mindset.  Testers, who were very good at finding things wrong, would find themselves sharing that info with the customers.   In general, it’s considered a Bad Business idea to send your Engineering team’s resident pessimists to the people who write the check.

Another thing has happened to me recently as well.  Those of you who find this blog via my LinkedIn profile are already aware that while my soul belongs to Test, my title no longer does.  I’m in a frontline Dev management role.

I feel like a sheep in wolves clothing.

 

I’m doing dev work, but I feel more like a SuperTester, than the Developer I expected to feel like.  I don’t know if it is because of all of my Test training or because I don’t have a Test team assigned to me as a partner, but I spend a *lot* of time thinking about quality.   A lot more than I did in test.  I also wonder what I would do differently if I had a test team as a partner.    I’m pretty sure I don’t want one right now.

There are a couple of reasons why:

1. The Burden of Accountability & Responsibility is about 3 times stronger.   The mother of the baby feels a lot more responsible than the midwife to make sure the child becomes a responsible adult.   I felt really accountable in Test, but it is just stronger when you know that you and your team is writing the code that will affect customers.
2. I feel closer to the customer than ever before.  Based on prior experience with other Dev managers, I don’t think Dev, in general, understands how good they got it.   They are directly providing quality to the customer.  I… Am… Actually… Delivering… Quality…   It’s fantastic.  In addition, I’ve really discovered that I don’t like my team writing code that the customer doesn’t use.   It’s a waste.   This forces me to understand my customer goals and intent, fail fast, and quickly iterate, so I can maximize the value my team is providing.  Non-theoretically.
3. I do not want my team getting sloppy and relying on a crutch.  I think far too many Test/Dev pairings end up with Test cleaning up after Dev’s messes; not about effectively partnering to maximize customer value.  It’s an easy pattern to fall back to.   I want my team mastering the ability to produce excellent designs/code on their own first (producing fewer features, if necessary).

As I sit back now, it’s still hard for me to disprove Mr. R.  My own experience is that I actually understand what Quality means to my Customer better than ever before.  I understand their intent and desires better than ever before.  My hope for today’s post is that you simply consider for yourself if you actually understand your customer.

There’s a motto that I use frequently on my team: “We don’t know what we’re doing”.  I’ve communicated to my team that the motto’s intent is to remind each and every one of us that until we verified with the customer that we delivered something they wanted.  Our understanding of the customer is simply an assumption we should constantly try to clarify.