Row, row, row your boat,
Gently down up the stream.
Merrily, merrily, merrily, merrily,
Life is but a dream.
Last week, I wrote about a phenomenon that I’ve noticed in the QA industry regarding our ability to move quality upstream or more precisely, our inability. Great thanks to fellow blogger, Phil Kirkham, whose comment led me to the term that describes it all: “Risk Compensation”.
We’ve all seen charts like this:
We all know that the cost of fixing a bug goes up huge as it travels down a stream. This is usually the chart we show younger testers to inspire them to “move quality upstream”. Note, though, that quality is *not* mentioned in the chart. This is because it’s assumed true that we know exactly what level of quality is needed throughout the entire software development cycle.
Really all this chart says is “if there is an important invalid assumption in your project code, it is better to find that sooner than later.”
But do we do that?
I wish I had access to a study that showed this same chart over time on the same team. (Seriously, if you have something like this, please send it my way. Even better would be something measuring unit of production over cost of production over time)
I suspect that all we have done is increased the cost of a project for the entire period. (Imagine the same chart shifted up one or two lines) We do more and more testing up front, but still do more and more testing in the middle and the end. I’ve seen, on some teams, the cost to actually go up much, much more on the back end of a project. They interpret “moving quality upstream” to mean we need more automation earlier. Sadly, they end up spending their time and money trying to maintain an increasingly burdensome automation suite meant to validate a static set of assumptions that are increasingly proving to be non-static.
In teams like this, your testers become full time automation maintainers and in the worst of cases, the actual testing of coverage gaps becomes a task for the customer (which is a sign, by some, that quality slipped downstream again).
So what is Quality?
So if we are going to move it, we need to know what it is. My view: the quality of your product is the aggregated subjective opinion your customers have of your product (more details will be the subject of a future post). If your product works, works the way your customers expect, and makes them happy, that’s higher quality than the reverse. The point: the quality of your product is determined by the customers of your product.
Where are we now?
In order to move something forward, we need to get a baseline of where we are at now. Since quality is determined by our customers, this means we need to get it in their hands. I find the modern techniques of constructing a Minimum Viable Product combined with plenty of instrumentation intriguing. The trick is to Fail Fast whenever the customer is behaving differently from our expectations. Then take the data that we have and Learn Fast to discover what it is we need to change.
Take a step forward
Winding this back to Risk Compensation, moving forward is often a sociological challenge, not a technical one.
What we want to do is Recover Fast and get a new version of the product in the customer’s hand in order to start the loop over. But to do this we need to change the minds of people:
1) Test to choose to focus more on understanding the depth and breadth of your customer’s concerns
2) Dev to choose to keep quality high WITHOUT relying on the safety net
3) Customer to choose to use your product more and more and pay more and more for it.
If you haven’t figured it out yet, the idea of micro-releases really appeals to me. Faster, smaller release cycles make it harder to lower the quality through bug churn. More importantly, we get more rapid feedback from the actual measurer of quality in the production line, The Customer.
IMHO, no one can sustainably ship garbage to their customer. They will either change their mind or go out of business.
A hodge podge of other ideas came to mind through the process of writing this post. I will share those next time.