“Test doesn’t understand the customer”

One thing I learned long ago is: if you want to make a breakthrough, you’ve got to try to look at the problem multiple ways.  Get different perspectives.   This is one reason why I am very fond of mentor relationships.   I currently have 4 mentors and about twice that number of mentees.   Maybe I’ll talk to their importance in some future blog, but needless to say, these folks are invaluable.    Especially, those who did not grow their careers following the same path, I did.

This is true of Mr. R.  He was an executive who had ascended via the Program Management path.  I had asked him to be my mentor because he had a shrewd eye for managing the business of software.  Something I was looking to understand more of and to master.  At the time, my title was Test Manager, which is essentially a middle manager role between the executives and frontline managers.  Mr. R’s role was to manage a Product Business.   This meant engineering, documentation, marketing, etc.

On the first day of our mentorship, I asked his opinion around something I had casually observed at Microsoft.   What I did not expect was how fundamentally his answer would shake my world.

Me:  “…In addition, given your experience, there’s another pattern I’ve observed that I’d love your opinion on.  I’ve noticed that around 15 out of 20 folks in your role got there via Program Management, about 4 via development, and around 1 out of 20 via test.”

Mr. R: “I don’t know if that’s totally accurate, but seems about right.”

Me:  “So why do you think test so rarely gets the role?”

Mr. R: “Oh, That’s easy!  It’s because Test doesn’t understand the customer.”

I remember the impact of those words.  Particularly, “That’s easy”.   Mr. R did not even pause to consider.  He knew that was the correct answer and he knew it confidently.

I have a reputation of not being afraid of any 1:1 debate and what this guy just told me would have been fighting words in just about any other situation.  Test is the damned Customer Defender, after all.   I flipped the Bozo bit.  I thought: “Well, I guess I’m going to have to find someone else to be my mentor for this.  This guy’s clueless.”  Thankfully, the problem solver side of me still didn’t have closure.   “Can’t have him telling everyone Test doesn’t understand the customer.  That’s not good for the Test discipline.  Let’s correct his thinking and go get lunch”, my thoughts continued.

But…

I couldn’t.

Nothing was coming to mind.  What did test, as a whole, do to understand the customer?  I wasn’t suffering Thinker’s Block. I simply was unable to come up with any argument to disprove him.

I thanked Mr. R for his time and said that I needed to think some more on this.

And think I did…   For the next several weeks.

I examined the activities that Testers did.  Test cases, specs, automation, bugs, etc.   Failing to find anything really relevant, I told several peer Test Managers my story and asked them the same question.   It was like I had just told them there was no Santa Claus.  Some fought.  And hard, but eventually they all came to the same conclusion:  Test had wrapped itself up in the Emperor’s Clothes.

So the “Test Thinktank” worked the problem.  We discovered there were 3 places where Test had a touchpoint with the customer:

1. Indirectly through PM’s feature requirements
2. Vaguely through trolling newsgroups.  Though, this usually resulted in aggregation of complaints.  Ie. Understanding what it takes to stop irritating the customer.
3. Actually interacting with the customer

Points #2 and #3, we noticed happened rarely.   Ironically, Testers were often too busy Testing.   Testers who were especially good at interacting with the customer were often considered misplaced and quickly moved to the PM organization.  Testers who weren’t good at it would be uninvited from future events.  This would be due to the natural tester mindset.  Testers, who were very good at finding things wrong, would find themselves sharing that info with the customers.   In general, it’s considered a Bad Business idea to send your Engineering team’s resident pessimists to the people who write the check.

Another thing has happened to me recently as well.  Those of you who find this blog via my LinkedIn profile are already aware that while my soul belongs to Test, my title no longer does.  I’m in a frontline Dev management role.

I feel like a sheep in wolves clothing.

 

I’m doing dev work, but I feel more like a SuperTester, than the Developer I expected to feel like.  I don’t know if it is because of all of my Test training or because I don’t have a Test team assigned to me as a partner, but I spend a *lot* of time thinking about quality.   A lot more than I did in test.  I also wonder what I would do differently if I had a test team as a partner.    I’m pretty sure I don’t want one right now.

There are a couple of reasons why:

1. The Burden of Accountability & Responsibility is about 3 times stronger.   The mother of the baby feels a lot more responsible than the midwife to make sure the child becomes a responsible adult.   I felt really accountable in Test, but it is just stronger when you know that you and your team is writing the code that will affect customers.
2. I feel closer to the customer than ever before.  Based on prior experience with other Dev managers, I don’t think Dev, in general, understands how good they got it.   They are directly providing quality to the customer.  I… Am… Actually… Delivering… Quality…   It’s fantastic.  In addition, I’ve really discovered that I don’t like my team writing code that the customer doesn’t use.   It’s a waste.   This forces me to understand my customer goals and intent, fail fast, and quickly iterate, so I can maximize the value my team is providing.  Non-theoretically.
3. I do not want my team getting sloppy and relying on a crutch.  I think far too many Test/Dev pairings end up with Test cleaning up after Dev’s messes; not about effectively partnering to maximize customer value.  It’s an easy pattern to fall back to.   I want my team mastering the ability to produce excellent designs/code on their own first (producing fewer features, if necessary).

As I sit back now, it’s still hard for me to disprove Mr. R.  My own experience is that I actually understand what Quality means to my Customer better than ever before.  I understand their intent and desires better than ever before.  My hope for today’s post is that you simply consider for yourself if you actually understand your customer.

There’s a motto that I use frequently on my team: “We don’t know what we’re doing”.  I’ve communicated to my team that the motto’s intent is to remind each and every one of us that until we verified with the customer that we delivered something they wanted.  Our understanding of the customer is simply an assumption we should constantly try to clarify.

Beware the JeDI

Last Friday, I had a meeting with a boss of mine from long, long ago.   Currently, he is one of the Directors of Test for Windows, which is a stronghold of waterfall.   He wanted my opinion on how to use what I’ve learned in my recent job positions.   These are teams that have deployed Agile techniques and have begun the march towards reducing headcount in Test Generalist positions.    So he wanted me to bring him up to speed.

In retrospect, I wish I had done my best spooky voice and stated:

“Beware the JeDI”

Over the years, I have personally witnessed several teams struggle to rollout Agile.  After doing my own analysis of common symptoms and behaviors, I have created a simple litmus test you can use to determine which your team is executing.  You may be surprised.  The first step to using it is to observe your team’s behaviors and ask: what is the question that these behaviors are trying to answer?

A Waterfall team asks: “Is the plan on track?”

An Agile team asks: “Is the plan correct?”

A JeDI team asks: “Plan? Just Do It!”

In Waterfall, the team is executing on a plan and getting the code out to market in accordance to the plan is pivotal for all of the synchronized machinations to work properly.   (Marketing, Sales, Customer Service, Hot Fix Engineering, etc.)

In Agile, the team has a baseline plan, but focused on shipping small bits of the code to the customer with the intent of determining if the plan is the right direction.   Based on observing the customer, the plan will change and a new direction set.  (Note:  I adore Elisabeth Hendrickson’s definition of Agile here)

A JeDI team believes they are Agile and each individual works 16 to 20 hr days in order to stay on track.    They think this is just a reality of software engineering.   Scaling way back on planning and documentation, they are heavily focused on “Code Velocity”, and their customer is someone they believe they will delight, but their view of the customer is theoretical for the most part.   When they actually observe customer behavior though, the runway is too short for them to make changes.  In these teams, Test Generalists are very often reactive to development.   In a world where “Code Velocity” is king, partnering with Test is an afterthought.  In some cases, I’ve even seen a total lack of documentation…  If it were constructed, that might help Test solve its own problems. But no.  There’s none.   It’s not dissimilar to Cowboy Development.

In a previous post, I was asked to explain why I didn’t recommend the Agile Manifesto.   It is simply this:

The Agile Manifesto creates JeDI teams.

 

 

On JeDI teams, there are 3 common characteristics I have noticed of the people driving the teams:

1. They are all wicked smart.  Often the best at what they do.
2. They are good at making things efficient
3. They are very focused on getting stuff done… and yesterday.

When combined with the Agile Manifesto, I’ve seen practices created that end up (ironically) being highly dysfunctional to the project overall.

Consider this:

• These folks like things simple.  Black & white is better than shades of grey.
• They will tend to use only the Agile Manifesto to understand Agile.  And there, only the 4 Values, not the 12 principles.  (In fact, I’ve had conversations with strong admitted Manifesto supporters who did not even know of the principles.)
• These folks will take the 4 values and oversimplify to:
  • Agile == Low Overhead  AND/OR
  • Agile == Flexibility
• They will roll out their new “Agile” program.  Unchecked, Low Overhead and Flexibility quickly becomes: “I get to do what I want (flexibility) without being accountable to management (low overhead). “

Once that has taken hold in your team, it is really, really hard to fix it (perhaps, a blog for another day).  In my experience, some key value gets subtracted from the Manifesto:

• The principle of Customer collaboration gets devalued.   Working with the customer “hand-in-hand” is important to succeed with Agile.   Learning your customer iteratively.
• “Responding to Change over following a plan”  is, IMHO, intended to mean Fail Fast and change the plan based on what you have learned, not to execute without a plan.

Are you in a similar situation and trying to understand Agile?

Here’s my advice: (modified from a George Paci quote here)

“[Treat Agile] like an off-the-rack suit: it’s not likely to fit you perfectly, but you’d better try it on before you make alterations.”

The best way to avoid the JeDI is to prevent your team from becoming one in the first place.  The Agile Manifesto does not tell you what to do.   It tells you what to value.  It’s a definition, not an implementation plan.  Taking its values to excess will cause dysfunction on your team.  Heed my warning.

Instead of starting with the Manifesto, implement the variant of Agile that most fits your teams skills, resources, and needs.   I think Alan Shalloway and his team has a good resource you can use.  (I’ve sent Alan a note asking.   If I get it or find another resource for you, I’ll update this post). [Edit: And he did! (thanks, Alan) He recommends this link (Overview) and this one  (additional resources) if you want to understand more]   The three most widely used variants to my knowledge are XP, SCRUM, and Lean.   My personal preference is Lean with Kanban.

Whichever you choose, implement it exactly as intended from the sources.  Once you have experienced it firsthand, then feel free to make changes.  Agile *is* about learning, after all.

Once you have tried it out, the Manifest will become clear and true to you and it is a thing of beauty.