User Stories and 5 Why Analysis

 

Today I stumbled upon a very useful technique I felt warranted sharing. I am in the process of drafting a blog around the frailty of introducing Agile techniques to a waterfall team. I’m confident I’ll post that either this weekend or next. In the meantime, one thing I have learned from you, the blog reader, is that you tend to appreciate and forward on useful techniques to others when they are practical solutions to present day problems. May this help you.

 

My love/hate relationship with User Stories

 

There are a lot of great User Story resources. My absolute favorite is Mike Cohn’s site, here. The term “User Stories” is often overloaded, so I recommend visiting Mike’s site to make sense of the rest of the post. My view is compatible with his (if not a blatant overlap).

One of the first things I try to do when I am helping a team shift to Agile is to teach them to stop scheduling workitems and, instead, start delivering outcomes. This approach enables teams to decouple the problem they are trying to solve from the solution they currently favor. There a lot of benefits to doing this. Not in the least is the ability to define DONE in terms of value added to the customer. There are other techniques to defining Done up front (such as ATDD), but I have found that User Stories is really palatable to newbies trying out Agile.

But even User Stories have problems. They can be hard to construct. When I know I need, say, a new report created, it is just easier to write “Create new report” on the ticket and place it on the task board in the backlog. They can be rather verbose and hard to communicate succinctly. “As a decision maker for the release, I want to see fresh execution reports online, so that I can weigh-in on readiness armed with the right data to make an informed choice“. The friction created in the longer format makes the shorter format very alluring. Especially for those who haven’t yet experienced the value of the longer version.

 

The problem

For folks from waterfall world, they get the workitem approach. “Create new report” seems easy to understand and execute on. Today’s problem came from one of the testers on the team, who quite honestly, had gotten tired of me complaining about Scope Creep in their “stories”. As the Agile Master for the team, I push hard on making sure we are maintaining a high, consistent and predictable velocity. Scope Creep makes this very difficult, causes delays, and creates the potential of significant waste of effort incurring within the system. My team is using Lean and Kanban. We do not timebox our iterations, but each story has a 2 week SLA. The story this tester was working on was about some tooling we are creating. They were coming to me to let me know that “the design has changed again” and wanted to know what to do about it. The ticket was already passed its 2 week SLA.
In addition, the ticket was similar to “enable performance thresholds”. IE. It was ambiguously worded and it was entirely unclear when we would be done. I had warned of this before, but my style of Agile Mastering is to let teams make the decision and the mistake in order to enable learning, so I let it stick.

The solution

It is insufficient to point out what not to do. If you want folks to learn, tell them what to do instead. Here I suggested that the problem with the ticket was that Done was not clear. I said use a User Story instead… now as well as in the future. This particular tester had a hard problem with that. Even after explaining Stories. They could not pivot the workitem into a story. It was just unnatural for them to think of the outcome they needed. Unfortunately, this a far too common experience for me.

However, during a rare flash of total insight, I fixed this for them, by throwing in SixSigma’s Why Analysis technique. Primarily used to determine root causes of things, basically, the way it works is you simply ask ‘why’ 5 times.

The dialogue

I started off: “Ok, let’s try a different thought process. What if I were to tell you I see no value in this “enable performance thresholds” task, so I am going to cut it. How do you feel about that?”

Tester: ” I hate that.”

Me: “Why?”

Tester: “Because we need it”

Me: “Why?”

Tester: “Because dev needs it”

Me: “Why?”

Tester: “So they can decide if the product is good or not”

Me: “so what you are saying is that ‘as a dev on this team, you want performance thresholds enabled, so that you can decide if the product is good or not’?”

Tester: “yes”

Me: <blank stare>

Tester: “ooooohhhhh!!!!!”

 

We then talked about how adding additional why’s adds precision to the outcome desired and clarity around Done, while in most cases, keeping the implementation decoupled from the outcome. In addition, we talked about how to determine when the story is too vague (likely an Epic) and needing to be broken down into smaller stories.

I will see how well it plays out over the coming weeks, but the tester, at least, believed they would be able to confidently break stories into smaller outcome-based stories and with that defend against scope creep while still handling undiscovered stories in an Agile fashion.

 

 

Kanban for Chores

Today’s post is a lot more practical than most. It’s fun to mix things up now and again. I really enjoy it when work related activities can improve the home. I feel like the time invested in learning pays off doubly so in those cases.

Today I am going to share a new chore system that has been rolled out in the Jensen family. So far, I’m very happy with the results. To give credit where credit is due: months ago, in one of Alan Page’s blog posts, he told a story about how he introduced his family to Kanban. His kids, in particular, really seemed to dig it. I would share out the link to the post, but, sadly, I cannot find it. I think it lies in one of Alan’s older posts. If somehow it shows up, I will post an update here. (EDIT: as pointed out in comments, it was a series of tweets) As I recall, Alan’s system was to put the chores up on a kanban-related taskboard and everyone works together to get the chores done on the weekend before heading out for family fun.

As you may recall from my last post, I have moved recently. The new house is much bigger than the old one and we have an active after work/school life, so chores were going to the wayside at times. This, to me, was annoying. I remember Alan’s post and said “hmm, that actually sounds like fun and with a few tweaks should work great for our household”.

Here’s how I did mine.

First, I acquired several supplies:

1. A 4′ Magnetic Dry Erase Board
2. A set of Dry Erase Pens (Although, I will probably change this to Painter’s Tape later… It looks cleaner.)
3. 4 sets of Planning Poker cards by using the PDF available here.
4. A set of Ink Jet Magnetic Business Cards by Avery

The board will serve as a base for the taskboard. Since both the cards and the board are magnetic, the cards will be a perfect medium for being task tickets. The dry erase pens will mark out the columns, WIP, and flow.

Next, I sat with my wife and we worked through the chores that we felt should/could be done by anyone in the household. Things like doing the dishes, taking out the trash, cleaning your room, etc. are not on the list. These chores we felt the kids should just do anyways… Family Tax. I then took each of those chores and printed it on its own business card. These ended up looking really sharp. Had I the patience or the time, I would have put pictures or decorated the cards so they showed the image of the chore. One of my sons really likes that type of work, so I may ask him to do it when these cards wear out. Plus I find if you get folks to help to work on the system, they feel more vested in its success. Can’t hurt.

At Dinner, I brought out the magnetic cards. This was the first time the kids had seen them and since all of the cards had work items on it that they recognized they were immediately suspicious. I trudged forward fearlessly and walked them and my wife through a variant of planning poker (without them realizing that what I was doing had that name).

Planning poker process we followed:

• Ordered the chore list from easiest to do to hardest
• I picked the one in the middle “Vacuum Downstairs” and announced it was worth 5 points. (My eldest, still suspicious, protested the use of points, claiming that it further proved that doom and gloom was coming his way… Teenagers!)
• I then took all of the chores and put them into a single randomized stack.
• I wrote a “5″ on “Vacuum Downstairs” and kept it out of the stack
• Then starting at the top, I asked “if Vacuum Downstairs is worth 5 points of difficulty, how many points is ___________________?” (for example, “Clean Refrigerator Door” or “Weed Front Yard”)
• I told them how to use the Planning Poker cards I had made and iterated through all of the chores.
• After we agreed on a point total for the chore (as well as got clear agreement on what Done meant), I wrote the final number on the card and went to the next card.

There was some discussion from the kids regarding the available point values, but I stuck to Fibonacci numbers. (they wanted a 4 and a 7)

 

Now I had all of the chores with points assigned, I went and made the simplest of taskboards (see below). Since I wanted the board to replace the current (and totally failing) allowance process, I then figured out the point to $ conversion by figuring out how much I would be willing to pay weekly and dividing that by the total of all of the chore points.

Then I told the kids (and my wife) the rules:

• Only mom and dad can move tickets from the Backlog column to the Ready column. We will do so when we think that chore needs to be done.
• Anybody can grab a chore they want from the Ready column, if they have available capacity.
• Each person can have, at most, 2 tickets in the Doing column. They are not required to have any.
• No one else can take that ticket as long as the owner completes it in 2 days. If they do not, then someone else can take it with notification.
• At the end of the week, each kid will get paid according to the sum of their points.
• Then the Done tickets will be moved to Backlog and my wife and I will re-fill the Ready column as needed.
• Also, at the end of the week, we will as a family talk about the chores and adjust points up or down. I will adjust the point conversation rate as necessary, so that I am paying a constant amount each week.

The Results

Truly amazing. For those who have deployed similar things at work, it really shouldn’t surprise, but it did me. My kids really picked it up excitedly. The first day half of the chores got done. It was kind of a whirlwind. Both kids viewed it as a game and had a blast just moving a ticket to done. They both are eager to get the big ticket items, but when one figured out that he could get the smaller tickets in faster time than the big ones, he quickly surpassed his brother. Right now, it is definitely a situation of everyone wins. Kids are in charge of earning what they want and when. Mom and Dad are in charge of prioritizing the work and the house is staying clean and spiffy and as another positive note an unexpected “bonus” has popped up: The kids do not want Mom and Dad doing tickets. Why? Because then *they* don’t get paid. Brilliant, I say! Might just give me more time for blog writing!

 

 

 

MyTestCareer++;

It has almost been 4 months since I have last blogged. Several big things (big to me, at least) have occurred in that period. The least relevant to this post, but the hardest on my schedule, has been a house move. We are still living 33% of the time out of boxes, but things for my family are slowly getting back to normal at our new location. In addition, the old house has most of its laundry list completed and will appear on the market very soon. <Fingers crossed> It has definitely been a rough adventure. If I wait another 15 years to move again, that will be fine by me.

That isn’t the only change for me. Today’s post, as is often the case, the result of deep thinking of themes that are problematic to me. As example, I wrote my last post when I was thinking through what I wanted to work on next in my career and where I wanted to do it. As a result, I have made a non-trivial decision to move back into a test role (Gory details? check out my LinkedIn profile). Several really great things happened after my last post. I has offered several great Development Lead roles and even a Development Manager role. These all were very interesting and if I hadn’t already committed to the new team, I might have taken that DM job. I definitely still think of it often. Why? Even though I am back in Test now, I suspect the Test discipline is not my final destination.

I have gone back to test because I have some new personal and “organizational” goals to aggressively pursue and I believe the best place to do that and contribute to the delivery of ground-breaking, customer delighting products is in that role.

To explain further, I want to improve lives and solve problems. I want to contribute to building businesses that satisfy a customer need and excel at doing it. In particular, I want to advance the science of producing Quality in the context of that business. This is a strong passion of mine.

In my last year in Development, I had zero testers. My team had to own their testing themselves. As a result, I learned 2 very important things. 1) Far too much of our focus on quality is around code correctness and 2) Testers are *not* needed to deliver improvements there. In fact, Testers who grew up the way I did, might actually be slowing down the product and contributing to a much more expensive bottom line. I think Whittaker is right when he claims “All that testing is getting in the way of quality“.

My developers, without a test team, had to focus on testing things themselves. Since we were an Agile team, we shipped each ticket when it was completed. Every day we were releasing something and every day if someone on my team messed up, the whole team would swarm to fix or revert. We discovered over time, what tests helped, what hurt, what policies protected the customer, and what were just burdensome costs. Over time, we learned to rely *more* on design practices make changes quickly and effortlessly than on costly overtesting. As we focused on our ability to react *quickly* to failure, our ability to prevent failure grew and grew. Our designs became much more cohesive, much less coupled together. Easily detangled and simple to test.

With code correctness managed, this enabled us to turn to quality and customer satisfaction of our feature (link here for what I mean by Quality). This ended up being a much harder and interesting problem. The team I left behind is still aggressively working hard to nail this. They are an Agile team and able to self-optimize for what they learn on a weekly basis. They will get there. I have no doubt of it.

They are working towards having pinpoint accuracy around customer insight and quality for their customer base. They will achieve QWAN.

So why the new team? And why test?

The test org of my new team is going a different direction than most. They are aggressively shifting functional testing over to development and changing their focus to higher value output. They want to create a much, much stronger investment in infrastructure that reduces the friction to high value customer insight and work prioritization. They want to shift to a monthly release cycle and a service orientation.

I went there to help them execute the shift and bring about an age of Data Science and Agility in the organization.

Just found out on Friday that I am going to be the leader for one of the key initiatives needed to make this happen.

My boss told me this along with the phrase “be careful of what you ask for….”

This is going to be one of the biggest tasks I have ever tackled. I do not yet know everything that I and my team will need to do, but it’s going to be very exciting and the final goal very rewarding.

It’s YOUR career. Lead it!

Microsoft’s review model is an odd beast. Some things about it I adore, some things about it I abhor, and some things that just don’t make sense to me. Why *do* we go through our “midyear” discussion in March and then write up our Annual reviews in June (3 mos later), then get our reviews back 3 months later? On which review are we supposed to take credit for the awesome things we did from that June to September? Shouldn’t next year’s goals be written right after we submit our reviews (tweaked when we find out our reviews)?

This post is not about complaining about the review system. No review system is perfect and it turns out the Peter Principle (“people get promoted to the point of incompetency”) is true. It was proven in 2010 by the Nobel Peace prize winners of Management Science (here). Microsoft follows a general “Up or Out” strategy, which is considered a typical “defense” to Peter Principle happening in your org.

But wanna hear something really neat?

These same guys proved that promoting the most competent people from the lower levels up is not as successful of a strategy as alternating between the Best and Worst when promoting. Why? Because being good at working at one level, does not mean you have the skills needed to do what is required at the new higher level.  Some of the “incompetent” folks *actually* have the skills to outperform their peers at the higher levels. If you are a teacher and/or a strategist, you aren’t going to be valued as much as an individual, but if you can land a leadership role where you have to coach folks, you can nail it.

Let that soak in. Alternating between best and least competent people is a *more* successful strategy to improve the overall corporate efficiency.

That tweaks my noodle a bit, but I’ve seen situations like this. Years ago, I told an employee (who had zero passion for testing) that I would give him 30 days to find a new job in Microsoft or I would accept his resignation. I also told him if he applied for a test role that I would not give his new manager my recommendation, but a PM role, he would have it 100%! With that motivation in hand, he did land a PM job… The next several reviews he received were stellar.

He was now competent in the position he was in.

This time of year is typically where all employees are done hearing their reviews (I hope so!!!) and have written up their goals for next year. People should now be looking ahead figuring out how to maximize their career growth and build solid value into the products their team is producing. It is common for me to get my opinion asked for by employees or mentees about how to get a better review score or get promoted.

My answer is personalized for each individual, but in general, I let people know first and foremost, they need to own their own career. I can help them understand how the game is played, but ultimately, they are the ones who must do the work.

Usually, people who ask me this question is because they have one of the following facets of their career out of whack:

• A product that drives their passion
• A team that highly values the application of their strengths
• A job that keeps them challenged

Passion

You need to find a product that will capture your heart. Sometimes mentees will come to me saying “I am thinking of joining that team, because I can learn [some important skill]“… Probing deeper, I will find that’s the only thing they value in the team. I usually respond by asking “Do you know what you will do next year once you have learned it and are bored and/or unhappy?” If they don’t, I recommend they look for a different team and work with them to try to find another means for them to acquire that skill. Passion is what keeps you engaged. If the product doesn’t do it for you, your value to the team will be constrained. Happy people are productive people.

Strength

I will write an article on this more deeply in the future, but I highly recommend some quality introspection around your strengths and how they are utilized in the organization. My favorite framework to use is StrengthFinders. Their tagline is “Do you have the opportunity to do what you do best every day?”.   Studies have shown (available off of the strengthfinders site) that a strategy around heavily self-investing in what you are good at naturally will yield a nearly 10x productivity rate. In this blog’s context though, your strengths catalog your *youness*… In a business context, this means the value you naturally bring to the workplace. It’s important to know this so you can help identify those roles in your workplace that will value what you bring,

My strengths (as example):

1. Ideation
2. Restorative
3. Individualization
4. Relator
5. Analytical

At a rudimentary level, it shows that I have talents in Innovating, Team Building, and Problem Solving. I love being a manager on teams that are on the cutting edge and trying to find that breakthrough that will take them to the next level.

As a corollary, I do not enjoy, nor thrive, in Individual Contributor roles that are purely execution in nature. These are very important roles, no doubt, but not something I am best suited for.

Other frameworks exist to help you identify your strengths, if you decide that strengthfinders is a “cross between good advice and a horoscope”.  The point is to find and use what will help you thrive. Both you and the team you are on should value your strengths and invest in them. If you find yourself in a position of feeling like square peg being forced into a round hole on a daily basis, talk with your manager and find a new role. If none exists, find a new team.

Challenge

In order to grow, you need to be challenged. The following diagram highlights Csikszentmihalyi’s Theory of Flow (not be confused with Kanban’s usage of the same term).

The chart above shows that the secret to happiness is to grow your skills and challenges at the same time. Growing your impact on your organization is quite similar to this process. Investing in your strengths is equivalent to investing in your skills (above). Being constantly in the flow will make you happier.

At Microsoft, there is a concept of a Business Justification for each role. What this means, in essence, is there is a business need for someone with a mastery of certain skills to be in the role. So what do you do when you have grown your strengths to the point that exceeds the business justification for your role? (or you have “automated” your role, so that it now has a lower level business justification?)

Really you have two choices:

• Find new challenges to add to your role (and continue the flow)
• Find a new role

Your manager should always be willing to partner with you on the first bullet above. If they are not, then you really need to find that new role elsewhere.

One last point here: If you are someone in upper levels (senior, principal, partner), then think about the business justification for your role in comparison to your peer group. If it implies a lower level justification by comparison (for example: a Senior can do your job and you are a Principal, and the rest of your peer group has Principal scoped jobs), then you *also* need to do one of the above choices if you want to have a chance to level the playing field for the review process. Now is the time to do this thinking, not 6 mos from now.

Alone in the Dark

By definition, you can’t know what you don’t know you don’t know (Armour’s 2^nd order of ignorance), but you can help defeat it by constructing a process which will enable you to learn those things that matter. Yes, *you* need to be the one that leads your career, but you don’t have to do it alone, nor without a clue. Surround yourself with those things and people that will enable you to learn. No matter where you are, I view getting a mentor as crucial. I, myself, have several mentors and value their honest assessments highly. I am lucky in that I don’t get offended easily and surround myself with “say it like it is” folks.

Visibility?

If your manager is telling you “You need to grow your visibility”, then shame on them. Your manager is optimizing for a dysfunctional loophole in how the process works. Yes, this tactic will work. It can and will get you promoted, but it elevates Peter’s Principle. I believe Visibility is a 4 letter word… It’s offensive. Unfortunately, this advice encourages folks to do what it takes to be visible. By making visibility a goal, you are not learning the skills to maintain and sustain yourself at the new level.

You are helping to ensure the Peter Principle comes true for *you*.

There’s a reason why it usually takes 2-3 years to get promoted. You are mastering a new set of skills to help you scale to the higher positions and win there. Don’t make visibility the goal; instead, making adding solid business value and growing your impact to that business the priority… Nail this and visibility will come with it, but as a result of your success not as the goal. Plus you will be more prepared to handle your next promotion.

Choose a job you love, and you will never have to work a day in your life.
Confucius

Need more to read?

Some other great blog posts on Microsoft Review and career advice from their authors:

Anita George:

How Important is the “How”?

If You Want It, Then You Aren’t Ready For It

Up, Up, and Away!

Eric Brechner:

Level Up

Permanently High Plateau

It’s not going to be okay

Alan Page:

Three Surprises

Dead Man’s Curve

Stack Overflow

Making QWAN actionable for Testers

Introduction

Shortly after writing my last post, QWANtifying Quality, I was having lunch with a longtime friend of mine who gave me some feedback. He loved the post, but he wanted something more to make it actionable. I received similar feedback from others… It was fairly esoteric. Today’s post will try to connect the dots. If you are reading this site for the first time, I recommend you go back and read the prior post before continuing. Though, in its conclusion, I share an opinion:

Delighting customers is good. Helping them live to their fullest potential is better.

For those who just want to know my answer to How? In a nutshell, it’s follow the Lean principles of “Fail Fast, Learn Fast, Recover Fast” and all that goes with it.

Otherwise, read on. I think Alexander’s knowledge on Quality sheds some great insight into how/why Agile actually works:

I asked my friend to explain to me his routine for watching TV. He likes to grab a huge pillow, a blanket, a bowl of snacks, and two drinks. When he enters into TV mode, he doesn’t want to get up for a while. He puts his drinks and bowl on a side table, covers himself up and then props his legs up on a coffee table. TV on. Laissez les bon temps rouler!

I asked him: how would it feel if you didn’t have that side table? He mentioned it would clearly be a loss. By holding his beverages and snacks in close proximity, the table is helping him to live. That table, the pillow, blanket… all of it… worked together to maximize his comfort level.

Using Alexander’s terms: my friend’s design of his living room has encouraged a Pattern of Events. Essentially, this is a verb or set of verbs that the design encourages over and over. In this case, my friend interacts with all of these objects in the same way most times he watches TV. It helps my friend live better. In addition, this pattern is deemed “alive”.

Why are Patterns of Events important?

Because they give you insight into the quality of your design. “Design” in this case can refer equally to a living room or a software package. I’ve said before that quality is the aggregated subjective POV of your customers. Today I’ll put it differently:

Quality is the aggregation of the Patterns of Events created by the design.

What this means:

• Patterns are those interactions that repeat themselves over and over.
• Interactions are more important than opinions. Actions do speak louder than the words of your users.
• If humans don’t interact with the design, the quality is, at best, indeterminate and at worst (and most likely), low.
• If humans do interact with the design, then the quality is the overall sum of Living patterns + Dead patterns. If the Deadening patterns are the stronger force, net quality will be low.

NOTE: A living pattern is one that either frees us or amuses us. Similarly, a dead pattern is one that traps us or adds difficulty to achieving our goals. Dead patterns often feel like “work”. One example of a dead pattern that immediately comes to mind is my Ping Pong table.

My youngest son and I love to play it together. Unfortunately, the table is currently folded up next to bicycles, baseball equipment, treadmill, etc. on one side of the garage. Getting to it is a challenge. My wife’s car occupies the other side of the garage. I installed a really nice light on her side of the garage, so we can play table tennis at night, but do we? No. There’s too much friction. I have to move out the car, untangle the table from the other stuff, move the table over, etc. So while the table has the potential of improving our lives, it doesn’t. There are too many dead patterns working against it.

Similarly, imagine what my friend’s living room would feel like to him if he moved his side table right next to the TV. Do you think he would still continue to use it? My buddy knows subconsciously the patterns for his living room. Until some new pattern comes into play, he will always design his living room in a similar way, even though he may not know why he did it.

Creators of software need to tap into the subconscious of its users to figure out the patterns. Note: Alexander mentions you cannot know QWAN while you are experiencing it. While that may not be entirely true, I think what he means is it is a lot easier to understand other people’s subconscious programming than your own. The second you tap into your own subconscious your ability to “see” what it’s doing dissipates. By definition, it’s operating under your conscious mind’s radar.

Ok, I get it. How do I use this?

I tell people the first thing they need to do is think of their software as a set of questions, not a set of answers. There are a lot of folks in Test who want to do everything possible to prevent harm to the customer and there is value in that. Quality means lighting up Living patterns. You simply cannot do that up front. Until you have it validated by your customer, it is just a guess.

Never forget that.

The questions you want to ask are going to be a variant of “Does this feel right to you?”, but they are likely going to be specific for your areas and feature. Another way to put it: The customer is already going to test your product. In addition, she/he is going to execute the Test cases that matters to them. Unfortunately, they will not tell you whether or not the test pass or failed. You will have to reverse engineer that from the telemetry. By knowing the questions up front, the design will be influenced to make this easier.

Next, add instrumentation and an instrumentation pipeline. Easy instrumentation method: Log the timestamp, object, and event for every interaction the user performs. Create a background process to send that info to a well-known site for aggregation. Can’t afford to receive and store all that info? Then, surgically, instrument the information that will help you to answer your questions.

Step 3: Get users using it. Slap a “Beta” label on it and get it out to your customers. Let them use it as *they* see fit.

Lastly, Analytics and lots of it. You need to look at heat maps and sequences of events. You need to look at failure/success rates, if present. You will have to make educated guesses, if not. I consider myself only a beginner at this Data Scientist stuff, but I can say it does get easier and easier over time to make an educated guess, ship something that tests that guess, then observe the behavior again. It’s hugely rewarding when you figure something out and the data validates it.

A note: If you have the ability to allow folks to customize their experience, I highly recommend you enable it. This isn’t just skinning. By observing how they arrange their experiences, they are giving you great insight into their behavior and Pattern of Events. This is no different than letting folks rearrange their own living room. Let them design how they like it. I love the Windows Phone live tile and pinning features as a great example. In addition, it’s never a bad idea to enable the like/dislike button on your design. If customers actually want to tell you that the test case they tried passed/failed, you should reduce that friction as much as possible.

Another note: Many folks challenge my optimize for reaction approach to quality. “Brent, does that mean you believe we shouldn’t test anything?” No, of course not. But I believe the proactive testing should be mostly focused on preventing those things that are most likely to cause an allergic reaction to users. I don’t think it takes much effort to strike the right balance. My suggestion: Get the right people in a room, spend 30 minutes, and come up with a 10 Minute Test Plan. By arbitrarily limiting time spent, the group will work together and focus on the stuff most important to prevent the customer from experiencing pain.

Making it real with an example

Here’s something I’ve observed with my children when they are on the XBOX. When they are on the XBOX dashboard, they almost never use the Voice interface, but they almost always use it when they are in a video application, such as Netflix.

After observing this behavior for many weeks, what I’ve noticed is:

• The kids spend very little time on the dashboard. They use it to get to where they want to go. They always have a controller in hand when they are on the dashboard.
• In Netflix, they almost always use the controller to navigate show selection
• Once a show is selected, they become almost allergic to the controller.
  • Voice becomes the preferred mechanism to interact with XBOX.
  • Controller is avoided. Accidental button pushes causes the show to skip ahead, back, etc.

In order to improve quality, here’s some possible conclusions:

• On the dashboard, the negative force appears to be voice misrecognitions.
  • Clearly, improving the engine should boost quality
  • But, optimizing for navigating with short easy to recognize statements could help too. Like “start Netflix”.
• In Netflix, any option that would prevent accidental pushes would improve quality. It’s a pain to navigate back to where the show was before the accidental push.
  • Allow keys to be remapped
  • Allow option to turn off controller when show is playing
  • Require dbl-press of buttons… etc…
  • OR instead of preventing, make it easy to undo the mistake. Some ability to go back to where I was before that button press.

If I had access and the data existed, some questions I would ask of the telemetry before implementing any code change:

• What are the common patterns for using voice on the dashboard? How are users being successful with voice on the dashboard? How are they failing?
  • Do my kid’s usage pattern show up?
  • What are the most recognized terms being used? How does word count affect success rate?
• How often does the *same* Netflix show get restarted after a controller button push?

There are a lot more I could ask. What’s key is that you are trying to understand the customer’s behavior and if *they* viewed what they are doing as positive or negative.

The next thing to do is to use your knowledge and get a fix out there. Ask the same questions and see if the success rate improved. Do this frequently enough, you’ll find your product iterating towards solving customer needs and improving quality. One thing I have really come to value: By pushing to find Living patterns, this process really helps to filter out those things your customer doesn’t care about. I hate it when my team delivers something that the customer doesn’t use. By following this model, we spend the minimal amount of time on those things while still pushing the envelope and innovating.

QWANtifying Quality

Introduction

2012 is fairly quickly becoming “the year I spent rethinking everything I knew about Test”. I don’t know what it’s like for you, but I believe Test is undergoing another reformation. This week alone I have been in far more than normal conversations around this phenomena. I find this hit count interesting. I view it as an expression of anxiety in the discipline AND as a problem seeking a solution. This year, though, unlike last year, there’s an aura of acceptance around this reformation. As folks are trying to work through how to handle what they view as inevitable change, they are trying to discover the value of the Test Discipline and what it should be. I see a dichotomy of opinion forming between the Proven and the Possible.

• The Proven camp knows how Test has always added value (asking questions, finding bugs, writing automation, etc) and trying to find ways to keep that value alive, but as ‘speed to release’ increasingly becomes a competitive advantage, they are struggling to scale cost-effectively (and therefore maintain their value).
• The Possible camp has a vision of the future, but a clear path for shifting the workforce to the new paradigm is really hard. First, most are not trained in behavior management and second, there are too many variants for “the vision” right now. It’s hard to tell when a technique works for a business context or not. It’s hard to tell if it will work unless you try it, and some may not be able to afford to experiment. Catch-22.

Within my own environment, I would estimate that there is another 2 to 3 years from a major tipping point. I believe we will see some of these new variants rise to the top as highly successful and models of success. Once these variants can be shifted from “possible” to “proven” at the business level, change will occur rather rapidly.

I have spent the majority of my career as a manager of Testers. IMHO, *NOW* is the time to put systems/practices into place that will help the team manage the future.

As I mentioned in A Tester’s Job, I believe a Tester’s job is to “to accelerate the achievement of shippable quality“. Please note: I did not state “to prevent the achievement of shipping crap“. This is, far too often, too theoretical. I believe the secret to moving forward as a discipline will be in changing our repertoire of techniques. Focus on delivering good to our users, instead of preventing bad. In order to do this, I believe we have to be reminded what it means to have achieved quality.

QWANtum Leap Forward?

Today’s article follows up from my last post. It is about the question of Quality. “What is it?” I am going to share some insight from one of the more important (my humble opinion) “fathers” of Modern Computer Science. You may know about Design Patterns and especially the book published by the “Gang of Four”. In addition, you may also know about Christopher Alexander, the inventor of design patterns.

For those who don’t, please note the use of lowercase there. What Alexander created was a system for identifying, cataloging, and using patterns in life for the purpose of design. Alexander, by training, was an Architect, Author, and until recently, a Professor. He is now retired, but in his work life, he contributed greatly to helping students learn better how to design and build houses, towns, buildings and the like through a set of principles he called The Pattern Language. The Gang of Four recognized that the principles he applied to architecture also applied to computer programming. The result of that connection was their book.

You may *not* know that Alexander’s designs and practices are an attempt to create and sustain something he calls Quality without a Name (often referred to as QWAN). Everyone in software engineering should learn, understand, and use Design Patterns. They offer many great benefits to you in order to create software.

However, I am reminded of a tweet:

I think Alexander would agree and further add they are looking for QWAN.

Described in A Timeless Way of Building, the “quality without a name”, which feels like a Taoist concept, is described as “the central quality which is the root of life and spirit”. As David Sheen synthesizes it, achieving QWAN “make us feel most alive, the most true to ourselves, the most unselfconscious, the most whole, the most complete, the most free”. Alexander calls it quality without a name as no single emphasis can be identified as the most important. All of these and in the right combination and context are needed.

Weird Science

QWAN can be, perhaps, expressed simply as “the feeling you get when interacting with a living design”.   We are trying feel alive and by doing so we create life.  Achieving QWAN is hard, but Alexander gives us some food for thought. For the purposes of today’s post, I present this in bullet form along with my own interpretation of meaning. Note: I am only presenting a subset of points that really reasonated with me. I highly encourage you to read the whole book for yourself.

Alexander tells us:

• There is *no* objective difference between a good or bad design.
  • This means Quality is subjective. QWAN comes from recognizing how designs make us feel.
• A design comes to life (is high quality) only to the degree life chooses to interact with it.
  • This means you can only understand Quality by observing how people interact with your design.
  • This also means you can only know Quality of your product reactively, by studying how life is interacting with it.
  • This also means Quality is measured by interaction level. Bugfree code that no one uses is low quality. Interaction level is measured both in terms of # of users, and depth of interaction.
• A design is given it’s character by a pattern of events
  • This means *how* life interacts with the components within the design helps to influence Quality.
  • A way to determine quality would be to study how life interacts with your design, enumerate those interaction patterns, and determine their quality level.
• Patterns that free us or entertain us are alive. (alive here means life is interacting with it in a way that it has itself become part of life. Eg. Texting is now an interaction pattern that is “alive”)
  • This means designs that enable personal freedom and/or entertainment have the potential to take on a “life of their own”.
  • This implies designs that are forced or feel “like work” will have a much lessor potential.
• The more living patterns a design has the more likely that it’s QWAN will be self-maintaining; it will feel like a part of nature.
  • By understanding the interactions, you understand the forces at play within the design/human interaction.
  • These forces can push quality in multiple directions, by replacing “deadening” patterns with “living” ones, you reinforce the design positively. It is not unlike planting grass on a hill to prevent erosion. Gravity and rain are working hard to destroy the hill, but by understanding these “forces” you can come to ‘planting grass’ solution.
• You can’t know QWAN during the moment you are experiencing it. Conscious thought causes it to dispel. It forces you to shift from living to analyzing.
  • You can not know when your design has achieved QWAN by yourself. You can only know this by observing others.

Quality is life

I encourage you to come to your own conclusion, but in my mind, achieving quality is not about delighting your customers then, it’s about helping them to feel alive.

On another note: I adore Alexander’s continual push towards trying to think of architecting designs in a light similar to that of the story Geppetto and Pinocchio. Viewing it this way: Isn’t Agile just another version of Natural Selection?

Quality is a 4 letter word too

Recommended prerequisite reading

If this is your first visit to my blog, then I recommend some pre-reading.

 I think of this blog post as the 3^rd part of a series.  I suggest reading “Test doesn’t understand the customer” first.   There I talk about how testers may have a glass ceiling in their careers because they are too far removed from Quality.   I then recommend reading TEST is a four letter word.   In that post, I bemoan the tendency of the software industry to overload the term TEST and thereby render it meaningless. 

Intro

Since I wrote that last article, I’ve been in a couple of heated discussions about Quality at work.   I was invited to a discussion of Test Leadership who was strategizing over how to best improve Quality.   When it occurred to me, there are a few topics where you can get 10 Testers into a room and come out with 11 definitions.   Quality is definitely one of these.

Based on these experiences and some alone-time spent pondering, I have come to grips with the idea that Quality has a problem.  It is a complex idea that is getting oversimplified.  It has occurred to me that Quality may also be a 4 letter word. 

By the end of this tongue-in-cheek article, I hope to convince you of this and inspire some to think a bit more deeply on the topic.

Quantity of Quality?

For those who would choose to believe that Quality is, in fact, a 7 letter word, I empathize.  Actually, I see this phenomenon quite often in test.  Testers are very good at figuring out metrics and claiming that these counts represent Quality.

They don’t.

 I started my career firmly believing this as well.   Even today, I still gravitate towards roles heavy in Beancounting, but I no longer view them as measuring quality.

Popular Metrics:

•          Tests Created
•          Tests Automated (or percentage)
•          Tests executed (or pass rate)
•          Bugs found (or Bugs found per tester)
•          Code Coverage percentage

In my experience, metrics of this sort serve to do one thing and one thing only.  They seek to modify behavior of the test team.  This is due to the Hawthorne Effect.  Peoples’ behavior change in accordance to how they are being measured.  As an experienced manager, I know the ones I chose above, if used “right”, will make sure more and more tests are being produced, but they tell me nothing of the quality of the code or product.

I offer you these words of advice:

                It may be important to your business to count metrics, but if you can count or measure it, it is called “Quantity”, not “Quality”.

FFFF

So what should we consider when thinking about Quality?  First off, Quality is the aggregated subjective point of view of your customers.  But how do we evaluate that?  What characteristics can we look at/for?   I offer you a 4 letter acronym to help, and as an added bonus, each word within it is an F-word.   Quality needs all of these things.  (characteristics, not F-words)

Features

First thing to consider are the features of the product.  Does it do what your customer expects?

Consider: A plain pinewood chair constructed without a seat.   While it is clearly recognizable as a chair, it’s going to have a low quality simply because with no seat, you cannot sit in it.

Faultlessness

Is the product working correctly?  Does the product do things your customer wished it didn’t do?

Consider: The same chair above with a seat, but now the front right leg is too short by 5 inches.   While you can sit in it now, if you lean to far forward, the chair will crash taking you with it.   It won’t be long before the irritation causes the high maintenance chair to be discarded.

Finish

How polished is the final product?  Is it desirable?  Does it feel special?

Consider: Now think of a plain pinewood chair as well as one made by artisans.  Think of one that was hand carved into hardwoods, polished to a sheen, comfortable cushions, etc.  It’s a thing of beauty and comfort.  It’s quality is higher.

Frame of Reference

Is the context right?  Is it relevant?  Does it do what the customer wants? This *might* just be the most important characteristic of Quality.  In my experience, getting the context wrong can really place you at odds with the customer.

Consider: This time the chair to consider is a toilet.   When it is placed in a restroom, it is viewed commonplace enough and no big deal.  But… Move it to the center of kitchen, and its context is dramatically out of place.  To some, disturbingly so.  Wouldn’t call that quality?  Now let’s say that same kitchen/toilet combo is part of a domicile located in a remote village in Africa… It suddenly isn’t so bad any more.  The quality feels improved.

Consider also: consider also my original chair (the one without a seat), what happens in your mind if I say that chair is actually a picture frame – I place a picture where the seat should be and hang the chair up in the ceiling?  By claiming its Art, I’ve changed the chair’s frame of reference and changed how quality will be perceived.

Consider lastly: My post today is titled “Quality is a 4 letter word too”.  Given, I started from a rant and I am following up from another article of similar intent, I might argue I have set things up so that the context of this blog title is more important that the correctness of the # of letters in the word.  Because of this context change, I argue today’s title is a quality one.  (but it’s subjective…  you may not agree)

This last item is an important item to consider when in test.   It implies there may be times in your career where the context of the feature is such that correctness may not matter that much…

What about Cost?

I was explaining this to one of my mentees the other day and he asked “Isn’t there a 5th? What about cost?”  To me, cost is not part of the quality equation.   Rather, both Cost and Quality are part of the Value equation.    Both Quality and Value are subjective, but basically, Value equals (figuratively) Quality divided by Cost.   Cost, imho, isn’t just money, btw, but barrier to entry.   A website that takes 15 minutes to render is likely too expensive (time is money) to be valueable in my way of thinking.

Closing

It is my hope that I helped give you some other things to consider when/if you are pushing to achieve higher quality.   While it is a part of it, quality is not simply the features, or the bugs you find.   It is the correct balance of several characteristics that, working together, will delight your customer.

By the way, there is another 4 letter word to describe quality, it’s QWAN.  Since this article is getting on the longer side of things, I will save that for my next post.