In Pursuit of Quality: Shifting the Tester mindset


Last time, I wrote a book review on Lean Analytics. Towards the end of that post, I lamented that I see a lot of testers in my neck of the woods trying to map their old way of thinking into what’s coming next. Several folks (Individual contributors and managers of same) have come to me wondering why should test move into this world of “data crap” and why is how they have been previously operating so wrong now. It is my hope today to explain this out.

But first, before continuing, I’d like to try something new and offer you a poll to take.

Please consider the following:

So which did you pick? Over time, it will be interesting to me to track how people are viewing this simple comparison. I have been doing this example for almost a year now. When I first started it, about 1 in 2 testers polled would select the bug-free code. Whereas with testers I talk to lately, about 1 in 3 will select it. I definitely view this as a good sign and that folks are starting to reflect on these changes and adapting. My ideal world is that 1 year from now the ratio is closer to 1 in 10.

Why is this poll so hard for folks?

Primarily, it is due to our training. Test was the last line of defense – a safety net – needed to assure we didn’t do a recall when we released product out to manufacturing. When I first started in the software development world, 1.44 floppy disks were the prevailing way customers installed new software on to their system. Windows NT 3.1, as example, required 22 of them. It was horrible. Installation of a new machine would take the better part of the day, disks would be asked for out of order, and lastly, people would often get to the end of the install to discover that a setting they were asked for at the very beginning was wrong and that it was easier to just redo install than to hunt through the manual to figure out how to fix it after the install.

Customers who got their system up and running successfully and found a major bug afterwards would be quite sore with us. Thankfully, I have not heard of this is quite some time, but back then, Microsoft had the reputation of shipping quality in version 3.0. There was a strong and successful push within the company to get our testers trained with a singular mission: find the bugs before our customers do and push to get them fixed. I was proud to state back then that Microsoft was the best in the world at doing this.

The problem I am attempting to address is the perceived value loss in Test’s innate ability to prevent bugs from hitting the customer. A couple of months ago I presented to a group of testers and one of the questions asked “All of this reacting to customer stuff is great, but how can we prevent bugs in the first place?” Thankfully, someone else answered that question more helpfully as my initial response would’ve been “Stop trying to do so“.

The core of the issue, imo, is that we have continued to view our efforts as statically valuable. That our efforts to find bugs up front (assuring code correctness) will always be highly regarded. Unfortunately, we neglected to notice that the world was changing. That, in fact, it was more dynamic: Our need to get correctness right before shipping was actually tied to another variable: Our ability to react to bugs found by customers after shipping. The longer the time it takes us to react, the more we need to prevent correctness issues.

“Quality redefinition” – from correctness to customer value

A couple of years ago, I wrote a blog, Quality is a 4 letter word. Unfortunately, it seems that I wrote it well before it’s time. I have received feedback recently from folks stating that series of posts were quite helpful to them now. One such person had read it then and had a violent allergic reaction to the post:

“Brent, you can’t redefine quality”.

“I’m not!”, I replied, “We’ve *always* had it wrong! But up until now, it’s been okay. Now we need to journey in a different direction.”

While I now refer to the 4 pillars of Quality differently, their essence remains the same. I encourage you to read that post.

The wholeness of Quality should now be evaluated on 4 fronts:

  • Features that customers use to create value
  • The correctness of those features
  • The extent to which those features feel finished/polished
  • The context in which those features should be used for maximum value.

Certainly, correctness is an important aspect of quality, but usage is a significantly greater one. If you take anything away from today’s post, please take this:

Fixing correctness issues on a piece of code that no one is using is a waste of time & resources.

We need to change

In today’s world, with services lighting up left and right, we need to shift to a model that allows us to identify and improve Quality faster. This is a market differentiator.

It is my belief that in the short term, the best way to do this is to focus on the following strategy:

    • Pre-production
      • Train your testers to rewrite their automation such that Pass/Fail is not determined by the automation, but rather, leveraging the instrumentation and data exhaust outputted by the system. Automation becomes a user simulator, but testers grow muscle in using product logs to evaluate the truth. This set of measurements can be directly applied to production traffic when the code ships live.
      • Train your testers to be comfortable with tweaking and adding instrumentation to enable measurement of the above.
    • Next, move to Post-production
      • Leverage their Correctness skillset and their new measurement muscle to learn to understand the system behavior under actual usage load
      • This is an evaluation of QoS, Quality of Service. What you want Testers learning is what & why the system does what it does under prodtraffic.
      • You can start here in order to grow their muscle in statistical analysis.
    • Then, focus their attention on Customer Behavior
      • Teach them to look for patterns in the data that show:
        • Places in the code where customers are trying to achieve some goal but encountering pain (errors, crashes, etc) or friction (latency issues, convoluted paths to goal, etc). This is very easy to find generally.
        • Places in the code where customers are succeeding in achieving goal and are walking away delighted. These are patterns that create entertainment or freedom for the customer. Unlike the above, this is much harder to find, it will require hypothesis testing, flighting, and experimentation, but are significantly more valuable to the business at hand.
      • Being stronger in stats muscle will be key here. Since Quality is a subjective point of view, this will force Test away from a world of absolutes (pass/fail) and into one of probabilities (likelihood of adding value to customers vs. not). Definitely, it is wise to befriend your local Data Scientist and get them to share the magic. This will help you and your team to scale sustainably.
      • This is an evaluation of QoE, Quality of Experience. What you want Testers learning is what & why the Customers do what they do
    • You will then want to form up a dynamic set of metrics and KPI’s that capture the up-to-date learnings and help the organization quickly operationalize their goals of taking action towards adding customer value. This will generate Quality!

Lastly, while executing on these mindshifts, it will be paramount to remain balanced. The message of this blog is NOT that we should stop preventing bugs (despite my visceral response above). Bugs, in my world view, fall into 2 camps: Catastrophes and Other. In order to have Quality high, it is critical that we continue to work to prevent Catastrophe class bugs from hitting our customers. At the same time, we need to build infrastructure that will enable us to react very quickly.

I simply ask you to consider that:

    As the speed to which we can react to our customers INCREASES, the number of equivalence classes of bugs that fall into Catastrophe class DECREASES. Sacrificing speed to delivery, in the name of quality, makes delivering actual Quality so much harder. Usage, now, defines Quality better than correctness.

Ken Johnston: a friend, former manager, and co-author of the book “How we test at Microsoft” recently published a blog on something he calls “MVQ”.    Ken is still quite active on the Test Conference scene (one next month), but if you ever get the chance to ask, ask him “If he were to starting writing the Second Edition of his book, how much of the content would still be important?“. His response is quite interesting, but I’ll not steal that thunder here. J

Here’s a graphic from his post for your consideration. I think it presents a very nice balance:

Thank you for reading.

Advertisements

22 thoughts on “In Pursuit of Quality: Shifting the Tester mindset

  1. Pingback: Five Blogs – 4 March 2014 | 5blogs

  2. Brent,
    your previous post left me wondering ‘What the hell is he up to with that?’ But this post dots the i-s quite eloquently (to my liking at least). Thank you so much for putting this down in writing.

    If you please, there’s one thing that still feels misterious to me. While I agree with you that our ability to react increased significantly, I’m not sure this covers everything. Can there be something else at play?

    For instance, quite useful definition of quality has been Jerry Weinberg’s infamous ‘value to some person, at some time, who matters.’ He came up with it much earlier than 1-2 years ago. Quality as relative value made sense even when we couldn’t react fast. In fact, I believe it makes sense even if we can’t react at all. Imagine we produce the only pacemaker on the market. People would find it valuable (i.e. of high quality) even if it would have been failing at regular intervals for 90% of patients (for argument’s sake, let us also imagine FDA doesn’t exist). It’s much better to have some chance to live than die with certainty.

    And if there are more drivers that help more people to ‘see the light’ now, other than increased ability to react?

    Furthemore, even nowadays there are still about 20% of people who choose ‘bug-free’ over ‘useful’ in the poll. How come? What’s your take on reasons that drive people to cling to such stance? I mean, it can’t be a mere habit — writing’s on the wall, isn’t it?

    • Ilya!!! Long time, my friend. Glad to see you still are around.

      Weinberg’s description is spot on, I think the problem was the timing. People didn’t have a way to act on his wisdom efficiently. In a nutshell, I think the drivers are 1) Speed is a major differentiator in today’s business environment, 2) Cost of storage has really been driven down, enabling more and more telemetry collection, 3) New Techniques operationalized – risk-taking companies have shown that they can knock down market leaders by smartly using analytics and Agile in concert.

      I’ve discovered 2 main reasons for the selection of “bug-free” over “used”.
      1) neophobia – people have been working hard at being great at the way they do things and have not be paying attention to the signs of change. This change is shocking to them.. It’s scary. For example, some deployments of the “future” model are moving all automation work to the dev team. People in test who like to code tend to be concerned. I tell them: no big deal… Move to dev then. They will love you.
      2) Entrenched Training – people have had the prior model beaten/baked into them for soooo long, they simply don’t get how anyone can state their value is diminished/diminishing…

      It was my desire to express 1) there is always hope and value in those folks who care about quality and 2) show a way to get there from here that’s valuable to the business and to the tester…

  3. Pingback: Testing Bits – 3/2/14 – 3/8/14 | Testing Curator Blog

  4. Brent, this is a fantastic post. Kind of along the same lines as Ilya suggests above I would like to see what your thoughts are when you work up and down the software stack. What you are saying very much aligns and is immediately able to be applied at the application, UI and Developer Platform levels but how do you take this thinking down to that System (HW/SW interface) layer? My feeling is that the lower in the stack you go the more and more the quality of components rests with the developer but when it comes to integration the customer more often than not is an internal one where dependencies are important and one small change at the system level causes branches at the top of the tree to sway vigorously. I think you can still apply the telemetry you speak of but a lot of times you can’t afford to let the “other” bugs go by due to my statement in the previous sentence; the software schedule quite honestly can’t accept any type of disruption. Thoughts? P.S. – great lunch today.

    • Thanks, Ben. I really appreciate your feedback.

      Your question really could be the topic for another post, but in my experience there are really 3 options available to teams in this situation. These options are, by no mean mutually exclusive, but do represent common practices:
      1) reorganize your team to deliver vertical slices (vs. organizing around architectural layers) – as the product grows, though, this option may not scale if too many teams take a dependency on the system component
      2) view your customer as both the final end user as well as the team taking a dependency on your code to come up with a more surgical definition of Catastrophe class issues.
      3) loosely couple your code with those who depend on you (eg. Web service), overemphasize prevention of Catastrophe class issues initially, then use instrumentation to scale back based on actual usage patterns.

      In all cases, decreasing the batch size of what is released into the wild as well as flighting changes to a small audience will help to manage risks of an undetected Catastrophe class bug hitting the end user.

  5. Really awesome post and very timely. Timely well coz it seems to be the code or perish , hop on to the cloud or you will be left behind era. I have some questions which may not be directly related to the above topic Brent. Is there an email id to send my questions to you so you would know if something around that’s already on your mind for the next blog ? 🙂
    Am going to be regular with your blogs for sure 🙂

    • Please feel free to write up your questions here on the blog itself. Might inspire others to comment or get others to clarify. Several folks who follow me have their own blogs and are deeply passionate about helping get us through this transition….

  6. Some of us testers had this question plaguing us .. Is coding the only way forward?
    As a tester, automation of course a given . Not that we are closed to the idea of coding but we chose to be testers because we did not want to code or develop code for a living.
    How does a Tester add value in concepts like Cloud, mobile services on cloud, the Devops model etc? Developers seem to fit into these at an early stage writing code or services or plugins etc but as testers is automation the only way we can add value to any of the above? Most testers seem to be worried that they are becoming extinct 🙂
    I read through your blog and found it very insightful and was sure we would get some good input

  7. Brent,

    Excellent post on running data analytics on product behaviors. I especially like the transition and symmetry between in-house testing and TIP.

    Also your intro is very valid. My understanding of that is: perfect software is not possible, or if you try to make it perfect, it will fail anyway because it won’t reach external customers in time for the business.

    But, IMO there’s something missing from your description of automation: regression testing, or as James Back would put it, “checks.” How do you know the latest code that was checked in did not break some aspect of the product, and how can you know it quickly enough to do a gated checkin?

    Maybe you meant to skip over functional and regression testing, but if automation is only actionable because somebody decided it was actionable after a lot of data analysis, then IMO that’s much too expensive.

    Did you mean to skip over it? Or if not, what am I missing?

    • Matt, thank you for your comment and response.

      I believe it was Michal Bolton who started the “checks” terminology and I know he works very closely with the Bach brothers.

      To your direct question, though, I did intentionally leave out many of the minute details around testing. Primarily, this blog is around the mindshift needed for testers for the new world.

      The current momentum regarding the topics of regression testing, checks, etc. is that the feature development team will take over ownership. My focus here is primarily around helping test move forward now that dev is taking over the duties that used to be one of the important test tasks.

      Thank you for your question. It occurs to me that maybe there would be some value in posting a similar Dev Mindshift article…

      • This is good news in that the owners of product quality will do it in a fully-powered language like C# (or C++, or Java) which enables much stronger quality regression. Actually, a powerful language like C# is a requirement for applying almost any part of my book on MetaAutomation (although, my book and the working code illustration is platform-independent). I’ll make sure to cover in my book the differences between coding quality standards (or touchstones) in test/check code vs. product code.

  8. Pingback: The future of quality is easy with EaaSy and MVQ | Data Driven Quality

  9. Pingback: The future of quality is easy with EaaSy and MVQ - Data Driven Quality - Site Home - MSDN Blogs

  10. Pingback: Systems Thinking: Why I am done with agility as a goal | Testastic

  11. A great, thought provoking post.

    I’m curious to know what conclusions you’re hoping to draw from your poll. I had emotional responses to the words “Certified” and “God” and “Bug free”, and framed my initial response around those gut reactions. It was only later that I took into consideration the relationship of quality (and value) to usage. If that’s what you’re hoping to gather a metric on, you should consider rephrasing those options to screen out other biases.

    I’m not sure about defining quality in relationship to user activity. I know what you’re getting at; user activity generates feedback and the development team can respond to quality that matters to the user. But what about software that doesn’t require user activity, the automation that runs behind the scenes? Would you call Chrome’s automatic updates user activity? The vast majority of users aren’t even aware that feature is running. There is a lot of software on the average laptop that the average user will, frankly, never use, but the super user cannot live without. Since this doesn’t add value to the majority of users, does this mindset frame the feature as low quality?

    • Hi Mark,
      Many apologies for the *very* late response. Unfortunately, my decision to go back to school takes quite a bit of time from me. Hopefully, this is a “better late than never” thing for you.

      The point of the poll and the associated hyperbole was to do one thing only. Inspire people to think. There is a lot of muscle memory in the Test discipline that tells us Quality and Bugs are synonyms. My point was they are not. Bug handling is a means. Quality is an ends. One that is only definable from the customer’s (very subjective) point of view.

      While the post was primarily about those features that have direct exposure to customers, those with indirect relevance should also be held to a similar standard. Your Chrome update example is a fantastic one. It is an indirect feature who’s purpose is to update and/or expose the direct ones. In my experience, most indirect features don’t necessarily drive happiness, but have a lot of power to make people unhappy. There are features I have worked on where it became super clear that happiness goes down when customer notice the feature. For example, update fails and causes corruption as example or update takes too long when all I wanted to do was a quick search for Pizza Hut.

      I would argue that these features can and should be held to same criteria as the direct ones, but with a different balance towards focusing on removing negative experiences (see above towards the end of the post). For example: it is more likely that these indirect features will have a higher degree of Catastrophe class concerns that need to be worked out proactively. The 4 pillars of Quality I mention above are also relevant but will be harder to measure due to their indirectness. Feature Value: We would want people to opt-in to the automatic updates so they can get our *other* new features in a timely fashion. Correctness: updates should just work. Polish: I’m sure there are other considerations, but for the most part, updates should be fast and seamless. The more I am involved with being able to use my system again, the unhappier I will be. Context: There is a right time and place for updates. Do not force me to reboot my computer when I am in the middle of a timed exam (this has actually happened to me).

      Using the above, I believe folks can discern the difference between what must be covered proactively (Catastrophes) and what should be covered reactively (precisely determine the best time to do the updates for a user as example)

      Lastly, you mentioned users versus super users. This, imho, is where context comes into play and is very important. Not all features provides value to all users, as you note. The quality of the feature is determined by the users and the problem they are trying to solve by using it. Certainly, having more users can help you build a better business, but the number of users doesn’t necessarily mean higher quality. It simply means more users. It can actually mean lesser quality in some cases. I recall a feature I worked on years ago. We’ll call it Feature A, but it order to use it, you had to navigate to Feature B. The team for Feature B was very proud of their usage statistics until it was discovered that 90% of the time users of Feature B went straight to Feature A. A hypothesis was formed that Feature A is what people are trying to do (heavily protested by the B team) so we deployed an experiment where feature A was exposed without having to go through B. Sure enough, users of A stayed strong (even went up), but B died off and eventually was removed from the product.

      Getting the context right helped to improve the quality of the product (by better understanding what customers were trying to accomplish) and resulted in us saving some $$ by refocusing efforts away from B.

  12. Sadly your poll lacked an option for ‘it depends’. What are the bugs?

    A highly used banking system that gives customers money but doesn’t subtract it from their account might be highly used, highly loved and still utterly wrong. Even having metrics showing that it is loved and used doesn’t help.

    What about buggy software where the software works correctly but has security bugs?

    What about software that is used often but each user that sees it decides never to do business with that company and goes else where?

    What about users who love the software, but it ultimately causes problems for them that they didn’t know was caused by your software? Even if they never blame your company (E.G. SimCity in http://www.joelonsoftware.com/articles/APIWar.html ), is that ethically right? Would you be happy releasing that code?

    What about code that currently is used, and works, but in the future will completely break? For example, what if an ID only supports 6 digits, and you are in the upper 5 digits. It will work and is used… up until it breaks because no new IDs will be inserted into the database.

    I am sure I can come up with more examples where no code (The same as perfect unused code) is better than broken used code. With that being said, I think your point is still reasonable. Often fighting to get fixed and fixing low priority bugs is less useful than looking for important bugs. But I don’t think that is news, trying to advocate for ‘bug-free, fix all known bugs’ model is not a great model in nearly all cases.

    As for your ideas for changing are interesting, and I think it again depends. Replaying user’s usage only tells you what people have done, not what they will do. That is a very reactive approach. Often a useful approach, but not in all cases, including when you have a new product and no users (yet). Worse yet, even with instrumentation, how do you know your instrumentation is working? Because it looks like it works? Does it always work? Do you know that when you push a bad build you only have 1% of the users seeing that and that it can be reverted quickly? I know there are cases where these ideas are great and useful, but do they apply universally? If you feel that the answer should be 1:10, is that because your experience has shaped you and perhaps your personal preferences rather than what the world really needs? Perhaps the whole world really does need to change but I am not convinced by this piece. Perhaps you just mean your neck of the woods needs to change? Thanks for making me think,

    – JCD

    • HI JCD,

      Thank you for your very thoughtful questions. I noticed as well as you have your own blog and I like to encourage folks to check it out as well. Note: I doubt he will respond (he is super busy nowadays), but James Whittaker is a friend and I just forwarded him your James v. James contrast blog post. Great read.

      I just expressed this in another response, but still appropriate here: “The point of the poll and the associated hyperbole was to do one thing only. Inspire people to think. There is a lot of muscle memory in the Test discipline that tells us Quality and Bugs are synonyms. My point was they are not. Bug handling is a means. Quality is an ends. One that is only definable from the customer’s (very subjective) point of view.”

      I completely agree with your response of “it depends”. My podcast partner, Alan Page, claims that those 2 words are always the correct response to any significantly complicated question.

      You asked a several thought provoking questions that I will attempt to address:
      1) software with security bugs does *not* work correctly
      2) RE: your SimCity example. As I stated in the post, “As the speed to which we can react to our customers INCREASES, the number of equivalence classes of bugs that fall into Catastrophe class DECREASES”. SimCity did not have and/or did not optimize for the ability to react quickly. In those days, it was a packaged product. That team should have (and likely did) proactive testing prior to the memory model being switched out from underneath them, so when they shipped, it was right. Likewise, Windows needs to support multiple applications and had one of the most extensive application testing labs on the planet, but yet this bug slipped through. I really don’t think ethics is an appropriate call out here. No one did anything unethical back then. The problem is more about scaling. The world has gotten so complex that you simply can not pre-test all possibilities in a timely fashion. In addition, new complexity releases into the wild on a daily basis. you can not predict how that may impact you. However, it would be unethical to release such software into the wild carte blanche. Ken’s MVQ system is a tried method for doing risk mitigation via exposure control that scales far better than the proactive testing approach (and cheaper too).
      3) I am a proponent of the Agile Architecture model. There is value in future proofing your code to the degree implied by your ID/Database example, but only limited value. It’s ok to only use 5 digit numbers now as long as you have abstracted the interface to the DB to not rely on digit count. Of course, this assumes you are not going to run out before you can get an update fully deployed. When the time nears, you can reachitect the db and point the abstraction to it without much difficulty. In the meantime, you have released your 5 digit id code and providing value to customers and positive impact to your business.
      4) RE: Instrumentation. In my proposal above, it is key to the pre-production phase. It will have to be solid and monitored when it’s in the wild. The data it generates and the resulting analysis is key to delivering quality in the modern world.

      “There are no absolutes, except this one”. The world is changing for the test discipline. It may not have impacted you just yet, but it likely will. Traditional Test positions will be around for a while, but they will rapidly decline. You can still apply for Cobol jobs today. Maybe a few companies can continue to afford a fully preventative approach, but one that adds using reactivity to it’s repertoire thoughtfully will have higher ROI, Scale, and speed while maintaining at minimum the same degree of risk exposure.

      I am not trying to convince you one way or the other. However, if in any way, I helped make your journey better, I have achieved my goal. Again, thank you very much for your feedback and questions.

      • RE: The ethics question, I just meant that if I shipped code that crashed, but people blamed someone else, ‘Oh, you upgraded to XP? Yeah, that’s ALL Microsoft’s fault.” would be unethical in my view. That is to say highly used code (Sim City) that is beloved but has bugs (the memory issue) might not be as good as unused code IF the interactions between software caused failures that were blamed on someone else. As a gamer I hate the blame game (It’s the drivers! No, it’s the game! It’s the OS!) and as a QA engineer, I personally have a distaste for playing the blame game. However, I don’t think Sim City or Microsoft did play the blame game, but it was a quick example where that could have happened. Perhaps it wasn’t the best of examples, but I wrote the whole thing rather quickly. I have a personal example where I didn’t want my name on a feature (involving sensitive data and a intentional lack of security), and in that case, I would have rather had no feature than the ‘working’ code.

        It seems the world changes often, and as we scale up of course things will change. I have no doubt about that. My personal doubt is that everyone will move to Microsoft scale. I suspect that there will be something closer to 3 scales. Small ‘startup’ business won’t hire testers, they only have a few employees. Perhaps just as they move out of startup size, they might hire one specialist if it really makes sense. As an organization grows a bit bigger, having some 20-30+ employees it seems that the scale they have, they can’t leverage all the data they collect and they can’t afford to hire senior developers who are generalized specialists (specialized generalists? 😉 ). This is a sweet spot for traditional testers. Often the company has equal parts growing pains and technical expertise and someone in test can help the process. Speaking just of my experience, often in those smaller companies only testers and product owners advocate for the customer. The fact that test is hard to define because it varies between organizations also reflects the fact that there are way way more small-to-medium businesses. There are few Googles, Facebooks and Microsofts in the world, but lots of 37signals (“Basecamp” now), Insomniac Games and Atlassians in the world. Not all of them will want testers, and some may prefer a more Microsoft approach, no doubt. Some will want more PM-oriented testers and some will want more automation-oriented testers. Some will have regulatory concerns. Some will …. etc. Speaking just of my experience, smaller companies often just can’t ramp up that fast. They need testers to help build the bridges to this data driven world, assuming it turns out to be the paradise advertised. This isn’t a case of Cobol vs Ruby, this is more a case of Java vs Perl. Java might be more stuffed shirt with its typing system compared to Perl’s regular expressions magic, but Perl is often harder to read because of it. Both tools have there place. Maybe one day the tools to do this sort of thing will get good enough even SMB (Small to Medium Businesses) will not need many testers.

        As a “SDET” like tester, I could easily be a developer, so this sort of change doesn’t worry me much. That doesn’t mean I am not looking at the different world views and trying to understand them. I think both sides have value. Even Alan Page’s latest blog post talks about exploratory testing (http://angryweasel.com/blog/hwtsam-six-years-later/ ). As an aside, your listener count went up by 1. I even mentioned it in my latest blog post: http://about98percentdone.blogspot.com/2014/12/shorts-ideas-described-in-less-words.html .

        I am pleased to hear Dr. Whittaker will get an opportunity to glance at my blog post. I spent several months trying to research his view points. I know for some his talk burnt bridges, but I wanted to know the underlying reasons behind it. He was redefining what he felt test was in 2008 and by 2011 he seemed to think testing was not that useful. It felt important to compare his views to something more pro test. Hopefully it turned out to be an honest and even handed post.

        Once again, thanks for the interesting feedback.

        – JCD

  13. Pingback: In pursuit of quality: shifting the Program Manager Mindset | Testastic

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s