Modern Testing Principles Explained

Nearly 8 years ago, I wrote my very first blog with a great deal of nervousness.   “Why does the world need yet another person espousing the value of test to software teams?”, I thought.   I am glad I did though. It greatly helped my communication skills and were instrumental in focusing my thoughts.    Looking back you can almost follow my own transformation into Modern approaches to Quality. Hell, just the first 5 posts contain many of the thoughts that are now expressed in the Modern Testing Principles (co-authored with Alan Page), including its mission statement of #ataosq.

Today officially marks another “milestone” of sorts for me in my shift away from Traditional Testing. This will be my last post on Testastic.    Why?  In essence, I no longer identify with the Testing brand.   I am a data scientist now;  Customer concerns and the delivery of actual quality is still (and will be forever) an important part of my job and my mission.

BUT, Test is now firmly entrenched in my mind as something I do and not something I am. 

Am I done blogging?   Who knows?!?  A wise friend once told me “I blog because I have something to say and when I don’t, I don’t”.  When/if I do come back to blogging, it will be under some new name that I identify with.  I am fond of the idea that I could further help Quality focused people enter into the data arena specifically. Maybe it will be oriented in that direction.

Before you worry (or rejoice), no, I am not leaving the podcast.  It has always been about Software Quality (not specifically Testing) and will remain so for the foreseeable future.   We named it ‘AB Testing’ as it enables us to talk on a broad array of topics.

The rest of this post is a follow up on Modern Testing Principles that I have been meaning to write for months.   It is my hope that it adds some form of clarity to the community at large.  

What are the Modern Testing Principles?

They are a path forward.  We authored them because we noted that a good deal of Test was struggling.  Struggling to make sense of several changes occurring in how software was being produced and their role in it.  Trying to make sense of memes such as “Test is Dead” or practices like “Testing in Production”.  In the traditional testing culture, these are nonsensical concepts. Several testers were (and are) left still believing that “what got us here, can still get us there”.

The Modern Testing Principles were developed to help Testers leverage their strengths into new positive directions that align better with producing software in the Age of the Customer. They are the antidote to Traditional testing.

Modern Testing is specifically named to reflect a need to leverage the benefits of contemporary practices and learnings.   The name “Modern Testing” is intended to distinguish present-day techniques, activities, and processes from their traditional “Test Last” counterparts.  It is absolutely expected that new learnings and techniques will continue to be discovered and leveraged in the pursuit of improving business’ ability to realize customer value.   Adapting and incorporating these learnings to towards “Accelerating the achievement of shippable quality” is thus a key tenet to Modern Testing.

Why are these needed?  

An astute observer of contemporary software development practices will notice that the principles, in fact, do not reflect anything truly Modern, nor anything about Testing.   Many companies have realized that techniques invented in the waterfall era of software development, no longer scale to current business demands.  As businesses adjust to the new landscape, many unprepared Testers are being left out in the cold.   Several studies have been coming out over the last few years which are giving credibility to a growing belief that while Test is not dead yet, it is dying.  Test is shifting from being a role to being an activity. As these efforts progress, Test, who has always been viewed as a cost to a business, is also being viewed as a bottleneck and irrelevant. BUT, not all software development business have adapted yet, so there is time for many.  Enough time for those, who care to do so, to learn new ways to apply their valuable skills into new positive directions. The Modern Testing Principles are for these Testers. They are guiding light for those looking for a way to create positive change in their lives in the face of a changing Test landscape.  

Markets have always been won by companies being able to adapt quickly to customer needs faster than their competition.   However, the software landscape has changed from the 1980’s waterfall process on several key aspects. Many companies today have learned and reaped the rewards of adaptable software development practices with low cycle times.  As such, it is easier than ever to compete; small teams can build great solutions at a fraction of the time and effort than ever in history. Lower switching costs give customers more freedom to move to competition in friction-free manner.  The online world grants more visibility to customer behavior telemetry than ever before seen. Additionally, software engineering techniques and processes have changed to allow continuous updates to software which makes risks smaller than ever to deliver solutions to problems that customers have today.

Important Note: one day a series of discoveries will occur that will remove the need for human involvement in quality evaluation and adaptation. As a data scientist, I suspect it will surprise no one that I believe Machine Learning will play a huge role in that. When those discoveries are found and implemented, Modern Testing approaches, just like Traditional Testing approaches before it, should immediately be abandoned.

Who is a Modern Tester?

Short answer: No one.  It is not a title. There will certainly never be a certification for it.  Modern Testing is a direction and a pursuit with a specific focus: Accelerating the achievement of shippable quality. A Modern Tester is a systems thinker who has experience in Traditional methods and continuously is learning to find more effective ways to achieve quality.  Some specific notions that a Modern Tester might reflect upon:

  1. The world has changed

    • Technologies like AI, Open Source, Big Data, and the Cloud have made it easier for competitors to win
    • Customers are empowered and likely indifferent.   They can switch to your competitor in seconds.
    • Customers care may only about their problems being solved.
    • They need those solutions today.   Not in 6 mos. Today.
    • Traditional Test approaches (such as scheduled test passes and separate Dev/Test disciplines) were required when we needed to ship Manufacturing and it was expensive to re-ship software.  They are may only be optional now.
    • Businesses need to care about Code Correctness and Craftsmanship in order to sustain speed of delivery.  Their Customers, though, don’t.
    • Several very successful companies have shipped bugs (and continue to do so) and don’t have Testers.
  2. Several Traditional Testing methods might no longer be needed or are unable to scale to current demands; keeping them alive may harm business goals by:
    • Creating unnecessary delays
    • Creating unnecessary costs
    • Being focused on code and spec correctness instead of quality
    • Enabling a detrimental ‘safety net’ culture and failing to deliver on the promise of moving quality upstream
    • Treating Test activities as an innate specialization that cannot be taught
    • Favoring dogmatic isolationism (“specialization”) over community and Whole team approaches
    • Being a passive contributor to decision making
    • Favoring Intuition and theory over work that measurably moves key positive business KPIs, such Customer Satisfaction or Cycle Time.
    • Failing to ACTUALLY measure and then, reduce risk

Explaining the Principles

In this section, I will explain the rationale around each of the principles.  More detail can be found in a variety of places. Visiting is an obvious first start, but I would also include with Episode 67 of the podcast and continuing through to 93.   The Ministry of Testing also has a fantastic synthesis here (and comes with a totally awesome printable graphic).   Lastly, many of the #Three (the term of endearment for our regular listeners) have created their own presentations or references.  My current favorite of which is a brilliant mind map of Modern Testing by Maciej Wyrodek.   Several of the #Three are active participants in MT community discussions on our Slack channel.   Click here to join.  (Don’t forget to go to the #iwantasticker channel.  The stickers are cool). I will also add additional book references for those who wish to understand the principles deeper.

My Nicknames

For each of the principles, I have “short titles” for them that simplify their intent. These are:

  1. Business First
  2. Bottlenecks out
  3. Learn
  4. Leadership
  5. Real Quality
  6. Product Hypotheses
  7. Embrace the Fear

#1 – Our priority is improving the business.

Book Reference: The Lean Startup by Eric Ries

This has turned out to be one of the least controversial principles in the list. Honestly, I fear this is due to a misunderstanding of the principle. In business leadership circles, Test is most often seen as a cost of doing business. Almost an insurance policy. But is it? When Test is questioned on the return, the answers are generally vague or subjective. The Test Community at Large does not have a good answer for this. The most common answer I’ve seen is “reducing cost”. However, digging in deeper often reveals that this is false. Its theoretical.

Test is generally very strong in Customer Empathy. The Modern Tester adds to this a non-theoretical look at what the goals the business is trying to achieve, looks deeply at how their work contributes to it, objectively measures it and becomes part of the solution moving that KPI. They remove the theory and execute in terms of ROI towards business goals. Generally, businesses are trying to do one of 2 things: Grow or Reap tangible rewards. Find out from your business leaders what your company is trying to do, connect your work to it, measure it, and improve. I highly recommend keeping an active eye on Anne-Marie Charrett and her work on Quality Engineering. She is going deep on this topic and imho, it is absolutely a critical step.

Last, we get feedback quite often around “Profit isn’t everything”. We agree, but it is important. A businesses primary mission is to serve its customers. Profit is a necessary secondary mission, though, so the company can exist and grow.

#2 – We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system.

Book Reference: The Principles of Product Development Flow by Donald Reinertsen

The Traditional Tester is building the reputation of slowing down ship cycles. The Modern Tester attacks this head on and works to instead speed them up. This is done by identifying bottlenecks and other causes of delay and work to remove them. Reinertsen’s book talks directly about Flow and goes deep into what practices work and why. While is not a book written for QA specifically, it does cover many topics on what causes delays. For example, a recent scientific study published by Dr. Nancy Forsgren in her book ‘Accelerate’, calls out that 1) Developers primarily creating and maintaining tests is correlated with business performance, but 2) Automation owned and maintained by QA is NOT. Why? 1) the code naturally becomes “more testable when Developers write the tests” and 2) they “care about them and will invest more effort into maintaining them”. QA has known for years that ‘moving quality upstream’ will result in the removal of many delays in the system. In this case, Dr. Forsgren’s conclusion is the same as my number one suggestion: Teach your dev’s TDD; help them succeed. Once that is done, discover the next major cause of delay and root it out too.

#3 – We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures.

Book Reference: Implementing Lean Software Development by Mary/Tom Poppendieck

The primary goal here is to incorporate a constant incremental “learning loop” into the Development process that shifts Code Correctness concerns and accountability to the people who write the code. I hear quite often that “Dev doesn’t want to do test” as a justification for keeping the Test Last approaches alive. First off, creating an identity for a discipline (QA) based on what a different discipline (dev) doesn’t want to do is not good. Modern Testers seek missions of direct value to the business. Next, Developers are perfectly capable of writing solid code and this includes Test Code. They are just not always incentivized to do so. Furthermore, with a good test team as their safety net, they don’t have to. The safety net keeps the customer protected from code correctness concerns; however, it also keeps dev insulated from those consequences and encourages the ‘it’s your job, not mine’ approach. Instead, start shifting away from this protector role and begin the process of teaching dev to succeed on their own.

#4 – We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture.

Book Reference: The Starfish and the Spider by Ori Brafman

“Culture eats strategy for breakfast, lunch, and dinner.” Creating and sustaining an inclusive community of people is perhaps one of the best ways to create and sustain positive initiatives. There’s no real good way to create successful software if one ignores people. People are key, especially in knowledge work. As an example, I have recently been diving into the science behind the push for Diversity and Inclusiveness. Multiple studies lately have shown that Women in leadership roles not only improve business outcomes, but greatly so. One failing of the movement (imho) is the lack of explaining why. Not surprisingly, it is NOT because what body parts a human does/does not have. Women are statistically more likely to value relationships between other people, collaborate, and include other ideas towards common goals.   These are all learnable skills.  Improvements in Knowledge Work requires new ideas. Ideas are born from old ideas coming together. Diversity grows the number of ‘old ideas’ in the mix. Diversity is not possible without inclusiveness. Building a community where people share their efforts and thoughts towards commons goals creates a breeding ground of practical ideas, including ones you can use now.  Leadership (especially servant leadership) is necessary to make these changes stick and self-sustain.

#5 – We believe that the customer is the only one capable to judge and evaluate the quality of our product

Book Reference: The Four Steps to the Epiphany by Steve Blank

One of the more controversial principles. When correctly interpreted, it can create an identity crisis for those entrenched in Traditional Test culture. It states the customer is the only real judge of quality. The corollary: Test is not a judge.

Who is the Customer? Whoever holds the power over the final buying decision. For example, are you working on a B2B product? Then I hope you are actively helping *your* customers to help *their* customers.

Why is the Customer the only one capable? Because Quality is not about bugs or features. It’s about the application of a solution to a recurring problem.  A problem your customer has.  Quality is a subjective point of view and the only subjective point of view that matters: the customer.  A friend of the podcast, Steve Rowe, said it best. “Test has lost its way“. The Modern Tester cares about quality over code correctness and requirements. And the Customer is the only one that knows whether or not the product is solving their problem. Quality is customer happiness.  (not leads to…   is)

#6 – We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact.

Book Reference: Lean Analytics by Alistair Croll

Without the data, you are just guessing. It may be an informed guess, but a guess nevertheless. Our job is to deliver value to the customer. Fixing bugs or adding features that don’t matter to the customer is simply wasting time. Time is the one crucial resource that once it is gone, you can not recover. Data is the key ingredient needed to take your “product requirements” (aka guesses) as a start and triangulate the actual needs of your customer. You don’t have to shift to being a Data Scientist as I did. (but it was absolutely a worthwhile journey for me.) However, it is critical that you become accustomed to using data for driving decisions. Traditional Testing was about being an information provider. Our suggestion is go further. Guide the decision and translate it into action. I did a webinar on this for AST. If you can power through all of ‘Ums’ I do in the presentation (I can’t), I do think there’s a lot of helpful content in here for you to start your journey.

#7 – We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist.

Book Reference: More Agile Testing: Learning Journeys for the Whole Team by Janet Gregory/Lisa Crispin

We acknowledge that principle #7 is scary. Some interpret it as Alan and Brent’s version of “Test is dead.” Others interpret it as Alan and Brent are saying to “Kill Test”. I will be very clear. We are saying none of these things. However, we are saying Testing is an activity (and a necessary one) and further, we are saying this is an activity better owned across the whole team. Even if it eliminates your job as a testing specialist.

Both Alan and I agree with the growing sentiment that Dev and Test positions will eventually merge, however, you don’t have to. Instead, I offer you this suggestion: whether we are eventually proven right or wrong, following the principles will grow you in your capabilities for delivering business results. If we are wrong, you will be a stronger test specialist for the experience. If we are right, then you will be prepared and have options and experiences that will make you even more valuable than before.

In recent years, I’ve had the opportunity to connect with Lisa Crispin a number of times and walked away better for the experience. They say never meet your heroes, because they will disappoint you. Lisa has not. In fact, I am proud of her. She’s acting on the opportunity the changes created for the Quality focused folk amongst us, which “led her down a whole new [exciting] path”.  Keep an active eye on Lisa as well.

Lisa Crispin and Janet Gregory have already been working the Whole team aspect for years. Heck, if all you do is start a book club with your devs, that will be a fantastic start.

While we can not guarantee your experience will match ours, but literally everyone I know who has made the journey away from Traditional Test has stated they would never go back.   Its quite a journey and not always easy, but the “grass is greener” here.   It is my hope that we will get their stories up on the Podcast soon.


Well, that’s it then. Thanks for taking the time to read this and for allowing me to help guide in the written form. I really appreciate the Quality Community and look forward to collaboration going forward. Hopefully, I will start a new blog site soon under a different theme, but until then, feel free to connect with me on Twitter or the Podcast’s slack chanel.




What’s so special about specialists?

A problem I am constantly asked to address is which are better Generalists, Specialists, or something in between? I have written about it, presented about it, and podcasted about it, but nothing as extensively as I plan for this post. Honestly, I don’t know how much additional value I will add to the community of thought on this topic, but at minimum, I will now have a URL to send people who want to know more details. So for me, this will be a big time saver. Much of the content comes from years of research and experience in coaching teams through an agile process.

The problem

Inevitably, the teams I am helping will face a period of rejecting agile because of some challenge a person or set of people are facing with the change. Usually, I get something like: “I am a specialist and this is doesn’t optimize for me”. This has always happened when I am coaching a team conversion from waterfall methods to agile. However, even the more successful teams face some challenge if they overdo the shift and swing the pendulum too far in the opposite direction. “I am a generalist now, just like everyone else. How do I differentiate for performance reviews?” If you facing similar questions on how your team should be staffed, then it is my hope that this post will help you answer what is right for you, why is that so, and some hints on considerations should you desire to implement changes in your environment. If you desire a simple one-size-fits all answer, then you want T-Shaped people. If you looking for a more nuanced answer, then I will encourage you to read on.


There multiple examples to draw from when comparing specialists and generalists. In the animal kingdom, a Koala is known as a specialist species. It can only live in one environment and has only one source of food. Compare it to a Raccoon, which has been known to be able to thrive just about anywhere. In the medical profession, you may have a general practitioner who will refer you to a heart specialist. Even jobs like Sign Spinners and Dog walkers are considered specialized jobs as they generally are only seen in specialized environments (heavily populated areas). Whereas, you can find a Handyman just about anywhere. It is interesting to note that Sign Spinners are becoming automated (crudely automated, sure, but automation always starts with basic beginnings). Lastly, there is the software world, where it is quite common to hear developers and testers to claim themselves to be specialists.


I am going to be exploring this topic as it relates to software development and the software development lifecycle. In addition, while almost every change requires shifts in people, processes, and technology, I am not going to spend a lot of time on tech. I am also going to assume (falsely) that delivered code is always perfect. This will help to keep the explanation simpler. As the patient reader will discover, this is a deep topic and I will only barely scratch the surface.


Some key terms used throughout this post.

Knowledge Worker – Thomas Davenport says: “Knowledge workers have high degrees of expertise, education, or experience, and the primary purpose of their jobs involves the creation, distribution or application of knowledge.” It is important to note that every human being is a bit different in terms of how they think and create, therefore it should not be a surprise that every knowledge worker has unique characteristics when it comes to producing results as well as the optimal environment needed to do so. However, this uniqueness is abstract to some degree. For example, I have knowledge for blogging, podcasting, agile practices, customer quality, data science/handling, testing, etc.

Generalist – someone with a wide array of knowledge; a person competent in several different fields or activities:

Specialist – a person who concentrates primarily on a particular subject or activity; a person highly skilled in a specific and restricted field.

Versatilist – someone who can be in a specialist role but at the same time switch to another role with ease; also known as a t-shaped person.

The Goals of Business

The goal of any business is to find a stable business plan that creates value and can grow the reach of the market to as many folks as possible. As part of this, optimizing the utilization of the resources used to produce new products is important to keep costs low and make more widgets. An important KPI when discussing Specialists and Generalists is Productivity. Productivity is defined as the number of outputs per unit of input. Since we are talking about software, let’s say the outputs are features that vary in their degree of value to the customer and the inputs are team members and time. However, while this is a necessary condition, unless you can easily replace those folks who leave your team, Employee Engagement is also going to be an important consideration. For the purposes of this post, I will define this to mean the degree in which your employees enthusiastically act to deliver business outcomes. Happy people are productive people.

Do NOT confuse productivity with efficiency. They are not synonyms. Efficiency is a measure of the ability to minimize waste while producing outputs.   It is common enough to misunderstand these terms.  Their differences can be subtle.  Here’s another reference comparing both.


The first known use of the term Specialist was in 1855 and it was coined to describe a medical practitioner who devoted their study to a particular branch of medicine. One year later the man would be born who would eventually create the system of management used most often today, Frederick Winslow Taylor. Taylor created the topic known as Management Science (aka Operations Research or Taylorism) back in the 1800’s deep within the industrial age. What started it was a study he did that recognized that deeply studying how workers performed their jobs and making small tweaks would greatly improve productivity. He used the term Generalists to describe workers that could be applied to most/any machine and specialists to describe those whose knowledge could only be applied to a very specific one. Taylor is accredited with noticing that the more repetitive the work could be made and the more inputs could be made consistent, then specialists would produce far more than generalists. He would prescribe folks to stay on a single machine and measure everything about it. Ultimately, he perfected the efficiency of the worker on that machine. But it came with a consequence, the resulting jobs required “less brains, less muscle, less independence”. In those days, this was fantastic. Getting new workers was relatively easy to come by and you could train them fairly readily on a new machine, and reap the rewards of high productivity, but it came at a cost. The workers became so specialized that they could not well function in a changing environment. Indeed, they could only take their inputs and turned them into the desired outputs. They were entirely unawares of the whole system in place. In order for the system to function, Taylor proposed a system of Managers to be constructed; it would be their job to 1) be the brains, 2) study productivity and 3) study and control the system. For the last 200 yrs, this was how the manager/employee relationship functioned, so when computer related jobs started to grow it was the business model most were used to.

The Pros and Cons of Specialists & Generalists

Specialists are optimized for “going deep” on repetitive tasks that require a minimum number of dependencies to succeed. In addition, they do not take a system view of problems, often requiring management to control their workloads in order to optimize productivity. In addition, as eluded to above, they are highly inflexible. Generally, they thrive in very specific environments with very specific workloads. Their inflexibility can be a bit of a problem and readily can cause workload starvation or bottlenecking of the process. For example, consider a UI development specialist whose UI is frozen (meaning no longer permitted to change for the release), what shall they do now? If it’s clear, the product will have a subsequent release, it may be fruitful to have them work on that now. This is more efficient, but if the product is stalled do to database changes then it would be more productive for the UI developer to apply their talents there. If they aren’t capable of doing so, I’ve seen teams go as far as add new UI features just so their developers aren’t idle. This creates a product that is optimized for the skills of your developers AND NOT for the needs of your customers. In the world where your customers can go to a competitor in a split second, this sadly is a really bad idea. However, there are really solid places where specialists make sense. In particular, those critical needs for a software release that are rarely called upon. Security and performance are examples. These require deep knowledge, generally valuable, and (unlike UI development) can be challenging for someone to pick up on a moment’s notice.

In contrast, Generalists are those folks optimized for flexibility. They are do take a system view, are very good at “filling in the cracks” and can be applied at just about any time to just about any problem. They are exceptional at solving 80% of any development problem (or 80% of all development problem… depending on how you want to look at it.. aside: both numbers totally made up), which leads them to be massively productive, since there is plenty of work them to pick up and move forward on. However, they tend to be inefficient with much of their time spent on ramping up on a new code base or learning a new skill. A frequent reader of this blog and/or listener to the AB Testing podcast will recall that I spent time as a manager of a dev team in the Unified Engineering model. There I encountered a phenomena: folks that had come from Test were out-producing those from Dev. I believe this was for 2 reasons: 1) Test development is a generalist skillset and 2) there was nothing particularly challenging about my product that required deep expertise in any particular area.

So what’s the silver bullet?

Ok, so specialists are more efficient than generalists, but generalists are more productive. But ideally, you’d want both: High Output and Low Waste. With a couple of caveats (see below), if you have a highly controllable business environment, then specialists are the way to go alongside a management team than controls how/when these folks are applied. Management will need to make sure all dependencies are cleared before the work gets handed off, of course. However, if you are operating in an chaotic business environment that needs to adapt to changing business requirements, you are better suited to hire a couple of key specialists and have the bulk of your team be generalists. Management may need to control costs directly, focusing the attention of the Generalists to only a few areas for each person. In both models, if the environment changes, you will need to be prepared to fire, then rehire your team if productivity/efficiency suffers as a result. This approach too may not be good enough though. It’s hard to quickly hire a team of any given size and time consuming. Once in my career, I completely hired a team of 30 people in just one month. It was a strong team, but it was grueling and required me to spend 3 weeks of interviewing 40 hrs a day, which was quite hard on my psyche (especially since I had other work to do as well). A better solution is hire versatilists. Think of them as specialists in 3 to 4 of the skills your team needs. As an example, I run a data science team today. I required every member of my team to be strong in at least 2 of the following: Data Science, Development, or Data Engineering. They need to also have a passion to learn the 3rd. As I hired, my emphasis for the remaining open positions focused on which area of my team showed the greatest weakness. This is strong and I am proud to claim they are known for being highly productive closers.


  • People have their own incentives. Above I mention a few caveats, much of these article is representing things from a pure emotionless business angle. However, until we’ve automated the development role, your employees will have emotions and care for more than just the business. Daniel Pink describes this human element as driving for Mastery (getting better at doing) something), Purpose (working towards something larger than ourselves), and Automony (self-direction). The value proposition of a product will, hopefully, drive their sense of Purpose. Specialists are going to be strong at pursuing Mastery and complain about being told what to do and when as will be required in order to keep your team productive. Likewise, Generalists are more easily granted autonomy (especially in an agile process such as Kanban), but they may not grow and may feel like they are constantly investing in and then abandoning skills. The Versatilist approach strikes a balance between these 2 extremes.
  • Software development is knowledge work. Very few things, in my mind, actually falls into the camp of “repetitive work” and low dependencies required to make specialists thrive. In fact, Steve Johnson’s work suggests that we would not want this even if we could construct it. Innovation comes from old ideas mixing together. A team of Versatilists, each required to switch areas every so often with 3-4 deep specialties are going to help knowledge sharing grow and help you innovate and adapt both productively and efficiency.

Send me your feedback and/or comment below. As I mentioned above, I’ve been meaning to write this for quite a while. I hope it’s helpful.

Book Review: Lean Analytics – great primer for moving to DDE world


So O’Reilly has this great program for bloggers, called the Reader Review Program. They will let me pick out a book of my choosing & read it for free, as long as I write up an honest review of the book here on my blog site. Because I know that I will eventually posting reviews here, I will be picking books that I think might have value to the audience that is following me. This is my first foray into this model. Right now, I think it’s an “everybody wins” thing, but I will pay heightened attention to how this affects the integrity & reputation of the site. Since I am generally reading 5-10 books at a time, I highly doubt that I will post blogs like this more than once or twice a year. Your feedback is welcome.


Lean Analytics by Alistair Croll & Benjamin Yoskovitz; O’ Reilly Media

    The title above will take you to OReilly’s site so you can delve further if you choose.



    As the title suggests, Lean Analytics is a solid combination of two powerful movements in the Software Engineering world, Lean Agile and Data/Business Analytics. While there are several books out there discussing the need for data science and growth in statistics, this book really covers the What, How, and Why for using data to drive decision making in your specific business. Without being too technicial or academic, it introduces readers to techniques, metrics, and visualizations needed for several common business start-up models in operation in today’s world.


    I am ***REALLY*** fond of the Head-First Series of books and that is just about the only thing that could make this book better. After The Lean Startup this is probably the most useful book for those trying to iterate fast in today’s software engineering world. I found the information to be very straightforward and easy to follow. While I think the authors really tried to cram everything they could into the book (at times, making it read awkwardly), they introduce you to practical examples of how to use the material and when.


Several sections of the book are quite good… looking at some lightweight case studies of startups and the analytics they used to navigate muddy waters. The book tries to make all types of software business accessible. Ranging from how to categorize the growth phase of your company, what things to use during your value phase, what analytics are appropriate for various types of companies (mobile apps versus SaaS e.g.), and even how to operate within enterprises. As a result, though, the depth at times can be lacking but if you are looking for a breadth book that covers all of the basics this one might be good for you. Reading it is one of the reason I have decided to start my Masters in Analytics. With more information in the case studies, and more examples of actual data to look at and suggestions on how to avoid false metrics and gives guidance on what to look for.


One of the struggles that I am seeing at my place of employ is that Test is shifting away from automation roles and into data pipeline roles. This means we are just changing the way in which we deliver information so that others can analyze it and make the “adult” decision. This, imho, is not good. But it falls within Test Wheelhouse, so it is safe. Please Please Please instead grab this book and take a leadership role. This book will help us start the disciple move into a direction setting role instead of just a measurement one.


This will likely be the topic of my next post. Thanks for reading…