‘Lean Startup’ definition of value

In the book “The Lean Startup Method”, Eric Ries, as CTO/co-founder of IMVU, describes an early experience with the startup.  After months of product development they realized that their customers weren’t interested in their product.  He realized that while he could console himself with the benefit of learning from the failure, such learning was little consolation to employees and investors.

The key concept for startups is what Ries defines as “validated learning”.

How do you validate learning?
You create increments of your software which you release to customers.  You track the response of specific groups of customers.  For example, you should be able to generate data which shows that after introducing the feature ‘simple login’ in February, the adoption of customers who used that feature has greatly increased compared to customers who don’t have that feature (as opposed to showing a general graph of customer adoption.)

Ries also introduces the concept of the scientific method and ‘experiments’.  This is how he describes an ‘experiment’.

“It begins with a clear hypothesis that makes predictions about what is supposed to   happen.  It then tests those predictions empirically. “

He gives the example of Zappos, the world’s largest online shoe store:

When starting the store, the founder, took pictures of shoes from stores and posted them online.  The question he was trying to answer was: “Is there already sufficient demand for a superior online shopping experience for shoes?”  The important takeaway from the example is that you should build a “minimalistic” product to conduct your experiment (as opposed to early e-commerce startups which invested in warehouses, inventory and a final product).

Definition of value

In the Lean Startup, anything which doesn’t result in validated learning is waste.  Instead of creating product features, you can create a ‘facade’ (see MVP below).  This is then used to get feedback from customers.  The feedback is in the form of actual customer adoption, not as a survey or a focus group.

While soliciting customer feedback is an important part of agile, sometimes teams may be a bit forgiving about the feedback they get.  Internal stakeholders may be used as proxies for customers (in agile).  The Lean Startup is unambiguous in the feedback (validated learning).  Teams must be able to check each user story with ‘validated learning’.  Without that the story is incomplete.

This is enough to understand the concept of ‘value’ in the Lean Startup.  I will write more about the ‘Lean Startup’ later.  There are a few additional concepts which are relevant to this post which are explained in the following paragraphs.

Additional concepts

Minimum Viable Product (MVP) : A Minimum Viable Product (MVP) is the smallest needed to sell to potential customers.  A good example of this is the start of Groupon.  Groupon started by rapidly assembling a set of deals without a lot of focus on the presentation.  The idea is to get rapid feedback on the concept which can be refined based on customer feedback. (The definition of MVP used in the book is from Frank Robinson of PMDI ).

Startup: What is a startup?  The book cover includes a definition of a startup, “an organization dedicated to creating something new under conditions of extreme uncertainty.  This is just as true for one person in a garage as it is in a group of seasoned professionals in a Fortune 500 boardroom.  What they all have in common is a mission to penetrate the fog of uncertainty to discover a successful path to a sustainable business.”  Although this may be partly marketing, the definition includes any type of software project/product.  In the book Ries gives an example of Intuit using his concepts.  He gives many other example of large companies.

Agile vs. Lean Startup: It might seem that the Lean Startup doesn’t really add anything new to agile.  One major difference is that the Lean Startup is insistent on ‘validated learning’.  In the case of agile, customer feedback is critical.  However, in smaller organizations or for products which are not web based, there may be limited customer feedback.  The product owner and the team can also be used as proxies for the customer.  Not so in the Lean Startup.  The other important difference is Lean Startup’s focus on business returns/value.  A key question that Lean Startup teams need to answer is to show growth in existing customers or growth in adoption by new customers – without smoke and mirrors presentations.  Similarly the Lean Startup also encourages teams to validate their business model by creating MVPs which are focused on the business and business processes, not necessarily the technology.

Understanding lean manufacturing ‘value’

Every other day, in the context of software testing, there is a discussion about ‘value’.  “Do testers provide value”, “Testers should provide value” and the related, “Can we eliminate testing?” or “Testing is dead”.  In many cases, the word “value” is used colloquially (gold is valuable; dirt isn’t) and even casually (value is value), without much thought to what it really means.  In the context of lean (manufacturing), value has a very specific definition/meaning.  I think it is essential to understand the meaning of ‘value’ in lean, before using the word in other [technical] contexts.  In this post I will not refer to software development.  For this discussion you can think of creating a product like a bolt in a factory or parts of a car.  I will also refer to the process of visiting a doctor.

In the context of lean, any activity used to produce a product or a service can be one of three categories:
1. value added
2. non value added
3. non value added but necessary

There are some activities which are obviously non-value added.  In the context of a visit to the doctor, a patient waiting for the doctor is non-value add.  Mistakes in a doctor’s lab report, which requires rework, is non-value add.  In the context of manufacturing, defective parts requiring rework is non-value add.

The categories for some activities are not obvious, e.g., suppose an administrator delivers lab reports to the doctors in the clinic – is that value add?  Can you prevent delivering the reports?  Suppose you keep records for defective parts in a manufacturing facility – is that value add?  Can you avoid keeping those?

In lean manufacturing, in order to qualify as value-add, an activity must meet the following three criteria:
1. It must change form or function of the product or service
2. The customer must be willing to pay for it
3. It must be done right the first time

Even with this definition, it can be challenging to figure out if an activity is value-added.

In the context of lean, when we discuss if the customer is willing to pay for it, we are not discussing personal preference.  It doesn’t matter if the customer likes red pens or not.  In the case of manufacturing, whether a customer ‘likes’ an item is a problem of it’s design or marketing.  That is not relevant to the discussion of ‘value’.  In the case of software, engineers are responsible for the design, and it is important that target customers see ‘value’ in the design.  I won’t discuss software in this post.  It’s enough to keep in mind that a customer’s willingness to pay (in lean manufacturing) is more related to wasteful activities, such as defects or excess inventory.

There are some activities which are not value-added, e.g., they do not change form or function of products, but which are necessary.  For example, sales and marketing of products or services.  Although these will not satisfy the three conditions for value-add, a business will not survive without these activities.

Why is value important (in lean manufacturing)?
When people pursue process improvement or try to make processes more efficient, they focus on the small percentage of value added activities – “make it faster”.  Instead, eliminating NVA can give much higher gains with less effort. Typically only 10 to 15 percent of steps in a process add value, and more often than not, these steps represent as little as 1 percent of the total process time (From Lean Six Sigma for Dummies).  Consider a typical visit to the doctor’s office, there are large periods of waiting with relatively little productive time spent on filling forms and talking to the doctor.  Focusing on reducing the wait times will result in higher gains compared to improving the time taken by the doctor.

Here are some of the ways to eliminate waste in a doctor’s office.
– have broad roles for admin staff, eliminating unnecessary hand-offs
– making sure physician’s have all the information needed to focus on the patient
– avoiding rework by different admin staff, the nurse and physician
– using technology to avoid mistakes and rework

Identifying activities which create value is the first step in applying lean (More on lean later).

The concepts of lean can be applied to any industry.  If you would like to know more, you may want to start with understanding how it is applied to health care/hospitals/doctors.  If you don’t have experience with manufacturing it may be easier to understand how lean concepts are applied to a doctor’s office.  There are many references on the subject available on the Internet.

There are no ‘requirements’ in agile!!

It surprises me to hear staunch agile followers still using the word “requirement”.  When questioned, they will respond, “…a story is a requirement.”  It isn’t.  Early in my reading of agile I came across this analysis of the word “requirements”:

“Software development has been steered wrong by the word “requirement”, defined in the dictionary as “something mandatory or obligatory.”  The word carries a connotation of absolutism and permanence, inhibitors to embracing change.  And the word “requirements” is just plain wrong.  Out of one thousand pages of “requirements”, if you deploy a system with the right 20% or 10% or even 5%, you will likely realize all of the business benefit envisioned for the whole system.  So what were the other 80%?  Not “requirements”; they weren’t really mandatory or obligatory.”

(I am leaving out the reference to this quote so there is no bias.  If you don’t recognize the quote or the style of writing it is worth your time trying to find the source.)

In agile, instead of requirements you have user stories.  It takes quite some time to make the transition from requirements to stories.  To me, after reading the previous quote, Mike Cohn’s statement about stories was essential: “A story is a reminder to have a conversation.”  Of course it is difficult to create stories.  Mike Cohn has written a book about that.

Alas, “requirements” are even more entrenched in the software testing world.  Interestingly, in 1999, a great tester wrote a wonderful article about requirements.  In that he wrote:

There are at least four alleged truisms about testing a product against requirements. Most  of the testing textbooks on my bookshelf promote these principles, and each principle reflects some truth about the dynamics of testing.

  1. Without stated requirements, no testing is possible.
  2. A software product must satisfy its stated requirements.
  3. All test cases should be traceable to one or more stated requirements, and vice  versa.
  4. Requirements must be stated in testable terms.

When we think in terms of risk, however, I believe a richer set of ideas emerges.

(I am again leaving out the source to avoid bias.  You can easily search for the pdf article. )

If you haven’t realized it by now, keep in mind there are no “requirements” in waterfall!!

In the “waterfall” world (pre-agile), there has been some good work on “requirements”.  Making the transition from that world to a world with “user stories” leaves many questions open.  I still believe, with a good agile coach, and a good testing coach, teams are better off making that transition.  However, writing stories and trying to determine what might go wrong can require a lot of work.

Agile Testing Days 2012 Buzz

Agile Testing Days 2012 is being held in Berlin on November 19th.  Here is my concise summary.  I have included links when available.  Some of my summaries are so concise that you’ll have to stare at the words for a few minutes.  If you get tired of staring, click on the links.

Management Successful automation Transition to agile
Ambler on agile Legacy code? Performance
“Mindful Team Member: Working Like You Knew You Should” Mindmaps Spec by example
Really hands-on no ppt Ambler on agile “5A – assess and adapt agile activities”
Data->quality story Endusers Distributed teams
Understanding agile Distributed teams Line Managers
Exploratory What is A testing Agile->Culture
World of Warcraft Communication Developers exploratory
“Self Coaching” “How to change the world” Factory requirements
Shorter releases Good news CI experience
New techniques Continuous Delivery Test developer
Markus on 21st century tester Advanced CI “Testers Agile Pocketbook”
Continous testing Sapient “Reinventing software quality”
“Fast Feedback Teams” User stories RIA BDD
Test Oracle Slack=creativity Requirements+testing
Javascript TDD BDD Open source tools
Test data Change “Technical Debt”
Bank context driven “The ongoing evolution of testing in agile development” Rigid environment
Right thing right Mobile automation CI
Winning the game

How I learned agile (far from done)

I started on agile late (yet again!!).   When trying to understand agile it is difficult to search for conceptual information on the web or in books.  You can search for standard terms such as TDD or pair programming.  I’ve listed the important sources which helped me understand agile.

James Shore has a great blog post on how he chooses different types of tests.  What is interesting is the sentiment that developers want to prevent defects (no matter what).  James provides his blueprint on how he plans to do that. (Sarcastic comments about developers won’t be appreciated here).  James uses exploratory testers (I think he means manual testers) to check if his process is broken.  I have seen many XP developers make such claims.  Without judging what they say and not having worked with such developers, I think just having such a mindset is great (it doesn’t matter if good testers find critical defects).

After sulking for months when developers in agile teams weren’t like James Shore, I was advised by Dave Nicolette on linkedin, that he would ‘let developers make that choice’ – what is their mission and how they will accomplish it.  You can’t hold a gun to a developers’ head and tell him that developers should have a mission statement like James Shore?  Not sure if I completely agree.  I wish someone would convince me.  I know Dave’s correct.

Brian Marick has a nice series of blog posts on agile testing.  Brian understands testing/exploratory testing and is generally a super sharp guy.  In circa 1995, Brian described how he would staff an agile test team – a few exploratory testers along with testers and developers (not his exact words).  From all I have seen (I’ve read it all), this perfectly summarizes how testing should be done in agile teams (not appealing to authority).

Somewhere along the line, I happened upon the context-driven testing school and the related yahoo newsgroup.  This group of testers exemplifies smart.  I also realized that “Lessons learned in software testing” (Kaner, Bach, Pettichord) is a scripture on testing.

Skill matters.  I figured this out from the responses on the yahoo newsgroups on scrum and xtreme programming.  Also gently reminded by Ron Jefferies wielding a sledge hammer on the scrum newsgroup.  (As an aside, both newsgroups are great sources of information.  The scrum newsgroup is probably more forgiving of silly questions (I know))

The one contribution which stands out from James Lyndsay’s excellent paper on agile is that in testing skill matters.  Of course, today many smart testers know this.  Process is not a proxy for skills. (All that remains is to define what you mean by testing skills – see Chapter 2 from the only book on testing you will ever need to read).

Lessons learned in Software testing explains the difference between iterative and waterfall
Lesson 165: Waterfall lifecycles pit reliability against time
Lesson 166: Evolutionary lifecycles pit features against time
If you have a task which has a fixed deadline/scope, you shouldn’t use an evolutionary lifecycle!!

Can you use scrum for non-development activity?  This was an important conceptual question for me.  Jeff Sutherland  has some articles on using scrum outside the development environment.  I think, in the development context, you shouldn’t bother.  If you want to learn agile, start with a meaty development project.

The term exploratory testing is used frequently among agile bloggers/writers.  What is exploratory testing?  Smart testers always (mostly) explore, i.e., testing is synonymous with exploratory testing.  (Does it mean you aren’t smart if you don’t explore – not necessarily.  If you tell me what you do, I can comment on whether you are a smart tester or not.  You may be doing exploratory under the hood.)  To understand exploratory testing look at Cem Kaner’s video (download this if it doesn’t load in the browser) and note that he has been doing exploratory testing for decades (it’s not new).  (Aside: If you make statements such as, “…do you create test plans in agile…”, you first need to figure out what you do/should do before agile/in waterfall.  You might have been wrong about testing all along).

Test-driven development is an extremely compelling concept in agile/software development.  If you are a tester (or developer) you should understand TDD.  The only way you can understand TDD is by actually trying to use it.  Use Brian Marick’s book to get a feel for TDD (if you are a developer there are plenty of choices).  Also note that TDD is not about testing, it is about design.

In James Shore’s book, Elizabeth Hendrickson has given a nice account of how an exploratory session looks like.

I love Kent Beck’s writing, especially “Extreme programming explained“.  If you do anything related to software development, you should own Kent Beck’s book.  This book gave me a great perspective on agile.

Mike Cohn is another amazing writer/thinker on agile.  He has two excellent books.  I read a lot of his books and website.