I’ve moved this blog to http://revelutions.com/counterfactuals
Here is a list of my curation of tweets from the software testing conferences in 2013. I haven’t included the smaller conferences. Let me know if I have missed any conference – I can include it next year. Let me know if you would like me to create a similar list for any other conference/topic.
The most common misunderstanding about software testing is that testers seek and find truth (corollary – testers find faults). They don’t — they make and test models.
Kepler packing Platonic solids to explain the observed motion of planets made pretty good predictions, which were improved by his laws of planetary motion, which were improved by Newton’s laws of motion, which were improved by Einstein’s general relativity. Kepler didn’t become wrong because of Newton being right, just as Newton didn’t then become wrong because of Einstein being right; this succession of models differed in their assumptions, accuracy, and applicability, not their truth.
This is entirely unlike the polarizing battles that define so many areas of life: either my political party, or religion, or lifestyle is right, or yours is, and I believe in mine. The only thing that’s shared is the certainty of infallibility.
Building models is very different from proclaiming truths. It’s a never-ending process of discovery and refinement, not a war to win or destination to reach. Uncertainty is intrinsic to the process of finding out what you don’t know, not a weakness to avoid. Bugs are features — violations of expectations are opportunities to refine them. And decisions are made by evaluating what works better, not by invoking received wisdom.
These are familiar aspects of the work of any scientist, or baby: it’s not possible to learn to talk or walk without babbling or toddling to experiment with language and balance. Babies who keep babbling turn into scientists who formulate and test theories for a living. But it doesn’t require professional training to make mental models — we’re born with those skills. What’s needed is not displacing them with the certainty of absolute truths that inhibit the exploration of ideas. Making sense of anything means making models that can predict outcomes and accommodate observations. Truth is a model.
This post is a copy of Neil Gershenfeld‘s response to the 2011 Edge question. I replaced the words ‘science’ and ‘scientists’ with ‘software testing’ and ‘testers’ in the first sentence. I also added the phrase in parenthesis in the first sentence. You could replace the title and the last sentence with ‘Expected results is a model’. This should explain why testers don’t/shouldn’t try to determine (or even worse prove) whether the software works. It also describes that ‘bugs’ are not an indication of (human/developer) failure ( When working with good developers, very few defects are of that category).
The agile movement’s principle of making developers responsible for testing is a good idea. While a developer can be confident of his technical skills (in programming/computer science), meeting a user’s needs, working with complex technology and large systems requires an empirical approach as followed by scientists.
I wrote an article for Tea Time with Testers with concepts similar to this post – a collection of links from the edge’s 2011 question on how to think about software testing. If the link doesn’t work, you can go to the archives of Tea Time with Testers (http://www.teatimewithtesters.com/#!magazines/galleryPage) and look for the July 2013 issue.
I have been curating tweets on software testing. I create a weekly list of popular tweets. I also try to curate tweets from conferences.
You can subscribe to my rss feed for regular updates.
Subscribe via email:
Subscribe to The Weekly Scowl – Popular Tweets on Software Testing by Email
Let me know if there is an event for which you want me to analyze data. Feedback on the tweet collections is welcome.
In the book “The Lean Startup Method”, Eric Ries, as CTO/co-founder of IMVU, describes an early experience with the startup. After months of product development they realized that their customers weren’t interested in their product. He realized that while he could console himself with the benefit of learning from the failure, such learning was little consolation to employees and investors.
The key concept for startups is what Ries defines as “validated learning”.
How do you validate learning?
You create increments of your software which you release to customers. You track the response of specific groups of customers. For example, you should be able to generate data which shows that after introducing the feature ‘simple login’ in February, the adoption of customers who used that feature has greatly increased compared to customers who don’t have that feature (as opposed to showing a general graph of customer adoption.)
Ries also introduces the concept of the scientific method and ‘experiments’. This is how he describes an ‘experiment’.
“It begins with a clear hypothesis that makes predictions about what is supposed to happen. It then tests those predictions empirically. “
He gives the example of Zappos, the world’s largest online shoe store:
When starting the store, the founder, took pictures of shoes from stores and posted them online. The question he was trying to answer was: “Is there already sufficient demand for a superior online shopping experience for shoes?” The important takeaway from the example is that you should build a “minimalistic” product to conduct your experiment (as opposed to early e-commerce startups which invested in warehouses, inventory and a final product).
Definition of value
In the Lean Startup, anything which doesn’t result in validated learning is waste. Instead of creating product features, you can create a ‘facade’ (see MVP below). This is then used to get feedback from customers. The feedback is in the form of actual customer adoption, not as a survey or a focus group.
While soliciting customer feedback is an important part of agile, sometimes teams may be a bit forgiving about the feedback they get. Internal stakeholders may be used as proxies for customers (in agile). The Lean Startup is unambiguous in the feedback (validated learning). Teams must be able to check each user story with ‘validated learning’. Without that the story is incomplete.
This is enough to understand the concept of ‘value’ in the Lean Startup. I will write more about the ‘Lean Startup’ later. There are a few additional concepts which are relevant to this post which are explained in the following paragraphs.
Minimum Viable Product (MVP) : A Minimum Viable Product (MVP) is the smallest needed to sell to potential customers. A good example of this is the start of Groupon. Groupon started by rapidly assembling a set of deals without a lot of focus on the presentation. The idea is to get rapid feedback on the concept which can be refined based on customer feedback. (The definition of MVP used in the book is from Frank Robinson of PMDI ).
Startup: What is a startup? The book cover includes a definition of a startup, “an organization dedicated to creating something new under conditions of extreme uncertainty. This is just as true for one person in a garage as it is in a group of seasoned professionals in a Fortune 500 boardroom. What they all have in common is a mission to penetrate the fog of uncertainty to discover a successful path to a sustainable business.” Although this may be partly marketing, the definition includes any type of software project/product. In the book Ries gives an example of Intuit using his concepts. He gives many other example of large companies.
Agile vs. Lean Startup: It might seem that the Lean Startup doesn’t really add anything new to agile. One major difference is that the Lean Startup is insistent on ‘validated learning’. In the case of agile, customer feedback is critical. However, in smaller organizations or for products which are not web based, there may be limited customer feedback. The product owner and the team can also be used as proxies for the customer. Not so in the Lean Startup. The other important difference is Lean Startup’s focus on business returns/value. A key question that Lean Startup teams need to answer is to show growth in existing customers or growth in adoption by new customers – without smoke and mirrors presentations. Similarly the Lean Startup also encourages teams to validate their business model by creating MVPs which are focused on the business and business processes, not necessarily the technology.
Every other day, in the context of software testing, there is a discussion about ‘value’. “Do testers provide value”, “Testers should provide value” and the related, “Can we eliminate testing?” or “Testing is dead”. In many cases, the word “value” is used colloquially (gold is valuable; dirt isn’t) and even casually (value is value), without much thought to what it really means. In the context of lean (manufacturing), value has a very specific definition/meaning. I think it is essential to understand the meaning of ‘value’ in lean, before using the word in other [technical] contexts. In this post I will not refer to software development. For this discussion you can think of creating a product like a bolt in a factory or parts of a car. I will also refer to the process of visiting a doctor.
In the context of lean, any activity used to produce a product or a service can be one of three categories:
1. value added
2. non value added
3. non value added but necessary
There are some activities which are obviously non-value added. In the context of a visit to the doctor, a patient waiting for the doctor is non-value add. Mistakes in a doctor’s lab report, which requires rework, is non-value add. In the context of manufacturing, defective parts requiring rework is non-value add.
The categories for some activities are not obvious, e.g., suppose an administrator delivers lab reports to the doctors in the clinic – is that value add? Can you prevent delivering the reports? Suppose you keep records for defective parts in a manufacturing facility – is that value add? Can you avoid keeping those?
In lean manufacturing, in order to qualify as value-add, an activity must meet the following three criteria:
1. It must change form or function of the product or service
2. The customer must be willing to pay for it
3. It must be done right the first time
Even with this definition, it can be challenging to figure out if an activity is value-added.
In the context of lean, when we discuss if the customer is willing to pay for it, we are not discussing personal preference. It doesn’t matter if the customer likes red pens or not. In the case of manufacturing, whether a customer ‘likes’ an item is a problem of it’s design or marketing. That is not relevant to the discussion of ‘value’. In the case of software, engineers are responsible for the design, and it is important that target customers see ‘value’ in the design. I won’t discuss software in this post. It’s enough to keep in mind that a customer’s willingness to pay (in lean manufacturing) is more related to wasteful activities, such as defects or excess inventory.
There are some activities which are not value-added, e.g., they do not change form or function of products, but which are necessary. For example, sales and marketing of products or services. Although these will not satisfy the three conditions for value-add, a business will not survive without these activities.
Why is value important (in lean manufacturing)?
When people pursue process improvement or try to make processes more efficient, they focus on the small percentage of value added activities – “make it faster”. Instead, eliminating NVA can give much higher gains with less effort. Typically only 10 to 15 percent of steps in a process add value, and more often than not, these steps represent as little as 1 percent of the total process time (From Lean Six Sigma for Dummies). Consider a typical visit to the doctor’s office, there are large periods of waiting with relatively little productive time spent on filling forms and talking to the doctor. Focusing on reducing the wait times will result in higher gains compared to improving the time taken by the doctor.
Here are some of the ways to eliminate waste in a doctor’s office.
– have broad roles for admin staff, eliminating unnecessary hand-offs
– making sure physician’s have all the information needed to focus on the patient
– avoiding rework by different admin staff, the nurse and physician
– using technology to avoid mistakes and rework
Identifying activities which create value is the first step in applying lean (More on lean later).
The concepts of lean can be applied to any industry. If you would like to know more, you may want to start with understanding how it is applied to health care/hospitals/doctors. If you don’t have experience with manufacturing it may be easier to understand how lean concepts are applied to a doctor’s office. There are many references on the subject available on the Internet.
Tivoli is one of IBM’s Software Group brands. IBM’s “Software group” is the business which creates software products. Tivoli has an annual conference called PULSE. IBM’s conferences are a great way to get insight into the industry and current trends. Here is my first attempt at making sense of the conference.
This is not written to promote IBM. I wrote this more for software testers/developers. If you work with enterprise software this might give you some ideas for your future career. I am writing this at a very high level. This is not a discussion of IBM's strategy. You can always refer to the main conference site for more information.
Pulse will be held in Las Vegas from March 3-6. The tag line for PULSE/Tivoli is “Optimizing the world’s infrastructure”.
Tivoli‘s official tag line is: “Manage your business infrastructure in real time.”
IBM Tivoli Software helps clients optimize the value of their business infrastructures and technology assets enabling greater visibility, control and automation across their end-to-end business operations.
A simple way to understand this is that Tivoli helps manage, improve (optimize!) your computer/hardware/software (you are working in an enterprise). However, now you can extend that concept to include managing machines like POS terminals, ATMs, ….everything….any computer related infrastructure.
Pulse 2013 has 4 streams:
- Cloud and IT Optimization
- Smarter Physical Infrastructure
- The Mobile Enterprise
- Security Intelligence
In this post I will be focusing on the mobile stream. The mobile stream has 55 sessions. There are 3 tracks:
- Becoming a mobile enterprise
- Managing and Securing the mobile enterprise
- Best practices and case studies in endpoint management
In the first track, “Becoming…mobile”, the focus is on better understanding mobile, the issues, what it takes to become mobile, tools, and information on various IBM products. These are some of the issues:
- Mobile Security
- Agile and Devops
- Mobile and Cloud trends
- User-Centered Design
- Managing multi-channel web experiences
- Mobile application testing
- IBM’s Collaboration strategy for mobile
The managing and securing track is about managing devices in a large/diverse enterprise and ensuring security.
- Managing BYOD
- Network management
- Social businesses are using mobile and cloud to enable collaboration and context. Existing business can also use these strategies to make better use of their resources. (read the session info – MME-2106)
- Using IBM Security Access Manager and IBM Tivoli Federated Identity Manager to address the BYOD Challenge
- Partner product to manage and secure two OS’s side-by-side on a mobile device
Some of the main issues addressed in the best practices and case studies track include:
- Large windows 7 migration
- Acquisition management with endpoint management
- Endpoint Manager integration with “pervasive smart home appliances”. Also “medical instruments, security devices, energy information systems, and smart equipment.
- License optimization and compliance
- Toshiba/IBM partnership to manage Toshiba devices
This is a good point to stop if you are overwhelmed or bored !! In the rest of this post, I list some of IBM’s products and services which are mentioned in Pulse2013’s mobile stream.
The following list includes most of the IBM products/services mentioned in the mobile stream in Pulse 2013. If you aren’t really into this market, I would recommend at least looking at IBM Endpoint Manager – just a nice product/technology and IBM Worklight which can be downloaded for free.
IBM Cast Iron
WebSphere Cast Iron Cloud Integration enables companies to rapidly connect their hybrid world of public clouds, private clouds, and on-premise applications.
Nilanjan says: Cast Iron allows you to quickly connect to various enterprise applications such as Salesforce and Microsoft Dynamics.
IBM Endpoint Manager
IBM Endpoint Manager for Mobile
IBM® Endpoint Manager can help you achieve smarter, faster endpoint management and security.
Nilanjan says: Endpoint Manager is a neat product which manages everything on endpoints (including mobile) in enterprises.
IBM Worklight Developer Edition. Download nowIBM Worklight provides an open, comprehensive and advanced mobile application platform for smartphones and tablets, helping organizations of all sizes to efficiently develop, connect, run and manage HTML5, hybrid and native applications. Note: IBM Worklight can be download free.
IBM Mobile Foundation
– an integrated package of Worklight, CastIron and Endpoint Manager
IBM Interactive is a leading interactive agency with a unique ability to imagine, discover, and deliver compelling user experiences.
Redbend Software’s TRUE solution
“The answer is Red Bend’s TRUE™ Solution for BYOD – an end-to-end solution for mobile device manufacturers and service providers to create and manage virtualized mobile devices that are “TRUsted by the Enterprise” for employee use.”
IBM Web Experience
IBM web experience software provides access to critical information and applications that is personalized to users’ needs – available anytime, on any device – to deliver an exceptional experience.
platform for versatile authentication, federated identity management and single sign-on across Cloud and Corporate resources
IBM Virtual Desktop
allows access to a personal PC desktop from anywhere, using any device, using the Virtual Enterprise Remote Desktop Environment (VERDE) from Virtual Bridges