Agile and the Lean Startup

This is part of my ongoing series devoted to understanding the connection between Disruption Theory, Lean Startup, Lean product development, and Agile software development.

The Lean Startup is most likely the best example of how to translate Lean into software development directly. This set of management tools uses nearly all the Lean tools identified in the famous texts of Lean, such as Toyota Way, Lean Thinking, and Toyota Production System. In fact there are so many similarities between the two people have accused the author of basically writing a book about Lean and not adding anything to the topic. I, personally, don’t agree with this as the portions about pivoting is definitely not part of the Lean framework and is novel capability he introduced.

Furthermore, he’s had to translate a great deal of the language from manufacturing or office settings into a development setting. While there are similarities there are differences between a piece of finished goods and a completed piece of code. This difference is that the code can be pushed to “production” servers and active for users, while the finished good has to be purchased by a customer, which still may take weeks to occur (if the production system is linked to purchase orders and directly to customers). To that end Reis pushed to make as many changes to production as quickly as possible – not without testing of course, but as soon as testing was completed the change went live. This is an extreme case of single piece flow. One change goes through testing and pushed live.

Pushing in the “Continuous Deployment” model of course causes a lot of issues if you don’t have robust testing. You need the testing to be fast but comprehensive. It’s impossible to do this from the beginning so over time the team has to develop more and more sophisticated tests based on how the team breaks the existing piece of software as they deploy changes. This is where Root Cause analysis plays a key role. Do a 5 Why’s analysis (ask why 5 times until you come to an underlying root cause) fix that problem so it never happens again. In the case where you cannot prevent the issue then you need to be automatically testing for the issue to fix it before you publish the change.

Agile has similar concepts in that they there are short cycles of development called “sprints” these sprints can last anywhere from a week to a month. At the end of each sprint the goal is to have a series of features that are deploy-able (including testing) that a customer would be willing to pay for. These pieces of work are broken down into something called a “minimum viable feature” which is smaller than a product but not so small that it wouldn’t completely work. For example, an MVF of WordPress would be the “publish button” initially it would need to convert all the text in the text editor and turn everything into a viewable website and create the URL associated with that article. Over time new features can be added like the auto-tweeting capability that exist now.

One common theme between both approaches is the concept of minimum viable product, which is a set of features that makes up the product. The difference between the two approaches in developing the MVP is that the Product owner does this in Agile, while the MVP is determined through a great deal of customer interaction in Lean Startup (Ideally the Product Owner is doing a similar activity for the customer but typically acts as a proxy).

There hasn’t been a lot of authors that have looked into combining these two approaches. There are a lot of similarities as they both emerged from the Lean philosophy, which means they should be fairly easy to combine into a larger framework. The only book I’m aware of that has even attempted to tackle this is the “Lean Mindset” by Tom and Mary Poppendieck. Even in this book though, they have decided to go with a 4 step approach that has more phases than what the Lean Startup recommends. They propose the stage gate to help determine minimum levels of details. I believe that this is because of the typical levels of bureaucracies in organizations, where a Project Management Organization will require adhering to some sort of phased gate approach. The Lean Startup uses Pivots as a way to manage projects that are failing, but not with the same level of rigor or structure as the approach the Poppendiek’s recommend. Instead, the Lean Startup approach recommends setting goals and targets for metrics (non-vanity like number of views) determining when to pivot. The timeline has to be long enough to get a clear view of what your customers actually want.

Methodology, managers, and projects

When working on a project there are a few different ways to manage those projects. One is the traditional waterfall approach, which is your top down project where you have to use Gantt charts to figure out how long you think it’s going to take up front, where you’re given a set date that can’t change without a lot of effort to do a certain amount of poorly defined requirements, and a set amount of money to do the project. This approach has been how Windows and many video games have been produced in the past. It’s not really extremely effective and really no one really likes working a project conducted using Waterfall methodologies. There are risks, projects get cancelled and the project management can seem to be capricious and opaque. This leads to lack of trust and belief that management has the best interest in mind for both the project and the employees on the project.

To address these concerns a group of people created the Agile project management methodology. The goal was to value working software over documentation. Which means that each bit of software is broken down into the minimum viable feature, or the smallest piece of working software that could be packaged and used by a customer. The goal is to manage the project through adjusting how many of these features are going to be finished by the go live date. Effectively you build small bits of work instead of finishing one giant massive piece of software.

This approach is effective for other types of technology that have a modular architecture. There’s some minimum viable product, where you need a minimum set of features for the product to actually work. For example a cell phone needs to have a combination of features to function properly. Things like bendable screens would not be in the minimum viable product, but an excellent touch screen would be. These minimum viable features can be modulated based on the Kano model – which is useful for determining if a specific feature is basic, a pleaser, or a delighter. If the feature falls into basic, you must include that feature if you’d like it to be a success. However, those minimum features don’t guarantee a successful product, you’ll need to include pleasers as well as delighters. Those are the pieces of scope that you will be able to eliminate to make sure you actually launch the product on time.

Issues with these projects come whenever there is a mixture of methodologies. When management believes projects must be managed through waterfall through a central project results office while the development team believes the project is being managed through the agile methodology. This creates serious issues whenever there is miscommunication, lack of information, or lack of understanding the real status of the agile team’s approach. This is exacerbated by the required openness in the agile approach (where you are supposed to continually learn from your mistakes and have a conversation about all the problems you’ve had – to fix them) while in waterfall it is better for people to hide and place blame elsewhere whenever things are not going well. Not because people are bad, but the incentives are in place to behave this way. With a single option of go/no go, it’s better to minimize the known risks as if things are misunderstood as going poorly it will drive management to take action. While in an Agile team, discussing the true status of the project is vital through self policing and partnering with other agile teams to address the problem. The greater the likelihood of success of the projects.

This conflict and a switch from governance in the agile methodology can and will destroy the trust the various agile teams have developed. An organization needs to fully commit to a single project management methodology or it will struggle to complete any project within scope and budget and will demoralize the leaders of projects being worked in agile, as waterfall would likely be the methodology that management selects. Leaders of Agile projects should leave organizations that undercut the agile teams, as it will not stop and will have dramatic impacts on their careers in the long run.