Is There a Disconnect Between Knowledge Workers and Business Leaders?

The past two weeks I’ve read a number of articles about the End of Agile and subsequent rebuttals. Yesterday I read an excellent article asking “Whatever Happened to Six Sigma?” Both of these articles are fascinating. The rebuttals to the first are incredibly enlightening. I don’t think these are the last of these sorts of articles. I expect to read “The End of Lean” down the road as well.

I think there are a few reasons, one of them is common between Six Sigma and Agile. Snake Oil Salespeople. Basically, what has happened with all methodologies (except for PMI’s Waterfall approach – which I’ll get to) there reaches a point in time where it becomes impossible to determine the quality of credentials for a given certification. At that point, the certification is valueless even if the training, like you dear reader received, was actually the top of the top. Because there’s no actual way to determine if the quality is any good or not.

I experienced this first hand while I was teaching Lean Six Sigma at AMD. I had some fantastic mentors when I was working there, that lived and breathed Six Sigma for their entire careers. They knew this stuff inside and out. Which made me gain a much deeper appreciation for the methodology. However, there was another small company in Austin that also taught Six Sigma to their employees, Dell. Whenever we hired people from Dell with a certification in Six Sigma, a Green Belt or even a Black Belt, we essentially had to retrain them. Many of them, did not truly internalize what they were taught, or the material was less rigorous. This was likely a trade off the Six Sigma training team had to make to ensure their team remained relevant to Dell.

However, whenever you cannot trust the training from an organization like Dell, it makes it a lot more difficult to trust training for any other organization. You just don’t know the standards. Agile’s currently experiencing the exact same issue. There’s been a huge influx of organizations giving certifications. Not all of them have the same level of quality.

I think as a reaction to this, the software development industry has created DevOps and DevSecOps. Which doesn’t have a certification process, but a general set of ideas, such as Trunk development, rigorous testing, continuous integration, and on the extreme continuous deployment.

I think all this goes back to a basic premise though. Knowledge workers, like engineers and software developers look at problems very differently than business leaders. I first experienced this while I was in college. I was studying Industrial Engineering (which pulls in elements from Six Sigma, Lean, Network Theory, Simulation, Human Factors, etc…) while a good friend was studying Business. We had a few conversations about how businesses should be run and it was very obvious to me, that we were talking about two completely different views of how a firm should be run.

I was arguing against (in 2003) off-shoring, because it decreased the efficiency of engineering and collaboration between manufacturing and engineering. Both Agile and Lean argue against off-shoring due to these reasons. Given the change in the approach, the salary savings, overall, didn’t make the effort worth it, because of the reasons I listed. My friend thought lowering cost was the right thing to do.

This isn’t just an anecdotal thing though. If you read books about Agile, Theory of Constraints, or Innovation, they all make the same arguments. The ideas taught in business school are causing business leaders to make bad ideas. Theory of Constraints was popularized in the book The Goal by Eliyahu Goldratt and came out in 1984. The ideas he espoused in that book were considered counter intuitive. If you read the Phoenix Project by Gene Kim which came out in 2013, which is written to model The Goal, you’ll find the characters running into, literally, the exact same type of thinking by managers and other team members. To me, this means there are other cultural organizations that are pushing back against the approaches technical leaders find work best and what our business leaders find work best for their goals.

The two most obvious cases for this are Accounting (which The Goal sets up as something of an antagonist) and Executives. Accounting has the weight of Law on its side, which is problematic, because Accounting organizations has their own reason for maintaining a status quo. They have their own certification process to become a CPA. From the executives standpoint, in many cases these folks are presented as having an MBA and excellent business training.

Despite that, they are still making poor business decisions for the technical team, poor decisions about how to structure their organization, and poor decisions about how to run projects. Most projects are run using a Waterfall approach, because that is the defacto approach we’re all taught throughout school. We manage to dates and push to get things done. The Project Management Institute has managed to corner the market on this approach, because most people can “do waterfall” without needing a certification. You learn through osmosis, by doing. The certification certainly elevates some PMs in some organizations. However, I don’t really think that having a PMP matters to most hiring managers.

So, where does this leave us? It leaves us with Knowledge Workers using ideas like Six Sigma, Lean, DevOps, Agile, and more to dress up their structured problem solving approaches to add structure and credibility. They need structure to compete with the Date Managed Waterfall approach. They need credibility of a methodology to put their approach on the same level as a PMP certification. In the case of Six Sigma, I’d argue its biggest success was internalizing the cost saving analysis. This helped translated the output in terms of money, which executives understand.

Every other approach tries to create an alternative measure. Elimination of Waste or “Working Software is the Measure of Progress” are nice alternatives to reducing costs or meeting due dates.

However, most share holders don’t care about that. They only care about what’s going to make them more money. Ethics and approach be damned. Until that is resolved. We’ll continue to have more fads or business fashions, as Knowledge Workers push back against Business Leaders.

Enabling Innovation Through Lean Improvement

This is part of my ongoing Lean Disruption series where I write about how to combine Innovation Theory, Lean, Lean Startup, Agile, and Lean product development.

Since values and metrics drive processes how can you enable innovation in an organization that isn’t completely willing to support innovation? As I mentioned before Skunkworks are a key component in this process. Using emergent strategies with Lean Startup tools such as the Lean Canvas are another key method to enable innovation with the skunkworks approach. However, not every organization can or will truly allow this approach.

In organization that cannot or will not support a skunk works approach to innovation what options are available? In the case where you are a manager of more than yourself you have the ability to implement Lean within your small organization. Owning the processes within your organization and becoming a coach to your employees, can begin enabling metrics that drive customer centric focus.

This is possible by taking a single step and picking a metric that will move the metrics that your bosses care about. Focus on improving one process at a time. Start with something that is completely in your control, such as how you hand off material within your group that goes to other groups. Measure the number of errors your group introduces, however do not blame people. If things are wrong investigate the root cause of the problem rather than accusing your people of causing the problems. Problems can come from the group that handed off the work. In those cases, create a process to review the work looking for errors and then work with the team supplying that work to eliminate those errors where you can.

In the case where there are errors within your team, you work together to uncover each source of the problem. As a manager you own the processes as much as the result of the work your team does. The final output of your work is 100% dependant on the processes that lead to those output. Another important step is to work on converting your daily meeting, assuming you have one, into a daily standup rather than a typical beat and greet. These meetings instead must cover the goals for the day, issues from the day before, and follow up to address those issues. In the case where the team has projects they are working on, this can become an opportunity to being pulling in some agile stand up practices, such as “What did you work on yesterday, what do you plan on working today, and do you have any impediments?” These questions can help change the way projects are run within your group.

To improve processes within your group, the best way to do this is to go to where work is being done. You must actually watch as work is being done, while keeping in mind that the way work is being done will change as you watch people work. Make sure they are aware that you are not going to be critical or even record who is doing the work at the time. Record the information and move on. You should not use this information in their next 1 on 1 or review. The goal is to learn how people are doing work especially across work types and workers. After you map the process review the results in your daily stand up to discuss potential issues. Partner with your team to find the root cause of the problem and empower them to make changes to their processes.

Before you change the process it is important to measure current state and measure the impact of those changes on the metrics that matter to you. Keeping in mind that you must make all changes from a customer’s perspective. These are the easier things to implement. However, making changes and getting support in this change from your leadership isn’t going to be easy, so you must show that you’re making wins and improving the result of your work. That’s all leaders care about.

Successful process improvement opens the way to broader innovations. Start small and then move into larger projects as you have a better understanding of what you are trying to do.

Values and Metrics Drive Emergent Strategies

This is part of my ongoing series on Lean Disruption. Where I write about combining Innovation, Lean, Lean Startup, Agile, and Lean product development methodologies.

Clay Christensen argues that there are two types of strategies corporate leaders engage with, deliberate and emergent. Porter’s 5 Forces analysis is an attempt to use tools to pull emergent strategies based on changing environmental landscape into the corporation’s deliberate strategy. A deliberate strategy is the strategy that leaders have vocalized and intentionally invested money and resources into. Emergent strategies on the other hand are strategies that develop through metrics and actual organizational behavior. While a leader may intentionally push resources one direction another metrics that has much more value to lower level managers may require those resources to be redeployed in another context. Resulting in a different strategy to develop than what executives had originally planned.

Hoshin Kanri (Policy Deployment) plays a similar role to the 5 Forces in the Toyota Production System where leaders start with their stated 3-5 year goals and turn those into annual goals, projects, and finally the metrics by which those projects will be measured. However, the process isn’t done after a single meeting, this policy is reviewed monthly and if conditions change enough can be completely reworked or modified based on what conditions are emerging. This is important because it can, in fact, feed changes the whole way back at the the 3-5 year goal levels where if serious issues are occurring in multiple projects associated with a goal, such as lack of resource commitment, that goal must be re-evaluated or there needs to be other changes to incentivize resource commitment to those projects.

The Lean Startup and Agile approaches are likely the most closely tied to emergent strategy development. The Lean Startup approach values experimentation and customer engagement above all else which can result in initially a great deal of change in project/corporate strategy. In the Running Lean the author uses the Lean Canvas as a tool to maximize the power of emergent strategy develop and smooth the transition from emergent to deliberate strategy development. In many cases that transition is relatively easy anyway, however it is possible to see that transition occur as a corporate leader iterates through versions of the Lean Canvas resulting in less and less changes to the Canvas.

Agile similarly promotes engagement with customers and using iteration to eliminate uncertainty. In this way Agile is closer to Lean Startup than traditional Project Management and leads to emergent based products. Where the customer need is truly met. Which, over time, results in a deliberate strategy to maximize the resulting product. This product is still, likely, within the initial deliberate strategy of the leaders of the company, but may be very different than what the leadership had initially wanted or planned. This is the best of both worlds as the leaders get a product that fits their strategy, but is more effective in serving market needs than what they ever could have planned.

The values and metrics the organization uses to manage the work that it does heavily influences the direction any project or product develops. The tighter the control over metrics with less flexibility for innovation leads to more tightly aligned products to deliberate strategies. However, this can come at a cost of less innovative ideas and poor-market fit. In the case where something might significantly change the direction of the company, for that product to survive it is best to move that into a skunkworks or protected space where funding is secure with appropriately aligned staffing levels. This will allow the metrics and values to coalesce around the product, the customer, and the market needs.

Values in an Agile/Lean/Innovative company

This is part of my Lean Disruption Series where I’m looking at Lean, Agile, Innovation, and Lean Startup.

None of these methodologies can be adopted for free. They require a great deal of firm introspection. Understanding how processes interaction with people and values is vital to adopting any of these approaches let alone a combination of these approaches.

Metrics are one of the best examples of how there can be conflicts between stated values, values in making decisions, how resources are handled and how processes are structured. The famous saying “You manage what you measure” is right in a lot of ways. Many companies claim that they value customer satisfaction, however many of these companies do not actually do anything with the satisfaction surveys they do get. Comcast is the most obvious example of this. Comcast doesn’t really value customer satisfaction because they measure their customer support on how much they can upsell to the customer anytime they are on the phone. This changes the processes their customer support must use, rather than designing processes to enable single call resolution, their processes are designed to enable selling more products. Their employees, the resources, are rated based on this and if they don’t meet those goals they are unlikely to do well. Considering the Verge’s Comcast Confessions series most of the resources at Comcast do not feel valued. This all points to the true values for Comcast being retention at all costs and more revenue per user measured in Churn and ARPU (Average Revenue per User) respectfully.

Agile Manifesto from ITIL’s blog

For a company to adopt an Agile approach to developing software, the paradigm of what the organization values must radically change to align to the Agile Manifesto. In most software development the concepts on the right are what are valued through a Project Management Office. The concepts on the left are typically considered only at the beginning or the end of the project or not at all. Working product is the goal of a project, while customer collaboration inclusive only in the beginning getting requirements.

Switching from the right to the left creates massive cultural upheaval at an organization, where power is shifted down and out. It is shifted down to the team level, where managers in the past made all the important decisions Product Owners, Scrum Masters, and developers make the decision now with the customers. Power is shifted out through increased collaboration with the customer. Customer centricity forces the company to understand what the customer really wants and more quickly respond to changes in their understanding of their needs. This does mean that the “requirements” change, however, in many cases due to the uncertainty in a technology, interface, or some other aspect it was impossible to properly articulate the actual need until there was an example in front of the customer.

With these value changes there must be process changes to that properly reflect the change in the way the values require work to be completed. In the case where Single Call resolution is the most important metric reflecting the value of true customer satisfaction, processes must be built to enable that – such as training, information repositories, and authority to truly address customer needs at a single point of contact. In software development rapid iteration with continual feedback is a process that must be built to enable that.

This changes are not free and require true commitment from leaders across the organization. Without their commitment any adoption of these frameworks is doomed to failure.

The Power of Kanban

This is part of my ongoing series on Lean, Innovation, Lean Startup, Agile, and disruption.

Kanban is a tool in Lean which is used to help send messages downstream without needing to actually talk to each other. Which is powerful in a manufacturing setting as communication might be difficult depending on the environment. Kanban translate as card from Japanese and typically has information related to the number of items that need to be in a bin and what the item is that needs to be in a bin.

Example of a Kanban board on Trello

Example of a Kanban board on Trello

Some Agile practitioners have adopted Kanban to be a set of steps in the process for developing software. This could be as simple as Ideas, To Do, Doing, Completed, or as complicated as Backlog, Wireframe, Coding, QA, Testing, Integration, Complete. Ideally, simpler is better, because more bins make management of the lists much more complicated than if there are that many phases in the Kanban board, but this is really up to the preference of the team and whatever works best based on the Build measure Learn approach.

Even developing a Kanban system for software development the team should view it as another possible to learn how to work with each other better. This includes the information that’s required for a piece of work to be started. For example, determining what information is the minimum for a User Story is something that must be agreed upon by the team. Furthermore, the Acceptance Criteria can help reduce the number of steps on the Kanban board as these may simply be part of the acceptance criteria for a complete story.

Using stories and Kanban board can help with Innovation through creating bite size pieces of work that can be designed to reduce the environment of uncertainty the product is being developed in. Since uncertainty is the greatest risk to the project it is vital to write user stories and use the kanban board to maximize efficiencies with developing a new product.

The Kanban board is also extremely effective for ensuring that startups keep their energy focused on the most important thing. No work should be done on an item if the work is not on the Kanban board. If the work is vital to the startup, then it needs to be recorded and validated that it’s the right thing to be doing. Yes, it might slow things now, but the Kanban board is an effective tool to help people understand who is working on what and how those interact with other problems out there. This doesn’t meant that immediate issues shouldn’t be resolved immediately, but for development purposes no new work should be done without having an item on the Kanban board.

Agile and the Lean Startup

This is part of my ongoing series devoted to understanding the connection between Disruption Theory, Lean Startup, Lean product development, and Agile software development.

The Lean Startup is most likely the best example of how to translate Lean into software development directly. This set of management tools uses nearly all the Lean tools identified in the famous texts of Lean, such as Toyota Way, Lean Thinking, and Toyota Production System. In fact there are so many similarities between the two people have accused the author of basically writing a book about Lean and not adding anything to the topic. I, personally, don’t agree with this as the portions about pivoting is definitely not part of the Lean framework and is novel capability he introduced.

Furthermore, he’s had to translate a great deal of the language from manufacturing or office settings into a development setting. While there are similarities there are differences between a piece of finished goods and a completed piece of code. This difference is that the code can be pushed to “production” servers and active for users, while the finished good has to be purchased by a customer, which still may take weeks to occur (if the production system is linked to purchase orders and directly to customers). To that end Reis pushed to make as many changes to production as quickly as possible – not without testing of course, but as soon as testing was completed the change went live. This is an extreme case of single piece flow. One change goes through testing and pushed live.

Pushing in the “Continuous Deployment” model of course causes a lot of issues if you don’t have robust testing. You need the testing to be fast but comprehensive. It’s impossible to do this from the beginning so over time the team has to develop more and more sophisticated tests based on how the team breaks the existing piece of software as they deploy changes. This is where Root Cause analysis plays a key role. Do a 5 Why’s analysis (ask why 5 times until you come to an underlying root cause) fix that problem so it never happens again. In the case where you cannot prevent the issue then you need to be automatically testing for the issue to fix it before you publish the change.

Agile has similar concepts in that they there are short cycles of development called “sprints” these sprints can last anywhere from a week to a month. At the end of each sprint the goal is to have a series of features that are deploy-able (including testing) that a customer would be willing to pay for. These pieces of work are broken down into something called a “minimum viable feature” which is smaller than a product but not so small that it wouldn’t completely work. For example, an MVF of WordPress would be the “publish button” initially it would need to convert all the text in the text editor and turn everything into a viewable website and create the URL associated with that article. Over time new features can be added like the auto-tweeting capability that exist now.

One common theme between both approaches is the concept of minimum viable product, which is a set of features that makes up the product. The difference between the two approaches in developing the MVP is that the Product owner does this in Agile, while the MVP is determined through a great deal of customer interaction in Lean Startup (Ideally the Product Owner is doing a similar activity for the customer but typically acts as a proxy).

There hasn’t been a lot of authors that have looked into combining these two approaches. There are a lot of similarities as they both emerged from the Lean philosophy, which means they should be fairly easy to combine into a larger framework. The only book I’m aware of that has even attempted to tackle this is the “Lean Mindset” by Tom and Mary Poppendieck. Even in this book though, they have decided to go with a 4 step approach that has more phases than what the Lean Startup recommends. They propose the stage gate to help determine minimum levels of details. I believe that this is because of the typical levels of bureaucracies in organizations, where a Project Management Organization will require adhering to some sort of phased gate approach. The Lean Startup uses Pivots as a way to manage projects that are failing, but not with the same level of rigor or structure as the approach the Poppendiek’s recommend. Instead, the Lean Startup approach recommends setting goals and targets for metrics (non-vanity like number of views) determining when to pivot. The timeline has to be long enough to get a clear view of what your customers actually want.

Adoption of Agile compared to Lean

This is part of my ongoing series devoted to understanding the connection between Disruption Theory, Lean Startup, Lean product development, and Agile software development.

In many organizations the adoption of Agile software development techniques have been adopted rather quickly. For those unfamiliar with Agile software development, the point is to focus on user experience through developing software that is created through user stories (there’s a lot more to it obviously). Agile grew out of an understanding of Lean practices, where there is a strong focus on learning, daily standups, and understanding where and why things went wrong. This includes the use of the 5Whys which was highlighted in the Lean Startup. In away the company becomes a learning organization as there is a set meeting every sprint to reflect on the last few weeks of work. Agile was initially conceived around the turn of the century and has seen steady adoption over the past 15 or so years.

Many of these ideas came from Lean. While retrospective isn’t specifically called out in Lean, it happens consistently through a Plan Do Check Act cycle. Furthermore it happens through Root Cause analysis which happens on nearly a daily basis whenever issues arise. Much of the most ardent Lean practitioners are found in manufacturing, however it is making headway in other areas of the organization over time. Lean was Popularized in the early 90’s by the book “The Machine that Changed the World” but is much older than that and had been adopted by Toyota competitors as early as the 70’s.

So, then why, when Lean is the inspiration for Agile (as well as the Lean Startup), has Agile been heavily adopted while Lean has not? I do not mean to say that Lean isn’t used; it’s used a lot of places. However, it’s not used in the bulk of companies around the world or used uniformly throughout an organization. Agile has been adopted as a methodology across many industries and within many different types of companies. It is also not possible to say that one works while the other doesn’t. Both have been shown to improve processes and drive cultural change within an organization. I think that Agile has been more quickly adopted in broader corporate America than Lean because, similarly to Lean, it was made by and for the people that use it the most; Corporate IT. IT/Software development has owned the deployment of Agile practices which has made them much more successful at adopting them than those same corporations adopting Lean practices.

Agile has had the luxury of being successful in many organizations in a very public manner. Which has lead to a lot of top down support and required adoption of the methodology. This makes adopting specific methods for project management significantly easier for adoption of any given methodology. In fact, Lean and Six Sigma deployments are only successful whenever those deployments are attached to specific strategic initiatives. With IT any project that is funded for development is by definition strategic resulting in clear alignment between the Agile deployment and the execution of the project. Lean typically does not have this luxury.

Furthermore, Lean is not well regarded in many leadership circles as it is not typically taught in high level MBA courses. Agile on the other hand has made it into Masters programs for both IT and Computer Science. Meaning that the leaders responsible for owning processes and project results learned about how to deploy Agile in their education and it was a highly stressed practice. On the other hand Lean is part of Industrial/Manufacturing Engineering and to specific business niches. Which means that there’s a clear misunderstanding of Lean and the management practices that accompany the tools.

Coming to Agile from the Lean perspective, I believe that Lean practitioners have a lot to learn from the Agile community around adoption. Much of the actual practices are extremely similar and anyone with a strong Lean background will be able to transition into an Agile environment easily. The transition the other direction will be only slightly harder, mostly because of the broader range of performance metrics associated with Lean compared to a handful with Agile.