Why Project Management Methodologies Are Often Incompatible With Innovation

You have probably heard engineers or scientists say, “You can’t schedule innovation.”  This is their first line of defence (or excuse) when reporting a schedule delay on a new product development project.  And of course the rejoinder by senior management is, “We have a window in the market and we must launch the new product in that window.”  The project team then goes into turmoil and often things get worse to the point of project failure or a major reduction in project scope in order to get “something” released to the market during the market window.

This scenario plays out again and again in many companies.  And the reason is because the disciplines of project management are incompatible with the realities of innovation.  A recent article by Hill, Brandeau, Truelove, and Lineback in the Harvard Business Review, “The Capabilities Your Organization Needs To Sustain Innovation,” explains the key components of an innovative environment in companies.  They use both Edison’s Menlo Park from the last century and modern day Pixar as examples.

In their article they highlight three capabilities of an innovative organization that work synergistically with each other. However, these three capabilities contradict some of the project management “best practices” that are embedded in many company’s New Product Development (NPD) project management methodology.  Let’s take a look at each one.

Creative Abrasion

The authors use the term “Creative Abrasion” to explain the collaborative problem solving process.  They say that this process “involves and should involve passionate discussion and disagreement.”  This can be emotionally difficult for some and will often be perceived as conflict on the team. 

While the authors say that this good and needed, many NPD methodologies seek to remove and eliminate this type of debate and conflict.  The reliance on specifications, contracts, and signed agreements serve to close off debate.  The problem is defined; the solution is defined; there is no reason to discuss anything.

Creative Agility

The authors use the term “Creative Agility” to explain the trial-and-error discovery process.  A key element of this capability is to quickly move from one experiment to the next, taking the lessons learned in one experiment and building on those in the next.  However, the planning process in most NPD methodologies requires that the project team define all of the tests and experiments that will be needed during planning. Of course they don’t yet know the results of the tests and experiments, so they must assume what the results will be in order to plan any follow on tests. 

This creates several problems for the project team.  First, they must explain to management why they need all of the tests and experiments in order to get their plan approved.  It is difficult to defend what you can’t define, so the schedule and budget are often under-estimated.  Second, if they start their experiments and the results lead them down a different path than what was in the plan, they either must follow a path they know is wrong or change the project; which is often a bureaucratic headache and viewed as a failure by the team to plan well.  So to avoid these problems the team plans a project with little or no innovation, leading to an inferior product that does not meet the product performance goals, but does meet the cost and schedule goals.  While being applauded for their project management skills, the actual project results are a failure. 

Creative Resolution

The authors use the term “Creative Resolution” to explain the process of integrating ideas in unexpected ways to create an innovative product.  This integration process requires an openness to explore ways to integrate multiple incompatible options into a solution that has capability that goes far beyond the capability of any one option.  This normally requires time to create and test alternatives that integrate the various options in new and unique ways. 

However, NPD methodologies often require a solution concept to be determined early in the project, before significant testing or analysis is done.  This is part of good project risk management, to resolve the risk questions as early as possible. In most cases, this leads to the selection of a concept that already exists and is not truly innovative.

Rethinking NPD Methodologies

So how do we modify NPD methodologies so that they embrace these innovation capabilities? There are two approaches. 

One plan is to separate Innovation from NPD.  Have an innovation methodology that focuses on encouraging that maximizing these capabilities.  The result of the innovation project is a proof of concept with a working prototype and analysis that is used as the starting point for NPD projects.  Some organizations with stand-alone R&D centers, like General Electric, use this approach.

The second plan is to change the existing NPD methodology to eliminate the prescriptive planning and forced low risk decision-making and allow the project teams to innovate using these innovation capabilities.  The Agile/SCRUM approach is an example of organizations attempting this with software development.

In both cases it will take senior management leadership to lead the way.  They must determine whether they want innovative new products or they want to be the fast follower to the market.  If they want innovative products, they need to change the NPD approach, which includes how they plan projects and the oversight expectations for the project team.  Finally, if they change their approach to rely on these innovation capabilities, they must also ensure their people are trained and have the resources to actually do creative abrasion, creative agility, and creative resolution.