The three keys to innovation are to seek new ideas, test these ideas on a scale where failure is survivable, and continuously monitor these trials for feedback. Failure is the path to success according to Tim Hartford’s Adapt: Why Success Always Starts With Failure. Hartford argues that the world is too complicated for top-down “big project” innovation-based purely on expert judgment. The best path to innovation is to try a lot of ideas simultaneously (even if they contradict each other), build in robust feedback loops, and use the winning ideas to start a new round of trials.
Learning from failure is not a new method of innovation; it is the oldest method of innovation around – evolution. Nature is continually creating variations of species and then selecting the species that best survive current conditions. Hartford applies that concept to organizations to see if a similar process works in determining what companies succeed and which close. The organizations that best survive a continually changing business environment combine incremental improvement along with the occasional long-shot idea to propel them into a better part of the business landscape ahead of their competitors
So, what does this have to do with government agencies? Hartford flatly states this innovation method will not work in government agencies because of several barriers. First, there is not enough time for political appointees to fully see these experiments through before a new administration comes in office. Second, the process depends on many failures for innovation, but failure carries a high stigma in government. Third, it is difficult to demonstrate that a policy innovation had an effect due to the lack of robust feedback loops in government.
Hartford’s opinion about government innovation is overstated. There have been numerous government projects that have been extremely innovative: the Hoover Dam, rural electrification, the Interstate Highway System, the Moon landings, the Space Shuttle, the Internet, and so on. When you examine how these agencies developed these projects you do see these agencies tried many ideas and learned from these trials. NASA has an amazing knowledge management culture, and DARPA’s successful record of innovation is built on the concept of trying many long-shot ideas at once.
What holds government back from being even more innovative is the stigma of failure. Many agency cultures are too cautious because of the constant external scrutiny and the internal cultural practices of not sticking your neck out and just waiting out the latest change effort. Often, this caution is well-warranted. Many people depend on government agencies, and thus agencies cannot fail in their primary mission of delivering Social Security checks, defending the nation, or enforcing laws and regulations
But failure to innovate will also lead to mission failure for agencies. In the sixth chapter, Hartford describes how the 2008 economic meltdown was inevitable, given the tight coupling of economic institutions and the failure of the government to prevent financial organizations from becoming too entangled. He argues that in any complex system, accidents will typically occur and that often our failure-prevention efforts will only increase the probability of failure. What is needed are the twin strategies of placing buffers between parts of the system and setting up feedback loops to warn us of emerging failure events.
Government must continually innovate so it can continuously deliver on its mission. Continuous innovation means that the culture must change so the agencies accept the small failures that teach to avoid the significant failures that cripple the agency and harm the people it serves. Whether we call it “experimentation,” “pilot tests,” or some other euphemism, the better the government is at innovation, the better it can serve its citizens.