Taking advantage of emergence for complex innovation eco-systems

Abstract

Our most pressing societal problems such as enhancing health care, developing alternate energy, revitalizing cities, and advancing the economy are complex innovation eco-systems. Complex innovation eco-systems are the next frontier in technology and innovation management, and require a transformation in strategic and institutional management so that managers can muster the staying power to persist and learn far into the future. I develop a framework to explain how participants can strategize across an entire eco-system, deal with the profound ambiguity from complexity, and handle the very long cycle times of complex innovations. Transformed strategic and institutional management combines abductive learning routines with clock-time and event-time pacing to map a portfolio of value creating opportunities far into the future. Eco-system participants use learning events, which represent available knowledge, to hypothesize new value creating opportunities and new kinds of governance structures for specific collaborations, evaluate these opportunities and governance structures by trying them out to explore assumptions and surface new possibilities, and reframe them over time.

Many of society’s most pressing problems are complex innovation eco-systems. Pharmaceuticals, health care more generally, alternate energy systems, climate control, education, and myriad social problems rely on ongoing innovation to continually generate new projects, products, programs, processes, capabilities, and value creation opportunities that address these critical challenges. These “grand challenges” (Ferraro et al. 2015) represent the next frontier in technology and innovation management, because they require new theory for managing multiple organizations, agents, and agencies, and new theory for collaborative learning about the co-evolution of sciences and technologies among vast, globally dispersed networks of knowledge workers.

Taking advantage of emergence (Dougherty 2016) is a central capability around which new theories for complex innovation eco-systems will build. Innovators in complex eco-systems must take advantage of the emergence of knowledge about sciences, technologies, user needs, delivery systems and so on for their new products and programs, because knowledge for innovation does not already exist. Taking advantage of emergence refers to grabbing the inherently fragmented and noisy information that abounds in complex eco-systems, configuring information bits into potential solutions for concrete problems, and using these configurations to learn about what might work, what else seems relevant, and how to reframe ideas to accumulate more noisy bits of information into better and better solutions.

To take advantage of emergence, new theory must tackle three distinctive challenges: 1) involve the entire system, not single firms; 2) encompass ambiguities of complexity, which means that the simple rationality typical of conventional economic-based strategy cannot work (Tsoukas 2005; Grandori 2010); and 3) incorporate very long term horizons rather than short term clock-time and schedules. I build on recent research by myself and colleagues to propose one new thrust that tackles these three challenges: building the strategic and institutional infrastructure to anchor and guide product, process, and program innovation.

First, to incorporate the entire system, Dougherty (2016) suggests a way to unpack the complexity without obscuring the inherent interdependencies among activities. Rather than chop complex problems into separate bits to be worked on separately, complex systems require that four entire problem-setting and problem-solving cycles of innovation be worked on simultaneously and continuously. These four problem cycles are: projects – working on the innovation project (which, like curing cancer, are enormous, global networks); processes – continually integrating diverse emergent sciences to support projects; strategies – experimenting with value creating opportunities that map far into the future; and institutions – generating collaborative commons. Second, to encompass ambiguities, Dunne and Dougherty (2016) develop abductive reasoning to work on all these problems. Third, to address long term emergence and mustering the requisite staying power to persist and learn despite short term pressures, Dougherty et al. (2013) develop a way to leverage diverse temporal structures (kairos along with chronos) to leverage emergent knowing and gauge progress over the very long term.

This essay focuses on the strategic and institutional cycles of problem setting and solving. These cycles are the least well developed, but unless people continually work on these particular problems to frame project and process development, innovators in complex eco-systems cannot take advantage of emergence. And so they cannot innovate very well at all. “Strategic and institutional” managers include the managers of the many businesses in the innovation system along with those in many other agencies, public and private, who work on pulling resources together to enhance, commercialize, make better use of, and rethink opportunities. I also rely on examples from bio-pharmaceuticals to illustrate points, but hope that those interested in other complex innovation eco-systems can think about applying these ideas to their domains as well.

Two limits of existing research in technology, science, and innovation management prevent effective development of theory for complex innovation eco-systems: looking mostly at the firm level, and assuming away emergence. First, most research concentrates on the firm level, and addresses eco-system concerns with generic dynamics such as technology trajectories or a presumed flow of knowledge from science to industry. But complex eco-systems require active, deliberate management of these dynamics that is focused on specific concrete problems so that new products and processes can continually emerge and be experimented with, rethought, and revamped over time. We know that all innovations at the firm level require strategic guidance that defines the goals of innovation, creates long term resource development to support particular thrusts, and provides a strategic direction to shape and guide day to day activities (Leonard-Barton 1995). The same logic applies to complex innovation eco-systems, but now the strategizing must occur across the system in collaboration among many agents and agencies. Managers not only need to develop and implement long term strategies, they also need to continually create governance structures of various kinds that enable collaboration.

An example helps to illustrate the strategic and institutional problem setting and solving cycles. Participants in the biopharmaceutical complex innovation system have already developed a large repertoire of possible strategies, so there is no shortage of ideas for what managers ought to do. Among these ideas are: 1) develop new classes of therapeutic compounds and treatment modalities; 2) deliver therapies via collaborations with hospitals and patient associations; 3) discover drug families that address diverse genetic make-ups among patients; 4) deploy emerging sciences and technologies such as diagnostics, or research areas like epigenetics into networks of commercial collaborations; and 5) shift to open innovation (Christensen et al. 2009; Pisano 2006; West and Nightingale 2009; Chesbrough 2003). A perusal of literatures on other complex innovation eco-systems would reveal a similarly rich repertoire of possibilities for addressing these various grand challenges. All these possible strategic thrusts listed above involve collaboration among a variety of agents and agencies, both public and private, so most new strategies also require ongoing development of governance arrangements so that diverse agents and agencies can collaborate on innovative strategies, processes, and projects.

However, societies fail to implement these possible strategies, in part, I suggest, because we do not develop the infrastructure for setting and solving strategic and institutional challenges. Pharmaceutical companies compete mostly in block-buster strategies, and throw away emerging possibilities in therapies that might not become major money makers. Bio-medical scientists generate astonishing new ideas in genomics, gene therapy, systems biology, bio-informatics, proteomics and so on, but they do not develop how particular ideas interact with other sciences to enable drug discovery – which continues to flounder (Scannell et al. 2012). Educational systems seem to blame teachers for all the failures, and do not interrogate how well the knowledge system, strategic, and institutional challenges are being addressed to enhance classroom teaching. Focusing on the healthcare complex innovation eco-system, Nelson says (2005: 208):

Today, some of our most difficult problems involve developing the social technologies needed to make new physical technologies effective. Arguably the lion’s share of the strains in our health care systems are the result of advances in physical and medicinal technologies that societies have not yet learned how to manage or pay for.

Nelson’s new social technologies include strategic and institutional problem setting and solving capabilities to figure out how to manage and pay for all the advances in physical and medicinal technologies. More importantly, it would take these two capabilities to figure out how to configure all of these advances into viable new products, programs, and strategies that actually improve health (Dougherty 2016). Participants in these complex innovation eco-systems must allow the needed knowledge to emerge and evolve more fully, muster the staying power to persist and learn so they can take advantage of emergence, and build interdependent product and program configurations from all these one off inventions, so that they can actually innovate, not simply invent.

The second limit in the literature now is the tendency to ignore emergence over the long term, and focus instead on sudden revolutions. Many scholars suggest that a biotechnology revolution will sweep away existing medicinal sciences and replace them with new abilities – no need for ongoing problem setting and solving for strategic and institutional challenges. However, Pisano (2006), Hopkins et al. (2007), and Gittelman (2015) argue that a biotechnology revolution is a myth, because the knowledge needed for innovation has already taken, and will continue to take, decades to emerge. This emergence depends on a co-evolutionary process in which changes in sciences, technologies, industries, clinical care, and regulations will mutually constitute one another (Nightingale 2004). This long time horizon subjects new technologies to large and increasing development costs, not to swift revolutions. Similarly, knowledge intensive endeavors in health, education, or environmental issues embody broad, diffuse objectives rather than clear ends with defined means for achieving them. According to Ansell (2011), these diffuse objectives become ongoing processes of working on them, or possibilities toward which problem solving processes tend.

The essay explains how strategic and institutional managers can shape and guide the complex innovation process productively by: 1) engaging in abductive learning routines to work through institutional strategic problems, and 2) balancing different temporal structures (kairos as well as chronos) to muster the staying power to persist and learn over the long term. I first draw on the history of industrial management and innovation to explain why transformation in managing strategic and institutional problem setting and solving for complex eco-systems is necessary in the early twenty-first century. Next, I explain how abductive reasoning and multiple temporal structures address these new challenges. Finally, I describe how participants can develop portfolios of value creating opportunities that map far into the future, and establish institutional collaborative commons in order to develop, experiment with, and implement these strategic opportunities.

Transforming institutional and strategic management to take advantage of emergence

The first transformation in strategic and institutional management in modern society took place during the nineteenth century, when enterprise management shifted from small, sole proprietorships to large organizations around new technologies. In his historical analysis of the rise of industrial society, Chandler (1977) argues that organization and strategy are as important in building economic growth as are investments in R&D. Technological innovation and organizational innovation are interdependent, because new forms of business organization and institutional arrangements are invented to solve specific economic problems (Pisano 2010). For example, technical advances in steam power, steel making, or mechanical engineering may have made railroads and mass production technically feasible, but a host of novel organizational and institutional arrangements made these technical advances economically feasible. These novel arrangements include administrative hierarchies, professional managers, university programs to train those managers and engineers, formalized capital budgeting systems, accounting and control systems, and corporate governance structures that separate ownership and management. Chandler had this to say about railroads (1977:120, quoted in Pisano 2010):

No other business enterprise up to that time had had to govern a large number of men and offices scattered over wide geographical areas. Management of such enterprises had to have many salaried managers and had to be organized into functional departments and had to have a continuing flow of internal information if it was to operate at all.

Other capital intensive businesses evolved in a similar way (Nelson 2005). According to Pisano (2010:467): “After reading Chandler, it is hard to think about technological innovation as anything but tightly intertwined with organizational and institutional innovation.” A variety of governance structures emerged as well to orchestrate the co-evolution of new sciences and technologies, even though it is often assumed that the “free market” does all the institutional work (Nelson 2005). Emerging technologies like railroads, telephones, and electrical power required considerable government investment and involvement, along with “new” innovations such as public monopolies or eminent domain. But the centrality of strategic and institutional work for innovation tends to get pushed into the background.

If we fast forward to the 1970s, we see that nineteenth century innovations in institutional and strategic management could not deal with new technologies and diverse markets and opportunities. These old innovations focused on economies of scale, and on optimizing existing functioning based on separating steps in the production process, top-down control mechanisms that assume defined means to reach clearly defined ends, and clock-time approaches to gauge progress (e.g., time and motion studies, quarterly reports, yearly plans). These approaches could not allow for continuous product and process innovation. The locus of innovation in many industries moved from the US and Europe to Asia where new businesses emerged to accommodate new technologies of scope and divergent opportunities (Hayes and Abernathy 1980; Clark and Fujimoto 1991).

During the 1970s and 80s, a second transformation in strategic and institutional management ensued to address continuous innovation in products, processes, and business models (Schon 1967; Souder 1985). Regarding strategy, managers and scholars of innovation realized that product and process innovation cannot occur without an effective innovation strategy to develop long term capabilities in technology, marketing, and manufacturing (Day 1990; Roussel et al. 1991; Cooper 1998), guide particular innovation efforts so they could draw on these resources, and leverage businesses into emerging markets. Strategic managers could not focus only on optimizing the current functioning of the enterprise. Technologies and markets emerge and evolve, so value creating activities must also continually transform qualitatively. Regarding institutions, monopolies no longer worked and were broken up. They were replaced with more flexible alliances, government sponsored development efforts (e.g., SEMETEC), technology platform business models (Microsoft WINTEL, airframe manufacturers), standard setting bodies, and other collaborative governance structures.

But as surveys by the Product Development Management Association show (Markham and Lee 2013), more than 90% of the product innovations are incremental. The strategic transformation from the 1980s relied primarily on leveraging existing knowledge, on leveraging architectures for streams of products, and on developing methods for innovation processes that proceeded sequentially through predefined stages, from clear product concepts to market launch. With notable exceptions (e.g., Van de Ven et al. 1999), many writers emphasized how to rethink managing and organizing to generate continuous streams of incremental new products (Cooper 1998; Beckman et al. 1994). But even incremental innovations required transformation from vertical to horizontal organizing based on multi-functional teams, and from staying the course to enabling new possibilities. Many studies detailed the trials and tribulations of organizations that tried to transform even for incremental innovations (Tushman and O’Reilly 1997), and the significant changes in management that being even incrementally innovative required (Van de Ven 1986; Garud et al. 2011; Jelinek and Schoonhoven 1990; Danneels 2008).

Another major transformation in strategic and institutional management processes for innovation is required now to deal with the ambiguities and long term horizons of complex innovation eco-systems. Waves of technological revolutions in digitalization, electronics, communications systems, and other domains generate complex shifts in technology capabilities and market applications. The current explosion in biomedical sciences and other fields generates many very new possibilities for better resolutions of social and economic challenges in health, climate management, economic revitalization and other pressing social needs. Pisano (2010:480) argues that new organizational forms and institutional arrangements are necessary again for science based businesses like pharmaceuticals:

Like the railroads and large scale manufacturing enterprises of 100 years ago, science-based businesses will be a potent source of economic growth in the 21^st century. And now, as then, these new businesses will demand new organizational forms and new institutional arrangements. In short, we are once again confronted by a serious need to invent new organizational forms and institutional arrangements to deal with a new set of economic problems.

Existing strategic and institutional approaches remain short term oriented and rigid, which preclude taking advantage of emergence. Scholars have already discussed the very new kinds of strategic and institutional processes that are needed. Stacey (1995) argues that managers need to shape and guide the complex innovation process and negotiate their direction in real time by focusing on process rather than content. According to Anderson (1999), managers cannot impose controls a priori, because complex systems react to direction in unpredictable ways. Anderson suggests that managers instead establish and modify the direction and boundaries within which innovations emerge by setting constraints on local actions, observing intermediary outcomes, and tuning the process by altering the constraints. And as already noted, Ansell (2011) points out that complex problems involve diffuse objectives that become ongoing processes of working on them, or possibilities toward which problem solving processes tend. In the next section, I suggest how managers can enact these new ways of eco-system managing by addressing the specific challenges of ambiguity and long term development.

Abductive reasoning and multiple temporal structures

Abductive learning routines to take advantage of emergence in complex eco-systems

Considerable research addresses complexity so its challenges are somewhat familiar (Tsoukas 2005; Plowman et al. 2007; Garud et al. 2011). Complexity means that relations between cause and effect are unknown and must be discovered. Innovators of complex products must figure out what the elements of the product or strategy will be because there are no architectures for complex innovations. Innovators must also figure out how various elements interact to generate the desired functionality, because unknown interdependencies among elements produce unpredictable and potentially disastrous consequences.

Complexity requires a discovery style of reasoning in which people try to understand what seems to be going on, rather than confirm pre-existing expectations. Nightingale (2004) summarizes literature in science and technology that indicates that scientists working at the frontiers of knowledge do not just confirm hypotheses, because existing knowledge is too weak to point sharply to a solution, and because experiments will mostly fail. Instead, scientists tinker with experimental conditions by actively intervening to create something to learn from, and use that knowledge to build up understandings, inform judgments, and create patterns to learn from (Pavitt 1999). Schon (1983) describes the process of problem setting in complex settings, where professional practitioners identify the “things” of a situation by reflecting in practice to uncover surprises, criticize their initial understandings, and impose a frame on the situation. Orr (1996), Mayr (2000), Denrell et al. (2004), Grinnell (2009), and Van de Ven (2007) among many others describe similar processes of reasoning used by scientists, management scholars, technicians, computer experts, and other knowledge professionals.

Management scholars propose abduction as the style of reasoning that enables the discovery style (Weick 2005; Grandori 2010). Abduction is the deliberate reasoning that leads to scientific discoveries (Nesher 2001). According to Peirce, abduction is the best answer we have to problems of discovery, since abduction alone among the forms of reasoning originates possible explanations and introduces new ideas. Weick (2005) describes the abductive process as “clues giving rise to speculations, conjectures, and assessments of plausibility rather than a search among known rules to see which ones might best fit the facts.” Simon (1977) also discusses the abductive process of discovering laws in raw data based on pattern recognition and abduction of hypotheses on laws that may regulate observed patterns.

Abduction “is the process of reasoning in which explanatory hypotheses are formed and evaluated” (Magnani 2001:18). Dunne and Dougherty’s (2016) research on drug discovery scientists extends this definition to encompass formulating, evaluating, and reframing hypotheses in continual cycles of research. I extend their work to explain the process of reasoning that underlies the strategic and institutional management of complex innovation systems.

Cycling through three abductive learning routines enables institutional and strategic managers to focus on process rather than content (Stacey 1995), and continually establish and modify the direction and boundaries within which innovations emerge (Anderson 1999). The first abductive learning routine is using clues to imagine a configuration of interdependencies among strategic or institutional elements that might constitute a viable strategic value creating opportunity or governance structure. There are no architectures for radical products, strategies, or institutional arrangements, so researchers begin by imagining possible architectures. Not any hypothesis will do for complex challenges, because hypotheses about one cause leading to one effect will not work. Dunne and Dougherty (2016) explain how drug discovery scientists imagine a configuration of interactions among molecular compounds, the disease in question, and the rest of human biology. They construct a coherent story about how a chemical compound will behave in the body against the disease. In the same manner, I suggest that strategic and institutional managers construct a story about how a way to collaborate or a way to create value will develop over time.

Each element of hypothesizing captures emergent and noisy information. For value creating opportunities in bio-pharmaceuticals, the content of the strategic hypothesis is a configuration of possible interdependencies among knowledge elements that would constitute a value creating opportunity such as alternate delivery processes or new variations on an established drug that meets different genetic make-ups. The content of the institutional hypothesis would be a configuration among relational elements and rules (e.g., types of leadership, rules for IP) that would comprise the governance system among participating agents and agencies or a particular set of value creating opportunities. Managers figure out the relevant parts of their strategic or institutional possibility, and more importantly, how these parts work together and depend on each other. Centering on the interdependencies rather than only on the parts highlights the major source of uncertainty in complex systems, where failures often arise because of unexpected interactions (Scannell et al. 2012). By focusing on a reasonable set of interdependencies among parts, managers attend to the possible strategy or program in action as it functions in the real world. The hypothesis reflects how elements mutually generate the desired outcomes.

Using clues grabs information, too, because clues convert existing information into directions that lead out of perplexity, as the dictionary definition suggests. According to Weick (2005), clues point to a world in which they are meaningful, and so give rise to speculations, conjectures, and assessments of plausibility rather than focus attention on a search among known rules to see which ones might best fit the facts. In complex innovation eco-systems, innovators do not have clear knowledge, but they do have clues. Clues capture knowledge resources that are available to managers for leveraging strategically. Managers use clues project and process learning to imagine configurations of interdependencies among strategic resources that might produce a viable new process, business model, collaborative network, and so on. Scientific conceptualizations and findings are clues, not answers, clues to possible configurations of interdependencies that may work in the real world.

Imagination also grabs noisy information. Weick (2005) emphasizes imagination, which "conceives a whole design almost at once, which it then fills out and gives body to by particular association.... The mind thinks simultaneously of specific parts and of their one organizing principle" (Engell 1981, 82–83). Imagination is “… the ability to conceive of something, seen only fragmentarily or superficially, as a complete, perfected and integral whole” (Merriam Webster’s dictionary 1984, 415). “Imagination is the power to present in concrete, particular forms and expressions what before had been only general and abstract knowledge, hazy feeling, or impression” (Engell 1981:101).

The second abductive learning routine evaluates imagined configurations of interdependencies among knowledge resources or relational elements. Evaluation assesses whether or not and how the predicted relationships between the resources and potential opportunities work, and what else can the system do better. The hypothesis imagines the configuration working in the context of action, and the evaluation process further contextualizes and situates the possible new strategic opportunity. Looking at the configuration in action assesses the nature of the mechanisms that animate the possibilities.

Evaluating enables managers to use the hypothesized configuration to sift through all the noisy information as they open up around possibilities to explore them, and then narrow down to situated aspects of interdependencies. Evaluating burrows into the mechanisms to explore how and why the configuration might work, what else may be going on, and what are the limits and contingencies. Evaluation questions can include can we learn more about this configuration in the real world; what else needs to be involved; and how much some interdependencies matter or not; how far out can we see in the emerging possibilities; can we surface new consequences? In keeping with the discovery styles of research, researchers need to collectively implement ideas and experiment with them to see whether and how they work.

The third abductive learning routine cycles back to the imagined configurations by refining or reframing the hypothesized interdependencies that are involved. Participants in the strategic or institutional problem setting and solving cycles critically examine assumptions, deliberate over different perspectives, and bridge possible differences into new shared directions (Ansell 2011). Inquiry is based on the clash of different perspectives and communication for probing, adjudicating, and bridging these differences. Different people see different aspects of the product possibility and how it might function in the application setting. Iteratively integrating helps to overcome competency traps, push ideas, cross check possibilities, and generate a joint representation. Managers might refine and replace milestones, and develop new performance objectives that reflect new alternatives and consequences learned from evaluation (Grandori 2010). By reframing, managers holistically assess what they know so far and what they have learned. Reframing cycles back with a new hypothesis for the configuration of interdependencies, to be evaluated and reframed again.

Strategically managing time with learning events and event-time pacing

In addition to complexity, the product cycle time for complex innovations is extremely long, averaging more than 8 years for radical products in established firms (Liefer et al. 2000), and thirteen years in pharmaceuticals (Collins 2011). Some problems never go away. Managers must muster the staying power to persist and learn so they can allow the complex processes of discovery and learning to emerge (Lynn et al. 1996). Researchers have developed time pacing (Gersick 1994; Brown and Eisenhardt 1998) as a primary mechanism for coordinating activities around specific innovations, by regulating the intensity and direction of people’s attention and efforts. However, for Gersick (1994) and Brown and Eisenhardt (1998), among other scholars, “time” refers only to clock-time. The passage of clock-time triggers change: a business launches new products every 6 months without regard for competitive actions, enters new markets every 3rd quarter and not when an opportunity appears, and starts product platforms every 24 months.

Despite its benefits for short term innovations like software products, clock-time makes near future deadlines most salient (Clark 1985), and shifts attention to exploitation, which undermines exploratory learning (Orlikowski and Yates 2002). Industrial society as developed in the 19th and 20th centuries has relied heavily on the social technology of clock-time with the invention of, for example, standard time, railroad schedules, and time and motion studies (Clark 1985). According to Mumford (1936, quoted in Clark 1985: 36), the clock is the most powerful metaphor in the western world, and has been more influential in the development of capitalism than the steam engine.

Time is more than what a clock counts. Time, according to Clark (1985:36), “…is a socially constructed, organizing device by which one set, or trajectory of events is used as a point of reference for understanding, anticipating, and attempting to control other sets of events.” People can develop longer time horizons if they combine diverse temporal structures to guide, orient, and coordinate their ongoing activities (Bluedorn 2002). Temporal structures are “…expressed in terms of clocks or events, and are created and used by people to give rhythm and form to their everyday work practices” (Orlikowski and Yates 2002:685). Plural temporal structures enable people to understand, anticipate, and attempt to control a wider variety of events. The literature on time and organizing captures these diverse temporal structures in two broad categories: chronos or clock time (the serial time of succession measured by the chronometer) and kairos or event time (the subjective living time of invention or people’s sense that the time is right) (Garud et al. 2011; Orlikowski and Yates 2002).

In an analysis of the tensions between strategic managers and discovery scientists in pharmaceuticals, Dougherty et al. (2013) find that managers rely on short term clock-time pacing, while scientists pace their work by anticipated but unpredictable learning events. Both clock-time and event-time pacing are temporal – about time – because both mark durations and map out future trajectories by indicating when activities start or stop. Clock-time pacing marks beginnings and ends of activities with clocks and calendars, while event-time pacing marks beginnings and ends of activities with learning events that can be anticipated but when those events might occur is unpredictable.

Learning events capture emerging understandings in innovation, and reflect current and anticipated knowledge resources. Learning events emerge endogenously when innovators learn enough about the possibility they are working on to indicate the next thrust of their innovation work. Learning events are moments of closure in the exploratory searching that capture enough of the whole configuration of interdependencies to enable people to see what they know so far and to identify plausible next thrusts in their innovation work. These moments of closure redirect the work toward the ultimate goal of developing a good strategic path or a good value creating opportunity. In complex systems, learning events capture the most central resources: what we know so far and think we will learn. By highlighting event-time pacing, people’s energy and attention focus on substantive learning, not only on the passage of clock time.

Dougherty et al. (2013) suggest that strategic managers in pharmaceuticals can use clock-time pacing in tandem with event-time pacing to map out drug discovery possibilities farther into the future and accommodate the very long product cycle times. Each kind of time pacing identifies different trajectories of events and experiences. If they are used together rather than treated as conflicting practices to choose between, they can map out more of the future by encompassing more alternatives to be explored. Learning events from event-time pacing also enable people to align clock-time with event-time pacing, because learning events reflect both the chronos and the kairos of drug discovery and development. A learning event reflects the time it has taken to achieve it (chronos), and the subjective sense that an innovation is emerging (kairos).

Event-time pacing directs attention and effort to cycling through the abductive learning routines, which leaves clock-time pacing for directing attention and effort to the efficient use of resources for that cycling. Clock-time pacing coordinates ongoing process improvements that will more productively and efficiently support learning in projects and knowledge integrating efforts. Managers and innovators can also use clock-time pacing to gauge progress by measuring how efficiently people learn to apply new insights, develop supporting infrastructures for particular value creating models, examine which activities can be done more efficiently to surface interdependencies, identify barriers to learning that can be overcome, and react to results of their experiments.

Taking advantage of emergence to continually set and solve strategic and institutional problems in complex innovation eco-systems

In this section, I describe how strategic and institutional managers can work on these problems over time. The institutional innovations in governance structures enable the development, evaluation, and reframing over time of value creating opportunities that map out directions for project (new drug therapies) and knowledge process (new integrations of sciences and technologies to support discovery) innovations.

Strategically managing value creation to anticipate deeper futures

Strategic managers need to work collaboratively across the eco-system to leverage all the resources embedded in learning events that are generated by the product, process, and strategic problem setting and solving cycles. The simple question for strategizing in complex eco-systems is how can we generate value creating opportunities to shape and guide complex innovation projects and processes that emerge over very long periods? I suggest a diverse portfolio of value creating possibilities that direct emerging innovations into actual commercial or publicly valuable applications. The portfolio overall emerges and changes over time as it maps out opportunities in various time periods, and informs the entire eco-system about possibilities. The portfolio affords a deeper look in time because various possibilities work as stepping stones into the future. Strategic thinkers imagine where they can go with what the system is learning, and they shape and redirect that learning with ongoing ideas of value creation. They implement imagined configurations of learning events to see how they might work, to consider what else seems to be going on, and to surface new possibilities. Different groups of agents and agencies would collaborate over different sets of value creating opportunities, but the entire eco-system can generate a portfolio of possibilities.

Formulating Hypotheses by Imagining Configurations of Interdependencies: Strategic managers would consider what certain learning events that arise in innovation efforts (including developing strategies) suggest for a future value creating possibility, and consider how these learning resources could work together to create a viable configuration. Going forward in time, managers would hypothesize how the configuration they imagine will emerge based on current and anticipated learning events, what interdependencies might be involved, and to acquire and deploy those resources. I am describing basic business planning, except that this planning reaches out over time, anticipates emergent changes based on learning, and emerges continually. Imagining a configuration of interdependencies among knowledge resources from learning events includes identifying certain assumptions about what would make configurations a good opportunity, what are people going to learn for value creation by developing the configuration, and how will it help muster the staying power to persist and learn. Different values might include opening a new niche in the market and/or in therapeutics, generating some protection from competition, and providing a long term foothold.

A good exercise to learn about the cycle of abductive learning routines would be to interrogate current business models such as shutting down internal R&D and outsourcing the work, or focusing on block-buster drugs. For example, what configuration of interdependencies among what knowledge resources generates value how by outsourcing R&D? What learning events are this strategy based on, and what are the underlying assumptions to be evaluated and reframed? What learning events give credence to the block-buster business model?

Other examples of possible value creating opportunities for pharmaceuticals include Christensen et al.’s (2009) suggestion that small firms can flourish with diagnostics that identify the specific customer base for particular drugs, and serve small markets and unserved customers. Rather than trumpet breakthroughs in biotechnology, with abductive reasoning one explores what a breakthrough depends on to be useful to innovation, or to effect in some way an important outcome. For example, what does this breakthrough technology depend on to help define a new business opportunity, and bring it into existence? And consider the Bill and Melinda Gates Foundation’s work on developing malaria vaccines and other ways to address this debilitating disease in impoverished societies. Their mapping would consider a few different configurations of resources (including not just drug discovery but also marketing and distribution to areas that are difficult to reach), and then hypothesize what each would achieve and how, and what learning events are needed to gauge progress.

Another type of pharmaceutical value creating opportunity is figuring out what works well and what does not. Scannell et al. (2012) point out that most of the R&D costs are in failures, so they recommend that firms develop a chief dead drug officer to investigate the failures. These failures are learning events. People in this role across the eco-system would set out the major factors responsible for the progressive decline in R&D productivity, and compare different therapeutic areas to explain the differences between them in productivity. Chief dead drug officers can explore the extent to which factors are tractable, such as where the molecular reductionism of rational drug design or the brute force screening of high throughput systems become distractions, and where do they help. Chief dead drug officers can measure the veracity of previous diagnostic forecasting exercises, and examine which clinical test requirements are most costly and least valuable.

Evaluating the Configuration of Interdependencies by Elaborating and Narrowing: These ideas are hypothesized configurations of interdependencies among learning events that might constitute a value creating opportunity that is to be empirically evaluated and reframed. Instead of assuming that outsourcing R&D to universities and biotech firms will automatically work, for example, strategic managers would implement this idea in a particular way (e.g., with a certain university or type of collaboration), and experiment with the hypothesized configurations to generate evaluative knowledge. By elaborating and narrowing around the interdependencies in the configurations, managers and innovators explore the actual interactions to see how and why their hypothesized configuration actually works. They elaborate out around a subset of interdependencies among resources to consider if these interdependencies are central or not, how and why, and what else they can learn. They narrow in on interdependencies that seem stable and useful, and then elaborate out again to see other possibilities. Elaborating and narrowing balances new knowledge with existing insights. For example, managers might narrow in on particular kinds of university collaborations, and elaborate out around how fully and usefully knowledge transfers as assumed, and if not why not. The goal is to evaluate if this configuration of resources can create value based on why managers thought it would, and to explore underlying assumptions in order to learn.

Evaluating hypothesized value creating opportunities combines clock time and event-time pacing. The goal is to make better judgments, not simply better decisions, about why and how this is a good business opportunity. Clock-time pacing questions include: 1) how long does it take us to figure out that we are at a good or bad point; 2) how quickly can we evaluate learning events; 3) how quickly do others provide input to our analyses; 4) how quickly do we identify alternatives and choose among them to take next steps. Event-time pacing evaluative inquiries include: 1) we think we are here, is here good enough for a possible value opportunity; 2) are we able to handle a larger variety of configurations; 3) how much are we willing to pay to explore potential, and 4) are the learning events that emerge getting better and better? Both temporal structures can address strategic issues such as does this opportunity open a new niche, protect us from competition, extend our existing franchise adequately, give us a long term foothold, and allow us to know more about the opportunity as we also generate revenues.

Tests of confirmation are part of evaluating the imagined configuration of interdependencies, but they do not stand alone. Strategic managers need to go beyond confirmation to develop insights into why and how a particular element affects the configuration, and what else may be involved. Since the goal is learning, managers go beyond does the configuration work or not, and consider how and why it works.

Reframing the Imagined Configuration of Interdependencies by Iteratively Integrating: The third strategic abductive learning routine cycles back to the imagined configurations by refining or reframing the interdependencies among knowledge resources that are involved. Participants in the strategic problem setting and solving process critically examine assumptions, deliberate over different perspectives, and bridge possible differences into new shared directions (Ansell 2011). Strategic managers might refine and replace milestones, and develop new performance objectives that reflect new alternatives and consequences learned from evaluation (Grandori 2010). Strategic managers would also rethink how clock-time and event-time pacing combine, and how well the combinations create a rich set of reference points that people can use to anticipate more possibilities further into the future. Reframing might revise the future trajectories of anticipated activities that need to be accomplished, and coordinate attention and effort to carry out those activities. Event-time pacing structures the inherently exploratory searching and helps to constrain the short term nature of clock-time by keeping the future open to emergent possibilities. Clock-time pacing helps to constrain the potentially expansive searching, and marshals the development of resources that can be clocked. Strategic managers in the innovation eco-system identify new future trajectories and eliminate others based on emergent learning.

Institutional innovating to generate collaborative commons

The strategic management problem setting and solving cycles cannot take advantage of emergence unless the eco-system also generates new governance structures that will enable the collaborations among disparate entities that must take place. As already noted, the emergence of new sciences and technologies into viable innovations has always depended on governance structures for collaboration, along with government and private support. However, complex innovation eco-systems require continuous innovation in governance structures, since the kinds of enterprises that require collaboration will continuously emerge and evolve. As Ansell (2011) explains, the objective of many of these collaborative problem setting and solving cycles will be to build up problem solving capabilities and to engage in ongoing problem solving efforts. Participants working on a particular category of strategies (e.g., using genomics, or developing different clinical trials) would identify, develop, and manage over time their particular configuration of rules and relations.

A simple question for institutional innovating in complex eco-systems is how can we work together effectively over time to plan, experiment with, learn from, and revamp value creating opportunities, ways to integrate sciences and technologies to support actual innovation work, and/or specific therapeutic innovations? The institutional challenge is to develop a configuration of interdependencies among rules and relations that form the particular governance structure. I suggest that participants build on the non-market structures that already exist to include many organizations and agencies around diffuse objectives. These existing systems provide models for different kinds of governing structures that enable collective, long term co-evolution of sciences and technologies for innovation. These systems include open innovation (Chesbrough 2003), regional clusters (Gilbert 2012), network innovations (Iansiti and Levien 2004), industry platform systems (Gawer and Cusumano 2002), and the more general ideas about technology trajectories that build on the efforts of many different organizations and innovators (Dosi 1982; Floricel and Dougherty 2007).

For example, “open innovation” emphasizes the idea that knowledge is widely dispersed and that some innovations require multiple actors (Chesbrough 2003). Research demonstrates that open innovation requires participating firms to develop strategies for the long term, build capabilities for absorptive capacity (i.e., abilities to spot, bring in, and use emerging ideas), and build up abilities for long term partnering (Di Minin et al. 2010). There is no simple “outsourcing.” Regional clusters that foster technological innovation depend on diverse sources of knowledge, government policies that foster new technologies rather than allow incumbent firms to dominate, and rich networks of relationships that enable the exchange of tacit knowledge (see summary by Gilbert 2012). Organizations willingly participate because they can tap into supplier networks or access customers. These clusters demonstrate that for complex innovation eco-systems, multiple organizations can co-create a context of mutual learning that enables participants to muster the staying power to persist and learn.

Firms that compete also collaborate over standards setting, because standards, for example in communications systems, enable all firms to keep innovating (Piepenbrink 2015). Participants in standards setting bodies follow rules for IP and appropriation, for responsibilities for reciprocity, and for how changes in standards are continually orchestrated. Network and industry platform systems (Nambisan and Sawhey 2011) depend on the active leadership of large firms that continually upgrade the core technology or architecture, enable ongoing negotiations among participants for IP, and provide market access. In return, the large firms receive continual innovation in components and other network externalities that keep their core technology valuable. Ansell (2011) summarizes studies in the public domain among collaborating agencies around improved policing that depend on a rule of accountable autonomy.

These existing governance structures suggest a variety of rules and relational elements for collaboration that institutional managers can select from and work into configurations of interdependencies that they can hypothesize, evaluate, and reframe over time. These institutional elements include: 1) a problem solving focus: centering on what people need to collaborate over such jointly as developing a kind of drug therapy (e.g., immune therapy for particular cancers; combining drugs to form “cocktails,” or trying out a new business model); 2) heterarchical organizing that enables participation and inclusion with various levels working on different aspects of the problem; 3) leadership by individuals and by large firms or coalitions of firms; 4) self-organizing, where organizations participate voluntarily because they gain value from that participation; 5) co-dependence among participants to reinforce active participation; 6) protected niches for value creation that includes end-to-end with customers, perhaps with disease foundations, clinical research groups; 7) intellectual property protection and development; and 8) articulation of basic rules for partnering and accountability.

Institutional managers in the drug discovery eco-system (and for any other complex innovation problem) can begin by developing collaborative commons around specific pressing problems such as clinical trials for diverse therapies or advancing immunotherapy. Experiments with these and similar problems are already on-going (albeit in localized or one-off modes), and should provide considerable insight for formulating, evaluating, and reframing hypotheses about governance structures that enable ongoing collaboration around specific concrete problems. The difference for complex eco-systems would be that these emerging governance structures would continually be developed, experimented with, and revised based on progress with actual problems. People would not simply hold a big meeting and then go on about their separate work.

As Dougherty (2016) details, cycling through the abductive learning routines for institutional problem setting and solving begins with imagining a configuration of interdependencies among relational elements and rules that can produce the specific kind of collaboration needed to address a certain problem. For example, if the problem is developing improved models for the early evaluation of drug possibilities, participants would include large firms, research hospitals, regulators, and small firms. Leaders need to be appointed, perhaps a coalition with clear rules for IP, participation, and task forces to oversee particular processes. Evaluating digs into the hypothesized interdependencies to examine assumptions and figure out what works and not, and why and how. Over time, experiments generate insights into how to develop and deploy metrics about how well the collaboration is doing, update and revise the problem, arrive at joint decisions and agreements, and foster ongoing participation. Some collaborative commons will be relatively short-lived, while others may continue for years.

Discussion

Taking advantage of the emergence of knowledge is central to developing the knowledge innovators need to work on important social and economic grand challenges. Complex innovation eco-systems need to take advantage of emergence to craft viable innovations in products, programs, or processes that tackle aspects of these grand challenges. The entire eco-system of different agents and agencies participates actively and collaboratively on innovation, because the requisite knowledge and skills are partial, fragmented, and widely dispersed. All this knowledge for innovation does not exist a priori: it will emerge unexpectedly and unpredictably as innovators engage in hands-on experiments with concrete problems. Dougherty (2016) details four integral cycles of innovation problem setting and solving in complex eco-systems and argues that each one must be addressed on its own merits, yet interactively with all of the other problems.

This essay argues that unless the strategic and institutional problem setting and solving cycles unfold productively and interactively with the project and process cycles, innovation cannot occur. The reason: the innovative problems and solutions unfold over many years, eco-system participants cannot muster the staying power to persist and learn for innovation over many, many years without strategies. Strategies for innovation integrate learning, direct activities, and shape ongoing knowledge co-evolution. Strategizing in complex innovation eco-systems requires a portfolio of future possibilities that maps out in various time periods potential value creating opportunities, because the knowledge needed for innovation emerges unpredictably and over a very long time. Imagine looking out 15–20 years and seeing a variety of possible value creating opportunities emerging, depending on various anticipated learning events. The imagined portfolio represents the competitive or strategic landscape out in time. Different firms and agencies would imagine the overall portfolio of possible opportunities out in time, but from their own perspective. The portfolio affords a deeper look in time because various possibilities work as stepping stones into the future.

However, for complex innovation, strategy making occurs across the eco-system, and involves multiple, qualitatively different kinds of value creating opportunities that are collaboratively produced by multiple, different kinds of agents. Institutional innovation to generate governance structures for collaboration among the specific participants is also essential.

Abductive learning routines enables innovators across the eco-system to configure available knowledge resources into potential applications for emerging project and process innovations, evaluate those possibilities by exploring their implications and seeing what actually emerges, and reframe them over time. Abductive learning builds on both clock-time pacing and event-time pacing to allow innovators in all four cycles to muster the staying power to persist and learn. Persisting does not mean sticking to a course of action mindlessly. Rather, persisting means continually probing the forward development of a new product, new way to enable innovation, or value creating opportunity and learning about how to improve or redirect their emergence, including choosing to stop developing this opportunity. Cycling through abductive learning routines takes advantage of emergence when innovators imagine where they can go with what the system is learning, and they shape and redirect that learning with ongoing ideas of value creation. They implement imagined configurations of learning events to see how they might work, to consider what else seems to be going on, and to surface new possibilities.

Three research thrusts will enable the development and integration of the strategic and institutional cycles of problem setting and solving. One thrust is to examine how to strategize across multiple players. Technology and innovation management researchers have developed considerable insight into systems of innovation (e.g., clusters, industry platforms, network innovators), and can start to apply these ideas to complex innovation. It seems likely that complex innovation eco-systems need to focus on concrete problems so that participants can tinker with possibilities and alternatives. Ansell (2011) explains that in pragmatism, which relates to abductive reasoning, meaning depends on hands-on experiments that confront concrete problems. He argues that problems disrupt existing assumptions, call for fresh discovery, and pin disputes about principles down to particulars. Research can explore how concrete problem solving works in complex eco-systems, and what other kinds of relational elements are needed to keep participants engaged for the long term.

The second thrust of research is to delve into abduction. Many scholars already discuss abduction (e.g., Weick, Garud, Grandori). However, few studies provide in-depth empirical analyses of how innovators and managers actually can using this form of reasoning, so more research into the day-to-day innovation practices in complex innovation eco-systems is needed.

The third research thrust is to develop time. Participants in complex innovation eco-systems need to direct activities and gauge progress in a way that encompasses the very long term, because co-evolutionary emergence take lots of time. The dominant approach to time management now relies on the inherently short-term methods of clocks and calendars. Clock-time is reinforced by institutional pressures for quarterly reporting and annual planning. Research can explore how different kinds of time pacing identify different trajectories of events and experiences, and how multiple temporal structures can map out more of the future by encompassing more alternatives to be explored.

One way to develop event-time pacing would be to study how strategic managers can use event metrics to gauge progress and assess performance that derive from event-time pacing. Innovators and managers can gauge their progress by examining how useful are the configurations they are imagining, if they are deepening their understanding well enough to tackle the new problems that emerge as they develop their new strategy and if not why not, and question how promising are the alternatives and consequences suggested by the imagined configuration. Managers can also clock-time how long it takes to determine if learning events have occurred, how quickly people recognize emerging perturbations and evaluate learning events, how quickly others provide input to the evaluating and reframing, and how quickly innovators identify alternatives and choose among them to take next steps. Both temporal structures can address strategic issues such as does this opportunity open a new niche, protect us from competition, or give us a long term foothold.

Participants in complex innovation eco-systems can construct a portfolio of possible value creating opportunities that map out in time, and provide stepping stones into the future. They can also construct innovative governance structures that enable them to continually revise and reframe the portfolio by adding new ideas and dropping others. The portfolio into the future opens up possibilities for projects and strategic paths, and helps to guide these developments as innovators encounter alternatives and choose options. The portfolio enables participants to understand, anticipate, and attempt to control their complex innovations using abductive learning routines to build on available knowledge, generate new knowledge about strategic and institutional opportunities, and tinker far into the future productively.