Untangling the Relationship between Innovation and Agility

Abstract

Many governments, global management consultancies, university researchers and top executives have strongly advocated that either Innovation or Agility is essential for twenty first century enterprises (both commercial and not-for-profit). However, the similarities and differences between Innovation and Agility, and how they interrelate, has been explored less frequently. In this chapter we explore Innovation and Agility by examining two cases where they were mission-critical during a period of extreme disruption due to VUCA (volatile + uncertain + complex + ambiguous) conditions. Using data from our case examples we suggest a specification of the contingency factors that determine where Innovation and/or Agility are likely to be beneficial. From a theoretical perspective we propose that Innovation and Agility are usefully viewed as instruments for deploying clusters of micro-foundations of Dynamic Capabilities, hence contributing to the development of middle-range theories of action. We conclude by reflecting on the implications of our study for researchers, teachers and managers and propose topics for further investigations.

1. Introduction

Innovation and Agility are hot topics. In 2023 a McKinsey & Company survey found that top Chief Executive Officers (CEOs) are confronting “a welter of uncertainties” so they need to find ways to make the most of novel opportunities—amid ongoing change [1] and, for many CEOs, “Agility ranks as a high strategic priority in their performance units” [2]. So, what are CEOs advised to do? New things need to be done (requiring Innovation) and opportunities need to be grasped prudently, effectively and quickly (requiring Agility).

Although arguable now more urgent, this message is not new. Innovation or Agility have been widely advocated as stand-alone organisational imperatives. For example, ter Haar [3] stated that: “(it) is broadly accepted in today’s literature that (the) concept of innovation … is considered one of the essential ingredients of competitive advantage”. Franco and Landini [4] assert that: “firm agility has recently emerged as one of the key organizational paradigms that managers should follow to build sustained competitive advantages and as a key critical business success factor”.

Questions arise from such divergent claims that include: ‘How do Innovation and Agility relate to corporate strategy?’ ‘Do Innovation and Agility have identical functions (i.e., purposes or deliverables)?’ ‘Is the work of innovating or being agile fundamentally the same?’ ‘Are Innovation and Agility interdependent theories of action?’ ‘Are there conditions where Innovation is a superior to Agility or vice versa?’ ‘What are the distinctive managerial challenges of undertaking Innovation or being Agile?’ ‘Are there situations in which either Innovation or Agility is a superior theory of action?’ and ‘Can Innovation or Agility be dysfunctional?’

Many of these questions concern the nature of the relationship between Innovation and Agility. It is this relationship that we investigate in this chapter. Our contribution has been derived from two main sources. First, from multiple research assignments undertaken by CENTRIM (the Centre for Research in Entrepreneurship Change and Innovation Management) in the University of Brighton (UK). Second, from extensive interviews analysing critical incidents in political life and business development with the second author (Woodcock) who was a UK Member of Parliament and is a highly successful entrepreneur and chairman of both commercial and not-for-profit enterprises.

The McKinsey & Company surveys that were mentioned above demonstrate that Innovation and Agility are quintessentially strategic capabilities as they: (i) are key topics for managerial study; (ii) are core capabilities needed by twenty first century organisations; (iii) require specialised managerial methodologies (more about this later) and (iv) directly affect an enterprise’s capability to gain competitive of comparative advantage. In this chapter, for reasons of space, we will not consider how Innovation and Agility are positioned in different strategic models rather we assume that they fit into the Dynamic Capabilities framework [5].

The construct of Dynamic Capabilities provides the theoretical capstone for our study. This high-level construct was developed by David Teece, and colleagues, in the 1990s to explain why certain companies were consistently successful. Teece and Pisano [6] wrote that “winners in the global market-place have been firms that can demonstrate timely responsiveness and rapid and flexible product innovation, coupled with the management capability to effectively coordinate and redeploy internal and external competences”. We can view ‘timely responsiveness’ and ‘effectively coordinating and redeploying internal and external competences’ as key deliverables required from Agility and ‘rapid and flexible product innovation’ as a key deliverable required from Innovation.

Helfat [7] succinctly described the function of Dynamic Capabilities as providing “a capacity of an organization to create, extend, or modify how the organization makes a living… Because capabilities are context-specific, there is no such thing as a generic dynamic capability that applies to all types of activities and settings”. Research undertaken by the first author (Francis) found that certain dimensions of Dynamic Capability are so prevalent at a particulate moment in time that every top management team should assess whether these should be developed as core capabilities. We place Innovation and Agility in this category. Whether and how the capability should be developed depends on the distinctive driving force of the type of enterprise [8], product-market ambitions and the selected strategic posture of the enterprise [9].

As mentioned above, Dynamic Capabilities is a high-level construct. Many researchers have sought to unpick the construct by investigating its micro-foundations, which are the range of assets, components and processes that combine to provide enterprise-specific Dynamic Capabilities (that can be seen as analogous to the ingredients and recipe for cooking a gourmet meal).

The identification of micro-foundations of Dynamic Capabilities is important as these provide orientating concepts for middle-range theories that add agency to grand or high-level theories by enabling theoretical principles to be deployed in real situations. The relevance of middle-range theories was explained by Cartwright [10] who stated that “if we are concerned with explaining, predicting or managing the world… middle-range theories are generally the only game in town”. The logic chain is this: grand theories provide a holistic perspective (that can be described as a ‘gestalt’); middle-range theories enable development initiatives to be planned; theories of action clarify how development initiatives can be managed successfully.

In summary, in this chapter we will view Innovation and Agility as being different instruments for deploying specialised clusters of the micro-foundations of Dynamic Capabilities thereby contributing to the development of middle-range theories that offer methodologies for Dynamic Capabilities to be intentionally developed by leaders and managers. This positioning roots our study within the fields of Strategic Management, Organisation Development, the Sociology of Occupations and Management Development.

The following sections of this chapter are structured as follows:

• First, we outline the constructs of Innovation and Agility and describe how they are relevant to management.

• Second, we explore two cases where Innovation and/or Agility were mission-critical during a period of extreme disruption due to VUCA (volatile + uncertain + complex + ambiguous) conditions.

• Third, using data from our case studies, we analyse the functions of Agility and Innovation, reconsider their role as instruments for the deployment of Dynamic Capabilities and present a provisional specification of the conditions where each is likely to be beneficial.

2. The constructs of Innovation and Agility

In this section we provide an overview of how the constructs of Innovation and Agility have been developed since they became topics for academic study.

2.1 Innovation

We define Innovation as ‘a deliverable from specialised work undertaken to transform opportunities, ideas, resources or needs into something of value that is new to the unit of adoption and strengthens either Dynamic or Ordinary Capabilities and/or contributes to achievement of competitive or comparative advantage’.

By defining Innovation as ‘a deliverable from specialised work’ we make an unusual distinction. An example helps to clarifies this point. For us, a pharmaceutical company that undertakes the research and development work needed to develop a new drug is undertaking an Innovation initiative. A different pharmaceutical company that buys a start-up company that has developed an identical new drug is being Agile as it is using its existing (financial and executive) resources to acquire the right to register the new compound but it has not done the specialised work required to transform an opportunity into something of value.

It is impossible to understate the importance of Innovation as it has transformed human experience in almost every conceivable dimension including in social structures, health, life chances, economic labour and warfare. The distinguished economist William Baumol [11] concluded that “virtually all of the economic growth that has occurred since the 18th century is ultimately attributable to innovation”. Many countries promote Innovation as a political and economic imperative. For example, in China, “Invention and innovation are (recognised as) the sources of technological change” [12] and South Korea sees “Research and Innovation as the driver of national economic and social advance” [13]. From an academic perspective, Innovation has received much greater attention than Agility. In 2022 there were approximately 58,000 hits in Google Scholar for articles with titles that included variants of the search-term ‘Innovation’ and an identical search procedure, using variants of the search-term ‘Agility’, yielded just 4480 hits (of which a significant proportion related to medical conditions or the training of dogs).

The work of manging and delivering innovative initiatives is demanding. Not all innovation initiatives can be successful, and some may be dysfunctional as a search to find, develop and exploit something new can be costly, unpredictable, distracting, resource-intensive, unproductive and sometimes exploitative, socially damaging or environmentally harmful.

Often, but not always, Innovation includes an element of risk. More ideas will be generated than can be implemented so many must be killed-off, sometimes without a full understanding of their potential. There may be a need for many aligned initiatives to be undertaken at the same time, most of which will be narrow in scope and improvement orientated but some may be transformational and require the destruction of existing assets, routines or mind-sets. In addition, Innovation requires that organisations possess a superior capability to execute by getting new things done effectively and efficiently.

For these reasons, it is wrong to assert that ‘Innovation is always needed and more Innovation is better’. Rather, Requisite Innovation is needed ‘not too much, not too little, of the right type and delivering wanted deliverables’.

2.1.1 Innovation as a (Untidy) process

Although the work of innovating may appear to be a systematic process there have been numerous case examples that demonstrate that it is rarely neatly structured and there will be twists and turns, set-backs, dead ends and, hopefully, lucky breaks along the way: more like the process of a jazz band than a string quartet [14]. For this reason, we describe the work of innovating as a journey with phases rather than a set of rigid steps. Capable managers know what skills are needed in each phase of an Innovation journey, how to deploy them and they will coach others in their use. As an Innovation initiative passes from one phase to another then those involved must change the mindsets, skills, structures and processes that they use to make progress.

There are five phases of any Innovation journey that, in practice, may not be sequential. Searching requires scanning the environment to detect existing or potential needs, ideas or possible improvement pathways that could possibly be captured and exploited by the unit of adoption. Exploring is when ideas or opportunities are carefully analysed to assess issues related to their possible development such as costs, risk factors, potential benefits, relationship with corporate strategy, rivals’ offers, difficulties in adopting and whether a proposed innovation is wanted by internal or external customers. Committing is when decisions are taken as to ‘what we will and won’t do’ and the resources that will be needed to move forward quickly, efficiently and effectively are defined and allocated. In Realising required development work is undertaken, often as a project, to transform a commitment into a deliverable. The ethic for this phase is ‘we get this done on-time and on-budget’. Optimising finds ways to maximise possible benefits from an Innovation journey. Here the ethic is ‘Let’s squeeze every last drop of advantage from what we have achieved’.

2.1.2 Targeting Innovation Capability

Once possessed Innovation Capability is ineffectual unless it is used. It can be targeted directed towards one or more of six potential target areas (we refer to these as the 6Ps). In brief, the targets are: (i) outputs (Product), (ii) how things are done (Process), (iii) interactions outside the enterprise (Position), (iv) business models (Paradigm), (v) acquisition of resources (Provisioning) and/or (vii) means of simplifying value chains (Platform) [15, 16]. The 6P target areas are explained in greater detail below.

2.1.2.1 Product Innovation (P1)

Product Innovation (P1) targets what is produced. Specifically, the outputs of an organisation or a sub-unit that are, or could be, provided for external and/or internal customers and/or other stakeholders. Products are tangible or intangible goods or services. Targeting innovation capability on developing new and/or improved products can involve multiple actors engaged in complex and inter-linked processes with a single end in view, which is creating superior value at an acceptable cost for users, distributors or customers. Many products targeted for external customers are inherently complex, so decisions need to be made about issues such as branding policies, market development trajectories, industry logics, resource availability, technological developments and other factors.

Modern methods for managing Product Innovation emerged in the late nineteenth century. Early movers found that units specifically dedicated to delivering successful innovative journeys (that became known as Research and Development or R&D Centres) strengthened Dynamic Capabilities, especially for science or technology intensive operations. For example, in Germany the Carl Zeiss company opened its first R&D laboratory in 1889 thereby creating an internal capability, and a facultative ecosystem, to enable multiple innovative journeys to deliver waves of superior science-based products until today.

Although R&D centres proved to be an effective wellspring of some forms of Product Innovation numerous other methods have been subsequently developed including skunkworks (for radical innovation), lean-start-up (for iterative innovation), open innovation (to reduce costs and gain access to a wider pool of capability) and, increasingly, Artificial Intelligence (using machine intelligence to draw from big data repositories to create novel solutions).

2.1.2.2 Process Innovation (P2)

Process Innovation (P2) targets how work is done. Processes are sequences of activities that enable tasks to be accomplished. This target for Innovation has many potential benefits including making processes faster, more responsive, cheaper, more reliable, technologically enabled, accurately measurable and/or better integrated. Processes can be extensive, interdependent, adaptive, socio-technical and integrated. Process Innovation is facilitated by systematic analysis, comparative benchmarking, consultancy and offers from specialist providers. Technological developments are extremely influential. For example, intelligent automation is widely adopted, the Internet of Things has changed processes for the maintenance of equipment fundamentally and intelligent digital systems are transforming processes for organisations to relate to their customers.

Modern methods for managing Process Innovation emerged in the early years of the twentieth century. The research, writings and lectures of Frederick Winslow Taylor [17] were a pivotal turning-point as they demonstrated that Process Innovation could be rigorously structured. Taylor’s methodology was developed by efficiency engineers who spent months in the Bethlehem Steel Company in Pennsylvania observing the micro-dimensions of inefficiency using laboratory-style techniques of scientific enquiry. Taylor concluded that: “It is no single element, but rather this whole combination, that constitutes scientific management, which may be summarised as: Science, not rule of thumb; harmony, not discord; cooperation, not individualism; maximum output, in place of restricted output; the development of each man to his greatest efficiency and prosperity” [18]. Taylor’s work demonstrated that organisational culture was critically important, as were human-centric values, a team-ethos, stretching goals, personal development and a reward system that recognised endeavour and output. Some of these principles were challenged in later research studies, especially in the Hawthorn Plant in the USA in the 1930s that found that non-rational factors profoundly influenced performance by the kind of intellectual rigour demonstrated by Taylor continues to drive Process Innovation.

2.1.2.3 Positional Innovation (P3)

Positional Innovation (P3) targets how meanings and interfaces between an organisation and external entities are managed. It has wide relevance as Positional Innovation includes all aspects of an organisation’s interplay with its environment, including, suppliers, customers, potential customers, influencing agencies, regulators and with the wider world.

Reactive Positional Innovation focuses on improving how an enterprise senses political and social changes, industry dynamics, technological developments, rival’s activities, needs of customers (internal and external), potential customers, entities in its ecosystems and other stakeholders or influential bodies.

Proactive Positional Innovation focuses on image, reputation, influence, differentiation and communicating the merits of products or services offered. In larger organisations marketing functions play a key role in determining positional strategies and tactics. Successful Positional Innovation improves the management of identities by reshaping advertising, marketing, media, packaging and the management of meanings. The use of digital technologies, including artificial intelligence powered analytics, is transforming the role of Positional Innovation in many enterprises. Now, facilitated by the internet and other forms of digital communication, information flows easily and enables organisations to be proactively outward looking and globally focused. Positional Innovation can: (i) provide higher quality and almost real-time macro- and micro-levels of customer intimacy; (ii) exploit collective task-specific collective knowledge easily and cheaply; (iii) increase the probability of beneficial partnering.

It was in the late 1980s that a method for structuring Positional Innovation known as Absorptive Capacity was developed [19]. This aims to: (i) strengthen the capability of people in an organisation to look outside of their own organisation and find ideas, opportunities, connections and other possible useful or advantageous inputs; (ii) consider (absorb) each new input so that they understand its strengths and weaknesses and possible relevance to their own organisation; (iii) select inputs that are most likely to be beneficial to their own organisation and (iv) effectively implement the selected inputs.

2.1.2.4 Paradigm Innovation (P4)

Paradigm Innovation (P4) targets what has been called ‘the Brain and the Heart of the Firm’ [20]. Thomas Kuhn [21] used the term paradigm to mean a coherent, shared body of knowledge, that becomes a largely unquestioned collective perceptual lens through which the world was interpreted.

All organisations are shaped by a de facto paradigm that may be implicit or explicit, weak or strong. Sometimes a paradigm is codified by a written creed or a statement of vision, mission and values. Paradigm Innovation is usually focused on an enterprise as a whole but its principles can also apply to sub-units. Not all paradigms are optimal as some will become obsolete. It is a key leadership task to review the efficacy of the current paradigm and redefine it if necessary.

Paradigm Innovation is helped when senior leaders examine questions like: ‘What Business Models are viable in the industries in which we choose to compete?’ ‘Are Change Drivers reframing what is viable?’ ‘What Vision, Mission and Values do we chose to adopt?’ ‘What organising principles are best for us?’ ‘What deliverables do we need from our Dynamic Capabilities?’ ‘What must we be excellent at doing?’ and ‘How do we gain the greatest value from technological change?’ ‘What would ‘success’ mean for us?’

An example from the financial services industry clarifies the importance of Paradigm Innovation. For more than a century retail banking in the UK was largely predictable. The market for financial services was dominated by a small number of national banks with branches in almost every city and town in the country. This enabled customers to have direct access to a broad portfolio of personal and business services. It was not to last. In the twenty first century this paradigm of retail banking almost collapsed as the extensive use of mobile digital devices enabled different concepts of service to revolutionise the banking experience with ‘Platform as a Service’ and ‘Software as a Service’ becoming widely accepted. In addition, Artificial Intelligence (AI) and Machine Learning techniques automated risk calculations and blockchain technology is being used to improve customer authentication. Traditional banks found that they had to practice radical Paradigm Innovation if they were to compete successfully in the digital era.

Modern approaches to paradigm innovation owe much to the recognition that organisations need a coherent driving force. Work by Tregeo and Zimmerman [8] in the 1970s provide useful typologies that have been subsequently extended.

2.1.2.5 Provisioning Innovation (P5)

Provisioning Innovation (P5) targets where and how resources are obtained and is relevant for enterprises and sub-units. The resources sought include financial, knowledge, technological, locational, contractual, reputational or legal inputs or assets. Successful Provisioning Innovation develops a facilitating ecosystem that includes users, supporters, actual and potential customers, service providers, kindred organisations, funding sources, special interest groups, networking sites, advisors and resource providers. This form of innovation is complex as the value that can be gained from external resources differs greatly. Nine categories of provisions provide a useful overview of the scope of opportunities for this target of Innovation. These are: (i) recognition (gaining prestige and stature); (ii) development of managerial capabilities; (iii) sector-specific requirements and opportunities; (iv) finance and investment; (v) human resource management; (vi) access to suppliers; (vii) insight into customers and potential customers; (viii) developments in science and technology and (ix) procurement policies and practises.

Although Provisioning Innovation is broader than supply chain strategy some of the policies and practises developed for this specialisation are relevant to the development of external ecosystems. These include legal and informal contracting; sustaining mutual trust; ease of maintaining relationships; risk resilience; transparency; sustainability; procurement strategy; real-time management; systems integration; supplier evaluation; negotiation and contract management; supply chain operations; inventory and operations; transport and project management.

The identification of provisioning as a target for innovation is relatively new although there are long-established processes in accounting to value assets and the significance of resources was clarified in the academic perspective known as the Resource-Based View of the Firm [22].

2.1.2.6 Platform Innovation (P6)

Platform Innovation (P6) targets how customers or users access products or services. There are two major types of platforms. One provides do-better and do-different concepts, tools and facilities for improving or redesigning conventional marketing and sales activities. The other is digital platforms: (i) that provide hitherto unavailable opportunities to contact previously unreachable market segments; (ii) proactively shape attitudes and behaviour in customers and others; (iii) greatly enhance opportunities to interact with users or customers; (iv) accelerate learning and (v) facilitate collaboration between individuals and groups that previously could not interact easily, if at all.

Currently, the most dynamic area is Digital Platforms as their presence has been transformational. Entire industries have been reshaped, often fundamentally, as opportunities for global enterprises have proliferated, often at the expense of local providers. Wealthy corporations invest millions to provide Platforms that can gain global competitive advantage. These are frequently enhanced and enriched by streams of new or improved products or services with Big Data and Artificial Intelligence acting as engines for further waves of innovation. Previously unformed or unreachable market segments may be reached and influenced. Marketing can be targeted at individuals in ways that were previously impossible. Interaction between buyers and sellers is enhanced as information is exchanged seamlessly and real-time data tracks trends. It becomes possible to grasp opportunities to gain advantage by connecting groups that previously could not interact easily, if at all, as Airbnb has so brilliantly demonstrated.

Platforms have a history buried deep in time. A simple marketplace in a town square is a form of Platform. The development of the Internet in the 1980s reconfigured the functions of Platforms in ways that continue to be developed.

2.1.3 An innovation Litmus Test

As a quick Litmus Test, we can describe Requisite Innovation as having seven characteristics.

1. Innovation is ‘the way we do things’ (it is deeply embedded in our organisational culture).

2. Our leaders proactively support Innovation initiatives.

3. We benchmark how effectively we are innovating, compared with rivals.

4. Blue-sky thinking is prized and practised.

5. Those who drive innovation forward are recognised and honoured.

6. We are skilled in each of the five phases for Managing Innovation.

7. We innovate in each of the 6P areas described above.

2.2 Agility

About 320 B.C.E., a Chinese General, Sun Tzu, wrote ‘The Art of War’ which is the earliest available description of the art and science of effective action in ambiguous and contested situations [23]. Tzu argued that, in warfare, leaders must direct and mobilise capabilities, mindsets, skills, processes, risk assessment processes and strategic flexibility. He advised that leaders should “avail yourself of any helpful circumstances over and beyond the ordinary rules. According as circumstances are favourable, one should modify one’s plans” [24].

Tzu’s description of the capabilities required for effective action in situations that are now described as being VUCA (volatile + uncertain + complex + ambiguous) is quintessentially Agile. His Middle-Range Theories of Action advocate gaining advantage by acquiring, upgrading, reconfiguring and directing specific sets of capabilities and using carefully considered stratagems to respond effectively, quickly and proactively to the circumstances of the moment and focus on winning, despite opposition and difficulty.

Five insights from Tzu’s conceptualisation of Agility are relevant today, specifically:

• In contested situations there will be winners and losers, but wise, timely action can change situational dynamics, thereby increasing the probability that strong Agile players will win.

• It is vitally important to execute decisions efficiently, effectively and, in most cases, quickly. This requires that needed resources be made available, reconfigured, aligned and be deployed where the greatest advantage can be gained (incidentally Tzu described key features of the ‘Resource-Based View’ that was to become an influential strategic concept 2000 years later [25]).

• It is necessary for those with leadership roles, not only at the top of the organisation, to understand, in depth, how to define the logic of a current situation as effective action needs to taken to exploit latent opportunities and reduce or eliminate threats.

• It is rarely possible to be perfectly prepared for all unfolding events but obstacles must be defined as targets for action, not reasons for failure.

• Prudent caution is a necessity. Risks, dangers and challenges are real. Agility cannot win every battle.

Today’s scholars of military strategy continue to explore how the leadership qualities outlined by Tzu can be developed. Alberts and Hayes [26] provide valuable insights from North Atlantic Treaty Organization’s (NATO) studies explaining that: “Agile organizations must be able to meet unexpected challenges, to accomplish tasks in new ways, and to learn to accomplish new tasks. Agile organizations cannot be stymied when confronted by uncertainty or fall apart when some of their capabilities are interrupted or degraded. Agile organizations need to be able to tolerate (even embrace) disruptive innovation. Agile organizations depend upon the ability of individual members and organizational entities to get the information that they need to make sense of a situation and to combine and recombine as needed to ensure coherent responses”.

2.2.1 The Agile Paradigm

In the late 1980s structured methods for Agility-Orientated Organisation Development began to be formulated. The context was a worsening crisis for manufacturing companies in the USA. Asian rivals had gained comprehensive competitive advantages and entire industries in the USA were at risk of collapse. Many generic weaknesses were identified in American companies, including slow responsiveness, a lack of flexibility, high costs, intractable quality problems, a weak capacity to reconfigure resources rapidly, ineffective project-based management and an inability to undertake rapid value-creating innovation [27].

A government-funded, but industry-led, think-tank was created to find ways to reinvigorate American manufacturing. It was located in the Iacocca Institute in Bethlehem, Pennsylvania [28]. It so happened that Bethlehem was the best possible backdrop for an effort to reset America’s industrial trajectory, as the city had been once one of the world’s great steel-making centres but was now scarred by rusting and silent furnaces and acres of decaying machinery.

The Iacocca task-force had 100 top managers from America’s industrial giants, along with consultants and academics, working for about six months to analyse problems and prepare a report to address the woes of the industrial West [29]. They concluded that American companies needed to be reconfigured radically, so that they became Agile, meaning that they: (i) were quick to create and seize opportunities; (ii) able to customise products for individual customers; (iii) were early and capable adopters of hard technologies (like digitalisation) and (iv) soft technologies (like quality control) and (v) utilised fully the latent talents of employees through directed empowerment. This combination of organisational attributes was dubbed by the Iacocca taskforce as the ‘Agile Paradigm’ and methods for a form of Agility-Orientated Organisation Development were designed to embed it widely. In the years that followed many companies in the USA and Europe adopted aspects of the Agile Paradigm to enable them to react faster to changing customer needs.

2.2.2 The Agile Manifesto

Agility-Orientated Organisation Development took a great leap forward in 2001 when a group of friends gathered in a lodge in Utah to enjoy skiing and discuss organisational solutions to the problem-ridden process of developing large software solutions. Collectively they produced a set of guidelines that advocated a radically different managerial approach that they called the Manifesto for Agile Software Development [30]. Derivatives of this approach, often described as Scrum, are now used extensively including in twenty first century enterprises, like Google, Microsoft, Amazon and Spotify, that have created vast and inherently Agile commercial organisations drawing from principles embedded in the Agile Manifesto [31].

2.2.3 Requisite Agility

Over the past 20 years a research programme into the practical management challenges of adopting Agility as a strategic capability has been undertaken in CENTRIM where researchers found that some organisations did benefit from adopting the Agile Paradigm or the Agile Manifesto whereas others used different approaches, such as investing in technology, establishing skunkworks, moving R&D facilities into dynamic ecosystems or acquiring new capabilities (by purchasing dynamic start-up firms etc.). In short, CENTRIM’s study found that ‘one size does not fit all’ [32] and that adopting Agility as a system-wide strategic imperative can be dysfunctional [33].

Possible dysfunctions include: (i) having too much or too little Agility; (ii) adopting a sub-optimal type of Agility; (iii) being Agile in the wrong places; (iv) prioritising short-term Agility without developing sufficient resilience to cope with future challenges and (v) failing to acquire the foundational capabilities required. For these reasons, it has been realised that a prudent managerial aim is to strive to become Requisitely Agile, meaning ‘not too much, not too little, of the right type and delivering wanted deliverables’.

Requisite Agility depends on possessing, or being able to access, and then using wisely and effectively agile-enhancing leadership and managerial capabilities and specialised (agility-enabling) portfolios of Dynamic Capabilities that enable an enterprise to be (i) proactively situationally adaptive; (ii) prudently opportunistic; (iii) effective in identifying and mitigating threats and (iv) actively preparing the organisation for a different future.

Overall there is a consensus in the third decade of the twenty first century that the need for Requisite Agility is growing as the forces driving competition are becoming stronger, more formidable and increasingly disruptive [34]. Requisite Agility requires the acquisition of capabilities, including human capital, that can be readily reconfigured to seize short- and long-term opportunities and, a high degree of intelligence, willingness and boldness amongst decision-makers to take advantage of the circumstances of the moment whilst avoiding the pitfalls that often beset the hasty.

New approaches to Agility-Orientated Organisation Development continue to enhance our understanding of how a high-level ambition to be Requisitely Agility can be achieved and the required specific capabilities can be developed. Of particular importance has been the realisation that socio-technical developments, such as the use of the Internet of Things and Artificial Intelligence, provide hitherto unavailable opportunities to increase Agility and the tools in the Lean Start-Up enable responsive micro-level strategic Requisite Agility. More recently, the EAfA (Exploiting Agility for Advantage) methodology has provided a comprehensive step-by-step process for developing Requisite Agility, identifying that the real management challenge is not to adopt agility principles universally but selectively, in ways that strengthen an organisation for both short- and long-term advantage [15].

2.2.4 Requisite Agility in context

Agility is a construct located in the realisation that aspects of the world are, and will always be, ruled by Darwinian forces where harsh competition, unexpected events and ongoing evolutions and revolutions are normal conditions [5]. Note that we use the phrase ‘aspects of the world’ as some organisations operate in less dynamic or relatively uncontested environments. For this reason, Agility needs to relate beneficially to three other grand driving forces that shape organisational capabilities. These are a need for: (i) order, (ii) alignment and (iii) prudence. Organisations need to be orderly so that they are understandable, efficient and predictable. They need alignment as parts cannot pull in divergent directions, so must be directed, just as a magnet causes iron filings to be oriented in the same direction. Prudence is the last of the grand driving forces as it combines a pursuit of economy with an enduring concern to fight tendencies towards unwise of excessive risk, neglect or systemic weaknesses.

2.2.5 An Agility Litmus Test

As a quick Litmus Test, we can describe Requisite Agility as having seven characteristics.

1. Senior leaders have a strong commitment to promoting requisite agility.

2. Entrepreneurial characteristics are greatly valued at all levels.

3. Resources are dynamically reconfigured to meet the needs of the moment.

4. Hands-on teams drive and coordinate initiatives in real-time.

5. Organisational systems are designed for rapid action.

6. Changes in customers’ needs and wants are intensively studied.

7. Progress is tracked so that urgent remedial action can be taken.

3. Case studies of organisations confronting VUCA (volatile + uncertain + complex + ambiguous) conditions

In this section we explore two cases where Agility and/or Innovation were mission-critical during a period of extreme disruption due to VUCA (volatile + uncertain + complex + ambiguous) conditions.

In March 2020 it became clear that the UK was about to experience a serious public health emergency caused by the rapid spread of Covid 19, then a new strain of coronavirus. At that time epidemiologists made predictions that up to a half a million premature deaths could occur in the UK with ice-rinks commandeered for use as temporary mortuaries and hospitals being unable to cope with the quantity of patients. The UK government moved to what was described as a ‘wartime footing’ and many initiatives took place to mitigate the effects of the pandemic, of which we will examine just two. First, was a fast-track programme to discover, develop, test, produce and deliver effective vaccines to protect populations against Covid 19 in about one-tenth of the time generally allowed for this kind of task. Second, was the construction of a new emergency hospital in London that would have the facilities to cope with hundreds of patients with urgent and severe respiratory failure.

3.1 Creating an effective vaccine against Covid 19

The only effective long-term solution for a global viral pandemic is a safe and effective vaccine. In the case of the Covid 19 epidemic the need was urgent, but it quickly became clear that conventional pharmaceutical drug development processes would be unable to meet a tight deadline as, conventionally, drugs or vaccines will be in discovery or development for up to a decade before they are ready to be licenced as medicines.

The UK Government was quickly briefed on the likely disastrous social, individual and economic consequences of the pandemic and an ad hoc executive organisation was quickly established with access to almost unlimited funds and the power to define policies and commandeer resources. An early step in the UK was to establish the Vaccine Taskforce, which was announced on the 17th April 2020 [35] with the mission to “drive forward, expedite and co-ordinate efforts to research and then produce a coronavirus vaccine and make sure one is made available to the public as quickly as possible”. This well-funded taskforce had the direct support of the UK Government to devise and deliver a strategy (that can be described as ‘backing many horses in the same race’) for directing and managing fast-tracked vaccine development.

It was quickly realised that the work to discover, invent, develop, test and manufacture at scale new vaccines at unprecedented speed would require extensive ‘do-different’ Innovation in multiple areas, including managerial processes, scientific discovery routines, intergroup-cooperation, trials management practises and preparation for constructing or upscaling vast, and largely robotic, production facilities. Innovation would be required in each of the 6P areas described in the previous section. As there was no certainty that a candidate vaccine would prove to be effective and safe in clinical trials multiple candidate vaccines were required to be developed.

Thanh et al. [36] provide deep insight into the scientific and technological challenges of this work programme when they observed that: “A striking feature of the vaccine development landscape for COVID-19 is the range of technology platforms being evaluated, including nucleic acid (DNA and RNA), virus-like particle, peptide, viral vector (replicating and non-replicating), recombinant protein, live attenuated virus and inactivated virus approaches”. The development of Covid 19 vaccines quickly became an international effort with leading pharmaceutical companies providing resources and expertise so that ‘many horses could be backed in the same race’.

Although one promising vaccine was being developed in the UK, the government took a decision to order supplies of different vaccine candidates before they had been certified for use from a range of international providers. Baraniuk [37] sheds light on the importance of speculative executive decision-making in this context when he observed that “(t)he UK’s hefty vaccine orders were made in part thanks to the 2011 film Contagion. Health secretary Matt Hancock was spooked by the ending of the film, in which countries ravaged by a respiratory disease are left fighting for a limited number of vaccine doses. He insisted on ordering 100 million Oxford-AstraZeneca doses despite receiving advice to order a mere 30 million”.

The Covid 19 vaccine developed by Pfizer and BioNTech was approved in about eight months after trials began. On the 8 December 2020 Grandmother Margaret Keenan became the first person in the world to receive the Pfizer COVID-19 jab in a hospital in the UK. The vaccine had been prepared four times more quickly than had ever been achieved before and in about a tenth of the time usually required for this kind of task.

Innovation was the core process driving the development of Covid 19 vaccines and it delivered the scientific breakthroughs, rapid upscaling of production facilities and other developments that enabled an effective and safe vaccine to be inoculated into millions of people within a timescale never previously accomplished. The scientific work of developing vaccines required interaction between multiple forms of ‘do-better’ and ‘do-different’ Innovation in each of the 6P areas (Product, Process, Position, Paradigm, Provisioning and Platform) that were described earlier. Although Innovation was central it depended on Requisite Agility to provide the conditions, and maintain ongoing support, that made this huge innovative endeavour possible. Agility was essential to establish, empower and drive the governance structure, acquire authority to create or deploy extensive and expensive resources, secure international cooperation, enrol pharmaceutical companies, provide funds, fast-tracked the building of production facilities and enable rapid clinical trials to be completed.

3.2 Building the Nightingale Hospital in 10 days

In March 2020 it was decided by the Government Minister responsible for Health that an emergency hospital was needed in London as it was predicted that the existing hospitals could be overwhelmed by the numbers of seriously ill patients with Covid 19. London’s emergency hospital was named after the nurse Florence Nightingale who had made huge contributions to the development of modern nursing techniques in the nineteenth century.

Once the decision had been taken to build the emergency hospital the UK government had to decide what ‘practice of action’ or ‘means’ could achieve this demanding and urgent task. Interestingly, the construction task was not delegated to the normal planning groups who were accustomed to take several years to construct a new hospital. Rather an empowered ad hoc organisation was constructed with the hands-on help of military officers who had the capabilities to deliver fast-track complex work programmes. The importance of ‘can-do’ competencies provided by the military was explained by a member of the project management team [38] in the following way: “(m)uch of the success so far is due to the collaboration with the military, whose leadership approach to clarity of task, clarity of mission, delegation of responsibility, execution and compassion is impressive to see”. Watts and Wilkinson provided additional insight into ‘can-do’ competencies in their interview with a chief medical officer of a Nightingale Hospital, who observed that “(t)hey (the military) will take your wish list and provide you with a solution… And if you have forgotten something or made an error, they are not phased at all: the attitude is one of problem solving not blame… They are very good at just making it happen; they don’t go away and debate it and have committee meetings…They take the instruction and operationalise that for you at great speed”.

Many actions were implemented within days of the government decision to go ahead. The Excel Centre in London was leased, vast funds were allocated, specialist architects instructed to design a new 4000 bed hospital, military planners tasked to act as project integrators, construction companies with proven competences were hired and hundreds of specialist workers were recruited. This required multiple rapid, decisive, integrated, funded and aligned actions to fulfil a design brief that [39] would “repurpose, with minimal new construction techniques, an exhibition centre into a hospital”. Notice that ‘minimal new construction techniques’ were specified. It was considered that there would not be time to embark on Innovation Initiatives. Those defining how the constructors of the Nightingale Hospital should operate had taken a decision similar to that already adopted by National Aeronautics and Space Administration (NASA), which was to strive to avoid undertaking any innovative endeavours during an actual space mission.

The Cfes construction company was selected as a principal contractor in the Nightingale Hospital project. Rob Doubtfire, Managing Director of Cfes, described the hectic pace of being a key actor in this agility-orientated endeavour [40]. Doubtfire received a phone call on Friday evening the 20th of March and he attended a briefing at Excel Centre the next day. One day later, on Sunday, Cfes designers gathered to plan the transformation of the conference centre into specialist hospital and employees started work, on site, on the next day. From then on teams from the UK’s National Health Service (NHS), the military, specialist architects and many subcontractors worked 16 to 18-hour shifts for nine days until the new hospital was ready to be handed over. The members of these aligned groups were totally dedicated with some sleeping in the temporary headquarters project office. Importantly, almost unlimited funds were made instantly available and government ministers took a close and proactive interest in maintaining rapid but considered progress.

Agility, not Innovation, was the dominant modality of action. Agile Capability was targeted at: (i) developing a resolute political will to get it done; (ii) making adequate funds available; (iii) assigning military personnel to provide the organisational expertise required to coordinate a large, complex and urgent project, (iv) ensuring that only experienced professionals were hired, (v) developing a culture that total dedication was expected, (vi) ensuring that workers were empowered, (vii) limiting innovation that was only sanctioned if existing capabilities were unavailable and (viii) maintaining a ‘wartime spirit’. As the task of building of the Nightingale Hospital was undertaken there were relatively minor Innovation Initiatives that were ‘do-better’, rather than ‘do-different’, in Process, Provisioning and Platform. These were supportive rather than mission critical.

3.3 Learning from the cases

When we review the cases outlined in the previous section it is important to note that data collected in one context cannot provide reliable generalizations although it can be used to develop hypotheses or to identify research questions. A limitation of our study was that the data available were secondary and the expertise of informants could not be checked, although the records of the UK Government’s Health and Social Care Committee, which has the power to take evidence under oath and to compel witnesses to attend, provided a valuable means to cross-check data and added veracity to the dataset.

Using of the Litmus Tests (see Sections 2.1.3 and 2.2.5) shows that the development of Covid 19 vaccines and the construction of the Nightingale Hospital benefitted from Innovation and/or Agility at different phases of these grand projects. At times the relationship between the two instruments was complex with Innovation being nested within Agility or Agility providing resources to increase the probability that Innovation would be successful.

3.3.1 Relevance of structural functionalism to the construct of Dynamic Capabilities

Whether reviewed separately or interactively it is helpful to describe Innovation and Agility as providing ‘functions’ that serve to facilitate or enable progress towards an overall goal. The term ‘functions’ needs explanation as we are using the term as it was defined in a sociological perspective known as Structural Functionalism. This conceptual framework is now rarely used for large-scale social analyses but, as Potts et al. [41] explain: “structural-functional approaches are highly useful and practical when used as a foundation for systemic analysis of real-world, multi-layered, complex planning systems”.

Structural Functionalism views a social system as an organism with multiple ‘organs’ each of which makes a distinctive contribution to the wellbeing of the whole. If an organ helps a social system to survive over time, remain healthy, achieve desired outcomes, withstand shocks and adapt to changing conditions then is described as ‘functional’. If an organ hinders the social system’s ability to survive over time, withstand shocks or adapt to changing conditions then it is ‘dysfunctional’.

From a theoretical perspective, viewing Innovation and Agility as providing (sociological) functions provides a useful, perhaps novel, perspective on Dynamic Capabilities that were originally identified largely using economic theoretical lenses. In recent literature [42]. Dynamic Capabilities have been described as being actualised by ‘micro-foundations’. If micro-foundations are also defined as functions, then their role in systematic analysis can be understood more comprehensively.

4. The interdependent functions of Innovation and Agility

Innovation and Agility are driven by a common intent (to facilitate beneficial development) but they have (often implicit) different Theories of Change and, therefore, different practises of action. What is the essence of the difference? It can be understood by considering a thought experiment. Imagine that in a pandemic thousands of additional medical ventilators are needed urgently. If Agility is the Theory of Change adopted then those with entrepreneurial power will provide direction and resources, motivate varied enterprises to engage in design and reward those who produce working prototypes. If Innovation is the Theory of Change adopted then action would be taken to bring together top scientists and engineers, establish a temporary development centre and use skunk-works methods to create new designs for ventilators that could be rapidly manufactured. What can we learn from this thought experiment? It is that the Agility’s modality is ‘do whatever it takes to get things done’ and Innovation’s modality is ‘do the work to create something new and valuable’. Briefly, Agility’s essence is proactive but prudent opportunism (characteristics of entrepreneurship) and Innovation’s essence is targeted structured creativity (characteristics of engineering).

This suggests that Innovation and Agility come from different conceptual worlds. Innovation integrates disciplines such as creative design thinking with the rigour of the scientific method and the problem-solving orientation of engineering. Agility draws from a Schumpeterian heritage as it embraces the wiliness of a hunter, the action orientation of a (good) politician, the pragmatism of an army commander and the unreasonable dedication of a serial entrepreneur [43].

Francis [15] summarised the essence of the relationship this way: “Agility and Innovation have a close, but complex relationship, rather like that sometimes seen between brother and sister… Is there a difference between Agility and Innovation? The answer is ‘yes’, as the nature of work, the driving force and the associated risk profiles, are different for each… Agility and Innovation have different clock-speeds. Innovation requires finding and exploiting new ideas and is frequently time-consuming, uncertain, expensive and difficult… Agility has a rapid heartbeat. It is rapid, lean and acquisitive… Why is the relationship between Agility and Innovation complex? Many organisations, like the ancient Roman God Janus, must face two ways and be both agile and innovative”.

5. Conclusions

There are three significant implications of the insights and frameworks that have been presented in this chapter.

First, it is noticeable that academic researchers who study strategic and organisational development favour investigating Innovation over Agility. A Google Scholar search for articles with the term ‘innovation’ or ‘innovative’ in their titles produced just under 5000 hits for the first four months of 2023. A similar search for the terms ‘agility’ or ‘agile’ produced far fewer: just over 1000 hits with some being related to medical issues or the training of dogs. Interestingly, the relationship between Innovation and Agility has been infrequently studied (just 22 articles were found by Google Scholar that included the terms ‘innovation’ and ‘agility’ in their titles the first four months of 2023). Hence, it is right to entertain the possibility that Innovation may have become an overused construct in scholarly work, resulting in what Kaplan [44] described as: “I call it the law of the instrument, and it may be formulated as follows: Give a small boy a hammer, and he will find that everything he encounters needs pounding”.

Second, the role of Agility is so central that there is a pressing need for managers, and for those who are studying management, to strengthen their personal and professional readiness to be prudent and effective in using this Theory of Action. Currently, in many Business Schools, managerial education is just that—education-centric. A VUCA (volatile + uncertain + complex + ambiguous) world needs managers with Agile competencies who take responsibility for key areas of their learning to develop their proactivity, organising skills and capacity to undertake enterprising ventures.

Last, the construct of Dynamic Capabilities has been extraordinarily influential since it was introduced to scholars and managers in the early 1990s. As mentioned earlier, the primary theoretical lenses used to develop this paradigm were economic analysis and strategic management. We consider that the use of a sociological lens (Structural Functionalism) will further enrich the construct as can help in clarifying how instruments (such as Innovation and Agility) play a significant role in both deploying and renewing Dynamic Capabilities.

Areas for future research can include: (i) examining in greater depth the relationship between Innovation and Agility and current models of strategy; (ii) the role of middle-range theories in organisation development; (iii) the role of Innovation and Agility in a larger number of cases and (iv) investigating and specifying the competences needed for effective ‘innovation work’ and ‘agile work”.

Acknowledgments

The constructs presented in this chapter were developed jointly by David L Francis during a twenty-year research programme and Mike Woodcock who is a serial entrepreneur, was a Member of Parliament in the UK and is chairman of large enterprises and charities. The research studies underpinning the frameworks presented in this chapter were undertaken in the Centre for Change, Entrepreneurship and Innovation Management (CENTRIM) in the University of Brighton (U.K.) with the support of Professor John Bessant, Dr. George Tsekouras and Dr. Stefan Speckesser. Valuable inputs were contributed by Amit Arora, Geordie Keith, Evelyn Simon and other thought-leaders in the Requisite Agility interest group. Helen Price of Henley Management College helped us to understand the management development implications of Innovation and Agility. Professor David Teece supported our perspective on the need for Requisite Agility, rather than seeing Agility as a universal requirement and Carlos Seaton provided valuable insights into the development of innovative ecosystems.