Decision-making in Risk Management

Abstract

The definition of risk introduced in the ISO 31000 standard of 2009 (2018) is uncertain goal achievement; thus, both negative and positive outcomes can be considered. It also implies that risk is not limited to life and health, but may cover all goals of a company. Risk management thus becomes a question of achieving and optimizing multiple goals. Since safety is but one of several considerations, safety may lose out to other more easily measured objectives of a company, such as economics and compliance with regulatory requirements. Risk analyses have a long history of quantification, a tradition that for various reasons has waned and should be revived if safety goals are to be treated together with other goals of a company. The extended scope affects not only company owners and employees but also neighbors, the local community, and the society at large. The stochastic nature of risk and the considerable time lap between decisions and the multiattributed consequences implies that managing risk is exposed to cognitive biases of many sorts. Risk management should be based on a quantitative approach to risk analysis as a protection against the many cognitive biases likely to be present, and managers should be trained to recognize the most common cognitive biases and decision pitfalls.

1. Introduction

Accidents happen, in the past and present, and efforts to analyze how to avoid their reoccurrence have always been the backbone for improvements in safety. Through the study of the causes and statistics of accidents, their frequency and consequence severity have been reduced. Analytical algorithms and tools were developed, mainly after WW2, supplementing the safety improvements of accident investigations. The analytical approach has evolved considerably over the years in terms of improvements in methodology and calculation capabilities. The evolution has also been a response to the extensions in the scope of both risk causes and consequences, i.e., goals.

Some of the mathematics and statistics of risk were developed to meet the need to decide the average remaining lifetime to estimate the cost of life insurance policies [1]. Practical risk reduction knowledge has accumulated since then in high-risk industries like shipping, chemical plants, oil and gas, nuclear power plants, aviation, and space exploration. Risk was defined in relation to unwanted consequences, as a function of the probability with which an event may happen and how severe it might be (see [2] for an overview). If the causes of risk are known and probability data exist, risk can be calculated in quantitative risk analyses (QRAs). Making decisions based on the results of risk analyses in a systematic way inspired the concept of risk management, with the aim to reduce risk based on findings from QRA. The quantitative nature of this approach made cost-benefit analyses possible. If properly carried out, the result was a better utilization of limited resources, be it money, experts, or other means.

The different applications of risk management in insurance, finance, and industry were developed with little mutual exchange between them. The risk management tradition of finance looked at risk including both gains and losses because of investments, while in industry and engineering, risk was associated with potential loss only. Because risk is an expression of events that may happen in the future, risk is intrinsically uncertain. The decisions that may trigger such events are often made to achieve multiple goals, e.g., profit while maintaining safety margins related to health and environment. The question of how to balance several goals is not trivial. Some might be in conflict, others might support each other. There can be different stakeholders affected by the decision, with different priorities and power of influence, and they might be involved directly or indirectly. The stakeholders can be owners, employees, neighbors, politicians, NGOs, or competitors. Some goals might be certain and others uncertain. Some of the effects of decisions can happen in some distant future, raising the issue of discounting. Since humans’ discount is differently than “econs,” the rational utility maximizing economic man, the stage was set for differences in opinions and priorities. Decision-making in risk management is therefore a practical application of judgment under uncertainty, a research field developed by Tversky and Kahneman [3, 4] leading to the study of cognitive biases and becoming the foundation for behavioral economics [5].

The definition of risk has undergone major changes, from the product of the severity and probability of unwanted events to uncertain achievement of multiple goals, as reflected in the ISO 31000 “Risk Management,” a guideline developed for risk management systems [6]. When the scope is lifted to include the whole company and all its objectives, the concept of enterprise risk management (ERM) is used. In parallel with the “engineering” approach, the auditing and accounting professions have developed an approach to ERM under the COSO label [7] with emphasis on fraud prevention and audit of accounting. Comprehensive systems on how to reduce risk to an acceptable level on a continuous basis are commonly described as Safety Management Systems (SMS), reflecting a broad approach including risk analyses, safety assurance, incident investigations, safety inspections, and audits. In aviation, SMS includes the evaluation of incidents with respect to quality the remaining barriers as well as safety issues that may require a more detailed risk analysis [8].

Concurrent with the development of SMS, vetting systems have emerged as background checks of both people and systems. Vetting is a case-based inspection used by a diversity of institutions, from public agencies in border control to oil majors in relation to suppliers. When an oil tanker is nominated to a charterer and considered for lifting cargo at a terminal which requires the consent of an oil major, the oil major will “vet” the vessel, i.e., inspect and approve the vessel for visits to that terminal. This is usually regarded as a more critical inspection than the internal audits performed by the shipowner because the consequence of a failed vetting is a loss of business. SMS and vetting systems complement each other as the former is a continuous and systems-based approach, while the latter is more detailed and adapted to a practical case.

The different definitions of risk and approaches to mitigate risk may have both a positive effect and a negative effect. On the positive side, competition can lead to improvements in achieving results at a lesser cost. Negative effects can be unnecessary activities and conflicts between the various safety assurance actors, with more bureaucracy and higher costs than necessary. A short history of how risk management emerged will be presented before possible improvements in risk management are discussed. The focus will be more on the practical and less on the theoretical aspects and more on management challenges and less on risk calculation issues.

Deliberate actions to harm, like sabotage and terror, are not covered, although some of the considerations made might be of relevance to acts of terrorism as well.

2. A brief history of risk management

2.1 Approaches to risk reduction

Risk causes included in risk analyses have increased in scope; from an initial focus on technical failures and extreme environmental conditions via operator errors to include problems originating in the culture of organizations and (lack of) management commitment. The trend to include a wider causal picture came gradually during the second half of the last century, motivated by the results of investigations of some spectacular major accidents, like the Bhopal gas disaster in India in 1984 [9], the space shuttle accident of Challenger in 1986 [10], and the capsizing of the roro ferry “Herald of Free Enterprise” in 1987 [11]. Major accidents also occurred in finance, like the bankruptcies of Enron in 2001 and Lehman Brothers [12] in 2008, just to mention a few well-known cases. Concurrent with the extension of the scope of causes, the range of risk consequence has widened and includes effects on the environment, social responsibility issues, and company reputation.

The extensions in scopes of both causes and consequences have made risk assessment more challenging as methods for quantitative risk analyses have not caught up with the increase in scopes. Software tools have made risk management easier as far as the “bookkeeping” of risk status and mitigations is concerned. The quality of the content of the risk registers is however another question, because the extension in scope has come at a cost. The “softer” causes and consequences are usually not quantified since cause-effect relations are difficult to identify and estimate. Authorities require risk assessments of new endeavors and major changes, to be followed up by the industry, although there is shortage on both proper methodologies and qualified analysts. One compromise to this issue was to relax the requirement for quantitative results, if not in theory so in practice. QRAs were no longer behind the wheel and were moved to the backseat of risk management. The lack of numeric rigor in expressions of risk opened a Pandora’s box of more subjective assessments. The result was a considerable growth in the number of accident investigations and risk assessments whose quality is dubious. It is difficult to judge whether this development was for the good or the bad, as even a poorly conducted qualitative risk analysis could produce interesting findings; at least risk workshops made people talk to each other over the border of department silos, thus improving in-house communications on risk issues.

Improvements in calculation methods, more powerful computers, and software may help to bring QRA back on track by making it possible to apply advanced modeling techniques capturing both stochastic aspects as well as the dynamic properties of evolving systems. The systems are “hybrid” in the sense that they consist of both technical and human parts, the combination of which raises a principal challenge as to how events taking place in the two are to be explained. This is not a new challenge, as it also was the case with man-machine systems and control room operators. Physical phenomena are explained through cause-effect relations, as are also human actions, and work well for simple cases. The ability to predict future states of physical systems is however reduced when complexity increases, e.g., in forecasts of weather. Prediction becomes even harder when the systems contain humans supposed to make multiattributed decisions, as in politics and economics [13], and one might add, as in risk management.

One remedy is to improve the utilization of knowledge from behavioral and cognitive science in the decision-making part of risk management. Behavior science is relatively young compared to physics and engineering. It is regarded as “softer” in nature and harder to quantify. Of more concern is that the different “schools” in the social and human tradition might give an impression of fragmentation, as disagreements rage between different professions and disciplines. This situation is real, unfortunately, and the main differences are related to what the core subject of behavior science should be and how to explain the phenomena studied. The approach promoted in this chapter is psychology as a natural science of behavior. The explanatory concepts are like those used in evolutionary biology, variation, selection, and replicators, to explain fitness in adaptation rather than cause-effect relations between the mind and body. Like natural or Darwinian selection explains phylogenetic evolution (genetics), so is behavior explained by the selection effects of the consequences of behavior during the lifetime of an individual (ontogenetic evolution) [14].

In the following, a historic review of how the origin of risk has been investigated and understood will be described before possible improvements to the decision-making part of risk management is discussed.

2.2 Accident investigations

An old saying states that fools never learn and the smart ones learn from their own mistakes, while the wise learn from others’ mistakes. In other words, improvement starts with efforts to prevent the unwanted event from occurring again, by observation and learning from own or others experience. The key is to identify and understand the causes that made the accident happen to prevent it from happening again. Although this was a reactive approach, over the years the gain was huge. Expressions like the anatomy of accidents and unsafe acts were introduced. Earlier, once a human error was identified, the analysis was believed to be complete, a misconception that could easily lead to a search for scapegoats. Without an understanding of what caused behavior failures, the search for human errors implied to find the responsible individual. Often, this was the man at the end of the chain of causes, the last contributor before the accident. As situational factors were understood as the main causes of human behavior, unsafe acts were considered the result of lack of training, time pressure, man-machine interface design, and other error-prone situations, and human errors could be reduced if precautions were taken [15].

Investigation boards covering several industries were established as national agencies in many countries. Accident causes were categorized as independent or contingent on other events, and as the physics of accidents were better understood, the logical relations between the events, their timing, and sequence leading up to the accident could be described more accurately. Unsafe acts no longer were the sole precursor to accidents, and a more complete causal picture appeared. When a human error was identified that had contributed to an accident, that signaled the start of the analysis, not the end. The chain of causes was further expanded when the investigation moved away from the immediate proximity to the operator and to the functions of the organization, management, owners, and the way the regulatory authorities operated. This extension of the causal scope was undoubtful of value, as people higher up in the organizational hierarchy could influence risk much more than the person at the sharp end of the line.

The change in focus from those executing operations to the designers and planners in management reflects the advance in technology which was about to change the primary human role in work life from manual labor to planning and decision-making. Management, organization, and culture were firmly included as topics to be addressed in accident investigations in most countries by the turn of the century. Aviation can serve as an example of successful accident investigation history. Flight anomalies and pilot error reporting are mandatory for both airliners and ground control. The pilots and air traffic controllers filing incident reports are not subject to legal prosecution unless there is a case of deliberate and serious misconduct. The fact that air flights are possibly the safest transportation means of all when exposure is measured per unit of distance and not per time unit is largely due to lessons learned from nonpunitive reporting systems and findings from thorough accident investigation efforts.

2.3 Technical risk

As accident investigation was established as the primary way of enhancing safety, accident causes were initially understood to be technical and human failures. Improvement both in the reliability of components and in how they were combined in systems resulted in fewer accidents. As reliability theory and calculation methods and tools became available to the engineering community after WW2, QRA methods were developed [16]. Techniques of a more qualitative nature were also developed, like failure mode and effects analysis (FMEA), hazard and operability analysis (HAZOP), and various barrier analyses. These new proactive analytical tools made it possible to improve safety before accidents happened and proved to be an important complement to reactive techniques like investigations of accidents and incidents.

The Reactor Safety Study [17] was probably the first “total” quantitative risk analysis (QRA), also called the Rasmussen report or WASH-1400. The report was published in 1974 after 3 years of work involving more than 50 contractors at a cost of about 4 million USD, equivalent to about 30 million today. The analysis was based on a system reliability approach, where component failure rates were combined using Boolean logic, represented graphically as logical gates in fault trees. The objective of the WASH-1400 study was to calculate a realistic estimate of the risk posed by nuclear power plants as a response to public claims that this new way of producing energy was very dangerous. The study concluded that it was about 1 million times more likely that car driving would be fatal. The study was criticized, partly because the nuclear risk was calculated, while the comparative risks, e.g., from traffic accidents, was based on statistics of real events [18]. The most influential result of WASH-1400 study was that it served as a recipe for similar analyses in other industries, e.g., the offshore oil and gas exploration in the North Sea.

One main reason for the early popularity of risk analyses was that the fault and event tree modeling approach was scalable to any plant type and size, if design drawings, P&ID, and component failure rates were available. The QRA made it possible to include the human as a system component that could fail, like a valve, a pump, or a vessel. In this way, the stage was set for the development of human reliability assessment methods that could feed human error probability data into the system reliability models.

The practice of applying risk analysis methods spread to other sectors and industries. Environmental impact studies were prepared built on the same logic. Consequence assessments were required before approval of large-scale industry and real estate development projects. Some years passed, however, before risk analyses became a required part of safety work in aviation. One probable reason for this late start is that accident and incident investigations had become quite advanced and were used to a large extent in aviation, providing ample evidence for their positive contributions to flight safety. As the saying goes, don’t change a winning team. The various safety methods are however better considered as elements of a broader safety effort, each contributing in their own way to improvements. Risk-based SMS are now mandatory for airliners, airport providers, and air traffic control service providers.

2.4 Human risk

Because the systems that failed also needed humans for operation, maintenance, and repair, human reliability became part of QRA. Assessment methods for human reliability for industrial and defense applications with high potential for major accidents were developed. One early example is control room operations in nuclear power plants [19]. Human tasks and their error probabilities were modeled using event trees like THERP, and tables of human error probabilities were published in a handbook for use in risk analyses [20]. Human errors could be omissions or commissions, meaning that something was forgotten or a wrong act was carried out. Later versions of human reliability models stated human error probabilities as a function of performance shaping factors (PSFs). The models were calibrated using data from experiments, statistics, and expert judgments. Examples of PSFs are quality of the man-machine interface, violation of stereotypes, too high or too low stress level, isolated acts, conflict of motives, quality of feedback, etc. [21, 22]. The human error models were mainly motoric tasks or simple decisions related to the execution of the tasks.

Safety research programs were nurtured by the growth in the British and Norwegian oil and gas offshore activities. In Norway, an increase in safety funding became available after the Aleksander Kielland accident in 1980 where a capsized floatel resulted in 123 fatalities. The Piper Alpha accident on the British sector in 1988 also served as a boost for increased safety efforts, resulting in the safety case approach [23]. The state safety funding in Norway was mainly devoted to occupational safety, workplace democracy, and socio-technical issues, while means to develop human reliability lessened. This was possibly due to pressure from labor unions who exerted considerable influence on the governmental financing of safety research. The focus on the worker as a contributing factor to risk was not politically acceptable, even though situational factors or PSFs were modeled as human error causes. When empirically based failure rate repositories were developed [24], and human error was included in the equipment failure rates, the need for human reliability data vanished, and the human reliability profession was history. In Britain the situation was better due to the larger industrial sector and cooperation between industry, universities, and consultancies [25].

The way humans contribute to risk ranges from simple motoric tasks to complicated decisions that include other people and other institutions. The former was developed quantitatively as human reliability, while the latter, decision failures, have so far not been formally included in QRA to this author’s knowledge. Decision errors have however been extensively studied by behavior science in the cognitive bias tradition. The absence of an analysis of decision failures in risk management is probably related to lack of empirical data, the high complexity of decisions under risk, and the shortcomings of behavior science in this area. Decision behavior is by no means understood sufficiently, although progress during the last couple of decades has been significant, as exemplified by cognitive bias research and the policy relevant “nudge” tradition [26]. Machine learning, Bayesian network, and self-learning AI robots are promising research disciplines. For now, a closer look at a few cognitive biases relevant for risk management will have to suffice.

2.5 Financial risk

Economics is probably the field of human endeavor that has been most concerned with risk, covering uncertain outcomes of both positive and negative values. New challenges appear as robots and artificial intelligence (AI) are being applied in finance and trading of stocks and derivatives. It is interesting to observe that AI algorithms use operant selection in AI self-learning, adaptive systems. The rapid innovation is a challenge for regulations because regulatory requirements usually are lagging new technology.

Angner defines economics as “the manner in which people make choices under conditions of scarcity and the results of those choices for society at large” [27]. In Anger’s textbook Adam Smith is considered the founder of modern economics and author of influential books like The Wealth of Nationsand The Theory of Moral Sentiments. Smith regarded the economy as a self-regulating system where the price mechanism would balance the supply and demand and thus result in the best allocation of scarce resources, aided by competition in the market and humans driven by self-interest. The self-governing system would reduce the need for a supreme regulatory power, being it the state or the church. The idea that liberty and individual freedom with a minimum of regulation would lead to prosperous outcomes for all members of society can be traced back to the Age of Enlightenment, a movement in Europe during the eighteenth century essential for the opposition against religious and feudal governing of people and commerce. The core of the economic system was the rational, utility-maximizing economic agent or “econ,” whose behavior was considered both as normative and descriptive. These assumptions were to be criticized from political and behavioral points of view, respectively.

The economic liberty and individual freedom resulted in a much higher productivity and thus accumulation of wealth, but not for all. The politically based critique of the self-regulating economy leading to prosperity was based on the resulting skewed distribution of the new wealth. This controversy still exists and fuels the conflict between capitalist and socialist ideas on governance of a society and attitudes to market economy.

The critique from the behavioral side was based on research showing that most people did in fact not behave like “econs.” An important contribution to understanding how decisions were made beyond the rational-agent concept was the work of Herbert Simon [28] on bounded rationality. When the article “Heuristics and Biases” by Tversky and Kahneman [3] was published in Science, human decision failures defined as deviations from the choices of an econ became a prime subject of psychological experiments. Prospect theory was published in Econometrica[29] by the same two authors a few years later, formalizing the basis for cognitive decision research. Although the lack of a clear definition of a “bias” was pointed out [30], the research on cognitive bias and heuristics flourished. The impact reached far beyond the field of psychology, as the empirical foundations of neoclassical economics came under attack (see Thaler [5] for a historic overview of the emerging field of behavioral economics). The Nobel Memorial Prize in Economic Sciences was given to Simon in 1978, to Kahneman in 2002, and to Thaler in 2017 as a recognition of their contributions.

That the assumption of the rational agent, economic man or “econ” for short, was disputed did not imply that the rationality of economics as such was rejected. Economic man was assumed to be the normative case of how decisions should be made if the goal was to maximize outcome for the decision-maker. The prescription fits the stereotype of an omniscient business executive doing transactions in a commercially competitive society. It is unclear why leading economists proposed the rational utility-maximizing agent as a generic, descriptive model for human decision behavior, thus confusing the descriptive with the normative. One reason could be to strengthen the legitimacy of economics as a science based on rational humans, to be backed further by the most rational scientific discipline of all, mathematics. Another reason for the misperception of normative and descriptive could be the lack of interest shown by psychologists, including behavior analysts, in decision-making in business and industry. The “invasion of economists” into the land of the social and behavior sciences might explain some of the skepticism toward economists from behavior scientists. The resentment between the two disciplines might have been strengthened because economists are preferred as managers and administrators, especially in the commercial private sector, and few with a background in behavior science seek such employments or are preferred as candidates.

The Enron case has been portrayed as an example of a major financial risk and an example of willful corporate fraud and corruption, and it led to the dissolution of the Arthur Andersen accounting firm. The various financial disasters that followed were probably not committed conscientiously, as the human’s capability to self-justification seems limitless. The Enron case was a major motivation for the Sarbanes-Oxley Act of 2002, leading to much stricter accounting rules. Paradoxically enough, it also led to a burst in the business of accounting companies who then later was delivering the services needed for companies to comply with the new rules. A better alternative would have been to introduce quantitative risk management methodologies developed in the engineering domain, and adapted that to finance, rather than to enforce stricter auditing philosophies that so far had been proven insufficient. The mitigation of finance risk has therefore proceeded more as a kind of compliance management rather than risk management.

2.6 Organizational risk

The change in production technology from manual labor, via mechanization to automation, resulted in more management type of activities like designing, planning, and decision-making. As a reflection of this development, a sociological perspective on risk was introduced to explain why accidents occurred [31]. Concepts like “normal accidents,” “an accident waiting to happen,” “tightly coupled systems,” and “interactive complexity” were introduced to describe the vulnerabilities of high-risk companies. This new understanding was first applied in accident investigations, and the stage was set for a search for causes to risk in the way the organization, i.e., management, was prepared for, or rather was not prepared for, safety. There is however little agreement on how an organization should look like for operations to be safe. Studies of the so-called high reliability organizations (HRO) might give an indication [32], but their way of conducting operations would hardly be accepted as role models for the industry in general.

The Swiss cheese metaphor was introduced to illustrate defense in depth [33] and is a visual representation of how barriers can fail simultaneously, visualized as slices of cheese with holes that are lined up. This was a failure situation that for decades had been modeled in reliability engineering by fault trees with the more precise Boolean and gates. The sociological perspective got a strong foothold in accident investigations but did never make it to the QRA teams other than to visualize and illustrate. A new metaphor is not necessarily an improvement, especially not when attention is diverted from logics and calculation. The sociological perspective on accidents reinforced a qualitative approach to the study of organizational risk causes.

As far as the human contribution to risk is concerned, the change resulted in a move in responsibility from the operator to management and owners. The sociological view on causes to accidents, helped by the auditing focus on compliance with regulations, could non-intentionally lead to the blame and shame culture being lifted from the shop floor to the board room.

The sociological perspective on risk came with a political flavor as management could be considered as potential culprits causing accidents. It put the focus on commercial pressure, reduced manning, budget cuts, insufficient training, and if management failed as role models for safety, by paying lip service to safety priorities while acting otherwise. Not walking the talk was a sure trail leading to a depraved safety culture. This critical attitude toward private enterprise and business is understandable given the safety scandals of the time, as the case of the Pinto can exemplify.

3. Risk management improvements

The story of the Ford Pinto illustrates two interesting issues of relevance to risk management and decision-making. The first issue is the low priority given to safety at the time, and the second one is how animosity toward setting a monetary value on life can hamper safety improvements. The priority issue has been rectified as safety is now given a much higher importance, partly because of negative press coverage and lawsuit compensations. The last issue is related to a reluctance to set a monetary value on life and is still controversial. These two issues need some explanations to bring home. It should be noted that the presentation given in the following is based on the report prepared by the Mother Jones magazine [34].

3.1 The issue of safety priority

During the 1960s the American car industry was met with fierce competition from European and Japanese manufacturers who were targeting the lucrative small-car market. The response from the Ford Motor company was the Pinto, a subcompact car that was put into production at record time. This was achieved by concurrency in engineering design and production of assembly line tools. The consequence was that design changes would be costly if they required any change in manufacturing tools. As the first cars were manufactured, collision tests revealed a serious safety issue related to the position of the fuel tank. It would easily burst by the impact from a rear-end collision, even at low speed. The car would be engulfed in flames if the gasoline was ignited, which was likely to happen because the impact itself would produce sparks. Another safety flaw was that the doors would be jammed at a moderate collision speed, rendering escape impossible. Ford management knew about the design flaws, but nothing was done about it although cheap measures were possible. Safety was not given priority, and money was rather spent on lobbying against safety regulations that were being prepared for the auto manufacturers. More than half a million cars were produced each year, making a huge profit for the Ford company. The number of rear-end collision fatalities has been estimated to be in the range of 500–900 during the 8 years before Ford finally incorporated safety improvements.

The many fire accidents caught the interest of the Mother Jones magazine. Several hundred reports and documents regarding rear-end collisions were studied, including the tests made by the company itself. It was also revealed that an internal memo sent to senior management had compared the cost of redesign of the hazardous position of the fuel tank with the off-court settlement cost of humans that would suffer from accidents. Applying the value of life provided by the National Highway Traffic Safety Administration (NHTSA) [35], the memo concluded that Ford would save almost $70 million by allowing accidents to occur. Mother Jones published the story in their August 1977 issue, and the reaction was devastating. Criminal charges and lawsuits were made, all Pintos were recalled, and the Ford Motor company got some of the worst press an American car manufacturer has ever received.

Although traffic safety had improved over many years [36], it was the investigative journalism by the Mother Jones magazine, and the following attention of the press in general, that made a whole industry set higher priority on safety.

3.2 The issue of the value of life

A cost-benefit analysis applied in the management of safety risk will require a monetary value of human life. The benefit from improved safety is calculated as fatalities and injuries avoided. The costs are mainly due to the mitigative measures and production loss needed for the implementation of the measures. A QRA estimating the risk reduction effect of the mitigation may provide the benefit value, if the values of life and health are given. This way of thinking is considered cynical and calculating by some people. Not doing cost-benefit calculations is for quantitative risk analysts equal to missing the opportunity to save lives.

Safety competes with all other objectives of a company, and the easiest to calculate are economics, production logistics, and marketing. Management must be convinced to initiate safety measures for unacceptable risks. The most cost-effective mitigation measures can be identified if a QRA is prepared. The CEO of a company has usually no training in safety and QRA methods, and neither has any of the other directors nor vice presidents.

The Pinto story is not unique, and the car manufacturing industry might not be the worst. A possible side effect of the media focus and lawsuits to track down the responsible individuals or company can have strengthened a reluctance to apply risk analyses to improve safety, because it is always possible that no-cure-no-pay lawyers could use the results in future lawsuits. A verdict in disfavor of a company is more likely if there is a reason to believe that the risks were known by management, even if being at acceptable levels.

It is likely that the Pinto and other cases that made the headlines prepared the ground for the introduction of the concept of the “amoral calculator company,” which is a way to describe different types of business firms on how they would respond to safety regulation and enforcement [37]. The amoral calculator type of companies and management was assumed to be mainly driven by self-interest and profit maximization, assuming they calculated costs and benefits in relation to safety measures to see what they could get away with. An ill-fated consequence of this kind of thinking is that doing cost-benefit calculation of safety measures by itself could be considered an amoral act. This is detrimental to safety because it will make the identification of the most cost-effective safety measures difficult.

4. Behavior analysis and risk management

Simply stated, risk management implies making decisions to influence risk in a predicted and controlled way. The expression above rests on the following criteria: knowledge about the risk level and safety margins are available, and the decision-makers are trained to observe and obey the risk acceptance limits. There are however several reasons why this may not happen: first, QRAs may not be carried out, so there is no reference, i.e., no quantitative measure of the current risk or trend. Second, risk acceptance limits are not defined, so if the risk was known, there is no knowledge of it being too high. Third, decision-makers are not trained to observe and act based on trends in the risk level, relative to risk acceptance limits. And finally, human decision behavior is vulnerable to a range of cognitive biases involving thinking and emotions. Failures to meet the assumptions above can result in a faulty risk management process. The most relevant cognitive biases for decision-making in risk management will be described in the following.

4.1 The base rate neglect and exposure

One of the best known cognitive biases is base rate neglect [3] which occurs when background information is disregarded and the decision is based on superficial and less relevant information. Risk is expressed relative to exposure, like the number of events or incidents divided by the number of opportunities for incidents to happen. When catchy and stereotypical descriptions dominate or replace base rates, decisions may be based on deceptive heuristics reflecting these stereotypes. The base rate neglect bias is especially relevant to qualitative risk estimations because this type of risk analysis does not require quantitative exposure data. The result can be hazard adaptation, an unnoticed slide toward a more lenient risk acceptance behavior. Prior to the faulty decision to launch the space shuttle Challenger in January 1986, the decision process failed on several of the criteria mentioned above. The outcome was the loss of the shuttle and the lives of seven crew members. Better knowledge of QRA and cognitive biases in decision-making under risk and uncertainty might have changed the fatal decision and avoided the accident [38].

4.2 The optimism bias and variation

The optimism bias can be described as a general overestimation of our performance in our favor. In a review of biases [39], it was concluded that optimistic illusions are the only group of misbeliefs that might be adaptive. The optimism bias is associated with harder and longer work periods, which may account for higher pay and promotions [40]. The optimism bias is also associated with an optimistic view of future events and an increased will, and thus ability, to predict and control future outcomes. In the inverse situation, as when prediction and control is not possible, the result is reduced ability to learn. The term “learned helplessness” was coined by Seligman to account for these effects [41].

The optimism bias or overconfidence is probably one of the most common and strongest human fallacies [42]. The bias is found in many different countries and cultures. Examples of the optimism bias are that we engage in more new ventures, establish new relationships, buy lottery tickets, etc., in areas where the expected benefit is much lower than the effort invested [43]. Many new businesses would probably not have been started and inventions not made, unless the effort required was underestimated. Many more activities are started than a realistic and rational utility-maximizing agent would initiate, making the optimism bias the mother of variation and innovation.

Translated into evolutionary terms, the optimism bias fuels the variation upon which selection operates. This is probably the case for both types of evolution: phylogenetic as in genetic inheritance and ontogenetic as when operant behavior is selected by the consequence it produces.

Overconfidence is rampant; we are all susceptible to it, and particularly in skills we do not master well. A large majority of drivers (above 90%) believe that they are better than the average driver [44]. The Dunning-Kruger experiments indicate that the less we excel in something, the more confident “experts” we tend to believe we are [45]. Training and education might however help, as the experiments indicate that high competence reduces neglect and overconfidence.

The optimism bias may account for the frequent lack of realism in project planning and budgeting, in addition to other more tactical causes like securing the approval of a project by promising too much. They provide good arguments for applying quantitative project risk analysis, an application of QRA to projects. Realistic means for time and money should be calculated before a project is launched, with defined confidence limits. The assurance arrangement for public projects above 750 million NOK issued by the Norwegian Finance Department around the turn of the last century [46] is an example.

4.3 The confirmation bias and selection

People tend to come up with a hypothesis and then to find support for it, instead of trying to prove it wrong. When we are sufficiently confident about our presumption through confirming, we stop searching even though there could be better alternatives. The presumptions can be beliefs and rules for conduct of the form “if you do this under those conditions, the result will be such and such.” Or they can be of simpler form, like stereotypes or weak correlations. Rules do not always produce the expected result. In behavioral terms, they are maintained on an intermittent or partial reinforcement schedule.

The confirmation bias may lead to the following of premature and false rules. Confirmation behavior also has positive effects as it serves to stabilize conduct and makes us more predictable, enhancing social acceptability. This is good if the rules are good. The downside is that one also becomes a reliable follower of rules that are not optimal and sometimes disadvantageous. Conformity and lack of innovation may be a high price to pay for social and political acceptance.

The Behavioral Insights Team (BIT), partly owned by the UK government, has identified the confirmation bias as one of the most prominent barriers against learning new skills and innovation [47]. In a recent study, the news consumption pattern of 376 million Facebook users was analyzed, showing that most users preferred to get their news from a small number of sources they already agreed with, further bolstering existing beliefs and preventing new insights [48].

4.4 Cognitive dissonance contingencies

Cognitive dissonance was introduced as an explanation of choice behavior in a situation of ambiguity [49] and is an example of an activity where both respondent and operant behaviors are present. The influence of affect in decision-making can be very significant [50]. One example of a bias is the “halo effect” that may occur when strong positive reactions are reflected over (conditioned) to otherwise neutral stimuli, an effect of stimulus generalization in classical conditioning. The opposite reaction is called the “horn effect.” One aspect of emotional reactions is that they tend to be either positive or negative, experienced as pleasure or pain, broadly stated. When there is correspondence between, e.g., our belief and what we perceive, i.e., confirmation, there is a feeling of pleasure. When there is a discrepancy of some sort, the feeling is aversive.

In behavioral terms, and somewhat simplified, we might say that cognitive consonance, i.e., confirmation, produces behavior that is positively reinforced, while dissonance produces behavior that is under aversive control, resulting in either escape or avoidance behavior. Repeated instances of a response that effectively terminates a dissonance may become an automated avoidance response. A similar argument can be stated regarding consonance; it keeps behavior on a steady course. Automated behavior escapes our attention, and we do not notice neither the dissonance nor the consonance. Automated decision behavior on “autopilot” is energy efficient as deliberate considerations are not carried out. The result might be a failure to notice changes that should have induced another decision outcome.

If strong conflicting emotions and values are involved, cognitive dissonance behavior is difficult to modify [51], in particular if there are automated reactions involved. Accusations of being biased may strengthen the aversive emotion, as it moves the attention from the outcome of the decision to the person making the decision. Attribution to a person often leads to reactions of defense in the form of self-justification, and the behavior might become extremely resistant to change [52]. The defense behaviors have become automated avoidance behavior, either as self-justification, as counterattack of some form, or a combination of the two. Automated behavior is not reflected upon; it is subconscious. Automated self-justification includes self-illusions or blind spots for everybody to notice except the self-justifier himself [53], a favorite subject in many comedies.

To favor beliefs that are not falsifiable is a powerful form of defense against cognitive dissonance as it will protect against being proven wrong, which is an unpleasant experience for most people. Confirmation behavior may be maintained by both negative and positive reinforcements. An individual might have a confirmation behavior that is simultaneously maintained as avoidance/escape behavior and as positively reinforced gratification-seeking behavior (ref. the Pollyanna principle). Multiple contingencies might explain why confirmation behavior is a very strong default option and why falsification is so rare as a belief-testing strategy in daily life as well as in making risk management decisions.

4.5 The power of inadequate rules

Behavior maintained on a thin reinforcement schedule is more resistant to extinction and change than behavior that has been reinforced according to a continuous schedule [54], a phenomenon called the partial reinforcement extinction effect.

Rare confirmation of rules or beliefs implies that the following of such rules is maintained on an intermittent or partial reinforcement schedule. The result is that vaguely formulated rules and beliefs as are typical for qualitative risk analyses often are more resistant to change than rules that are more precise and correctly formulated. Unclear and rarely confirmed beliefs tend to have more dedicated and convinced followers than rules and beliefs that reflect reality more precisely. It is a paradox that, at least within certain limits, the less correct a belief is, the more convinced the believers are.

A similar phenomenon is observed in the Dunning-Kruger effect commented earlier, if less skilled implies have beliefs that are less correct and adequate than more skilled individuals. The Dunning-Kruger effect states that low-competence individuals tend to believe that their ability is higher than it really is. High-competence individuals have a more realistic view and may even slightly underestimate their performance. The authors comment that when people are incompetent in their strategies to achieve success, they suffer a dual burden: not only do they reach erroneous conclusions and make unfortunate choices, but also their incompetence robs them of the ability to realize it. Instead, they are left with the mistaken impression that they are doing just fine. This is also an example of the blind spot bias. As it is said: “First rule of the Dunning-Kruger Club: You do not know you’re in the Dunning-Kruger Club.” This statement is unfortunately valid for most biases, as being aware of a bias does not protect you from being biased.

5. Conclusion

In this chapter, a short history of risk management was presented before the most prominent cognitive biases were discussed. Due to their evolutionary past, they are natural to our behavior repertoire and difficult to change and avoid. To make choices under uncertainty constitutes an error-prone situation typical for risk management decisions that are influenced by both our phylogenetic and ontogenetic histories. The two learning processes have hugely different timescales and mechanisms of fitness for variation and selection. Comparing the two is like relating the elephant with the man sitting on the back of the elephant. They are both better off if they cooperate, in other words: how shall we minimize the negative effects of cognitive biases and how to utilize the positive effects of cognitions, i.e., thinking and verbal behavior, man’s most precious virtues? The evolution of language and thinking gave man a crystal ball enabling imagination of a future that also contain age-related sickness, decline of physical and mental abilities, and inevitable death. Without the optimism bias, the evolution of mankind might have stopped when humanoids reached the stage of language and abstract thinking. Overconfidence is essential for innovation as it induces variation for selection to work on. Some of it ends in budget overruns, delays, and products that never make it to the market. The crucial question is to keep the “good” and avoid the “bad” variation and selection. How to balance these must be situation specific as there is a large difference between risk management in aviation and risk management in the development of digital consumer products. Flight safety leaves little room for variation, while digital gadgets must get to the market first with the new innovative product. Then, it is ok to fail given that you now and then hit a blockbuster. The market does the selection; the employees and management must secure sufficient variation, biased or not.

Confirmation behavior usually serves us well as it stabilizes conduct and makes it easier for others to predict our behavior, which is beneficial for building social relations. It also boosts self-confidence, because we perceive ourselves as more consistent and coherent than we actually are. Repeated often enough, the confirmation behavior can be automated, making us unaware of it. And when our behavior for whatever reason becomes inconsistent, as viewed by other people or by ourselves, the dissonance leaves us with an unpleasant feeling we seek to escape. We usually succeed, due to a well-equipped escape and avoidance behavior repertoire, developed over many years of our upbringing.

If the confirmation or avoidance behavior is maintained on a partial reinforcement schedule, it can become very resistant to change. The thinner the schedule, the more resistant the behavior is likely to become; within certain limits. Beliefs or rules that are less correct are confirmed more seldom than more correct beliefs. This opens for a subconscious and callous effect; the more wrong you are, the stronger you believe you are correct. In the discourse of science, falsification has therefore been proposed as the preferred scientific method for verification rather than confirmation [55].

Cognitive biases are human legacies from our behavioral past that may strongly influence decision-making in risk management. The research on these items is only in its infancy. That should however not prevent us from considering what we can do today, although we should be modest in what it is possible to achieve. The resistance to change and stealth like character of cognitive biases make them almost impossible to avoid. It is however possible to reduce their effect to some degree by reinstating QRA as the basis for risk management, extended with a review of relevant cognitive biases. The not-so-surprising solution is to let only well-qualified people take management positions. This might constitute a challenge as there normally are many to choose from. The best qualified are not necessarily those with highest self-confidence. It is rather the opposite, as the Dunning-Kruger experiments indicate. The less skilled people tend to unknowingly exaggerate their abilities. They are probably not the people you would like to make crucial decisions regarding risk, but you are likely to find them overrepresented among wannabe managers.

Regardless of industry, effective risk management implies that regulations must be in place that require QRA of high quality, and if risk levels are not acceptable, the measures that are most cost-effective must be identified and implemented, until the risk is within acceptable limits. This is the essence of a risk management system. If any of the steps are missing or carried out without the proper knowledge, compensation to those who suffered from the accident can be enforced. Using lawyers to promote safety by making the responsible pay compensation for damage is an example of corrective action; it is reactive as it is initiated after the accident has occurred. Accident prevention is however a much better strategy than damage compensation. A proactive way to promote safety is to set a much higher value on life. This policy should be made external to the involved stakeholders, e.g., by an official, public institution like NHTSA for road safety. A still better solution is to raise the decision of value of life above the different industry branches, as there is no good reason why the value of life should depend on what kind of work you do.

Acknowledgments

The author thanks Gunnar Ree for his helpful comments.

Conflict of interest

The author declares that he has no conflict of interests.