Project and Enterprise Risk Management at the California Department of Transportation

1. Introduction

A better understanding of risk management processes and practices within a government agency is crucial for enhancing the project delivery process and for implementing formally risk management. This chapter outlines the whole implementation process carried out with the risk management team formed from different functional units and backgrounds. In addition, a discussion is held over the critical steps and aspects for performing project and enterprise risk management in the real world.

Risk management is not new for the transportation industry in the United States, specifically in highway projects. The California Department of Transportation (Caltrans), a leading authority in public transportation projects in the US, has used basic project management principles along its statewide Districts offices. Risk management has been part of the project management menu; nevertheless its application was limited only to developing a risk register and a qualitative analysis at the most. The Office of Statewide Project Management Improvement (OSPM), has developed a Project Risk Management handbook which is a guide for project managers at Caltrans for using risk management. Unfortunately, the latest version of the manual which is from 2007, did not included a detail explanation of the benefits for performing quantitative risk analysis while determining the risks impacts into the project objectives, in terms of cost and time. The term quantitative risk analysis is merely described, lacking a sound description of the tools and methodologies that have been in place and use in the industry for many years and even with other government agencies around the world. Cost overruns caused by a lack of using risk management in the practice for infrastructure and transportation projects, has been mentioned in the literature for many years. However, only few examples of how risk management can be use in the real life are available, including how can a risk team be formed and how to educate the team for performing a sound and trusted risk management exercise.

2. Risk management planning

As any other process in project management, risk management has to be planned in order to forecast the total effort required by the project team for developing the full scope of risk management. The California Department of Transportation (Caltrans), developed a Project Risk Management Handbook which is being used as a reference for planning the steps for applying risk management into specific projects. The purpose of the study, roles and responsibilities, the scope of the Risk Register, risk identification, analysis methods, implementation period, schedule and budget allocation need to be defined with the plan. Special attention should be placed into the human resource (method selection) aspect and in identifying the right phase of the project to initiate with the study. The roles of the Project Manager (PM) and the Risk Manager (RM) are critical for developing a realistic implementation plan (Figure 1). In addition, before starting working wit the RMT, the PM and RM should ensure that important project data is available. For example the project report, cost estimate, project plan, etc.

Figure 1.

Planning the risk management plan

It is ideal to have the project charter for developing the risk management plan, since in the charter it is possible to identify critical information about the project like scope, conceptual cost estimate, delivery milestones, conceptual risks, stakeholders, etc.

3. Risk management training and education

Caltrans’s Project Risk Management Handbook (California Department of Transportation (Caltrans, 2007) is the reference for performing training to the project risk management team. It covers the basics of risk management applied to transportation projects. Nevertheless, additional knowledge is provided to the team members for assessing properly the risks and opportunities during the qualitative and quantitative analysis. Although there are considerable resources for learning about risk management, Caltrans has adopted this process into its project development process (Figure 2). This approach has assisted for providing on the job training for the Project Development Team (PDT).

Figure 2.

Risk management process flow diagram (Caltrans, 2007)

It is important to notice, that the risk assessment is responsibility of the PM and the project team. Nevertheless, it is recommended to use whenever is possible a RM. The RM is a neutral element of the project team and can reduce the bias, which can seriously affect the outcome of the risk management study.

4. Risk manager role

The relevance that a risk manager plays during the implementation phase is crucial for the success of the study. The RM as a risk expert should be able to lead, coordinate, educate, explain, convince, propose, monitor and evaluate the entire process; plus he or she needs to be able to have experience in leading teams from different backgrounds and coming from different functional units and agencies. Vose (2008) enounces the following as the characteristics of the risk analysts: creative thinkers, confident, modest, thick-skinned, communicators, pragmatic, able to conceptualize, curious, good at mathematics, a feel for numbers, finishers, cynical, pedantic, careful, social and neutral. The reality is that we not always can find individuals that have all the virtues mentioned before, therefore; we need to select the most critical characteristics that a risk manager should have. Definitely a risk manager should be a good communicator, must have an analytical mind and needs to be able to think outside the box. The skills of a risk manager are somehow related to the project manager’s, in the sense of managing and controlling. However, the risk manager needs to deal with risk assessment that in the quantitative arena requires analytical modelling skills that the project manager is usually not trained for.

It is a demanding list and indicates; that risk analysis should be performed by people who have a proven track record of doing risk management for several projects ideally. It is also rather unlikely to find these skills in one person: the best risk analyst units with which we work are composed of a number of individuals with complementary skills and strengths Vose (2008).

The Washington State Department of Transportation ([WSDOT], 2010) in their Guidelines for CRA-CEVP Workshops, enlisted some of the risk manager responsibility: risk elicitation, risk modelling, cost validation/review, lead meetings, and provides reports and Develops or implements workshops on topics such as project definition, and risk identification and management. The California Department of Transportation (Caltrans, 2007) describes the risk manager (Risk Officer) responsibilities as risk management planning, identification, qualitative and quantitative analysis, risk response and risk monitoring and control. See Figure 3.

It is very important to take into consideration that the person selected or named to become the risk manager, should be someone that in addition of having the minimum capabilities, should be a neutral team element that can guide the risk team towards a non biased risk assessment of the project.

Although the position of RM in public agencies and private companies is rather new; currently the importance of having such an expert formally within the organization structure has become more formal and demanded. It has been realized that the benefits of having such an expert are much higher than the losses caused by not properly assessing and managing risks.

Figure 3.

Risk management key responsibilities (Caltrans, 2007)

5. Project stakeholders

Due to the nature of the transportation projects, several stakeholders are usually involved. In particular, there is a closed communication between other local, regional and federal agencies which need to be involved and provide assessments and feedback. Caltrans as a state agency, deal internally as well with several stakeholders coming from the different functional units or divisions.

For the above reasons, it is extremely important to develop project communication plans, so the right stakeholders can be identified, together with investigating the best ways of communication with them critical project information for decision making. Figure 4 shows a typical stakeholder analysis used at Caltrans, which is part of the project communication plan.

As can be seen, the stakeholder analysis aid in selecting those stakeholders that could play a relevant role in the project. For the PM, is important to know which is the prefer way for communicating with each stakeholder, since he needs to keep them informed at all times. The frequency is another factor which needs to be addressed carefully, some people like to be informed about every change or movement in the project and others, only want to receive updates related to considerable changes in scope, time or cost for example.

6. Subject matter experts

The Subject Matter Experts (SMEs) have two objectives along the risk management study; firstly they provide specialized knowledge about specific project risks and uncertainties and secondly; in lieu of having a set of data available, they can fill the gaps of insufficient data while defining the uncertainty of a project variable. In practice and with transportation projects, usually two or three variable values are asked to the SMEs. For other type of projects or industry this approach may be different for example if the project is very unique and complex, like an energy plant or even an oil platform.

Name	Function	Contact Information	Preferred Method of Communication	Goals on this project	Frequency
Project Sponsor	Progress/Status Report	email	email	Budget	Monthly
Project Manager	Progress/Status Meetings and Report. Information sharing and issue resolution	email	email	Time, cost, quality	Monthly
Project Manager Assistance	Progress/Status Meetings and Report. Information sharing and issue resolution	email	email	Time, cost, quality	As occurs

Table 1.

Stakeholder analysis (Caltrans, 2007)

If for example the goal is to work with two values for defining the uncertainty behaviour for a particular risk variable, then the probability distribution called Uniform Distribution is used, which assumes all the values between the minimum and the maximum values have the same probability of occurring. In this case, a worst and best case scenarios are asked from the SMEs for defining a minimum and a maximum value of the risk. The other common probability distribution function use in this field is the triangular distribution function. This function assumes that there are three different values assigned to a variable of risk; a minimum, a most likely and a maximum. It is difficult to define which function is better for obtaining the data from the SMEs, since it will depend of their experience, the type of project and the project data available at the moment of the study. In general, the uniform distribution is the most popular since it is rather easy to estimate only two values.

A key part for soliciting the information for the SMEs, are the questions asked by the Risk Manager or Risk Facilitator. In some cases, the SMEs are reluctant to participate and optional methods should be place on the table by the risk manager for getting the opinion needed from the SME. An alternative and perhaps a highly recommended approach is to have one on one interviews with the SMEs for asking the questions and getting the answers. However, the recommendation is to have this information during the meetings, so the rest of the team can discuss and in some cases even do they can challenge the SMEs opinion, which is a very productive and healthy action for the project. The downside of conducting these interviews is that they are rather time consuming.

In some cases, we can get different SMEs opinions for the same variable or risk as mentioned by Vose (2008). Experts will sometimes produce profoundly different probability distribution estimates of a parameter. This is usually because the experts have estimated different things, made differing assumptions or have different sets of information on which to base their opinion. However, occasionally two or more experts simply genuinely disagree.

Figure 4.

Combining three dissimilar expert opinions (Vose, 2008)

As can be observed, from the input contribution to the risk model from the SMEs is a combination distribution. This is not a common situation but there is always a chance for encountering these types of challenges. Especially for complex projects where a considerable expertise is required. Figure 4 illustrates an example of combining three differing opinions, but where expert A is given twice the emphasis of the owing to the greater experience of that expert (Vose, 2008).

It is relevant to notice that not all the SMEs are willing to participate actively in a risk management exercise at the first time, especially if they have no been exposed before into one. When SMEs are first asked to provide probabilistic estimates, they usually won’t be particularly good at it because it is a new way of thinking (Vose, 2008).

For overcoming this and in order to provide the SMEs some sort of support in the field of the probabilistic risk management, we held a sort of training and educational meeting with all the RMT and SMEs.

It can be noted from Figure 5 that the expert’s knowledge can extend beyond the knowledge base into the absolute truth area as a result of creativity and imagination of the expert. Therefore, the intersection of the expert’s knowledge with the ignorance space outside the knowledge base can be viewed as a measure of creativity and imagination. Another expert (i.e., Expert B) would have her/his own ellipses that might overlap with the ellipses of Expert A, and might overlap with other region by varying magnitudes (Ayyub, 2000).

Figure 5.

Knowledge and Ignorance of Humans (Ayyub, 2000),

SME`s are crucial for giving credible data for the risk model. If SMEs are not part of the risk management study, the results will not be trusted, causing this a failure of the process, the PM and the team.

7. Risk management team elicitation

The selection of members for the Risk Management Team (RMT), is not an easy task at all. A depth analysis of background and education was carried on for selecting the potential members of the team. The analysis, brainstorming, experience, background of the RMT members is critical not only for the risk management identification; it is for all the process including the monitoring and control. Eliciting the risk team members can be such a critical milestone for success or failure for the rest of the risk management process and the project itself. At the end, risk management is an input-output process that if wrong data or knowledge is feed into, then the results expected, most likely would be trustless or would included biases which at the end affect seriously the integrity and best practices in risk management.

Figure 6.

Overview of the Expert Elicitation Process (USEPA, 2011)

Expert Elicitation (EE) is a multidisciplinary process (Figure 6) that can inform decisions by characterizing uncertainty and filling data gaps where traditional scientific research is not feasible or data are not yet available. Although there are informal and nonprobabilistic EE methods for obtaining expert judgment. The goal of an EE is to characterize, to the degree possible, each expert’s beliefs (typically expressed as probabilities) about relationships, quantities, events, or parameters of interest. The EE process uses expert knowledge, synthesized with experiences and judgments, to produce probabilities about their confidence in that knowledge. Experts derive judgments from the available body of evidence, including a wide range of data and information ranging from direct empirical evidence to theoretical insights. Even if direct empirical data were available on the item of interest, such measurements would not necessarily capture the full range of uncertainty. EE allows experts to use their scientific judgment transparently to interpret available empirical data and theory (Ayyub, 2011).

As mentioned by Vose (2008), it is usually impossible to obtain data from which to determine the uncertainty of all the variables within the model accurately, for a number of reasons:

• The data have simply never been collected in the past

• The data are too expensive to obtain

• Past data are no longer relevant (new technology, changes in political or commercial environment, etc)

• The data are sparse, requiring expert opinion “to fill in the holes”

• The area being modelled is new

The uncertainty in subjective estimates has two components: the inherent randomness of the variable itself and the uncertainty arising from the expert’s lack of knowledge of the parameters that describe that variability.

In cases where there are no subject matter experts available, it is recommended to even hire external experts for obtaining the uncertainty ranges per variable or risk. Caltrans’s practices towards project management and the project delivery process use a Project Development Team (PDT) which changes alongside the phases of the project. These teams are rather big in size and usually integrate stakeholders, subject matter experts and individuals coming from the same or different functional units (right of way, environmental, design, construction, etc.). Firstly, the downside of using such a team like the PDT is the size since it could be complicated for the RM to facilitate the meetings for obtaining better results towards assessing the uncertainty. In addition, the team members should feel free to talk about risk and in some cases having more than one member coming from the same division of functional unit, can cause some limitations for discussion and brainstorming.

In general, risk teams are rather small in compare with the traditional project teams. As a general rule, to invite a representative from each department or division is recommended. This approach definitely will help the whole process and will assist the RM to maintain the team focused in talking only about risk.

8. Risk identification, analysis and response

The whole process of risk management was implemented for three different projects. Regardless of the project scope, cost, schedule, location, type of funding, etc; the same steps were followed in order to determine the overall risk cost 8cost contingency). Figure 7 shows the risks identified for each project; it can be observed the source of risk, which in Caltrans relates to the functional units (e.g. environmental, right of way, design, project management, construction etc.)

Figure 7.

Risk identification

Project A was a Historical Bridge, Project B a Direct Access Ramp (DAR) with a Transit Station (TS) and Project C a standard bridge. The main goal of the risk analysis for each project was to determine the cost risk contingency associated to each project risk register. Figure 10 shows the critical risks obtained for each project once the qualitative analysis was performed. The qualitative assessment matrix (Figure 9) used for ranking the risk criticality, forms part of Caltrans standard process for risk management.

The risk sources indicated in Figure 7 are: design, right of way (rw), environmental (env), program project management (ppm), construction (const), exterior (ext), engineering (eng) and organizational (org).

Figure 8.

Risk matrix (Caltrans, 2007)

The RM was responsible for educating the RMT along each phase of the risk management implementation. For the qualitative risk analysis in particular, it was useful to provide tangle examples to the team before and during the meeting.

As can be observed, the critical risks are only a few ones in compare with all the risks identified at the initiation phase. The purpose of the qualitative risk analysis was to select those risks that represent a major negative or positive impact into the project objectives. A critical part for performing the qualitative analysis was to define the probability and impact ranges. The impacts were defined in terms of cost only for these projects. In addition, construction, engineering and organizational risks were not part of the quantitative risk analysis, since there were not scored as critical in the qualitative risk analysis.

Figure 9.

Qualitative risk analysis

The output of the qualitative risk analysis formed the basis or inputs for conducting the quantitative risk analysis. A set of cost ranges where developed for each project for matching the impact value selected. A model was built for running the Monte Carlo Simulation technique for obtaining the risk cost contingency for each project. Figure 10 shows the contingency values obtained per project. It is important to notice that instead of assigning a randomly selected contingency percent, these values are based directly on the critical risks identified with the Risk Registers.

Figure 10.

Quantitative risk analysis

The cost contingency per project assessed with the quantitative analysis provides a substantial support for justifying the overall project uncertainty. It was a common practice at Caltrans to assign a merely flat rate for risk contingency, without referring to specific project risks or by justifying the percentage upon a formal risk management study.

For each project, a set of risk responses were developed by the RMT and placed with the Risk Register. A Risk Owner was named for implementing the responses. It is recommended to match the risk owner with the source of the risk. In other words, if the risk has a source in construction, then the best suitable risk owner should come from that division.

The whole risk management implementation, including the risk identification, analysis and response was conducted in three meetings with the RMT. This is a standard set at Caltrans. The time and effort invested from the RM is not included since most of this work is done outside the meetings.

9. Risk management meetings

It was not common at all at Caltrans to have risk management meetings for the PDT members and even for executives. What is a reality is that those meetings are critical for performing in depth thoughts about what can go wrong with the project. The most critical of the meetings is the planning meeting; which usually includes educating the RMT. Questions like: why a risk management exercise is needed? What could be the possible project risk contingency? What are the restrictions? What is the overall project uncertainty rate? Need answers for properly doing a risk management study.

The following aspects are quite relevant in order to keep an order and sense within the risk management planning meeting (Vose, 2008):

• Rank the questions that need answering from “critical” down to “interesting”

• Discuss with the risk analysis the form of the answer

• Explain what arguments will be based on these outputs

• Explain whether the risk analysis has to sit within a framework

• Explain the target audience

• Discuss any possible hostile reactions

• Figure out a timeline

• Figure out the priority level

• Decide on how regularly the decision-maker and risk analysis will meet

It was interesting to notice along the meetings that when the meetings were held, the invitees were already prepared just to talk about risk. In some cases, critical project issues were discovered thanks to the risk discussions.

The risk management meetings were properly planned, one for the identification, analysis and results. No meeting took more than two hours and instead of meeting minutes, the Risk Register was used as the deliverable for discussion and follow up.

10. Risk monitoring and control

Risk monitoring and control has been mentioned as one of the most common failures of risk management. In part because the follow up process is usually forgotten by the project and risk manager and as results, there is no comparison between the baseline and actual Risk Registers or risk results. Risk metrics are fundamental for determining and assessing how risk management is contributing or enhancing the project delivery process. If the benefits of risk management are tangible and can be promoted with management and executives, the chances for formalizing the risk management process are very high. Otherwise, there will be less interest and support for keeping such a program.

Figure 11 shows the risk contingency behaviour for a given project. As can be notice, the contingency is dynamic thought the project delivery process. For example, in can be assumed that the baseline risk contingency was the one marked in grey colour. Then, after the risks responses implementation, risks are mitigated and the contingency is reduced (red colour). Nevertheless, the monitoring and control process continues and a new risk arises, increasing again the project contingency (blue colour).

Figure 11.

Risk contingency behaviour

Caltrans has implemented risk monitoring through the project development team meetings, where the project manager and the risk manager address any changes within the current risks or document new risks. In addition, an alternative way for risk monitoring has been through email. In this case, the risk manager is in charge of contacting the risk owners for updates and feedback regarding their risks.

Risk status reports have been developed with the purpose of maintaining informed the executive though a proper risk management communication. These reports are developed by the RM and are updated every time there is a change with the Risk Register.

Risk monitoring and control must be maintain through the project life cycle until the closeout phase, were the lessons learned can include feedback from risk management. The PM and RM should put extra care in keeping a constant review of the baseline Risk Register with the purpose of actively managing the project contingency and for assessing the effectiveness of the risk responses.

11. Risk management system

A risk management system could be considered the ultimate tool for managing risks for a portfolio of projects. In might be a powerful instrument for communication, monitoring and control. In addition, the basis for a future risk management database could be established.

In practical terms, for having a risk management system it is necessary to have developed a sufficient number of risk management studies and to have performed formal training to project managers. By having a set of projects in which risk management was previously implemented, the system can be fulfilled with data, including lessons learned. This data can provide an additional support for developing new risk management plans. A system can represent a considerable advantage for executives and project managers since can provide risk management status reports for a set of projects for specific data dates. The project manager can view, edit and direct actions to the team while using the system. Alternately, the RM can use the system for the risk management implementation. The following figure shows a view of Caltrans risk management system.

Figure 12.

Caltrans Risk Management System

The RMT can view all the information for their project(s). In addition, some members for example the project manager, the design manager or the risk manager can have rights for edition. Usually with these systems, electronic alerts or messages are sent via email to keep all the team informed. As well, reports can be generated for supporting the decision making process.

Although a risk management system is a great tool for supporting the decision making along the project delivery process. It is recommended first to start risk management with education, a pilot project and training. After several studies, the role of a system can be justified for enhancing the overall process of risk management and its communication.

12. Caltrans enterprise risk management

Although the term of enterprise risk management (ERM) is not new, Caltrans started looking into its current process for doing business. The nature of risk management is intimately related to Caltrans functional units. Program Project Management, Construction, Environmental, Design, Right of Way and Surveys are the most representative functional divisions existing at Caltrans. Each division plays a role with the project delivery process; usually representatives of each division formed the project development team (PDT) which has the responsibility of carried on the project since the initial phase until the closeout.

Caltrans has four major risks. These are project risk, program risk, operations risk, and organizational risk. Taken together, management of these components constitutes a Department-wide Risk Management Strategy. In some organizations, this is known as “Enterprise Risk Management.” Each of these risk components relate directly to the different types of the work performed by the Department’s functional areas. For example, project risk is related to the project delivery process; those things that, if they occurred, could impact a project’s schedule, cost, or scope. Project Delivery staff can assess and take intelligent risks in delivery because taking intelligent risks fosters innovation and responsible decision-making. But, at the same time, Caltrans needs to follow its project-related processes and controls to manage that risk. Caltrans must monitor the quality of our performance over time, and evaluate any deficiencies and adjust our processes accordingly. Actively incorporating the concept of Risk Management into Caltrans’s programs and activities is a dynamic process and must be constantly evaluated and adjusted to meet the strategic goals of the Department.

According to the Minesota Department of Transportation (MnDOT, 2012), ERM is a risk-based approach to managing an enterprise, incorporating concepts of internal control, planning and budgeting. It addresses the needs of various stakeholders who want to understand the broad spectrum of risks facing complex organizations so they can be appropriately managed.

Figure 13.

Enterprise Risk Management System (MnDOT, 2012)

The ERM at the Minesota Department of Transportation (Figure 13) is a clear example of how risk management can evolve for an organization. It is important to notice that outside of the common risk sources (corporate, programmatic, project and operations); other variables such the quality of life indicators, market research and performance measures are included. These variables have a direct impact into the risk sources, which at the end could influence the results and benefits of risk management.

Risk management has to be implemented for projects or within projects, but this is only the first step. Risk management means a change of doing business. For that reason, the culture of implementing Risk management should be brought by the executives and the company’s policies. Risk management has evolved into the “Enterprise Risk Management” (see Figure 14).

Figure 14.

Risk Management evolution (Protiviti, 2006)

This new methodology explains that there are different philosophies about risk management with their own methods and focus; these are normally classified in four different types.

• the Risk Silo Management (Operational Risks),

• the Integrated Risk Management (Economic and Capital Risks),

• the Risk and Value Management (Management and Performance) and

• the Strategic Risk Management (Senior Management and Strategical Risks),

ERM is in charge to unify all this philosophies and confer the determination of one whole risk in a corporation. One of the most important topics handled by the ERM is the determination of the Risk Appetite. The Risk Appetite is: “the quantum of risk that the firm is willing to accept within its overall capacity” (Barfield, 2008).

In order to propitiate the ERM in every company, the standard “ISO 31000:2009 Risk management - Principles and guidelines” is a comprehensive guide of how companies should implement a formal risk management process. It sets out principles, a framework and a process for the management of risks that are applicable to any type of organization in public or private sector. It does not mandate a "one size fits all" approach, but rather emphasizes the fact that the management of risk must be tailored to the specific needs and structure of the particular organization. It depends at the end of the company’s desire to be competitive and willing to manage proactively risks and opportunities.

Caltrans` goal towards ERM is related directly to its current position as a leader in the United States in the field of transportation projects. Caltrans expects to implement ERM in a near future with the intention of reinforcing its project and business processes.

13. Conclusions

Caltrans has evolved considerably in the past five years in the field of project risk management. The most common project risk management techniques used in the private sector are currently part of Caltrans project delivery process. The implementation of project risk management has assisted Caltrans executives and project managers in assessing properly the project contingency cost based upon specific identified risks. Enterprise Risk Management is still new in Caltrans. Nevertheless, management is taking currently formal steps in implementing it through all the state of California with the intention of managing and controlling not only project risks. The goal is to standardise the best practices in risk management from an enterprise perspective.