Streamlining project delivery through risk analysis [Aug. 2015]

GEORGIA DOT RESEARCH PROJECT 13-05
FINAL REPORT
STREAMLINING PROJECT DELIVERY THROUGH RISK ANALYSIS
OFFICE OF RESEARCH

GDOT Research Project No. 13-05
Final Report
STREAMLINING PROJECT DELIVERY THROUGH RISK ANALYSIS
Prepared By: Baabak Ashuri, Ph.D., DBIA, CCP, DRMP
Gordon Kingsley, Ph.D. Juan Rogers, Ph.D.
Mostafa Reisi Gahrooei Mohammad Ilbeigi Elie Ji-Yun Sung
Shahaboddin (Sean) H. Toroghi
Georgia Institute of Technology
Contract with Georgia Tech Research Corporation
August 2015
The contents of this report reflect the views of the author(s) who is (are) responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the Georgia Department of Transportation or the Federal Highway Administration. This report does not constitute a standard, specification, or regulation.
1

1. Report No.:

2. Government Accession No.: 3. Recipient's Catalog No.:

FHWA-GA-15-1305

5. Report Date:

4. Title and Subtitle:

August 2015

Streamlining Project Delivery through Risk Analysis

6. Performing Organization Code:

7. Author(s): Baabak Ashuri, Ph. D., DBIA, CCP, DRMP Gordon Kingsley, Ph.D. Juan Rogers, PhD. Mostafa Reisi Gahrooei Mohammad Ilbeigi Elie Ji-Yun Sung Shahaboddin-Sean H. Toroghi

8. Performing Organ. Report No.:

9. Performing Organization Name and Address: Economics of Sustainable Built Environment (ESBE) Lab Georgia Institute of Technology 280 Ferst Drive, Atlanta, GA 30332-0680
12. Sponsoring Agency Name and Address: Georgia Department of Transportation, Office of Research 15 Kennedy Drive, Forest Park, Georgia 30297-2599

10. Work Unit No.:
11. Contract or Grant No.: 0011829
13. Type of Report and Period Covered: Final; April 25, 2013 - August 31, 2015 14. Sponsoring Agency Code:

15. Supplementary Notes:

16. Abstract: Project delivery is a significant area of concern and is subject to several risks throughout Plan Development Process (PDP). These risks are attributed to major areas of project development, such as environmental

analysis, right-of-way (ROW) acquisition, utilities coordination, third-party agreements, etc. The major

challenge for GDOT is that the risks can negatively impact the project outcomes as cost overrun and schedule

delay. However, if these risks could be identified early in the concept and scope development processes, their

respective negative impacts on project outcomes could be reduced. The research objective of this study is to

develop a comprehensive guidebook that advances the adoption of risk analysis tools in GDOT, in order to

expedite project delivery. To achieve the research objective, various project risk management processes

developed by different organizations were reviewed. Furthermore, current practice of risk management in

different state DOTs were studied. Several state DOTs were surveyed and interviewed regarding their risk

management programs. The results indicate that typically state DOTs determine the level and methods of risk

management based on project size and complexity of the project. The level of risk management may vary from

a simple risk register to a complex quantitative analysis. Then, a semi-structured interview was conducted with

nine subject matter experts at GDOT. The key factors that influence risk management practices within GDOT

were explored. The results were analyzed and a model explaining the current risk management practice and future needs was developed. A comprehensive list of potential risks for transportation projects was developed

based on reviewing the academic/professional literature, current state of practice in risk management among

leading state DOTs, and current state of practice of GDOT. During several meetings with higher level risk

management experts at GDOT, the most important risks were indented and a short list of major potential risks

was developed for each office at GDOT. Finally, a software tool specifically designed for identification and

qualitative assessment of highway project risks during the pre-construction phase of the project was developed

based on the shortlisted risk factors. The software program is equipped with the modification capability of

adding new risk items and/or removing some of the predetermined risk factors from the assessment.

17. Key Words: Project Risk Management, Project Delivery, Transportation

18. Distribution Statement:

19.Security Classification (of this report): Unclassified

20. Security Classification (of this page): Unclassified

21. Number of Pages: 200

22. Price:

2

TABLE OF CONTENTS
EXECUTIVE SUMMARY ............................................................................................................ 9 CHAPTER 1 INTRODUCTION .................................................................................................. 11 CHAPTER 2 LITERATURE REVIEW ....................................................................................... 15
2.1. WHAT IS RISK? ............................................................................................................... 15 2.2. WHAT IS PROJECT RISK MANAGEMENT?................................................................ 15 2.3. WHY RISK MANAGEMENT?......................................................................................... 17 2.4. RISK MANAGEMENT PROCESS .................................................................................. 18 2.5. REVIEW OF EXISTING RISK IDENTIFICATION METHODS ................................... 28
2.5.1. Assumption Analysis ................................................................................................... 28 2.5.2. Reviewing Previous Projects' Documents .................................................................. 28 2.5.3. Eliciting Experts' Opinions ......................................................................................... 29 2.5.4. Diagraming Techniques............................................................................................... 31 2.6. REVIEW OF EXISTING RISK ASSESSMENT METHODS.......................................... 32 2.6.1. Qualitative Risk Assessment ....................................................................................... 32 2.6.2. Quantitative Risk Assessment ..................................................................................... 34 2.6.3. Identification of Barriers for Risk Analysis in Organizations ..................................... 37 2.7. REVIEW OF EXISTING RISK RESPONSE METHODS ............................................... 39 2.8. SUMMARY ....................................................................................................................... 42 CHAPTER 3 REVIEW OF THE CURRENT PRACTICE OF STATE DOTS IN PROJECT RISK MANAGEMENT................................................................................................................ 43 3.1. INTRODUCTION.............................................................................................................. 43 3.2. WASHINGTON STATE DEPARTMENT OF TRANSPORTATION............................. 45 3.2.1. Risk Management Organization .................................................................................. 45 3.2.2. Risk Management Process ........................................................................................... 46 3.2.3. Lessons Learned and Challenges................................................................................. 50 3.3. CALIFORNIA DEPARTMENT OF TRANSPORTATION ............................................. 53 3.3.1. Risk Management Organization .................................................................................. 53 3.3.2. Risk Management Process ........................................................................................... 54 3.3.3. Lessons Learned and Challenges................................................................................. 61 3.4. UTAH DEPARTMENT OF TRANSPORTATION .......................................................... 63
3

3.4.1. Risk Management Organization .................................................................................. 63 3.4.2. Risk Management Process ........................................................................................... 66 3.4.3. Lessons Learned and Challenges................................................................................. 70 3.5. NEW YORK STATE DEPARTMENT OF TRANSPORTATION .................................. 72 3.5.1. Risk Management Organization .................................................................................. 72 3.5.2. Risk Management Process ........................................................................................... 73 3.5.3. Lessons Learned and Challenges................................................................................. 77 3.6. MINNESOTA DEPARTMENT OF TRANSPORTATION ............................................. 79 3.6.1. Risk Management Organization .................................................................................. 79 3.6.2. Risk Management Process ........................................................................................... 80 3.6.3. Lessons Learned and Challenges................................................................................. 86 3.7. LOUISIANA DEPARTMENT OF TRANSPORTATION ............................................... 90 3.7.1. Risk Management Organization .................................................................................. 90 3.7.2. Risk Management Process ........................................................................................... 91 3.7.3. Lessons Learned and Challenges................................................................................. 96 3.8. MISSOURI DEPARTMENT OF TRANSPORTATION .................................................. 99 3.8.1. Risk Management Organization .................................................................................. 99 3.8.2. Risk Management Process ......................................................................................... 100 3.8.3. Lessons Learned and Challenges............................................................................... 103 3.9. MONTANA DEPARTMENT OF TRANSPORTATION............................................... 104 3.9.1. Risk Management Organization ................................................................................ 104 3.9.2. Risk Management Process ......................................................................................... 106 3.9.3. Lessons Learned and Challenges............................................................................... 108 3.10. MICHIGAN DEPARTMENT OF TRANSPORTATION............................................. 110 3.10.1. Risk Management Organization .............................................................................. 110 3.10.2. Risk Management Process....................................................................................... 110 3.10.3. Lessons Learned and Challenges............................................................................. 111 3.11. MASSACHUSETTS DEPARTMENT OF TRANSPORTATION ............................... 113 3.11.1. Risk Management Organization .............................................................................. 113 3.11.2. Risk Management Process....................................................................................... 113 3.11.3. Lessons Learned and Challenges............................................................................. 114
4

3.12. RHODE ISLAND DEPARTMENT OF TRANSPORTATION.................................... 116 3.12.1. Risk Management Organization .............................................................................. 116 3.12.2. Risk Management Process....................................................................................... 116 3.12.3. Lessons Learned and Challenges............................................................................. 117
CHAPTER 4 ALIGNING PROGRAM-LEVEL AND PROJECT-LEVEL OBJECTIVES FOR RISK MANAGEMENT: CASE STUDY OF GEORGIA DEPARTMENT OF TRANSPORTATION................................................................................................................. 119
4.1. INTRODUCTION............................................................................................................ 119 4.2. MODEL AND CASE DESIGN ....................................................................................... 125
4.2.1. Identification of Factors and Design of the Protocol................................................. 125 4.2.2. Interviews .................................................................................................................. 126 4.2.3. Preliminary Analysis and Model ............................................................................... 127 4.2.4. Coding Procedure to Test the Model ......................................................................... 130 4.3. FINDINGS ....................................................................................................................... 131 4.3.1. Categories of the Model ............................................................................................ 131 4.3.2. Relationship in the Model.......................................................................................... 140 Relationship 1: Existing project development process-Main risks ..................................... 142 Relationship 2: Existing project development process -Typical projects or problems ....... 142 Relationship 3: Internal environment-Main risks ................................................................ 142 Relationship 4: Internal environment-Change factors......................................................... 143 Relationship 5: Internal environment-Benefits and costs.................................................... 144 Relationship 6: Internal environment-New risk assessment................................................ 144 Relationship 7: External environment-Typical projects or problems.................................. 147 Relationship 8: External environment-Change factors........................................................ 147 Relationship 9: External environment-New risk assessment .............................................. 149 Relationship 10: Main risks- New risk assessment ............................................................. 150 Relationship 11: Typical projects or problems-New risk assessment ................................. 151 4.4. CONCLUSION AND DISCUSSION .............................................................................. 153 CHAPTER 5 COMPREHENSIVE RISK ASSESSMENT FOR TRANSPORTATION (CRAFT) SOFTWARE MANUAL......................................................................................... 157 5.1. INTRODUCTION............................................................................................................ 157 5.2. RISK IDENTIFICATION PROCESS ............................................................................. 159
5

5.2.1. Review the Project Development Process ................................................................. 159 5.2.2. Interview with the GDOT Subject Matter Experts .................................................... 160 5.2.3. Literature Review ...................................................................................................... 160 5.3. SOFTWARE DEVELOPMENT...................................................................................... 164 5.4. STEPS TO USE CRAFT .............................................................................................. 166 5.4.1. Step 1- Run the Tool.................................................................................................. 166 5.4.2. Step 2- Identify Risk and Assess its Impact .............................................................. 168 5.4.3. Step 3- Create the Risk Register and the Risk Heat Map for Each Office ................ 170 5.4.4. Step 4- Create the Risk Breakdown Structure for the Project & Generate the Comprehensive Summary of All Identified and Assessed Risk Factors ............................. 172 5.4.5. Step 5- Save the Risk Analysis Results ..................................................................... 175 5.4.6. Step 6- Open the Risk Analysis File.......................................................................... 175 CHAPTER 6 CONCLUSIONS .................................................................................................. 178 APPENDIX A: SURVEY QUESTIONNAIRE.......................................................................... 181 REFERENCES ........................................................................................................................... 193
6

LIST OF FIGURES
FIGURE 2-1: RISK MANAGEMENT PROCESS DEFINED BY WORLD ROAD ASSOCIATION (WORLD ROAD ASSOCIATION, 2012) ...................................................................................................................................................................19
FIGURE 2-2: RISK MANAGEMENT PROCESS DEFINED BY APM (DIXON, 2006) ............................................................21 FIGURE 2-3: PROJECT RISK MANAGEMENT PROCESS DEFINED BY NSW GOVERNMENT (NSW 2011).........................22 FIGURE 2-4: RISK MANAGEMENT PROCESS DEVELOPED BY CANADIAN STANDARD ASSOCIATION (2002) ..................25 FIGURE 2-5: CYCLICAL NATURE OF RISK MANAGEMENT ADOPTED FROM FHWA (2006)...........................................27 FIGURE 2-6: RISK MANAGEMENT PROCESS INTRODUCED BY SHRP2 REPORT (MOLENAAR ET AL., 2014) ..................27 FIGURE 2-7: INPUTS, TOOLS, AND OUTPUTS FOR QUALITATIVE PROJECT RISK ASSESSMENT, ADOPTED FROM PROJECT
MANAGEMENT INSTITUTE (2013) ........................................................................................................................33 FIGURE 2-8: QUALITATIVE RISK ASSESSMENT ADOPTED FROM MOLENAAR ET. AL (2010) .........................................33 FIGURE 2-9: PROBABILITY DENSITY OF A RISK EVENT AS AN INPUT FOR MONTE CARLO SIMULATION, ADOPTED FROM
MCGOEY-SMITH ET AL. (2007) ...........................................................................................................................36 FIGURE 2-10: PROJECT COST DISTRIBUTION OBTAINED FROM THE MONTE CARLO SIMULATION, ADOPTED FROM
MCGOEY-SMITH ET AL. (2007) ...........................................................................................................................36 FIGURE 3-1: RISK MANAGEMENT TOOL SCREENSHOT .................................................................................................52 FIGURE 3-2: CALTRANS RISK MATRIX .........................................................................................................................58 FIGURE 3-3: RISK COST PROBABILITY DISTRIBUTION ..................................................................................................60 FIGURE 3-4: DECISION TREE FOR IDENTIFYING APPROPRIATE LEVEL OF RISK ANALYSIS............................................66 FIGURE 3-5: UTAH QUALITATIVE RISK ANALYSIS TOOL..............................................................................................69 FIGURE 3-6: RISK IDENTIFICATION TOOL AT NEW YORK STATE DOT .........................................................................75 FIGURE 3-7: RISK ANALYSIS TOOL AT NEW YORK STATE DOT ..................................................................................76 FIGURE 3-8: NON-LINEAR IMPACT SCORING ................................................................................................................94 FIGURE 3-9: LINEAR IMPACT SCORING.........................................................................................................................94 FIGURE 3-10: RISK ASSESSMENT SPREADSHEET USED BY MISSOURI DOT ................................................................102 FIGURE 3-11: RISK MANAGEMENT TOOL SCREENSHOT .............................................................................................109 FIGURE 4-1: MODEL OF THE ADOPTION OF RISK MANAGEMENT PRACTICES IN GDOT..............................................128 FIGURE 4-2: MODEL REVISED IN LIGHT OF THE EVIDENCE ........................................................................................154 FIGURE 5-1: MAIN MENU OF THE CRAFT SOFTWARE ............................................................................................166 FIGURE 5-2: COPYRIGHT AND CONTACT INFORMATION OF THE CRAFT SOFTWARE ..............................................167 FIGURE 5-3: BRIEF INSTRUCTION IN THE HELP MENU ................................................................................................167 FIGURE 5-4: IDENTIFYING LIKELIHOODS AND IMPACTS OF RISK FACTORS.................................................................170 FIGURE 5-5: RISK HEAT MAP FOR THE OFFICE OF RIGHT-OF-WAY.............................................................................172 FIGURE 5-6: RISK BREAKDOWN STRUCTURE (RBS) OF THE PROJECT ........................................................................174 FIGURE 5-7: SAVING THE RISK ANALYSIS RESULTS FOR THE PROJECT ......................................................................175 FIGURE 5-8: OPENING THE FILE FOR RISK ANALYSIS RESULTS OF THE PROJECT ........................................................176 FIGURE 5-9: CRAFT SUB-PROGRAM FOR ROW RISK ANALYSIS ...........................................................................177
7

LIST OF TABLES
TABLE 2-1: DEFINITION OF (PROJECT) RISK .................................................................................................................15 TABLE 2-2: GENERIC 9-PHASE PROJECT RISK MANAGEMENT PROCESS (RMP) IDENTIFIED BY CHAPMAN (1997) .......18 TABLE 2-3: ROAD MAP OF PROJECT RISK MANAGEMENT DEVELOPED BY KAHKONEN (1997) ....................................20 TABLE 2-4: RISK IDENTIFICATION METHODS (AYYUB, 2001; PROJECT MANAGEMENT INSTITUTE, 2013; CRETU ET AL.,
2011) ...................................................................................................................................................................31 TABLE 2-5: QUANTITATIVE RISK ANALYSIS METHODS................................................................................................37 TABLE 2-6: SUMMARY OF STUDIES ABOUT RISK ASSESSMENT AND MANAGEMENT BARRIERS ADOPTED FROM
CHILESHE ET AL. (2013) ......................................................................................................................................39 TABLE 2-7: EXAMPLE OF RISK RESPONSE ADOPTED FROM CRETU ET AL. (2011) ........................................................41 TABLE 2-8: EXAMPLES OF RISK RESPONSE ADOPTED FROM CALTRANS (2012) ...........................................................42 TABLE 3-1: THRESHOLDS FOR CONDUCTING RISK ANALYSIS FROM WSDOT (2014) ..................................................46 TABLE 3-2: SAMPLE RISKS FOR ENVIRONMENTAL ISSUES............................................................................................57 TABLE 3-3: DEFINITIONS OF IMPACT AND PROBABILITY RATINGS FROM CALTRANS (2012)........................................58 TABLE 3-4: EXAMPLE OF THE RISK REGISTER FOR ROW RISKS FROM NEW YORK STATE DOT..................................74 TABLE 3-5: SAMPLE LIST OF RIGHT OF WAY RISKS .....................................................................................................92 TABLE 3-6: RISK PROBABILITY RANKING THRESHOLDS ..............................................................................................93 TABLE 3-7: THRESHOLDS FOR CONDUCTING RISK ANALYSIS (MDT, 2014) ..............................................................105 TABLE 4-1: CODING THEMES AND CORRESPONDING CATEGORIES OF THE MODEL ....................................................131 TABLE 4-2: CROSS TABULATION OF THE NUMBER OF REFERENCES COMMON TO CATEGORIES .................................141 TABLE 4-3: SUMMARY OF THE COMPARISON BETWEEN ROLE OF INTERVIEWEES AND INTERVIEW CONTENT LEVEL OF
CONSIDERATION ................................................................................................................................................156 TABLE 5-1: MAJOR RISKS FOR THE OFFICE OF BRIDGE DESIGN .................................................................................161 TABLE 5-2: MAJOR RISKS FOR THE OFFICE OF PROJECT MANAGEMENT ....................................................................162 TABLE 5-3: MAJOR RISKS FOR THE OFFICE OF CONSTRUCTION..................................................................................162 TABLE 5-4: MAJOR RISKS FOR THE OFFICE OF ROADWAY DESIGN.............................................................................162 TABLE 5-5: MAJOR RISKS FOR THE OFFICE OF DESIGN POLICY AND SUPPORT...........................................................162 TABLE 5-6: MAJOR RISKS FOR THE OFFICE OF RIGHT OF WAY (ROW)......................................................................162 TABLE 5-7: MAJOR RISKS FOR THE DISTRICTS ...........................................................................................................163 TABLE 5-8: MAJOR RISKS FOR THE OFFICE OF TRAFFIC OPERATIONS ........................................................................163 TABLE 5-9: MAJOR RISKS FOR THE OFFICE OF ENVIRONMENTAL SERVICES ..............................................................163 TABLE 5-10: MAJOR RISKS FOR THE OFFICE OF UTILITIES .........................................................................................163 TABLE 5-11: MAJOR RISKS FOR THE OFFICE OF MATERIALS AND TESTING................................................................163
8

EXECUTIVE SUMMARY
Development of highway projects takes substantial amount of time and resources from the Georgia Department of Transportation (GDOT). Project delivery is a significant area of concern and is subject to several risks throughout Project Development Process (PDP). These risks are attributed to major areas of project development, such as environmental analysis, right-of-way (ROW) acquisition, utilities coordination, third-party agreements, etc. The major challenge for GDOT is that the risks can negatively impact the project outcomes as cost overrun and schedule delay. However, if these risks could be identified early in the concept and scope development processes, their respective negative impacts on project outcomes could be reduced. GDOT needs to enhance its understanding regarding source and natures of these risks early in concept and scope development phases.
The research objective of this study is to develop a comprehensive guidebook that advances the adoption of risk analysis tools in GDOT, in order to expedite project delivery. To achieve the research objective, various project risk management processes developed by different organizations were reviewed. Furthermore, current practice of risk management in different state DOTs was studied. Several state DOTs were surveyed regarding their risk management programs. Some of them have a standard process and guidebook to implement risk management. However, some of the other surveyed state DOTs rely mostly on their project managers' experiences for a successful risk management. After analyzing the results of the survey, a semistructured interview was conducted with subject matter experts to achieve more detailed information about the current practice of state DOTs for risk management. The results indicate that typically state DOTs determine the level and methods of risk management based on project
9

size (i.e. dollar value) and complexity of the project. The level of risk management may vary from a simple risk register to a complex quantitative analysis. Moreover, several factors such as lack of training of personnel, lack of sufficient internal infrastructure such as database, lack of existing policies, and lack of risk culture are among the most important challenges and barriers to implement a successful risk management program. After reviewing the literature and current risk management practices by state DOTs, a semi-structured interview was conducted with nine subject matter experts at GDOT. The interviewees were responsible for project management, program level roles, and specialists from different offices. The key factors that influence risk management practices within GDOT were explored. Then, the results were analyzed and a model explaining the current risk management practice and future needs was developed. A comprehensive list of potential risks for transportation projects was developed based on reviewing the academic/professional literature on risk analysis, current state of practice in risk management among leading state DOTs, and current state of practice of GDOT. The identified risks were categorized based on the responsible offices at GDOT. During several meetings with higher level risk management experts at GDOT, the most important risks were identified and a short list of major potential risks was developed for each office at GDOT. Finally, a software tool specifically designed for identification and qualitative assessment of highway project risks during the pre-construction phase of the project was developed based on the shortlisted risk factors. The software program is equipped with the modification capability of adding new risk items and/or removing some of the predetermined risk factors from the assessment.
10

CHAPTER 1 INTRODUCTION
Development of highway projects takes substantial amount of time and resources from the Georgia Department of Transportation (GDOT). Project delivery is a significant area of concern and is subject to several risks throughout Plan Development Process (PDP). These risks are attributed to major areas of project development, such as environmental analysis, right-of-way (ROW) acquisition, utilities coordination, third-party agreements, etc. Inadequate project scope, insufficient information on the extent of utility relocation requirements, insufficient knowledge of right-of-way costs and locations, required environmental mitigation costs to avoid certain impacts, traffic control requirements, and work-hour restrictions have been identified as major issues that inhibit streamlining project delivery during concept and scope development phases (Anderson et. al 2007). The major challenge for GDOT is that the risks can negatively impact the project outcomes as cost overrun and schedule delay. However, if these risks could be identified early in the concept and scope development processes, their respective negative impacts on project outcomes could be reduced. GDOT needs to enhance its understanding regarding source and natures of these risks early in concept and scope development phases. Enhanced understanding of risks is critical for identifying risks at their sources and assessing their impacts on project outcomes. Key bottlenecks throughout the process of project delivery should be identified and their impacts on the budget, scope, and schedule of the project should be examined. Also, there is a need for thorough risk analysis that helps GDOT effectively mitigate identified risks, in order to avoid undesirable events and streamline project delivery. Effective risk analyses can reduce both cost and time of delivering transportation projects, particularly when they are
11

utilized at early phases of project development, such as concept and scope development phases and preconstruction. Proper mitigation plans should be identified to remove bottlenecks from project delivery in important areas, such as environmental analysis, right of way (ROW) acquisition, and utilities coordination. Research is needed to aid the systematic identification of major project risks during concept and scope development processes. Main risks should be categorized into appropriate risk factors that represent sources (or causes) of project cost overrun or schedule delay for different project types. Identified risk factors may trigger events that induce direct damages to project outcomes (cost and schedule). Also, there is a research need for developing proper qualitative and/or quantitative risk assessment methods that can help determine the impact of and the magnitude of identified risks on project cost and schedule. This risk assessment approach can be the basis to make an informed decision about contingency in cost estimation for proper budgeting and scheduling. The research objective of this study is to develop a comprehensive guidebook that advances the adoption of risk analysis tools in GDOT, in order to expedite project delivery. The deliverable will be a set of best practices for risk analysis that helps GDOT reduce undesirable project outcomes (cost overrun and schedule delay) and streamline project delivery. Identified risk factors can help GDOT address project issues at their sources and reduce the chance of undesirable events that negatively impact project outcomes. Practical risk-based tools that are fairly easy to understand and implement, can be significant in streamlining project delivery for GDOT. The proper use of risk analysis will be significant to improve the image of GDOT as the leading public agency that maximizes the utilization of tax payers' dollars with streamlining much-needed transportation projects.
12

The project will lead to the development of a comprehensive guidebook that advances the adoption of risk analysis tools to enhance the process of project delivery in GDOT. The deliverable will be a set of best practices for risk analysis that helps GDOT to reduce undesirable project outcomes (cost overrun and schedule delay) and streamline project delivery. Identified risk factors can help GDOT address project issues at their sources, develop proper measures to resolve risks, and reduce the chance of undesirable events that negatively impact project outcomes. To achieve the research objectives, the following tasks have been done, and the report is structured as follows: Chapter 2- Review the academic/professional literature on risk analysis: In the first step, we conducted a thorough academic/professional literature review. The main goal of this task is to collect information and data related to the state of knowledge in risk analysis for transportation project development. Chapter 3- Review the current state of practice in risk analysis among leading state DOTs: Selected state DOTs were reviewed to understand their current processes for risk analysis strategies, guidelines, and records. A survey and structured interviews were conducted to explore risk analysis in state DOTs, such as Washington, California, Utah, New York, Minnesota, Louisiana, Missouri, Montana, Michigan, Massachusetts, and Rhode Island. Their project risk management processes were investigated from policy standpoint and organizational perspective. Chapter 4- Study the current state of practice in GDOT related to risk analysis: Semi-structured interviews were conducted with employees responsible for project management, program level roles, and specialists from different offices. The key factors that influence risk
13

management practices within GDOT were explored. Then, the results were analyzed and a model explaining the current risk management practice and future needs was developed. Chapter 5- Develop Comprehensive Risk Assessment For Transportation (CRAFT) software: Comprehensive Risk Assessment For Transportation (CRAFT) is a software tool specifically designed for identification and qualitative assessment of highway project risks during the preconstruction phase of the project. After developing the comprehensive list of potential risks based on the review of the academic and professional literature, current state of practice among leading state DOTs, and the current state of practice in GDOT, several interviews and meetings were conducted with professionals at GDOT to investigate the applicability of the initial list of potential risk factors for the GDOT. The most important and probable risk factors have been identified for each office. The software has been designed and developed based on the shortlisted risk factors. The software program is equipped with the modification capability of adding new risk items and/or removing some of the predetermined risk factors from the assessment. Chapter 6- Conclusions: The summary of the research methodology and findings are presented.
14

CHAPTER 2 LITERATURE REVIEW

2.1. WHAT IS RISK?
Project Management Body of Knowledge (PMBOK) defines project risk as "an uncertain event or condition that, if it occurs, has a positive or a negative effect on at least one project objective." (Project Management Institute, 2013). Other organizations provide similar definitions, in which they characterize risk as an uncertain event with positive or negative impact on project objectives. These definitions are illustrated in Table 2-1. Although project objectives can be anything from cost and schedule to quality, sustainability, and public acceptance, the primary focus of the available literature is mostly on the project's cost and schedule.

Table 2-1: Definition of (project) Risk

Definition
An uncertain event or condition that, if it occurs, has a positive or a negative effect on at least one (project) objective The exposure to the chance of occurrences of events adversely or favorably affecting project objectives as a consequence of uncertainty Project risk is an uncertain event or condition that, if it occurs, has a positive or a negative effect on at least one project objective. A risk may have one or more causes and, if it occurs, one or more impacts.
The effects of uncertainty on objectives
The exposure to the change of occurrences of events adversely or favorably affecting project objectives as a consequence of uncertainty.

Organization or Author
Project Management Institute, 2013 Washington State Department of Transportation, 2014
Caltrans, 2007
ISO3100 (Standard Organization for Standardization, 2009) Al-Bahar & Crandall, 1990

2.2. WHAT IS PROJECT RISK MANAGEMENT?
Project Management Body of Knowledge (Project Management Institute, 2013) defines project risk management as "the systematic process of identifying, analyzing, and responding to project risk. It includes maximizing the probability and consequences of positive events and minimizing
15

the probability and consequences of adverse events to project objectives." Similarly, The Association for Project Management (APM) defines the risk management process as "a structured process that allows individual risk events and overall project risk to be understood and managed proactively, optimizing project success by minimizing threats and maximizing opportunities" (Dixon, 2006). This process begins with the development of risk management plan, which specifies how to approach and complete the project risk management activities. Risk management planning requires the following inputs (Project Management Institute, 2013): Project scope definitions, organization risk management policies, defined roles and responsibilities, stakeholder risk tolerance, and Work Breakdown Structure (WBS). The output of risk management planning is a plan which describes how each of the risk management activities should be conducted.
The international Organization for Standardization (ISO) mentions that a risk management should (Standard Organization for Standardization, 2009):
create value, be an integral part of organization processes, be part of decision making, explicitly address uncertainty, be systematic and structured, be based on the best available information, be tailored, take into account human factors, be transparent and inclusive, and
16

be iterative and responsive to changes.
Based on this definition, a risk management should be structured, explicit in identifying risks, scalable, and more importantly, create value for money.
2.3. WHY RISK MANAGEMENT?
In his study, Flyvbjerg (2002) showed that 86% of the construction projects are completed with the cost that is greater than the original estimated cost. This study identifies the lack of proper risk management as a main reason for this cost overrun. Hence, the ability to better understand and manage potential project risks yields benefits that are in excess of the costs to adopt risk management practices. A clear understanding of project-specific risks will help project teams make effective decisions by tackling uncertainty during project delivery and development. This process assists the agency to better estimate the project costs and schedule, and to assure that the project is completed within the estimated budget and time while all other objectives are satisfied. Risk Management helps an agency to systematically identify, assess and control potential risks and obtain higher value for money. Moreover, a systematic risk management technique is methodical rather than intuitive, and clarifies how to measure and document risks (Al-Bahar & Crandall, 1990). To support the necessity of a structured risk management system, the U.K. Highways Agency in its January 2001 report (U.K. Highways Agency, 2001) argues, "If Agency colleagues take decisions in ignorance of the associated risks, regardless of their possible impact on business, they are likely to reduce Value for Money (VFM) rather than enhance it. This is exacerbated if the Agency is actively encouraging a more well thought approach towards risk taking, without defining the framework or criteria within which colleagues are expected to do so."
17

2.4. RISK MANAGEMENT PROCESS
Risk management, in its general form, is the process of identifying, analyzing, mitigating, monitoring, and controlling potential uncertain events that negatively or positively affect the project objectives. This process is for the project team to better understand the project risks and to effectively mitigate them. Over years, different variations of this process have been introduced by several authors. Chapman introduced a generic 9-phase process for project risk management. The nine phases of this process are: Define, focus, identify, structure, ownership, estimate, evaluate, plan, and manage (Chapman, 1997). Table 2-2 describes each of these phases.

Table 2-2: Generic 9-phase Project Risk Management Process (RMP) Identified by

Chapman (1997)

Phase
Define Focus
Identify
Structure Ownership Estimate Evaluate Plan
Manage

Purpose
Consolidate relevant existing information about the project. Fill in any gaps uncovered in the consolidation process Scope and provide a strategic plan for the RMP Identify where the risks might arise Identify what we might do about the risks Identify what might go wrong with our responses Testing simplifying assumptions Providing more complex structure when appropriate Client-Contractor allocation of ownership and management of risks and responses Identify areas of clear and possible significant uncertainty Synthesis and evaluation of the results of the estimate phase Project plan ready for implementation and associated risk management plan Monitoring Control Developing plans for immediate implementation

Al-Bahar (Al-Bahar & Crandall, 1990) introduces a systematic risk management approach for construction projects with three major steps: Risk identification, risk assessment, and risk response. This approach is methodical, systematic, objective, and contains quantitative measurement.
18

Similarly, Kahkonen (Kahkonen, 1997) defines project risk management with fewer phases: organization and scope, risk identification, risk analysis, decision and risk strategy, response planning, and continuous control and feedback. Kahkonen also introduced a risk management roadmap as illustrated in Table 2-3. The World Road Association (World Road Association, 2012) identified risk management as a five-phase process: Establishing the context, assessing the risk, treating the risk, communication and monitoring the risk. Figure 2-1 illustrates the phases of World Road Association's risk management process.
Figure 2-1: Risk Management Process Defined by World Road Association (World Road Association, 2012)
19

Table 2-3: Road Map of Project Risk Management Developed by Kahkonen (1997)

Organization and scope of project risk management
Personal task for project manager
Risk management workshops
Facilitator involvement needed

Risk identification
Experience and intuitive awareness
Interviewing
Generic checklists: broad headings

Risk analysis
Project risk list
Verbal risk description Project risk list and additional data: Causes and timing

Decision and risk strategy
Modify Project objectives
Risk avoidance
Risk prevention

Planning and decision on responses
Response list
Response list with additional data: Cost of response and timing Quantification and charting: Effects of planned responses

Systematic procedures for project risk management

Generic checklists: hierarchical list including more detailed risk drivers

Quantification and charting: Impacts of risks Risk mitigation on project outcomes

Quantification and charting: trade-off analysis

Risk mitigation
Develop contingency plans Monitor situation Accept risks

Quantification and charting: trade-off analysis

Project risk knowledge base: problems encountered, close events

Continuous control and feedback
Responsibility control
Advanced reporting practice
Regularly updated experiential checklist Project risk knowledge base: problems encountered, close events

20

The Association for Project Management (APM) (Dixon, 2006) defined an iterative process for risk management consisting of five phases as: 1) initiate, 2) identify, 3) assess, 4) plan responses, and 5) implement resources. Figure 2-2 shows the process. As it is illustrated, after each step, the project team should revise and update the previous phases. During the "initiate" phase, the scope, objectives, and context for the risk management process are determined. The risk events relevant to a project are identified as comprehensively as possible during the "identify" phase. For some risk events, the risk responses may also be identified naturally during this phase. The identified risks will be assessed and evaluated for their impact and probability during the assess phase. Based on this evaluation, the project goals and scope and the risk list might be updated. In the final stage, the project team should find the potential responses for each of risks.
Figure 2-2: Risk Management Process Defined by APM (Dixon, 2006) A risk management guidebook developed by New South Wales government (NSW 2011) in Australia defined six major steps for project risk management as follows (Figure 2-3):
1. Establish the context: Determine the scope of risk management process considering organization and project environment, project characteristics, goals and objectives, dependencies and stakeholders.
21

2. Identify and define risks: Identify project risks by involving experts and stakeholders. The identified risks should be recorded in the project risk register.
3. Conduct risk analysis: Analyze the risks to determine their causes, recognize their impacts, and estimate their probability.
4. Conduct risk evaluation: Prioritize the identified risks according to their impacts and likelihood. At the end of this step, the risks that require treatments are identified.
5. Develop and implement risk treatments: Develop risk treatments to effectively reduce and control the risks.
6. Monitor, report, update and manage risks: Monitor and update the identified risks as they might change during the project. New risks may be identified as project progresses and some existing risks may be eliminated through the effectiveness of the risk treatments.
Figure 2-3: Project Risk Management Process Defined by NSW Government (NSW 2011) 22

The Joint Standards Australia/Standards New Zealand Committee (2004) developed the Australian/New Zealand Standard AS/NZS 4360:2004 and identified a risk management process, very similar to the one developed by Road World Association, with seven major elements as follows:
1. Communicate and consult: Communicate and consult with internal and external stakeholders at each stage of the risk management process.
2. Establish the context: Establish the context of risk management and define the structure of the risk analysis.
3. Identify risks: Identify the events that could negatively affect the achievement of the project objectives.
4. Analyze risks: Determine consequences of the risks and the probability of their occurrence. This analysis helps to rank the risks.
5. Evaluate risk: Compare estimated impact and probability of risks against the preestablished criteria to make decisions about the extent and nature of treatments.
6. Treat risks: Develop and implement specific strategies to increase potential benefits and reduce potential costs.
7. Monitor and review: Monitor the effectiveness of all steps of the process, revisit the risks, and check for new risks.
The Canadian Standard Association (2002) presented a process for risk management consisting of six steps as follows (Figure 2-4):
1. Initiation: - Define problem or opportunity and associated risk issues
23

- Identify risk management team - Assign responsibility, authority, and resources - Identify potential stakeholders 2. Preliminary Analysis: - Define scope of the decisions - Identify hazards using risk scenarios - Begin stakeholder analysis - Start the risk information library 3. Risk Estimation: - Define methodology for estimating frequency and consequences - Estimate frequency of risk scenarios - Estimate consequences of risk scenarios - Refine stakeholder analysis through dialogue 4. Risk Evaluation: - Estimate and integrate benefits and costs - Assess stakeholder acceptance of risk 5. Risk Control: - Identify feasible risk control options - Evaluate risk control options in terms of effectiveness, cost, and risks - Assess stakeholders' acceptance of proposed actions - Evaluate options for dealing with residual risk - Assess stakeholder acceptance of residual risk 6. Action/Monitoring:
24

- Develop an implementation plan - Implement chosen control, financing, and communication strategies - Evaluate effectiveness of risk management decision process - Establish a monitoring process.
Figure 2-4: Risk Management Process Developed by Canadian Standard Association (2002) The US Army Corps of Engineers developed Cost and Schedule Risk Analysis (CSRA) guidance in 2009. Their proposed risk management process contains four main building blocks as follows: Identification, Assessment, Response, and Documentation. Similar to other risk management process, National Cooperative Highway Research Program (NCHRP) and Federal Highway Administration (FHWA) formalized the process of risk management as a five-phase process: identification, assessment, mitigation, allocation, and updating (Molenaar et. al, 2010; FHWA, 2006). Figure 2-5 illustrates the connection between these phases.
25

1. Identification-The primary objectives of risk identification are to categorize and document risks. A team compiles the project risk events and the risk management skills of the members after a review of the issues found in the project management process. Classification of risks helps in reducing redundancy and adds an organizational component to the process of identification. For example, Caltrans classifies risks into the technical, external, environmental, organizational, project management, construction, and right of way risk categories.
2. Assessment- It is the process of evaluating risk events that were documented in the Identification stage. The risk frequency and consequence severity are assessed in this phase. The assessment analyzes a combined effect of risks on project scope, schedule, and cost. This assessment can be done qualitatively or quantitatively.
3. Mitigation- The team seeks to reduce the probability or consequences of a risk event to an acceptable threshold. It accomplishes this via many different means that are specific to the project and the risk. Mitigation steps, although costly and time consuming, may still be preferable to going forward with the unmitigated risk allocation.
4. Risk Allocation- It defines the roles and responsibilities for risks and the fundamental aspect of risk management is to allocate risks to a party that is capable of managing it.
5. Tracking & Updating-The objectives of risk tracking and updating is to systematically track identified risks, identify new risks, manage the contingency reserve, and develop experience for further risk assessment and allocation.
26

Figure 2-5: Cyclical Nature of Risk Management Adopted from FHWA (2006) SHRP2 report (Molenaar et al., 2014) provides similar cyclical diagram with few more steps for the risk management process. This report emphasizes on project scope/strategy condition, structuring and risk management implementation in the risk management process. Figure 2-6 illustrates this process.
Figure 2-6: Risk Management Process Introduced by SHRP2 Report (Molenaar et al., 2014) 27

The review of risk management process illustrates that risk identification, assessment, and response are the major parts of every risk management process. In the next section, we will review the methods and techniques that have been used for implementing these three major steps.
2.5. REVIEW OF EXISTING RISK IDENTIFICATION METHODS
To identify risks affecting construction projects and to assess the impact of the risk on project's overall cost and schedule, one should use information gathering techniques such as analyzing assumptions, reviewing previous projects' documents, eliciting experts' opinions, and diagraming techniques. These techniques have been used in several research studies and are explained in this section. Table 2-4 summarizes all the risk identification methods.
2.5.1. Assumption Analysis An assumption analysis involves examining cost, schedule, and design assumptions, documenting these assumptions, and analyzing them as potential risks. Each assumption should be evaluated for validity, accuracy, consistency, and completeness. If uncertainties in these assumptions are identified, then risks should be developed based on these uncertainties (FHWA, 2006; WSDOT, 2010). As part of the risk identification process, Washington State DOT (WSDOT) employed assumption analysis to identify the risks of nine mega projects. These projects were evaluated in a formal workshop with the presence of several experts from different disciplines in which project assumptions were presented and assessed (Molenaar, 2005).
2.5.2. Reviewing Previous Projects'Documents To identify the risks associated with a specific project, one can refer to similar projects undertaken in the past years. Although each project has its own specifications (e.g. the project
28

time and location is specific to that project), most projects have several overlapping issues. For example, most of the highway construction projects end up relocating utility systems, obtaining environmental permits, or purchasing Right of Way (ROW) (Franklin, 2009; Molenaar, 2005). By considering these common issues through reviewing the project documents and examining the similarity between those projects and the current project, one can generate a risk registry for the project in hand. Previous projects' documents and published reports on project risk management (such as FHWA, SHRP, and DOT's reports) provide information about the risks that have been identified during lifetime of a (highway construction) project, as well as those that were neglected but impacted the project. This technique has been employed by (Franklin, 2009). Franklin, based on his experience in Federal Transit Administration (FTA), identified several risks associated with transportation projects and grouped them in the categories such as general risks, project specific risks, and geotechnical risks. As another example, Creedy et al. reviewed historical highway projects to identify risks factors that cause cost overrun on these projects (Creedy et al., 2010).
2.5.3. Eliciting Experts'Opinions To elicit experts' opinion, one can employ the following techniques: Brainstorming: Another method of identifying project risks is holding formal and informal brainstorming sessions with project team members or extended project team members such as specialty groups, stakeholders, and regulatory agency representatives (WSDOT, 2010). Brainstorming sessions should have a facilitator to lead the meeting and assist the group to generate a list of risks. One specific technique in holding the brainstorming session is Crawford slip method. This brainstorming method has the following steps:
Give a short introduction about the project and about the brainstorming process,
29

for ten minutes, ask each participant to write one potential risk in each minute, and
at the end of the ten minutes, gather all the identified risks and remove the overlapping ones.
The benefit of this method is that a large number of risks can be identified in a very short time (FHWA, 2006; Project Management Institute, 2013; Molenaar et al., 2014).
Delphi method: Instead of holding a brainstorm session to identify risks, the project team can use a technique called the Delphi method. In this technique, the facilitator sends out questionnaires to the experts, collects their responses, and while keeping the respondents anonymous, circulates the obtained answers among respondents. After viewing other responses, the experts answer the questions again (they might change their answer after learning about other thoughts.) This process continues until the facilitator decides that the experts converged to similar answers (Ayyub, 2001; Berg, 2010).
Survey and interview: In addition to brainstorming sessions and the Delphi method, one might elicit experts' opinion about project's potential risks via a survey or an interview (Tran & Molennar, 2012; Project Management Institute, 2013). For example, Strassman and Wells (Strassman & Wells, 1988) surveyed experts to identify project risks from both contractor and the client perspective. Also, Kangari (Kangari, 1995) surveyed several contractors and owners to identify project risks. Interviewing experts assists not only to identify new risks, but also to validate those risks that have already been identified from documents of similar projects or a brainstorming process (Molenaar, 2005; Project Management Institute, 2013). An interview can be structured or semi-structured. While structured interview consists of a set of predefined
30

questions, a semi-structured interview allows discussions about new relative topics. In addition to the interview, survey questionnaire can be used to elicit expert's opinions about potential risks. Rather than identifying risks, a survey might ask for expert's assessment of the probability of a potential risk or the impacts of the risk on a particular aspect of a project (Sensi et al., 2012; Hallowell & Molenaar, 2013; D'Ignazio et al., 2011).
Risk identification workshops: In highway megaprojects, the risks should be identified in formal workshops where an interdisciplinary group of experts, including a risk analyst, identifies and evaluates potential risks (Molenaar, 2005; WSDOT, 2010). For example, in a study (Molenaar, 2005), WSDOT identified risks associated to 9 megaprojects using formal workshops. In these workshops, several risks were identified. The most critical risks were: Market Condition, Environmental, Third Party, Right of Way, WSDOT Management, Geotechnical, and other risks.

2.5.4. Diagraming Techniques Diagraming techniques are tools which illustrate the causes of risks, the time ordering of the events, and other relationships between risks (Project Management Institute, 2013; FHWA, 2006).

Table 2-4: Risk Identification Methods (Ayyub, 2001; Project Management Institute, 2013;

Cretu et al., 2011)

Risk Identification Method
Assumption analysis Reviewingpreviousprojects' documents
Eliciting expert opinion
Diagramming

Description
examining cost, schedule and design assumptions
Identify risk that appeared in previous projects
Gathering expert's opinion about potential risk and their impact using interview, survey, or brainstorming techniques. Graphical tools to show root causes and their relationship

31

2.6. REVIEW OF EXISTING RISK ASSESSMENT METHODS
The next step after identifying risks is to analyze risks for their eventual impact on cost, schedule and quality of a project. Risk analysis techniques can be categorized into two general groups, qualitative and quantitative methods. 2.6.1. Qualitative Risk Assessment The qualitative risk analysis, which is usually conducted simultaneously with the risk identification, applies similar techniques such as a survey questionnaire, interview, or brainstorming to qualitatively assess the chance and impact of a risk (Al-Bahar & Crandall, 1990; World Road Association, 2012; FHWA, 2006). For example, an expert might assess a risk as very likely with low impact, and another risk as unlikely with high impact. This process has been used to evaluate risks for several projects. For example, California and Washington State Departments of Transportation (Caltrans, 2012; WSDOT, 2010) employ this assessment method for smaller projects that cost less than five million dollars. Figure 2-7 shows the input, tools, and output of the qualitative risk assessment (Project Management Institute, 2013). Figure 2-8 illustrates how risks can be qualitatively categorized based on their probability and impact.
32

Figure 2-7: Inputs, Tools, and Outputs for Qualitative Project Risk Assessment, Adopted from Project Management Institute (2013)
needed. .
Figure 2-8: Qualitative Risk Assessment Adopted from Molenaar et. al (2010) 33

The output of qualitative risk assessment is the overall risk ranking of a project, the list of prioritized risks, and the list of risks for additional analysis.
2.6.2. Quantitative Risk Assessment Quantitative risk analysis methods employ probabilistic models to evaluate the impact of risks on project's objectives, especially cost and schedule. This process may use following techniques, summarized in Table 2-5 (Project Management Institute, 2013; Baloi, 2012):
Traditional method: The traditional method of risk analysis considers effects of risks on project cost by adding cost contingencies to the project baseline cost. In this process, the cost contingency is calculated by multiplying the risk factor, determined based on experience, to the estimated cost of the project element, which might be affected by that risk. The more complicated techniques to analyze the risks are sensitivity analysis, Monte Carlo simulation, and fault trees.
Sensitivity analysis: Sensitivity analysis identifies the effect of changes in the probability or impact of a risk on a project objective by fixing all other risks' values. As a result, this method can illustrate what risks have the highest impact on the project, and require special attention. Several studies used this technique to evaluate project risks. For example, Wu (2006) conducted sensitivity analysis to compare the effect of risks on transportation infrastructure cost overrun. Molenaar (2005) used sensitivity analysis to rank the megaproject risks identified and assessed in a formal workshop. Also, Alarcon et al. (2010) used sensitivity analysis to assess the impact of risks on the cost variation of Panama Canal expansion project. Monte Carlo simulation: Monte Carlo simulation gathers the information about the probability and impact of risks to generate overall distribution of the final project cost. In this method, since
34

the project cost depends on the probability and the impact of each risk, one should sample the cost of each risk based on its probability distribution and calculate the total project cost. This process should be continued for large number of times. The final result will be a distribution of the project cost. This method requires large amount of data such as mean and variance of the distribution of each identified risk or the three point (minimum, average, and maximum) values of the probabilities and impact of each risk. Molenaar (2005) assessed risks associated to 9 megaprojects using Cost Estimate Validation Process (CEVP). CEVP is an intense workshop process, resampling value engineering, in which several experts and stakeholders gather to identify and assess project risks. In this process, each of the nine megaprojects is thoroughly evaluated by a multidisciplinary team of experts from both the public and private sectors. During these workshops, the identified risks were analyzed using sensitivity analysis and Monte Carlo simulation to determine their impact on final cost and schedule of the projects. Similarly, Touran (2006) employed Monte Carlo simulation technique to forecast the cost escalation due to the market condition risks, and identified the overall project's cost distribution. Finally, McGoeySmith et al. (2007) conducted Monte Carlo simulation to estimate the cost of Highway 11 Twinning project in Canada at year 2005 and the year of expenditure (YOE). Figure 2-9 and Figure 2-10 adopted from McGoey-smith et al. (2007) show the input and output results of the Monte-Carlo simulation respectively for this project.
35

Figure 2-9: Probability Density of a Risk event as an Input for Monte Carlo Simulation, Adopted from McGoey-Smith et al. (2007)
Figure 2-10: Project Cost Distribution Obtained from the Monte Carlo Simulation, Adopted from McGoey-Smith et al. (2007)
Fault tree analysis: A Fault tree identifies and analyzes factors, conditions and causes that potentially contribute in occurrence of a top event. This method not only identifies the root causes, but also illustrates how the causes come together to trigger an event. This technique requires estimates of probability and impact of several causes to allow the calculation of probability and impact of the top event. Carr & Tah (2001) employed cause diagrams to represent the relationship between the causes, risks, and their consequences in construction projects.
36

Table 2-5: Quantitative Risk Analysis Methods

Risk analysis method
Traditional Method Sensitivity Analysis
Monte Carlo Simulation
Fault Tree Analysis

Description
Contingency = risk factor * base cost Gives the impact of a risk on an objective when all other risks remained fixed. Combines the distribution of all risks (as uncertain events), and provides a final distribution of the value of interest. Calculates the probability of a top event by considering the chances and the relationships of the causal events.

2.6.3. Identification of Barriers for Risk Analysis in Organizations In 1996, Akintoye & MacLeod (1997) surveyed 30 general contractors and 13 project managers to evaluate the reasons that the contractors and construction mangers did not apply the more sophisticated techniques (e.g. fault tree, Mote Carlo simulation, Bayesian methods) in risk analysis. The main reasons provided in this study were:
Lack of familiarity with the techniques. The degree of sophistication involved in the techniques is unwarranted for project
performance. Time plus lack of information and knowledge. Doubts whether these techniques are applicable to the construction industry. Most construction projects are seldom large enough to warrant the use of these
techniques or research into them. They require availability of sound data to ensure confidence. The vast majority of risks are contractual or construction related and are fairly subjective,
hence they are better dealt with based on experience from previous contracts undertaken by the firm. It is difficult to see the benefits.
37

Risk analysis of construction projects is seldom formally requested by clients (clients expect project management practice to set up projects risk-free).
Risk analysis in commercial terms is not always viable on projects. Project risk management is about people not scientific models. Lack of expertise in the techniques.
More recently, Sensi et al. (2012) evaluated the barriers to applying probabilistic risk analysis. At the preliminary stage, this study surveyed 104 organizations including owners, Engineering, Procurement and Constriction (EPC) firms, contractors, and design firms. Based on the survey questionnaire, researchers selected 12 firms, from which data were collected thorough interviews and reviewing documents. Considering these data, the following barriers were identified:
Difficulty in Interpreting Results Lack of Organizational Support Lack of Policy or Procedures Lack of Technical Expertise Lack of Transparency amongst Stakeholders
These two studies highlight the need for risk analysis workshops and simple easy-to-understand risk analysis tools. Moreover, they underline the importance of the alignment of risk management policies and techniques in all the levels of an organization. Risk maturity and policy analysis can be used to bridge the existing gaps.
Similarly, Chileshe et.al (2013) reviewed several studies that identify barriers in conducting risk assessment and management. Table 2-6, adopted from Chileshe et.al (2013), summarizes these studies.
38

Table 2-6: Summary of Studies about Risk Assessment and Management Barriers Adopted

from Chileshe et al. (2013)

Study and context
Kim and Bajaj (2000): Interview of 13 Korean contractors
Lynos and Skitmore (2004): Survey of 17 contractors, 11 consultants, 10 clients, and 6 Developers in Australia
Chileshe and Yirenyi Fianko (2012) Survey of 34 contractors, 46 consultants and 23 clients (public and private) in Ghana

Risk assessment barriers
Lack of familiarity with techniques Lack of expertise, and Owner's interest in tangible calculations and results Lack of experience Lack of time Lack of resources Unclear benefits Lack of coordination Lack of resources and expertise Lack of information and knowledge

As it is summarized, lack of resources and expertise are the major reasons all around the world that inhibit the owners and contractors to conduct risk assessment. Moreover, some contractors and owners don't find any value in conducting more sophisticated risk analysis.
2.7. REVIEW OF EXISTING RISK RESPONSE METHODS
Project management body of knowledge (Project Management Institute, 2013) introduces several risk response strategies:
Avoidance: The project team might avoid risk by changing the project plan. Although it is not possible to avoid all risks, some of them can be avoided. Some examples of risk avoidance are:
Reducing scope to avoid high-risk activities, Adding resources or time, Adopting a familiar approach instead of an innovative one, or Avoiding an unfamiliar subcontractor.

39

Transference: This technique transfers risks to a third party. Transference can be conducted using contracts. Insurance, warranties and guarantees are sometimes required for risk transference. Mitigation: In this technique, the project team tries to reduce the probability or the impact of the risk to an acceptable threshold. Examples of risk mitigation methods are:
Adopting less complex processes, Conducting more seismic or engineering tests, Choosing a more stable seller, or Adding resources or time to the schedule. Acceptance: This technique indicates that the project team has decided not to change the project plan to deal with a risk or is unable to identify any other suitable response strategy. Following techniques are the most well-known techniques of risk mitigation, and are offered by other organizations or researchers. For example, Franklin (2009) identified the following risk mitigation strategies:
Transferred to a third party via insurance (this is common for certain construction risks such as accidental injury or death),
Revising or developing an alternate design to be considered should a low probability but high consequence risk occur,
Contract provisions that share risk between the owner and contractor, and Build-phase workarounds suggested by the contractor.
40

Cretu et al. (2011) provide examples for each of the risk response as illustrated in Table 2-7. Similarly, California Department of Transportation provided examples for risk response. Table 2-8 shows examples of risk response strategies for different risks identified by Caltrans (2012).

Table 2-7: Example of Risk Response Adopted from Cretu et al. (2011)

Strategy
Avoid Transfer
Mitigate

Risk
Geotechnical conditions increase the cost of retaining wall
In a subway project, the owner first decided to furnish tunnel boring machine equipment for the contractor, but the project team felt that there is a risk that the contractor can blame the owner for any machine inefficiency.
A roadway project requires extended period of heavy construction within 10 feet of several residences. This might cause lawsuit and delay the project, and increase the cost.

Response
If the cost is lower than the risk cost, purchase extra right of way to replace the wall with embankment.
Transfer the risk to the contractor so that they furnish their own equipment.
Negotiate with the residents to relocate them for the project duration. This increases the project cost significantly, but remove the large impact that a lawsuit can have on the project.

41

Table 2-8: Examples of Risk Response Adopted from Caltrans (2012)

Risk description

Inaccuracies or incomplete information in the survey file could lead to rework of the design.

Design Environmental Right of Way

A design change that is outside of the parameters contemplated in the Environmental Document (ED) triggers a supplemental Environmental Impact Report (EIR) which causes a delay due to the public comment period.
Potential lawsuits may challenge the environmental report, delaying the start of construction or threatening loss of funding.
Nesting birds, protected from harassment under the Migratory Bird Treaty Act, may delay construction during the nesting season. Due to the complex nature of the staging, additional right of way or construction easements may be required to complete the work as contemplated, resulting in additional cost to the project. Due to the large number of parcels and businesses, the condemnation process may have to be used to acquire ROW, which could delay start of construction by up to one year and increase construction costs.

Response
Mitigate: Work with Surveys to verify that the survey file is accurate and complete. Perform additional surveys as needed.
Avoid: Monitor design changes against ED to avoid reassessment of ED unless the opportunity outweighs the threat.
Mitigate: Address concerns of stakeholders and public during environmental process. Schedule additional public outreach. Mitigate: Schedule contract work to avoid the nesting season or remove nesting habitat before starting work.
Mitigate: Re-sequence the work to enable ROW Certification.
Mitigate: Work with Right-of- Way and Project Management to prioritize work and secure additional right-of-way resources to reduce impact.

2.8. SUMMARY
Project risk management is an important process that can objectively identify, evaluate, and analyze project risks. This process can increase the value of the project and assure that the project is completed within the budget and schedule, and that other project objectives are effectively satisfied. Several organizations proposed different processes for risk management. However, risk identification, assessment, and response are at the heart of every proposed process. Several methods and techniques for risk identification and assessment, and different examples for risk responses have been introduced in this chapter. In the next chapter, several state DOTs will be reviewed for their current state of risk management.

42

CHAPTER 3 REVIEW OF THE CURRENT PRACTICE OF STATE
DOTS IN PROJECT RISK MANAGEMENT
3.1. INTRODUCTION
Currently, a common understanding of the importance of risk and risk management is pervasive throughout many state DOTs. Some of them such as Washington State DOT and California DOT (i.e. Caltrans) have a specific process integrated to their project development process and guidebook for project risk management. Some other state DOTs such as Massachusetts DOT (MassDOT) and Rhode Island DOT do not have a standard procedure for project risk management and they rely heavily on project managers' experiences to manage the uncertainties and risks. Studying and reviewing the current practices by other state DOTs is an important step to develop a standard process for project risk management at GDOT.
In this part of the research project, we conducted a survey with different state DOTs with various levels of risk management process. The questionnaire had ten major sections as follows:
1- Survey goals and objectives 2- General information of the respondent 3- State of risk management in the agency 4- Risk management process and organization 5- Candidate projects for conducting risk analysis 6- Risk identification process 7- Risk assessment process 8- Project risk control process 9- Organizational issues
43

10- Project risk management challenges and barriers The results of the survey helped identify leading state DOTs in project risk management and also compare their current practices to implement project risk management. Furthermore, valuable information about state DOTs that do not have a standard procedure to conduct risk analysis and rely mainly on project managers' experiences was provided. The complete version of the questionnaire is available in Appendix A. After studying and analyzing the results of the survey and documents related to state DOTs' project risk management such as guidebooks, reports, and real examples of implementing risk management in major projects, semi-structured interviews were conducted with the subject matter experts that responded to the survey in the first step. During the interviews, more detailed questions about their project risk management process were asked and the interviewees shared their experiences about implementing risk management process, challenges, barriers, and important factors that should be considered for a successful implementation of project risk management. In the following sections, the review of the current state of practice in risk analysis among leading state DOTs that resulted from the survey, interviews, and their related documents are presented.
44

3.2. WASHINGTON STATE DEPARTMENT OF TRANSPORTATION
3.2.1. Risk Management Organization Washington State Department of Transportation (WSDOT) devoted a special office, Strategic Analysis and Estimating Office (SAEO), to evaluate and manage project risks. The major role of this office is to support project managers who are responsible for project risk analysis. This department has also integrated the project risk management procedure into the Project Development Process (PDP) and published a risk management guidebook that detailed the risk management process (WSDOT, 2014). In the Washington State Department of Transportation, the main person responsible for the project risk analysis is the project manager. However, project manager is supported by the SAEO and receives assistance from internal experts. For larger projects that require more accurate risk analysis, the Washington State DOT employs consultants to support the project team. The process of risk management should start as early as possible. In the Washington State DOT, this process usually begins in the Concept Development stage and follows in the Preliminary Design and Environmental studies. Washington State DOT conducts project risk management for every project size and duration, but this process is scalable. In particular, depending on the project cost or schedule, the methods and techniques used to analyze risk can change. Table 3-1 illustrates the thresholds for using qualitative and quantitative risk evaluation methods.
45

Table 3-1: Thresholds for Conducting Risk Analysis from WSDOT (2014)

Project size
< $10 M Between $10M and $25M Between $25M and $100M More than $100M

Risk analysis method
Qualitative risk analysis: Use of risk register Informal workshop using self-modelling spreadsheet1 Cost Risk Assessment (CRA) workshop Cost Estimate Validation Process (CEVP) workshop

Beside the project time and duration, the WSDOT identified the following project types as the most frequent and critical candidates for the risk management process:

Road-Rehab/Reconstruct Projects Road-Resurface/Renewal Projects Interchange-Construct/Improve/Modify Projects Managed Lanes-Construct-Modify Projects Bypass Projects Bridge and Tunnel Projects Grade Separation Projects

3.2.2. Risk Management Process 3.2.2.1. Risk identification Risk identification is the first step of the risk management process, in which the project team predicts the potential surprises that might appear during the project life cycle. There are several techniques that can be employed in the risk identification phase. Washington DOT employs

brainstorming sessions,

1 http://www.wsdot.wa.gov/Projects/ProjectMgmt/RiskAssessment/Information.htm#Self%20Modeling, accessed July 1, 2015
46

structured risk identification workshops, structured interviews with project participants, and Risk Breakdown Structure (RBS)
to identify project risks. During the brainstorm sessions and risk identification workshops, representatives from engineering, environmental analysis & permitting, roadway design, bridge design, geotechnical, design policy & support, construction, materials, utilities, railroad, traffic operations, maintenance, and estimation offices (depending on project needs) are usually among attendees. Moreover, different stakeholders such as district offices, FHWA Division offices, railroad companies, public utilities companies, tribal governments, engineering consulting firms, and highway contractors might be invited to attend these sessions. In this department, the workshops and brainstorming sessions are facilitated by professional facilitator from the consulting world or by an in-house expert.
3.2.2.2. Risk assessment Using the results from the risk identification sessions, the project team employs qualitative or quantitative method to analyze risks. Software such as @risk and spreadsheets are the major tools for risk analysis at Washington State DOT. Figure 3-1 illustrates the spreadsheet used by the Washington State DOT. As illustrated, each risk has an ID, status, group (e.g. ROW, environmental, etc.) description, and trigger. Moreover, in case of quantitative risk evaluation, the team will identify the risk's probability and cost impact. A heat map in the spreadsheet illustrates how severe a risk is. This tool has the capability to run Monte Carlo simulations for more advanced risk analysis.
47

For projects below the $10M, the Washington State DOT uses qualitative methods such as heat maps and for values greater than that, they employ quantitative methods such as Monte Carlo simulations. In particular, WSDOT uses Cost Risk Assessment (CRA), and Cost Estimate Validation Process (CEVP) workshop for the projects with cost greater than $25M.
Cost Estimate Validation Process and Cost Risk Assessment Workshop: In 2002, Cost Estimate Validation Process (CEVP) is developed and trade marked by WSDOT to identify risks and assess their impact on the project cost and schedule. This process is an intense series of workshops in which the risk managers, project managers, high level engineers, risk analyzers and senior experts from local agencies and private sectors review project materials to identify risks, to evaluate their impacts, and to plan for mitigation strategies. CEVP is mostly conducted on mega projects (> $100 M) and uses more complex probabilistic methods such as Monte Carlo simulation to evaluate the consequences of risks on project costs. The major goal of CEVP is to better estimate project costs, and to provide accurate, robust information for stakeholders and public. This process has been conducted in nine mega projects which are summarized by Molenaar in 2005. In addition to CEVP, Washington State DOT coined the term Cost Risk Assessment (CRA) to refer to a similar process of risk assessment, which should be conducted in a smaller scale for projects with costs from 25 to 100 Million dollars. The overall purpose of CEVP and CRA is to establish a logical base cost estimate and to incorporate risk events that might cause the project to turn out differently than planned (WSDOT, 2014).
3.2.2.3. Risk response WSDOT considers four different strategies for risk response: Avoid, Transfer, Mitigate, or Accept. When it is possible, the project team should avoid the identified risk, for example, "If the particular risk concerned wetland impacts, re-aligning the road to avoid the wetlands would be
48

an avoidance strategy." In the cases where the contractors or other parties are more equipped to handle a risk, the project team might want to transfer the risk to that party by using contract terms. If the risk could not be avoided completely or transferred to other parties, the project team should consider different engineering techniques to mitigate that risk. For example, by conducting early environmental studies, the project team can mitigate the probability of environmental approval delays. Using new strategies is sometimes useful in mitigating the risks, as mentioned by a respondent:
"Also, sometimes the risk management is helpful to avoid some risk by thinking of new strategies. For example, in a project, a risk was identified with 50% chance of occurrence which costs 8 million. They avoided this risk by spending 3 million and adding that money in the baseline. "
3.2.2.4. Evaluation and control Washington State DOT has different mechanisms to evaluate and control the risks. First, to share and capture the obtained knowledge, they generated their own guidebook, and published papers in conferences. Moreover, after project completion, this department compares the final cost and schedule with the estimated one to evaluate the performance of the risk management process. Moreover, quarterly meetings are a method to keep the team updated:
"In the 520-bridge project, in the quarterly meetings, the project managers were called to discuss different risks. This is an incentive to think about the risks. People are busy with their daily tasks, which make them avoid think about risks on daily basis."
49

Finally, they generated risk culture by great support from executives and upper level management.
3.2.3. Lessons Learned and Challenges Measuring the effectiveness of the risk management is the main difficulty of the risk management process. An interviewee from WSDOT mentioned:
"Risks are hard to be identified, but the process of risk analysis generates the expectations for these types of events. For example, in the Alaska tunnel project, a machine hit the pipe and caused a long delay in the project schedule. The risk management process might not be able to identify this risk, but it generates awareness. The biggest advantage of the risk analysis is avoiding surprises. Things happen regardless of how well the risk analysis is done or how much money is spent on mitigation strategies. However, it helps to avoid surprises."
The Washington State DOT considers the following as the most important challenges in the project risk management:
1. Lack of staff or resources for complex tasks 2. Overall lack of adequate funds 3. Lack of existing policies 4. Lack of risk culture 5. Lack of communication among offices 6. Lack of support from the top 7. Lack of training of personnel 8. Inaccurate forecasts
50

9. Inefficient organizational frameworks 10. Issues with the risk management tools 11. Lack of desire to use new procurement methods 12. Lack of best practices and available training Risk reserve for incentivizing risk management implementation: Previously, WSDOT budgeted projects, for which the risk assessment was conducted, at the 90th percentile of the cost distribution. This generous budgeting minimized the incentive of effective risk management, and did not create an environment for aggressive risk management. To incentivize project managers to follow the risk management plans, WSDOT introduced risk reserve. Risk reserve is defined as the difference between the 70th percentile of cost distribution and the base cost, which is mostly about 40th percentile of cost distribution. Project managers should follow the risk management plans to complete the project within the base cost which include contingencies for construction adjustment. To access the risk reserve, project managers should describe the mitigation strategies they have implemented to mitigate the identified risks, and receive approval from Region Program management. The risk reserve is monitored and as risks are retired, the amount is adjusted.
51

Figure 3-1: Risk Management Tool Screenshot 52

3.3. CALIFORNIA DEPARTMENT OF TRANSPORTATION
3.3.1. Risk Management Organization California Department of Transportation (Caltrans) has established Office of Enterprise Risk Management and developed their special enterprise risk management that considers the whole department and identifies risks related to the department strategic plan, goals, and objectives. Their enterprise risk management system consists of department level risk management, program level risk management, and project level risk management. Department level risk management is the responsibility of executives and deals with risks that impact achievement of department goals and objectives. This process involves multiple functions in programmatic, organizational, and operational levels.
Program level risk management is the responsibility of program managers and handles risks that are common to organization, clusters of projects, programs or entire business units. Project level risk management is the responsibility of project managers and considers risks that are specific to individual projects. Project risk management is conducted under the Division of Project Management at headquarters. Caltrans defines their project risk management process an integrated component of their standard project development process. They have a project risk management handbook that presents a scalable approach for risk management based on size and complexity of projects. Their risk management process is applicable to all projects and the level of risk management is determined primarily by total cost of the project. In addition to total costs, other factors and considerations such as political sensitivity, project type, location of the project,
53

duration of the project, stakeholders, and the sponsor's sensitivity to the primary objective of the project may lead to utilizing a higher scalability level.
The following project types typically go through the risk analysis in California:
Road-Rehab/Reconstruct Projects Road-Resurface/Renewal Projects Interchange-Construct/Improve/Modify Projects Managed Lanes-Construct-Modify Projects Bypass Projects Bridge and Tunnel Projects ITS (Intelligent Transportation systems) Projects Grade Separation Projects
3.3.2. Risk Management Process The risk management process in Caltrans is implemented during the entire project life cycle from project inception to completion of construction. However, the most important components of the process are during long-range planning and programming, preliminary design and environmental studies, right of way acquisition, and final design stages. Project risk manager who has been trained in the process directs the project risk management team. Typically, the risk management team consists of Caltrans project personnel from design, construction, project management, and functional units involved in the project. Representatives from other agencies may be invited to participate at risk management team meetings as well. Generally, the project manager acts as the project risk manager for projects with total estimated cost of less than $100 million dollars.
54

The risk management process of Caltrans categorizes the projects into three scalability levels. The first level is projects with estimated costs of less than $5 million and approximately consists of 67% of the projects of Caltrans. The second level is projects with the estimated costs from $5 million to $100 million and covers around 30% of the projects. The third level is projects with estimated costs greater than $100 million and comprises only 3% of the projects in Caltrans. Risk identification process is utilized for all three levels. However, the minimum risk analysis requirements (i.e. qualitative and quantitative analysis processes) differ for these three levels. For level 1 projects, a simple qualitative analysis is conducted that only rates the identified risks based on their overall importance. For level 2 projects, a probability/impact matrix analysis is conducted to assess the identified risks. For level 3 projects, risks should be analyzed quantitatively.
3.3.2.1. Risk management plan The risk management process begins with determining the scalability level for the project. Then, the risk register format is chosen based on the level. The frequency of risk management meetings for the project and the checkpoints are determined by project manager. Furthermore, project manager decides who will be on the project risk management team. The outcomes of these steps can be gathered in the risk management plan that defines the risk management level, frequency of meetings, project risk management team members, and the budget for the risk management activities. A written risk management plan is not required for all projects. Based on project size and complexity, the project manager decides if it is necessary.
3.3.2.2. Risk identification The first responsibility of project risk management team is to identify and assess risks. Caltrans project risk management handbook emphasizes that team members should be careful to avoid
55

any confusion between cause of risks, genuine risks, and the effects of risks. To identify the risks, the team members may use any combination of brainstorming, challenging of assumptions, looking for newness (e.g. new technologies and materials), developing risk checklists, consultation with others who have significant knowledge about the project or similar projects, organizing structured interviews with project participants or project stakeholders in the organization, and conducting structured risk identification workshops. Caltrans may use an external professional facilitator from the consulting world for organizing and leading the risk identification workshops for major projects (level 3 projects) that require quantitative analysis. Typically, experts from estimation, project management, maintenance, traffic operations, railroad, materials, construction, design policy and support, bridge design, roadway design, environmental, and engineering offices participate in the risk management workshop. Furthermore, stakeholders from metropolitan planning organizations, FHWA, and district offices may attend the workshop as well. Caltrans risk management handbook includes a list of typical risks from previous Caltrans projects. The risk list is categorized into environmental, external, design, engineering services, right of way, construction, project management, and organizational risks. For example, Table 3-2 shows the sample identified risks for environmental category. However, this list is for guidance only and is not a substitute for other methods of risk identification. At the end of risk identification step, each risk is assigned to a member of the risk management team who becomes its risk owner.
56

Table 3-2: Sample Risks for Environmental Issues

Risks

1

Environmental analysis incomplete

2

Availability of project data and mapping at the beginning of the environmental study is insufficient

3

New information after Environmental Document is completed may require re-evaluation or a new

document (i.e. utility relocation beyond document coverage)

4

New alternatives required to avoid, mitigate or minimize impact

5

Acquisition, creation or restoration of on or off-site mitigation

6

Environmental clearance for staging or borrow sites required

7

Historic site, endangered species, riparian areas, wetlands and/or public park present

8

Design changes require additional Environmental analysis

9

Unforeseen formal NEPA/404 consultation is required

10

Unforeseen formal Section 7 consultation is required

11

Unexpected Native American concerns

12

Project may encroach into the Coastal Zone

13

Project may encroach into a Scenic Highway

14

Project may encroach to a Wild and Scenic River

15

Unanticipated noise impacts

16

Project causes an unanticipated barrier to wildlife

17

Project may encroach into a floodplain or a regulatory floodway

18

Project does not conform to the state implementation plan for air quality at the program and plan level

19

Unanticipated cumulative impact issues

3.3.2.3 Risk analysis Qualitative analysis for level 1 projects: For level 1 projects, the qualitative risk analysis process assigns a risk rating to each identified risk. The ratings can be high, medium, or low. High risks are first priority for risk response. For medium risks, risk responses should be provided as time and resources permit. For low risks, no risk response is required at this (current) time. This process help Caltrans improve project performance by focusing on high-priority risks.
Qualitative analysis for level 2 projects: Qualitative risk analysis for level 2 projects prioritizes the identified risks based on their probability of occurring and the corresponding impact on project objectives if the risks occur. Caltrans defines five ratings (i.e. very low, low, moderate, high, and very high) for risk probability and corresponding impacts on cost and schedule of the project. Table 3-3 shows the definition of impact and probability ratings. Cost impact is based on the sum of Capital Outlay (CO) and Capital Outlay Support (COS) costs.
57

Table 3-3: Definitions of Impact and Probability Ratings from Caltrans (2012)

Rating Cost Impact of Threat Cost Impact of Opportunity Schedule Impact of Threat Schedule Impact of Opportunity Probability

Very Low Insignificant cost increase Insignificant cost reduction Insignificant slippage
Insignificant improvement
1-9%

Low <5% cost increase <1% cost decrease <1 month slippage
<1 month improvement
10-19%

Moderate 5-10% cost increase 1-3% cost decrease 1-3 months slippage
1-2 months improvement
20-39%

High 10-20% cost increase 3-5% cost decrease 3-6 months slippage
2-3 months improvement
40-59%

Very High >20% cost increase >5% cost decrease >6 months slippage
>3 months improvement
60-99%

After determining the ratings, a risk matrix (Figure 3-2) is used to determine the overall importance of each risk based on the combination of probability and impacts. For each rating level (i.e. very low to very high) of probability and impact, a standard number is associated. The product of the probability number and the impact number defines the risk score. For a particular risk, the combination of probability and impact positions the risk into one of the three colored zones in the matrix.

Figure 3-2: Caltrans Risk Matrix 58

Quantitative risk analysis for level 3 projects: Quantitative analysis is conducted for level 3 projects. Typically, Caltrans utilizes three-points estimate to quantify the degree of uncertainty in cost and duration of each activity and uses Monte Carlo simulation to produce a probability distribution of possible completion dates and project costs. Several software such as @Risk, Crystal Ball, and Primavera are used by Caltrans to conduct the quantitative risk analysis. The results of quantitative analysis help risk management team determine contingency reserves of time and money to provide a sufficient degree of confidence. Figure 3-3 shows an example of risk cost probability distribution developed during the quantitative risk analysis. This graph shows the curves for the current and previous assessment and indicates, for example, that there is a 90% chance that the risk cost is greater than $144 million in the current assessment. However, this number was $164 million in the previous assessment.
59

Figure 3-3: Risk Cost Probability Distribution 3.3.2.4 Risk response Following identifying and analyzing the risks, the project risk management team determines which risks require a response and identifies the best strategies for each risk. For each particular risk, its risk owner identifies various options to reduce the probability or impacts of that risk. Then, the team selects the best options for the project and assigns the required actions to the risk owner to execute the selected responses. The risk owner takes the lead but he/she may involve other experts to have the responses implemented and documented.
3.3.2.5 Risk monitoring and controlling The risk management team meets during a project based on the project risk management plan and predetermined check points. They periodically review the risk register and risk response actions to update the project risk information. During this process, they identify, analyze, and
60

plan actions for newly arising risks, review the execution of risk response actions and evaluate their effectiveness, assign additional risk response actions to risk owners, and retire risks whose significance have elapsed.
For level 1 and level 2 projects, the risk register with a cover sheet will serve as the risk communication medium and the project manager submits it at each checkpoint. The cover sheet summarizes the changes to the risk register since the previous communication. Level 3 projects require a more detailed report that includes the probability curves and their relation to project objectives. In addition to the communication meetings and checkpoints, project manager schedules accountability checkpoints and meetings to review the project and its risks by deputies to ensure that the project risk management team has managed the risks acceptably.
3.3.3. Lessons Learned and Challenges Although Caltrans does not have any systematic approach to capture lessons learned from conducting risk analysis on different projects, the project risk management team reviews the history of the retired risks to record any lessons learned regarding the implemented risk management processes. The project risk manager conducts a periodic review of all lessons learned with the risk management team members to capture knowledge as the project moves forward. Caltrans implements specific policy and training sessions to establish a culture of risk management for enhancing project delivery. They do not have a specific performance metrics to measure the success of the risk management program. However, they review the change orders based on the risk register. This can help them to use change orders for cost-benefit analysis of risk assessment efforts as change orders refer to the issues that may have been documented as the project risks in the risk assessment efforts. So better risk identification and more appropriate risk
61

assessment mean less issues as far as change orders and can be translated into time and cost savings for the project.
"Regarding the actual benefits of risk management, we are not yet there. For funding, they require risk register. Change orders are based on what was in the risk register. However, there is no database in which the cost benefits of risk analysis are recorded." Based on the results of the survey with risk management experts in Caltrans, the most important challenges for a successful implementation of risk management process in Caltrans are as follow:
Issues with the risk management tools Lack of risk culture Lack of communication among offices Lack of desire to use new procurement methods Inefficient organizational frameworks Inefficient coordination and communication between the agency and other local, state,
and federal government entities Also, the following factors are the secondary important barriers for successful execution of risk management process:
Lack of support from the top Inefficient risk allocation Poor prospects for economic growth
62

3.4. UTAH DEPARTMENT OF TRANSPORTATION
3.4.1. Risk Management Organization The main purpose of the Utah Department of Transportation to implement project risk management is to "control costs, increase efficiency, and reduce risk exposure throughout the Project Development life-cycle."2 One of the interviewees mentioned that:
"Because of a law suit, the Utah DOT decided to begin a risk management program. To do so, we invited WSDOT to help in the development of this program."
Although Utah Department of Transportation did not dedicate an office for the project risk management, it defined a role as a risk manager, whose "primary job is to assist project teams in developing risk registers, managing risks, etc." Moreover, Utah DOT has integrated the risk management procedure into the project development process, as it is mentioned by an interviewee:
"We have a Project Development Network (PDN) which defines the steps necessary to deliver a project. A project level risk analysis is required on all projects in this process."
Furthermore, this DOT has developed a document explaining the risk management practices, and a series of videos explaining the process and tools3:
2 http://www.udot.utah.gov/main/f?p=100:pg:0:::1:T,V:3164 from Instructions for Using the Qualitative Risk Worksheet (Word) accessed July 1, 2015 3 http://www.udot.utah.gov/main/f?p=100:pg:0:::1:T,V:3164 accessed July 1, 2015
63

"We have a document explaining how to use one of the tools that we have developed. We are in the process of developing a series of videos explaining the process and tools."
Unlike the project risk management, the Utah Department of Transportation devoted a separate enterprise-wide office for handling safety risk and dealing with insurance. One of the roles of this office is to be involved in insurance analysis during the pre-construction. Moreover, this office pre-qualifies contractors for safety using a scoring system. They score the contractors in five different areas such as E-mod, OSHA citation, OSHA repeat violation, National Incident Rate and Fatality. The contractors should receive at list 60 or 70 points in order to be considered for a job.
In the Utah Department of Transportation, depending on the size of the project, the risk management process can be conducted by a consultant, risk manager, or a project manager. As it is mentioned by an interviewee:
"Risk management process can be conducted by any one of the three-risk manager, project manager, or a consultant, depending on the complexity of the project. On the very large projects, we will hire a consultant to lead the process. Risk manager has been conducting the meetings for small and medium projects but he/she has been training the project managers to conduct their own meetings for the small jobs." In the Utah DOT, the process of risk analysis usually begins in the Concept Development stage and continues in the Preliminary Design and Environmental studies stage, and in the Final Design phase:
64

"Risk manager's guidance to the project managers is to conduct the meeting as soon in the development process as possible. Usually, it occurs at scoping. On large environmental documents, we will conduct a workshop to identify risks in the environmental process and a second workshop after the document is ready to finalize where we identify design and construction risks. Of course, the risk register is a living document that should be updated regularly throughout the development process." Utah DOT conducts project risk management almost for every project as a respondent mentioned:
"We require some form of risk analysis on all projects. Are we there yet? No. But at this time, the majority of our projects are doing it. Some of the smaller projects (e.g. pavement preservation projects) aren't there yet."
Utah DOT identified the following project types as the most frequent and critical candidates for the risk management process:
Road-Rehab/Reconstruct Projects Road-Resurface/Renewal Projects Interchange-Construct/Improve/Modify Projects Managed Lanes-Construct-Modify Projects Bypass Projects Bridge and Tunnel Projects ITS projects Grade Separation Projects
65

Finally, to identify what level of risk analysis is appropriate for a particular project, Utah DOT has developed a decision tree, illustrated in Figure 3-4.

No
Conduct a risk identification workshop at the beginning of the Environmental process using the
"QualitativeRiskWorksheet". Develop a risk register identifying risks in the
Environmental Phase, the Design Phase, and the Construction Phase.
Develop Mitigation strategies
Revisit and update the Risk Registry on a Regular basis
(recommend bi-weekly). Update the status of each risk
and confirm that the probabilities, impacts, and owner are still correct. Review the risk triggers and determine if any of
them have occurred. Identify any new risks.
Review the Mitigation strategies. Are they still appropriate or do they need to be revised? Are the strategies working?
Continue monitoring risks throughout the Environmental,
Design, and Construction Phases.

Total PV $20M?

No

Total PV $100M?

Yes

Env. Doc. Complete?

Start

Yes

Yes

No

Conduct a risk identification workshop at the beginning of the Environmental process using the
CRA or CRAVE method. Develop a comprehensive risk register identifying risks in the
Environmental Phase, the Design Phase, and the Construction Phase.

Conduct a rigorous risk identification workshop at the beginning of the Environmental
process using the CEVP method. Develop a
comprehensive risk register identifying risks in the
Environmental Phase, the Design Phase, and the Construction Phase.

Yes

EA or EIS?

No

Yes

State ES?

No

Develop Mitigation strategies

Develop Mitigation strategies Yes Yes

New Alignment, widening, env. Sensitive area?

Revisit and update the Risk

Revisit and update the Risk

Registry on a Regular basis

Registry on a Regular basis

(approximately every month or at

(approximately every two months

No

Milestones).

or at Milestones).

Update the status of each risk

Update the status of each risk

No

and confirm that the probabilities, impacts, and owner are still correct. Review the risk triggers and determine if any of

and confirm that the probabilities, impacts, and owner are still correct. Review the risk triggers and determine if any of

Any potential for

Yes

controversy (Location,

political, etc)?

them have occurred. Identify

them have occurred. Identify

any new risks. Re-run the Monte

any new risks. Re-run the Monte

Carlo model and examine the

Carlo model and examine the

No

output for risk elements that

output for risk elements that

have increased in potential

have increased in potential

impact.

impact.

Review the Mitigation strategies. Are they still appropriate or do they need to be revised? Are the strategies working?
Compare the probability curves ("S"curves) for schedule and cost to the estimated cost and
the committed advertising date.

Review the Mitigation strategies. Are they still appropriate or do they need to be revised? Are the strategies working?
Compare the probability curves ("S"curves) for schedule and cost to the estimated cost and
the committed advertising date.

Technical Risk factors (Urban construction, full depth pavement reconstruction, significant underground
work, bridges, complex MOT, restrictive Limitations)?

Figure 3-4: Decision Tree for Identifying Appropriate Level of Risk Analysis4

3.4.2. Risk Management Process 3.4.2.1. Risk identification The first step of the risk management process is risk identification, in which the project team predicts the potential surprises that might appear during the project life cycle. There are several techniques that can be employed in the risk identification phase. Utah DOT employs
4 http://www.udot.utah.gov/main/f?p=100:pg:0:::1:T,V:3164 from Downloads, Instructions for Using the Qualitative Risk Worksheet (Word), accessed July 1, 2015
66

brainstorming sessions, and the Delphi method
to identify project risks. The brainstorming sessions are facilitated by the risk manager, project manager, or a consultant depending on the size of the project. In the brainstorming sessions, representatives from engineering, environmental analysis & permitting, roadway design, bridge design, geotechnics, construction, materials, safety, right of way, utilities, railroad, traffic operations, maintenance, project management, and estimation offices are usually among the participants. Moreover, different stakeholders such as district offices, engineering consulting firms, and highway contractors might be invited to attend these sessions.
3.4.2.2. Risk assessment Using the results from the risk identification sessions, the project team employs qualitative or quantitative methods to analyze risks. As it is mentioned by a respondent, spreadsheets and @risk are the major tools for risk analysis at Utah DOT:
"We use two different spreadsheets for in-house workshops. If we hire a consultant to perform the work, they use whatever they want. Typically, we've seen @risk."
Similar to the Washington State DOT, the Utah Department of Transportation uses Cost Estimate and Validation Process (CEVP) (developed by Washington State DOT) for large, complex projects. In the CEVP workshop, several experts gather to identify, and assess potential risks using Monte Carlo simulations (WSDOT, 2014). For smaller projects, the Utah DOT performs Cost Risk Analysis (CRA), which can be performed in-house at a reasonable cost. Utah DOT might combine the CRA with Value Engineering (VE) in a process called Cost Risk Analysis
67

and Value Engineering (CRAVE). For example, CRAVE has been used for the SR-108, Antelope Drive to 1900 W project5. As it is mentioned in their report, the "primary objective of the CRAVE study was to:
Conduct a thorough review and analysis of the key project issues and conceptual design using a multi-discipline, cross-functional team
Identify high risk areas in delivering this project Perform a cost risk assessment on both the baseline design and the VE recommended
design Review project estimate Investigate ways to construct project with the least amount of construction impacts
("Constructability")."
For projects below the $20M, the Utah DOT uses qualitative methods such as heat maps. Figure 3-5 illustrates a screenshot of the excel tool that the Utah DOT employs for qualitative risk analysis. As it is illustrated in Figure 3-5, each risk has an ID, status, and description. Also, the project team should identify the probability that a risk might appear, and its cost impact. For each risk, the heat map illustrates how severe the risk is. The project team should as well identify what actions they will take if a risk appears (risk response).
5 http://www.udot.utah.gov/main/uconowner.gf?n=21892816727365033 accessed July 1, 2015
68

Figure 3-5: Utah Qualitative Risk Analysis Tool6
6 http://www.udot.utah.gov/main/f?p=100:pg:0:::1:T,V:3164, from UDOT Qualitative Risk Worksheet (Excel), accessed July 1, 2015
69

3.4.2.3. Evaluation and control Utah DOT has different mechanisms to evaluate and control the risks. First, to share and capture the obtained knowledge, they generated a guidebook. Moreover, during the project, the project team conducts "Project risk response review regularly to capture the changes in risks and their prioritization."
3.4.2.4. Risk response The Utah DOT considers four different strategies for risk response: Avoid, Transfer, Mitigate, or Accept. When it is possible, the project team should avoid the identified risk, for example, if the particular risk concerned wetland impacts, re-aligning the road to avoid the wetlands would be an avoidance strategy. In the cases where the contractors or other parties are more equipped to handle a risk, the project team might want to transfer the risk to that party by using contract terms. If the risk could not be avoided completely or transferred to other parties, the project team should consider different engineering techniques to mitigate that risk. For example, by conducting early environmental studies, the project team can mitigate the probability of environmental approval delays.
3.4.3. Lessons Learned and Challenges The Utah DOT considers the following as the most important challenges in the project risk management:
Lack of communication among offices Lack of staff or resources for complex tasks Issues with the risk management tools Lack of training of personnel
70

Lack of risk culture Lack of best practices and available training Inefficient organizational frameworks The Utah Department of Transportation developed a set of training videos for project managers. These videos can help to improve the risk culture in this department. As a respondent mentioned:
"This [training video] is in addition to Instruction Manual, and the two Excel tools for qualitative and quantitative risk assessment that are posted on the UDOT website. This is a great way to easily access all information about the risk management tools in one place."
Moreover, Utah DOT uses excel spreadsheets for both of its qualitative and quantitate risk analysis without a need to purchase expensive tools and software. Finally, the quality of risk management plan has a significant role in the selection of contractor (best-value selection) of the design-build team and is part of pre-construction management services for Construction Manager/General Contractor (CM/GC) projects:
"For CM/GC projects, UDOT considers risk analysis as a requirement of the projects. For these projects, they do the risk analysis when the contractor comes on-board. In the Design-Build (DB) projects, risk management and mitigation strategies are considered as selection criteria. That is, to select a DesignBuilder, UDOT evaluates the contractors' strategies for mitigating the potential risks."
71

3.5. NEW YORK STATE DEPARTMENT OF TRANSPORTATION
3.5.1. Risk Management Organization New York State Department of Transportation defines risk management as "the intentional, systematic process of planning for, identifying, analyzing, responding to, monitoring and controlling project related risks 7 ." This agency considers risk management as a tool for achieving project outcome within the defined cost, schedule, and scope.
The New York State Department of Transportation did not allocate a separate office for project risk management, but they have integrated this process into the project development process. This process has been incorporated in an appendix of project development guidebook, and the design build procedure manual. The purpose of the risk management appendix is to define the risk management principles, risk management process as it is applied to the project, roles and responsibilities, and examples of the risk management implementation.
The New York State Department of Transportation conducts risk management only for DesignBuild delivered projects and some selected major projects. For these projects, they start the process of risk management in the concept development and continue it in the preliminary design, environmental studies, final design, and right-of-way acquisition phases. The main person responsible for this process is the project manager that should call for risk management meetings, and work to identify and analyze project risks.
7 https://www.dot.ny.gov/divisions/engineering/design/dqab/dqabrepository/Risk%20Management%20for%20Project%20Development%20Guide_Final%20Draft_010514_0.pdf, accessed July 1, 2015
72

The New York guidebook of risk management requires every project to go through the risk management process:
"Risk management including a risk management plan should be incorporated into every project. The extent of the plan should be consistent with the complexity of the project, e.g. a simple project may only require a strategy to be developed and a list of red flag items, whereas a complex project will most likely warrant an in-depth plan."
However, in practice, only more complex, large projects and Design-Build projects have been evaluated for their risks: "Our risk manual requires a risk analysis be completed for every project, but in practice, this is not the case." This department defines complex project as projects with extensive public and outside agency involvement, projects with environmental issues, all projects that require Environmental Assessment (EA) based on The National Environmental Policy Act (NEPA), and all projects that require Environmental Impact Statement (EIS).
3.5.2. Risk Management Process 3.5.2.1. Risk identification The first step of the risk management process is risk identification, in which the project team predicts the potential surprises that might appear during the project life cycle. There are several techniques that can be employed in the risk identification phase. New York State DOT employs:
brainstorming sessions structured risk identification workshops risk checklists Delphi method
73

structured interviews with project participants questionnaires and conducting surveys Risk Breakdown Structure (RBS)

to identify project risks. The brainstorming sessions are facilitated by project manager with the exception of mega projects that are facilitated by a consultant. In the brainstorming sessions, representatives from engineering, environmental analysis & permitting, roadway design, bridge design, geotechnical, construction, materials, right of way, utilities, and project management are usually among the participants. Moreover, different stakeholders such as district offices, FHWA representatives, and consultant companies might be invited to attend in these sessions.

To simplify the process of risk identification, the New York State DOT developed a risk register as an initial checklist for risk identification. In this risk register, they divided risks into project development risks, external risks, environmental risks, project management risks, right-of-way risks, and construction risks. Table 3-4 illustrates the ROW risks in this risk register.

Table 3-4: Example of the Risk Register for ROW Risks from New York State DOT

Insufficient ROW available for all operations ROW Clearance not received in time for advertising Unanticipated need for public hearing due to ROW acquisition notdeemed"diminimus" Unforeseen railroad involvement Unanticipated escalation in right of way values or construction cost Need for "PermitstoEnter"notconsideredinprojectschedule development Condemnation process takes longer than anticipated Access to adjacent properties is necessary to resolve constructability requirements Acquisition of parcels controlled by a State or Federal Agency may take longer than anticipated Discovery of hazardous waste in the right of way phase Inadequate pool of qualified appraisers Landowners unwilling to sell

COST SCHEDULE



























74

Based on this risk register, the project team identifies the risks and inputs them in either a spreadsheet or the in-house software. Figure 3-6 illustrates a screenshot of the in-house risk identification tool.
Figure 3-6: Risk Identification Tool at New York State DOT 3.5.2.2. Risk assessment Using the results from the risk identification sessions, the project team employs qualitative or quantitative methods to analyze risks. As it is mentioned by a respondent, spreadsheets and a customized in-house software system are the major tools for risk analysis at New York State DOT. Selection of the risk analysis method is project specific and depends on the complexity of the project. As it is mentioned before, the complex projects are those with extensive public and outside agency involvement, with environmental issues, all NEPA EA projects, and all EIS
75

projects. For more complex projects, the project team might conduct Monte Carlo simulation or scenario analysis to evaluate the project risks. Figure 3-7 illustrates the in-house tool for risk analysis at New York State DOT. In this tool, the project team can identify the probability of a risk and its cost, schedule, and scope severity. Also, the project team should clarify the assumptions behind their selected values and reasons for changing their estimation.
Figure 3-7: Risk Analysis Tool at New York State DOT 3.5.2.3. Evaluation and control New York State Department of Transportation developed a manual for the risk management to assure a consistent risk management practice among projects. This agency as well provided trainings for Design-Build projects. This department is currently in the process of soliciting a consultant to update their risk manual, and to provide training. Finally, this department might
76

interview the experts or ask them to document their previous experiences to better identify and manage risks.
3.5.2.4. Risk response The New York State DOT considers the risk avoidance and mitigation plan as a part of the risk management plan. The risk avoidance and mitigation plan should identify what action should be done in what time frame in response to the identified risks. This plan should as well identify the responsible party for each risk.
3.5.3. Lessons Learned and Challenges The New York State DOT considers the following as the most important challenges in the project risk management:
Lack of staff or resources for complex tasks Overall lack of adequate funds Lack of risk culture Issues with the risk management tools Lack of sufficient internal infrastructure such as database Lack of training of personnel Inaccurate forecasts Inefficient organizational frameworks Lack of communication among offices Lack of desire to use new procurement methods Inefficient coordination and communication between the agency and other local, state, and federal government entities
77

Inefficient risk allocation Lack of best practices and available training New York State DOT has developed an In-house tool for the risk management process. This tool uses a database system to record the risks that are identified and analyzed in each project. This mechanism can help to better capture the previous experiences and to help the project managers in identifying risks for new projects.
78

3.6. MINNESOTA DEPARTMENT OF TRANSPORTATION
3.6.1. Risk Management Organization Minnesota Department of Transportation (MnDOT) currently does not have a specific office for project risk management. Risk management is not a formal and documented part of their standard project development process and they do not have a specific risk management guidebook. However, they consider risk management as an integrated component of project cost estimation and cost management. In 2008, MnDOT developed a Technical Reference Manual (TRM) for cost estimation and cost management during planning, scoping, design, and letting phases of project development. Risk management process was considered as part of this manual. Based on this manual, baseline cost estimates should include an initial assessment of risks and uncertainties. Typically, project managers are in charge of project risk management process. For large and major projects, they have had consultants in charge of helping the project managers through a risk workshop and a probabilistic risk analysis.
Based on the cost estimation manual, project risk management is required for all projects. However, the level of risk analysis varies due to the complexity of the project. Project complexity is not determined necessarily based on the project size. The general risk analysis process remains the same for all projects, but, the tools and level of effort vary with risk analysis level. Typically, road reconstruction, resurfacing, interchange, bridge, tunnel, and grade separation projects go through the extensive risk analysis process. During the interview, the interviewee provided a rough estimate of the total cost of projects with different level of risk analysis:
79

"Project size does not matter; however, we would like to implement a scalable risk management process. Projects under $5M will likely use more of a qualitative analysis and projects under $1M may need only a risk register. We anticipate that projects over $5M will require a quantitative analysis."
3.6.2. Risk Management Process The MnDOT project delivery process consists of five phases: planning, scoping, design, letting, and construction. The cost estimating manual focuses on the first four phases. The cost estimation framework requires a conceptual estimating during planning phase to estimate potential funds needed and prioritize needs for long range plans, scope estimating during scoping phase to establish a baseline cost for the project, design estimating during design phase to manage project budget against baseline, and finally plans specification estimating during letting phase to compare with bids and obligate funds for construction. The manual defined five management policies as follows:
1- Project Cost Estimation Policy 2- Uncertainty, Risk, and Contingency Policy 3- Communications Policy 4- Project Cost Management Policy 5- Program Management Policy
As noted above, the second policy deals with risks and uncertainties. This policy requires that the total project cost estimate for each project development phase includes a risk and uncertainty analysis. Then, project team estimates the needed contingency amount based on the risk analysis to be included in the total project cost estimate. A contingency estimate based on risk analysis is
80

required for all projects. However, the extent of risk analysis is determined based on the project complexity. Risk analysis during the all four phases (i.e. planning, scoping, design, and letting) consists of six steps as follows. However, the extent of analysis to address these steps vary in different phases based on the available information.
1- Review Risk Information 2- Determine Level of Risk Analysis 3- Identify Risks 4- Estimate Contingency 5- Document Risk and Contingency Basis 6- Prepare Total Project Cost Estimate The key inputs to conduct the above six steps are project definition assumption, estimating assumption and concerns, individual expertise, and project complexity categorization. The output of this process is a contingency estimate, a documentation of the risk and contingency basis, and the total project cost expressed in year of construction dollar.
3.6.2.1. Review risk information In this step, overall risk information is prepared for analysis in the following steps. The risk information consists of review of all estimating assumptions and project scope assumptions. Since in the planning phase very little details are available regarding the project definition, the estimator makes estimating assumptions in a planning level estimate. Likewise, the planners make initial project definition assumptions because of the limited information. These assumptions serve as triggers for risk identification. Risk checklist and risk analyses from similar
81

projects can be other sources for risk identification and should be included in the risk information document.
In the scoping phase, since the complete design information is not available, the estimator makes some assumptions. Also, functional groups make initial design assumptions at the conceptual level. Estimating and design assumptions should be gathered and considered for risk identification in the next steps. In the design phase, the risk documentations are updated using new information in the design estimate basis and base estimate package. Finally, in the letting phase when the design is 100 percent complete, the risk documentations are updated as estimating assumptions are minimal at that point.
3.6.2.2. Determine level of risk analysis All transportation projects in Minnesota require some form of risk analysis. However, the level of risk analysis differs based on the project complexity. The project complexity evaluation at MnDOT categorizes the project as minor, moderately complex, and major projects. For each level of complexity, a special risk analysis type is defined (i.e. type I risk analysis, type II risk analysis, and type III risk analysis).
Type I risk analysis is the simplest form of risk analysis and applies only to minor projects. A type I risk analysis involves a list of potential risks and the use of a percentage to estimate the contingency. The estimator determines the percentage contingency based on his/her experiences and judgement.
Type II risk analysis is a qualitative risk assessment and applies to moderately complex projects. In this process, a risk register containing a probability-impact matrix is developed to rank the identified risks.
82

Type III risk analysis is a quantitative risk analysis and applies to major projects. Typically, this analysis begins with a risk analysis workshop that is facilitated by consultants who are experts in quantitative risk management processes. The results of the interview revealed that typically experts from environmental analysis, roadway design, bridge design, geotechnical, construction, materials, traffic operations, maintenance, right of way, utilities, project management, and estimation participate in the workshop. Furthermore, stakeholders from district offices attend the workshop.
"Project management is held at the District Office level, so it is mainly District Office staff and some Central Office (support/specialty groups) that participate. We may include a local agency (city, county) if they have work/costs involved with the project."
The outcome of the type III risk analysis is a stochastic estimate of cost and schedule which are updated during the project development. MnDOT utilized Excel spreadsheets and Acumen for quantitative risk analysis. Also, their consultants typically use @Risk. At each phase, the process to determine the level of risk analysis is reconsidered and if a major change occurs, the type of risk analysis will be updated.
3.6.2.3. Identify risks The MnDOT cost estimating manual defines the risk identification as a creative brainstorming process. The outcomes of the first step, review risk information, including risk checklists, risk analyses from similar projects, and scope and estimation assumptions are used for risk identification in addition to some other tools and methods such as assumption analysis and expert interviews. Upon completion of the risk list, the identified risks are categorized into logical
83

grouping. The manual recommends establishing a standard risk breakdown structure to categorize the risks similarly across the department and develop a historical database of risks. Furthermore, the necessity of a work breakdown structure has been mentioned during the interview as well:
"Having the strong work break down structure and having detailed activities not only helps to have a good risk analysis, but more importantly, it generates a basis for asking questions, and a basis to think clearly about it."
The tools and techniques to identify the risks depend on the level of risk analysis. For type I risk analysis (i.e. minor projects), red flag item lists, risk checklists, and assumption analysis can be used. For type II risk analysis (i.e. moderately complex projects), experts interviews and Crawford slip methods can be utilized. For type III risk analysis (i.e. major projects), a risk workshop in conducted. The risk identification process is a continuous and repetitive process. The identified risks are reviewed and updated at each phase. New risks might arise and some previously identified risks might be retired. The interviewed professional from MnDOT believes that asking basic questions is the most important factor in a successful risk identification.
"Asking basic questions during the risk identification and risk management is critical. For example, sit down and ask, "Have we seen this risk before?" It is more important to slow down and think hard about the risks."
And eventually, a useful risk management process should lead to making decisions:
"The risk analysis process is only useful if it results in making different decisions. For example, the risk analysis tool showed that one of the projects cannot be completed within the base schedule. Based on these results, the
84

project manager could show that he needs more resources to finish the project on time."
3.6.2.4. Estimate contingency An appropriate contingency for the project is estimated in this step. This process varies based on the applied risk analysis type. For type I risk analysis (i.e. minor projects), the identified risks should be ranked qualitatively using probability-impact matrix. Then, estimator determines percentage contingencies from the allowable ranges.
For type II risk analysis (i.e. moderately complex projects), the identified risks should be ranked using probability-impact matrix. Then, similar to type I analysis, percentage contingencies are determined. Then, top 20 percent of the prioritized risks are reviewed to ensure that the contingency is enough. Also, Expected Value (EV) estimate is conducted for top-ranked risks based on the product of impact and probability. Finally, if warranted by the expected value analysis, additional contingency can be used.
For type III risk analysis (i.e. major projects), at first step, a type I or II analysis is performed. Then, a quantitative risk analysis is utilized to develop a stochastic estimate for cost and schedule. The results of the interview with a risk management expert from MnDOT revealed that they are considering other approaches such as Bayesian network and fault detection to improve their quantitative risk analysis. As risk register and analysis level are updated during the project development phases, the contingency estimate should be updated.
3.6.2.5. Document risk and contingency In this step, estimators and planners document the list of identified risks and uncertainties in the project estimate file and keep it for communication of the cost estimate. For minor projects, this
85

list is just a red flag list. For moderately complex and major projects, this documentation includes a comprehensive risk register consisting of a detailed description of the risks, their probability, their impact if they occur, strategies to manage the risks, risk owners, and a schedule for risk resolution. For major projects (i.e. type III risk analysis), a formal risk management plan is a requirement. Generally, the risk management plan consists of the risk management approach, responsibilities, budgeting, timing, reporting format, and tracking.
3.6.2.6. Prepare total project cost estimate Finally, the estimated contingency is added to the base cost estimate to develop the total project cost. In the planning phase, the total project cost should be expressed as a range. For type I and II risk analysis, this range is generated using a three-point range estimate. For type III risk analysis, this range is generated by a stochastic estimate model.
3.6.3. Lessons Learned and Challenges MnDOT does not have any systematic approach to capture lessons learned from conducting risk analysis on different projects. Furthermore, they do not have a standard process to evaluate the identified risks to check if they have occurred and if their assessed impacts were accurate. MnDOT has not developed specific performance metrics to measure the success of their risk management process.
In general, the results of the interview revealed that the following factors are the most important challenges to implement the risk management process successfully:
- Lack of training of personnel - Lack of existing policies - Lack of risk culture
86

Moreover, lack of staff or resources for complex tasks, lack of desire to use new procurement methods, inefficient risk allocation, and lack of best practices and available training are important barriers for a successful risk management program.
Although the 2008 technical reference manual for cost estimation and cost management is a great guidebook for cost estimation and managing the associated risks, MnDOT could not fully implement the proposed process. In 2013, MnDOT defined a research project to review the implementation and effectiveness of the manual. The University of Colorado Boulder (CU) and Parsons Brinkerhoff (PB) conducted the research (Molennar and Harper 2013). During the research project, they surveyed 104 experts, held a workshop with 28 MnDOT personnel, and conducted 10 focus interviews. The results of this study indicated that the department and its experts have an appropriate level of understanding about the importance of risk and risk management. However, there is a lack of consistency in the application.
"A common understanding of the importance of risk and risk management is pervasive throughout the department. However, the consensus of data showed inconsistencies and a lack of uniformity in the application and use of risk management. While the Technical Reference Manual (TRM) has specific instructions on how to link risks and contingency, there was not a clear understanding of the process with the survey or workshop participants. Likewise, it was noted that the risk management process should be more scalable. Few participants were aware of the scalable process provided in the TRM."
87

Also, they found that the most important barriers to implement the risk management successfully is related to establishing a project baseline and retiring risks.
"Perhaps the most significant implementation issue with risk management relates to establishing a project baseline and retiring risks. There is little consistency and guidance available for retiring risks and management contingency. Several interview comments stated that there is a lack of clarity in retiring risks, which makes it difficult to be consistent from district to district and even project to project."
During the interview with risk analysis expert at MnDOT, he mentioned about the inherent limitation of existing quantitative risk assessment methods. He believes that the actual benefit of the whole process is to have a formal platform to talk about the project issues and risks.
"Instead of risk registers and giving some estimates for the cost and schedule, it is much more important to sit and talk seriously about the risks. This is because people are poor in defining the probability, and are overestimating good outcomes, and are underestimating the bad results."
Also, the interviewee mentioned some innovative practices for implementing risk management successfully in the organization.
"To have headquarter people, district people, and different offices support and follow the risk analysis process, two mechanisms have been used:
1- It is suggested to add a thorough risk analysis to the agenda of monthly meetings of the transportation program investment committee, which decide about
88

adding money to projects, abandoning projects, or moving the projects to another year. 2- The department, rather than having dozens of goals or initiatives, decided to have only one- increasing financial effectiveness. This goal consisted of four parts: 1) having a better system of managing assets over their lifetime (looking at first cost, and also the operation costs), 2) tying works with activities better (differentiating the projects and services into useful buckets), 3) continuing to make significant changes in management system, and 4) telling the stories better to the stakeholders. (Documents talking about initiatives)."
89

3.7. LOUISIANA DEPARTMENT OF TRANSPORTATION
3.7.1. Risk Management Organization LA DOTD considers risk management process as part of their value engineering (VE) process. VE study is required for projects with a total estimated cost (including right of way, utilities, and construction) over $50 million and any bridge project over $40 million by FHWA and LA DOTD engineering directives and standards. The Value Engineering Director (VED) determines whether a project is a candidate for a formal VE study based on the scope and budget of the project. Then, after receiving the Chief Engineer's approval for the VE study to proceed, the VED identifies the appropriate sections of the project for participation. If a VE study is required for a project, the project manager may request a risk analysis as part of the VE study. The extent and level of risk management for each project depends on the size and complexity of the project. Project managers are responsible for risk management process and determine the level of the risk management needed for the project.
LA DOTD does not have an especial office or division for managing project risks. They rely mostly on consultants to perform risk assessment. However, they developed the first draft of their project risk management manual in 2008. The manual explains the desired risk management process at LA DOTD. However, the results of the survey with risk management expert at LA DOTD revealed that they do not have any formal mechanism that ensures them that the risks are being managed to the fullest extent.
"...Ideally, the Project Managers are responsible for managing these identified risks; however, we don't have any formal mechanism for ensuring that they
90

do. It is likely that these risks are not being managed to the fullest extent that they should be."
3.7.2. Risk Management Process Risk management process at LA DOTD begins with overall evaluation of the projects in terms of their potential uncertainties to determine if it is necessary to incorporate a risk management plan into the project. Projects are evaluated and rated to three levels: low, moderate, and high. Any project with rating of moderate or high should be considered for risk management process.
Risk management process in LA DOTD is conducted in the preliminary design and environmental studies stage and is continued at each stage of the project development process. Typically, major projects or FHWA projects go through the risk analysis process. These projects usually have estimated total costs of higher than $100 million. There is no restriction for implementing risk management based on the duration of the projects.
3.7.2.1 Risk management plan The first step of risk management process in LA DOTD is risk management planning that indicates how to approach, plan, and execute the risk management activities for a project. Risk management plan defines the role and responsibilities of the team members. Furthermore, the decisions regarding the level and type of analysis, frequency of risk evaluation checkpoints and meetings, and basis for an acceptable level of risk from project team and stakeholders are made in this step.
3.7.2.2 Risk identification Different methods based on information gathering techniques such as brainstorming sessions, Delphi technique, interviewing with experts, and root cause identification are utilized to identify
91

potential risks. Furthermore, other methods including checklist analysis (i.e. a risk checklist developed based on historical information and lessons learned from previous similar projects) and assumptions analysis (i.e. inspecting all possible hypothesis, scenarios, and assumptions to identify the potential risks) can be employed. LA DOTD risk management guidebook consists of a sample risk list that has been identified based on the past experiences. The risk list is categorized into design risks, external risks, environmental risks, organizational risks, project management risks, right of way risks, construction risks, and engineering services risks. Table 3-5 shows the sample risks of right of way.

Table 3-5: Sample List of Right of Way Risks

Risks

1

Utility relocation requires more time than planned

2

Unforeseen railroad involvement

3

Resolving objections to right of way appraisal takes more time and/or money

4

Right of Way datasheet incomplete or underestimated

5

Need for "Permits to Enter" not considered in project schedule development

6

Condemnation process takes longer than anticipated

7

Acquisition of parcels controlled by a State or Federal Agency may take longer than anticipated

8

Discovery of hazardous waste in the right of way phase

9

Seasonal requirements during utility relocation

10

Utility company workload, financial condition or timeline

11

Expired temporary construction easements

12

Inadequate pool of expert witnesses or qualified appraisers

It should be noted that when risk analysis is performed by a consultant, the risk identification approach might be different and depends upon the consultant performing the analysis. Consultants performing risk analysis may hold meetings with the design team as needed. Furthermore, for risk analysis performed during the VE studies, the team meets for a whole week in one of the headquarters meeting rooms. Typically, experts from engineering, roadway design, bridge design, construction, project management, and estimation offices participate in the risk management meetings. Also, stakeholders, especially from district offices and FHWA division
92

offices, may participate either as a part of the VE study where risk assessment is performed or as part of meetings with the consultant conducting the risk assessment for LA DOTD.

3.7.2.3. Risk analysis Quantitative Risk Analysis: After risk identification, the identified risks are prioritized based on their probability and impacts. The project team and stakeholders estimate and rank the probability and impacts on each project objective (i.e. time, cost, scope, and quality). All probabilities are ranked from 1 (i.e. lowest probability) to 5 (i.e. highest probability) subjectively. Table 3-6 shows the thresholds to rank the probabilities. Also, all expected impacts are ranked as very low, low, moderate, high, and very high subjectively. Then, the estimated probabilities and impacts are placed into a probability/impact matrix and use either linear or nonlinear impact scoring to calculate the overall risk scores.

Table 3-6: Risk Probability Ranking Thresholds

Ranking
5 4 3 2 1

Probability of Risk Event
80-90% 60-79% 40-59% 20-39% 1-19%

Linear impact scoring method shows a one to one correlation between the probability and impact of the risk. However, non-linear impact scoring gives a greater impact for risks that have smaller probabilities of occurring. Project manager decides to use a linear or non-linear method. Figure 3-8 and Figure 3-9 show the linear and non-linear impact scoring tables.

93

Probability
5 4 3 2 1

Threats or Opportunities

Very Low

Low

Moderate

High

5

10

20

40

4

8

16

32

3

6

12

24

2

4

8

16

1

2

4

8

1

2

4

8

Impact on Selected Objective

Figure 3-8: Non-linear Impact Scoring

Very High 80 64 48 32 16 16

Probability
5 4 3 2 1

Threats or Opportunities

Very Low

Low

Moderate

High

5

10

15

20

4

8

12

16

3

6

9

12

2

4

6

8

1

2

3

4

1

2

3

4

Impact on Selected Objective

Figure 3-9: Linear Impact Scoring

Very High 25 20 15 10 5 5

After calculating the risk scores, the risks are rated as low (score equal to or less than 6), moderate (score between 7 and 14), or high (score equal to or higher than 15).
Quantitative Risk Analysis: If a project involves a high level of risk, a quantitative risk analysis may be conducted. Typically, a quantitative risk analysis in LA DOTD involves modeling and Monte Carlo simulation to determine probabilistic estimates of cost and time. The output of the quantitative analysis is an updated risk register including prioritized list of quantified risks and probabilistic models of the project cost and time. Software may vary depending on the consultant performing the risk assessment. Also, LA DOTD has experts capable of performing Monte Carlo simulation using Excel spreadsheet macros. However, this is not used very often.

94

Furthermore, LA DOTD takes advantage from FHWA Cost Estimate Review (CER) process. They have implemented this process in several projects such as I-69 Section of Independent Utility (SIU) 15 projects. The primary purpose of this process is to conduct an unbiased riskbased review of the current total cost and schedule estimates to verify their accuracy and reasonableness and utilize a probabilistic approach to estimate the total cost as a range rather than a point value. Typically, the first review of the CER process is conducted during NEPA process and it is recommended that the first review be conducted 30 days before the completion of the NEPA document. For most major projects, cost estimates are once again reviewed along with the initial finance plan before the beginning of the construction phase (FHWA 2007). During CER process, subject matter experts are interviewed and their opinions regarding uncertainties in the project estimates such as base variability, inflation, market conditions, and risk events are modeled by the review team. Then, a Monte Carlo simulation is used to incorporate the uncertainties into forecast curves that represent a range estimate for cost and duration of the project. The risks are identified during the review process. Typically, binomial and triangular distributions are used to model the likelihood of occurrence and cost impact in the simulation, respectively.
3.7.2.4 Project risk response After analyzing the risks (qualitatively or quantitatively), the project team identifies possible strategies to deal with the risks rated high or moderate. At first step, project manager assigns an owner for each risk. The risk owner identifies the best options to respond to the risk.
95

3.7.2.5 Risk monitoring and controlling The identified risks are reviewed at regular, scheduled intervals. Project managers coordinate with the risk owners to monitor the risks and update the risk management process. During monitoring and controlling sessions, the experts update the status of the identified risks. Some of the risks might be eliminated and some new risks may arise. Furthermore, they analyze the residual risks that remain after implementing responses to the original risks. Secondary risks that may arise from the responses to the original risks are considered in this process as well. Also, interactions of two or more risks occurring simultaneously and that may create a greater effect are monitored.
The following tools and techniques might be utilized for risk monitoring and controlling at LA DOTD:
1- risk reassessment: new and old risks are reassessed at project team status meeting 2- risk audit: examine and document the effectiveness of risk responses 3- variance and trend analysis: analyzing the deviation from the baseline plan 4- reserve analysis: comparing the amount of contingency reserves remaining to the amount
of risk remaining to determine if the remaining reserve is adequate 5- status meeting: checking the status of the risks, eliminating risks that have been resolved,
and adding newly identified risks
3.7.3. Lessons Learned and Challenges LA DOTD risk management guidebook requires that lessons learned from the project risk management implementations be added to the organization's database. However, there is no systematic approach at LA DOTD to capture the lessons learned from conducting risk analysis.
96

Also, LA DOTD does not use a standard procedure to evaluate the risk management results and measure the performance of their risk management program. The results of the interview with cost estimate and value engineering director of LA DOTD revealed that having more experienced personnel to document procedures and lessons to capture risk management knowledge is the only method that has been experienced at LA DOTD.
"...I've been working with our Value Engineering (VE) Program to try and introduce a culture of risk management through our week long VE Studies. During some of the VE Studies, we identify risks and recommend strategies for mitigating identified risks and capture them in a Risk Register."
Moreover, the interviewee emphasized that the most important challenge for successful execution of risk analysis at LA DOTD is lack of staff or resources for complex tasks.
"...Lack of staff is a major hurdle. Many are acting as designer, project manager, and cost estimator all in one, not to mention some of their other duties... Ideally, the Project Managers are responsible for managing these identified risks; however, we don't have any formal mechanism for insuring that they do. It is likely that these risks are not being managed to the fullest extent that they should be due to the fact that many Project Managers wear multiple hats such as Designer, Cost Estimator, and etc. As a result, they don't feel that Risk Management is that high of a priority in light of all their other duties. Many Project Managers consider Risk Management and Value Engineering to be synonymous with what they are already doing as Project Managers; however, they don't realize that these are totally different and formal processes from what
97

they may be doing as Project Managers. This is a considerable obstacle to overcome." Other important barriers to implement the risk management process successfully at LA DOTD are issues with the risk management tools, overall lack of adequate funds, lack of training of personnel, inaccurate forecasts, lack of existing policies, lack of support from the top, lack of risk culture, lack of communication among offices, lack of best practices and available training, and lack of desire to use new procurement methods.
98

3.8. MISSOURI DEPARTMENT OF TRANSPORTATION
3.8.1. Risk Management Organization As it is stated in the Missouri Department of Transportation (MoDOT) risk management documentation, "Performing a project risk assessment is necessary to determine and document the most appropriate delivery method for a project. A thorough risk assessment allows MoDOT to clearly identify, prioritize and assign resources to risk avoidance and mitigation opportunities in order to help eliminate or reduce risk to the project. The resultant risks and an understanding of the efforts required to properly manage the risk provide the necessary perspective by which the appropriate project delivery method should be selected.8" MoDOT did not allocate a separate office for project risk management, and did not yet integrate the process of risk management in the project development process. However, this State DOT has developed documentation for the process of project risk assessment in which they briefly explain the goal and purpose of the process and the projects that should go through this process. In the MoDOT, project manager is responsible for conducting risk management process which is usually conducted in the Concept Development stage. In this State DOT, the project size and duration are not the definitive attributes for conducting risk management. MoDOT conducts project risk management only for the Design-Build projects irrespective of the size and duration of the project.
8 http://epg.modot.mo.gov/index.php?title=Category:149_Project_Delivery_Method_Determination_and_Risk_Assessment accessed July 1, 2015
99

3.8.2. Risk Management Process 3.8.2.1. Risk identification The first step of the risk management process is risk identification, in which the project team predicts the potential surprises that might appear during the project life cycle. There are several techniques that can be employed in the risk identification phase. MoDOT employs
structured risk identification workshops brainstorming sessions
to identify project risks. The brainstorming sessions are facilitated in-house without employing any consultant company. In the brainstorming sessions, representatives from communication, engineering, environmental analysis & permitting, roadway design, bridge design, geotechnical, design policy and support, construction, materials, safety, right of way, utilities, railroad, traffic operations, maintenance, project management, and estimation offices are usually among the participants. The major tool for conducting risk identification is spreadsheets, in which all the risks are recorded and ranked. The MoDOT risk assessment documentation 9 considers the following as the most important areas for risk identification:
"1. Drainage: Are there third party approvals necessary for drainage design? 2. Environmental: Are there environmental permits that MoDOT can obtain? How can the NEPA document allow for flexibility in the ultimate project solution? 3. Noise Walls: Can MoDOT agree with the public to a height, elevation, etc. of a noise wall or to a process to reach agreement on a noise wall?
9 http://epg.modot.mo.gov/index.php?title=Category:149_Project_Delivery_Method_Determination_and_Risk_Assessment accessed July 1, 2015
100

4. Method of Handling Traffic: Can MoDOT agree to detour routes with a public entity? 5. Public Information: Are there research efforts that can assist in formulating a public information plan or method of handling traffic plan? Are there key audiences that could derail the project? 6. Right of Way: Are there parcels that acquisition can be avoided? Can the amount of right of way acquired be minimized? 7. Roadway Design: Are there variances or exceptions that will be required? Is an Access Justification Report (AJR) required? 8. Structures: Are there approvals or variances that need to be obtained? 9. Third Party Agreements and Permits (other than environmental): Are there railroad agreements, process agreements, standards agreements that need to be obtained? 10. Utilities: What utilities are possible conflicts? Are there utilities with long relocation schedules? Should a Subsurface Utility Engineering (SUE) be pursued?"
3.8.2.2. Risk assessment Using the results from the risk identification sessions, the project team only employs qualitative methods to analyze risks. As it is mentioned by respondent, spreadsheets are the major tools for risk analysis at MoDOT. This department only uses risk heat maps to evaluate risks. Figure 3-10 illustrates the screenshot of the risk assessment spreadsheet used by MoDOT.
101

Figure 3-10: Risk Assessment Spreadsheet Used by Missouri DOT
3.8.2.3. Risk response The MoDOT considers four different strategies for risk response: Avoid, Transfer, Mitigate, or Accept. When it is possible, the project team should avoid the identified risk. In the cases where the contractors or other parties are more equipped to handle a risk, the project team might want to transfer the risk to that party by using contract terms. If the risk could not be avoided completely or transferred to other parties, the project team should consider different engineering techniques to mitigate that risk. For example, by conducting early environmental studies, the project team can mitigate the probability of environmental approval delays.
102

3.8.2.4. Evaluation and control MoDOT has developed documentation and standard spreadsheet for conducting risk assessment. They review the risk assessment process regularly as well: "Project risk response review regularly done by project team to capture the changes in risks and their prioritization." Finally, they developed a lessons learned database to capture the knowledge from previous projects. 3.8.3. Lessons Learned and Challenges The Missouri DOT considers the following as the most important challenges in the project risk management:
Lack of staff or resources for complex tasks Overall lack of adequate funds Issues with the risk management tools Lack of sufficient internal infrastructure such as database Lack of training of personnel Inaccurate forecasts
103

3.9. MONTANA DEPARTMENT OF TRANSPORTATION
3.9.1. Risk Management Organization As it is mentioned in the Montana department of transportation (MDT) project risk management guidebook, project risk management is an important step in efficient project delivery, and will contribute to public safety:
"Understanding project risks will better enable project teams in making decisions regarding project development and delivery. These decisions contribute to public safety and the projects we deliver add value to Montana on many levels."
Although MDT did not dedicate an office for the project risk management, this process is, to some extent, an integrated component of the standard project development process in this agency:
"The process of risk management is new and not fully integrated, but it has been in place for the higher risk projects."
Furthermore, this DOT has developed a guidebook (MDT, 2014) to provide the project managers and project teams a consistent methodology for conducting risk analysis. Depending on the size of the project, the risk management process can be conducted by a consultant or a project manager in MDT. In this department, the process of risk analysis is usually conducted at the preliminary design, environmental studies, final design, and right of way acquisition phases. The MDT risk management guidebook notes:
104

"Schedule risk assessments at appropriate times. Risk assessment should begin early, but there must be enough known about the project to understand what is being assessed. This will be to varying levels of detail depending on the point in project development at which the risk assessment is conducted (planning, scoping, or design)."

MDT mostly conducts project risk management for projects with the cost of greater than a million dollar, or those that last for longer than a year. However, they consider an informal process for low cost projects. Table 3-7 illustrates the thresholds for conducting risk analysis. For very complex projects, MDT conducts Cost Risk Assessment workshop, developed by WSDOT. If this process has done prior to the value engineering, its result is used in the value engineering process. The value engineering process might identify more risks that should be considered in the risk analysis. For every other project, the MDT uses the spreadsheets (shown in Figure 3-11) to conduct the risk analysis process.

Table 3-7: Thresholds for Conducting Risk Analysis (MDT, 2014)

Project Size < $1M <$20M >$20M Very Complex

Required Process Risk identification Qualitative risk analysis Quantitative risk analysis Cost Risk Assessment workshop

Level of Analysis Informal Informal Informal Formal

This department identified the following project types as the most frequent and critical candidates for the risk management process:
Road-Rehab/Reconstruct Projects Interchange-Construct/Improve/Modify Projects Bridge and Tunnel Projects
105

3.9.2. Risk Management Process 3.9.2.1. Risk identification The first step of the risk management process is risk identification, in which the project team predicts the potential surprises that might appear during the project life cycle. There are several techniques that can be employed in the risk identification phase. Montana DOT employs
brainstorming sessions structured risk identification workshops risk checklists scenario planning
to identify project risks. The brainstorming sessions are facilitated by the project manager. This agency does not use an external professional facilitator from the consulting world for organizing and leading risk identification workshops for projects. In the brainstorming sessions, representatives from engineering, environmental analysis & permitting, roadway design, bridge design, geotechnical, construction, materials, safety, right of way, utilities, railroad, traffic operations, and maintenance are usually among the participants. Moreover, different stakeholders such as FHWA Division Offices, Tribal Governments, and Engineering Consulting Firms might be invited to attend these sessions. One of the respondents mentioned:
"Tribal Governments haven't been involved to date, but they would be invited to participate in a project analysis for projects on tribal lands."
106

3.9.2.2. Risk assessment Using the results from the risk identification sessions, the project team employs qualitative or quantitative method to analyze risks. As it is mentioned by respondent, spreadsheets10 are the major tools for risk analysis at MDT. Figure 3-11 illustrates a screenshot of the spreadsheet used by the Montana DOT. This spreadsheet is used from both the qualitative and quantitative risk analysis. As it is illustrated in Figure 3-11, each risk has an ID, status, group (e.g. ROW, environmental, etc.) description, and trigger. Moreover, in case of quantitative risk evaluation, the team will identify the risk probability and cost impact. A heat map in the spreadsheet illustrates how severe a risk is. This tool has the capability to run Monte Carlo simulations for more advanced risk analysis.
3.9.2.3 Risk response After analyzing the risks (qualitatively or quantitatively), the project team identifies possible strategies to deal with the risks rated high or moderate. At first step, project manager assigns an owner for each risk. The risk owner identifies the best options to respond to the risk.
3.9.2.4. Evaluation and control Utah DOT has different mechanisms to evaluate and control the risks. First, to assure the consistent implementation of the risk management, they developed a guidebook and an excel spreadsheet. Moreover, during the project, the project team conducts "Project risk response review regularly to capture the changes in risks and their prioritization." Finally, as a respondent mentioned, this agency developed risk culture for enhancing project delivery:
10 http://www.mdt.mt.gov/business/contracting/cost.shtml under RMP Spreadsheet (Excel) Accessed July 1, 2015
107

"Yes, to a minor extent. Training workshops have been held and the message of guidance, tools, and training availability has been reiterated at different venues."
3.9.3. Lessons Learned and Challenges The Montana DOT considers the following as the most important challenges in the project risk management:
Lack of existing policies Lack of support from the top Lack of communication among offices Inefficient risk allocation Lack of best practices and available training Lack of staff or resources for complex tasks Lack of training of personnel Inaccurate forecasts Lack of risk culture Inefficient organizational frameworks Lack of desire to use new procurement methods Inefficient coordination and communication between the agency and other local, state,
and federal government entities Lack of sufficient internal infrastructure such as database
108

Figure 3-11: Risk Management Tool Screenshot11
11 http://www.mdt.mt.gov/business/contracting/cost.shtml From RMP spreadsheet Accessed July 1, 2015
109

3.10. MICHIGAN DEPARTMENT OF TRANSPORTATION
3.10.1. Risk Management Organization Michigan Department of Transportation (MDOT) is currently in the process of including project risk management in their bridge design process. This organization does not have a separate office devoted for project risk management, but they recently developed a draft of a guidebook for project risk management.
Michigan DOT only considers bridge and tunnel projects for the risk management process. This process is conducted by the project managers at the final design phase. In general, the size and the duration of the projects do not determine the need for this process. That is, any bridge and tunnel project should have go through the risk management procedure.
3.10.2. Risk Management Process 3.10.2.1. Risk identification and assessment The first step of the risk management process is risk identification, in which the project team predicts the potential surprises that might appear during the project life cycle. There are several techniques that can be employed in the risk identification phase. Michigan DOT employs
risk checklists
to identify project risks. This department has also developed a general risk register for the bridge and tunnel projects to simplify the process of risk identification and knowledge
110

sharing. The risk checklist is usually filled based on the project team brainstorming results and the general risk register.
To assess the identified risks, the project manager uses the qualitative methods such as heat maps. Spreadsheet is the major tool for risk identification and assessment at Michigan DOT.
3.10.2.2. Risk response After analyzing the risks (qualitatively or quantitatively), the project team identifies possible strategies to deal with the risks rated high or moderate. At first step, project manager assigns an owner for each risk. The risk owner identifies the best options to respond to the risk.
3.10.2.3. Evaluation and control Michigan DOT is in the process of developing a guidebook for risk management. This guidebook will help the project team to conduct the process in a consistent fashion. Other than this guidebook, the Michigan DOT has not developed any particular technique to control project risks.
3.10.3. Lessons Learned and Challenges The Michigan DOT considers the following as the most important challenges in the project risk management:
Lack of staff or resources for complex tasks Lack of training of personnel Lack of existing policies
111

Lack of risk culture Lack of best practices and available training
112

3.11. MASSACHUSETTS DEPARTMENT OF TRANSPORTATION
3.11.1. Risk Management Organization Office of project control and performance oversight at Massachusetts Department of Transportation (MassDOT) is responsible for managing project risks. Currently, MassDOT does not have a specific guidebook for risk management process. An internal expert in risk management is in charge of conducting project risk management process and an external professional facilitator from the consulting world may organize and lead the risk identification workshops and meetings.
3.11.2. Risk Management Process MassDOT begins risk analysis process in the stage of preliminary design and environmental studies. Typically, MassDOT conducts risk analysis for projects with the estimated cost of greater than $5 million and there is no limitation based on the duration of the projects. Most of the projects that go through the risk analysis process are bridge, tunnel, grade separation, and interchange projects.
3.11.2.1. Risk identification MassDOT develops and utilizes risk checklist to identify the potential risks for each project under risk analysis process. Typically, experts from the offices of finance and budgeting, project management, railroad, utilities, contract management, bidding administration, safety, construction, geotechnical, bridge design, roadway design, environmental analysis and permitting, and engineering participate in the risk management workshops to identify the risks. Furthermore, some stakeholders such as
113

FHWA division offices, district offices, and engineering consulting firms may participate in the workshops.
3.11.2.2. Risk analysis Typically, MassDOT conducts quantitative risk assessment for projects with total estimated costs of more than $15 million. They use Excel spreadsheets and Primavera risk analysis software for their risk analysis process.
3.11.3. Lessons Learned and Challenges MassDOT does not have any systematic approach to capture lessons learned from conducting risk analysis on different projects. Also, they do not evaluate the identified risks to check if they have occurred and if their assessed impacts were accurate. However, office of project control and performance oversight utilizes performance metrics to measure the success of the risk management process. Furthermore, they write final project reports and capture knowledge as the project moves forward to improve their knowledge about risk management.
The results of the survey revealed that the most important challenges to implement risk management process successfully are issues with the risk management tools, lack of sufficient internal infrastructure such as database, inaccurate forecasts, lack of existing policies, and lack of best practices and available training. Also, overall lack of adequate funds, lack of training of personnel, lack of communication among offices, inefficient organizational frameworks, lack of desire to use new procurement methods, inefficient coordination and communication between the agency and other local, state, and federal
114

government entities, and poor prospects for economic growth are important barriers for successful execution of their risk analysis.
115

3.12. RHODE ISLAND DEPARTMENT OF TRANSPORTATION
3.12.1. Risk Management Organization Rhode Island Department of Transportation (RIDOT) does not have an especial office or division for project risk management. Project risk management process is not an integrated component of their standard project development process and they have not developed a guidebook for project risk management process. Typically, project managers are in charge of conducting project risk management. They may hire consultants for major projects as well.
Typically, interchange, bypass, bridge, tunnels, and grade separation projects go through the risk analysis process. However, there is no specific standard threshold based on project size or duration to determine if the project is a candidate for conducting risk analysis.
3.12.2. Risk Management Process RIDOT conducts risk management analysis during preliminary design, environmental studies, and final design. RIDOT does not have a standard process to implement risk management and project managers decide about the extent of the analysis based on their experiences.
3.12.2.1. Risk identification RIDOT typically employs risk checklist for risk identification. They may hold a workshop for this process as well. Typically, experts from engineering, environmental
116

analysis, roadway design, construction, utilities, and project management participate in the risk management workshop. This workshop is facilitated by project manager and they do not use an external professional facilitator from the consulting world for organizing and leading the workshop. Furthermore, other stakeholders from engineering consulting firms might be required to attend the workshop.
3.12.2.2. Risk analysis RIDOT does not employ a formal risk assessment method. They do not have predetermined thresholds for qualitative or quantitative risk assessment.
3.12.2.3. Risk monitoring and controlling Since RIDOT does not use a standard process to conduct risk management, they do not have a specific process, methods, and tools to monitor and control the identified risks.
3.12.3. Lessons Learned and Challenges RIDOT does not have a systematic approach to capture lessons learned and evaluate the identified risks to check if they have occurred and if their assessed impacts were accurate. Furthermore, they have not developed a specific performance metrics to measure the success of the risk management program. RIDOT relies mainly on having more experienced personnel to captures risk management knowledge.
The results of the survey revealed that the following items are among the most important challenges to implement the risk management process successfully:
- Lack of sufficient internal infrastructure such as database - Overall lack of adequate funds
117

- Lack of existing policies - Lack of support from the top - Lack of risk culture - Lack of communication among offices - Inefficient coordination and communication between the agency and other local,
state, and federal government entities - Inefficient risk allocation Moreover, lack of staff or resources for complex tasks and issues with the risk management tools are important barriers to have an appropriate risk management process.
118

CHAPTER 4 ALIGNING PROGRAM-LEVEL AND PROJECT-
LEVEL OBJECTIVES FOR RISK MANAGEMENT: CASE STUDY OF GEORGIA
DEPARTMENT OF TRANSPORTATION
4.1. INTRODUCTION
In this chapter, we explore factors that influence risk management practices within the Georgia Department of Transportation (henceforth called `GDOT' or `the department'). An early observation on the relevant literature is that the definition of risk management varies depending on the level of management. At the project management level, risk is defined as "an uncertain event or condition that, if it occurs, has a positive or negative effect on at least one project objective" and risk management's objective is to "increase the probability and impact of positive events, and decrease the probability and impact of negative events on the project" using planning risk management, risk identification, quantitative analysis, qualitative analysis and planning response (Project Management Institute, 2013). At the level of an organization or a program, the scope is, "To successfully confront the effects of project risk, risk analysis must be applied with a broad view of risk; concentration on the technical risks can lead to oversights in other project dimensions." (Molenaar et al., 2010). As a result, risk management involves "risk mitigation and planning efforts [which] may require that agencies set policies, procedures, goals, and responsibility standards".
119

The literature typically focuses on these two levels for risk management that we will call project-level and program-level in this study. As the literature indicates, risk management is a process that involves project-level practices and program-level practices; however, the link between the two types of practices that are happening simultaneously within an organization is missing from the literature. The question of the alignment or at least the compatibility of risk management overall seems like an important condition for successful implementation of risk management. Therefore, in this study, we will focus on factors that influence risk management practices and analyze whether they are the same at the project-level and the program-level.
Concrete examples of risk management practices vary in scope and level in the academic literature and industry reports. The scope designates both the type of objective of the practice and whether it is aimed at improvement managing risk at a project level or program level. The types of objectives can include: improving the conditions for successful risk management, use of models and tools, use of formal or informal processes, etc. Perhaps the most comprehensive discussion of the conditions for successful risk management, including the identification of variables can be found in the literature review by the Network of Major European Cities (2013) and the Risk Analysis Tools Guidebook by Biehler et al. (2010). These factors have been identified in them as conditions for success in regard to successful policy implementation. Although there is a multitude of factors contributing to success, the factors most commonly discussed are: the use of scientific evidence, stakeholder inclusion, communications of risks, transparency and visibility, and political support. Also, FHWA (2012) enumerates external factors such as the economy, business environment, community issues, financial
120

environment, natural disasters, climate change, and environmental issues, among other contextual considerations. Agencies have to be able to address risks by taking advantage of the positive ones, mitigate the negative ones when it is possible, and when it is not, make decisions about the tolerability of some risks. To be able to make these decisions, agency risks can be classified by level of severity, likelihood, potential consequences, area of impact, period of impact and impact on agency's accomplishment of strategic objectives. These practices are related to the internal and mostly external environment from the point of view of a state DOT. But they also seek to deal with the main risk in terms of probability and impact.
The policy literature points out that these kinds of discussions implicitly assume a constant and sound reliable organizational structure that enables effective risk management if these measures are taken. However, according to Heimann (2010), this foundation is only reliable if the organizational and political leaders develop a sense of risk culture through practicing effective communication and actively making safety a high priority. Yet, many factors influence the development of this type of organization and the ability and motivations of leaders and stakeholders to accept, adopt, and abide by this common culture. These are the types of organizational and institutional factors the policy literature adds to the industry reports (FHWA, 2006; FHWA, 2012a; FHWA, 2012b).
Industry reports (FHWA, 2006; FHWA, 2012a; FHWA, 2012b) tackle other organizational and institutional aspects. They seek to have a formal structure and a set of processes to implement work management. One of the main recommendations given at the federal level is to formalize risk management. Risk management practices exist but
121

not many states have formal strategies and processes. According to the US DOT's Office of International Programs, a clear process for risk management includes the development and documentation of a clear strategy, the definition of methods to be used and planning of resources12.
Risk management is not expected to be comprehensive from the beginning and is expected to rely on experience. This process will therefore be iterative and will need to be adapted to each project. Also, the development of documented records is meant to help future risk management. Therefore, the federal recommendations include a formal form of knowledge management. On this topic, it is on the same page as the academic research literature that indicates that risk management should be sufficiently formalized to avoid relying entirely on employees' knowledge. Retirements, turnover and downsizing have negative impact on the knowledge assets of agencies. Therefore, learning and transmission of knowledge is an aspect that organizations need to focus on in risk management. In addition, Moynihan's report (2005) points out that the lack of resources and support can weaken the development of a managerial process. A process can be successfully institutionalized if there is an effort to foster it "not just through a series of planning and reporting procedures, but also through building a results-oriented organizational culture". The involvement of employees at all levels in planning and increasing their expectation of greater receptivity to their ideas can help generate concrete ideas in risk management and increase their sense of responsibility and organizational culture. Moynihan also found that "the continued existence of the old culture and rules" can make a shift difficult; it can generate frustration and ineffective managerial processes
12 http://international.fhwa.dot.gov/riskassess/risk_hcm06_05.cfm Accessed July 1, 2015.
122

will be unchanged. The author suggests that one solution is to communicate the information as much as possible to elected officials and senior management to make the decision-makers as involved in the issues as possible. This literature seems to concur with our concern about the alignment between program-level and project-level employees and also points out the influence of old culture and practices when implementing new solutions.
A review of the literature reveals different characteristics related to risk management. In this study, we choose to group these characteristics into eight relevant categories and propose a model to understand how they impact risk management. The analysis of evidence for this model takes into account the alignment between project-level and program-level employees as a potential factor that affects the success of risk management.
A case study is developed of GDOT in a process of reviewing and updating risk management practices. This study is an embedded case design of preconstruction design engineers and technical specialists working in the context of the plan development process. The simplified organizational structure of GDOT is the following. Project managers are responsible for delivery of projects and different stages of the project require different input. These inputs are obtained from employees in offices specialized in different fields such as: environmental issues, right of way, design, and so on. A third class of employees has a program-level role, which involves being detached from one particular project and looking with a holistic view at the portfolio of projects under development. Because of this organizational structure, we add a third class of employees to the analysis: office-level employees. We compare and contrast perspectives on the
123

implementation of risk management across the three classes of managers responsible for the program level, project level of operations and office level. The case study was developed as part of a larger agency project aimed at formalizing a comprehensive risk management approach for the Plan Development Process (PDP) and developing a register of the highest risk factors at each stage of the process. Semi-structured interviews were conducted with employees responsible for the three levels of operation. Each set of interviewees was asked to identify and describe the factors influencing risk management practices within the agency during critical steps in the PDP. We found that program-level considerations influence the decision to adopt technological and administrative solutions to improve project delivery but ultimately it is the responsibility and decision of project managers to use them. Also, we also found that individual offices within the agency have their own views of risk related to office operations and that these can often be at odds with views of risk management held by program level and project level managers. The analysis of interviews reveals that the consequence of these varying views of risk management lead to alignment gaps between the program level strategic objectives of project delivery and their actual impact after implementation at the project level.
124

4.2. MODEL AND CASE DESIGN
Our research was designed in the following four steps. First, we identified potential factors and used them to create the interview protocol. Second, we conducted interviews to get empirical evidence about the factors and the evolution and current situation of risk management. Third, using a preliminary analysis of the data, we inferred a model explaining the change from existing processes to new risk management practices with these factors. Fourth, we coded and analyzed the data to test the model.
4.2.1. Identification of Factors and Design of the Protocol In the first step, a list of risks potentially relevant to GDOT was identified based on our review of risk management practices in other states (see Chapter 3), a review of professional and academic literature, and discussions during preliminary meetings with employees with a role related to risk management in the department. We identified strategies and methods to identify, assess and mitigate risks and issues in project delivery. Based on the same sources, the protocol was designed to conduct semi-structured interviews. As a result, the protocol contains all the themes we identified as relevant for interviewing GDOT's personnel on how the department deals with risks that can impact project delivery. An initial list of interviewees was provided by the department and included employees with roles at different levels and offices. Some of the interviewees from the initial lists recommended other employees that we interviewed later on. We used one unique protocol for all of them.
125

The protocol covered a number of themes grouped into four sections. It was designed to let the interviewee focus relatively freely on the themes they deem important. In addition, rather than mentioning risks as the main topic, we asked about the threats for on-time delivery of transportation projects and solutions available to solve the issues. The questions in the first section of the protocol are related to: the interviewee's role, GDOT's current and past interest in risk related to project delivery and approach taken by the interviewee's office. The second section of questions is related to risk model selection process: who is involved in the choice of models, what models are or have been used and does risk management vary depending on classes of project. The third section deals with risk-related practices and deliberations: types of project and issues that have been determinant for changes in risk management practices, practices in risk assessment and risk analysis if they exist, and decision making on risks. The last section involves questions to the interviewee about expectations in terms of benefits and costs when changing risk management practices.
4.2.2. Interviews Using this protocol, we interviewed nine employees of the department in the second step of our research. Interviews followed the semi-structured protocol and lasted approximately 45 60 minutes each13. Figure 4-1 summarizes the key concepts explored with the interviewees.
13 All interviews were recorded and transcribed. Eight interviews were conducted between April 22 and April 29 2014, and one in July 2014. Each interviewee was asked at the end of the session if there was a person in GDOT that he/she would recommend us interviewing. As a result, we conducted one additional interview in July 2014.
126

Given the organizational structure of GDOT, we identified three categories of employees. Among the interviewees, two were project-level employees, three were office-level employees and four were program-level employees. Project-level employees are project managers or program managers, previously referred to as senior project managers. They are responsible for delivering projects and work with various offices in GDOT to accomplish that task. Office-level employees are people who work within an office and report to the head of the office or are head of an office themselves. They provide services and inputs necessary to deliver projects. Program-level employees are those who have roles with strategic objectives at a program or at the agency-level. 4.2.3. Preliminary Analysis and Model A preliminary analysis of transcripts of the interviews was done manually to identify whether factors listed in the first step are relevant according to the interviewees and how they relate not only to change from existing processes to new risk management but also to each other. Eight categories of information have been revealed by the data using this method: internal environment, external environment, main risks, typical projects, change factors, costs/benefits, existing project development process and new risk assessment. Using this preliminary analysis, we inferred the model in Figure 4-1.
127

Figure 4-1: Model of the Adoption of Risk Management Practices in GDOT
The model is offered as a hypothetical explanation of changes in the risk management approach and processes over the last few years. The key factors in the model are drawn from two of the main frameworks in organization theory and policy development, namely, contingency theory and institutionalism. These frameworks articulate ways in which both the internal and external environments might affect an organization and lead to changes in its structure and/or its key processes and routines. The model also takes into account the point of departure for the changes in risk management processes against which the needs statements and/or implementation of new approaches have occurred. Both the prior approaches and the roles and structure of the organization constrain the pathways of change that might follow. Finally, the model considers the specific content
128

of risk assessment in the process of project delivery to understand how it may contribute to the change of approaches in the context and is constrained by the previously mentioned factors.
The resulting model shows the change process from the prior conditions in the organization ("Existing project development process") to the final stage of a new risk management approach. Key elements of this change process are the motivating factors of main risks perceived by its members that continue to present problems and the characteristics of typical projects in which those critical risks tend to appear. In between, possible change factors for the organization are hypothesized, which may be both internal and external to the organizations and the moderating effect of strategic calculations of costs and benefits. The entire process of chain may be affected by a variety of aspects of the internal and external environment.
As represented by the dotted lines that go through the boxes for change factors and cost benefits, main risks and typical projects become change factors from prior experience and are taken into account for possible costs and benefits of new practices. The relationship between main risks and typical projects with change factors and costs/benefits will not be detailed separately in the findings section because it can be interpreted from the relationship between each of these categories with existing and new practices.
On the basis of this model, interview and document analysis protocols were developed to gather evidence and identify the specific content of each component of the model and the role they played in the observed change to organizational risk management. The
129

categories of the model in Figure 4-1 and the relationship between them are presented in more detail along with the empirical evidence in the following section. 4.2.4. Coding Procedure to Test the Model The preliminary analysis through manual coding revealed the previously listed eight categories. Higher details within the categories involve sub-categories; we will refer to them as themes. These themes are used to understand the content within categories and how categories relate to each other. An extensive list of themes has been coded to capture any connections that might not have been revealed by the preliminary analysis14. References are the specific parts of the text that are coded from the interview data. A count of references and sources for each theme is used to assess the importance of the themes and variance across interviewees. However, because a word, a sentence or a paragraph may be counted the same, it is an indicative figure and not a precise estimation of the relative importance.
14 To test the model, we coded all transcripts using NVivo software. Relevant parts of the text data are associated with one or more corresponding themes. Only the themes we found relevant are presented in this study. In addition, a relationship between themes, and therefore between the eight categories can be identified through an analysis of the network of themes in the transcripts.
130

4.3. FINDINGS
4.3.1. Categories of the Model The categories coded are summarized in Table 4-1.

Table 4-1: Coding Themes and Corresponding Categories of the Model

Model category
Change factors
Benefits and costs External environment New risk assessment Existing project development process Main risks Typical projects or problems
Internal environment

Description
Factors identified as having entailed some change in risk management. This can include events, influence, goals of different actors, strategy, etc. Benefits and costs expected from a change in risk management practice
External factors related to risk and project delivery
Current (new or unchanged) practices related to risk management and project delivery Past practices related to risk management and project delivery, that have been changed Main risks, using risks in a large sense: uncertainty about funding-time-scope, lack of control, potential impact, etc.
Project or issues that have had an impact on practices
Internal factors related to risk and project delivery. This can include characteristics of the environment at GDOT, knowledge management practices, organizational issues, roles, etc.

Coded sources 9
7 7 9 6 9 7
9

Coded references 97
24 38 173 15 166 19
155

Overall, change factors in risk management at GDOT are thought of mainly at the program and department level by the three categories of employees. All the interviewees identify economic downturn, reduced federal funding and decrease in budget as a reason for GDOT's interest in having more efficient risk management to reduce delays and costs. A program-level interviewee indicated that the decision to adopt certain tools such as Primavera is influenced by the industries' practices:
"Our construction office was more familiar with Primavera, the contracting industry pretty much uses that."

131

Another office-level finding is that tools enhancing efficiency are sometimes adopted at the office level and an interviewee who is head of an office assessed that it greatly improved the performance of their office.
The interviews reveal that employees consider different costs and expect different benefits depending on their role. Program-level employees expect that a better monitoring of information concerning time and budget and a better management of uncertainty at the project-level will aggregate into more accurate scheduling and savings at the program level. Program-level employees want to improve forecasting accuracy to manage portfolios of projects at the department level to fit the federal fiscal year's budget. One interviewee who has a role at the program-level expressed concerns about increasing requirements of monitoring and reporting having an impact on the role of project managers turning into schedulers if the new approach requires a lot of reporting of information and scheduling:
"So the question is: do you want your project managers turning into schedulers because you have included thousand activities? Which means they are monitoring all those and checking those off, they don't have time to actually do their work. So the trade-off is: you need to make it reasonable."
Office-level employees think of costs and benefits at the project and the office level, especially in terms of improvement of cooperation between the two. Project-level employees think of project-level improvement to deliver on time and within budget. Project-level employees' and program-level employees' agendas seem aligned but
132

concern about communication of project-level information to the program-level, which is critical for program-level forecasting, is missing from project-level employees' concerns.
The factors external to GDOT identified as having an influence on risk management are: the economic situation, federal legislation, the natural environment, other state DOTs, contractors, politics, public and state legislation. Interviewees from the three levels all seem aware of the impact of the economic situation that caused a reduction in federal funding and impacts GDOT but office-level and project-level employees see consequences in terms of administrative requirements to deliver a project:
Program-level interviewees think about the external environment mainly at the program level. Federal funding reduction is the main concern for them. They mention project-level impacts of change of federal legislation and procedures.
Office-level interviewees also think mainly at the program-level and view risk management as a result of adaptation to economic downturn and changes in funding: reduction of federal funding, reliance on a combination of federal and local funding resulting in changes in documents required.
Project level employees think of the external environment both at the program and the project level. The program-level impact of the external environment is also related to the economy and the federal funding in terms of budget and also in documents required for the projects.
In addition, for project-level employees, the external environment involves politics and public accountability for projects. The natural environment and cooperation with contractors impact them. Therefore, there is only a partial alignment on the external environment factors between employees from the three levels.
133

Overall, regarding the internal environment, employees from the three levels are concerned by the reduction of human resources capacities due to the reduction of the budget in the department. Knowledge management is now encouraged by the federal level regulations and discussions. It is motivated by the reduction of human resources and the fact that staff is getting younger and therefore less experienced. Especially, given the length of transportation projects, people who work on the projects are usually not the same over a period of 7 or 8 years. Currently, knowledge management is not standardized and exchange of information between office-level experts and project managers depend on meetings. Interviewees mention several forms of transmission of information:
Communicating between offices, face-to-face or by email, for project delivery Environmental forms required communicating back and forth with environmental
services Formal meetings such as the project team initiative plan (PTIP) are used Meetings between managers enable them to help each other solve problems in
their own projects and profit from others' experience. The internal environment involves considerations about project-level and program-level issues for program-level employees.
There is frustration for employees who do not see projects completed because of delays. A program-level employee highlights the need for a cultural shift to have people think of efficiency at the program level and expresses frustration about the fact that the incentives are not the same as in the private sector to meet milestones. In addition to this cultural
134

issue, program-level employees are concerned with organizational issues related to risk management. There is a difficulty in measuring current employees' performance and hiring the right new employees. Human resource management is gaining importance because of the constrained budget and need for efficiency. In that respect, GDOT as an organization has evolved. Projects managers were created and are in one division and the risk management office was created recently.
Office-level employees and project-level employees both think at the department, office and project levels when it comes to the internal environment. In practice, because the project managers and different offices have to interact, sometimes several times, during a project, people need to be proactive to keep the project on schedule according to the head of an office. The heterogeneity of experience and skills of project managers is impacting cooperation between offices and projects. Regarding knowledge management, an officelevel interviewee pointed out that employees don't necessarily know what, in their knowledge, is valuable to share. People in design keep project notes for themselves but there is no comprehensive collective knowledge repository.
Everyone does not perceive knowledge management as necessary and sufficient:
An office-level interviewee believes that employees learn by doing things rather than having in manuals, so documentation is useful but not enough.
Although training and information sharing practices are available (but not imposed), passing information and training involves additional work for employees. The reduced budget froze promotions, which does not incentivize employees to take on more work or responsibilities.
135

Another interviewee, a project manager, considers that delays are also caused by the lack of resources, especially negotiators and contract specialists. Also, the office of environmental services is identified as the one generating the biggest causes for delay.
In conclusion, for the internal environment, employees from all three levels are concerned about the reduction of human resource capacities in the department but the practices to deal with it are not formalized and not aligned.
Program-level interviewees think of current risk management approaches in terms of program-level objectives. Some of the current formal risk management practices mentioned by the interviewees are PTIP, Primavera P6 Project Management Module and the use of value engineering. These practices are mainly focused on risk analysis by early identification and evaluation of the potential issues during meetings between project managers and experts. However, they are less concerned with the mitigation of the risks. An interviewee mentions that a change of culture is needed for everyone to have a program-level view. They also consider what it means in terms of measures at the project level to improve early consideration of risk and updates on project status to have information to forecast at the program-level. Office-level interviewees and project-level interviewees think of current risk management mainly at the project level but also to some extent at the three levels. Interviewees focus on different kind of practices, they do not seem aligned but do not seem confined to their own level either.
Existing project development processes used within the agency were identified by interviewees. Program-level employees mention that before the department created project managers, experts working on a project met to exchange information and did risk
136

management intuitively without a formal process and without having one person responsible for the whole project. The design office handled the coordination. Other risk management solutions that GDOT adopted, when the focus on risk management became a strategic issue, like software tools that were expected to improve scheduling were not adopted with success in GDOT. They also mention changes in legislation regarding some practices such as value engineering. Existing project development processes are barely mentioned by office-level interviewees and project-level interviewees. An office-level interviewee mentions the change in legislation for value engineering and a project-level interviewee expresses frustration at not being able to work with experts from different public organizations as easily as before. There is a lack of evidence that existing project development processes impact current practices at GDOT, which might be due to the fact that previous measures were not adopted at a systemic level.
Interviewees from the three levels mention the same kind of main risks: the resolution of environmental issues as the biggest and the resolution of right of way and utilities.
The environmental issues are the change of scope, schedule and budget related to historic sites and endangered species. They are the biggest risk because they are most frequent.
"Environmental risks, to me, [are the biggest risk] in all my years of designing, I don't know if I ever had a project that didn't have some environmental risk."
"Environmental is the critical risk, probably about 90-95% of the time."
In addition, these issues are difficult to tackle because the nature of the environmental risk varies from project to project.
137

"One of the biggest environmental challenges is that practically every significant project is different [in terms of environmental risk]."
"I might have a small project that takes me two years to get environmental [work done]. I might have a small project that takes me six months... Until I am out there, I don't know what is out there."
The management of this risk could be improved, according to interviewees, by having knowledge management to provide an early anticipation of the issues. In addition, the frequent changes in environmental rules prevent GDOT from having a standard process in preparing the required documents. This issue could be tackled by having a standard monitoring of the changes and knowledge management.
Right of way creates issues because GDOT has to deal with multiple landowners within each project. Maintaining an up to date repository of property status and classification of sites could reduce them.
According to the interviewees, the Office of Utilities has a system to anticipate issues. In coordinating with the utility companies, the limitations come mainly from the utility companies' side since they have limited resources devoted to reviewing the needs of projects in each district.
Some other risks that are commonly encountered by GDOT projects at different levels are the following:
Small projects undergoing a scope change that makes them become big projects Political issues related to change in government
138

Public involvement Delays due to unforeseen issues that have a cascade effect on the yearly budget of
organization Interviewees think of the above risks' impact at different levels. Project level interviewees and office-level interviewees consider the uncertainty in scoping they introduce because of a lack of control over external factors. A project-level interviewee points out that certain offices such as the Office of Procurement can sometimes be bottlenecks for project delivery. This is due to the lack of staff available compared to the work required.
"[Project managers] use the instrument we call PRF, procurement requisition form, and they send it down to [the Office of Procurement]. It's always taking some time for them to allocate that assignment to contract specialist and a contract negotiator. And they finally assign two (a contract specialist and a contract negotiator), [however,] the project takes a long time because they have a big volume of work to do. I mean, they have a long queue."
Program-level interviewees seem aware of these problems and worry about their impact on time and money because of the precision of information they can use to forecast and manage project portfolio. Interviewees from the three levels seem to agree on the main risks and their impact on scope, schedule and budget although they think about the impact at different levels.
139

The types of projects that are most problematic from a risk management perspective involve highway widening and bridge projects as mentioned in the previous subsection. Highway widening and bridge projects are the ones that involve a lot of land and therefore usually entail right-of-way, utility and environmental issues. Office-level and project-level interviewees think of their impact at the project level only while programlevel interviewees talk about their implications for schedule and budget of projects and their repercussions on financial forecasting. The alignment is also only partial because they think of the same projects and problems but think about the impact at different levels. 4.3.2. Relationship in the Model Table 4-2 shows the cross tabulation between categories in the model. The content of references common to categories related in the model are analyzed in this section in order to test if data support the existence of the relationships described in the model.
140


Change factors

Table 4-2: Cross Tabulation of the Number of References Common to Categories

Change factors

CostsBenefits

External

Internal

environment environment

Main risks

New risk assessment

Existing project development process

Typical projects or problems

45

2

10

3

5

8

2

0

Costs-Benefits

2

23

0

2

0

1

0

1

External environment

10

0

38

8

12

8

3

4

Internal environment

3

2

8

129

14

30

8

5

Main risks

5

0

12

14

82

10

1

6

New risk assessment

8

1

8

30

10

78

7

11

Existing

project development

2

0

3

8

1

7

15

0

process

Typical

projects or

0

1

4

5

6

11

0

39

problems

141

Relationship 1: Existing project development process-Main risks The number of references coded for both themes is 1. There is little evidence that existing project development process has an impact on the main risks. The link is made by a program-level employee who reveals that in the old approach, especially before there were project managers at GDOT, there was no formalized process to anticipate risks. There is little evidence supporting this link and therefore, there is no possible discussion about the alignment.
Relationship 2: Existing project development process -Typical projects or problems The number of references coded for both themes is 0, so there is no evidence supporting this relationship. The lack of evidence for relationship 1 and relationship 2 does not mean necessarily that there is no relationship between the existing practices and the new ones, but rather that employees do not perceive the link as directly relevant to the current risks and issues faced by GDOT. According to the interviewees, in the past, offices acted individually, and some successful practices, such as the ones in the Office of Utilities, were adopted. In addition, some of the practices that GDOT tried to implement, such as Artemis for example, were not widely adopted within the organization. To have a global improvement of how main risks and typical problems are tackled, a bigger commitment, at the level of GDOT as a whole is needed.
Relationship 3: Internal environment-Main risks The number of references coded for both themes is 14 and come from 5 sources. This link is made by two program-level interviewees, one office-level interviewee and two project-level interviewees. Program-level interviewees worry about having the internal human resource capacities adequate to deliver projects. Main risks impact the internal
142

environment because risks create delays and projects are several years long so employees experience frustration because of never ending projects. The internal environment creates risks because of the reduced staff, heterogeneous experience and skills of project managers and inefficient interaction between offices and projects. Program-level interviewees indicate that the main risks impact scoping and project goals so they prevent them from doing accurate financial forecasting. The office level employee indicates a concern about managing coordination right between external parties, contractors, and project staff. Project level employees mention the lack of tools to manage information efficiently and issues due to the lack of available contract specialists in the department; so the Office of Procurement can be a bottleneck. There is evidence that the internal environment impacts how main risks are managed. However, we see that there is no alignment between employees of different levels who are concerned mainly with their own level.
Relationship 4: Internal environment-Change factors The number of references coded for both themes is 3 from 3 different sources. This link is made by two program-level employees and one office-level employee. Program-level employees mention that knowledge management practices have started to emerge within GDOT with changes in federal regulation and discussions. Also, value engineering practices have been changed according to federal rules but GDOT used to have a lower threshold. An office-level employee refers to the lack of funding having an impact on bids, which creates risks.
Evidence shows that most internal environment factors such as the variation in project managers' skills and the reduced offices' resources that explain GDOT's strategic focus
143

on risk management are a result of the reduction in funding due to external environment factors. This is a concern for program level employees and office-level employees at their respective levels so there is no alignment for this relationship.
Relationship 5: Internal environment-Benefits and costs The number of references coded for both themes is 2 as provided by one source (a program-level interviewee). This employee describes the costs and benefits of changing risk management practices upon the internal working environment. The chief cost could be to overwhelm managerial roles through the addition of numerous functional tasks: "So the question is: do you want your project managers turning into schedulers because you have included thousand [new] activities? Which means, (if) they are monitoring all those and checking those off, they don't have time to actually do their work. So the trade-off is you need to make it [i.e. risk management] reasonable." The benefits could be an increased satisfaction for employees if delays are reduced and they can be more effective at closing projects out. There is not enough evidence to study the alignment in this relationship but we note that program-level employees are concerned with repercussions of risk management changes at the project level.
Relationship 6: Internal environment-New risk assessment The number of references coded for both themes is 30 from 9 different sources. Some of the most important aspects of this relationship revealed by the interviewees are:
Communication between offices, especially with environmental services is key for risk management practices in the current internal environment.
144

Organizational changes: There is a newly formed risk management office in order to formalize risk management. In addition, project managers were created a few years ago as well as an office of program delivery. Senior project managers were turned into program managers, a recently created role, and are expected to mentor less experienced project managers.
Currently, project managers are responsible for project delivery. Rather than using formal models, they rely on meetings. These meetings are used to discuss between the different offices within projects or to discuss with other managers for knowledge exchange.
Knowledge management practices are emerging because the new staff is less experienced and outsourcing has become more common. The agency is moving towards more documentation and tutoring between employees.
Sharing lessons learned and training are not a formal requirement at the moment. There is a formal training to be a certified value specialist that is available.
There are projects "on shelves" that are ready but waiting for funding because the budget does not allow to do them.
For some information, such as constructability, they find useful to rely on consultants. For other information, like cost estimations, GDOT relies on employee experience and there is no formal process. All interviewees made this link in their interview as below:
Program-level employees mention changes in the organizational structure at the department level such as the current role of project managers being the person responsible of project delivery, the creation of divisions of program delivery and
145

risk management. They also mention changes in approaches related to office and project level work such as value engineering, review meetings, etc. Office-level employees mention the importance of early communication in current practices, especially when it comes to dealing with the environmental office. They mention that there is no formal risk model but managers get together and discuss outside projects to share knowledge. They also refer to value engineering usefulness for. Project-level employees mention that there are tools with information for projects in GDOT but many people have not adopted them and previous information has not been migrated, so they are not used. On the other hand, current process involves meetings such as Project Team Initiation Process (PTIP). There are other practices that have been adopted but are not formal such as the risk assessment process, which depends on the project managers and the subject matter experts (SME) on the project. A practice to overcome the lack of experience of some project managers is to have more experienced employees mentor and tutor them. In this regard, senior project managers have been renamed as program managers. There is significant evidence of the relationship between the internal environment and current risk management practices. Employees from different level focus on different things. Program-level employees focus on the organizational situation, office-level employees focus on practices related to office work and project-level employees focus on projects and offices.
146

Relationship 7: External environment-Typical projects or problems The number of references coded for both themes is 4 from 2 different sources. The major external factor impacting typical projects or problems are federal regulations:
They set requirements for many aspects of a project such as environmental impact and right of ways, for example.
One of the typical issues that projects face are the frequent changes in federal regulation, especially for environmental issues, that they have to cope with.
Federal recommendations influence the way project delivery is measured. Everything is based on schedules.
Two program-level interviewees make this link. The typical problem they mention is the very frequent changes in federal regulation and process that impact all projects and therefore forecasting at the program-level. Also, because of the need for GDOT to be in phase with federal fiscal year, it impacts decisions on which projects are chosen. Other factors, such as market values of materials and properties can also impact the types of projects and locations of projects. In this case, there is little evidence to discuss the alignment but we note that program-level employees are concerned with the impact of external environment changes on project-level schedules, although ultimately they are interested in the aggregate delays at the program-level.
Relationship 8: External environment-Change factors The number of references coded for both themes is 10 from 5 different sources. External environment impacts changes in risk management practices:
147

Risk management does not have a formalized reaction to changes in administration. An interviewee indicates that there is not much to do to mitigate them so the department does not focus on them even though some projects, on which resources have already been allocated, are sometimes stopped.
As mentioned before, federal regulation is the source of changes in several processes and in knowledge management, like value engineering.
Public support is affecting funding. Therefore, risk mitigation involves communication and public involvement.
Federal and state law affect the way the budget and GDOT's fiscal year are structured.
The recession impacted the right of ways. It is impacting budget that has effects on the department practices.
Other states' practice impacts GDOT's risk management practices. For example, the department started focusing on risk management after seeing in a conference that other departments had risk management models. Also, project managers were created based on other states' practices as well.
This link has been made by 3 program-level employees, 1 office level employee and 1 project level employee. Program-level employees mention political risks, such as changes in administrations, as being change factors. Federal regulation and discussion, for example for knowledge management and value engineering, trigger changes in GDOT.
"I think you could almost tie it back to the, when we started trying to document knowledge, [...] knowledge transfer, knowledge management, something like that.
148

But really, I think it came to the forefront when it started appearing in federal regulations, federal discussions".
State law and public opinion can also be factors for changing practices. Another external factor, mentioned by an office-level employee, is the influence of other states. The project-level employee mentions the economic situation as being a potential cause for GDOT's focus on project delivery:
"I would say, it is the number one major reason that projects, [which are] sometimes meaningful, can't move forward. The dollar amount is very high. So the department is just looking for more ways to streamline the high costs of a lot of good projects."
There is a relative alignment between the three different level employees given their awareness of the impact of the economic situation discussed earlier. However, officelevel and project-level employees have different concerns of other factors at their own levels.
Relationship 9: External environment-New risk assessment The number of references coded for both themes is 8 from 4 different sources. Two program-level employees and two office level employees make this link.
Both office-level and program-level interviewees suggested that federal and state law impact the way budget is structured and how projects are let at a department level. As a result, risk management is altered too. One of the examples given is value engineering. Interviewees from both levels also mention that the lack of control of some external
149

elements, such as right of ways, is dealt with using specific risk management practices such as standard charts for parcels to improve scheduling.
Office-level interviewees mention that other DOT influenced organizational changes in the department such as the creation of project managers and the strategic focus on risk in the first place that resulted in the creation of a risk management office. There is alignment between program and office level employees.
Relationship 10: Main risks- New risk assessment The number of references coded for both themes is 10 from 5 different sources. A few ways mentioned by interviewees about how main risks have an influence on current risk management practices are the following:
There is a standardized chart to predict the time that this aspect will take depending on the number of parcels to deal with risks related to right of way.
Contractors' work and coordination is one of the main risks. There is a formal procedure in case contractors defect to account them responsible for delays and damages.
The biggest risks are identified at the beginning of projects using a risk management scale.
Inexperience of staff is a big risk in GDOT and is dealt with by tutoring. This link is made by two program-level employees, two office level employees and one project-level employee. Program-level employees mention the use of contingency at a project level to solve uncertainty on budget. The uncertainty related to main risks should drop as the project progresses. They also mention having to make early decisions to kill
150

projects if they think the risk of non-delivery is too high. This approach is meant to cut losses and replace them with other projects that are kept as a backup. The office-level employee also mentioned the contingency that can be moved from project to project within a program. The project-level employee mentions a formal form for issues with contractors at a project level and the tutoring of inexperienced project managers to help them manage risks efficiently. There is an alignment between program-level and officelevel employees but not with project-level employees.
Relationship 11: Typical projects or problems-New risk assessment The number of references coded for both themes is 11 from 7 different sources. The typical projects or problems do not seem to be directly related to current practices. There is not a lot of evidence that road widening or bridge projects have affected new risk assessment. Right of ways and environmental issues are mentioned but the interviewees do not make a direct link with new risk management practices.
Scope, schedule and budget are mentioned related to current practices. The main risks category shows that the above typical projects and problems are the ones that have a large effect on these three factors. Therefore, there could be a connection but interview data do not show a direct link. The current risk management practices mentioned are all related to them. Early meetings are used as a formal process to make decisions about doing work in-house or outsource. Cost estimate tools are used in the planning or concept stages of projects. Rights of way are handled with using a standard chart. An early constructability assessment is done with consultants. The use of scheduling tools and processes helped improve delivery rate for certain offices.
151

This link is made by three program-level employees, two office-level employees and two project-level employees:
Program-level employees mention a tool, which helps make decisions to do work in-house or outsource early. They also mention cost estimation tools for projects as being not very precise.
Office level employees mention that an office implemented tools to minimize the risks, specific to their office, which had the most impact on projects' schedules. Cost estimation is a typical problem but there is no formal process: "there is no formal process for that; it's really based on their experience."
According to project-level employees, meetings could be more efficient if combined with a software. Another typical problem that affects efficiency is the fact that cost estimations are often not accurate.
There is no evident alignment between the concerns of employees of the three levels.
152

4.4. CONCLUSION AND DISCUSSION
The analysis of the interviews shows that there has been a strategic shift towards risk management due to the national economic situation. Certain issues such as a reduction in human resources are related to this shift. However, other issues existed before the change happened. The main risks that interviewees identify are related to the management of scope, budget and schedule of projects themselves and portfolio of projects at GDOT level. Risks that seem to exist regardless of the change are concentrated on widening and bridge projects because of the large impact on budget and schedule that a change in scope has. Existing project development processes were attempted to management of such risks. Some of these practices that were created at the program level are the creation of the role of project managers instead of having the design office as a tacit project management office, the creation of a risk management office, and also software tools for management of data and for scheduling. Projects managers are an established role. However, software tools were not widely adopted and some are used by only a few people and some were abandoned. In addition, solutions were developed by certain offices at the level of their activities. Overall, there was not a global solution at the organization level that tackled the issues but solutions were developed and adopted based on different perceptions of the same problem in the organization. These solutions tackled part of the problems only.
This way of solving issues was disrupted by the reduction in federal budget due the national economic situation. The reduction of federal funding for GDOT encouraged a clear focus on efficiency and risk management. In addition, it created additional issues such as a reduction in human resources that revealed that the organization lacked formal processes and knowledge management practices. All interviewees point out that it is an issue at least in two ways. Project managers with different level of skills and experience are responsible for projects. Project
153

managers need to figure out when and how to cooperate with offices to complete a project and this is done more or less efficiently depending on the person in charge. This variability is made possible by the fact that there are a lot of informal processes involved. The other issue is the lack of office staff in certain offices compared to the amount of work that is required from projects. As a result, there are delays in accomplishing the tasks necessary to complete projects. The model, revised in light of evidence, is shown in Figure 4-2.
Figure 4-2: Model Revised in Light of the Evidence When considering the perception of interviewees, there is little evidence that existing project development processes have an impact on risk considerations at GDOT. The internal
154

environment is related to main risks because of human resources' reduction and cooperation between offices; and projects are having difficulties dealing with some of the main risks related to scoping, scheduling and budget. However, the issues related to the internal environment are not perceived as a change factor. External environment factors, such as the economic situation and reduction of federal budget, are perceived as the main cause although they impact risk management at GDOT partly indirectly through changes in the internal environment. The external environment also impacts risk management directly at the office and project-levels: with federal and state requirements, funding source requirements, public pressure, and contractors. Weighing costs and benefits of new risk assessment is not the primary concern of employees at GDOT. It looks like almost anything that could improve project delivery would be welcomed.
A comparison of the levels at which risk management related elements are mentioned, the interviews show that in most of the cases, there is little alignment between the focus of program, office and project level employees.
The categories of the model on which all three level employees seem to focus on the same elements are: change factors, external factors, main risks, and typical projects and problems. However, the narrative shows that they are concerned with their impact at their own levels.
For the relationship between the categories of the model, all three levels seem to agree on the external factors that generate change, but then again, they are concerned with the impact at their own level. There is evidence of a greater alignment between office and project levels, both focusing more on projects. And there is evidence of alignment between program level and office level for the relationship between the internal environment and change factors, the relationship between the external environment and new risk management, and the impact of main risks on
155

new risk assessment. An analysis of the narrative per interviewee level, summarized in Table 4-3 confirms this low level of alignment and the focus mainly on their own level.

Table 4-3: Summary of the Comparison Between Role of Interviewees and Interview

Content Level of Consideration

Program-level employees
Office-level employees
Project-level employees

Approach revealed by the interviews Overall focus on program-level. Only program level for cost and benefits, change factors. Also project-level issues mentioned for main risks, new risk management approach and impact of internal and external environment but mainly in terms of aggregate impact on program level budget and schedule. Mention of impact on forecasting of organizational issues related to project manager experience and skills or process for interaction and transmission of information but almost no reference to office level objectives and delivery. Overall focus on office-level and project-level. Program-level for change factors: external environment with reduction of federal funding and increased interest in managing risk at GDOT. Department, office and project issue: heterogeneous capacities among project managers. Overall focus on project-level. Program-level for change factors: external environment with reduction of federal funding and increased interest in managing risk at GDOT. Office not mentioned or mentioned as bottleneck for projects.

These results do not show directly a negative impact on efficiency. Absence of alignment does not lead to incompatibility necessarily. There is a consensus on the cause of change of strategic focus towards risk management and the issues. However, different perceptions about what should be done might undermine the implementation of a global solution for GDOT if the differences are not addressed both when crafting the solutions and when implementing them.

156

CHAPTER 5 COMPREHENSIVE RISK ASSESSMENT FOR TRANSPORTATION (CRAFT) SOFTWARE
MANUAL
5.1. INTRODUCTION
Comprehensive Risk Assessment For Transportation (CRAFT) is a software tool specifically designed for identification and qualitative assessment of highway project risks during the preconstruction phase of the project. Several GDOT offices, such as the Office of Environmental Services, the Office of Right-of-way, and the Office of Utilities, are involved in pre-construction services of the project. The project manager needs to understand the project issues from the perspective of each office since the overall project success (i.e., on-time and on-budget delivery of the project) depends on the smooth execution of various pre-construction tasks. Any issue (i.e., risk factor) that may impact the project progress should be identified early and its impact should be assessed. The project manager in charge of the project needs to reach out to subject matter experts in different offices to identify the critical issues that may negatively affect the project progress.
Currently, several project managers, including newly-hired project managers, work for the GDOT Office of Program Delivery on multiple concurrent projects. These project managers need assistance to structure meaningful dialogues with subject matter experts in different offices, in order to elicit their knowledge about the issues that may adversely affect the progress of the project. New project managers may not know where to start, what offices they should contact, and what issues should be studied as part of pre-construction services for the project. A
157

preliminary checklist identifying possible issues that may affect the progress of the project can be extremely helpful for the project manager to begin investigating project risks from the perspective of each office involved in the project. The GDOT Office of Program Delivery expressed interest in the customizable risk registers that can be utilized by the project manager to retrieve the risk information from subject matter experts in various GDOT offices. The need for a structured approach to search for issues that may adversely affect the progress of the project (i.e., the risk factors) was the main motivation behind the development of the Comprehensive Risk Assessment For Transportation (CRAFT) software. The risk register for each office has been developed using the outcome of the research project conducted at Georgia Tech in collaboration with the Office of Program Delivery at GDOT. The CRAFT software is equipped with a visual interface for each office to identify and assess the related risks. A list of commonly found risk factors is built in into the software, but the subject matter expert can add a customized risk factor to the list. Central documentation of the information items related to the project risks is a critical missing element in the current GDOT's plan development process. A single source of access to project issues allows the project manager to engage all stakeholders from different offices in identifying project issues, determining mitigation solutions, and finding opportunities to expedite project delivery. The GDOT Office of Program Delivery has recognized the need for an automated tool that allows different project stakeholders to interact with each other and with the project manager to identify the project issues. Automation is the key as the project manager needs to establish contacts with multiple subject matter experts in several different offices to retrieve their assessments of the project issues.
158

A critical task before developing the software was risk identification. A short list of major potential risks is an important input for the software development process.
5.2. RISK IDENTIFICATION PROCESS
The risk identification process consists of three major steps based on the possible sources to identify the risks. The first step is to review the project development process at GDOT and conduct a scenario analysis to develop a primary risk list. The second step is to interview with subject matter experts at GDOT to identify the risks based on their experiences and what they have observed in the projects. The third step is to conduct a comprehensive literature review to study the identified risks in the previous studies, manuals, and guidebooks. Those risks that might be applicable in GDOT project were added to the developed risk register in the previous steps.
5.2.1. Review the Project Development Process At first step, an extensive risk list was developed based on the GDOT plan development process manual. The development process for major projects at GDOT consists of different steps such as concept development, preliminary design, environmental process, right-of-way plans development and approval, utilities plan development and coordination with utilities and railroads, value engineering study (if required), public hearing, preliminary field plan, acquiring right-of-way, final utility relocation plan development, final field plan, final design, and letting to contract. For each step, the recommended process in the Plan Development Process (PDP) manual of GDOT was studied and different scenarios that can impact the plan development were analyzed. Based on the scenario analysis, a primary comprehensive list of possible events that can trigger risks during the plan development process was prepared. In this step, the risks were categorized based on the plan development phase (i.e. concept development, preliminary design,
159

final design). It should be noted that some of the risks and trigger events may belong to more than only one phase of the plan development. For example, the risk of errors in design may occur in both preliminary and final design or a significant change in the scope of the project can be a risk during all phases of concept development, preliminary design, and final design.
After identifying the potential risks based on the scenario analysis, possible risk responses were identified based on reviewing the plan development process. These risk responses are general strategies and actions that can be considered to align with plan development process to reduce the probability or impact of the negative risks. The identified risk responses were gathered together to develop a primary list of risk responses integrated into the identified risk list.
5.2.2. Interview with the GDOT Subject Matter Experts In the second step, nine subject matter experts at GDOT were interviewed. Interviews followed a semi-structured protocol. Among the interviewees, two were project level employees, three were office-level employees and four were program level employees. During the interviews, the subject matter experts were asked to mention the most important and probable risks based on their past experiences at GDOT. The final deliverable of this step was another comprehensive list of potential risks. These risks were added to the developed risk register in the previous step. The identified risks in this step were highlighted in the risk register to emphasize their importance indicating that subject matter experts think that those risks are more important than other possible risks.
5.2.3. Literature Review As noted in the previous chapters of this report, several state DOTs have standard risk management process or sample risk registers that help their employees for risk identification
160

process. Furthermore, several risk management guidebooks developed by FHWA and other organizations contain sample risk lists. In this step, the identified risks in the literatures were studied and those that can be applicable in GDOT projects and were missed in the risk register developed in the previous steps were added to the risk register.

The final deliverable of these three step was an extensive and comprehensive risk register developed based on the three different sources for risk identification. After developing the comprehensive risk register, several meetings were held with higher level GDOT professionals to present the risk register and get their feedbacks. Considering their feedbacks, the identified risks were categorized based on the responsible offices. This categorization will help project managers to assign the risk to the risk owners more efficiently.

After categorizing the risk register based on GDOT offices, GDOT subject matter experts reviewed the risks during several meetings to determine the most important risks from GDOT's point of view. They determined the most important risks based on their past experiences at GDOT considering the likelihood and impact of the risks. At the end of this step, a short list of major risks for each office was developed. GDOT subject matter experts preferred to present the risk in the format of questions. Table 5-1 to Table 5-11 show the major risks for each office in question format.

Table 5-1: Major Risks for the Office of Bridge Design

Risk Questions

1

Are there hydraulic issues that could significantly impact project development?

2

Are there structural or foundation issues that could significantly impact project development?

3

Are there constructability issues that could significantly impact project development?

4

Have environmental issues been identified that could impact, delay or require mitigation of bridge plans?

161

Table 5-2: Major Risks for the Office of Project Management

Risk Questions

1

Are there funding issues now or in the future?

2

Are there schedule issues?

3

Are there scope issues?

Table 5-3: Major Risks for the Office of Construction

Risk Questions

1

Are there constructability issues that could significantly impact project development?

2

Are there access issues that could significantly impact project development?

3

Are there issues with payment on the project?

Table 5-4: Major Risks for the Office of Roadway Design

Risk Questions

1

Are there geometric issues that could significantly impact project development?

2

Are there potential drainage issues?

3

Are there traffic analysis or capacity issues?

4

Are there utility conflict issues?

5

Are there staging or constructability issues?

Table 5-5: Major Risks for the Office of Design Policy and Support

Risk Questions

1

Are there survey availability issues that could impact project development?

2

Are there erosion control issues that could impact project development?

3

Are there MS4 issues that could impact project development?

Table 5-6: Major Risks for the Office of Right of Way (ROW)

Risk Questions

1

Is the project in a residential area?

2

Is the project in a commercial area?

3

Are there access issues in the project corridor?

4

Are there a significant number of displacements in the project corridor?

5

Are there properties with potentially contaminated soils?

6

Are there known environmental issues that may significantly impact ROW acquisition?

162

Table 5-7: Major Risks for the Districts

Risk Questions

1

Is there local government support for the project?

2

Is there local stakeholder (citizens) support for the project?

3

Will the project require coordination among different entities that could result in impacts or delays to the

project schedule?

Table 5-8: Major Risks for the Office of Traffic Operations

Risk Questions

1

Are there safety issues along the project that require additional time or information to address?

2

Are there traffic signal justifications or permits required?

3

Are there new equipment requirements?

Table 5-9: Major Risks for the Office of Environmental Services

Risk Questions

1

Are there major natural environmental issues that could significantly impact project development?

2

Are there major human environment issues that could significantly impact project development?

3

Will significant coordination beyond the norm be required with external partners?

4

Are there significant time constraints for studies or permits beyond the norm for this type of project?

5

Is an environmental impact statement (EIS) required?

Table 5-10: Major Risks for the Office of Utilities

Risk Questions

1

Is railroad involvement required for the project?

2

Are there major utilities located in the project corridor?

3

Will the design require relocation of major utilities?

4

Are there known utility coordination issues?

5

Will a Subsurface Utility Engineering (SUE) or Public Interest Determination (PID) be required?

Table 5-11: Major Risks for the Office of Materials and Testing

Risk Questions

1

Is the project located in an area with less than desirable soil?

2

Are there pavement design issues that could significantly impact project development?

The short list of major risks categorized based on the GDOT offices was the key component to develop the CRAFT software.
163

5.3. SOFTWARE DEVELOPMENT
Not all of the identified risk factors have the same degree of importance from the perspective of their effects on the progress of the project. The developed software provides an interactive platform for the subject matter experts to express their beliefs about the relative significance of each identified risk factor. The overall effect of each risk factor on the progress of the project is assessed on two dimensions: (a) the chance (or likelihood) of occurrence of the risk event; and (b) the impact of the risk event on the project schedule. This qualitative assessment approach is utilized in the CRAFT software to allow subject matter experts to analyze the impact of each indented risk on the progress of the project and relatively rate their impacts on the project schedule.
The software processes the information collected from various subject matter experts in the risk identification and risk assessment steps and generates two types of outputs. First, the tool takes the information collected from the subject matter expert in each office and creates a risk register and a risk heat map for each office. The software summarizes the outcome of the identified risks in a table called risk register that assigns a unique risk ID to each identified risk. Also, the software presents the outcome of the qualitative risk assessment as a risk heat map specifically developed for each office.
Risk heat map is a tool that represents the results of a risk assessment process in a visual way. A risk heat map shows the identified risks and their relative significance in the progress of the project. The relative significance of the identified risks is determined based on the likelihood and impact of risks. Considering the likelihood and impact of each risk, the identified risks are placed in a two-dimensional map that is color-coded indicating the significance of a risk. The horizontal axis shows the potential impact of the risk factor on the project schedule. The vertical axis shows
164

the likelihood of a given risk occurring. The colors represent risk areas; e.g., green-colored boxes are in the low area (bottom left corner of the risk heat map); yellow boxes are in the medium area (center of the risk heat map); red boxes in the high area (upper right corner of the risk heat map). The risk factors are plotted on the heat map based upon the "Potential Impact on the Project Schedule" and "Likelihood of Risk Occurring." An automated tool not only facilitates the retrieval of the risk information items from several subject matter experts in multiple offices, but also provides a platform that integrates different risk inputs into a single repository of the project issues. One of the major outputs of the CRAFT software is a risk breakdown structure (RBS) that organizes the entire list of the identified risks in a color-coded risk matrix. An RBS is a hierarchical representation of the identified risk that is arranged based on different categories of risks. The categorization in CRAFT software is based on the divisions of work (i.e., offices) identified in the GDOT organizational chart. The CRAFT software provides a framework that helps project managers and subject matter experts identify, assess, and rank major project risk factors in a systematic, transparent, and collaborative manner.
165

5.4. STEPS TO USE CRAFT
5.4.1. Step 1- Run the Tool Open the folder on your computer where the CRAFT tool is located. Double click on the software icon. The tool will open up and the following main menu form will appear on your screen:
Figure 5-1: Main Menu of the CRAFT Software As shown in Figure 5-1, the software provides a list of different GDOT offices that are involved in preconstruction services of the project. Click on any of the eleven buttons representing different offices to access the respective risk register for the office. Subject matter experts in each office can evaluate the project risks related to their offices by visiting the risk register developed for their offices. Click on "About" button to see the copyright and contact information related to this software (Figure 5-2). Click on "HELP" button to access a brief instruction for using CRAFT software (Figure 5-3). Note that the brief manual is a searchable text document.
166

Figure 5-2: Copyright and Contact Information of the CRAFT Software Figure 5-3: Brief Instruction in the Help Menu 167

5.4.2. Step 2- Identify Risk and Assess its Impact The software allows the subject matter experts from different offices to provide their assessments of the project risks from the perspective of their own offices. When clicked on one of the office buttons (e.g., office of "Right of Way"), a new window will appear as it is shown in Figure 5-4. This window shows the list of risks related to the selected office (i.e., the risk register template for the office). Risks are presented in the format of specific questions and the subject matter expert can specify whether the project risk is applicable to the project. If the risk is not applicable in your project, check the "NO" box. For those risks selected to be applicable to the project, the subject matter expert needs to specify the likelihood of occurrence and impact on the project schedule for each identified risk factor. Use the colorcoded scale bars (as shown in Figure 5-4) to qualitatively assess the effect of the identified risk. Adjust the ruler proportional to the level of the risk impact with regard to its likelihood and the respective impact. If needed, the subject matter expert can add a new risk factor to the risk register template. Answer YES to the last question and enter the description of the new risk item in the provided box. Then, specify its likelihood and its potential impact. For instance, Figure 5-4 shows an example of risk identification and assessment conducted on a project. The Right of Way subject matter expert specified the following risk information for the project:
The subject matter expert believed that the project did not run into any residential and commercial right of way risk. Hence, NO was selected for these two risk factors.
Access issues in the project corridor were considered a risk factor for this project. The subject matter expert specified relatively low likelihood for the occurrence of the risk
168

(identified by the yellow color on the scale bar) but believed that its potential impact on the project would be relatively high (identified by light red color on the scale bar). A significant number of displacements in the project corridor were determined as a risk factor for this project. The subject matter expert specified relatively low likelihood for the occurrence of the risk (identified by the yellow color on the scale bar) but believed that its potential impact on the project schedule could be devastating (identified by solid red color on the scale bar). Existence of properties with potentially contaminated soils was identified as a risk factor for this project. The subject matter expert strongly believed that the project would run into this problem. Therefore, she selected very high likelihood for the occurrence of the risk (identified by solid red color on the scale bar). The potential impact of this risk on the project schedule was defined as moderate (identified by orange color on the scale bar). Existence of environmental issues that may significantly impact ROW acquisition was determined as a very minor risk factor, low likelihood of occurrence with low impact on the project schedule (both were identified by solid green on the scale bars). In addition to the above risk factors, the subject matter expert specified a new risk factor for this project and added it in the designated textbox. The new risk factor is "Railroad involvement." The subject matter expert defined this risk as a moderate risk with low likelihood (identified by solid green color on the scale bar) and above average impact on the project schedule (identified by light red color on the scale bar).
169

Figure 5-4: Identifying Likelihoods and Impacts of Risk Factors
5.4.3. Step 3- Create the Risk Register and the Risk Heat Map for Each Office After determining the likelihoods and impacts of the project risk factors, you have three choices:
1- Click on "Save and Exit" to go back to the main menu and continue identifying the risk factors for other offices.
2- Click on "Exit" to come back to the main menu without saving your inputs for the office. 3- Click on "Report to Excel" to see the heat map risk for the office in an Excel file. For
instance, Figure 5-5 shows the Excel file developed by "Report to Excel" button for the Office of Right of Way. It should be noted that, if you save and exit from this page and come back later, you still can click on "Report to Excel" and see the results. Furthermore,
170

you can get the risk heat maps for each office separately as part of the comprehensive report that will be explained in the next step as well.
Figure 5-5 shows the developed risk heat map and the risk register for the office of ROW. The table on the right side of the Excel sheet shows the description of the risk factors and assigns a unique risk identification code (ID) to each risk factor. The subject matter expert's efforts in the risk identification step is summarized as Yes/NO answer in the column next to the risk factor. The risk heat map on the left side of the Excel spreadsheet shows the results of the subject matter expert's efforts in qualitative risk assessment. The risk ID of each identified risk factor is placed on the 2-diemnsion risk heat map. The vertical axis of the risk heat map represents the likelihood of risk occurrence as specified by the subject matter expert in the previous step. The horizontal axis of the risk heat map represents the impact of the identified risk on project schedule as specified by the subject matter expert in the previous step. The identified risk factors are placed in the risk heat map according to the subject matter expert's assessments in these two dimensions. The risk heat map is color coded to depict the relative significance of the identified risks.
For instance, Figure 5-5 shows the relative significance of the five identified risk factors: access issues in the project corridor (ROW03), a significant number of displacements (ROW04), existence of properties with potentially contaminated soils (ROW05), existence of environmental issues (ROW06), and railroad involvement (ROW07). Based on the subject matter expert's assessment, access issues in the project corridor (ROW03), a significant number of displacements (ROW04), and railroad involvement (ROW07) are moderate risk factors and therefore, are placed in middle of the heat map (color-coded in yellow). Existence of properties with potentially contaminated soils (ROW05) is a high-risk factor in this project and hence, is
171

placed in the upper-right corner of the heat map (color-coded in red). On the other end of the spectrum, existence of environmental issues (ROW06) is a low-risk factor for this project and thus, is placed in lower-left corner of the risk heat map (color-coded in solid green).
Figure 5-5: Risk Heat Map for the Office of Right-of-way 5.4.4. Step 4- Create the Risk Breakdown Structure for the Project & Generate the Comprehensive Summary of All Identified and Assessed Risk Factors The CRAFT software allows the project manager to combine all risk identification and assessment done by various subject matter experts in different involved offices, in order to create a single spreadsheet summarizing all of the identified risk factors. After you received risk inputs
172

from all offices, go back to the main menu and click on "Report to Excel" button to generate the comprehensive output of the risk analysis software. An Excel file consisting of several sheets will be automatically generated. The first sheet of the Excel file shows the project risk breakdown structure (RBS) that is organized by the involved offices in risk analysis. Each column represents the identified risk factors for an office. The identified risk factors are color-coded based on their relative significance. Dark green, light green, yellow, orange, and red are used to provide correspondence with low, medium-low, medium, medium-high, and high risk factors, respectively. Risk factors that were identified by the subject matter experts as not applicable to the project will be shown in shadowed light grey color in the developed RBS. Figure 5-6 shows an example of the comprehensive RBS for the project. Each column summarizes the identified risks for an office. For example, take a look at the column related to the risks for the office of ROW. The first two top cells (e.g., residential and commercial ROW issues) are shown in shadowed light grey. This means that these two items were not considered as risk factors in the project. The next cells represent medium-level risks, access and displacement issues, and hence, are highlighted in yellow. The cell for potentially contaminated soil is highlighted in red, which means that this item was defined as a high risk factor. The cell for environmental issues is highlighted in green, which means that this item was determined as a minor high risk factor. Finally, the last cell at the bottom of the ROW column represents the additional risk factor identified by that the ROW subject matter expert. If you move your mouse onto this cell, a note box will appear to describe the additional risk (here defined as railroad involvement).
173

Figure 5-6: Risk Breakdown Structure (RBS) of the Project
The Excel file also has a separate risk analysis spreadsheet for each office. Each sheet represents the risk register and the risk heat map for each office based on the assessment inputs provided by the subject matter expert. Save the Excel file as the risk analysis report for the project. The project manager and the other stakeholders can open the Excel file without any need to access the CRAFT software. One of the distinctive features of the CRAFT software is that it provides the risk analysis output as an Excel file with several spreadsheets. The Excel output automates the process of generating report for risk analysis exercise performed on the project. The developed spreadsheets including individual risk heat maps for each office and the combined risk breakdown structure (RBS) can be directly exported to the risk analysis report file that will be prepared by the project manager for the project. The Excel output file can be utilized by the project manager or the other project stakeholders to provide further notes and comments about the project. For instance,
174

comment boxes can be added to elaborate the content of any cell in the risk analysis spreadsheets. 5.4.5. Step 5- Save the Risk Analysis Results The CRAFT software allows the project manager to save partially-completed risk identification and assessment efforts. The saved file can be opened later to resume risk analysis. Click on the "Save" button to store the current state of risk analysis efforts performed on the project. A new window will pop up that allows you to find the right folder for saving the results of risk analysis efforts. Choose an appropriate file name and click the "Save" button (Figure 5-7). The risk analysis file will be saved as a ".dat" file.
Figure 5-7: Saving the Risk Analysis Results for the Project 5.4.6. Step 6- Open the Risk Analysis File If you want to open a risk analysis project in the CRAFT software, click on the "Open" button in the main menu. An open window will pop up. If you need to browse the folder that you stored your risk analysis file in, select the file, and click on the "Open" button (Figure 5-8). The entire risk information including the identified risks, their likelihoods, and potential impacts will be
175

imported into the CRAFT software. You can continue working on the risk assessment and modify any part of, get the Excel outputs or continue working on the project and make further changes.
Figure 5-8: Opening the file for Risk Analysis Results of the Project
Finally, the CRAFT software allows subject matter experts from different GDOT offices to work on risk analysis independently from each other and later consolidate different risk assessment modules into a single unified file for the whole project. The project manager can share a portion of the CRAFT software with the subject matter expert in each office and request his/her risk assessment inputs. There are eleven CRAFT sub-programs, each contains only a portion of the whole software that is applicable to one of the eleven offices involved in risk analysis. Each subject matter expert can conduct step 2 (risk identification & assessment) on his/her own, save the file, and send it back to the project manager for compiling the results into a single risk analysis file for the entire project. For instance, Figure 5-9 shows the snapshot of the CRAFT sub-program for ROW risk analysis that will be shared with the ROW subject matter expert for receiving her inputs. Once the project manager runs the master software program, she
176

can open individual risk analysis files received from various offices. The master software automatically appends all files into a single risk analysis file.
Figure 5-9: CRAFT Sub-program for ROW Risk Analysis Disclaimer: The CRAFT software has been developed in the C# (C Sharp) environment. It works on any computer with the Microsoft Office 2003 or more recent versions installed.
177

CHAPTER 6 CONCLUSIONS
A common understanding of the importance of risk and risk management is pervasive throughout many state DOTs. Uncertainties and risks can negatively impact the project outcomes as cost overrun and schedule delay. GDOT needs to enhance its understanding regarding source and natures of these risks early in concept and scope development phases. Project risk management is an important process that can objectively identify, evaluate, and analyze project risks. This process can increase the value of the project and assure that the project will be completed within the budget and schedule.
In this study, a comprehensive guidebook that advances the adoption of risk analysis tools in GDOT, in order to expedite project delivery, was developed. To achieve the research objective, various project risk management processes developed by different organizations were reviewed. Furthermore, current practice of risk management in different state DOTs were studied. Several state DOTs were surveyed regarding their risk management programs. Some of them have a standard process and guidebook to implement risk management. However, some of the other surveyed state DOTs relay mostly on their project managers' experiences for a successful risk management. After analyzing the results of the survey, a semi-structured interview was conducted to achieve more detailed information about the current practice of state DOTs for risk management. The results indicate that typically state DOTs determine the level and methods of risk management based on project size (i.e. dollar value) and complexity of the project. The level of risk management may vary from a simple risk register to a complex quantitative analysis. Moreover, several factors such as lack of training of personnel, lack of sufficient internal
178

infrastructure such as database, lack of existing policies, and lack of risk culture are among the most important challenges and barriers to implement a successful risk management program.
After reviewing the literature and current risk management practices by state DOTs, a semistructured interview was conducted with nine subject matter experts at GDOT. The interviewees were responsible for project management, program level roles, and specialists from different offices. The key factors that influence risk management practices within GDOT were explored. Then, the results were analyzed and a model explaining the current risk management practice and future needs was developed.
A comprehensive list of potential risks for transportation projects was developed based on reviewing the academic/professional literature on risk analysis, current state of practice in risk management among leading state DOTs, and current state of practice of GDOT. The identified risks were categorized based on the responsible offices at GDOT. During several meetings with higher level risk management experts at GDOT, the most important risks were indented and a short list of major potential risks was developed for each office at GDOT.
Finally, a software tool specifically designed for identification and qualitative assessment of highway project risks during the pre-construction phase of the project was developed based on the shortlisted risk factors. The software program is equipped with the modification capability of adding new risk items and/or removing some of the predetermined risk factors from the assessment. The CRAFT software allows subject matter experts from different GDOT offices to work on risk analysis independently from each other and later consolidate different risk assessment modules into a single unified file for the whole project. The project manager can share a portion of the CRAFT software with the subject matter expert in each office and
179

request his/her risk assessment inputs. An automated tool not only facilitates the retrieval of the risk information items from several subject matter experts in multiple offices, but also provides a platform that integrates different risk inputs into a single repository of the project issues. One of the major outputs of the CRAFT software is a risk breakdown structure (RBS) that organizes the entire list of the identified risks in a color-coded risk matrix. An RBS is a hierarchical representation of the identified risk that is arranged based on different categories of risks. The categorization in CRAFT software is based on the divisions of work (i.e., offices) identified in the GDOT organizational chart. The CRAFT software provides a framework that helps project managers and subject matter experts identify, assess, and rank major project risk factors in a systematic, transparent, and collaborative manner.
180

APPENDIX A: SURVEY QUESTIONNAIRE
GENERAL INFORMATION
* 1. Respondent Information Name Title Name of Agency Organizational Unit E-mail Phone Number 2. How long have you been with this agency?
3. How long have you been at your current position?
STATE OF RISK MANAMGNENT AT YOUR AGENCY
* 4. Does your agency have a special office/division for managing project risks? Yes No
If yes, what is the name of the unit? * 5. Is the risk management process an integrated component of the standard Project Development Process (PDP) in your agency?
Yes
181

No
Please briefly explain. * 6. Does your agency have a guidebook for project risk management process?
Yes No If yes, could you please email the guidebook to baabak@gatech.edu?
RISK MANAGEMENT PROCESS AND ORGANIZATION
* 7. Typically, who is in charge of conducting project risk management process in your agency? Project manager An internal expert in risk management Consultant Other (please specify)
* 8. Typically, which of the following projects go through the risk analysis process? Road-Rehab/Reconstruct Projects Road-Resurface/Renewal Projects Interchange-Construct/Improve/Modify Projects Managed Lanes-Construct-Modify Projects Bypass Projects Bridge and Tunnel Projects ITS (Intelligent Transportation Systems) Projects
182

Grade Separation Projects Other (please specify)
* 9. Typically, at what stage of project's development process, does your agency conduct risk analysis? Visioning and Policy Long-Range Planning and Programming Concept Development Preliminary Design and Environmental Studies Final Design Right-of-Way Acquisition
Additional comments
CANDIDATE PROJECTS FOR RISK ANALYSIS
* 10. Typically, for what project size, does your agency conduct risk analysis? Less than $1M Between $1 M and $5 M Between $5 M and $50M Between $50M and 100M Between $100M and $500M Greater than $500M (mega projects) All project sizes Project size does not matter Other
183

* 11. Typically, for what project duration, does your agency conduct risk analysis? Less than six months Between six months and one year Between one and three years Between three to five years More than five years All project durations Project duration does not matter Other (please specify)
12. How does your agency identify complex projects that can be prime candidates for conducting risk analysis? Please describe briefly or refer to a document.
RISK IDENTIFICATION
* 13. Which of the following methods does your agency employ to identify potential risks? Conducting brainstorming sessions Conducting structured risk identification workshops Developing and utilizing risk checklists Using the Delphi method
184

Organizing structured interviews with project participants Developing questionnaires and conducting surveys Developing and utilizing Risk Breakdown Structure (RBS) Utilizing scenario planning to identify risks Not using any formal approach Other (please specify)
* 14. Has your agency developed any risk register as an initial checklist for risk identification? Yes No
If yes, could you please email a copy of your risk checklist to baabak@gatech.edu? * 15. Does your agency use an external professional facilitator from the consulting world for organizing and leading risk identification workshops for your projects?
Yes No
If No, who facilitates your risk identification workshops? * 16. Typically, which of the following offices/experts participate in your risk management workshops?
Engineering Environmental Analysis & Permitting Roadway Design Bridge Design
185

Geotechnical Design Policy & Support Construction Materials Safety Bidding Administration Contract Management Right of Way Utilities Railroad Traffic Operations Maintenance Project Management Project Control Estimation Project Delivery Systems Procurement Legal Services Construction Claims Finance & Budgeting General Accounting Organizational Performance Management Human Resource Management
186

Training & Development Communications Commissioning & Planning Government & Legislative Relations Local Grants
17. Typically, which of the following stakeholders participate in your risk management workshops? District Offices Federal Highway Administration (FHWA) Division Offices Railroad Companies Public Utilities Companies Private Utilities Companies Property Owners State Environmental Protection Agencies State Departments of Natural Resources Toll, Port, or Turnpike Authorities MPOs (Metropolitan Planning Organizations) U.S. Army Corps of Engineers (USCOE) U.S. Fish and Wildlife Service (USFWS) Tribal Governments Environmental Interests Groups Affected Neighborhoods and the Public
187

Historically Underserved Public by the Transportation System, Including Minority and Low-income Populations
Other Modal Administrations within the U.S. DOT Engineering Consulting Firms Highway Contractors
RISK ASSESSMENT
* 18. Typically, what methods does your agency employ for assessing the identified risks? Quantitative methods Qualitative methods Risk heat maps Scenario analysis Monte Carlo simulation No formal risk assessment method Other (please specify)
* 19. Typically, for what types of projects does your agency conduct quantitative risk assessment? Please briefly explain the decision process (e.g., project characteristics, size, cost, etc.)
* 20. What are the software programs that your agency typically uses for conducting risk assessment?
188

Excel spreadsheets Primavera Risk Analysis Crystal Ball @Risk In-house customized software system Other (please specify)
PROJECT RISK CONTROL
* 21. Typically at your agency, which of the following tools are employed to monitor and control risks? Project risk response audit done by auditors to evaluate the effectiveness of risk response plans Project risk response review regularly done by project team to capture the changes in risks and their
prioritization Technical performance measurement Additional risk management planning No specific process Other (please specify)
ORGANIZATIONAL ISSUES
* 22. Does your organization have any systematic approach to capture lessons learned from conducting risk analysis on different projects?
Yes No If yes, would you please email some of the lessons learned documents to baabak@gatech.edu? * 23. After project completion, does your agency evaluate the identified risks to check if they have occurred and if their assessed impacts were accurate?
189

Yes No
If yes, please explain. 24. Has your agency developed specific performance metrics to measure the success of the risk management program?
Yes No
If yes, please explain.
25. How has your agency established a culture of risk management for enhancing project delivery? (e.g. providing training workshops and incentives, etc.). Please describe briefly.
26. Which of the following techniques are typically used in your agency to capture risk management knowledge? Writing final project reports Using formal story telling Interviewing the experts involved in previous projects Developing a lessons learned database Capturing knowledge as the project moves forward Having more experienced personnel to document procedures and lessons, and develop standards Other (please specify)
190

191

PROJECT RISK MANAGEMENT CHALLENGES/BARRIERS

27. Considering the experience of your agency in implementing risk management process, what is the relative importance of the following barriers/challenges for successful execution of risk analysis?

Lack of staff or resources for complex tasks
Issues with the risk management tools
Lack of sufficient internal
infrastructure such as database
Overall lack of adequate funds
Lack of training of personnel
Inaccurate forecasts
Lack of existing policies
Lack of support from the top
Lack of risk culture
Lack of communication among offices
Inefficient organizational
frameworks
Lack of desire to use new procurement methods
Inefficient coordination and communication between the agency and other local, state,
and federal government entities
Inefficient risk allocation
Lack of best practices and available training
Poor prospects for economic growth

Not important

Slightly important

Important

Very important

192

REFERENCES
Akintoye, A. S., & MacLeod, M. J. (1997). "Risk Analysis and Management in Construction". International journal of project management, 15(1), 31-38.
Alarcn, L. F., Ashley, D. B., de Hanily, A. S., Molenaar, K. R., & Ungo, R. (2010). "Risk Planning and Management for the Panama Canal Expansion Program". Journal of Construction Engineering and Management, 137(10), 762-771.
Al-Bahar, J. F., & Crandall, K. C. (1990). "Systematic Risk Management Approach for Construction Projects." Journal of Construction Engineering and Management, 116(3), 533-546.
Anderson, S., Molenaar, K.R., and Schexnayder, C. (2007). "Guidance for Cost Estimation and Management for Highway Projects during Planning, Programming, and Preconstruction", NCHRP 574, Transportation Research Board of the National Academies, Washington, DC.
Ayyub, B. M. (2001). "A Practical Guide on Conducting Expert-Opinion Elicitation of Probabilities and Consequences for Corps Facilities", Institute for Water Resources, Alexandria, VA, USA.
Baloi, D. (2012). "Risk Analysis Techniques in Construction Engineering Projects", Journal of Risk Analysis and Crisis Response, 2(2), 115-123.
Berg, H. P. (2010). "Risk Management: Procedures, Methods and Experiences", Risk Manage, 1, 79-95.
193

Caltrans (2007). "Project Risk Management Guidebook: Threats and Opportunities", California Department of Transportation. Sacramento, CA.
Canadian Standards Association (2002). "Risk Management: Guideline for Decision Makers", Canadian Standards Association: CAN/CSA-Q850-97, Mississauga, Ontario.
Caltrans (2012). "Project Risk Management Handbook: A Scalable Approach." Report of the California Department of Transportation (Caltrans), Office of Project Management Process Improvement. Sacramento, CA.
Carr, V., & Tah, J. (2001). "A Fuzzy Approach to Construction Project Risk Assessment and Analysis: Construction Project Risk Management System", Advances in Engineering software, 847-857.
Chapman, C. (1997). "Project Risk Analysis and Management (PRAM): The Generic Process", International Journal of Project Management, 15(5), 273-281
Chileshe, N., and Yirenki -Fianko, A.B. E. (2012). "An Evaluation of Risk Factors Impacting Construction Projects in Ghana", Journal of Engineering Design and Technology, 10 (3), 306-329.
Chileshe, Nicholas, and Geraldine John Kikwasi. (2013) "Perception of Barriers to Implementing Risk Assessment and Management Practices by Construction Professionals in Tanzania", Management, 1137-1146.
Creedy, G. D., Skitmore, M., & Wong, J. K. (2010). "Evaluation of Risk Factors Leading to Cost Overrun in Delivery of Highway Construction Projects", Journal of construction engineering and management.
194

Cretu, O., Stewart, R. B., & Berends, T. (2011). "Risk Management for Design and Construction", (Vol. 77). John Wiley & Sons.
D'Ignazio, J., Hallowell, M., & Molenaar, K. (2011). "Executive Strategies for Risk Management by State Departments of Transportation". NCHRP project 20-24(74), Administration of Highway and Transportation Agencies.
Dixon, M. (2006). "APM Project Management Body of Knowledge", Association for Project Management, Peterborough, England.
Federal Highway Administration, (2006). "Risk Assessment and Allocation for Highway Construction Management". Report No. FHWA-PL-06-032. Prepared by Ashley, D. B., Diekmann, J. E., Molenaar, K. R. Federal Highway Administration, Washington, DC.
Federal Highway Administration, (2012a). "Risk-Based Transportation Asset Management: Evaluating Threats, Capitalizing on Opportunities", Report 1: Overview of Risk Management. Report No. FHWA-HIF-12-035. Prepared by Proctor, G., & Varma, S. Federal Highway Administration, Washington, DC
Federal Highway Administration, (2012b). "Risk-Based Asset Management: Examining Riskbased Approaches to Transportation Asset Management Report", Report 2: Managing Asset Risks at Multiple Levels in a Transportation Agency. Federal Highway Administration, Washington, DC
Flyvbjerg, B., Holm, M. S., & Buhl, S. (2002). "Underestimating Costs in Public Works Projects: Error or Lie?" Journal of the American planning association, 68(3), 279-295.
Franklin, P. (2009, January). "Risk Analysis for Transportation Projects", In Reliability and Maintainability Symposium, 2009. RAMS 2009. Annual (pp. 274-278). IEEE.
195

Hallowell, M. R., Molenaar, K. R., & Fortunato III, B. R. (2012). "Enterprise Risk Management Strategies for State Departments of Transportation", Journal of Management in Engineering, 29(2), 114-121.
Heimann, C. L. (2010). "Acceptable Risks: Politics, Policy, and Risky Technologies", University of Michigan Press.
Kahkonen, K. (1997). "Implementation of Systematic Project Risk Management In Companies From Immediate Needs to The Prospects of the Future", In Managing Risks in Projects. In Proceedings of the IPMA Symposium on Project Management. E&FN Spon: London, 75-83.
Kangari, R. (1995). "Risk Management Perceptions and Trends of US Construction", Journal of Construction Engineering and Management, 121(4), 422-429.
Kim, S. and Bajaj, D. (2000). "Risk Management in Construction: An Approach for Contractors in South Korea", Cost Engineering, 42(1), 38-44.
Lofstedt, R. E. (2004). "Risk Communication and Management in the Twenty-First Century", International Public Management Journal 7(3), 335-346
Lynos, T. and Skitmore, M. (2004). "Project Risk Management in the Queensland Engineering Construction Industry: A Survey", International Journal of Project Management, 22 (1), 51-61.
Manski, C. F. (2011). "Policy Analysis with Incredible Certitude". The Economic Journal, 121(554), F261-F289.
196

McGoey-Smith, A., Poschmann, A., & Campbell, L. (2007). "Quantitative Risk Assessment and Risk Management of a Large Transportation Project", In Transport Association of Canada, Annual Meeting, Saskatoon, Canada.
Minnesota DOT (2008). "Cost Estimation and Cost Management." Technical Reference Manual.
Molenaar, K. R. (2005). "Programmatic Cost Risk Analysis for Highway Megaprojects", Journal of Construction Engineering and Management, 131(3), 343-353.
Molenaar, K. R., Anderson, S. D., & Schexnayder, C. J. (2010). "Guidebook on risk analysis tools and management practices to control transportation project costs", NCHRP 658: Transportation Research Board.
Molenaar, K. R. and Harper, C. M. (2013). "Cost Estimating and Cost Management Implementation Review." Research Report for Minnesota Department of Transportation.
Molenaar, K., Loulakis, M., & Ferragut, T. (2014). "Guide for the Process of Managing Risk on Rapid Renewal Projects", SHRP 2 Report S2-R09-RW-02.
Montana Department of Transportation (2014). "Risk Management Guidelines: Managing Project Costs through Identification and Management of Risks." Montana Department of Transportation, Helena, MT.
Moynihan, D. P. (2005). "Goal-Based Learning and the Future of Performance Management", Public Administration Review, 203-216.
Network of Major European Cities (2013), "Pilot Project: Literature Review on Research and Study Results Relevant to SUTP". 197

New South Wales (NSW) Government (2011), "Project Risk Management Guideline", Department of Finance and Services.
Project Management Institute. (2013). "A Guide to the Project Management Body of Knowledge (PMBOK Guide)". Project Management Institute, Newton Square, PA.
Salling, K. B., & Leleur, S. (2012). "Modelling of Transport Projects Uncertainties: Risk Assessment and Scenario Analysis", European Journal of Transport and Infrastructure Research, 12(1), 21-38.
Senesi, C., Javernick-Will, A., & Molenaar, K. (2012). "Barriers to applying probabilistic risk analysis in design and construction projects", In Construction Research Congress West Lafayette, Indiana.
Standards Australia/Standards New Zealand (2004). "Australian New Zealand Standard AS/NZS 4360:2004: Risk Management", Homebush, NSW: Standards Australia/Wellington: Standards New Zealand.
Standard Organization for Standardization. (2009). "Committee Draft of ISO31000 Risk Management", Standard Organization for Standardization.
Strassman, P., & Wells, J. (1988). "The Global Construction Industry". Croom Helm, London.
Touran, A., & Lopez, R. (2006). "Modeling Cost Escalation in Large Infrastructure Projects", Journal of construction engineering and management, 132(8), 853-860.
Tran, D. D., & Molenaar, K. (2012, May). "Critical Risk Factors in Project Delivery Method Selection for Highway Projects", In Construction Research Congress. West Lafayette, Indiana. 198

U.K. Highways Agency, (2001). "Highways Agency Framework for Business Risk Mangement". Report of Highways Agency, London, England.
US Army Corps of Engineers (2009). "Cost and Schedule Risk Analysis Guidance." Washington Department of Transportation. (2014). "Project Risk Management Guide",
Washington Department of Transportation, Olympia, Washington. World Road Association. (2012). "Managing Operational Risk in Road Organization", World
Road Association. Wu, Q. (2006). "Transportation Infrastructure Project Cost Overrun Risk Analysis", Doctoral
dissertation, University of British Columbia, Vancouver, Canada.
199