Strategies and resources for strengthening the implementation of the construction quality acceptance firm (CQAF) model in the innovative project delivery environment

GEORGIA DOT RESEARCH PROJECT 19-27
FINAL REPORT
STRATEGIES AND RESOURCES FOR STRENGTHENING THE IMPLEMENTATION
OF THE CONSTRUCTION QUALITY ACCEPTANCE
FIRM (CQAF) MODEL IN THE INNOVATIVE PROJECT DELIVERY ENVIRONMENT
OFFICE OF PERFORMANCE-BASED MANAGEMENT AND RESEARCH
600 W. PEACHTREE STREET NW ATLANTA, GA 30308

TECHNICAL REPORT DOCUMENTATION PAGE

1. Report No. FHWA-GA-21-1927

2. Government Accession No. N/A

3. Recipient's Catalog No. N/A

4. Title and Subtitle Strategies and Resources for Strengthening the Implementation of the Construction Quality Acceptance Firm (CQAF) Model in the Innovative Project Delivery Environment

5. Report Date March 2021
6. Performing Organization Code N/A

7. Author(s) Baabak Ashuri, Evan Mistur, Jung Hyun Lee, and Lier Liu

8. Performing Organ. Report No. 19-27

9. Performing Organization Name and Address Georgia Institute of Technology Office of Sponsored Programs 505 Tenth Street, NW Atlanta, Georgia 30332-0420

10. Work Unit No. N/A
11. Contract or Grant No. P.I. NO.: 0016970

12. Sponsoring Agency Name and Address Georgia Department of Transportation Office of Performance-based Management and Research 600 W. Peachtree Street NW Atlanta, GA 30308

13. Type of Report and Period Covered Final Report (November 2019-March 2021)
14. Sponsoring Agency Code N/A

15. Supplementary Notes Prepared in cooperation with the U.S. Department of Transportation, Federal Highway Administration.

16. Abstract Considering the social and economic values of high-quality transportation infrastructure in the U.S., major projects delivered using innovative project delivery require a new model to ensure effective quality management. Several State Departments of Transportation (DOTs) have adopted a new quality management system requiring use of the developers' construction quality acceptance firm (CQAF), also known as an independent quality firm (IQF). Better understanding is needed to develop innovative methods of conducting quality assurance (QA) in the construction engineering and inspection (CEI) industry. Therefore, this research aims to provide guidance on ways to strengthen the QA process in the innovative project delivery environment. To identify the current state of understanding in the CQAF model compared to traditional quality management, this study conducted a combination of content analysis, survey, and follow-up interviews. The content analysis identified the major differences in the approach taken by different state DOTs and compiled the primary similarities and differences in QA between designbuild (DB) and publicprivate partnership (P3) projects. A survey of the CEI industry provided valuable lessons for future improvement in quality management services in federal-aid designbuild projects, and we further consolidated our interpretation of QA topics and investigated strategies for QA at GDOT through in-depth interviews with 15 survey participants. Our findings offer a better understanding of the roles and responsibilities of the CQAF model and contribute to adopting the new quality assurance program and enhancing the CQAF model for current and future projects in the DB and P3 environments.

17. Key Words

Quality Assurance; Design-Build; Public-Private Partnership;

Project Delivery

19. Security Classification (of this report)

20. Security Classification (of this page)

Unclassified

Unclassified

Form DOT F 1700.7 (8-72)

18. Distribution Statement No restrictions.

21. No. of Pages: 22. Price:

74

Free

Reproduction of completed page authorized

GDOT Research Project RP 19-27 / T.O. 2014-96
Final Report
STRATEGIES AND RESOURCES FOR STRENGTHENING THE IMPLEMENTATION OF THE CONSTRUCTION QUALITY ACCEPTANCE FIRM (CQAF) MODEL IN THE INNOVATIVE PROJECT DELIVERY ENVIRONMENT
By Baabak Ashuri, Ph.D., DBIA
Evan Mistur, Ph.D. Jung Hyun Lee Lier Liu
Georgia Institute of Technology Contract with
Georgia Department of Transportation
In cooperation with U.S. Department of Transportation Federal Highway Administration
March 2021
The contents of this report reflect the views of the authors who are responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the Georgia Department of Transportation or the Federal Highway Administration. This report does not constitute a standard, specification, or regulation.
ii

TABLE OF CONTENTS
LIST OF FIGURES ....................................................................................................................v
LIST OF TABLES .....................................................................................................................vi
EXECUTIVE SUMMARY .........................................................................................................1
CHAPTER 1 INTRODUCTION .................................................................................................5 Research Background ............................................................................................................5 Research Objectives ..............................................................................................................6 Research Literature ...............................................................................................................7 Quality Management in the DesignBuild Environment ..................................................7 Changes from Traditional Delivery Methods .................................................................10 Quality Management and Quality Assurance .................................................................11 Challenges for Effective Quality Assurance...................................................................13
CHAPTER 2 RESEARCH METHODOLOGY .........................................................................18 Research Objectives ............................................................................................................18 Research Roadmap ..............................................................................................................19
CHAPTER 3 CONTENT ANALYSIS ......................................................................................21 Purpose ...............................................................................................................................21 Materials and Methods ........................................................................................................21 DesignBuild ......................................................................................................................24 PublicPrivate Partnership (P3) ...........................................................................................25
CHAPTER 4 SURVEY ANALYSIS.........................................................................................27 Overview............................................................................................................................. 27 Survey Demographics .........................................................................................................28 Survey Responses................................................................................................................30 CEI Firms' Roles and Responsibilities...........................................................................30 Stringency of CEI Firms' Decisions...............................................................................34 Challenges for CEI Firms ..............................................................................................36 Advantages for CEI Firms .............................................................................................40 Owner Verification Firms' Roles, Responsibilities, and Values .....................................42 Respondent Recommendations ......................................................................................46 Discussion ........................................................................................................................... 48
CHAPTER 5 INTERVIEW SUMMARY ..................................................................................50 Overview............................................................................................................................. 50 Interview Protocols .............................................................................................................51 Interview Responses............................................................................................................52
iii

Typical Roles and Responsibilities of CEI Firms ...........................................................52 Areas of Improvement for CEI Firms' Understanding....................................................52 How DOTs Reduce Gaps in Misunderstanding ..............................................................53 Most Impactful Challenges for CEI Firms......................................................................53 Stringency of CEI Firm's Decisions...............................................................................54 Biggest Challenges in the QA Process ...........................................................................54 Owner Verification Firms' Roles and Impacts ...............................................................54 Discussion ........................................................................................................................... 55 CHAPTER 6 CONCLUSIONS .................................................................................................57 ACKNOWLEDGMENTS ......................................................................................................... 61 REFERENCES .........................................................................................................................62
iv

LIST OF FIGURES
Figure 1. Pie chart. Survey respondents' professions. ................................................................29 Figure 2. Bar graph. Understanding discrepancy regarding CEI firms' tasks..............................32 Figure 3. Bar graph. Understanding the inconsistency regarding CEI firms' responsibilities. .....33 Figure 4. Bar graph. CEI professionals' different views on CEI firm's responsibilities when
hired by DB versus DOT.......................................................................................................33 Figure 5. Stacked bar graph. Stringency of CEI firms' quality acceptance decisions. .................35 Figure 6. Stacked bar graph. Stringency of CEI firms' administrative oversight. .......................36 Figure 7. Stacked bar graph. OVF's responsibilities. .................................................................44 Figure 8. Bar graph. OVF and CEI firms' contradiction.............................................................45 Figure 9. Bar graph. OVF's added value....................................................................................46 Figure 10. Pie chart. Professional background of interview participants.....................................51
v

LIST OF TABLES
Table 1. Archival data sample. ..................................................................................................22 Table 2. Key quality process characteristics. .............................................................................23 Table 3. Designbuild quality management practices. ...............................................................25 Table 4. P3 quality management practices. ................................................................................26 Table 5. Respondents' experience. ............................................................................................30 Table 6. Respondents' preferences for quality management models...........................................30 Table 7. Relative significance of challenges for CEI firms.........................................................38 Table 8. Relative significance of issues related to project shutdowns. ........................................40 Table 9. Relative significance of benefits for CEI firms.............................................................42
vi

EXECUTIVE SUMMARY
Considering the significance of high-quality transportation infrastructure to social and economic success in the United States, major projects delivered using innovative project delivery, such as designbuild (DB) and publicprivate partnership (P3), require a new model for ensuring total quality management. To act as an integrated part of a quality assurance program (QAP), several state departments of transportation (DOTs) have adopted a new quality management model requiring use of the developers' construction quality acceptance firm (CQAF), also known as an independent quality firm (IQF). A better understanding is needed of how novel methods of conducting quality assurance (QA) have developed in the construction engineering and inspection (CEI) industry. The overarching goal of this research is to provide guidance on ways to strengthen the QA process in the innovative project delivery environment. To accomplish this goal, first, we identified and analyzed the gaps between the DOT's expectations of CQAF roles and responsibilities and the industry's understanding of how the CQAF should function in the DB environment. Second, we identified the significant barriers to effective QA and developed appropriate solutions to enhance the new QA method's effective adoption in the CEI industry.
The research assignment began with an in-depth analysis of the current literature regarding published academic papers. These resources include federal and state reports published by various organizations, such as the Federal Highway Administration (FHWA) and National Cooperative Highway Research Program (NCHRP); numerous state DOTs' designbuild/P3 documents, including agreements, master contracts, requests for qualifications (RFQs), and requests for proposals (RFPs); and presentations and other published records from professional associations such as the DesignBuild Institute of America (DBIA). The next stage of this
1

research was a content analysis of DOT documents, including master contracts, RFQs, RFPs, quality management plans, and quality manuals. To enhance the understanding of the state of the practice in quality management in the DB and P3 environments, we selected the criteria representing each DOT's unique quality management practices based on project delivery types for the analysis. We chose the following five criteria: (1) CQAF model; (2) independent quality manager as key personnel; (3) designbuilder quality manager as key personnel; (4) quality manual/quality management plan (QMP) as evaluation criteria; and (5) authority to stop work. The results identified 12 different quality management practices in DB projects from 28 DOTs, and 5 quality management practices for P3 projects among 12 DOTs.
Following the content analysis stage, the research methodology continued with a CEI industry survey to analyze construction quality management practices in the DB environment. To understand industry perspectives, we surveyed subject-matter experts from different professional groups, including CEI experts, design consultants, general contractors, DOT staff, and owner's representatives. We analyzed the survey results to identify survey questions with high levels of variance or consistency. The major findings of the survey analysis are the following:
CEI's roles and responsibilities. The stringency of CEI firm's decisions. Challenges for the CEI firms. Advantages for the CEI firms. Owner verification firm's (OVF) roles, responsibilities, and values. Recommendations.
We identified survey questions with high levels of variance or consistency, as well as high divergence under different contracting structures. The industry has inconsistent beliefs
2

regarding CEI firms' typical roles and responsibilities when hired by designbuilders versus state DOTs. We observed conflicting perspectives about the stringency of CEI firms' quality acceptance decisions. CEI professionals, design consultants, and owner's representatives responded that quality acceptance decisions are relatively more stringent under the CQAF model than the traditional quality management model. On the other hand, most DOT personnel felt that the CEI firm's quality acceptance decision is substantially more lenient and somewhat more lenient under the CQAF model. The stringency in administrative oversight aligns with results showing discrepancies in contract requirements. It is necessary to reduce this gap in understanding how contracting influences the level of stringency of CEI firms' roles and responsibilities between the traditional quality management model and the CQAF model.
Our analysis also showed an inconsistent understanding of the OVF's roles and its impacts on how respondents are uncertain about the OVF's responsibilities. Many respondents agreed that the OVF and CEI firms' decisions rarely contradict; however, this is disproportionate to the fact that not many respondents agreed on the added values of the OVF. Some additional comments from the survey participants also provided valuable lessons and resources for future quality management services. To further study the survey topics and focus on inconsistencies in the survey responses, we conducted semistructured interviews with subject-matter experts who expressed their willingness to participate in our research and share their insights on quality management services.
We followed the survey analysis with the in-depth interviews and embedded qualitative analysis of experienced CEI perspectives. After completing the survey, we selected a subsample of experienced respondents and conducted semistructured interviews to reveal their understanding of QA topics introduced in the survey. This provided the opportunity to pursue
3

survey themes in more detail and explore new questions that the survey responses raised. We aimed to obtain opinions of the subject-matter experts on the following areas:
Typical roles and responsibilities of CEI firms. Areas of improvement for CEI firms' understanding. How DOTs reduce gaps in understanding. The most impactful challenges for CEI firms. The stringency of CEI firms' decisions. The biggest challenges in the QA process. The owner verification firm's roles and impacts.
The interviews help improve the alignment of the CEI industry and DOTs' understanding and lessen the gaps between the two. Some typical comments included having clear expectations and specific minimum requirements for every aspect of work, and that proactive strategies can effectively mitigate these problems. While some CEI professionals agreed on similar stringency levels of the CEI's decisions when hired by either the designbuilder or the state DOT, which is consistent with our survey results, a few others thought CEI firms tend to be more stringent when hired by the designbuilder. The interviewees believed that the OVF adds value and brings impact on QC and quality acceptance.
The findings of this research provide a better understanding of the roles and responsibilities of the CQAF model. This research contributes toward adopting the new quality assurance programs and enhancing the CQAF model for current and future projects in the designbuild and P3 environments.
4

CHAPTER 1 INTRODUCTION
RESEARCH BACKGROUND
High-quality transportation infrastructure is vital to social and economic prosperity in the United States. State departments of transportation (DOTs) around the country are responsible for fulfilling a continually rising demand for project development and maintenance (ASCE 2013, USDOT 2013). However, many DOTs struggle to keep up while using traditional project delivery methods alone. Traditional designbidbuild (DBB) processes involve numerous, serially connected tasks and are vulnerable to internal agency issues with staffing, funding, task complexity, and communications (An et al. 2018, Kingsley et al. 2017, Mallett and Luther 2011). These issues can lead to substantial delays and cost overruns for project delivery, making the DBB system unsuitable for some situations. In response, the Federal Highway Administration (FHWA) has approved a number of alternative contracting methods and, since 1990, has allowed state DOTs to use nontraditional contracting techniques under Special Experimental Project No. 14 (SEP-14) (FHWA 2019). In 2002, FHWA issued a final rule for designbuild (DB) project delivery systems that allows DB to be applied to qualified projects without SEP-14 approval and to unqualified projects when approved for use through SEP-14. Consequently, many DOTs are moving toward innovative methods of project delivery in order to increase efficiency and effectiveness of infrastructure development (Ashuri and Kashani 2012).
DB project delivery systems are becoming increasingly common in the U.S. (DBIA 2016) and are used, at varying capacity, in 46 states, as well as the District of Columbia (DBIA 2020). The DB environment is designed to lessen administrative burden (Gad et al. 2015) and reduce the transaction costs associated with high-quality infrastructure development (Vrijheof
5

and Ridder 2007). It can provide substantial benefits and has been linked to shorter schedules; reduced costs (Allen et al. 2002, Gransberg et al. 2003, Hale et al. 2009, Ibbs et al. 2003, Liang et al. 2019, Molenaar et al. 1999, Roth 1995, Sanvido and Konchar 1997, Warne 2005); and, in some cases, improved quality (Bennett et al. 1996, Ernzen and Feeney 2002) compared to traditional DBB systems. Other publicprivate partnership (P3) systems, such as designbuild finance (DBF) and designbuildfinanceoperatemaintain (DBFOM), offer numerous benefits, as well (Mostaan and Ashuri 2016b). These systems generate mutually beneficial, long-term contracts where private-sector entities provide operating services for the public sector (Garvin 2010), making it possible to achieve broader objectives, reduce prices, and make schedules shorter and more consistent (Brown et al. 2009).
However, questions remain over whether these innovative delivery methods improve all areas of project delivery. Uncertainty exists over the level of transaction costs incurred by the project owner in these project-delivery environments (Li et al. 2014, Whittington and Dowall 2006), raising questions about the potential costs attached to DB efficiency gains. While the quality of DB projects is required to be at least as high as that of commensurate DBB projects (Gransberg et al. 2008b), concerns persist over whether project quality remains at the same level (Ernzen and Feeney, 2003). These concerns demand in-depth investigation and analysis of DOT quality assurance (QA) practices to ensure that the high-quality standards of DOT infrastructure development are consistently maintained in the DB environment.
RESEARCH OBJECTIVES
Like all state DOTs, the Georgia Department of Transportation (GDOT) is responsible for orchestrating total quality management for public infrastructure projects. GDOT has already
6

implemented a new quality management model for DB projects, requiring the developers' construction quality acceptance firm (CQAF), also known as the independent quality firm (IQF), to act as an integrated part of the project QA. However, the impact of the new methods of QA for innovative project delivery systems is still under review. Better understanding is needed about how the novel methods of conducting QA have developed in the construction engineering and inspection (CEI) industry. It is critical to know how well members of the CEI industry understand their roles in this reformed process, as well as to identify the implementation challenges that the CEI industry faces in the innovative delivery environment. GDOT is interested in aligning agency and CEI industry expectations for this new QA method, identifying major challenges and issues facing the process and developing effective solutions for ensuring high QA standards.
The chief goal of this research is to provide guidance to GDOT on ways to strengthen the QA process in the innovative project delivery environment. In this study, we focus on two main objectives. First, we identify and analyze the gaps between GDOT's expectations of CQAF roles and responsibilities and the industry's understanding of how the CQAF should function in the DB environment. Second, we identify the major barriers to effective QA and develop appropriate solutions to enhance the effective adoption of the new QA method in the CEI industry.
RESEARCH LITERATURE
Quality Management in the DesignBuild Environment Quality management is a critical component of project delivery for public infrastructure development and is heavily driven by the efficacy of the QA methods employed on projects. While state DOTs recognize the importance of QA in ensuring project success (Gad et al. 2015),
7

the nuances and environmental factors guiding the process need to be understood in greater depth. The DB environment presents many unique challenges for ensuring project quality (Lee, Jallan, and Ashuri 2020). These characteristics must be thoroughly understood in order to consistently deliver high-quality infrastructure using innovative project delivery methods.
DB delivery methods are often used on large-scale, fast-paced development projects. This creates challenging constraints for QA personnel as they are tasked with overseeing extensive amounts of work on a rapid schedule. In DB projects, quality planners need to evaluate their risks (Ashuri et al. 2015). While major DB projects often carry large amounts of risk as highbudget, heavily used infrastructure, the rapidity of delivery tasks in the DB environment places additional stress on the QA process. Additionally, the flexibility inherent to DB operations makes it difficult to predict exactly what a project's final designs will entail, generating uncertainty that must be accounted for when conducting QA and requiring post-construction quality management to be undertaken.
These unique environmental constraints are accompanied by structural changes governing the responsibilities of the owner and designbuilder of projects (Ashuri et al. 2013). In the DB environment, the project owner (typically a DOT) minimizes its involvement in quality management and moves to an oversight role, leaving the designbuilder in charge of both quality control (QC) and quality acceptance. While quality acceptance is required to be completed by an independent third party and is often outsourced to a CQAF by the designbuilder, these changes create a significantly different organizational environment from that of traditional DBB projects. Due in part to these constraints, quality management plans (QMPs) vary by project owner and often look very different between different DOTs (Scott and Molenaar 2017). This results in a
8

high degree of heterogeneity between QA processes in use on public DB infrastructure projects in the U.S.
Other P3 methods in the DB environment come with their own unique requirements. In DBF projects, project financing is taken over by the designbuilder, allowing the project owner to defer payments until after the delivery process (FHWA 2020). This creates additional risk for the designbuilder as they are faced with new financial considerations and uncertainty associated with the project owner's future budgetary appropriations (Ashuri and Mostaan 2015). Similarly, DBFOM projects allow the designbuilder to assume both the project's financing and responsibility for its long-term operation and maintenance (Ashuri and Mostaan 2014). This system comes with risks similar to DBF, but it also gives the designbuilder a vested interest in the long-term quality of the project (FHWA 2020). DBFOM incentives to maximize project performance in the long run can encourage designbuilders to ensure high-quality QC and quality acceptance.
This added incentive to produce high-quality products can help ensure successful project QA, but P3 projects that do not have operations and maintenance components, such as DBF and standard DB projects, often lack adequate incentives for project quality in the QA process (Mostaan and Ashuri 2016a). While the structural requirements of DB and other P3 delivery methods create many differences in how they must approach quality management, there is a need for deeper understanding of how QA functions for each of these innovative delivery methods. The relevant similarities and differences of QA across these systems in U.S. DOTs are poorly understood and require additional comparative study.
9

Changes from Traditional Delivery Methods The typical QA process used for DBB project requirements imposed by DB, DBF, and DBFOM projects precludes them from using traditional QA methods. The flexibility of DB systems allows designbuilders to pursue different stages in the project development concurrently, but this leads to a lack of clarity about how the final design and budget of a project will look. This makes it difficult to estimate what QC efforts will be needed, making the already challenging QA process more difficult to plan for (Beard et al. 2001, FHWA 2006).
Additionally, the transfer of quality acceptance roles and responsibilities from the project owner to the designbuilder creates challenges for traditional quality acceptance methods in the DB environment. As the designbuilder takes on additional responsibility over the project and assumes liability for design (Ashuri et al. 2019, 2020; Gransberg et al. 2008b; Lee et al. 2020), it also takes charge of the QC and quality acceptance of the project. Thus, the project owner moves to an oversight role, giving it less control over the QA process as many of the checks and balances it has in DBB systems are eliminated (Gransberg and Molenaar 2004, 2008; Lee and Arditi 2006). Without the project owner exercising direct control over the project, quality acceptance must be completed by an independent party, such as a CQAF, which is outsourced by the designbuilder. This outside firm is directly responsible to the public and has a powerful position over the project process, but having it contracted by the designbuilder introduces the potential for principle-agent problems and perverse incentives to arise if it does not understand its status as fully independent. Furthermore, as the roles of the project owner and designbuilder shift, the responsibilities of project personnel change, creating ambiguity over what needs to be done in order to ensure effective QA. Institutional norms and organizational expectations can
10

hinder effective workflow if project personnel are slow to adapt to new expectations or struggle to understand their new responsibilities in the new environment.
Finally, heterogeneous QA standards at different state DOTs give rise to additional ambiguity over the expectations for agents working on DB projects in multiple states. While state differences allow DOTs to be flexible and cater their QA processes to unique project requirements (Gransberg and Molenaar 2004), the lack of uniformity can make QA roles, responsibilities, and standards more difficult to follow for designbuilders and CQAF personnel if DOTs fail to explicitly communicate their expectations. Understanding the extent and ramifications of these challenges in the DB environment is critical to ensure that the new processes and institutional arrangements these innovative delivery systems bring do not undermine the high standards of quality on public infrastructure projects in the U.S.
Quality Management and Quality Assurance As is the case for all construction projects, QA is essential for project success in infrastructure development (Chan et al. 2002). An essential component of the "iron triangle" of cost, time, and quality (Atkinson 1999), ensuring high quality of infrastructure design and construction is critical for effective project delivery. Failure in any component can undermine project outcomes, but poor quality can be particularly deleterious in public infrastructure projects for many reasons (Huges 2005). If the quality, reliability, and safety of public infrastructure projects are eroded, the public can be put at risk, opening state DOTs up to significant legal and financial liability. Innovative QA methods that effectively ensure project standards are critical for project managers to consider (Harman and Sillars 2013).
This has led to many changes to quality management in the DB environment. The need for innovative methods has spurred widespread reorganization of QA. In 1995, revisions to
11

Title 23, Part 637 of the Code of Federal Regulations (23 CFR 637) under FHWA allowed state highway agencies to use quality acceptance testing results from contractors in their acceptance decisions (FHWA 2016). Additionally, FHWA has moved away from the old quality control/quality assurance (QC/QA) terminology toward total "quality assurance" (FHWA 2008, 2012). Under QC/QA, the expectation was that project contractors are responsible for QC while project owners are responsible for QA. However, the shift away from this system toward more holistic QA emphasizes that quality control and assurance are not separate functions (FHWA 2008, 2012). This establishes that QC is a core element of any quality assurance program and that QA refers to the overall system for assuring project quality, redefining the quality management responsibilities of contractors.
In traditional DBB systems, quality acceptance is typically handled in-house by the DOT, but in the DB environment this role can be outsourced. Some state DOTs still choose to do QA in-house for DB projects, but most lack staff and resources to take on this process themselves, and they outsource it (Lee et al. 2020). DB contractors take on the responsibility of QC themselves, but need to hire an independent quality acceptance firm, often as a CQAF or an IQF, to fulfill other quality acceptance roles on the project. These include roles such as conducting testing, reporting, and ensuring contract fulfillment by the designbuilder. While the CQAF is directly contracted by the designbuilder, it is intended to represent DOT interests, so its independence from the contractor remains paramount. Once the CQAF has completed its quality acceptance process on the project, it must be verified by an external review. A second reviewer, often the owner's verification firm (OVF), double-checks project quality acceptance to validate the CQAF's work, paving the way for final project approval.
12

While in the DB environment the project owner transfers quality acceptance responsibilities to the contractor (Gransberg et al. 2008b), the level of transfer and precise form of these steps in the QA process varies between state DOTs (Scott and Molenaar 2017). Some DOTs require designbuilders to follow internal DOT manuals or agreements specifying quality management activities, while others require the designbuilder to develop their own quality assurance management plan (QAMP). Consequently, wide variation exists in quality acceptance roles at different DOTs (Gransberg et al. 2008a, Huges 2005).
Challenges for Effective Quality Assurance In addition to the structural and environmental constraints already discussed, DB projects are prone to several issues complicating project owners' attempts to effectively ensure project quality. Displaced responsibility, uncertainty, and ambiguity challenge QA in the DB environment (Ashuri et al. 2017). Transaction cost economics provide a useful framework to understand the root of these QA issues for public infrastructure development. All market interactions and service delivery systems accrue transaction costs (North and North 1992). These costs can either be explicit, incurring direct commitment from the service provider, or implicit, incurring indirect costs through changes in efficiency, effectiveness, or risk. For public agencies tasked with providing high-quality public infrastructure, the transaction costs associated with project delivery can guide the level of success achieved by numerous tasks, including QA. Generally speaking, the more transaction costs involved in a service or project process, the more difficult it will be for project personnel to produce efficient and effective outcomes.
Institutions governing how different service transactions are organized are key in determining the level of transaction costs (Williamson 1985). The DOT organizational systems structuring how projects are organized, assigned, and delivered are critical drivers of the costs
13

accrued, and, consequently, how efficiently and effectively those projects can be delivered. Innovative project delivery methods such as DB systems are designed to reduce administrative burden (Gad et al. 2015) and alleviate the amount of transaction costs incurred by traditional DBB methods (Vrijhoef and Ridder 2007). However, alternative delivery structures can create issues of their own. P3 systems can create behavioral issues and misplaced incentives as the responsibility for different project roles changes hands (Ho et al. 2015), and they have been linked to high transaction costs (Dudkin and Vlil 2006). Previous studies question whether DB methods are actually effective at reducing the transaction costs associated with organizing, administering, and implementing project functions. Some studies find that the costs are roughly the same as those in DBB systems (Li et al. 2014), while others estimate that they are actually higher in the DB environment (Whittington and Dowall 2006). Concerns about whether these costs undermine project quality consistently arise for DB project delivery (Ernzen and Feeney 2002). While DB methods facilitate the efficient delivery of a product, the added costs associated with QA that are inherent in the DB system may act as a barrier to the final quality of that product. This raises serious questions about the institutional arrangements guiding the QA process for innovative project delivery systems and demands an investigation of the transaction costs they can incur on project performance.
Displaced incentives are a major area of concern for project QA. The main difference between DB and DBB delivery systems is who holds responsibility over the project (Gransberg et al. 2008b). DB contractors are in charge of a large suite of project responsibilities, including QA, as the project quality roles are transferred to them from the project owner (Kraft and Molenaar 2013). Under this organization, QA is still meant to represent the project owner's interest, but the owner abandons authority over the process and moves to an oversight role
14

(Gransberg and Molenaar 2004, 2008; Lee and Arditi 2006). This creates misplaced incentives for the CQAFs conducting quality acceptance on projects. While they are designed to represent the project owner, CQAFs are hired and work directly for the DB contractor. CQAF personnel may be encouraged to side with the designbuilder and approve substandard work if disagreements arise over quality compliance. This institutional arrangement creates a perverse incentive to expediently approve projects rather than stand in the way of quality acceptance, even though they are intended to act as the project owner's safeguard against substandard production.
Project personnel, including those working for the CQAF, are boundedly rational and are restricted to making decisions based on the limited information, time, and cognitive capacity they have available (Greve and Agote 2015). Furthermore, they can be generally expected to behave opportunistically and pursue their own self-interest whenever possible. This makes it impossible to specify all work processes or predict the exact ways project personnel will approach their responsibilities ex ante during the contract formation stage. Bargaining is pervasive and contract fulfillment is impossible to completely specify up front (Williamson 1985). Even if DB contracts could be perfectly specified before a project began, their fulfillment would still be subject to project personnel's interpretation and opportunism when carrying them out. How CQAF personnel fulfill their contractual quality acceptance tasks will, in part, be defined by their working environment (e.g., implicit expectations of the DB contractor) and the institutional arrangements that frame that environment (e.g., incentives created by contract structure). This creates added costs for the CQAFs conducting quality acceptance. They are forced to navigate divergent incentives, interpret compliance with incomplete contracts, and learn the real expectations of the DB contractor they are hired by as well as the project owner
15

they represent. These activities can generate substantial search and bargaining costs for the CQAF while conducting project quality acceptance.
The lack of contract specificity in DB projects also introduces uncertainty to the QA process. Economic organization is an issue of contracting, but it is impossible to explicitly specify everything in a contract (Williamson 1985). Contracts simply cannot be written to predict every potential situation, least of all for DB projects. Quality management is especially prone to this issue, as the numerous moving parts in the DB process make it difficult to predict the final requirements necessary for QA (Beard et al. 2001, FHWA 2006). The flexibility and lack of final designs of DB projects make it difficult to appropriately specify a QAMP up front, generating uncertainty and the necessity for additional search costs. Liang and Ashuri (2020) suggested that straightforward payment mechanisms can reduce the ex post transaction cost incurred in contract enforcement.
In the DBB process, the costs of overcoming this uncertainty are internalized by the project owner conducting quality acceptance in-house; however, quality acceptance for innovative delivery methods is not able to follow this model (Harman and Sillars 2013). DOTs need to adapt to new methods (Gransberg and Molenaar 2004). Consequently, there is much variation in QA processes across project owner (Harman and Sillars 2013). This generates even more transaction costs for CQAFs as they must search for new information on how their QAMP should be modeled for each different project leader they work with. They must spend time and resources determining how best to comply with the implicit requirements, organizational norms, and quality standards of each project owner they work with. If these transaction costs are not dealt with, miscommunication can lead to discrepancies and undermine project quality.
16

Finally, changing roles and responsibilities in innovative project delivery systems create ambiguity and additional transaction costs for CQAF personnel. DB packages often fail to clearly define QA roles and responsibilities (Gransberg and Molenaar 2008). Organizational standards that are driven by agency norms and work culture are often left implicit and must be learned by external consultants. Furthermore, job expectations are often not explicitly communicated, leaving ambiguity over who is responsible for each part of QA and incus search costs for CQAF personnel to learn. Additionally, many project owners fail to make the mental shift to thinking about innovative project delivery methods as different from traditional methods (Scott 2006), and they approach DB quality management issues with a DBB mindset (Gransberg and Molenaar 2007).
It is critical that QA roles and responsibilities are clearly defined and understood (Gransberg et al. 2008a). Information asymmetries between CQAFs and DOT project owners can lead to QA requirements being unfulfilled or poorly performed, undermining project quality. Simultaneously, eliminating these asymmetries can be very costly for CQAFs, creating a barrier to completing the QA process effectively and also undermining project quality.
Innovative delivery methods offer many advantages but create a different set of institutional arrangements and procedures to deal with. These can incur significant transaction costs for project personnel and lead to complications for effective QA. Understanding the root of the transaction costs that lead to these challenges is important to finding new ways of improving performance in the DB environment.
17

CHAPTER 2 RESEARCH METHODOLOGY
RESEARCH OBJECTIVES
Deeper understanding of quality management for innovative project delivery methods is needed. This research project investigates the QA processes used on innovative projects at state DOTs across the U.S. to increase clarity about the process and examine problem areas involved in delivering high-quality infrastructure. The major objectives of this research are to:
Examine gaps between how GDOT and the industry understand the CQAF model of QA in the DB environment.
Analyze discrepancies between CEI understanding and GDOT expectations for QA roles and responsibilities.
Identify appropriate strategies to enhance the effective adoption of the CQAF model and improve QA performance.
We use a multifaceted research methodology in pursuit of these goals, relying on several methods of data collection and analysis to elicit understanding of the QA system. We employ a content analysis of archival data from nationwide DOTs to understand QA methods used for DB and P3 projects, survey data from different professional groups to reveal industry and state professional perspectives, and semistructured interviews with experienced QA personnel to analyze important aspects of the CQAF model. Coordinating these data sources through a diverse research design allows us to triangulate our results and ensure robust conclusions from our analysis. This makes it possible to develop a series of informed conclusions and recommendations for QA in the DB environment moving forward.
18

RESEARCH ROADMAP
We conduct a detailed review of the academic and professional literature to examine the current state of the practice for quality management in the innovative project delivery environment. This allows us to identify key differences between DB, P3, and DBB delivery methods and develop a structured analysis of QA for these systems. Each step of our research is informed by this review.
In chapter 3, we conduct a content analysis of DOT documents to develop a systematic understanding of the QA process in DB and P3 environments. We examine the population of U.S. DOTs that regularly employ DB, DBF, and DBFOM project delivery systems in order to identify key similarities and differences between different systems' quality management processes and pinpoint major differences between U.S. DOTs. Archival DOT documents provide high-resolution data describing different DOTs' QA methods. We sample these documents, code them using a structured protocol to capture relevant detail and variation, and analyze the data using a comparative analysis of project delivery systems with an embedded cluster analysis of state DOT practices. This analysis clarifies ambiguities related to the roles assigned to quality management personnel, provides a contextual understanding of the field of practice, and defines key issues for us to address in our examination of the QA process in later steps in this research. Furthermore, it allows us to develop a guide to QA for innovative delivery methods in the U.S.
In chapter 4, we conduct a survey of the CEI industry to analyze construction quality management practices in the DB environment. To understand industry perspectives, we survey subject-matter experts from different professional groups, including CEI experts, design consultants, general contractors, DOT staff, and owner's representatives. We then investigate internal variation between industry professionals and differences between CEI perspectives and DOT expectations for QA. By sampling CEI respondents from professional groups around the
19

U.S., we are able to gather a robust dataset to conduct our analyses. This involves a gap analysis of CEI and DOT expectations for QA tasks, roles, and responsibilities and a qualitative comparative analysis identifying issues and barriers in the process. The survey analysis allows us to identify discrepancies between how CEI firms understand their roles in QA and what DOTs expect from them, describe the major challenges facing effective implementation of QA in the DB environment, and determine appropriate strategies for improving the process.
In chapter 5, we follow the survey analysis with in-depth interviews and an embedded qualitative analysis of experienced CEI perspectives. This research step builds from the survey analysis by exploring its most critical topics in more depth. After completion of the survey, we select a subsample of experienced respondents and conduct semistructured interviews to reveal their understanding of QA topics introduced in the survey. This provides the opportunity to pursue survey themes in more detail and explore new questions that the survey responses raise. We code the data from these interviews and conduct a qualitative comparative analysis of expert perspectives, allowing us to triangulate the results from our survey analysis, increase our depth of understanding about QA topics, and develop more effective strategies for QA at GDOT.
Finally, in chapter 6, we synthesize the outcomes from across each step of our overall research plan. Here, we present our final conclusions, discuss our results, and provide a series of recommendations for GDOT to help overcome barriers to effective QA and improve quality management in the innovative project delivery environment.
20

CHAPTER 3 CONTENT ANALYSIS
PURPOSE
We conducted a content analysis of DOT procedures to develop a systematic understanding of QA in the innovative project delivery environment. QA is critical to the successful development of infrastructure projects (Gad et al. 2015). However, DB and P3 delivery methods make traditional systems for ensuring quality infeasible (Harman and Sillers 2013). The unique requirements of different innovative delivery systems make it impossible to apply traditional DBB methods of QA. Since different delivery systems, such as DB, DBF, or DBFOM, present unique challenges for QA (Lee, Jallan, and Ashuri 2020), they necessitate different strategies for ensuring quality compliance. It is crucial for practitioners to understand the differences between these systems in order to guarantee high-quality infrastructure development when they are used. To date, little work has examined QA in different project development environments. While some research has focused on how QA functions in P3 projects (e.g., Mostaan and Ashuri 2016a), a systematic analysis comparing DB and other P3 systems is needed. We used this content analysis to develop an in-depth understanding of the primary similarities and differences in QA between DB and P3 projects, identify the leading drivers of variance between systems, and identify major differences of approach between U.S. DOTs.
MATERIALS AND METHODS
We examined the population of U.S. DOTs that regularly employ DB and P3 (i.e., DBF and DBFOM) systems to investigate QA procedures. Using public archival evidence from these agencies, we explored key areas of variance between groups. First, we investigated variance
21

between the three delivery systems to see how they differ; then, we examined differences between U.S. DOTs to identify states with varying procedures and cluster them into groups.
The archival data we used consisted of DOT project documents, guidelines, and report outlining requirements in the QA process. We sampled these documents from DOT online archives to observe details of the QA process in place, focusing on DOT quality manuals, requests for proposals (RFPs), requests for qualifications (RFQs), QMP, key personnel listings, and P3 project reports post-2013. We sampled 96 documents from 36 states, offering a representative look at DOT practices and procedures around the United States. Table 1 outlines the states and the documents covered in our analysis.

DB P3 Total

Table 1. Archival data sample.

States 30 13 36

Documents 74 24 98

We systematically reviewed these documents for rule specifications, coding on explicit requirements related to QA tasks with a focus on key quality process characteristics. These characteristics came from an exhaustive list of items critical to the QA process that we drew from the professional literature. After analyzing a subset of documents from our sample to assess the accuracy and relevance of the items on this list, we reduced our focus to eight key characteristics, as outlined in table 2. This allowed us to explore the most important areas of divergence for QA in different DOT delivery systems.

22

Table 2. Key quality process characteristics.

No.

Characteristic

1

CQAF Model

2

Independent Quality Manager as Key Personnel

3

DesignBuilder Quality Manager as Key Personnel

4

Quality Manual / Quality Management Plan as Evaluation Criteria

5

Authority to Stop Work

6

Specified Key Personnel and Responsibilities

7

Organizational Structure

8

Line of Reporting

We coded each document on these characteristics qualitatively, conducting periodic intercoder reviews to ensure and maintain intercoder reliability. We compiled this information into a dataset of DOT QA practices, allowing us to conduct the content analysis. Finally, we combined the results to find similarities and differences across the DOTs and project delivery types. For the analysis, we selected the criteria that represent each DOT's unique quality management practices based on project delivery types. Among the eight characteristics shown in table 2, we selected the following five criteria: (1) CQAF model; (2) independent quality manager as key personnel; (3) designbuilder quality manager as key personnel; (4) quality manual / QMP as evaluation criteria; and (5) authority to stop work. The results found 12 different quality management practices in DB projects from the 28 DOTs. Also, we defined any forms of designbuild with a financing component as P3, which includes DBFOM and DBFM; the results indicated 5 different types of quality management practices in P3 projects among 12 DOTs.

23

DESIGNBUILD
Based on the content analysis, table 3 presents 12 different subsets of the state of the practice in quality management for designbuild projects. The results show that half of the 12 subsets have used a CQAF model. Although the quality management trend indicates an evenly equal number of the overall subsets between the traditional and CQAF models, most state DOTs still implement the traditional quality management model. Only 8 of 28 state DOTs implement the CQAF model. This shows that the CQAF model is still a relatively new concept in the DB environment.
In those six groups that have used the CQAF model, more than half of the groups require an independent quality manager as key personnel. A designbuilder quality manager is not seen as a required key person across those six groups, except by the Maine DOT. Also, only the Maine DOT requires both an independent quality manager and a designbuilder quality manager as key personnel. A quality management plan seems to be an essential evaluation criterion, either at the RFQ or RFP phase, because five of the six CQAF groups contain it in the evaluation phase. Four DOTs--Mississippi, Texas, Virginia, and Louisiana DOTs--are explicit that a quality manager has authority to stop work.
When it comes to the traditional quality management model, the DOTs require a design builder quality manager as key personnel, rather than an independent quality manager. The trend here shows that half of the groups that have used the traditional model require the designbuilder quality manager as key personnel. The total number of DOTs is slightly higher than those who do not need a designbuilder quality manager as key personnel, representing 11 and 9 DOTs, respectively. Like key personnel, the trend in QMP also indicates that half of the traditional groups evaluate the QMP. Considering the actual number of DOTs, 13 DOTs evaluate QMP
24

during the procurement phase. The remaining seven DOTs do not explicitly use the QMP provided by the designbuilder as an evaluation criterion. We observed that only two of the traditional groups indicate that a quality manager has the authority to stop work. Compared to the CQAF groups, the traditional groups tend not to transfer this authority to the designbuilder.

Table 3. Designbuild quality management practices.

No.

CQAF Model

Independent Quality
Manager as Key
Personnel

Design Builder Quality Manager as Key Personnel

QMP as Evaluation
Criteria

Authority to Stop Work

State DOT

1

Yes

Yes

Yes

Yes

No

ME

2

Yes

Yes

No

Yes

Yes

MS, TX

3

Yes

Yes

No

Yes

No

UT, AR

4

Yes

Yes

No

No

Yes

VA

5

Yes

No

No

Yes

Yes

LA

6

Yes

No

No

Yes

No

VT

CA, NE,

7

No

No

Yes

Yes

Yes

MI, NM,

AZ

MO, NY,

8

No

No

Yes

Yes

No

AL, NH,

MT

9

No

No

Yes

No

No

NV

10

No

No

No

Yes

No

TN, KY, CT

11

No

12

No

No

No

No

Yes

WA

No

No

No

No

RI, SC, WI, FL

PUBLICPRIVATE PARTNERSHIP (P3)

We considered any designbuild forms with a financing component as a publicprivate partnership, including DBFOM and DBFM. The dataset for P3 is relatively smaller than those for DB. We reviewed 24 documents from 13 state DOTs and used 12 state DOTs for the analysis

25

because of their available relevant information from the documents. We obtained five subsets of quality management practices in the P3 environment. The second group in table 4 represents quality management practices widely used in most state DOTs. Four state DOTs--Texas, North Carolina, Ohio, and Georgia--require an independent quality manager as key personnel, not a designbuilder quality manager. The independent quality manager has the authority to stop work, as well, and those four DOTs evaluate a QMP during the procurement phase. The Pennsylvania DOT is impressive because it requires both independent and designbuilder quality managers; however, they do not have the authority to stop work.
The remaining three groups that keep with the traditional quality management model do not include an independent quality manager as key personnel. The three DOTs in the third group shown in table 4 require a designbuilder quality manager as key personnel, who is given the authority to stop work. Those three DOTs also evaluate a QMP. The fourth and fifth groups do not include quality managers as key personnel.

Table 4. P3 quality management practices.

No.

CQAF Model

Independent Quality
Manager as Key
Personnel

Design Builder Quality Manager as Key Personnel

QMP as Evaluation
Criteria

Authority to Stop Work

State DOT

1

Yes

Yes

Yes

Yes

No

PA

2

Yes

Yes

No

Yes

Yes

TX, NC, OH, GA

3

No

No

Yes

Yes

Yes

MI, AZ, CO

4

No

No

No

No

No

IN, MD, FL

5

No

No

No

No

Yes

VA

26

CHAPTER 4 SURVEY ANALYSIS
OVERVIEW
The research team conducted a survey of the CEI industry to further study construction quality management practices in the DB environment for federal-aid highway projects. We distributed the survey to a wide range of professional associations to achieve high-quality responses. The professional associations that participated in the survey are listed below:
American Council of Engineering Companies of Georgia. American Council of Engineering Companies of Georgia Partnership for Transportation
Quality (GPTQ) Task Force. American Society of Civil Engineering Claims Avoidance and Resolution Committee. American Society of Civil Engineering Construction Institute Board of Governors. American Society of Highway Engineers. DBIA Transportation and Aviation Markets Committee. DBIA P3 Committee. Transportation Research Board Committee on Quality Assurance Management
(AFH20). Transportation Research Board Joint Subcommittee on Quality Management for
Alternative Project Delivery (AFH20 (1)).
We obtained valuable opinions from subject-matter experts (SMEs) in quality management and investigated differences between the traditional quality management model and the alternative quality management model (i.e., the CQAF model). The survey participants included professionals who have experience in quality management across the nation to collect
27

reliable data input to analyze the survey results. Those professionals come from CEI firms, engineering firms, highway contractors, owner's representatives, and public owners. We investigated the gaps between the CEI industry and DOT expectations for QA's tasks, roles, and responsibilities and analyzed barriers and issues in the QA process. We aimed to identify survey questions with high levels of variance or consistency through the analysis of our survey results, indicating discrepancies or agreement in how SMEs understand the roles and responsibilities of the CEI firms in the QA process and DOTs' expectations of those duties. The survey also determined high divergence under different contracting structures when different actors hire the CEI firm. We planned to identify impactful challenges in implementing QA in the DB environment and develop effective strategies to improve the process. This analysis draws attention to four main subject areas concerning the CQAF model's performance in the design build environment, including roles and responsibilities of the CEI firms, CEI firm stringency, impactful challenges for the CEI, and roles of the owner verification firm.
SURVEY DEMOGRAPHICS
This survey was conducted as an in-depth analysis to investigate CEI practices in the design build environment. We received 106 sets of responses in total, of which 53 were relatively complete and useful for further analysis. Survey participants were sampled from nine professional engineering organizations across 25 states in the U.S. The participants' backgrounds included general contractors, design consultants, CEI specialists, agency employees (DOT staff), and owner's representatives. A pie chart, shown in figure 1, presents the professions of the survey respondents. Approximately one third of the respondents have worked in their current professions for over 10 years, and 90 percent of the participants expressed their familiarity and
28

prior experience with designbuild project delivery. Around 60 percent of the participants mentioned that about a quarter of their firms' projects are federal-aid designbuild projects. A majority of them have worked for projects in multiple states across the nation. Through their experience providing construction quality management services, shown in table 5, approximately 40 percent have experience in both the traditional quality management model (hired by DOTs) and the CQAF model (employed by the designbuilder). When asked about the participants' preference of their employer, table 6 indicates that 43 percent of the participants stated that they prefer working with state DOTs to working for the designbuilder when performing construction quality management services. This preference is 22 percent higher than working for the design builder, suggesting respondents have better contractual experiences with the state DOTs.
Figure 1. Pie chart. Survey respondents' professions.
29

Profession
CEI Firms Design Consultants General Contractors
DOT Staffs Owner's
Representatives

Table 5. Respondents' experience.

Traditional Quality Management 14.29 25 75 90

CQAF
7.14 31.25 N/A N/A

16.67

16.67

Both
78.57 43.75
25 10 66.67

Table 6. Respondents' preferences for quality management models.

Profession
CEI Firms Design Consultants General Contractors
DOT Staffs Owner's Representatives

Traditional Quality Management
Employed by State DOTs (%) 35.71
41.18
100
44.44
33.33

CQAF Model Employed by DesignBuilder (%)
35.71 29.41 N/A N/A N/A

No Preference (%)
28.57 29.41 N/A 55.56 66.67

SURVEY RESPONSES
CEI Firms' Roles and Responsibilities Regarding CEI firms' roles and responsibilities, we asked the survey question, "How frequently is the CEI firm responsible for the following tasks in federal-aid designbuild projects?" We observed the frequency of the following tasks: (1) notifying the DOT of key times in the quality management schedule; (2) issuing noncompliance reports (NCRs) to address deficiencies in the materials; (3) exercising the approved engineering judgment to accept deficiencies in the material test results; (4) auditing quality management procedures and records; (5) ensuring
30

compliance of project payroll; (6) ensuring compliance of report submission; (7) ensuring contract compliance; and (8) conducting construction measurements to certify payments to the designbuilder. We found that the respondents have inconsistent beliefs about the CEI firms' roles and responsibilities, indicating a gap in understanding between the traditional quality management model and the CQAF model. Our survey demonstrated on four out of eight tasks that the industry has inconsistent beliefs regarding CEI firms' responsibilities. Figure 2 shows the inconsistency in the understanding of the CEI's tasks in the four areas of auditing quality management procedures and records, ensuring project payroll compliance, ensuring contract compliance, and conducting construction measurements to certify payments to the design builder.
Per contract requirements, achieving disadvantaged business enterprise (DBE) goals is critical for the public owner (Amekudzi-Kennedy et al. 2016). We observed whether CEI firms are responsible for ensuring DBE requirements and the DavisBacon Act. The survey addressed the following question: "How frequently is the CEI firm responsible for ensuring the following aspects of the contract in federal-aid designbuild projects?" We found discrepancies in ensuring the designbuilder's compliance with the contract's goal for using DBEs and paying the local prevailing wages on public works projects for laborers and mechanics (i.e., the DavisBacon Act). Figure 3 shows the differences in respondents' perspectives on CEI firms' responsibilities. A focus on future educational materials on these tasks can help reduce the gap in CEI industry and DOT understanding. Additionally, our survey revealed that CEI personnel understand that their roles and responsibilities differ depending on whether they are hired directly by the design builder or the DOT. Figure 4 indicates that CEI personnel are far more likely to consider the following responsibilities: ensuring the designbuilder's compliance to the contract's goal for
31

using DBEs and paying the local prevailing wages on public works projects for laborers and mechanics.
Figure 2. Bar graph. Understanding discrepancy regarding CEI firms' tasks.
32

Figure 3. Bar graph. Understanding the inconsistency regarding CEI firms' responsibilities.
Figure 4. Bar graph. CEI professionals' different views on CEI firm's responsibilities when hired by DB versus DOT. 33

Stringency of CEI Firms' Decisions We investigated respondents' views on the stringency of quality acceptance decisions and the CEI firms' administrative oversight when the designbuilder employs them, compared to the typical enforced level of stringency when the state DOT directly hires the CEI firms for quality management. We asked, "On average, how stringent are the acceptance decisions of the CEI firm when it is directly hired by the designbuilder compared to the typical level of stringency enforced when it is directly hired by the state DOT?" We observed varying CEI firm stringency levels regarding quality acceptance under different contracting structures reported by CEI firms, as shown in figure 5. The CEI firms perform quality acceptance on behalf of the public owner under the traditional quality management model. When using the CQAF model, the design builder or the developer will hire CEI firms to perform quality acceptance. The CEI firm performs the same duties but works for different employers. Interestingly, about 30 to 40 percent of the CEI professionals, design consultants, and owner's representatives responded that quality acceptance decisions are substantially more stringent and somewhat more stringent under the CQAF model than the traditional quality management model. On the other hand, more than 60 percent of the DOT personnel felt that the CEI firm's quality acceptance decision is substantially more lenient and somewhat more lenient under the CQAF model. These results indicate an inconsistency in understanding among CEI personnel, and such differences in stringency may result from misplaced incentives in the process. This inconsistency may indicate a lack of clarity about how contracting influences the level of stringency of CEI firms' acceptance decisions.
We then observed CEI firms' stringency on administrative oversight by asking, "On average, how stringent is the administrative oversight of the CEI firm when it is directly hired by the designbuilder compared to the typical level of stringency enforced when it is directly hired
34

by the state DOT?" Across different professions, figure 6 indicates that a considerable portion of survey participants disagreed on the same or similar level of stringency of the administrative oversight of the CEI firm under the CQAF model, compared to the traditional quality management model. This discrepancy aligns with the results of the contract requirements that we discussed in the previous chapter. It also shows that developing future educational materials on these tasks can reduce the understanding gap between the CEI industry and the DOTs.
Figure 5. Stacked bar graph. Stringency of CEI firms' quality acceptance decisions.
35

Figure 6. Stacked bar graph. Stringency of CEI firms' administrative oversight.
Challenges for CEI Firms In questions 21 to 23, we aimed to gather individuals' perspectives on challenges by asking, "How frequently do each of the following challenges impact the CEI firm's ability to perform quality management services?" The survey question asked for weighting of the element of five potential challenges: (1) designbuilder reluctance to accept the role of independent quality acceptance firm; (2) owner reluctance to accept role of quality acceptance firm; (3) limited understanding about the new dynamics of designbuild project delivery (e.g., new roles and responsibilities); (4) limited understanding of state DOT's expectations and norms for quality management; and (5) fixed amount considered for conducting quality management services.
To identify each element's relative significance based on a question's cumulative responses, we applied the metric of relative importance index (RII) to represent CEI personnel opinions on challenges by comparing the traditional quality management model and the CQAF
36

model. The RII method (Amarasekara et al. 2018, Awwad et al. 2016, Jarkas et al. 2012) is used to determine the overall importance of challenges from the perspective of the survey respondents in each industry group. The RII value allows us to identify the most significant factors based on experts' responses and rankings of all factors (Shah et al. 2014). Equation 1 shows the specific RII equation designed for this survey:

= 55+44+33+22+11

(1)

5

where, n1, n2, n3, n4, and n5 represent respondents who selected "Never," "Rarely," "Sometimes," "Often," and "Always," respectively, and N is the total count of respondents. The greater the RII value for a challenge, the more critical the challenges from the perspective of the respondents. The values were calculated and ranked for the significance of the CEI firms' challenge from the highest to the lowest. Table 7 summarizes the overall rankings of the potential challenges for CEI firms from the overall respondents. Understanding these differences more thoroughly can help DOTs prepare to mitigate for these challenges more effectively in project delivery.

37

Table 7. Relative significance of challenges for CEI firms.

Rank

When hired by DB

Challenges

RII

Limited understanding of

1

state DOT's expectations and norms for quality

0.6364

management

Fixed amount considered

2 for conducting quality

0.6272

management services

Limited understanding

about the new dynamics of

3 designbuild project

0.5773

delivery (e.g., new roles

and responsibilities)

Owner reluctance to 4 accept role of quality
acceptance firm

0.5556

Designbuilder reluctance

5

to accept the role of independent quality

0.5511

acceptance firm

When hired by DOT

Challenges

RII

Limited understanding about the new dynamics of designbuild project delivery (e.g., new roles and responsibilities)

0.5590

Limited understanding of state DOT's expectations and norms for quality management

0.5487

Designbuilder reluctance to accept the role of independent quality acceptance firm

0.5350

Fixed amount considered for conducting quality management services

0.5179

Owner reluctance to accept role of quality acceptance firm

0.4450

The results indicate the inconsistencies in the significances of challenges that the CEI firms may encounter while performing quality assurance tasks, depending on their employer. When implementing the CQAF model, the respondents think that the fixed amount considered for conducting quality management services is the most critical challenge. Some respondents mentioned that most public DB projects are derived from a price-driven nature. We noticed that each stakeholder might interpret their roles and responsibilities differently. For instance, one said that the DOT's project management consultant and the OVF might have different views toward the tasks that are assigned to the firm. Also, the designbuilder's reluctance to accept the independent quality acceptance firm's role is the second-most considerable challenge from the respondents.
38

We received comments from several respondents. Respondents who provided comments from all different professional groups expressed similar challenges. Some respondents mentioned that many professionals are not familiar with the CQAF model and hesitate to issue an NCR. Contractual ambiguity can also result in potential challenges; these challenges were further described as staffing plans, organizational structure, a line of reporting, definition of processes surrounding acceptance, and engineering judgments. When the state DOTs hire the CEI firm for quality assurance (i.e., the traditional quality management model), the most frequent challenges are limited understanding of the state DOT's expectations and norms for quality management and limited understanding about the new dynamics of designbuild project delivery (e.g., new roles and responsibilities). In terms of challenges that impact the CEI firm's ability when implementing the traditional QA model, most of the respondents mentioned that there are no differences from designbidbuild. CEI firms and general contractors expressed concerns that the owner may push the CEI inspection requirements to an elevated level in DB projects. We also received concerns from owners that CEI firms and designers tend not to be involved in dictating solutions or correcting issues.
Concerning project shutdowns, we aimed to identify the significant issues by asking, "How frequently are project shutdowns triggered by each of the following issues?" The survey question includes the frequency of seven issues that can trigger project shutdown: (1) breach of contract; (2) design errors; (3) noncompliance with design specifications; (4) faulty materials/workmanship; (5) noncompliance with construction specifications; (6) failure to disclose information; and (7) safety concerns. We then observed high consistency in responses when hired by either the designbuilder or the state DOTs. Applying the RII metric, we found when hired by either the designbuilder or state DOT, respondents agreed that the most
39

significant impacts come from safety concerns, noncompliance with construction specification, and faulty materials/workmanship. Table 8 indicates such relative importance ranking among all issues related to project shutdowns. The results show that survey respondents with different professions still had similar views on the issues closely related to project shutdowns. They believed safety concerns, noncompliance with construction specifications, and faulty materials/workmanship are most closely related to project shutdowns. To further investigate and detect differences between professional groups, we also calculated each listed issue's relative importance index from the survey respondents' perspective in each industry group. The result of each group's RII almost aligns with the entire group.

Table 8. Relative significance of issues related to project shutdowns.

Rank

When hired by DB

Challenges

RII

1 Safety concerns

0.5833

2

Noncompliance with construction specification

0.5833

3

Faulty materials/ workmanship

0.5833

4

Noncompliance with design specifications

0.5389

5 Design errors

0.5511

6

Failure to disclose information

0.4333

7 Breach of contract

0.4111

When hired by DOT

Challenges

RII

Faulty materials/ workmanship
Noncompliance with construction specification

0.5944 0.5889

Safety concerns

0.5833

Noncompliance with design specifications
Design errors
Failure to disclose information
Breach of contract

0.5611 0.4833 0.4778 0.4389

Advantages for CEI Firms We aimed to gather individuals' perspectives on whether CEI firms benefit from several advantages in performing quality management services in federal-aid designbuild projects. To identify their relative significance, this survey question was added: "How frequently does the
40

CEI firm benefit from each of the following advantages in performing quality management services in federal-aid designbuild projects?" The question provided four potential options of advantages that can bring benefits to the CEI firms: (1) innovative approaches to quality management; (2) incentives to provide enhanced performance; (3) implementing advanced technologies; and (4) project schedule sensitivity. We used the relative importance index method to obtain the relative significance of each element based on the cumulative responses to a question.
Table 9 shows the difference in rankings of benefits for the CEI firms when either the designbuilder or the state DOTs hire the CEI firms for quality assurance. Understanding these differences more thoroughly can help DOTs prepare to harness these characteristics more effectively in project delivery. The respondents had different views when it comes to project schedule sensitivity and innovative approaches to quality management. CEI personnel reported that sources of benefits such as project schedule sensitivity and implementing advanced technologies arise more often when the firm is hired directly by the designbuilder. In contrast, when the state DOTs hire CEI firms for quality assurance, innovative approaches to quality management were reported as the most significant benefit source.
41

Table 9. Relative significance of benefits for CEI firms.

When hired by DB

Rank

Challenges

RII

1

Project schedule sensitivity

0.6528

2

Implementing advanced technologies

0.6070

3

Innovative approaches to quality management

0.5840

4

Incentives to provide enhanced performance

0.5500

When hired by DOT

Challenges

RII

Innovative approaches to quality management

0.6320

Implementing advanced technologies

0.5886

Project schedule sensitivity
Incentives to provide enhanced performance

0.5704 0.4667

Owner Verification Firms' Roles, Responsibilities, and Values As the owner has ultimate responsibility for verifying that the project complies with the contract document, the owner or its representative will perform owner verification sampling, testing, and inspection to confirm that the CQAF acceptance is completed per the approved construction quality management plan (GDOT 2014). On behalf of the owner, the owner's verification functions oversee the CQAF inspections and validate or verify the QC/QA test results (GDOT 2014). After we examined the CEI firms' roles and responsibilities as the quality assurance position that determines quality acceptance, we inquired about the CEI firms' duties as the OVF. The survey questions obtained opinions about the roles, responsibilities, and values of the owner verification firms. In detail, we first asked, "How frequently is the OVF responsible for the following tasks in federal-aid designbuild projects?" and added the following question: "How frequently do the OVF's decisions contradict the primary CEI firm's decisions?" We then asked, "How frequently do the OVF's decisions create added value in the QA process?"
Concerning the CEI industry's understanding of the OVF's tasks, the question included eight elements: (1) notifying the DOT of key times in the quality management schedule;

42

(2) issuing noncompliance reports to address deficiencies in the materials; (3) exercising the approved engineering judgment to accept deficiencies in the material test results; (4) auditing quality management procedures and records; (5) ensuring compliance of project payroll; (6) ensuring compliance of report submission; (7) ensuring contract compliance; and (8) conducting construction measurements to certify payments to the designbuilder. Our results indicated inconsistency in the following items: (3) exercising the approved engineering judgment to accept deficiencies in test results, and (5) ensuring compliance of project payroll. Figure 7 describes the uncertainties we received. We observed relatively equally distributed answers from "Never" to "Always," and the answers followed no explicit pattern as to whether respondents consider these tasks as the OVF's responsibilities, as described in figure 7. One interesting element is ensuring compliance of project payroll. About 22 percent of the respondents selected it as the OVF's responsibility, while another 22 percent of the respondents thought the OVF is never asked to take on this duty.
43

Figure 7. Stacked bar graph. OVF's responsibilities.
Furthermore, our responses indicated that while most CEI personnel believe the OVF rarely contradicts the primary CEI firm's decisions, their responses about whether OVF decisions create added value to the quality assurance process varied widely. Figure 8 demonstrates such inconsistency that over 97 percent of the respondents believed OVF and CEI firms' decisions are rarely or sometimes contradicting each other. Simultaneously, there was no clear pattern of their attitudes toward the OVF's added values, as shown in figure 9.
In response to areas of disagreement between the OVF and the CQAF in quality acceptance, most of the respondents mentioned that the OVF and CEI firms disagree with nonconforming works, includeing correcting minor nonconformance items, resolving NCRs, allowing nonconforming work to continue, and materials acceptance using "engineering judgment." We also observed other comments on sources of disagreements between the OVF
44

and the CQAF from respondents with different backgrounds. Owner's representatives and general contractors mentioned the allowed means and methods and compliance with the plans and specifications. Owners and CEI firms commented about staffing and appropriate oversight levels in the field, and design consultants commented about design flexibility that does not precisely follow the DOT's preferred method. These variances may result in their current state of understanding concerning each party's role. Still, future education and materials that elaborate the public owner's expectations regarding the roles and responsibilities could increase the CEI industry's understanding of the role of the OVF and reduce the gap in understanding between the OVF and the CQAF.

How frequently do the OVF's decisions contradict the primary CEI firm's decisions?

70.00% 60.00%

58.33%

50.00% 40.00%

38.89%

30.00%

20.00%

10.00% 0.00%

0.00% Always

2.78% Often

Sometimes

Rarely

0.00% Never

Figure 8. Bar graph. OVF and CEI firms' contradiction.

45

50.00% 45.00% 40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00%

How frequently do the OVF's decisions create added value in the QA process?
45.45%

27.27%

27.27%

0.00% Always

Often

Sometimes

Rarely

0.00% Never

Figure 9. Bar graph. OVF's added value.

Respondent Recommendations To enhance the quality management services in federal-aid designbuild projects, we gathered recommendations from our survey respondents regarding four areas: (1) the reporting process; (2) reporting frequency; (3) communications with the designbuilder or DOT; and (4) the dispute resolution process. CEI experts and designers offered common recommendations on the reporting process, including a single document-storage location through a database, standardization of required areas that need to be reported, and reduction of the required paperwork. CEI experts, public owners, and owner's representatives suggested real-time cloud information sharing in the reporting process. With respect to reporting frequency, designers and owner's representatives recommended a risk-based approach, such as monthly reports for lowrisk items and weekly reports for high-risk items. At the same time, CEI experts mentioned that reporting should be as often as the data requires, use real-time dashboards on the job site, and comply with the contract. In addition, owner's representatives commented that reporting should follow the CEI contract or DOT manual(s), which are explicit but rarely read.

46

Regarding communications, CEI firms and owner's representatives mentioned that CEI firms should be, at a minimum, part of the weekly progress meeting during the construction phase. Owners and owner's representatives recommended co-location and providing one point of contact for each party, such as designbuilder, DOT, or the CEI firm as QA. Finally, CEI firms and owner's representatives provided advice on the dispute resolution process to include designing a formalized workflow, such as a tiered process, when dispute resolution is required. For instance, the CEI specialist working in Florida mentioned that the Florida DOT has a Dispute Resolution Board (DRB) on every designbuild project over $10 million and follows an issueescalation process specified in the RFP. Some other recommendations from the survey respondents included the use of additional software like Bluebeam for documentation of changes and compensation models. Two CEI personnel mentioned that this software encourages innovation. Additionally, DOT personnel commented that meetings between DOT and CEI firms before construction tend to help communication and CEI understanding.
We also received some additional comments regarding areas that we did not include in the question. CEI experts and general contractors proposed two ideas related to the cost of quality management services. CEI professionals proposed to set minimum bid amounts for quality management services and possible negotiation of a change order with owners for the quality scope upon award. General contractors suggested sharing costs for CEI services between the designbuilder and the DOT (i.e., an actual percent split on all costs). Comments on evaluation were made by CEI professionals and owners, where CEI professionals mentioned assigning higher quality-based value to the DB's CEI firm during procurement. In contrast, owners mentioned including the CEI proposal in RFQ/RFP reviews to ensure adequate qualifications and hours.
47

Furthermore, owners, designers, and CEI firms commented about roles and responsibilities. Owners emphasized field-level observations and providing field personnel requirements, while designers suggested reducing the paperwork and overlap between the OVF and the construction quality firm. Another two CEI professionals recommended adopting a datacentric approach that proves conformance to requirements and provides guidance for inspection oversight when acceptance is allowed by the designbuilder.
DISCUSSION
This chapter provides an in-depth analysis to study current understanding of the CEI's roles and responsibilities, the stringency of the CEI firm's decisions, challenges and benefits for the CEI firm, and the owner verification firm's roles and added values. We analyzed our survey results to identify survey questions with high levels of variance or consistency and high divergence under different contracting structures. The industry has inconsistent beliefs over CEI firms' typical roles and responsibilities, and different roles are identified when hired by designbuilders versus state DOTs. We observe conflicting perspectives about the stringency of CEI firms' decisions, resulting from the lack of clarity about how contracting influences the level of stringency of CEI firms' acceptance decisions between the traditional quality management model and the CQAF model. Our analysis also shows an inconsistent understanding of the OVF's roles and impacts on how respondents are uncertain about the OVF's responsibilities.
Moreover, many respondents agree that the OVF and CEI firms' decisions rarely contradict, while we observe this is disproportionate to the fact that not many respondents agree on the added value of the OVF. Some additional comments from the survey participants also provide valuable lessons and resources for future quality management services. In the next
48

chapter, to further study the survey topics and focus on inconsistencies in the survey responses, we describe semi-structured interviews that we conducted with those subject-matter experts who expressed their willingness to participate further in our research and share their insights on quality management services.
49

CHAPTER 5 INTERVIEW SUMMARY
OVERVIEW
Following the survey analysis, we conducted semistructured interviews with 15 survey participants followed by an embedded qualitative analysis of their experienced CEI perspectives. Critical topics in the survey analysis were chosen to be the interview protocols for further exploration. A sample of experienced survey participants who voluntarily offered their contact information for interviews was selected as our interviewees to gather an in-depth understanding of the key topics in the survey. The in-depth interviews allowed us to explore survey topics in more detail and investigate new questions raised from the survey responses. We summarized the data collected from interviews and conducted a qualitative analysis of expert perspectives. From this analysis, we can enhance our understanding of QA topics and determine more appropriate strategies for QA at GDOT.
Among over 100 survey participants, approximately one third offered contact information and expressed their willingness to further share their views and experience about their answers and quality management research. We successfully reached out and arranged 15 interviews with these professionals and experts. We recorded many insights and recommendations from our interviewees that bring value and innovation to our research. We followed up with 15 individuals nationwide, consisting of two DOT personnel, three general contractors, two design consultants, five CEI professionals, and three owner's representatives, as shown in Figure 10. Most of the interviewees have over 10 years of experience working in the industry and expressed their considerable familiarity with the designbuild project delivery method. Such diverse backgrounds and professions add robustness and comprehensiveness to our interviews.
50

13% 20%
20%

33% 13%

DOT CEI Firm

General Contractor

Design Consultant

Owner's Representative

Figure 10. Pie chart. Professional background of interview participants.

INTERVIEW PROTOCOLS

To increase the depth of our research about the QA topics and formulate competitive strategies for QA, we set interview protocols covering seven major areas. We planned to start with each experts' background or experience with QA in the DB environment and progressively gather the thoroughness of their understanding of some critical topics in our survey. The next question covered the CEI firms' roles and responsibilities. From the survey responses, we observed an inconsistency in how different CEI industry members understand their roles and responsibilities. To assist CEI firms in fulfilling their roles and responsibilities, we felt that the interviewees' recommendations on how CEI firms' understanding could be improved would be valuable. We also found that some survey responses showed possible gaps between the DOTs' and the CEI industry's understandings. We were interested in how the subject-matter experts think that the

51

industry can improve the alignment between their understanding and owner expectations. Next, considering the discrepancy in the survey responses about the difference in the stringency of CEI decisions and actions when hired by the designbuilder versus by the state DOTs, we hoped to hear a more detailed explanation and elaboration from the experts. To make the CEI firms' job easier and improve the quality management process, we wanted to identify the most impactful challenges for CEI firms and the biggest challenges in the QA process. Finally, as we had observed a lot of inconsistency in how members in the CEI industry understand and view the owner verification firm, we asked an interview question about the added values and impacts of the OVF.
INTERVIEW RESPONSES
Typical Roles and Responsibilities of CEI Firms When asked to describe a CEI firm's typical roles and responsibilities, two DOT officials, a CEI professional, and an owner's representative agreed on inspection as a critical role for CEI firms. The owner's representative and the CEI professional emphasized that CEI firms need to be aware of any deficiencies. The CEI professional also described its role as the "eyes and ears" of owners for taking care of checks and balances. We also received feedback from the owner's representative that CEI firms' roles have no difference between DB and DBB projects.
Areas of Improvement for CEI Firms' Understanding We received some recommendations in multiple aspects from our interviewees on where the alignment between state DOTs and the CEI industry's understanding can be improved. General contractors, design consultants, owner's representatives, and CEI firms suggested that specifying expectations for every aspect of work and minimum testing levels in the RFQ and RFP, and
52

including clear minimum requirements and responsibilities would be effective for improving CEI firms' understanding. An owner's representative mentioned that having an oversight process will promote DOTs' and CEI firms' understanding. A CEI firm professional suggested that being proactive, communicative, and having engaged mindsets will also be a helpful strategy.
How DOTs Reduce Gaps in Misunderstanding To make the CEI firms' job easier, we gathered some feedback from experts about state DOTs to reduce gaps in misunderstanding. Interviewees among owner's representatives, DOT personnel, and CEI professionals commonly advised having a clear RFQ and RFP, specific quality management plans, detailed documentation and daily reports, and clear responsibility and requirements of positions. Two owner's representatives suggested that having an independent quality firm such as the CQAF model can help reduce the chance of misunderstanding. The CEI professional mentioned that dedicating time to critical items and performing risk-based inspection can help lessen gaps in misunderstanding.
Most Impactful Challenges for CEI Firms Regarding the most impactful challenges that CEI firms may encounter, some CEI professionals mentioned the complex contractual relationships, different roles expected, and their unfamiliarity with designbuild plus the pressure on CEI firms when working directly with the contractor. A general contractor specified the significant risks when all responsibilities are on the CEI firms, which corresponds to the above statement of the CEI firms' pressure. Simultaneously, the CEI firms are directly involved with the contractor, compared to when they work with DOTs. DOT personnel responded that negotiation for enough man-hours and funds to do the works could be
53

challenging because the designbuilder could have contradictory views. Furthermore, an owner's representative mentioned that CEI firms' ambiguity in the contract can be a potential challenge.
Stringency of CEI Firm's Decisions Regarding the difference in the stringency of the CEI firms' decisions and actions when hired by the designbuilder versus the state DOT, some CEI firms agreed that when hired by the design builder, they tend to have more proactive mindsets before potential issues occur. Another CEI professional commented that the stringency could be the same when hired by either the design builder or the state DOT, which shows consistency with our survey results. A view from the CEI professional was that CEI firms' performance could be dependent on their experience levels, personalities, and preference. For instance, if the DB construction manager has more experience, he tends to be more stringent about decisions or actions.
Biggest Challenges in the QA Process In response to the biggest challenges in the QA process, we received a variety of answers from different professions. Two CEI professionals identified that the fixed amount of work and schedule, and that reaching satisfaction for both entities--general contractors and the DOTs--are challenging in the QA process. They noted that contractors emphasize budgetsaving, while state DOTs aim to meet specifications and needs. Other challenges include the paperwork and the absence of performance measures in testing specified by a design consultant and an owner's representative, respectively.
Owner Verification Firms' Roles and Impacts When asked about an owner verification firm's impacts on project delivery, all interviewees agreed that it added value. Two CEI professionals pointed out that having an OVF complies with
54

federal requirements for acceptance purposes. Another two CEI firm interviewees mentioned that the OVF could bring inherent value to ensure jobs are built correctly. The OVF serves as a second QC, reviewing both QC and quality acceptance that materials and end-products meet the requirements. One CEI professional commented that having an OVF can ensure a CQAF is honest, which adds more value, and verifies the CQAF's work according to the quality management plan and project documents. We received feedback that an OVF's value and impacts come from CEI professionals working as the OVF. A design consultant also mentioned that the value and importance of the OVF go up when the complexity of the project increases.
DISCUSSION
We designed the in-depth interviews to gather more detailed and thorough perspectives from our survey participants. We successfully covered the prepared interview protocols during a series of 15 interviews. With a common understanding from interviewees of CEI firms' roles and responsibilities, CEI firms could have a more comprehensive view of the roles expected of them by different professional groups and reduce the gaps in their understanding from that of state DOTs. Similarly, the second and third areas we covered in the interviews help improve the alignment of the CEI industry and DOTs' understanding of CEI firms' roles and responsibilities and lessen the gaps between their understanding. Some typical comments included having clear expectations and specific minimum requirements for every aspect of work and a clear RFP, RFQ, and QMP. With the common challenges for CEI firms identified by the interviewees, proactive strategies can effectively mitigate these problems. The interviewees further elaborated on the difference in the stringency of their decisions when hired by the designbuilder versus by the state DOTs, explaining their incentives to be more lenient or stringent. While some CEI
55

professionals agreed on similar stringency levels of CEI firms' decisions when hired by either the designbuilder or state DOT, consistent with our survey results, a few others thought CEI firms tend to be more stringent when hired by the designbuilder. On the next topic, the insights shared by some interviewees about the most impactful challenges for effective quality assurance can assist in facilitating efficient and effective project delivery. Lastly, the comments on OVF's value and impacts from our interviewees consistently stated that the OVF brings added values regarding QC and quality acceptance. The OVF's involvement matches the federal requirement, as well.
56

CHAPTER 6 CONCLUSIONS
This study is proposed to offer guidance to GDOT on ways to strengthen the QA process in the innovative project delivery environment. The major concern is that many state DOTs struggle to continue using traditional project delivery methods alone. The traditional designbidbuild system is exposed to internal agency issues in staffing, funding, the complexity of tasks, and communications (An et al. 2018, Kingsley et al. 2017, Mallett and Luther 2011). Such shortcomings can lead to critical cost and time overruns for project delivery, making the DBB method inappropriate under certain circumstances. Correspondingly, many state DOTs are moving toward innovative project delivery methods and replacing the traditional system. Along with this transition, GDOT has already implemented a new quality management model for DB projects requiring the developers' construction quality acceptance firm to serve as an essential part of project QA.
Unlike the traditional quality management model in which the owner hires the CEI firm to conduct quality acceptance, the CQAF, also known as an independent quality firm, performs the frontline acceptance sampling and testing and inspection acceptance determination (Lee et al. 2020). Still, the CEI firms perform quality acceptance, but they now work for the designbuilder (or its developer) with the same duties. CEI firms need to satisfy both designbuilders and DOTs with different priorities. Thus, it is necessary to ascertain to what extent members of the CEI industry understand their roles in such a reformed process and discover the potential challenges to its implementation that the CEI industry faces in the innovative delivery environment.
In this study, we emphasized two primary goals. We first identified and analyzed the gaps between GDOT's expectations of CQAF roles and responsibilities and the CEI industry's
57

perspectives of how it should function in the designbuild environment. Second, we determined the key barriers to effective QA and developed strategic solutions to enhance the effective adoption of the new QA method in the CEI industry. Specifically, we conducted a comprehensive review of professional and academic literature on current practices for quality management in the new project delivery environment. This evaluation allowed us to discover a variance in different project delivery methods, and the literature review informed the following steps of this research.
The next stage of this research was a content analysis of DOT documents, including master contracts, RFQs, RFPs, quality management plans, and quality manuals. To examine the state of the practice in quality management in both the DB and P3 environments, we selected the criteria representing each DOT's unique quality management practices based on project delivery types for the analysis. For the content analysis, we chose the five following criteria: (1) CQAF model; (2) independent quality manager as key personnel; (3) designbuilder quality manager as key personnel; (4) quality manual / quality management plan as evaluation criteria; and (5) authority to stop work. The results showed the similarities and differences across the DOTs. We obtained 12 different quality management practices across the 28 state DOTs for design build projects. The results also showed 5 other quality management practices in P3 projects from the 12 DOTs.
We then developed a survey to analyze construction quality management practices in the DB environment. We discovered inconsistent beliefs over CEI firms' typical roles and responsibilities, different perspectives about the stringency of CEI firms' decisions, and different attitudes toward the added value of owner verification firms among survey participants from various professional disciplines. The respondents also offered recommendations on the reporting
58

process, reporting frequency, communications with the designbuilder or DOT, the dispute resolution process, and advice on cost and evaluation. Some individuals recommended standardizing the reporting process, implementing frequency based on risk levels, and using advanced software. Additional comments included sharing costs for CEI services between the designbuilder and the state DOT and setting a minimum bid amount for quality management services. These comments provide valuable lessons for future improvement in quality management services in federal-aid designbuild projects.
We followed up with 15 survey respondents for further interviews. This step builds from our qualitative survey analysis by exploring its most important topics more thoroughly. We consolidated our interpretation through the analysis of perspectives and recommendations from our interviewees. They shared their understanding of CEI firms' roles and responsibilities, and it helps CEI firms have a better view of expectations and facilitate better performance. Interviewees also guided reducing gaps in understanding between the CEI industry and state DOTs. Their further elaboration on impactful challenges for CEI firms and barriers in the QA process informs us of productive strategies to mitigate these potential issues under the innovative system. Lastly, the comments on OVF enhance our understanding of OVFs' added value for project delivery.
The CEI industry plays a vital role in assisting the public owner and ensuring the quality of constructed projects. The CQAF acts as an extended arm of the public owner to ensure the designbuilder upholds the highest construction standard in developing major projects in the state. Considering the increasingly critical role of the CQAF in the innovative delivery environment, it is valuable to ensure that the CEI industry has a very good understanding of the new requirements of quality assurance in the innovative delivery environment. This study
59

identifies the current state of understanding in the CQAF model compared to the traditional quality management from different professional groups: CEI professionals, design consultants, general contractors, public owners, and owner's representatives. Its findings provide a better understanding of the roles and responsibilities of the CQAF model. It is anticipated that this research will empower the public owner to enhance the CQAF model for current and future designbuild projects. This research contributes to adopting the new quality assurance program in the designbuild and P3 industries.
60

ACKNOWLEDGMENTS
The research reported herein was sponsored by the Georgia Department of Transportation through Research Project 19-27/T.O. 2014-96. The authors acknowledge and appreciate the help of Mr. Darryl VanMeter, GDOT Assistant P3 Division Director / State Innovative Delivery Administrator; Ms. Sarah Lamothe, Research Program Manager at GDOT; and Ms. Supriya Kamatkar, Assistant Office Head for the Office of Performance-based Management and Research.
61

REFERENCES
Allen, L.N., Gransberg, D.D., and Molenaar, K.R. (2002). "Partnering Successful DesignBuild Contracts in the Naval Facilities Engineering Command Southwest Division." The Military Engineer, 94(616), pp.4748.
Amarasekara, W.D.L., Perera, B.A.K.S., and Rodrigo, M.N.N. (2018). "Impact of Differing Site Conditions on Construction Projects." Journal of Legal Affairs and Dispute Resolution in Engineering and Construction, 10(3), 04518006.
Amekudzi-Kennedy, A., Ashuri, B., Amoaning-Yankson, S., Cao, Y., Boadi, R., Brodie, S., and Mostaan, K. (2016). Effective Utilization of Disadvantaged Business Enterprises (DBE) in Alternative Delivery Projects: Strategies and Resources to Support the Achievement of DBE Goals. Research Project RP14-42, Georgia Department of Transportation, Atlanta, GA, April 2016.
American Society of Civil Engineers (ASCE). (2013). "Report Card for America's Infrastructure". Retrieved March 30, 2020, from: http://www.infrastructurereportcard.org/.
An, Y., Rogers, J., Kingsley, G., Matisoff, D.C., Mistur, E., and Ashuri, B. (2018). "Influence of Task Complexity in Shaping Environmental Review and Engineering Design Durations." Journal of Management in Engineering, 34(6), 04018043.
Ashuri, B., Cao, Y., and Mostaan, K. (2017). Deploying SHRP2 Renewal 10 Guidebook for Project Management Strategies for Complex Projects in the Georgia Department of Transportation. Research Project RP15-12, Georgia Department of Transportation, Atlanta, GA, January 2017.
Ashuri, B. and Kashani, H. (2012). Recommended Guide for Next Generation of Transportation Design Build Procurement and Contracting in the State of Georgia. Research Project RP10-23, Georgia Department of Transportation, Atlanta, GA, September 2012.
Ashuri, B., Kingsley, G., Rogers, J., Reisi, M., Ilbeigi, M., Sung, E.J-Y., and Toroghi, S.H. (2015). Streamlining Project Delivery through Risk Analysis. Research Project RP13-05, Georgia Department of Transportation, Atlanta, GA, September 2015.
Ashuri, B., Lee, J.H., and Zhou, Y. (2019). Emerging Trends in Design Professional Liability Policies in Innovative Project Delivery. Research Project RP17-34/T.O. No. 2014-72, Georgia Department of Transportation, Atlanta, GA, November 2019, 82 pp.
Ashuri, B., Lee, J.H., and Zhou, Y. (2020). "Key Challenges Related to the Design Professional Liability in the Transportation DesignBuild Environment." ASCE Construction Institute (CI) Summit, Los Angeles, CA, February 2022, 2020.
62

Ashuri, B. and Mostaan, K. (2014). Innovative Project Delivery Using Alternative Financing Mechanisms: Assessment of Benefits, Costs, and Risks. Research Project RP13-01, Georgia Department of Transportation, Atlanta, GA, December 2014.
Ashuri, B. and Mostaan, K. (2015). "State of Private Financing in Development of Highway Projects in the United States." ASEC Journal of Management in Engineering, 04015002.
Ashuri, B., Mostaan, K., and Hannon, D. (2013). How Can Innovative Project Delivery Systems Improve the Overall Efficiency of GDOT in Transportation Project Delivery? Research Project RP11-21, Georgia Department of Transportation, Atlanta, GA, April 2013.
Atkinson, R. (1999). "Project Management: Cost, Time and Quality, Two Best Guesses and a Phenomenon, It's Time to Accept Other Success Criteria." International Journal of Project Management, 17(6), pp. 337342.
Awwad, R., Barakat, B., and Menassa, C. (2016). "Understanding Dispute Resolution in the Middle East Region from Perspectives of Different Stakeholders." Journal of Management in Engineering, 32(6), 05016019.
Beard, J., Loulakis, M., and Wundram, E. (2001). DesignBuild: Planning Through Development. McGraw-Hill, New York.
Bennett, J., Pothecary, E. and Robinson, G. (1996). Designing and Building a World-Class Industry. Report No. ISBN 07049 1, University of Reading. Reading, United Kingdom.
Brown, J.W., Pieplow, R., Driskell, R., Gaj, S., Garvin, M.J., Holcombe, D., Saunders, M., Seiders, J., and Smith, A. (2009). PublicPrivate Partnerships for Highway Infrastructure: Capitalizing on International Experience. No. FHWA-PL-09-010. U.S. Federal Highway Administration, Office of International Programs.
Chan, A.P., Scott, D., and Lam, E.W. (2002). "Framework of Success Criteria for Design/Build Projects." Journal of Management in Engineering, 18(3), pp. 120128.
DesignBuild Institute of America (DBIA). (2016). DesignBuild Done Right: Transportation Sector. Washington, DC.
DesignBuild Institute of America (DBIA). (2020). DBIA State Maps: DesignBuild Authorization for Transportation 2020. DesignBuild Institute of America.
Dudkin, G. and Vlil, T. (2006). "Transaction Costs in PublicPrivate Partnerships: A First Look at the Evidence." Competition and Regulation in Network Industries, 1(2), pp. 307 330.
Ernzen, J. and Feeney, T. (2002). "Contractor-Led Quality Control and Quality Assurance Plus DesignBuild: Who Is Watching the Quality?" Transportation Research Record, 1813(1), pp. 253259.
63

Ernzen, J. and Feeney, T. (2003). "Compressing the Schedule with Design Build Plus Incentives: Is the Quality as Good?" In Construction Research Congress: Wind of Change: Integration and Innovation, pp. 19.
Federal Highway Administration (FHWA). (2006). DesignBuild Effectiveness Study. FHWA, Washington, DC.
Federal Highway Administration (FHWA). (2008). Transportation Construction Quality Assurance Reference Manual. Washington, DC.
Federal Highway Administration (FHWA). (2012). Techbrief: Construction Quality Assurance for DesignBuild Highway Projects. McLean, VA.
Federal Highway Administration (FHWA). (2016). Federal-Aid Policy Guide. FHWA. Retrieved from: https://www.fhwa.dot.gov/legsregs/directives/fapgtoc.htm.
Federal Highway Administration (FHWA). (2019). Construction Program Guide: Special Experimental Project No. 14 Alternative Contracting. FHWA, September 4, 2019. Retrieved from: https://www.fhwa.dot.gov/construction/cqit/sep14.cfm.
Federal Highway Administration (FHWA). (2020). Design Build Finance (DBF). Center for Innovative Finance Support, April 17, 2020. Retrieved from: https://www.fhwa.dot.gov/ipd/alternative_project_delivery/defined/new_build_facilities/ dbf.aspx.
Gad, G.M., Adamtey, S.A., and Gransberg, D.D. (2015). "Trends in Quality Management Approaches to DesignBuild Transportation Projects." Transportation Research Record, 2504(1), pp. 8792.
Garvin, M.J. (2010). "Enabling Development of the Transportation PublicPrivate Partnership Market in the United States." Journal of Construction Engineering and Management, 136(4), pp. 402411.
Georgia Department of Transportation (GDOT). (2014). GDOT DesignBuildFinance Construction Quality Assurance Program (Northwest Corridor Project).
Gransberg, D.D., Badillo-Kwiatkowski, G.M., and Molenaar, K.R. (2003). "Project Delivery Comparison Using Performance Metrics." AACE International Transactions, CS21.
Gransberg, D.D., Datin, J., and Molenaar, K.R. (2008a). NCHRP Synthesis 376: Quality Assurance in DesignBuild Projects. Transportation Research Board, Washington, DC.
Gransberg, D., Datin, J., and Molenaar, K. (2008b). Quality Assurance in DesignBuild Projects. National Cooperative Highway Research Program (NCHRP).
Gransberg, D.D. and Molenaar, K. (2004). "Analysis of Owner's Design and Construction Quality Management Approaches in Design/Build Projects." Journal of Management in Engineering, 20(4), pp. 162169.
64

Gransberg, D.D. and Molenaar, K.R. (2007). The Impacts of DesignBuild on the Public Workforce. USC Keston Institute for Public Finance and Infrastructure Policy, Los Angeles, CA.
Gransberg, D.D. and Molenaar, K.R. (2008). "Does DesignBuild Project Delivery Affect the Future of the Public Engineer?" Transportation Research Record, 2081(1), pp. 38.
Greve, H.R. and Argote, L. (2015). "Behavioral Theories of Organization." International Encyclopedia of the Social & Behavioral Sciences, 2nd ed. Elsevier, Amsterdam, pp. 481486.
Hale, D.R., Shrestha, P.P., Gibson Jr., G.E., and Migliaccio, G.C. (2009). "Empirical Comparison of Design/Build and Design/Bid/Build Project Delivery Methods." Journal of Construction Engineering and Management, 135(7), pp. 579587.
Harman, L. and Sillars, D.N. (2013). "Case Studies in Innovative Quality Assurance Methods for Alternative Delivery Projects." Journal of the Transportation Research Board, 97214, 503 pp.
Ho, S.P., Levitt, R., Tsui, C.W., and Hsu, Y. (2015). "Opportunism-focused Transaction Cost Analysis of PublicPrivate Partnerships." Journal of Management in Engineering, 31(6), 04015007.
Huges, C.S. (2005). State Construction Quality Assurance Programs: NCHRP Synthesis 346. Transportation Research Board, Washington, DC.
Ibbs, C.W., Kwak, Y.H., Ng, T., and Odabasi, A.M. (2003). "Project Delivery Systems and Project Change: Quantitative Analysis." Journal of Construction Engineering and Management, 129(4), pp. 382387.
Jarkas, A.M., Kadri, C.Y., and Younes, J.H. (2012). "A Survey of Factors Influencing the Productivity of Construction Operatives in the State of Qatar." International Journal of Construction Management, 12(3), pp. 123.
Kingsley, G., Matisoff, D., Rogers, J., Ashuri, B., An, Y., Mistur, E., Ruthotto, I., and Agnew, D. (2017). Strategies for Communicating Quality Expectations for Environmental Service Contracts. Report No. FHWA-GA-17-1506, Georgia Institute of Technology, Atlanta, GA.
Kraft, E. and Molenaar, K.R. (2013). "Project Quality Assurance Organization Selection for Highway Design and Construction Projects." Transportation Research Record, 2347(1), pp. 2937.
Lee, D. and Arditi, D. (2006). "Total Quality Performance of Design/Build Firms Using Quality Function Deployment." Journal of Construction Engineering and Management, 132(1), pp. 4957. https://doi.org/10.1061/(ASCE)0733-9364(2006)132:1(49).
65

Lee, J.H., Jallan, Y., and Ashuri, B. (2020). "Key Issues and Differences in Practical Components of Quality Management in DesignBuild Highway Projects." Journal of Legal Affairs and Dispute Resolution in Engineering and Construction, 12(1), 04519029.
Lee, J.H., Zhou, Y., and Ashuri, B. (2020). "Key Challenges to Design Professional Liability in the DesignBuild Environment." ASCE Journal of Legal Affairs and Dispute Resolution in Engineering and Construction, 12(3), 04520031.
Li, H., Arditi, D., and Wang, Z. (2014). Transaction Costs Incurred by Construction Owners. Engineering, Construction and Architectural Management, 21, pp 444458.
Liang, Y. and Ashuri, B. (2020). "Analysis on the Obstacles of Implementing Small and Medium PublicPrivate Partnership for the U.S. Infrastructure Development." The 2020 ASCE Construction Research Congress (CRC), Tempe, AZ, March 810, 2020.
Liang, Y., Ashuri, B., and Sun, W. (2019). "Identifying Patterns in DesignBuild Projects in Terms of Project Cost Performance." The 2019 ASCE International Conference on Computing in Civil Engineering, Atlanta, GA, June 1719, 2019.
Mallett, W.J. and Luther, L. (2011). Accelerating Highway and Transit Project Delivery: Issues and Options for Congress. Congressional Research Service, Washington, DC, pp. 1018.
Molenaar, K.R., Songer, A.D., and Barash, M. (1999). Public-sector Design/Build Evolution and Performance. Journal of Management in Engineering, 15(2), pp. 5462.
Mostaan, K. and Ashuri, B. (2016a). "Challenges and Enablers for Private Sector Involvement in Delivery of Highway PublicPrivate Partnerships in the United States." ASCE Journal of Management in Engineering, 04016047.
Mostaan, K. and Ashuri, B. (2016b). "Recommended Opportunities for the Development of Highway PublicPrivate Partnership Projects in the United States." 2016 Construction Research Congress (CRC), ASCE, San Juan, Puerto Rico, May 31June 2, 2016.
North, D.C. and North, D.C. (1992). Transaction Costs, Institutions, and Economic Performance. ICS Press, San Francisco, CA, pp. 1315.
Roth, M.B. (1995). An Empirical Analysis of United States Navy Design/Build Contracts. The University of Texas at Austin, Department of Engineering.
Sanvido, V.E. and Konchar, M.D. (1997). Project Delivery Systems: CM at Risk, DesignBuild, DesignBidBuild. Technical Report No. 133, Construction Industry Institute (CII), Austin, TX.
Scott, S. (2006). Best-value Procurement Methods for Highway Construction Projects. Vol. 561, Transportation Research Board.
Scott, S., and Molenaar, K. (2017). Guidelines for Optimizing the Risk and Cost of Materials QA Programs: NCHRP Synthesis 838. Transportation Research Board, Washington, DC.
66

Shah, M.P., Pitroda, J.R., and Bhavsar, J.J. (2014). "Analysis of Factors Influencing Productivity: Survey of Construction Projects in Central Gujarat Region of India." International Journal of Engineering Sciences & Research Technology, 3(4), pp. 3082 3087.
U.S. Department of Transportation (USDOT). (2013). "National Transportation Statistics." Bureau of Transportation Statistics. Retrieved March 3, 2020, from http://www.fhwa.dot.gov/reports/designbuild/designbuild.pdf.
Vrijhoef, R. and Ridder, H. (2007, May). "Supply Chain Systems Engineering in Construction." In Proceedings CIB World Building Congress, pp. 1417.
Warne, T.R. (2005). Design Build Contracting for Highway Projects: A Performance Assessment. Tom Warne & Associates, LLC, South Jordan, UT.
Whittington, J. and Dowall, D.E. (2006). Transaction-cost Economic Analysis of Institutional Change Toward DesignBuild Contracts for Public Transportation. No. 2006, 09, Working Paper.
Williamson, O.E. (1985). The Economic Institutions of Capitalism. Free Press, New York.
67