Project development improvement and streamlining: assessing the effectiveness of process improvments in consultant management [2008]

Georgia Transportation Institute Project

Georgia

Transportation Title: Project Development Improvement and Streamlining: Assessing

Institute

the Effectiveness of Process Improvements in Consultant Management

Investigator(s):

P.I., Gordon Kingsley, School of Public Policy, Georgia Institute of Technology, Atlanta, GA 30332-0345, phone: 404-894-0454, fax: 404-385-0504, email: gordon.kingsley@pubpolicy.gatech.edu

Co-PI, Barry Bozeman, School of Public Policy, Georgia Tech Email: barry.bozeman@pubpolicy.gatech.edu

Co-PI, Chris Weible, School of Public Policy, Georgia Tech Email: chris.weible@pubpolicy.gatech.edu

Co-PI, Cliff Eckert, Georgia Tech Research Institute Email: cliff.eckert@pubpolicy.gatech.edu

Graduate Research Assistants :

Jeff Jones, Wanda Spivey, Jennifer Chirico Love, and Mike Waschak, Georgia Institute of Techology Mary Feeney and Craig Smith, University of Georgia

Institution(s):

Georgia Institute of Technology Georgia Tech Research Institute University of Georgia

Acknowledgments We acknowledge the guidance and leadership given to this project by the late Mr. Babs Abubakari, former Chief Design Engineer for the State of Georgia. Babs was the inspiration and the guiding force behind this project in his efforts to improve the performance of the Office of Consultant Design and the interactions of the Georgia Department of Transportation with the engineering design and professional services community of consultants. The State of Georgia and the transportation community has lost a valuable colleague and friend with his untimely passing. We would also like to acknowledge the men and women of the Georgia Department of Transportation for the assistance and cooperation in the development of this project. The dedication of these individuals to pursuing high standards of performance is a credit to the public service. We also acknowledge the able contributions from our graduate assistants at Georgia Tech. Mr. Jeff Jones provided substantial leadership on this project. Ms. Wanda Spivey, Ms. Jennifer Chirico Love, and Mr. Mike Waschak all made valuable contributions. Students from the University of Georgia also made substantial contributions in particular Ms. Mary Feeney and Mr. Craig Smith. This report is the product of collaboration between researchers at the Georgia Institute of Technology, the University of Georgia, and the Georgia Tech Research Institute. Any errors or omissions in the findings or interpretations are the authors alone and do not reflect the official position of our respective institutions or the Georgia Department of Transportation.
ii

Table of Contents Acknowledgements...................................................................................ii Table of Contents ....................................................................................iii List of Tables and Figures..........................................................................iv Abstract................................................................................................1 Executive Summary................................................................................2-9 Task Report 1: Cost Proposals and Man-Hour Estimates.................................10-35 Task Report 2: Changes in Tacit Knowledge of Contracts and Contracting............36-82 Task Report 3: Knowledge of Prime and Subcontractor Relations.....................83-142
iii

List of Tables and Figures
Section I: Cost Proposals and Man-Hours Estimates
Figure 1 Data Import Control Form............................................................15 Figure 2 Roadway Type Analysis Input Screen..............................................17 Figure 3 Roadway Type Analysis Report Pages 1 and 2................................18 Figure 4 Selected Cost Analysis by PI Input Screen.......................................19 Figure 5 Selected Cost Analysis by PI Number Report Pages 1 and 2..............20-21 Figure 6 Consultant Input Screen...............................................................22 Figure 1 - Consultant Summary Report Page 1.............................................. 23
Section II: Tacit Knowledge, Capacity and Contracting: A Survey of Consultants and GDOT Managers
Table 1 - Age Distribution for GDOT Respondents............................................44 Table 2 - Have you ever worked in the private sector in a managerial or professional job?.....................................................................................................................................45 Table 3 - Correlation for In-House Functionality Index and Selected Predictor Variables.............................................................................................49 Table 4: Means for Work Task and Hours Worked.............................................51 Table 5. Views about Qualities of "Good Consultants"........................................52 Table 6 Consultant Management Perspective: Dimensions Emerging from Factor........54 Table 7. Attributes of Respondents' Companies...............................................58 Table 8. GDOT Offices with which Respondents Worked Analysis ..............................................................................................59 Table 9. Time Required for Core Tasks..........................................................69 Table 10. Number Sign-off for Contracts........................................................71 Table 11. Red Tape Constructs and Correlates.................................................72 Table 12. Consultant Responses for Contracting Relations Items............................74 Table 13. GDOT Staff Responses for Contracting Relations Items.........................76
Figure 1 - GDOT Respondents' Education......................................................43 Figure 2 Outsource: Core Competency.........................................................46 Figure 3 Outsource: Do in-house to compete.................................................47 Figure 4 Outsource: Monitor Consultants......................................................48 Figure 5 Outsource: Keep Up With Technical and Program Change......................48 Figure 6. Education of Consultant Respondents.................................................57 Figure 7. Composition of Contracts...............................................................60 Figure 8. Types of Contracts for Consultants, by Percentage (Means).....................61 Figure 9. Contractor Preference for Contract Types...........................................62 Figure 10. Assessments of "Red Tape"..........................................................64 Figure 12. Sources of Contracting Red Tape: GDOT Respondents..........................67 Figure 13. Sources of Contracting Red Tape: Consultant Respondents.....................67
iv

List of Tables and Figures (continued)
Section II: Tacit Knowledge, Capacity and Contracting: A Survey of Consultants and GDOT Managers
Table 1 - Data Quality and Rejected Data for CMS, CMIS, and combined data sets.....91 Table 2 - Number of Prime/Sub Consultants and Contract Totals per Year...............93 Table 3 - Number of Contracts per Year in Combined Dataset..............................94 Table 4 - Ratio of Primes / Subs..................................................................96 Table 5 - Number of Relations and Contract Dollars for Arcadis U.S., Inc. by Year...113 Table 6 - Number of Relations and Contract Dollars for PBS&J U.S., Inc. by Year....117 Table 7 - Number of Relations and Contract Dollars for United Consulting by Year...122 Table 8 - Number of Relations and Contract Dollars for Street Smarts by Year.........125 Table 8 - Most Central Firms (by degree or total number of relations)....................130 Table 9 - Most Central Firms (by average degree or number of relations per year)......131 Table 10-Average Contract Dollars and Connections for DBE and Non-DBE Firms by Year.................................................................................................132
Figure 1. Screen Shot of Combined Dataset....................................................90 Figure 2. GDOT Prime and Sub Consultant Network 1995................................98 Figure 3. GDOT Prime and Sub Consultant Network 1996................................99 Figure 4. GDOT Prime and Sub Consultant Network 1997..............................100 Figure 5. GDOT Prime and Sub Consultant Network 1998..............................101 Figure 6. GDOT Prime and Sub Consultant Network 1999..............................102 Figure 7. GDOT Prime and Sub Consultant Network 2000..............................103 Figure 8. GDOT Prime and Sub Consultant Network 2001..............................104 Figure 9. GDOT Prime and Sub Consultant Network 2002..............................105 Figure 10. GDOT Prime and Sub Consultant Network 2003.............................106 Figure 11. GDOT Prime and Sub Consultant Network 2004.............................107 Figure 12. GDOT Prime and Sub Consultant Network 2005.............................108 Figure 13. GDOT Prime and Sub Consultant Network 2006.............................109 Figure 14. GDOT Prime and Sub Consultant Network 2007.............................110 Figure 15. Firm Level Network for Arcadis U.S., Inc.......................................114 Figure 16 - Firm Level Network for PBS&J...................................................118 Figure 17 - Firm Level Network for United Consulting.....................................123 Figure 18. Firm Level Network for Street Smarts...........................................126 Figure 19 - 2000 Network Map with Large Circles and Squares Indicating DBE Status................................................................................................134 Figure 20 - 2000 Map with Contract Dollars Shown by Size, and DBE Status in Black................................................................................................135
v

vi

Abstract The purpose of this study is to examine the effectiveness of organizational and process changes that have taken place in the Office of Consultant Design (OCD) and with the Consultant Management Information System (CMIS) of the Georgia Department of Transportation (GDOT). This project is the third in a series of projects aimed at monitoring improvements in consultant management systems and in identifying promising techniques for making further improvements. This study examines the effectiveness of these initiatives to determine areas where the quality of information available to project managers can be improved. There are three tasks for this study: 1) to review the data collected using the semi-structured spreadsheets to determine if the changes improve the quality of man-hour estimates from cost proposals; 2) to examine changes in the amount of project management information that is held in tacit form by individual project managers or captured in GDOT standard operating procedures and information systems; and 3) to develop more effective ways of visualizing resource allocations for program and project managers by applying social network analysis to basic contract data. Recommendations on the next steps for GDOT are provided.
1

Executive Summary The purpose of this study is to examine the effectiveness of organizational and process changes that have taken place in the Office of Consultant Design (OCD) and with the Consultant Management Information System (CMIS) of the Georgia Department of Transportation (GDOT). This project is the third in a series of projects aimed at monitoring improvements in consultant management systems and in identifying promising techniques for making further improvements. Since the previous study, completed two years ago,1 GDOT has accomplished the following: A reorganization of OCD into a resource allocation unit, a contracting unit,
and a contract management unit. Revisions to the internal management processes associated with consultant
management and the further development of the Consultant Management Information System (CMIS). Development of a semi-structured spreadsheet designed to improve the data quality of cost proposals submitted as part of consultant project applications. This study examines the effectiveness of these initiatives to determine areas where the quality of information available to project managers can be improved. There are three objectives for this study. Each objective has been addressed through one of the three reports attached to this executive summary: 1. The first report (Task 1) reviews the data collected using the semi-structured spreadsheets to determine if the changes improve the quality of man-hour estimates from cost proposals. In an earlier project it was determined that there
1 Jennifer, please insert the title of the previous studies that we completed. 2

was not sufficient standardization across cost proposals to allow for generating man-hour estimates. This current study finds substantial improvements have been achieved but that sufficient data quality issues remain before reliable estimates can be generated from cost proposals; 2. The second study (Task 2) examines changes in the amount of project management information that is held in tacit form by individual project managers or captured in GDOT standard operating procedures and information systems; and 3. The third study (Task 3) is the most experimental of the studies in attempting to develop more effective ways of visualizing resource allocations for program and project managers by applying social network analysis to basic contract data. This study created network maps of the system of contracts linking GDOT to prime and sub-consultants. A summary of the findings across the three studies is provided here. This work was conducted under the leadership of Dr. Gordon Kingsley, School of Public Policy, Georgia Institute of Technology. Each of the reports was lead by a team leader including the following: Task 1 which examined cost proposals and man-hour estimates was led by Mr. Cliff Eckert, of GTRI; Task 2 which examined the tacit knowledge associated with the management of consultant projects was led by Dr. Barry Bozeman, University of Georgia; and Task 3 which examines the contract networks with the consulting community was led by Dr. Chris Weible of the School of Public Policy at Georgia Tech. Task 1: Cost Proposals and Man-Hour Estimates Task 1 examines the quality of information captured in GDOT's cost proposal spreadsheet compared to the quality of data captured in an earlier database, the Georgia
3

Road Improvement Projects (GRIP). To accomplish this research goal, a Cost Proposal Database (CPD) was created. The CPD generates aggregate and average man hour estimates over different project types.
A close collaboration was established with GDOT managers in reviewing cost proposals that employed the data from the new spreadsheet format in order to develop cost estimates by project type and contract type in a searchable database. Estimates were made of man-hour per mile and cost per mile of phase level data. OCD contract officers were contacted to help answer questions about the negotiations and content of the cost proposals that utilized the new format.
The result of this work is a searchable Cost Proposal Database (CPD) of cost proposals. The CPD allows analysis of man-hours per mile and cost per mile of different project types, contract types and phases of work. It is important to note that the reports generated are for demonstration only, as the data in the CPD has not been verified for accuracy by GDOT and values may be incorrect. However, the usability and quality of CPD will improve through the incorporation of additional data standards and validation processes.
GDOT will be able to utilize the CPD to analyze man-hours per mile, cost per mile, contract types and phases of work. At some point in the future as the data quality improves, GDOT will be developing a web-based cost proposal submittal system. However, these systems still need to be developed and implemented. By having data standards and validation processes already incorporated, the adjustments to a web-base submittal system will be less difficult for both the consultants and OCD.
4

Task 2: Changes in Tacit Knowledge of Contracts and Contracting One of the issues of concern to GDOT is the retention of knowledge about
contracting with the consulting community. Transportation agencies are vulnerable to the loss of knowledge when key people retire or take jobs elsewhere. A key question is how much knowledge is retained through to standard operating procedures and management systems maintained by GDOT. Task 2 examines the relationship of contracting to GDOT knowledge capacity in consultant management and tacit knowledge of GDOT officials. This research includes an analysis of survey responses from GDOT staff and consultants and how they compared in their perceptions about contractual relationships.
The research was based on surveys that were administered to GDOT managers, administrators, and consultants. The final report is comprised of three sections: 1) the GDOT survey; 2) the consultant survey; and 3) a comparison of GDOT and consultant survey responses. The three primary questions considered in the survey were:
1. What are the GDOT managers views about out-sourcing, including maintaining in-house capacity? What factors mitigate the capacity impacts of contracting?
2. How do GDOT staff and consultants diverge in their views about administrative procedures, including the extent and sources of "red tape?"
3. How do GDOT staff and consultants diverge in their views about contractor-staff relationships and the role of contractors?
The surveys were designed to provide a comparison between the GDOT and consultant samples with respect to perceptions and activities about contracting and contract management. The report also provides an analysis about views on
5

organizational procedures and "red tape" in both consultant firms and GDOT. The overall survey results indicate:
The average consultant contact in the sample was involved with four GDOT offices, though the Office of Consultant Design (OCD) had the most contracts with consultants;
On average, the consultants in the sample did more business with GDOT than any other public or private organizations and "cost plus contracts" were the most common type of contract;
On a scale of 1-10, the questions that assessed the level of "red tape" had much higher ratings for GDOT (7) than for consultant firms (3);
On a scale of 1-10, the GDOT sample reported a slightly higher level (6.5) than the consultant sample (6.5) with respect to the level of "red tape" in the contracting relationship between GDOT and private firms;
GDOT respondents thought that federal acquisitions regulations, GDOT employees, and GDOT rules and regulations were the biggest contributors of "red tape", while consultants claimed that federal acquisitions regulations and GDOT rules and regulations contributed the most to "red tape";
"Red tape" appears to have a negative impact on the manager, as managers that spend more time communicating with consultants and on contract paperwork have increased assessments of red tape, and therefore, significantly lower job satisfaction; and
6

GDOT's views were more favorable about consultants than consultants' views of GDOT (though consultant views of GDOT contracting relations were positive overall). The report provides information on the tools that contract managers need to
maximize the translation of tacit knowledge into contracts and GDOT consultant management systems. It also examines how GDOT can minimize the capacity losses from contracting. Finally, the study reviews those areas of the consultant management system where GDOT is vulnerable to perceptions of "red tape" among GDOT managers and consultants. Task 3: Knowledge of Prime and Subcontractor Relations
The purpose of Task 3 was to build a database from GDOT databases of prime and subprime contract awards. One of the primary challenges of managing a large number of contracts is keeping track of the flow of projects through prime and subprime consultants. Social network maps can be used as an effective tool for visualizing and tracking the relationships between prime consultants and sub-consultants. Dr. Weible and his research team utilized social network software tools to create network maps of these contractual relationships.
The data for Task 3 was collected through two datasets: The Consultant Management System (CMS) and the Consultant Information System (CMIS). These two datasets were combined into one dataset that included the relationships among prime and subprime consultants, Disadvantaged Business Enterprises (DBE), and consultant contract amounts between 1992 and 2007. Two software tools, Ucinet and Pajek, were used for analyzing and providing network maps of GDOT consultant networks. The tools
7

assisted with demonstrating the growth in contracts over time for prime consultants and the relationships between prime and subprime consultants over time.
The resulting combined database was used to document the contract relationships by prime consultant and sub-consultant. The information was then used to create network maps that presented the changes in contract relationships over time. The primary findings from this task indicate:
While the number of prime and subprime consultants increases over time the increases are not proportional on a project basis;
The complexity of the relationships among prime and subprime consultants increases as the contract dollars and number of contracts increased over time as a web of relationships emerges linking primes and sub consultants across projects;
When maps are created that include both the prime and the sub-consultant the firms with the most relationships with GDOT and with other consultants were PBS&J, Arcadis, PBQD, J.B. Trimble, Edwards-Pittman Environmental, Carter & Burgess, Earth Tech, and Columbia Engineering;
Disadvantaged Business Enterprise (DBE) firms were nearly equal to non-DBE firms in terms of likelihood of establishing contract relationships and achieving proportional contract dollar amounts. These network maps can assist GDOT managers in understanding the complex
relationships inherent within the GDOT consultant community. GDOT can visually monitor the frequency and monetary amount of contractual relationships with consultants and subprime consultants. For instance, GDOT managers can easily assess whether subprime consultants are overcommitted to projects with several different prime
8

consultants or if DBEs are not receiving the required percentage of contract funding. In addition, the combined CMS and CMIS dataset provides GDOT managers with a more complete and accurate source for tracking prime and subprime consultants over time. Recommendations on Next Steps for GDOT
The deliverables for this project are the final task reports that summarize the findings from all three tasks. Along with the final report, two databases, the Cost Proposal Database (CPD) and the combined CMS/CMIS database were developed during the research process, and are provided to GDOT. These databases are, in effect, a proof of concept for how GDOT might adapt existing data sources available through CMIS or the existing cost proposals to build managerial tools for GDOT personnel. In present form these databases are not operational tools. Issues remain in terms of the data quality and in terms of GDOT systems for collecting and maintaining data in order to create an operational tool. In the future, we recommend that GDOT expand the current method of contract management to include a systems-based approach, which would encompass the entire portfolio of contracts, including all prime and subprime consultants. Adopting a systems approach would require GDOT to 1) continue documenting all contracts in the CMIS database; 2) introduce further standardization to the cost proposal data entry tools as a means of improving data quality. In order to develop visual maps of contract relationships with the consulting community an analytic tool such as UCInet would need to be employed as an auxiliary to the CMIS system. Specific recommendations for future research associated are available in each of the individual task reports.
9

SECTION I: COST PROPOSALS AND MAN-HOURS ESTIMATES
Cliff Eckert Georgia Tech Research Institute
10

Introduction The objective of this task was to examine the quality of the information captured
in GDOT's cost proposal spreadsheet format, used over the last two years, and to compare this data with the quality of information that was compiled in the previous database of GRIP projects. To accomplish this task, a database was developed that allows for the comparative analysis of data and also provides a searchable database of recent cost proposals that will permit the analysis of man-hours per mile and cost-per mile of different project types, contract types, and phases of work. Background
As part of the Consultant Management Process Improvement Initiative (CMPII) a proof -of-concept database was developed to assess the quality of information contained in the cost proposals for a select set of Georgia Road Improvement Projects (GRIP). A task report titled "Developing Data and System Resources for Consultant Management: Preparatory Assessments for the Standardization of Cost Proposals and Contract Templates" was submitted to GDOT February 2005. This report analyzed the capacity and hurdles for the standardization of cost proposals in GDOT and described the processes and issues of building the database prototype as well as implications for the GDOT's efforts towards the standardization of cost proposals. Subsequent to this report, GDOT developed and implemented an improved cost proposal collection method that with greater standardization. The new spreadsheets gave greater standardization with more detailed information.
To determine the quality of the data being collected by GDOT, a Microsoft Access 2007 database of cost proposals was developed by importing cost proposal
11

spreadsheets provided by GDOT. By developing the database, researchers were able to determine the quality of information being submitted by consultants and the existing degree of data standardization. Additionally, researchers were able to determine what areas of cost proposal data needs more standardization as well as provide other recommendations for improving data quality. Cost Proposal Database
The Cost proposal Database was developed with three objectives in mind, 1) Simplify the import process so that GDOT OCD users could easily import data from the consultants cost proposal spreadsheet, 2) Develop reports that would be beneficial to GDOT OCD users and management, and 3) develop the necessary tables and queries that would allow analysis of the database for answering questions about data quality and data standardization. All three objectives were met to various degrees and are discussed below. Data Import
In order to easily import data from consultant cost proposal spreadsheets, a multistep process was developed. Researchers decided to use spreadsheet cell formulas to "find and select" necessary data rather than VBA code within MS Access to find and select the data. The logic behind this decision was if enhancements or upgrades to the import process were needed in the future, it would be easier for an experienced spreadsheet user to make the changes rather than trying to find a skilled VBA programmer.
The first step of the import process involves opening a spreadsheet template that uses cell formulas for linking to the cost proposal spreadsheet. Once linked, the template
12

spreadsheet finds and selects data from the consultant's cost proposal spreadsheet. The template spreadsheet picks up such information as project/discipline/phase information, labor rates and labor descriptions by discipline, hours by labor category within each phase and other costs by phase. The template spreadsheet does not pick up any calculated amounts from the consultants cost proposal spreadsheet, including the cost summary worksheet. It was evident early in the database development process that the consultant cost spreadsheets were being altered even though GDOT issues a "standardized" spreadsheet to consultants. An estimated 75% of the consultant cost spreadsheets that were imported had been modified by consultants. Often rows and columns were added or deleted to worksheets, and formulas changed or deleted. It was evident that consultants needed to modify GDOT's standardized cost proposal spreadsheet to accommodate their proposal cost data. Because the standardized spreadsheets were not "locked" the consultants could alter any of the standardized worksheets. For this reason, the import process performs all calculations once the data is imported into the database rather than rely on spreadsheets that could have formula errors. There was also the problem with GDOT changing their standardized spreadsheet for Turnkey 4 and above proposals. GDOT added columns and rows to the spreadsheet for Turnkey 4 to gather additional information from the consultants so these changes required the development of a special Turnkey 4 spreadsheet template. Because so many of the standardized cost spreadsheets were being altered by consultants, the ability to develop a simple automated import process was not feasible. Developing programming code to anticipate all the possible spreadsheet alterations would be costly to develop and would be costly to accommodate future changes. Therefore, the high
13

probability that a spreadsheet had been altered required a thorough manual review of each spreadsheet before importing. If consultant alterations were discovered the spreadsheet would need to fixed so that the data would be in the correct location and could be picked up by the spreadsheet template.
The remaining steps in the import process are done from MS Access following step by step instructions. (See Figure 1) The steps trigger the import of data from the template spreadsheet to a temporary table which then appends the data to other appropriate tables. The last step of the import process runs a query that shows the total cost, with breakdowns by discipline and phase, and should be compared to the original consultants cost spreadsheet total costs. Of the 81 turnkey cost proposals that were imported, differences in total costs ranged from pennies (due to rounding MS Access rounds differently than MS Excel) to $1.6 million. Many of the large differences were due to whether Right of Way Acquisition costs were imported. Some consultants included acquisition costs in the ROW disciplines while others did not. Most of the differences in totals, however, were due to the fact that the formulas on the consultant spreadsheets were incorrect, missing or in the wrong cells. Attachment A lists the 81 turn key cost proposals that were imported and show the differences in total amounts between the spreadsheet and original consultant cost proposal spreadsheet.
14

Figure 2- Data Import Control Form
Reports A requirement of the database was the ability to develop cost estimates by project
type and contract type in a searchable format and to come up with estimates of man-hour and cost per mile of phase level data. GT and GDOT also agreed that no attempt would be made to develop cost and man-hour estimates at the task level due to the wide variance by consultants of task descriptions.
15

It is important to note that the reports generated from this Cost Proposal Database are for demonstration purposes only. The data in the database have not been verified for accuracy by GDOT therefore the values in the report may be incorrect. As mentioned above, many of the proposals that were imported to the database have different totals than the consultant spreadsheets therefore until the differences are reconciled by GDOT, reports should not be used for business purposes. Also, the data provided in the various fields by consultants in their proposal spreadsheet is not standardized among the consultants. GDOT has not yet set up any validation processes on consultant's spreadsheet data and, since values in many fields were quite diverse, validation could not be performed during the import process or even once the data was in the database. Before the database can used for business purposes GDOT will need to set standards for data fields and should set up validation algorithms so that non standard data is rejected and returned to the consultant for correction. Attachment B is a list of spreadsheet data fields which shows the variations in data that have been submitted by consultants in the 81 proposals imported in the database.
Three developed reports can be run from the database Introduction Form. They were developed to show possible methods of determining man-hour estimates based on similar roadway data or from aggregated discipline or phase data as well as providing aggregate information about consultants, consultant type, and project costs,. Roadway Type Analysis: Users can select roadway criteria such as roadway length, mainline function class, area type, terrain, number of interchanges and number of bridges. Once the criterion is selected a report can be run and printed. See Figures 2 and 3.
16

Figure 3 - Roadway Type Analysis Input Screen 17

Figure 4- Roadway Type Analysis Report Pages 1 and 2 18

Selected Cost Analysis: Users can select the proposal(s), type of cost, discipline(s), and phase(s) then run and print the report.
Figure 5 - Selected Cost Analysis by PI - Input Screen
19

Figure 6 - Selected Cost Analysis by PI Number Report - Pages 1 and 2 20

21

Consultant Reports: Users can select the type of consultant(s) such as EEO, non EEO or a specific consultant and a report is generated which shows summary cost information by proposal, discipline and phase.
Figure 7 - Consultant Reports Input Screen
22

Figure 8 - Consultant Summary Report Page 1
Numerous other reports can be generated from the database by querying the many tables that are in the database. Attachment C shows the database tables that will help users familiar with MS Access to develop and run needed queries and reports.
23

Data Quality GDOT OCD has made very good progress in their efforts to standardize the data
provided by consultants. They have developed spreadsheet templates for consultants to use for submitting proposals which has provided a higher degree of standardization. Due to the complexities of submitting cost proposals, GDOT is reluctant to lock many of the fields in the spreadsheet however the consequence is consultants can alter the spreadsheet fields and formulas. Unfortunately, OCD often does not catch the altered spreadsheets or the costs and totals are often incorrect. Also, because validation is not performed on the data, a significant amount of data is non-standard and these non-standard fields cannot be used adequately for reporting. An example of this problem is in the Roadway Type Analysis Report. Because of the diversity of values in the Mainline Function Class field it is difficult to aggregate costs using this field. This is common problem throughout the database. With some data clean up work on existing data and implementing validation processes for new data, all of these problems can be corrected which will make the database extremely useful for OCD. Conclusion
GDOT is close to having a useable, high quality Cost Proposal Database. By incorporating additional data standards and incorporating data validation processes, data quality will improve significantly. At some point in the future, GDOT will be developing a web-based cost proposal submittal system. By having data standards and validation processes already incorporated, the adjustments to a web-base submittal system will be less difficult for both the consultants and OCD.
24

Next Steps To get the database to a point where the data can reliably be used for GDOT
business purposes, such as man-hour estimates and summary reporting, researchers recommend the following tasks:
Reconcile differences between consultant spreadsheet totals and database totals. This task should not be difficult since researchers have already identified most of the reasons for differences but need OCD personnel to approve changes.
Clean up existing data by reentering data in a standardized format and entering missing data. Some of the clean up can be performed using OCD personnel project knowledge however contact with consultants for clarification and missing data will be necessary.
Import remaining cost proposal spreadsheets. A number of cost spreadsheets were not available for import during this project. Also, a number of spreadsheets are in PDF format and will need to be entered manually.
Enhance the existing standardized spreadsheet template o Work with consultants and OCD to provide consultants with flexibility for submitting cost proposals yet maintains a standardized format needed for importing data and validating data. o Lock the spreadsheet so that consultants cannot change formulas and data location.
Establish control processes for importing and maintaining data quality.
25

Attachment A List of Imported Data

Project Nos.

PI Database

Ori gin al Sp re ad she et

Di fferen ce

STP-012-1(79)
STP-012-1(81) CSSTP-0006-00(252) CSSTP-0006-00(253)
STP-002-4(26)

PI low $.01 due to rounding. MAP1 - the other direct

costs were entered manually in the summary section -

not carried to sum mary section from detail section

$

PI high due to Geo1 not adding to total labor cost to the

other direct cost.

$

exact match - original spreadsheet needed realignment a $

due to rounding

$

Map 1 did not carry the correct direct cost detail left to

the summary box. The cost spreadsheet did not have

any worksheet for RW acquistion. Added RW manually

for PI database

$

2,933,527.98 $ 2,933,527.99 $

1,308,074.99 $ 5,617,373.70 $ 4,216,839.06 $

1,308,075.00 $ 5,617,373.70 $ 4,216,839.05 $

3,043,753.91 $ 3,043,753.92 $

NHS-0000-00(764) NH-009-2(93) BRST-065-2(19) STP-001-6(35) NH-STP-75-3(203) STP-00MS(7) CSSTP-0006-00(416)
STP-187-1(15) CSSTP-0006-00(049)

PI high due GSP Brdg CR246 did not escalate all the

rates and did not use the correct rates for engineer and

technician

$

PI high due to rounding

$

PI high due to rounding

$

PI high due to rounding

$

exact match

$

due to rounding

$

PI low due to rounding

$

Envir 1 incorrectly m ultiplies Project Manager and Chief

Ecologist rates and hours

$

$

4,729,271.09 $ 1,671,655.45 $ 1,942,352.09 $
539,553.80 $ 1,089,778.12 $ 2,431,303.02 $ 2,048,398.55 $

4,729,271.12 $ 1,671,655.44 $ 1,942,352.03 $
539,553.79 $ 1,089,778.12 $ 2,431,303.01 $ 2,048,421.70 $

3,554,702.18 $ 3,554,072.15 $ 4,506,971.45 $ 4,506,971.43 $

NHS-000-00(804)

PI high due to Srvy 1 (LandAir) and Srvy 2 (Long)

divided the product of rate and hours by 2 for Special

Studies phase and also did not multiply in the fixed fee

rate of 10%. Also carried the wrong total to the

summary workseet for Sue 2

$ 3,642,088.95 $ 3,629,826.96 $

(0. 01 )
(0. 01 ) 0. 00 0. 01
(0. 01 )
(0. 02 ) 0. 01 0. 06 0. 01
0. 01 (2 3. 15 )
63 0. 03 0. 02
1 2 ,26 1. 99

CSSTP-0007-00(217) PI high due to original RW 1 not including the 10% fee $ 2,497,146.24 $ 2,497,146.11 $

STP-002-6(48) BRF-002-6(49) BRF-002-5(50) STP-0000-00(412)
STP-0003-00(682) BR-0001-00(221)

PI is high by $136,404.31 - Cost Summary does not include

the SUE2 costs. The SUE2 discipline does not use escalated

labor rates for Principal and Project Mgr. Other SUE2 rates

used are escalted rates.

$

exact match

$

exact match

$

due to rounding

$

NO ROADW AY DATA - PI high due to SUE 1 worksheet did

not total the direct labor costs correctly. Off by

$30,397.56 plus OH and fee costs

$

PI low due to rounding

$

4,468,480.70 $ 233,071.75 $ 218,445.53 $
2,478,449.96 $

4,329,703.36 $ 233,071.75 $ 218,445.53 $
2,478,449.57 $

5,625,784.81 $ 5,625,783.07 $ 378,413.70 $ 378,414.38 $

0. 13
13 8 ,77 7. 34 0. 00 (0. 00 ) 0. 40
1. 74 (0. 68 )

STP-014-1(69) STP-014-1(67) CSSTP-006-00(431)
STP-076-1(32)

PI low due to rounding

$

PI high due to Map 2 - multplies the rate x hrs incorrectly,

Also, Geo 1 did not multiply the current rate times the

escalation rate. On Sue 2 - principal , computed the escalated

rate incorrectly.

$

Unusual Rdwy 1 labor costing - different labor rates were

used within the same discipline - first 4 phases had one labor

rate, remaining phases uses another labor rate. Handled by

splitting the Rdwy 1 discipline into two separate disciplines

each with a different escalation rate. Also for ROW , all costs

were manually entered into the P I database due to the non

standard Phase descriptions. After all the manual entries

there is an exact match.

$

3,574,241.60 $ 3,574,242.27 $ 2,841,309.21 $ 2,841,319.30 $
2,717,475.94 $ 2,717,475.94 $

Unusual Rdwy 1 labor costing - different labor rates were

used within the same discipline - first 4 phases had one labor

rate, remaining phases uses another labor rate. Handled by

splitting the Rdwy 1 discipline into two separate disciplines

each with a different escalation rate. Also for ROW , all costs

were manually entered into the P I database due to the non

standard Phase descriptions. After all the manual entries

PI was high by $.47 due to rounding.

$

26

2,657,986.66 $

2,657,987.13 $

(0. 67 ) (1 0. 09 )
(0. 00 )
(0. 47 )

STP-2992(2)

In the E nvr2 discipline the Fixed fee formula in Concept

Development was missing. Also, in MAP 1 the formulas for

Other Direct Costs were incorrect in the detail section of the

spreadsheet.

$ 1,874,545.79 $

1,874,670.16 $

(124.37)

NHS-0000-00(297) STP-159-1(14) STP-155-1(21)
BRST-093-1(44)

due to rounding

$ 2,803,283.96 $ 2,803,317.61 $

(3 3. 65 )

Envir1 - P roject Ecologist labor rate was not escalted correctly

$ 1,837,080.42 $ 1,837,470.36 $

(389.94)

PI low by $.01 due to rounding

$ 3,822,509.86 $ 3,822,509.87 $

RW Y1 - Cerical Labor rate not entered or link ed. Also

the Cost Summ ary Tab has aded the costs for other

consultants that were not included as separate tabs -

Only prime was in cost spreadsheet.

$

498,223.17 $ 1,200,998.62 $

(0. 01 ) (70 2 ,77 5. 45 )

STP-001-1(55)

PI low due to rounding

$ 5,869,668.38 $ 5,869,668.32 $

0. 06

BHFLB-001-1(57) BRG-0005-00(879) CSBRG-0007-00(022)

exact match

$

Rdwy2 had no item description for two other direct

costs. Also, Srvy1 had an arbitrary labor rate for Project

Manger, was not linked lik e other labor rates. Also, the

spreadsheet Cost Summary has ROW acquisition cost

of $51,700 however there is no ROW worksheet in the

cost proposal.

$

PI low by $799,486.67. Cost Summ ary worksheet has

Geo1 cost of $696,086.66 however the actual Geo1

worksheet only has $202,972. Also the Cost Summ ary

has ROW Acquisiton costs of $103,400 but there is no

RO W worksheet.

$

339,582.60 $ 339,582.60 $ 998,932.99 $ 1,045,799.08 $ 2,287,533.95 $ 3,087,020.62 $

(4 6 ,86 6. 09 ) (79 9 ,48 6. 67 )

NH-017-1(22)

PI low due to rounding

$

971,962.11 $ 971,962.12 $

(0. 01 )

STP-021-1(24) STP-021-1(25)

Svry1 did not escalate the labor rate with form ulas or use formula for labor rate in hours X labor rate.
Svry1 did not escalate the labor rate with form ulas or use formula for labor rate in hours X labor rate.

$ 4,100,681.79 $ 4,091,968.10 $ $ 4,807,958.95 $ 4,808,677.74 $

8 ,71 3. 69 (718.79)

STP-0002-00(862) STP-0004(001) CT 2 STP-065-3(55) STP-198-1(20)

Envir1 had error in PI num ber and did not use escaled

rate for hours X rate. SUE1 used the wrong rate (non

escalated rate) to multiply by hours.

$

Envr 4.1 Other direct costs started on rown 57 and

should have started on row 58 creating errors in other

direct costs. PI $.01 high due to rounding.

$

PI high by $.01 due to rounding. Discipline 9/1 for RW Appriasal and Acquisiton was added to balance 1P,I6h7i6g,h60b0y $.03 due to rounding. Needed Discipline 9/1 $ RW Appriasal & Acquisition, entered directly to database, did not enter in cost spreadsheet or summary $

2,755,901.11 $ 2,769,026.31 $ 900,411.86 $ 900,411.85 $
5,556,951.74 $ 3,880,351.73 $ 5,994,154.11 $ 5,994,154.08 $

(1 3 ,12 5. 20 ) 0. 01
1 ,67 6 ,60 0. 01 0. 03

PI high by $8.36, On G eo1 had to enter pre escalted

MSL-0004-00(646)

rates manually to tblDiscipline.

$ 3,549,269.75 $ 3,549,261.39 $

8. 36

STP-002-7(22) STP-002-7(20)

PI high by $13,079.29 due to wrong amounts entered in

summary worksheet. The amount in the Geotechnical

worksheet is not the amount carried to the summary

costs worksheet.

$

2,151,928.07 $

2,138,848.78 $

PI low by $10.24 due to rounding

$ 4,413,179.97 $ 4,413,190.21 $

1 3 ,07 9. 29 (1 0. 24 )

27

APD-056-2(29)

PI high by $11,677.66 due to Geotech not using the

labor escaltion rate entered in K23. A 10% is entered

but the rates are not escalted by 10%

$

STP-1336(11)

PI low by $.01 due to rounding

$

STP-0003-00(701) Hall PI low by $28.94 due to rounding.

$

CSBRG-0006-00(319) PI high by $20.56 due to rounding

$

BRSLB-1320(3)

PI low by $.02 due to rounding

$

BRSLB-1320(4)

PI low by $.02 due to rounding

$

CSBRG-0006-00(432) STP-1583(12)

PI high by $.01 due to rounding.

$

PI high by $5,950.94 due to StreetsSmart Survery 5/2 -

there is a fixed fee of 10% indicated in cell F6 but no fee

in cell F10 - Should there be a fee? Changed to data

table to 0%

$

PI high originally high by $6,539.66 because the Geotechnical Discipline did not use the 5% escalated

CSBRG-0007-00(162) STP-0000-00(820)

rate called for in cell K23. Also, the wrong totals from the

Brdg 1 worksheet were carried to the Cost Summary

W orkSheet. Corrected the database by changing the

escalated rate to the base rate. No correction m ade to

database for wrong totals being carried to the summ ary

she et .

$

PI high by $24.45 due to rounding and recom puting

escalation rate from FV vlaues to straight percent

increase. Some disciplines used FV based on num ber

of periods.

$

STP-0000-00(821)

PI high by $2.46 due to rounding

$

CSSBRG-0006-00(468) PI low by $.01 due to rounding

$

PI high by $6,597.16 due to Bridge 1/Construction

Services labor costs not picked up in the calculations

CSBRG-0005-00(963)

therefore not in the top summ ary section of the Bridge 1

worksheet. Also Construction Services other direct cost

was not carried left to the total column therefore not

picked up in the Cost Summary worksheet. Database

does not include these costs therefore agrees with

spreadsheet

$

3,788,156.93 $ 3,237,117.45 $ 3,406,634.57 $
669,154.57 $ 601,679.61 $ 756,928.46 $ 729,392.99 $
6,099,127.14 $
609,908.70 $
2,635,432.32 $ 2,084,652.44 $
577,006.68 $
562,999.31 $

3,788,099.62 $ 3,237,117.46 $ 3,406,663.52 $
669,134.01 $ 601,679.63 $ 756,928.48 $ 729,392.98 $
6,098,924.15 $
607,423.12 $
2,635,438.29 $ 2,084,670.04 $
577,006.69 $
562,999.30 $

STP-065-2(13)

Geo1 did not have any labor hour breakdown in the

Hours & Cost Estimate section therefore only the other

direct costs were entered into the database. The

Summary includes R/W Acq amounts but no detail

worksheet was found in the pdf file therefore no RW

amounts entered in database

$

1,528,709.10 $ 2,144,496.69 $

STP-187-1(17) STP-9408(3)
STP-2868(1) STP-0005-00(750)
STP-128-1(13)CT 2

PI high by $4,608.42 due to not carrying Concept

Validation total costs to the summ ary sheet.for Bridge 1. $

PI high by $.03 due to rounding

$

Difference due to Original speadsheet Summary Page

does not include ROW costss

$

exact match

$

RO W Acquisition costs not itemized - only totals

provided. Traffic Ops costs were not correctly

transferred to beginning summ ary section.

$

3,290,361.32 $ 3,285,752.90 $ 5,065,890.55 $ 5,065,890.52 $

4,752,377.32 $ 3,830,603.25 $

81,346.59 $

81,346.59 $

6,465,682.07 $ 6,460,821.06 $

5 7. 31 (0. 01 ) (2 8. 95 ) 2 0. 56 (0. 02 ) (0. 02 ) 0. 01
20 2. 99
2 ,48 5. 58
(5. 97 ) (1 7. 60 )
(0. 01 )
0. 01
(61 5 ,78 7. 59 ) 4 ,60 8. 42 0. 03
92 1 ,77 4. 07 -
4 ,86 1. 01

STP-3600(2) STP-0002-00(871) STP-0004-00(913)
NH-005-5(37)SP

ROW - formulas were not used throughout the cost summary.

$ 1,842,374.03 $ 1,845,897.39 $

RD2 did not have the correct sumproduct formula for Concept Development

$ 5,156,514.03 $ 5,148,337.21 $

$

PI high by $32,240.6 due to rounding and the following:

Srv - rate for Clerical not carried down correctly. Env -

row 39 calculates the escalated rate. OMR1- the rates

are escalated to row 39. Also, the ROW costs did not

include a contingent amount.

$

3,192,569.11 2,809,886.87 $

3192568.61 $ 2,809,901.40 $

(3 ,52 3. 36 ) 8 ,17 6. 82
0. 50
(1 4. 53 )

28

N H-0 68 -2(2 0) N H-0 68 -2(2 5)

PI high by $126,817.57 due to Envir1 - the first three

rate categories were escalated by the ecscalation rate,

the others were not escalated - which is correct? . Also,

the ROW costs did not include a contingent amount.

Also, OMR did not have form ulas in the totals of OMR to

add in the Other Direct costs thus the other direct costs

of $87,147 were not added in .

$

3,260,442.30 $

3,173,298.31 $

8 7 ,14 3. 99

Envir1 amounts did not carry to the Cost Summary

worksheet correctly, also OMR did not carry up the other

direct cost of $16,310.45 from ROW Acquistion into the

top of page summary nor did it carry to the Cost

Summary W orksheet correctly.. Also, did not include

the ROW contingency.

$

4,004,899.25 $

3,561,019.70 $

44 3 ,87 9. 55

BH N-0 68 -2(2 3)

data loaded - no comparision done

$ 1,216,254.00 $ 1,216,276.22 $

(2 2. 22 )

BR N-0 68 -2(2 4)

no roadway data bridge replacem ent only-data loaded -

no comparision done

$

821,342.21 $

821,351.17 $

(8. 96 )

STP-068-2(26)

$ 5,758,736.42 $ 5,719,758.72 $

3 8 ,97 7. 70

STP-068-2(27)

$ 6,922,212.00 $ 6,922,182.93 $

2 9. 07

STP-068-2(28) STP-074-2(24) STP-0003-00(681)
STP-017-3(64) STP-0001-00(420) ED S-4 41 (2 8)
STP-2576(2)

PI high by $.01 due to rounding

$ 2,140,483.83 $ 2,140,498.91 $ $ 3,395,246.92 $ 3,395,246.91 $

(1 5. 08 ) 0. 01

$

Srv1 - Survey Party Chief Labor rate was inconsistent. The Summary Cost Sheet does not pick up the Trf2

costs. There is no detail on ROW Services cost of

$61,110 that is found in the Summary W orksheet. Not

included in database.

$

Off due to rounding

$

Did not import ROW costs of $246,624.73. Also

rounding differences

$

Aerial Mapping spreadsheet had other direct costs

entered in summary section. No supporting detail was

available. Did not import ROW costs of $65,241.12.

$

4,373,788.10 $ 4,373,788.12 $
706,027.74 $ 704,735.27 $ 1,837,170.87 $ 1,837,277.28 $ 3,493,507.74 $ 3,740,094.93 $
705,789.46 $ 771,030.58 $

(0. 02 )
1 ,29 2. 47 (10 6. 41 )
(24 6 ,58 7. 19 ) (6 5 ,24 1. 11 )

29

Attachment B Data Quality

List of fields (cell names) included in cost proposal spreadsheets showing the variation in consultant responses.

Cell Name
New Location_Desc Widening_X Overlay_X Lanes_No DesignSpeed_No DesignClass_Type Access_Type OutsideShoulder_Type Median_Type MedianWidth_No ProfileChg_YN ApprxAADT_No MainlineFuncClass_Type RuralUrban_Type Terrain_Type CountOfRoadwayTotalLength_NO Interchanges No UnsignalizedInterchanges_No SignalizedInterchanges_No Parcels_No ParcelsperRoadwayMile Driveways_No DrivewaysperRoadwayMile_No ConstPlanScale_No PublicInfoMeetings_No PublicHearing_YN StakeholderMeetings_No AdvisoryCommittee_YN CACMeetings_No WebSite_YN Newsletters_YN Hotline_YN PIOther1_Desc PIOther1_No PIOther2_Desc PIOther2_No PIOther3_Desc PIOther3_Desc PIOther3_No ConstPlanScale_No PIOther4_Desc PIOther4_No PIOther5_Desc PIOther5_No

Valid Valid Numeric Text "x" Values

263

26

30

66

258

41

339

87

244

277

266

433

31

14

6

102

77

78

78

81

11

58

51

70

70

70

73

80

57

29

1 18

"No" "Yes" or or "Y" "N"

0

other blank alpha/numeric

7

49

5

3

22

19

39

4

2

5

87

24

96 1

3

67

74

4

93

4

138

175

92

72 108

43

231

2

1

1

70

20

29

10

4

11

5

3

23

25

10 25

21

6 40

6

3

38 38

1

15 52

14

2

51 28

2

51 28

2

51 28

80

6

74

1

63

1

10 70

81

81

80

6

73

1

81

81

81

81

30

Cell Name
PIOther6_Desc PIOther6_No PIOther7_Desc PIOther7_No EnvironmentDoc_Type Section4fDoc_YN Section4fProperties_No ParkRecRefugeSites_No HistoricSites_No ArchaeologicalSites_No ArchaeologicalHighProb_No ArchaeologicalSitesTests_No FederallyOwnedProp_No Cemeteries_No ChurchesCommSites_No EnvJusticePotential_YN WetlandCrossings_No StreamCrossings_No FishMusselSpecialSurvey_YN TimeSensitiveTESpecies_YN 404Permit_YN PAR_YN FloodPlanInvolvement_YN USTHazardousWasteSite_No NoiseAnalysis_YN EnviroOther1_Desc EnviroOther1_YN EnviroOther2_Desc EnviroOther2_YN EnviroOther3_Desc EnviroOther3_YN ReqdPermit1 ReqdPermit2 ReqdPermit3 ReqdPermit4 DesginMapNegScale_No DesignMapExposurers_No SerialModels_No MappingArea_No DesignMapObsecureArea_No DesignMapExistingPhotoUse_YN DesignMapPhotoFlight_Date DesignMapOther1_Desc DesignMapOther1_No DesignMapOther2_Desc DesignMapOther2_No ConLvlPhotoNegScale_No ConLvlPhotoExposures_No ConLvlPhotoDTMArea_No ConLvlPhotoExistingPhotoUse_YN

Valid Valid Numeric Text "x" Values
58
16 13 29 8 7 14 3 24 26
28 35

26 5 2 2 26 1

75 26 27 27 25

-

-

-

21 17 19
31

"No" "Yes" or or "Y" "N"

0

other blank alpha/numeric

81

81

81

81

23

16

22 32

11

47

53

34

3

51

1

53

46

1

60

43

41

15

16 33

17

41

35

23

14 38

6

24

16 38

3

35

2 33

11

31

13 33

4

32

12 34

3

41

38

5 35

3

76

4

77

79

81

79

81

55

80

81

81

45

1

50

49

46

50

6

33 41

1

-

-

-

-

-

-

81

81

81

81

54

1

58

58

3

25 52

1

Cell Name
ConLvlPhotoFlight_Date ConLvlPhotoOther1_Desc ConLvlPhotoOther1_No ConLvlPhotoOther2_Desc ConLvlPhotoOther2_No FieldSurveyObscureArea_No LakePondSurveyArea_No BridgeSurvey_No HydrologicSurvey_No DrainageSurveyLength_No SurveyEnhancementAreaTotal_No FieldSurvOther1_Desc FieldSurvOther1_No FieldSurvOther2_Desc FieldSurvOther2_No FieldSurvOther3_Desc FieldSurvOther3_No LAMPNegScale_No LAMPExp_No LAMPSteroModels_No LAMPLinearDist_No LAMPOther1_Desc LAMPOther1_No LAMPOther2_Desc LAMPOther2_No LAMPOther3_Desc LAMPOther3_No UrbanResidential_No UrbanBusinessPublic_No SuburbanResidentail_No SuburbanBusinessPublic_No RuralResidential_No RuralBusinessPubic_No OpenPastureCultivated_No Woods_No OvrhdElectrical_YN UndrgrdElectrical_YN UtilElecLength_No OvrhdComm_YN UndrgrdComm_YN UtilCommLength_No OvrhdGas_YN UndrgrdGas_YN UtilGasLength_No OvrhdSewer_YN UndrgrdSewer_YN UtilSewerLength_No OvrhdWater_YN UndrgrdWater_YN UtilWaterLength_No

Valid Valid Numeric Text "x" Values

-

-

-

28 13 20 25 23 9
18 8
2
1 1 15

5 8 11 16 27 23 25 49
32
23 31 6
24
2 23
2 19
1 3 23
32

"No" "Yes" or or "Y" "N"

0

other blank alpha/numeric

-

-

-

-

-

-

81

81

81

81

49

66

60

55

55

72

63

73

79

80

1

80

80

65

81

81

81

81

81

81

81

81

81

76

71

70

65

54

58

56

30

41

8

19

4

55

3

57

44

6

23

2

48

2

55

2

1

14 62

4

29

50

57

1

2

13 61

1

4

22

4

51

2

60

2

2

13 60

1

4

28

1

47

2

56

2

Cell Name
OvrhdTransElec_YN UndrgrdTransElec_YN UtilTransElecLength_No OvrhdTransPetro_YN UndrgrdTransPetro_YN UtilTransPetroLength_No UtilOther1_Desc OvrhdOther1_YN UndrgrdOther1_YN UtilOther1Length_No UtilOther2_Desc OvrhdOther2_YN UndrgrdOther2_YN UtilOther2Length_No UtilOther3_Desc OvrhdOther3_YN UndrgrdOther3_YN UtilOther3Length_No UtilImpactRtg_Type UtiliytPoles_No blank1 LevelATestHoles_No LevelBLength_No LevelCLength_No LevelDLength_No SoilSurveryLength_No (feet) WallSites_No WallSurveyLength_No (feet) BridgeSites_No BridgeBents_No PavementCorings_No blank2a blank2b blank2 blank3 Phase2nvirSiteAssmntsIndComm_No Phase2nvirSiteAssmntsUST_No blank4 blank5 blank6 ResidentialNoRelocation_No ResidentialRelocation_No CommercialNoRelocation_No CommercialRelocation_No NPOGovernmentNoRelocation_No NPOGovernmentRelocation_No CondmntnCasesEstimate_No blank7 blank8 blank9

Valid Valid Numeric Text "x" Values
2 3
2 4 1
3 3
1
1 12
23 36 51 13 2 18 3 10 33 28 5
2 4
33

"No" "Yes" or or "Y" "N"

0

other blank alpha/numeric

17

3 53

6

4

9 65

3

75 1

2

13 64

4

3

10 65

3

77

2

77

2 79

79

1

76

2

78

3 78

1

79

1

79

1

80

1

81

81

80

69

56 1

81

42 1

28 1

65 1

76 1

41 1

21

46 1

64

7

46

2

50

3

45 1

31 1

31

1

81

81

79

77

81

81

50

1

3

1

3

1

3

1

3

1

3

1

3

1

22

1

22

1

9

22

1

9

22

1

9

Cell Name
blank10 blank11 RoadwayTotalMiles_No PercelsperTotalMiles_No DrivewaysperTotalMiles_No

Valid Valid Numeric Text "x" Values

"No" "Yes" or or "Y" "N"

0

other blank alpha/numeric

22 1

9

22 1

9

81

81

81

34

Attachment C Database Relationship Diagram
35

SECTION II: TACIT KNOWLEDGE, CAPACITY AND CONTRACTING: A SURVEY OF
CONSULTANTS AND GDOT MANAGERS Principal Author
Dr. Barry Bozeman University of Georgia
Co-Authors Dr. Mary Feeney University of Illinois-Chicago Dr. Craig Smith University of Arizona
36

Executive Summary
This report describes results of a survey of Georgia Department of Transportation (GDOT) staff and a companion survey of consultants who have contracted with GDOT. The survey results and analysis comprise the major deliverable from the project "TACIT KNOWLEDGE, CAPACITY AND CONTRACTING: A SURVEY OF CONSULTANTS AND GDOT MANAGERS."
Among the most important questions considered here are these: What are GDOT managers views about out-sourcing, including maintaining in-
house capacity? What factors mitigate the capacity impacts of contracting? How do GDOT staff and consultants diverge in their views about administrative
procedures, including the extent and sources of "red tape?" How do GDOT staff and consultants diverge in their views about contractor-staff
relationships and the role of contractors?
Two separate questionnaires were designed to permit comparison in the symmetry between consultants and GDOT staff with respect to perceptions and activities pertaining to contracting and contract management. The sample of GDOT staff included, after data cleaning and adjustment for incorrect addresses, 208 contacts. After a variety of request procedures (see the Appendix), the survey yielded 96 responses. Similarly, 208 GDOT consultants were contacted and 95 responses were received.
Regarding GDOT respondents' views about outsourcing, the questionnaire asked GDOT respondents a series of questions about the circumstances under which tasks and functions should be outsourced and the consequences of doing so. GDOT respondents agree overwhelmingly (99% "strongly agree" or "agree") that there are core competencies that should not be outsourced. There is somewhat less consensus about the
37

need for "functional redundancy." On responses to the item "All tasks outsourced by GDOT should also be done in-house to ensure that GDOT can compete with consultants," 40% either disagreed or disagreed strongly.
To further consider views about outsourcing and in-house functionality, the researchers developed an "In-House Functionality Index," an additive scale based on all the items focusing on all the items pertaining to possible reasons for preserving in-house capability to perform functions that are outsourced. The question, then, is what factors are related to respondents' general views that functionality should be retained even with outsourcing? Correlation results show that those who define a "good engineer" in terms of technical activities are more likely to have a high score on the In-House Functionality Index, as are those who feel that GDOT managers are not able to concentrate sufficiently on engineering.
The GDOT questionnaire included a number of items asking about characteristics of a "good consultant." The two areas about which there is greatest agreement are that consultants should be technical experts and that they should be familiar with and closely follow GDOT procedures. There is less agreement about what might be referred to as matters of personal style, especially control.
Since the response seemed to have underlying scale properties that would signify a few archetypal modes of managing consultants, a factor analysis was performed to test for possible dimensional properties. The results of the factor analysis yielded five dimensions. By far the leading dimension is "The Boss," defined by the respondents' view that he or she is in control and the consultant should, first and foremost, be working
38

for the manager. Consultant management roles were examined in terms of correlations predicting the extent to which respondents embrace a given role. The results showed:
(1) "The Boss" consultant management role is correlated with (a) being a member of a professional association (negative correlation), (b) the In-House Functionality Index; (2) The "Stick to the Contract" consultant management role is correlated with the respondents' reported level of job satisfaction (negative correlation); (3) the "Stick to GDOT rules" consultant management role is correlated with (a) the attitudinal item "I feel better prepared to perform technical work rather than managing consultants" and (b) the attitudinal item "my work should be more technical rather than managing consultants"; (4) The "Flexible Consultant" role is correlated with the respondents' age; (5) The "Keep Distance" role is correlated with, among other items, the percent of work week typically devoted to oversight of consultants.
The focus of the questionnaire mailed to consultants is contractor relations with GDOT. One question asked respondents to indicate the GDOT offices they had worked with during the past year. For the total of 392 contracts, the greatest number was with the Office of Consultant Design (78) and the Office of Environmental Location (66). Respondents, on average, do more business with GDOT than with any other category. Nevertheless, there are many respondents in the sample (12%) whose work with GDOT represents 10% or less of their overall work, just as there are some (5%) who depend on GDOT for all its contracts.
Among respondents, the most common contract type is cost plus contracts (mean percentage 51.4%), with the other categories being about roughly equal. The questionnaire also asked respondents about their preference for contract type and the most common choice was "no preference."
39

The respective questionnaires (consultants and GDOT staff) were designed to permit some comparison in perceptions and activities pertaining to contracting and contract management. Both GDOT staff and consultants were asked about the level of organizational red tape in their organization and its sources.
The ratings for red tape in GDOT are higher than for the consulting firms, with the mode red tape assessment for consulting firms being "3" versus "7" for GDOT and with means being 6.7 for GDOT and 4.3 for the consulting firms. An issue more directly relevant to the topic at hand is the level of red tape in contracting relations between GDOT and private firms.
A question on both the GDOT and the consultant questionnaires was- "How would you assess the level of red tape in the contracting relationships between GDOT and private firms?" The differences in perspective between GDOT employees and consultants are not marked. In general, the GDOT respondents tend to report a somewhat higher level of red tape in contracting than do the consultants. The respective respondents differ somewhat with respect to their views about origins of red tape in contracting. GDOT respondents identify three origins of red tape that are roughly of equal impact- actions of GDOT employees (23.5%), federal acquisition regulations (23%) and GDOT rules and regulations. The consultant respondents' assessments are similar except they provide somewhat lower scores to individual GDOT personnel and somewhat higher scores for the Georgia General Assembly.
The GDOT respondents were asked to respond to the following two items, one asking how many GDOT employees are required to sign off on the paperwork for a contract, the other asking how many sign-offs are required for a contract modification.
40

The average is about six sign-offs for both the initial contract and for the modification of the contract. The range for each is considerable, with 2-20 sign-offs for the initial contract and 2-16 for the modification.
The final research question examined pertains to GDOT staff and consultants respective views about contracting relations. In general, the consultants' views of GDOT contracting relations seem quite positive. A sizable majority agrees that GDOT is "evenhanded" in contracting; the vast majority feels that GDOT keeps its promises and that GDOT is trustworthy; more than three-quarters agree that their experience with GDOT managers has been positive. One interesting result is that more than one-third feel that "GDOT often acts opportunistically" at their firm's expense.
With respect to GDOT views of consultants, GDOT is somewhat less likely to wish to work with the consultants (compared to consultants' willingness to work with GDOT) when specifications are vague; the consultants, according to GDOT, generally "keep promises" but according to the consultant respondents, not at as high a rate as GDOT does; similarly, GDOT scores higher on being "trustworthy." Introduction
This report describes results of a survey of Georgia Department of Transportation (GDOT) staff and a companion survey of consultants who have contracted with GDOT. The survey results and analysis comprise the major deliverable from the project "TACIT KNOWLEDGE, CAPACITY AND CONTRACTING: A SURVEY OF CONSULTANTS AND GDOT MANAGERS."
A distinctive feature of this study is that it includes some questions common to both the GDOT managers' survey and the consultants' survey and, thus, allows
41

comparison of perceptions and opinions. However, many of the issues in the respective surveys are distinct from one another.
Among the most important questions considered here are these: What are GDOT managers views about out-sourcing, including maintaining in-
house capacity? What factors mitigate the capacity impacts of contracting? How do GDOT staff and consultants diverge in their views about administrative
procedures, including the extent and sources of "red tape?" How do GDOT staff and consultants diverge in their views about contractor-staff
relationships and the role of contractors?
The report is divided into three sections. The first section focuses on the GDOT survey, the second on the Consultant survey, and the third on comparison and overall conclusions. SECTION I. THE GDOT SURVEY
Who are the GDOT Respondents? The sample of GDOT staff (hereafter "GDOT sample") included, after data
cleaning and adjustment for incorrect addresses, 208 contacts. After a variety of request procedures, the survey yielded 96 responses (46.6% response rate). While this is a somewhat lower rate than received by some recent GDOT surveys,2 the response rate is somewhat better than average for general surveys of organizational employees.3 Details about the sample and sampling procedures for the study are provided in the Appendix.
2 The fact that the rate is lower than earlier surveys at GDOT is not surprising inasmuch as de facto panel surveys almost always have diminishing returns due to respondent fatigue and a lack of novelty with increased exposure to surveys. 3 L. Kanuk and C. Berenson, "Mail Surveys and Response Rates: A Literature Review. Journal of Marketing Research, 12, 4, November, 1975, pp440-453.
42

Figure 1 gives the educational characteristics of the respondents to the GDOT sample. The figure shows that most of the respondents (67%) are graduates from fouryear colleges and a quarter (25.5%) either received a graduate degree or attended graduate school.
Figure 1. GDOT Respondents' Education
60.0%

Percent

40.0% 20.0%
0.0%

67.0%

2.1% HS Grad

5.3% Some College College Grad

7.4%
Attended Graduate

18.1%
Graduate/Prof Degree

Table 1, below, provides the age distribution for respondents, according to birth year. The median birth year for all respondents is 1965, meaning that the typical respondent would have been about 42 years old at the time the questionnaire was returned. The range among respondents is between the birth years 1946 and 1983.

43

Table 1. Age Distribution for GDOT Respondents

Birth Year Frequency Percentage

Range

1945-1955 16

17.5%

1956-1965 36

39.5%

1966-1975 28

30.7%

1976-1983 8

8.8%

The gender and race distribution among respondents is highly skewed with 74 male respondents (80.4%) and 19 female respondents (19.6%); 71 respondents are white, 13 black, and 19 did not care to provide a response about race.4
Given the nature of the study, it is particularly useful to consider the private sector experience of GDOT respondents. As we can see from Table 2, 41.5% of the respondents have had a managerial or professional job in the private sector (in a general sample of Georgia state government managers, 33.5% of respondents have had any significant employment in the private sector).5 In a 2003 survey 6of project managers, only 21% of respondents reported they had private sector experience (however, the character of the two samples is not identical; the current studies focuses exclusively on managers and professionals, with all others removed from the sample).

4 Other racial groups are in such trace amounts that, for confidentiality's sake, we do not provide a number or percentage.
5 B. Bozeman and M. Feeney, "The National Administrative Studies Project-III: An Overview" University of Georgia, June, 2007.
6 G. Kingsley, et al., 2003.
44

Table 2. Have you ever worked in the private sector in a managerial or professional job?

NO YES
Total

Frequency 55 39
94

Cumulative Percent 58.5
100.0

45

Outsourcing Views We begin with one of the most important questions motivating the study- views
about outsourcing. The survey asked GDOT respondents a series of questions about the circumstances under which tasks and functions should be outsourced and the consequences of doing so.
Figure 2 gives the distribution of agree-disagree responses to the item "There are certain tasks which GDOT must do in-house in order to maintain its core competencies."

Percent

Figure 2 Outsource: Core Competence

60.0%

40.0%

68.1%

20.0%

30.9%

0.0%

1 1%

1

2

3

There are certain tasks which GDOT must do in-house in order to maintain its core competencies (Strongly Agree=1, Strongly Disagree=4)

46

As we can see from Figure 2, respondents agree overwhelmingly (99% "strongly agree" or "agree") that there are core competencies that should not be outsourced.
There is somewhat less consensus about the need for "functional redundancy." Figure 3 gives responses to the item "All tasks outsourced by GDOT should also be done in-house to ensure that GDOT can compete with consultants." There is considerable distribution of responses on this item with about 60% either agreeing or agreeing strongly and about 40% disagreeing or disagreeing strongly.

Figure 3 Outsource: Do In-House to Compete

30.0%

Percent

20.0% 10.0%

28.7%

31.9%

30.9%

8.5%

0.0%

1

2

3

4

All tasks outsourced by GDOT should also be done in-house to ensure that GDOT can compete with consultants (Strongly Agree=1, Strongly Disagree=4)

Figure 4 takes the same premise but a different rationale. This item asks respondents if functional duplication should occur in order to ensure that GDOT can monitor consultants. The distribution in this case is monotonic, with the largest percentage strongly agreeing (39.4%) and the smallest percentage strongly disagreeing (7.4%).

47

Percent

40.0%

Figure 4 Outsource: Monitor Consultants

30.0%

20.0%

39.4%

33.0%

10.0%

20.2%

7.4%

0.0%

1

2

3

4

All tasks outsourced by GDOT should also be done in-house to ensure that GDOT can monitor consultants (Strongly Agree=1, Strongly Disagree=4)

Another rationale for maintaining functional competence when outsourcing is to ensure that GDOT will be current with technical and policy change. Figure 5 gives responses in connection with this rationale. As before, the responses are monotonic, with most respondents strongly agreeing (47.9%) and the least respondents strongly disagreeing (3.2%).

Percent

Figure 5 Outsource: Keep Up with Technical and Program Change
50.0%

40.0%

30.0% 20.0% 10.0%

47.9%

28.7%

20.2%

0.0%

3.2%

1

2

3

4

All tasks outsourced by GDOT should also be done in-house to ensure that GDOT can keep up with new techniques in engineering design, policies, programs and trends (Strongly Agree=1, Strongly Disagree=4)

48

Another way to analyze views about maintaining in-house functionality in

activities that are outsourced is to consider this motivation apart from particular

rationales. One way to do this is to develop a simple additive scale focusing on all the

reasons why respondents feel that some duplication of outsourced tasks is useful. We can

term this the "In-House Functionality Index" and it is created simply by adding responses

to each of the four items identified above. For convenience, we reverse the scale such

that a higher number implies stronger support for maintaining in-house capacity.

Theoretically, the scale ranges from 0 to 16, but the actual range of responses is 3 to 12

(median 9). A number of attitudinal variables were correlated (see Table 3) to the scale

and three were significant at the .10 level or greater.

Table 3. Correlation for In-House Functionality Index and Selected Predictor

Variables

Selected Predictor Variable
Year Born Highest Level of Education Worked Previously in the Private Sector Job Satisfaction No Intention to Leave GDOT Number of Hours Worked per Week "A good engineer is concerned with managing the technical aspects of our projects, not with consultant management." "GDOT managers are not able to concentrate sufficiently on engineering..."

In-House Functionality Index (Pearson r) n.s. n.s.
n.s. n.s. n.s. n.s. .183*
.419****

Key: n.s. = Not statistically significant * = Significant at .10 level ** = Significant at .05 level *** = Significant at .01 level **** = Significant at .001 level or >

49

The results from the zero order correlation show that differences in the In-House Functionality Index pertain to attitudinal variables. Specifically, those who define a "good engineer" in terms of technical activities are more likely to have a high score on the In-House Functionality Index, as are those who feel that GDOT managers are not able to concentrate sufficiently on engineering. While these findings certainly are not inconclusive, it seems likely that attitudes about maintaining in-house functions are closely related to respondents' role definitions (as either consultant managers or technical engineering mangers). Views about Consultants
One might expect that GDOT managers would have some diversity of views about relationships with consultants. In this section, we consider difference in views as to the characteristics of a "good consultant." First we consider the extent to which respondents are involved in consultant management.
Respondents were asked to indicate the composition of their work according to a variety of task categories, among these "oversight of consultants." Presumably, many of the views about consultants and consultant management might be affected by the relative work time devoted to consultant management. Table 4 gives the means for each of the task categories as well as for responses to the question "during the typical work week, about how many hours do you work (including work done away from the office but as part of your job)?"
Table 4 indicates that, on average, about one-quarter of GDOT managers' work time is devoted to consultant management. However, the responses showed considerable
50

variance, with a range from zero to 100% of work activity devoted to the task of consultant management. The standard deviation for all respondents is 20.7.

Table 4: Means for Work Task and Hours Worked

Means for Work Task Categories Oversight of GDOT employees (%) Oversight of consultants (%) Contract design (%) Auditing (%) Technical work (%) Other (%)7
Hours worked during a typical work week

Mean
36.9 23.7 7.1 1.5 25.6 17.8
42.72

The GDOT questionnaire included a number of items following this lead question: "The following items deal with your opinions about consultants and their work with you for GDOT...Please indicate your level of agreement based on the scale indicated below" (where the scale is "Very Important," "Important," "Somewhat Important," and "Not Important." A good consultant should...
Table 5 below provides the frequency distributions for responses to each of the fourteen items pertaining to views about "good consultants." As we can see from Table 5, there are several items about which there is great agreement and others about which there is relatively little consensus. The two areas about which there is greatest agreement are that consultants should be technical experts and that they should be familiar with and
7 The most common "other" response is general administration. 51

closely follow GDOT procedures. There is also widespread agreement that consultants

should manage communications with sub-consultants. There is less agreement about

what might be referred to as matters of personal style, especially control. For example,

respondents disagree about how important it is for a consultant to "do what I say" and

while there are few who feel that it is very important for consultants to "be loyal to me,"

responses are evenly distributed among the other three categories. Respondents vary in

their views about flexibility, both on their part and the consultants.

Table 5. Views about Qualities of "Good Consultants"

"A good consultant should...
Be an expert in a technical or specialized area Take the initiative on points not covered explicitly in the contract Have sufficient staff to handle unanticipated work change Closely follow GDOT rules and procedures Contact me only for really important problems Do what I say Have good professional ties to the consulting community Know how my GDOT office differs from other GDOT offices Understand that he/she is working for me Stick to the details of the contract Be intimately familiar with GDOT rules and procedures Be loyal to me Be flexible when I suggest needed changes not covered in the contract Manage all the communications with sub-consultants for a project

Very Important 75 19 39 75 9 18 10 11 21 26 52 4 17
56

Important 18 56 45 18 41 45 43 46 43 45 30 22 44
36

Somewhat Important 2 16 8 2 37 27 38 26 22 22 12 38 31
4

Not Important 0 3 2 0 8 5 4 12 9 1 1 30 2
2

52

Since the response seemed to have underlying scale properties that would signify a few archetypal modes of managing consultants, a factor analysis was performed to test for possible dimensional properties.8 The results of the factor analysis yielded five dimensions. Table 6 provides the dimensions (as is conventional, defined on the basis of the highest coefficients or "factor loadings"). The dimensions are in the order of their statistical importance (i.e. the extent to which they represent the attributes of the respondents). By far the leading dimension is "The Boss," defined by the respondents' view that he or she is in control and the consultant should, first and foremost, be working for the manager. The next most important dimensions are two that seem related, "Stick to Contract" and "Stick to the Rules." Respondents scoring high on these dimensions view their roles and the consultants' as very much one based on formalism and documentation. Less common roles (dimensions) are the "Flexible Contractor" and "Keep Distance." In the former case, the primary concern is exhibiting flexibility, very much at odds with the two previous dimensions. In the latter case, the manager is chiefly concerned, at least in this role, that he or she not be bothered unless a very important issue emerges; otherwise, the consultant should make sure there is sufficient capacity for the job and then do the work independently. It is important to understand that these roles are archetypes; that is, most respondents embrace multiple roles, with some being more important than others. However, it is clearly the case that "The Boss" role is much more prominent than others.
8 For those not familiar with factor analysis, it is mathematically complex in its derivation, statistically complex in its application, but, once valid results are obtained, it is usually not difficult to interpret. One way to think about the result is that it provides (at least in the most popular "orthogonal solution") a set of indices that are composed of inter-related items, but the indices are not correlated with one another. It is a good way to determine dimensionality of a set of variables and to define the dimensional properties.
53

Table 6 Consultant Management Perspective: Dimensions Emerging from Factor

Analysis (continued on next page)

Be an expert in a technical or specialized area Take the initiative on points not covered explicitly in the contract Have sufficient staff to be able to handle unanticipated work change Closely follow GDOT rules and procedures
Contact me only for really important problems Do what I say
Have good professional ties to the consulting community Know how my GDOT office differs from other GDOT offices Understand that he/she is working for me Stick to the details of the contract Be intimately familiar with GDOT rules and procedures

Rotated Factor Matrix9

Factor

"Stick to "Stick

the

to

"The Contract GDOT

Boss"

"

Rules"

"Flexible Contractor
"

"Keep Distance
"

.033

.287 -.056

-.111

.050

.060

-.118 -.007

.501

.198

-.031

.130 .120

.156

.520

.080

.019 .756

-.076

.164

.028 .048

-.018

.377

.112 .224

-.008

.058

.314 .194

.008

.259

.392 .235

-.073

.964

-.012 -.041

.184

.134

.660 .346

-.169

.196

.195 .665

.196

.095 .678 -.014 .322
.174 .187 .261 .194

9 The specification for the factor analysis: maximum likelihood factor analysis of Pearson correlation matrix, Varimax rotation, favors extracted to an eigenvalue = 1 (i.e. at least the amount of variance accounted for by any single variable in the matrix).
54

Be loyal to me
Be flexible when I suggest needed changes not covered explicitly in the contract Manage all the communication with sub-consultants for a project

.518

.287 .288

.174

.117

.116

-.073 .002

.900

-.052

.029

.704 .128

.026 -.020

Given the presence of several well-defined, easily interpreted roles, it is useful to understand the manager attributes and attitudes most closely associated with the respective roles. Thus, the researchers correlated a number of such variables with the scores for the factor dimensions (the managerial roles). The most important predictors (each significant at least the .1 level, most at the .05 level are greater) are given below.
1. "The Boss" consultant management role is correlated with: Being a member of a professional association (negative- those who are a member of one or more professional organizations are less likely to score high on this dimension). The In-House Functionality Index (positive)
2. "Stick to the Contract" consultant management role is correlated with: The respondents' reported level of job satisfaction (negative- those who are less satisfied are more likely to score high on this dimension)
3. "Stick to GDOT rules" consultant management is correlated with: The attitudinal item "I feel better prepared to perform technical work rather than managing consultants" (positive)

55

The attitudinal item "my work should be more technical rather than managing consultants" (positive)
4. "Flexible Consultant" consultant management is correlated with: The respondents' age (positive)
5. "Keep Distance" consultant management is correlated with: The attitudinal item "if I were to leave GDOT to work as a contractor or consultant I think my work would be more fulfilling" (positive) The attitudinal item "professional reputation is more important to me than rank" (negative) The attitudinal item "I identify myself as a professional more so than a public employee" (negative) Percent of work week typically devoted to "oversight of consultants" (positive)
While there are several ways in which these findings can be interpreted, the following interpretation seems a conservative one: there are distinct dimensions or roles for managing consultants, the most prominent ones among the respondents relate to personal control ("The Boss") and control through formalism ("Stick to Contract" and "Stick to GDOT Rules") and that these various roles are amenable to prediction from attribute and attitudinal variables.
56

SECTION II. THE CONSULTANT SURVEY
Who are the Consultant Respondents? In this section the consultant sample is compared on many of the attributes of the
GDOT respondents. We included project managers/contact individuals from consulting firms who have been awarded GDOT contracts. It is important to note that we only included firms who have contracted with GDOT, not those who are pre-qualified to contract with GDOT. Figure 6 examines the level of education of respondents. Nearly 96% are college graduates, with more than 42% having graduate degrees (compared to the 18% of GDOT respondents who have graduate degrees).
Figure 6. Education of Consultant Respondents
50.0%

40.0%

Percent

30.0% 20.0%

44.6%

42.4%

10.0% 0.0%

4.3% Attended College College Grad

8.7%
Attended Grad School

Grad Degree

Just as there is some difference in the educational mix for the consultant respondents, other demographic attributes are dissimilar. The race distribution for the

57

GDOT sample was somewhat more diverse than the consultant sample and included

about 14% black compared to no black respondents in the consultant sample.

For the consultant sample 15% are female compared to 20% for the GDOT sample.

What are the Characteristics of Consultant Respondents' Companies?

Regarding company characteristics of the respondents', the respondents were

asked when their firm was established (both in Georgia and worldwide) and how many

employees work for the firm (again, both in Georgia and worldwide). Table 7, below,

provides the means, range and standard deviation for these company attributes. As

indicated in Table 7, the average size of the respondents' companies is 103, but none of

the companies is exceptionally large with the largest being 700 Georgia employees (but

many of these companies are part of much larger worldwide companies). The typical

firm was established in Georgia in 1983 and worldwide in 1948.

Table 7. Attributes of Respondents' Companies

Company Attribute What year was your firm established in Georgia? What year was your firm established worldwide? How many employees work for your firm in Georgia?
How many employees work for your firm worldwide?

Minimum Maximum

1900

2005

1800

2002

Mean 1983.72

Std. Deviation
18.452

1948.86

41.322

2

700 103.38

133.194

0

30000 4037.52 7020.832

What are the Consultants' Work Relations with GDOT? The consultants were asked a number of questions about the nature of their work
relations with GDOT, including which offices they have worked with recently, the percentage of engineering consulting work performed for GDOT vs. other clients, the
58

types of contracts experienced with GDOT and the company's preference for contract type.
Table 8 provides the distribution of responses to the question "Please indicate which of the following GDOT office(s) you have worked with in the last 12 months." Since many of the respondents had worked with more than one office, they were not limited to indicating one office. Indeed, if we examine the totals (392 contracts), the average company is involved with four GDOT contracts during the past 12 months. The Office of Consultant Design and the Field Districts were the offices with the most contracts among respondents' companies but all offices were well represented.
Table 8. GDOT Offices with which Respondents Worked

GDOT Office
Office of Consultant Design Office of Environmental Location Office of Urban Design Office of Road Design Office of Preconstruction Field Districts Other Total

Frequency10
78 66 58 46 47 70 27 392

Next, we consider GDOT contracting as a percentage of the company's contracting. As we can see from Figure 7, the respondents, on average, do more business with GDOT than with any other category (which is not surprising given the way in which the sample was constituted). Nevertheless, there are many respondents in the sample

10 Only frequencies are given because the number of respondents in this case is 99 and, thus, the frequencies are very similar to the percentages.
59

(12%) whose work with GDOT represents 10% or less of their overall work, just as there are some (5%) who depend on GDOT for all its contracts.

Figure 7. Composition of Contracts
50

40

30

Mean

42.7
20

23.7

10

19.1

14.3

0

GDOT (%)

Other DOT's (federal, state,
local) (%)

Other public

Private sector

organizations (%) organizations (%)

There is also considerable variation with respect to the type of contract under which consultants work. Respondents were asked to "estimate the percentage of your work that is under each of the following contract types," with the types including "cost plus contracts, task orders, turn key contracts, and subcontracts to other prime contractors." The results appear in Figure 8. The figure shows that cost plus contracts are the most common for these respondents (mean percentage 51.4%), with the other categories being about roughly equal.

60

Figure 8. Types of Contracts for Consultants, by Percentage (Means)

60

50

40

Mean

30
51.4
20

10

21.9

21.5

23

0

Cost Plus Contracts (%)

Task Order Contracts (%)

Turn Key Subcontracts to Contracts (%) other Prime
Contractors (%)

The fact that this is the distribution of contract types says nothing about contractor preferences for contract types. Figure 9 gives the respondents preference for contracts among the choices above. The survey allowed the respondent to check more than one contract type (though none did so) and also to indicate indifference about the contract type. The most common response was that respondents did not care about

61

contract type, but among those who registered an opinion, the most preferred contract type was the task order. The sub-contract is the least often the preferred type.
Figure 9. Contractor Preference for Contract Types
25

20

Count

15

24
10
18

13

5

11

2
0

1

2

3

4

5

Type of contract form that your organization prefers to have in working with GDOT? (1=Cost Plus; 2=Task Order, 3=Turn Key, 4=Subcontract, 5=Don't
Care)

SECTION III. COMPARING GDOT AND CONSULTANT RESPONSES The respective questionnaires were designed to permit some comparison in the
symmetry between consultants and GDOT staff with respect to perceptions and activities pertaining to contracting and contract management. The primary foci are on (1) comparing views about organizational procedures and "red tape," especially with regard to contracting, and (2) comparing views about contracting relations. Comparing Assessments of "Red Tape": Its Extent and Sources
Both GDOT staff and consultants were asked about the level of organizational red tape in their organization and its sources. The consultant respondents were asked "if red
62

tape is defined as `burdensome administrative rules and procedures that have negative impacts on the organization's effectiveness,' how would you assess overall the level of red tape in your consulting firm." The parallel question for GDOT respondents was identical expect "in GDOT" was substituted for "your consulting firm." 11 Figure 10 includes two histograms, one gives the responses of consultants about red tape in their firm; the other for GDOT respondents about GDOT.
Not surprisingly, the ratings for red tape in GDOT are higher than for the consulting firms, with the mode red tape assessment for consulting firms being "3" versus "7" for GDOT and with means being 6.7 for GDOT and 4.3 for the consulting firms. This is entirely consistent with the empirical literature. 12 Most researchers feel that government organizations tend to have stronger red tape tendencies owing to, among other factors, higher levels of external control and, related, accountability requirements.
11 This is a ten-point scale item that his been used in many different studies of red tape and proved stable and reliable. For an overview of the results of studies using this and related measures see Sanjay K. Pandey and Patrick G. Scott, "Red Tape: A Review and Assessment of Concepts and Measures," Journal of Public Administration Research and Theory, 12, 2002, 553-580. 12 Hal G. Rainey, Sanjay Pandey, Barry Bozeman, "Research Note: Public and Private Managers' Perceptions of Red Tape Journal," Public Administration Review, 55, 1995, pp. 110-121.
63

Figure 10. Assessments of "Red Tape":

Consulting Firms

GDOT

25 15
20

Percent Percent

10

16.9 15.7

13.5 12.4

5

10.1

7.9 6.7
3.4

5.6 4.5 3.4

0

0

1

2

3

4

5

6

7

8

9

10

If red tape is defined as "burdensome administrative rules and procedures that have negative impacts on the organization's effectiveness," how would
you assess overall the level of red tape in your consulting firm? (0-10)

15

25 22.83
10

15.22 14.13

5

4.35 5.43

5.43 6.52

1.09
0

2

3

4

5

6

7

8

9

10

If red tape is defined as "burdensome administrative rules and procedures that have negative impacts on the organization's effectiveness," how would
you assess overall the level of red tape in GDOT? (Scale 0-10)

An issue more directly relevant to the topic at hand is the level of red tape in contracting relations between GDOT and private firms. Again, we have comparable responses from GDOT and consultant respondents to the identical item: "How would you assess the level of red tape in the contracting relationships between GDOT and private firms?" Figure 11 provides histograms depicting the responses for GDOT respondents and consultant respondents.

64

Figure 11. Assessments of "Red Tape" in GDOT Contracting:

GDOT

Consultants

As Figure 11 shows, the differences in perspective between GDOT employees and consultants are not marked. In general, the GDOT respondents tend to report a somewhat higher level of red tape in contracting than do the consultants. The median for all consultants is a value of 6 (mode 6) whereas the median for GDOT respondents is 6.5 (mode 7).13
13 The interpretation of these assessments is not entirely straightforward. One possibility is that consultant respondents are, due to confidentiality concerns, somewhat reluctant to report high levels in order to maintain future positive working relationships. More likely, the GDOT employees have a broader and more intense exposure to rules and procedures and are affected by them more regularly. If so, the issue might be more salient to them and, thus, the assessment might diverge on this account.
65

This leads to a related issue: assessment of the sources of red tape in contracting. Again, there are comparable data for GDOT and the consulting respondents. Each group was asked to assess the contributors to red tape (on a percentage basis, regardless of the actual amount of red tape) from the following sources-
U.S. Congress and the statutes, laws and policies it creates Federal Acquisition Regulations Georgia General Assembly and the statutes, laws and policies it creates GDOT rules and regulations Actions of individuals at GDOT Practices of consulting firms Other
Figure 12 gives the assessments of GDOT respondents regarding causes of red tape in GDOT contracting (apart from the actual level of red tape); Figure 13 gives the consultants' responses on the identical item.
66

Figure 12. Sources of Contracting Red Tape: GDOT Respondents
25.0

20.0

Mean Mean

15.0 10.0
5.0 0.0

23.0

22.7

23.5

15.3

8.9

10.0

3.4

U.S. Congress statutes, laws, or policies (%)

Federal Acquisition Regulations
(%)

Georgia General Assembly statutes, laws, or policies (%)

GDOT rules and
regulations (%)

Actions of individual
GDOT employees
(%)

Practices of consulting firms (%)

Other (%)

Figure 13. Sources of Contracting Red Tape: Consultant Respondents

30

25

20

15

26.9

26.7

10
13.8
5

13.7

14.8

0

2.4

1.3

U.S. Congress statutes, laws, or policies: Red Tape Origins
(%)

Federal Acquisition Regulations: Red Tape Origins (%)

Georgia General Assembly statutes, laws, or policies: Red Tape Origins
(%)

GDOT rules and
regulations: Red Tape
Origins (%)

Actions of individual
GDOT employees:
Red Tape Origins (%)

Practices of consulting firms: Red
Tape Origins (%)

Other: Red Tape Origins
(%)

The two figures show that the respective respondents differ somewhat with

respect to their views about origins of red tape in contracting. GDOT respondents identify

three origins of red tape that are roughly of equal impact- actions of GDOT employees

67

(23.5%), federal acquisition regulations (23%) and GDOT rules and regulations. The consultant respondents' assessments are similar except they provide somewhat lower scores to individual GDOT personnel and somewhat higher scores for the Georgia General Assembly.
The questions about red tape assessment, both in general and in contracting, are based on overall perceptions rather than being anchored in discrete functions. For the GDOT sample, we asked respondents to report the amount of time typically taken to accomplish certain core functions. While this only an indirect indicator of red tape, inasmuch as some functions may take longer due to task complexity, limited capacity and other reasons, these items are "red tape relevant" questions asked in many other studies and, thus, there is some benchmark for the amount of time required for core functions.14 Table 9 gives the mean and range for GDOT respondents' assessments of how long is required to (1) chose a contractor; (2) get a large contract in place once the contractor has been chosen; (3) get a small contract in place once the contractor has been chosen; (4) to hire a new GDOT manager; (5) to fire a poorly performing GDOT manager; (6) to purchase new equipment costing more than $10,000.
14 Some of the studies using this measure are reported in B. Bozeman, Bureaucracy and Red Tape, Upper Saddle River, NJ: Prentice-Hall, 2000.
68

Table 9. Time Required for Core Tasks

Core Function To choose a contractor once a request for a bid has been made public (# weeks)
To get a large contract in place once the contractor has been chosen (# weeks)
To get a small contract in place once the contractor has been chosen (# weeks)
To hire a new GDOT manager (# weeks) To fire a poorly performing GDOT manager (# weeks)
To purchase new equipment costing more than $10,000 (# weeks)

Minimu m Maximum Mean
(weeks) (weeks) (weeks)

Std. Deviation

1.0

90.0

9.8

12.09

1.0

60.0

11.8

11.00

1.0

60.0

8.3

8.72

2.0

26.0

7.2

4.60

.0

780.0

50.3

111.64

1.0

144.0

17.7

25.51

69

As is customary for public organizations, the time required to fire poorly performing personnel is much greater than other functions. What is remarkable about the table, however, is the great range of responses received from respondents. While there may some problems in respondents' ability to provide estimates, it is quite possible that the divergence is explained in part by the fact that GDOT respondents have had very different experiences even within the same organization.
Another traditional indirect measure of red tape15 is the number of persons required to sign documents. The GDOT respondents were asked to respond to the following two items:
"Once it is agreed to award an average to larger size consulting contract, approximately how many GDOT employees need to sign off on the paperwork?"
"Once it is agreed that a contract that is already in place needs a significant modification, approximately how many GDOT employees need to sign off on the paperwork?"
Table 10 gives descriptive statistics for these two items. As we can see from the table below, the average is about six sign-offs for both the initial contract and for the modification of the contract. The range for each is considerable, with 2-20 sign-offs for the initial contract and 2-16 for the modification.
15 Pandey and Scott, 2002, opp cit. 70

Table 10. Number Sign-off for Contracts

Once it is agreed to award an average to larger size consulting contract, approximately how many GDOT employees need to sign off on the paperwork? (# of employees) Once it is agreed that a contract that is already in place needs a significant modification, approximately how many GDOT employees need to sign off on the paperwork? (# of employees)

Minimum Maximum

2.0

20.0

2.0

16.0

Mean

Std. Deviation

6.4

2.91

5.6

2.63

Drawing from the literature on red tape16 we examined a number of hypothesized impacts of red tape on respondents work attitudes and behaviors. Table 11 is a correlation table that gives results, for each sample of respondents, for relationships between red tape and selected attitudinal and behavioral variables. One implication is that different red tape constructs show different patterns of association, implying, perhaps multiple dimensions of red tape. Overall, the anchored items (number of weeks to choose a contractor and number of sign-offs required for contract to be put in place) show stronger correlation patterns than the purely perceptual ones (assessments of overall red tape and contracting red tape).

16 For a summary see B. Bozeman, Ibid. 71

Table 11. Red Tape Constructs and Correlates

Questionnaire item

Overall Red Tape

Contracting Red Tape

Wks. to Choose Contractor

Sign-offs Needed to Get Contract in Place

Be intimately familiar with GDOT n.s. rules and procedures (4) Be flexible when I suggest needed n.s. changes not covered explicitly in the contract (4) A good engineer is concerned with n.s. managing the technical elements of our projects; not managing consultants (1) There are certain tasks which GDOT n.s. must do in-house in order to maintain its core competencies (1) Hours worked during a typical work .219** week I am highly satisfied with my job (1) -
.320**** Percentage of time (per week) doing n.s. paperwork for contracts (%) Percentage of direct communications .216** with consultants (%) Key: n.s. = Not statistically significant
* = Significant at .10 level ** = Significant at .05 level *** = Significant at .01 level **** = Significant at .001 level or >

n.s. n.s.
n.s.
n.s.
.192* -.363**** n.s. .272**

.239** n.s.
-.201*
-.215*
n.s. -.267** .288** .225**

.385**** n.s.
.239**
-.237**
n.s. n.s. n.s. .244**

One striking finding is the apparent negative impact of red tape on the manager. If managers spend more time communicating with consultants and if they spend more time on paperwork for contracts, they have increased assessments of red tape. If

72

managers have higher assessments of red tape, they have significantly lower levels of job satisfaction. Comparing Views about Contracting Relations
The final research question examined, and the one most central to this analysis, pertains to views about contracting relations. Both questionnaires asked a series of questions about contracting relations between GDOT and consultant contractors. Table 12 provides the items as asked in the consultant questionnaire, along with the responses (in percentage terms).
73

Table 12. Consultant Responses for Contracting Relations Items

Contracting Relations Item
GDOT has always been evenhanded in its negotiations with my consulting firm GDOT often acts opportunistically at my firm's expense Based on past experience, my consulting firm cannot with complete confidence rely on GDOT to keep its promises My consulting firm is hesitant to work with GDOT when the specifications are vague Compared with other organizations with which my company has worked, GDOT is trustworthy My experience with GDOT managers has generally been positive.

% Strongly Agree 12.1 7.1
2.0
6.1 41.4
24.2

% Agree 51.5 26.3 2.0
6.1 41.4 52.5

% Somewhat Disagree 18.2

% Strongly Disagree 4.0

(Missing Data)17
14

35.4

17.2

14

40.4

20.2

14

36.4

19.2

14

8.1

1.0

12

9.1

1.0

12

In general, the consultants' views of GDOT contracting relations seem quite positive. A sizable majority agrees that GDOT is "evenhanded" in contracting; the vast majority feels that GDOT keeps its promises and that GDOT is trustworthy; more than three-quarters agree that their experience with GDOT managers has been positive. One interesting result is that more than one-third feel that "GDOT often acts

17 The missing data percentages are included here, first because they are larger for these items than most and, second, because consultants may be reluctant to criticize or otherwise evaluate GDOT and the extent of missing data is suggestive.
74

opportunistically" at their firm's expense. This may or may not be negative; it could be that this is a result of GDOT managers wringing as much value as possible out of contracts.
One difficulty of the present research design is that consultants interact with only "one GDOT" whereas GDOT managers interact with many consultants.18 Thus, seeking comparable data from GDOT about consultants posed a challenge. The approach taken entailed asking GDOT very similar questions about "Consultant X." Specifically, the wording was as follows:
"[W]e are interested in learning about your perceptions of working with a specific consultant. Instead of thinking about consultants in general, please think about your MOST RECENT experience with a specific consultant to answer questions...For purposes of this survey, we will refer to this most recent consultant as `Consultant X.'"19
The logic behind this item specification is (1) a better, more valid response will be received if there is a particular referent or anchor to the question and, (2) by asking about the "most recent" the total responses should, essentially, reflect a random distribution of contracting experiences (assuming that there was not unusual set of contracting-related events during the period of the study).
Table 13 provides the response frequency percentages for the GDOT survey's parallel questions about consulting relations.
18 Obviously many consultants also interact with other contract providers but that is not of interest for present purposes due to the methodological difficulty of sorting out responses for many different contact providers.
19 The respondents were asked about the number of times they have worked with "Consultant X." The average (mean and median) is three times, but the range is between 1 and 60.
75

Table 13. GDOT Staff Responses for Contracting Relations Items20

Contracting Relations Item
Consultant X has always been evenhanded in its negotiations with GDOT Consultant X often acts opportunistically at GDOT's

% Strongly Agree 43.2
8.4

% Agree 45.3
31.6

% Somewhat Disagree 9.5

% Strongly Disagree 0

35.8

24.2

Based on past experience,

17.9

35.8

42.1

GDOT cannot with complete confidence rely on

4.2

Consultant X to keep its

promises

GDOT is hesitant to work 5.3

20.0

41.1

33.7

with Consultant X when the

specifications are vague

Compared with other

28.4

46.3

20.0

5.3

consultants GDOT has

worked with,

Consultant X is trustworthy

A simple, but useful, comparison is to consider whether GDOT is more favorable about consultants with regard to a given item or consultants are more favorable about GDOT. For the "evenhanded" item, clearly GDOT has a more favorable view of consultants than vice-versa; there is little difference on the "opportunistic" item (at least when we take into account sample differences in missing data); GDOT is somewhat less likely to wish to work with the consultants (compared to consultants' willingness to work with GDOT) when specifications are vague; the consultants, according to GDOT, generally "keep promises" but according to the consultant respondents, not at as high a rate as GDOT does; similarly, GDOT scores higher on being "trustworthy."

20 It is perhaps noteworthy that there is no missing data for the GDOT response to the questions, but for the consultant version missing data ranges from 15-20%.
76

SECTION IV. CONCLUSIONS AND RECOMMENDATIONS
A distinctive feature of this study is that it allows comparisons of the views, perspective and attitudes of GDOT staff with those of consultants. If there is one overarching conclusion it is that those views or not significantly out of alignment, whether one is examining contractor relationships, red tape, or perceptions of GDOT. Moreover, the study has produced no evidence of glaring difficulties in contracting, red tape, or maintaining in-house functions. Thus, the recommendations are not ones for fundamental change but ones to consider for possible incremental change.
GDOT may wish to consider developing a strategic document that systematically examines the need for retaining in-house capabilities and functionality with respect to specific tasks and domains of knowledge. Clearly, there is a strong consensus about the need to retain capacity and tacit knowledge but there seems less agreement as to the reasons to do this and the types of knowledge and capacity that is especially worth emphasizing.
Maintaining capacity and tacit knowledge is in part a matter of personnel turnover and the reasons for turnover. In many instances, GDOT managers are working for GDOT contractors and, in a sense, this represents a storehouse of usable know-how and institutional knowledge. It would perhaps be useful for GDOT to conduct formal, systematized exit interviews for the purpose of constructing a career tracking data base. This would help GDOT better understands the reasons behind mobility and, at the same time, provide knowledge of possible sources of "on tap" knowledge. Currently, many managers retain this knowledge, but when they, too, leave it is diminished.
77

The number of sign-offs required for contracts and contract modifications may in some instances be excessive. This should be specifically addressed and, if possible, greater discretion should be available to managers.
While there is no policy or procedure available to change managerial style, the findings about managerial style are significant and worth diffusing. While there is a great deal of variance in managerial style, it seems that the most common approaches rely heavily on formal controls and personal loyalty. In some cases, it might well be useful to moderate the control orientation especially inasmuch as contracts already detail specific deliverables and obligations.
78

Appendix
Consultant Sample Development
Sampling Frame
We included project managers/contact individuals from consulting firms who have been awarded GDOT contracts. It is important to note that we only included firms who have contracted with GDOT, not those who are pre-qualified to contract with GDOT.
Sample Development
Step 1: We obtained a list of awarded contracts from GDOT. For each awarded contract, we obtained a contact individual from each consulting firm. We removed numerous duplicates, since many firms have multiple contracts with GDOT and listed the same individual as the contract manger/contact person. Also, many individuals have worked for more than one firm over time. We conducted some research to identify the most current firm for these individuals.
Step 2: We conducted missing address/telephone searches via the internet and telephone calls. We were suppose to check to see if the names we had were correct, but in all honestly I am not sure how well this was done. Based on the number of names we were able to remove this summer due to employee turnover, I am guessing it was not done, or at least not done well. Clearly this was my responsibility and I dropped the ball, I'm sorry.
Sample
The final sample included 280 managers/contact individuals from 106 different consulting firms.
GDOT Staff Sample Development
Scope
The first step was to decide whether to take a broad or a narrow approach to the sample development. The broad approach would include those GDOT employees with experience working with consultants from a variety of different areas (transportation, infrastructure, etc.), not just engineering design. The narrow approach would only include those individuals working with engineering design consulting firms. Based on the potentially small sample associated with the narrow approach, we opted for the broad approach. Therefore, we contacted all of the GDOT offices/division and each of the field offices to build the sample.
79

Clearly, this was a tradeoff. On the one hand we have a larger sample, with the potential for more respondents. But on the other hand, we had several respondents who felt the instrument wasn't very relevant to their work.
Sampling Frame The sample included:
1. GDOT "project managers" with direct experience working with consultants (not everyone has the title project manager)
2. Division and office heads from offices working with consultants
Sample Construction Step 1: Babs contacted all office heads via email to let them know about the project and that we would be contacting them via email to collect the names. Step 2: We contacted each office head via email and telephone to get a list of employees who work directly with consultants. In all, we contacted 49 division and office heads to build the sample (8 divisions and 41 offices). Step 3: We conducted a series of follow up contacts for those offices failing to respond.
Sample The final sample consisted of 208 GDOT employees from 32 different offices and divisions. The offices and divisions that reported having no interaction with consultants were not included in the final sample.
80

GDOT SURVEY ADMINISTRATION
Precontact letter: Mailed June 1, 2007
Mailing #1: Mailed June 7, 2007
Postcard Mailing: Mailed June 14, 2007
Mailing #2: Mailed and delivered June 26, 2007. Hand delivery was for GDOT employees working at 2 Capital Square. All other GDOT surveys were mailed.
Mailing #1 Issues
Consultants removed from sample:
1. #1108 and #1142 RBA (RBA is not in GA anymore) 2. #1120 Not at organization 3. Remove #1182 b/c responded to pretest. 4. HDR Engineering removed b/c not in GA: #1018, #1149, #1051, #1121 5. #1057, #1116, 6. Researchers called all consultants who had not responded. The following are no
longer working for the company: 1003, 1007, 1042, 1053, 1055, 1056, 1065, 1068, 1087, 1111, 1133, 1135, 1145, 1154, 1155, 1156 (resigned), 1173, 1185, 1186, 1199, 1215, 1219 7. Josh's calls resulted in the following removals: 1038 (repeat on the list), 1010, 1011, 1012, 1013, 1014, 1022, 1028, 1032, 1036, 1045, 1048, 1050, 1059, 1064, 1073, 1077, 1082, 1092, 1094, 1126, 1138, 1148, 1151, 1159, 168, 1189, 1190, 1193, 1196, 1202, 1213, 1222, 1224, 1233. 8. #1074 removed, #1072 name changed to new employee, #1156 name changed to new employee.
GDOT individuals removed from sample:
1. Offices of subsurface utility engineering program and railroad safety program received two surveys each. #1244, #1263, #1245, #1246. A GDOT employee contacted us by email and reported that two individuals would respond to one survey for the subsurface utility engineering program and two would fill out one survey for the railroad safety program. Reduced sample by TWO.
2. #1398 removed 3. #1247 maternity leave 4. #1361 quit, #1366 gone, #1389 gone, #1288 resigned, #1293 retired, #1304
resigned, #1297 gone, #1325 removed 5. 6. #1277 retired and changed to replacement 7. #1280 original person retired and changed to replacement 8. #1354 and #1356 are a duplicate. Removed #1354.
81

Efforts to uncover bad addresses 1. Respond to returned mail: Researchers took all mail that was returned by the USPS and followed up to try to correct the addresses. I began by looking online for the company's address. Researchers then moved on to contacting the individual by phone and email. In some cases, the returned mail was stamped "Doesn't work here" or "not at address." These individuals were removed from the sample because they no longer work for the firm or GDOT. Mail that was stamped "Unable to forward" were readdressed, stamped, and resent to the new forwarding address (e.g. PBS&J). 2. Call all consultants who were nonrespondents: Researchers called each individual or the consulting firm to confirm if the individual currently works at the firm. They then confirmed the mailing address and asked the respondent to please respond. 3. Hand deliver GDOT surveys to 2 Capital Square: Researchers hand delivered the WAVE II mailings to the main GDOT office. At that time, with the help of GDOT staff, we were able to identify individuals who had retired, were on leave, or no longer worked at GDOT. 4. Call GDOT employees at all other facilities: Researchers called the GDOT nonresponents who do not work at the main GDOT office. From those calls a few individuals were removed who are no longer working at GDOT. #1328 and #1330 not working at GDOT. #1331 not appropriate for sample. #1268 removed, #1345 removed.
82

SECTION III: KNOWLEDGE OF PRIME AND SUBCONTRACTOR
RELATIONS
Dr. Christopher M. Weible Jeffrey C. Jones
Georgia Institution of Technology School of Public Policy
83

Introduction This report is one of three parts of a larger study. The two other parts focused on
tacit knowledge among Georgia Department of Transportation's (GDOT's) project managers and contractors (Bozeman), and generating work level estimates from consultant cost proposals (Eckert). These two parts were based on methodologies with long histories of proven techniques and applications in transportation research. This third report involves a methodology new to contract management: the analysis of the relations among prime and sub consultants using social network analysis techniques.
Social network analysis is a relatively new technique used for analyzing connections among objects. In this report, for example, social network analysis is used to understand the connections based on contracts among prime and sub consulting firms. In contrast, traditional research techniques usually focus on an object's internal characteristics, such as the number of employees in a consulting firm. Social network analysis can provide GDOT (i) broad visualizations depicting the relations among the entire universe of prime and sub consultants, (ii) specific visualizations showing how single firms have contracted with other firms over time, and (iii) the names of firms with the most or the least contractual relations. A major objective of this report is to show the potential of social network analysis in helping GDOT officials manage its portfolio of contracts and communicate this portfolio to people internal and external to GDOT.
This report focuses on the relations among prime and sub consultants involved in GDOT's pre-engineering contracts from 1995 through 2007. As GDOT increased the amount of work outsourced to consultants, it also increased the number of prime
84

consultants and sub consultant firms under contract. The study reports, from both macro and micro perspectives, on the relations among GDOT consultants. The objectives of this report are five-fold:
1) To understand the growth in pre-engineering design contracts over time among prime and sub consultants;
2) To provide a macro perspective of the relationships among selected prime and sub contractors, along with changes over time;
3) To provide a micro perspective of the relationships among selected prime and sub contractors, along with changes over time;
4) To identify firms most central in GDOT's pre-engineering design contract system, along with the changes in centrality over time;
5) To investigate the network relations of Disadvantaged Business Enterprises (DBEs) over time.
This report finds that GDOT is not dealing with an atomized or independent collection of contracts marked by a single prime teamed with a group of sub consultants. Instead, GDOT faces a system of interdependent contracts, where primes share the same sub, or primes act simultaneously as sub consultants. Indeed, one of the difficulties in managing a large number of contracts is keeping track of the flow of projects through many of the same prime consultants, sub consultants, or both. If GDOT starts five new projects in a year, and the same sub contractor is providing a key service on all of the contracts, a bottleneck is likely to develop in the flow of work from the dependence of the five contracts on the same sub contractor. A major thrust of this report focuses on
85

visualizing this interdependent system of prime and sub relations, and how these relations have changed over time.
This report is based on a combined dataset built from two previous, smaller datasets: the Consultant Management System (CMS) and the Consultant Management Information system (CMIS). The combined dataset provides the relations among prime and sub consultants, DBEs, and their contract amounts from 1992 through 2006. Beginning with a description of the datasets and methodology, the report contains five analysis sections. The conclusion offers recommendations for managing a system of contracts and future research directions. Datasets and Methodology
The dataset used for this study is derived from two GDOT sources: the Consultant Management System (CMS) and the Consultant Management Information System (CMIS). Data in the CMS primarily captured information on contracts initiated between 1992 and 2003. The CMIS data includes contracts initiated between 2003 and 2006 as well as contracts initiated in earlier years, but still active between 2003 and 2006. The CMS and CMIS datasets were merged to help interpret changes in relations among prime consultants and sub consultants over time and to fill many of the missing values in both datasets. The report does not analyze data from 1992, 1993, or 1994 because so few contracts or firms were active in those early years.
To merge the CMS and CMIS datasets, a number of steps were taken. The first step was to sort both the prime and sub contractors by name to ensure that consistent names were being used across time. For example, the CMS data referred to a particular firm as PBQD, Inc, while the CMIS dataset referred to this firm as Parsons Brinckerhoff
86

Quade & Douglas, Inc. When firm names were inconsistent, a find/replace edit was done on the dataset to update the older CMS name with the newer name used in the CMIS data. Thus, in the case above, the firm name was changed to Parsons Brinckerhoff Quade & Douglas, Inc.
The next step was sorting the data by project identification (PI) number to identify contracts that were common in both the CMS and CMIS datasets. If, in the two datasets, the original contract amounts were only slightly different, the CMIS value was used. For example, PI# 110620 matched on prime contractor name, but differed on the original contract amount. The CMS data reported the original contract amount at $970,564, while the CMIS data reported the contract amount at $970,411. In this instance, the CMIS value of $970,411 was used in the combined dataset. When contract dollars differed by a large amount, the contract was not included in the analysis, and flagged for later follow up. The two datasets were also used to fill in missing data in each. For instance, if a contract in the CMS dataset included all of the sub consultant information but was missing a value for contract year, this data could be found in the CMIS dataset.
Once the data was sorted by firm name, firms that were listed DBEs for any one contract were updated to include their DBE status on all contracts. Additionally, DBE lists were collected online for 2004 and 2008.21 These lists were then added to the full database. Firms identified as DBEs in either the 2004 or 2008 lists were listed as DBEs on all contracts within the dataset.
21 Current GDOT DBE list can be found at: http://tomcat2.dot.state.ga.us/ContractsAdministration/uploads/rptDBE_Directory_CA_New.pdf. The 2004 list can be found at: http://tomcat2.dot.state.ga.us/ContractsAdministration/uploads/Sept%202004_DBE_Directory.pdf.
87

Over the past decade, the method of contract identification has changed. Currently, all contracts are given an alphanumeric number. One alphanumeric number can possibly be associated with several project identification (PI) numbers. Prior to 2004, however, contracts were primarily identified by PI numbers. The researchers attempted to correct this problem by combining some of the contracts identified by PI numbers with a new alphanumeric identification number. However, the current combined dataset does not reflect this attempt at reclassification. Nonetheless, almost all of the results presented in this report are robust regardless of the classification scheme because the report focuses on the relations between primes and subs, which would remain the same if classified by PI numbers or by alphanumeric numbers. Only Table 3, which shows the number of contracts over time, will change if contracts with a PI number were combined under a new alphanumeric number.
Two of the data fields with many missing values were start dates and end dates for contracts. In June of 2008, GDOT personnel shared a Consultant Contract Database that included more complete start and end date information to be added to the master dataset. Dates were reformatted to reflect only the year, and the data was then merged with the master dataset, further increasing data quality.
Figure 1 includes a screenshot of the final combined dataset. The four highlighted lines represent one contract (e.g., PI# 262165). For this project, American Engineers, Inc. is the prime consultant. There are three sub consultants: (1) Heath & Lineback Engineers, Inc.; (2) Photo Science; and (3) Transportation Systems Design, Inc. Column J shows the original, full contract amount, while Column K shows the amount that went to each sub consultant as well as the remaining amount that the prime was left with after
88

paying all of the sub consultants. In this example, the full contract amount was $606,661. Heath & Lineback Engineers, Inc., Photo Science, and Transportation Systems Design, Inc. respectively received $76,544, $31,352, and $108,742 as sub consultants. The prime, American Engineers, Inc., was left with $390,023 after paying the subs. Additionally, this contract started in 1997 and has one DBE (Transportation Systems Design, Inc.). There is also missing end date information for this contract.
Because each distinct row in the dataset includes information about a contractual relationship between a prime consultant and sub consultant, it could also be said that each individual line (or record) in the database represents a connection between two firms. In the above example, the database included information pointing out a contractual relationship between American Engineers, Inc. with its three sub contractors and well as a relationship with itself.
89

Figure 1. Screen Shot of Combined Dataset
90

One way to assess the improvements and benefits of a combined CMS/CMIS dataset is to compare the amount of data discarded because of missing values before and after the data were combined. Using the sample data included in Figure 1, a record (or row) would be rejected for missing data if it did not include a PI number, prime and sub consultant names, or a start date. Projects where information was incomplete for full or sub contract amounts were left in the dataset because some of the analysis in this report did not require this information.
Table 1 shows the number of records (lines in the Excel file as in the screen shot in Figure 1) between sub and primes in each of the datasets. For example, in CMS, there were 914 total records of which 774 were complete and 140 were incomplete. The incomplete records were discarded, leaving 85% of the total number of records for analysis. As Table 1 shows, the combined dataset discards 19% of the data because of missing or conflicting data, a slight improvement over the CMIS data (25% discarded). Table 1 also highlights the increased depth of the data found in the combined dataset, as the final combined dataset contains 1,983 records while the CMIS contains only 1,255.

Table 1: Data Quality and Rejected Data for CMS, CMIS, and combined

data sets.

Good

Rejected

Total

% Kept

Records

Records

Records

CMS

774

140

914

85%

CMIS

1255

410

1665

75%

Combined

1983

465

2448

81%

See Appendix A for a discussion of the reasons for rejected records for the three datasets.

The datasets were analyzed using UCINET, a software package designed for social network analysis.22 Published by Analytic Technologies, UCINET is comprehensive toolkit for analyzing network relations. It also comes with NETDRAW, a program for visualizing network relations in diagrams. UCINET is used because Microsoft Office programs, such as Excel or Access, do not have the capacity for analyzing or visualizing networks. Changes in the Number of Primes and Subs over Time
How has the number of prime and sub consultants changed over time? To answer this question, Table 2 compares the original CMS and CMIS datasets with the final combined dataset for the number of consultants, number of prime and sub consultants, and dollar amounts for each year.
22 UCINET can be found at http://www.analytictech.com. 92

Table 2. Number of Prime/Sub Consultants and Contract Totals Per

Year

Money

Money

Year Total Primes Subs

(real) (2002 dollars)

1995 18

7 11 6,349,106

7,494,778

1996 21

9 13 8,218,110

9,422,804

CMS Data

1997 27

16 11 9,367,779

10,500,083

1998 29

10 20 9,396,688

10,370,945

1999 41

13 31 14,841,116

16,025,910

2000 35

11 26 13,331,153

13,927,261

2001 87

25 74 82,434,833

83,738,150

2002 24

5 24 16,622,526

16,622,526

1999 12

3 8 12,652,022

13,662,057

2000 19

4 12 19,205,128

20,063,893

CMIS Data

2001 52

7 37 64,067,669

65,080,596

2002

8

2 4 8,693,511

8,693,511

2003 40

4 33 34,793,218

34,017,934

2004 18

3 14 18,797,007

17,901,437

2005 36

8 25 33,196,723

30,579,060

2006 128

43 110 328,609,501

293,238,339

2007 64

15 52 131,789,070

114,346,604

1995 22

12 11 8,378,253

9,890,077

1996 24

13 13 9,369,415

10,742,879

1997 28

17 11 11,010,640

12,341,520

1998 28

14 15 16,559,790

18,276,725

Combined Data

1999 48

14 40 17,655,570

19,065,048

2000 48

19 36 19,718,344

20,600,059

2001 91

30 78 112,409,718

114,186,947

2002 26

9 20 18,850,116

18,850,116

2003 46

14 36 70,543,326

68,971,437

2004 22

5 18 13,320,733

12,686,077

2005 76

27 63 75,975,568

69,984,663

2006 137

50 127 206,402,859

184,185,885

2007 82

25 72 70,909,698

61,524,702

Note: Money (2002 dollars) has been adjusted for inflation by using the

CPI calculator available via the U.S. Department of Labor, Bureau of

Labor Statistics.

93

Looking at the combined data, the increase in the number of prime and sub

consultants over time is evident. The total number of contractors increased 273%, from

22 in 1995 to 82 in 2007. The number of primes increased 108% from 12 in 1995 to 25

in 2007. Showing the most growth, sub contractors increased from 11 in 1995 to 72 in

2007 (a 555% increase). Similar increases can also be found for contract amounts. In all,

GDOT awarded about $650 million in contracts from 1995 through 2007. While there is

a general increase in the number of contractors over time, there are also fluctuations with

2001 and 2006 standing out as spikes in the number of consultants and contract dollars.

Table 3 shows the total number of contracts found in the combined dataset. In all,

there were 453 contracts awarded from 1995-2007 within the dataset. In general, there

was a steady increase in the number of contracts awarded over time, with 19 contracts in

1995 and 43 contracts in 2007. As noted in the datasets and methodology section, it is

possible that the number of contracts prior to 2004 was inflated because of a different

method of contract classification.

Table 3. Number of Contracts

Per Year in Combined Dataset

Year

Total Contracts

1995

19

1996

15

1997

21

1998

23

1999

22

2000

34

2001

75

2002

18

2003

29

2004

8

2005

55

2006

91

2007

43

Total

453

94

There are some anomalies within the dataset that should be discussed before moving forward with the analysis. In some instances, such as 1996 in the CMS data, the total number of contractors was not equal to the total numbers of subs and primes over time. Within these years, there was at least one firm, and possibly many firms, that acted as both a sub consultant and prime consultant on different projects.
Two years, 2001 and 2006, especially stand out in the data. In these two years, the contract networks not only grew in terms of consultants, they also grew greatly in terms of contract dollars awarded. The spikes in contracts can be explained by the influx of state and federal projects. In 2001, the Governor's Road Improvement Project (GRIP) project led to an influx of state and federal dollars for road construction in Georgia. GRIP expanded the total awarded contract amount in 2001 to $112.4 million, a 470% increase from the previous year. A similar pattern can be seen in 2006. In that year, the state began spending a large amount of pre-engineering dollars on Preliminary Engineering Projects; the $206 million in contracts in 2006 was a 171% increase from the previous year.
Of note, the trend of increasing sub consultants relative to primes is highlighted in Table 4, where the ratio of primes and subs is calculated from the combined data in Table 2. In early years, such as 1995 and 1996, the number of primes and subs was almost equal (ratio = 1.09). In 1997, there were more prime consultants than sub consultants in the contracting network. Beginning in 1998, this trend reversed, as more and more subs entered the network. By 1999, there were consistently twice as many sub consultants than prime consultants in any given year. This trend was most evident in 2004, a year in which there were 18 sub consultants and only five prime consultants. By 2007, there
95

were approximately three subs for every prime (ratio = 0.35). This trend suggests a growing reliance on sub consultants over time and the increasingly important role that subs play in GDOT's contracting network. It also shows that primes are playing a more central role in latter years compared to earlier years.

Table 4: Ratio of Primes / Subs

Ratio of

Year

Primes/Subs

1995

1.09

1996

1.00

1997

1.55

1998

0.93

1999

0.35

2000

0.53

2001

0.38

2002

0.45

2003

0.39

2004

0.28

2005

0.43

2006

0.39

2007

0.35

Macroviews of Relations among Prime and Sub Contractors

How has the relation among prime and sub contractors changed over time? A

visual map, created with NETDRAW software, displays yearly trends of prime and sub

consultants; there are 13 maps describing the data. In these maps, prime contractors are

represented as circles, with relationships flowing out as lines from prime contractors to

the sub contractors, who are represented by squares. A line between a circle and a

square, or a prime and a sub, represent a contractual interaction or relation between those

firms within a given year. For example, in 1995, HDR Engineering, Inc. served as a

96

prime contractor who then subcontracted with R.K. Shah & Associates and Rochester & Associates.
The 1995 data further demonstrates sub consultants playing key roles in the contracting networks, as well as holding a large number of connections to primes. For example, Transportation Systems Design displayed a pattern of high connectedness; they were linked with five primes. A more in-depth analysis of firm connectedness will follow, but it is important to notice that some firms were more connected than others, according to the available data.
In contrast to firms that are highly connected, sometimes firms are sparsely connected. In 1997 for example, Browder, LeGuizamon & Associates and HDR Engineering were only connected to each other. This relationship represents a case of one prime (HDR Engineering) using only one sub consultant (Browder, LeGuizamon & Associates).
In 2001, 2005, and 2006 the firm names were removed from the maps because the network size and complexity within these years made it difficult to read. The maps with firm names can be easily reproduced, but they are not as aesthetically pleasing. In these three years, there were several subs on the periphery that were only connected to one prime. In other instances, sub consultants play a key role in connecting prime contractors through their relationships with well-connected sub consultants.
The maps also show that primes and their teams do not operate in isolation. Instead, primes and subs are usually interconnected. One implication of this data is a broadening of contract management from individual, atomized contract administration to a system of interdependent contracts.
97

Figure 2. GDOT Prime and Sub Consultant Network 1995 (primes circles, subs squares)
98

Figure 3. GDOT Prime and Sub Consultant Network 1996 (primes circles, subs squares)
99

Figure 4. GDOT Prime and Sub Consultant Network 1997 (primes circles, subs squares)
100

Figure 5. GDOT Prime and Sub Consultant Network 1998 (primes circles, subs squares)
101

Figure 6. GDOT Prime and Sub Consultant Network 1999 (primes circles, subs squares)
102

Figure 7. GDOT Prime and Sub Consultant Network 2000 (primes circles, subs squares)
103

Figure 8. GDOT Prime and Sub Consultant Network 2001 (primes circles, subs squares)
104

Figure 9. GDOT Prime and Sub Consultant Network 2002 (primes circles, subs squares)
105

Figure 10. GDOT Prime and Sub Consultant Network 2003 (primes circles, subs squares)
106

Figure 11. GDOT Prime and Sub Consultant Network 2004 (primes circles, subs squares)
107

Figure 12. GDOT Prime and Sub Consultant Network 2005 (primes circles, subs squares)
108

Figure 13. GDOT Prime and Sub Consultant Network 2006 (primes circles, subs squares)
109

Figure 14. GDOT Prime and Sub Consultant Network 2007 (primes circles, subs squares)
110

Microviews of Relations among Prime and Sub Contractors How have the relations for specific firms changed over time? The previous
section took a macroview by looking at the entire universe of prime and sub consultants. This section takes a microview by zooming in on a single consulting firm. A microview gives GDOT insight into how a single firm has worked with other firms over time. This information, in turn, can help negotiate future contracts or manage current contracts. With 245 firms in the dataset, this analysis focuses only on a sample of firms that may be of interest to GDOT.
Exemplary firms in total and real contract dollars, in contract dollars as a subconsultant, and in total and real contract dollars as a DBE were chosen for the microanalysis. Our goal is to show proof of concept and demonstrate the value of a microview of the relations among primes and subs. This type of analysis could be run on any subgroup of firms that are of interest, such as all DBE firms. In addition, this analysis could be run for all 245 firms in the network, although many of these firms participate rather sparsely over time.
Prime consultants that were awarded large sums of money in the pre-engineering design contracts are analyzed first. Arcadis U.S., Inc., awarded $106,301,406 over the 12 years, was included. PBS&J was also included in the analysis because it was awarded $37,289,092 over 12 years. Sub-consultants that earned the most contract dollars can also be analyzed, but because so many served as primes, analysis can be difficult. United Consulting, which earned $15,959,400 in just subcontracts, was well in front of the rest. Next, among DBE firms, Street Smarts / Transportation Consulting MDA, Inc. earned the largest amount of money and was included in the most contracts over time. Street
111

Smarts was awarded $15,495,988 in real contract dollars and $14,707,262 in primeconsultant dollars (before having to pay their sub-consultants).
112

Firm Level Analysis: Arcadis U.S. Inc

Below, Table 5 and Figure 15 show a firm-level analysis of Arcadis U.S., Inc.

over time. Table 5 displays the number of relations (i.e., the number of other primes or

subs Arcadis worked with in a given year) and the contract dollar amount. In both 1997

and 1998, the firm had no relations, despite being awarded nearly $2.5 million in

contracts; however, in other years, Arcadis had as many as 42 relations. This is because

while the firm did receive contract dollars in those years, they did not use sub

consultants. In other words, all of the money was kept by Arcadis. In other years,

Arcadis had many relations. For example, in 2002, the firm had 28 relations and, in

2006, it had 42 relations.

Table 5. Number of Relations and Contract Dollars for

Arcadis U.S., Inc. by Year

Year

Number of Relations Contract Dollars

1995

0

0

1996

2

$827,896

1997

0

$584,160

1998

0

$1,858,389

1999

0

0

2000

0

0

2001

4

$1,917,141

2002

28

$10,789,760

2003

0

0

2004

0

0

2005

11

$1,414,598

2006

42

$1,858,467

2007

5

$705,146

It is interesting to note that the number of relations does not seem to be related to

the contract dollar amounts. For example, in both 1998 and 2006, Arcadis earned $1.8

million in contracts, yet connections varied considerably in those years (with zero

relations in 1998, and 42 in 2006).

113

The patterns in Table 5 can also be seen in the individual maps for Arcadis for each year they were included in the consultant network (see Figure 15 on the next two pages). For example, the large spike in connections found in 2006 is easily confirmed by looking at the network map for that year. The network maps also show that Arcadis largely served as a prime consultant, but also served a sub consultant in 2002, 2005, and 2007. This facet is shown in these years by the inclusion of Arcadis as a square (or sub consultant within these maps). Figure 15. Firm Level Network for Arcadis U.S., Inc. (primes circles, subs squares)
1996
2001
114

2002
115

Figure 15(continued). Firm Level Network for Arcadis U.S., Inc. (primes circles, subs squares)
2005 2006
2007
116

Firm Level Analysis: PBS&J U.S. Inc.

The number of PBS&J's relations and contract dollars for each year are displayed

in Table 6. Figure 16 presents the network maps of PBS&J for the years it worked with

other consultants (on the next four pages). PBS&J has been awarded nearly $37 million

in contracts, with larger contract amounts in more recent years. The number of relations

tended to increase over time. PBS&J usually served as a prime (circles in Figure 16) in

the early years and then as both a prime and a sub in the subsequent years. The increase

in the number of relations in Table 6 shows PBS&J's increasing number of subs on the

contract teams.

Table 6. Number of Relations and Contract Dollars for

PBS&J U.S., Inc. by Year

Year Number of Relations

Contract Dollars

1995

7

$768,095

1996

3

$278,562

1997

2

$467,504

1998

0

0

1999

6

$3,804,548

2000

5

0

2001

18

$4,768,407

2002

0

0

2003

13

$5,084,488

2004

0

0

2005

30

$11,552,742

2006

28

$5,680,983

2007

5

$4,498,744

117

Figure 16. Firm Level Network for PBS&J (primes circles, subs squares) 1995
1996
1997
118

Figure 16 (continued). Firm Level Network for PBS&J (primes circles, subs squares)
1999
2000
2001
119

Figure 16 (continued). Firm Level Network for PBS&J (primes circles, subs squares)
2003
2005
120

Figure 16 (continued). Firm Level Network for PBS&J (primes circles, subs squares)
2006
2007
121

Firm Level Analysis: United Consulting Table 7 and Figure 17 (next two pages) describe United Consulting, one of the
subs with the most relations. Like most firms, United Consulting increased their contract dollars and the number of relations over time. Altogether, they earned approximately $16 million from 1999 through 2007. The combined results show how one firm a sub consulting firm in this instance might serve as a bottleneck in the execution of multiple contracts. In 2006, for example, United Consulting was involved with 20 different primes on 20 different contracts).

Table 7: Number of Relations and Contract Dollars for

United Consulting by Year

Year

Number of Relations Contract Dollars

1995

0

0

1996

0

0

1997

0

0

1998

0

0

1999

2

$149,933

2000

0

0

2001

11

$2,309,824

2002

2

$23,050

2003

0

0

2004

1

$1,237,175

2005

6

$784,811

2006

20

$7,100,534

2007

4

$4,354,073

122

Figure 17. Firm Level Network for United Consulting (primes circles, subs squares) 1999 2001
2002 2004 2005
123

Figure 17 (continued). Firm Level Network for United Consulting (primes circles, subs squares)
2006
2007
124

Firm Level Analysis: Street Smarts Among DBE firms, Street Smarts earned the most money and was included in the
most contracts over time. Table 8 and Figure 18 (next 3 pages) present their information. Street Smarts brought in $15,495,988 in real contract dollars and $14,707,262 in prime consultant dollars (before having to pay their sub consultants).
Street Smarts contracted mostly as a sub consultant, particularly in the early years. But, in the more recent years, the firm has been both a prime consultant and a sub consultant. As a sub consultant, Street Smarts worked with 16 different primes in 2006 and 8 different primes in 2007.

Table 8. Number of Relations and Contract Dollars for

Street Smarts by Year

Year

Number of Relations Contract Dollars

1995

0

0

1996

1

0

1997

0

0

1998

0

0

1999

1

$26,337

2000

4

$145,508

2001

19

$2,805,959

2002

1

$383,758

2003

0

0

2004

1

$883,479

2005

13

$4,762,335

2006

23

$4,212,913

2007

16

$2,275,699

125

Figure 18. Firm Level Network for Street Smarts (primes circles, subs squares) 1996 1999 2000
2001
126

Figure 18 (continued). Firm Level Network for Street Smarts (primes circles, subs squares)
2002 2004 2005
127

Figure 18 (continued). Firm Level Network for Street Smarts (primes circles, subs squares)
2006
2007
128

Centrality Analysis The results thus far have shown relations among sub and prime consultants
forming a system of interdependent firms operating under a number of contracts. One way to analyze a system of contracts and the firms within that system is through centrality analysis. This type of analysis focuses on the firms that are involved in the most relations in a given year. Degree is the term used to describe the total number of other firms that a particular firm is connected to within a given year. For example, Street Smarts, in Figure 18 and in Table 8, had a degree of 16 in 2007; that is, they were connected with seven other subs and eight other primes. (While only connected to 15 other consultants in 2007 on Figure 18, their degree score is 16 because they worked with QORE Property Sciences on two distinct contracts.)
From a managerial perspective, it is useful to know which firms have the most relations because firms with the most relations are also most likely in a position to impede the execution of contracts. Alternatively, if GDOT is concerned with how innovations and information spreads through their contracting networks, they could use centrality as a measure of possible information exposure with the assumption that firms with more relations are also more likely to receive timely information within a network. The centrality of a firm can also be used as a measure of power. For instance, central firms (that is, firms with higher degree) can possibly influence other firms linked to them. If GDOT is concerned with the competitiveness of their contracting network, degree scores over time could measure firm dominance and power. In an ideal competitive environment, no firms would emerge as dominantly central over time.
129

There are a number of ways in which GDOT can utilize centrality measures to examine their contracting networks; the simplest is to create lists of firms based on their degree. Table 8 shows the twenty most central firms by tabulating the total number of degrees across all years. Five firms emerge as most central: PBS&J, Arcadis, PBQD, J.B. Trimble, and Edwards-Pittman Environmental.
Table 8, however, does not take into account that some firms might have had a year or two with many relations, but other years where they had no relations and were not central. To solve this problem, the average number of relations (degree) can be calculated for each year that they had a contract. The average degree is a good measure of the consistency of a firm's involvement in the contracting network over time. The average degree is shown in Table 9. PBQD and J.B. Trimble remained in the top five, but Carter & Burgess, Earth Tech, and Columbia Engineering emerged as the most central actors. PBS&J are no longer in the top position because the majority of their connections occurred within a particular year. Looking back at the firm level analysis of PBS&J, this assumption is confirmed, as PBS&J had very high degree scores in 2005 and 2006, and more modest degree scores in other years.

Table 9. Most Central Firms (by degree or total

number of relations)

Firm

Degree

PBS&J, Inc.

117

Arcadis U.S., Inc.

92

Parsons Brinckerhoff Quade & Douglas, Inc.

90

J. B. Trimble, Inc.

83

Edwards-Pitman Environmental, Inc.

82

Earth Tech

81

Street Smarts /Transportation Consulting mda,

Inc.

79

Wilbur Smith Associates, Inc.

72

QK4

67

130

Heath & Lineback Engineers, Inc.

66

Kimley-Horn and Associates, Inc.

61

Moreland Altobelli Associates, Inc.

60

Parsons Transportation Group, Inc.

60

Transportation Systems Design, Inc.

60

Willmer Engineering, Inc.

58

Greenhorne & O'Mara, Inc.

57

Jordan, Jones & Goulding, Inc.

48

United Consulting

46

Hoffman & Company / now / PhotoScience

44

URS Corporation

44

Table 10. Most Central Firms (by average degree or

number of relations per year)

Firm

Degree

Carter & Burgess, Inc.

26

Earth Tech

16.2

Parsons Brinckerhoff Quade & Douglas, Inc.

15

Columbia Engineering

14

J. B. Trimble, Inc.

11.9

PBS&J, Inc.

11.7

Shaw Environment & Infrastructure Inc.

11

Arcadis U.S., Inc.

10.2

Kimley-Horn and Associates, Inc.

10.2

QK4

9.6

Edwards-Pitman Environmental, Inc.

9.1

DMJM Harris, Inc. / dba / CTE Engineering

9

Ecological Solutions

9

Long Engineering, Inc.

9

Professional Engineering Consultants, Inc.

9

Washington Group International, Inc.

9

Street Smarts /Transportation Consulting mda,

Inc.

8.8

Moreland Altobelli Associates, Inc.

8.6

Parsons Transportation Group, Inc.

8.6

Georgia Transportation Partners

8

A DBE Analysis What is the role of DBE firms in GDOT's system of contracts? Are DBEs
earning more money than non-DBE firms? Are DBEs more central within the system of

131

relations compared to non-DBE firms? These questions are analyzed by comparing the

number of relations and dollars of DBE and non-DBE firms.

Table 10 compares the average degree (average number of relations), average

dollars per firm, and total dollars in each year for both DBE and Non-DBE firms. DBEs

were awarded $104 million in real contract dollars (including sub contract amounts and

prime contract amounts after paying subs) during the 13-year sample period. Comparing

the $104 million awarded to DBEs with the $655 million in total contracts (see Table 1),

DBEs are earning approximately 16% of all available dollars. This exceeds the 10%

federal and state requirements. Once the total number of DBEs is taken into account, 61

DBE firms among 245 total firms operated in the network over time. In other words,

25% of firms are qualified as DBEs, and these firms earn 16% of the contract dollars.

Table 11: Average Contract Dollars and Connections for DBE and Non-DBE Firms by Year

NON-DBE

DBE

Average

Average

Average Dollars per Total Dollars Average Dollars per Total Dollars

Year

Relations Firm

for Year

Relations Firm

for Year

1995

1.76 $449,074

$7,634,251

2.40 $148,800 $744,002

1996

1.79 $428,860

$8,148,339

2.80 $244,215 $1,221,076

1997

1.65 $404,310

$9,299,122

3.20 $342,304 $1,711,518

1998

1.86 $682,835

$14,339,537

1.86 $317,179 $2,220,253

1999

3.41 $415,276

$15,365,202

2.36 $208,215 $2,290,368

2000

3.47 $458,123

$16,492,413

3.25 $268,828 $3,225,931

2001

7.58 $1,464,365 $98,112,464

4.72 $571,890 $14,297,254

2002

3.16 $891,956

$16,947,166

2.57 $271,850 $1,902,950

2003

2.17 $2,327,110 $67,486,195

1.24 $179,831 $3,057,131

2004

2.44 $655,644

$10,490,301

1.00 $404,347 $2,830,432

2005

3.80 $1,336,104 $68,141,289

2.88 $313,371 $7,834,279

2006

8.03 $1,589,818 $157,391,941

7.09 $1,139,789 $49,010,918

2007

3.60 $1,061,790 $55,213,096

3.08 $565,333 $14,133,326

TOTAL

3.44 $935,789

$545,061,316

2.96 $382,766 $104,479,438

When comparing the average degree, or relations, for DBE and Non-DBE firms,

the Non-DBE firms are slightly better connected than DBE firms overall. However, in

132

some years, such as 1995-1997, DBE firms were better connected than their non-DBE counterparts. Figures 19 and 20 incorporate DBE status into the network maps. In Figure 18, the network map for 2000 is shown again, but the firm size has now changed to indicate DBE status. The large circles and squares represent firms that are DBEs, while the smaller circles and squares represent firms that are not DBEs.
In the next map, node color is used to represent DBE status, where solid colored nodes are DBEs and open nodes are non-DBEs. In this example, node size is dependent upon the contract dollars that each firm earned for 2000, with the larger node size reflecting larger contract amounts. While many DBE firms such as Edwards-Pittman and Street Smarts were fairly well connected, their intake of contract dollars in this particular year is dwarfed by non-DBE firms, regardless of their connectedness. While there is no absolute standard on what the best role of DBEs in GDOT's network of pre-engineering contracts, the results from Table 10 and Figures 19 and 20 suggest that DBEs were not on the periphery of the network, but play just as central of a role as non-DBEs. DBEs were awarded well over the 10% federal minimum.
133

Figure 19. 2000 Network Map with Large Circles and Squares Indicating DBE Status
134

Figure 20. 2000 Map with Contract Dollars Shown by Size, and DBE Status in Black
135

Conclusions and Recommendations This report focuses upon the relations among primes and sub consultants. Major
findings include the following: The number of prime consultants and sub consultants has increased over time, with the number of subs increasing in greater proportion compared to the number of primes. As the contract dollars and number of contracts have increased over time, so has the complexity of the relations among prime and sub-consultants. Prime and sub consultants were increasingly enmeshed in an interdependent system of contracts. As the number of sub consultants has increased over time, they often link two or more different primes together. The most central firms (firms with the most relations) have been PBS&J, Arcadis, PBQD, J.B. Trimble, Edwards-Pittman Environmental, Carter & Burgess, Earth Tech, and Columbia Engineering. DBE firms were just as central in their relations and almost as equal in their contract dollars as non-DBE firms. DBE firms were not on the periphery of the network of relations.
Along with these findings, one of the deliverables from this project is a dataset that combines the CMS and CMIS data, providing more complete information about GDOT's pre-engineering design contracts from 1992 through 2007. This combined dataset improves upon both individual datasets with more complete and accurate records.
136

The principle recommendation from this report is for GDOT to expand the current method of contract management to include a systems-based approach. Whereas conventional approaches to contract management often focus on a single contract as well as a single prime, a system-based approach would encompass the entire portfolio of contracts, including all the primes along with all of their subs. A system-based approach to contract management builds from the findings in this report, especially the macroviews in Figures 2-14, that GDOT's contracts are interrelated. Contracts become interrelated when multiple primes share the same subs or when primes simultaneously act as subs. A major focus of this report was showing that techniques exist for visualizing the relations among a large number of prime and subs.
A system-based approach to contract management offers several benefits to GDOT. First, it would help GDOT identify bottlenecks in contract execution, such as when one sub slows the execution of multiple contracts. For example, this report found United Consulting has worked as a sub across multiple contracts at the same time. Second, it would help GDOT identify highly centralized primes. A case in point would be Arcadis. The removal of Arcadis might put at risk previously productive relations and destabilize the market. Third, a system-based approach to contract management would help GDOT communicate its contract portfolio to consultants or political officials. For example, the microview of single firms can be used to be better informed in contract negotiations. Similarly, the macroviews can be used to help illustrate to political officials the growing complexity of GDOT's portfolio of contracts over time.
To adopt a system-based approach to contract management, GDOT should consider the following steps:
137

(i) Continue to record contract information, including identification numbers, primes and subs, contract amounts, start dates, and end dates. This is the basic information needed to conduct the analyses in this report.
(ii) Develop the capacity to translate contract information into (i) a macroview of all contracts and/or (ii) microviews of individual firms. This requires the purchase of network analysis software, such as UCINET ($150) and an interface with current databases. A regular input into UCINET is Excel spreadsheets.
(iii) Develop a monitoring system of prime and sub consultants. Once the macro and microviews of contracts are generated and disseminated, GDOT can regularly incorporate this information into the management of single contracts or in largescale programmatic processes.
Along with managerial recommendations, several research strategies emerge from this report that can help GDOT strengthen the management of pre-engineering contracts:
Develop an Evaluation of Contracts Linked to Contract Teams and Network Position. This report does not show which relational configurations are better than others. Evaluating the performance of contracts, and then combining the evaluations with network structures (teams of consultants or position within the contracting system) would help GDOT adapt to an increasingly complex network of relations among prime and subs.
Analyze Informal Consultant Relations. This report investigates the formal relations based on contracts among prime and sub consultants. Prime and sub consultants, however, relate in many ways outside of formal contracts. In the future, research should consider investigating the informal social networks of
138

GDOT's consultants. This might include their information sources or who they seek assistance from in winning and executing contracts. Such an analysis would give GDOT a glimpse into how their consultants overcome their obstacles, thereby suggesting productive ways to provide assistance. Analyze Unawarded Bids. This report analyzes contracts that have been awarded. An alternate approach would be to investigate teams of primes and subs that were not awarded a contract. This would provide insight into the extent that the contracting process is competitive and, thereby, help frame discussions with contractors and political officials. It would also provide a more complete understanding of GDOT's contracting universe.
139

Appendix A: Data Quality Assessment

Table 1 presents distribution statistics for good records and rejected records for

the combined dataset. The largest reason for rejecting data is for missing year

information. In 333 records representing 78 distinct contracts, both the CMS and CMIS

data was unable to provide any information for contract start date.

The remainder of the rejected records was removed from the final dataset because

of conflicts between the CMS and CMIS data that could not be resolved. The most

common of these types of conflicts is different start dates between CMS and CMIS data.

In this instance, 95 records representing 25 distinct contracts were thrown out of the

dataset because CMS and CMIS data did no agree on project start date.

Table 1: Combined Dataset Quality
Good Rejected
Missing Start Date/Year Year CMS/CMIS Conflicts Contract Amount CMS/CMIS Conflicts Subcontractor CMS/CMIS Conflicts TOTAL

Records 1983
333 95 16 21 2448

Contracts 462
78 26 2 2 570

While not nearly as prevalent, data were also rejected because of differences

between contract amounts or subcontractors. In the first case, 16 records representing 2

contracts were removed from the dataset because the full contract amount listed in the

CMS and CMIS data was vastly different. As noted in the GDOT final report, if the

discrepancy between CMS and CMIS was small (e.g. less than $10k) the CMIS amount

was used and the data was collapsed into agreement. In the cases noted above, the

difference was so great that this was not possible. Finally, 21 records representing 2

contracts were removed from the combined dataset because of differences in subcontract

140

information. For example, CMS would list subcontractors that were not included in the

CMIS data, and vice-versa.

In the instances in which things did not match up, the records would be left intact.

So, using the above example, if PI# 232 had the start year at 2001 in CMS, and 2003 in

CMIS, these records would not be collapsed, and (assuming the sub contractors matched

in both datasets) all 8 records would be copied into the worksheet for "conflicting start

dates".

It should also be noted that in the combined datasets, records were not rejected if

they were missing contract (full or sub) information, they were only rejected if these

amounts conflicted between CMS and CMIS. So if both CMS and CMIS listed the same

prime and sub contractors, as well as year, but were missing the prime contract amount,

or the sub contract amounts, these records were left in the dataset. This was because,

while contract amount was important to our research, the relationships between firms

were the most important.

Table 2: CMIS Dataset Quality
Good Rejected
Missing Start Date / Year Missing Prime Missing Subs TOTAL

Records 1255
383 20 7 1665

Contracts 264
130 20 7 421

Turning to the CMIS data in Table 2, in total there were 1665 records

representing 421 distinct contracts in the original CMIS dataset received from GDOT.

Because of missing contract start date, 383 records, representing 130 distinct contracts,

were removed from the dataset. Because they included subs, but lacked information on

prime contractor name, 20 records representing 20 contracts were removed. Finally, 7

141

records representing 7 contracts were removed because they were missing sub contractor information. After these records were removed the final dataset contained 1255 complete records representing 264 distinct contracts.
It was possible for a record (or contract) to be missing both year and prime contract information (or year and sub, prime and sub, etc). This situation, however, did not occur in the data.

Table 3: CMS Dataset Quality
Good Rejected
Missing Start Date / Year Conflicting Primes/Year TOTAL

Records 774
103 37 914

Contracts 241
28 6 274

Looking at the CMS data in Table 3, there were 914 total records representing

274 distinct contracts in the CMS dataset. 103 records, representing 28 contracts were

removed from the dataset because they information on the contract start dates. 37 records

(6 contracts) were removed from the dataset because they either listed two prime

contractors or because there were conflicting dates for the PI#. After removal of the bad

records, 774 records representing 241 contracts remained in the dataset.

142