Transportation project selection and prioritization: more data-driven, transparent process needed for selecting capacity projects [2016]

Special Examination Report No. 16-17

December 2016

Georgia Department of Audits and Accounts
Performance Audit Division
Greg S. Griffin, State Auditor Leslie McGuire, Director

Why we did this review
This special examination of the Georgia Department of Transportation's (GDOT) project selection and prioritization process was conducted at the request of the Senate Appropriations Committee. The Committee requested that we review how GDOT determines which highway projects it will fund and to what extent GDOT follows industry standards or best practices for setting priorities and selecting highway projects. The Committee also asked that we examine what opportunities exist for making the process for selecting highway projects more transparent.

Transportation Project Selection and Prioritization
More data-driven, transparent process needed for selecting capacity projects
What we found Best practices and national trends in project selection and prioritization emphasize linking selection criteria to long-term planning goals, using performance-based criteria and analytical tools to prioritize projects, and involving stakeholders to ensure transparency in selection and prioritization processes. However, GDOT's Planning Division does not appropriately employ many of these practices.

About Capacity Projects
The statewide transportation planning process for capacity projects is overseen by GDOT's Planning Director, who is appointed by and reports to the Governor. Capacity projects (e.g., widening, new roadways, managed lanes, etc.) are intended to reduce congestion.
From 2014 to 2017, approximately $740 million annually was allocated to capacity improvement projects. The Planning Division selects and prioritizes capacity projects in coordination with 16 Metropolitan Planning Organizations and nonurban local officials. As of July 2016, 676 capacity projects were programmed.1

While the Planning Division has developed a project scoring methodology, it is not used to decide which projects to select and program.1 Instead, projects go through an informal review process. Depending on the project, planning studies and analyses are sometimes conducted, and the amount of information gathered can vary substantially. It is not until projects are programmed that they are scored to determine funding order, but the scoring method and criteria are problematic. Additional problems exist with documentation and transparency. These issues are described in more detail below:
Project selection criteria and scoring methods The Planning Division lacks initial selection criteria, including benefit-cost analyses. Instead, projects are scored after the Planning Division has decided to program the project. The scoring determines implementation order, but several scoring criteria are not well-aligned with GDOT goals or outcomefocused. Also, the possible points assigned to the criteria are not

1 Program refers to GDOT's process of allocating funds to specific transportation projects.

270 Washington Street, SW, Suite 1-156

Atlanta, Georgia 30334

Phone: (404)656-2180

www.audits.ga.gov

reflective of their relative importance. Finally, congressional balancing requirements may cause lower priority projects to be advanced ahead of projects of greater need or benefit.
Process Improvements The Planning Division lacks detailed policies and procedures to guide key selection and programming decisions, and the basis for the decisions are not well-documented. For example, the Planning Division does not require any documentation or explanation for programming a low scoring project ahead of a high scoring project. In addition, the Planning Division lacks a way to systematically track all projects from proposal to implementation. As a result, it is difficult to review and analyze the total number of project proposals received and initially rejected or rejected after the project review.
Communication with Stakeholders The Planning Division could improve its communication of its overall project selection process and its criteria and scoring methodology. We surveyed officials from the 16 Metropolitan Planning Organizations and seven (44%) respondents disagreed or strongly disagreed that GDOT's overall process was transparent. In addition, the Planning Division does not provide information regarding reasons why specific projects were selected or not selected.
What we recommend The Planning Division should consider revising its criteria and scoring methodology and incorporating benefit-cost analyses to better assess project merit. In addition, the Planning Division should study the impact of the congressional balancing requirement on project prioritization decisions. The Planning Division should also improve its selection and prioritization approach by creating a more comprehensive, streamlined process and establishing more detailed policies and procedures. To improve transparency, the Planning Division should better communicate its process and decisions to stakeholders.
Like other states, the General Assembly may wish to include best practice requirements in statute.
See Appendix A for a detailed listing of recommendations.
Agency Response: GDOT and the Planning Division collaborated on this response. The Director of Planning reports to the Governor and oversees the Planning Division which identifies and selects projects for implementation. Other GDOT Divisions work closely with the Planning Division.
GDOT noted that it "strives to provide Georgians the best transportation system possible with our available resources in a transparent manner." In addition, GDOT acknowledged "that there is always opportunity for ongoing improvement and is giving due consideration to several areas referenced in the report." With regard to best practices in project selection and prioritization, GDOT stated that it is "an active member and participant in the American Association of State Highway Transportation Officials (AASHTO), where we gain and share national best practices." GDOT further noted that "there is not a `one size fits all' prioritization formula but rather that multiple criteria are utilized by each state in responding to its transportation needs." In addition, GDOT indicated that its ongoing focus on project prioritization resulted in the 2015 implementation of the customized prioritization tool. Lastly, GDOT stated its commitment to "continued evaluation of and modification to its selection criteria especially as federal programs evolve or change" (e.g., the creation of the national freight program as the result of the fiscal year 2016 federal transportation bill).

Transportation Project Selection and Prioritization

i

Table of Contents

Purpose of the Special Examination

1

Background

Long-Range Planning and Programming

1

GDOT Organization and Governance

3

Transportation Project Selection Process

4

Capacity Project Selection and Prioritization

5

Federal and State Laws and Regulations

8

Best Practices and National Trends

9

Performance Measures

11

Financial Information

13

Findings and Recommendations

Selection Criteria and Scoring Methods

15

The Planning Division should revise its process to ensure projects are formally

evaluated against a set of standard criteria before they are selected and

programmed.

15

The Planning Division should establish controls to ensure projects are consistently

evaluated against scoring criteria. The Planning Division should also prioritize

projects according to their scores to help inform decisions about which projects to

select and program.

17

The Planning Division should utilize project scoring criteria that most effectively

assess a project's need and potential impact.

20

The Planning Division should incorporate benefit-cost analysis as part of its project

selection process.

22

The Planning Division should revise its scoring methodology to better ensure that project scores accurately reflect the need for the project and the potential benefit. 24

GDOT, the Planning Division, and the General Assembly should explore alternative

methods for considering regional needs.

27

Process Improvements

29

The Planning Division should streamline and automate the project selection and

prioritization process and ensure that all relevant information is tracked and

accessible. As part of this effort, the Planning Division should consider the need for

decision-making software or tools with greater functionality.

29

Transportation Project Selection and Prioritization

ii

The Planning Division should establish more specific policies and procedures to

guide the project selection and prioritization process.

32

Transparency

34

The Planning Division should work with MPOs to clarify its level of input and

assistance in the development of the TIPs.

34

The Planning Division should better communicate its overall project selection

process and its criteria and scoring methodology. In addition, the Planning Division

should improve its communication on project selection decisions.

36

Appendices

Appendix A: Table of Recommendations

40

Appendix B: Objectives, Scope, and Methodology

42

Appendix C: MPO Map

45

Appendix D: Project Selection Processes for Maintenance/Pavement, Bridge, Safety

Enhancements, and Operational Improvements

46

Appendix E: Prioritization Criteria for Capacity Projects

49

Appendix F: FAST Act Federal Funding Programs

50

Transportation Project Selection and Prioritization

1

Purpose of the Special Examination
This review of the Georgia Department of Transportation's (GDOT) project selection and prioritization process was conducted at the request of the Senate Appropriations Committee. We limited our review to capacity projects, which are intended to reduce roadway congestion. Our examination focuses on the following:
1. How does GDOT determine which highway projects it will fund? 2. To what extent does GDOT follow industry standards or best practices for
setting priorities and selecting highway projects? 3. What opportunities exist for making the process for selecting highway
projects more transparent?
A description of the objectives, scope, and methodology used in this review is included in Appendix B. A draft of the report was provided to GDOT for its review, and pertinent responses were incorporated into the report.

Background

Programming is the process of allocating funds to specific transportation
projects.

Long-Range Planning and Programming
GDOT is responsible for planning, maintaining, and operating Georgia's highway system. GDOT's planning responsibilities include both general, long-range planning activities and project-level programming decisions. Long-range planning activities, such as conducting corridor studies, help assess future transportation problems and identify solutions. While long-range planning activities identify overall investment priorities, programming is the process of allocating funds to specific projects. Programming involves matching project schedules and cost estimates to the available funding sources, which have various eligibility restrictions. Generally, programming decisions can be influenced by both data-driven analyses and less quantitative factors including public support and project momentum.

Long-range planning and programming activities are largely driven by federal laws and regulations (23 CFR 450 Planning Assistance and Standards), which require certain documents outlining transportation needs, investment priorities, and project selection decisions be developed by GDOT and the 16 Metropolitan Planning Organizations (MPOs). MPOs are transportation policy-making organizations representing urbanized areas with populations over 50,000 (See Appendix C). Along with GDOT, MPOs have a significant role in the planning process as any transportation improvement project within an MPO's boundaries must be a part of its adopted plan in order to receive federal funding. Key planning ad programming documents required by federal (and state) law are discussed below and shown in Exhibit 1.
Statewide Long-Range Plans In prior years, GDOT produced two longrange planning documents the federally-required Statewide Transportation Plan (SWTP) and the state-required Statewide Strategic Transportation Plan

Transportation Project Selection and Prioritization

2



The STIP is the document that reflects GDOT's
programming decisions. It includes all planned federally-
funded projects, including those projects proposed in rural areas and metropolitan
areas (through the MPO TIPs).


(SSTP).2 In an effort to meet both the federal and state requirements in a single document, GDOT completed the 2040 SWTP/2015 SSTP in 2015. This Plan, developed in consultation with local officials in rural areas, MPOs, and other stakeholders, provides a comprehensive review of transportation issues, including growth trends and projections, economics, existing conditions, future needs, and an investment strategy. The Plan does not include specific projects (and is not required to do so) but does provide an overall programmatic assessment of the state's transportation systems.
MPO Long-Range Transportation Plans (LRTP) Each MPO, along with local government agencies within the MPO's boundaries, develops a longrange plan (in consultation with GDOT) that covers a 20-year timeframe and is updated at least every four or five years.3 These plans include a list of specific projects, as well as other information such as performance measures, projected demand for transportation services, cost estimates, and financial sources. As previously noted, a project within a metropolitan area must be included in the MPO's long-range plan in order to receive federal funding.
Statewide Transportation Improvement Program (STIP) The STIP is a four-year program that lists all highway, public transit, and multimodal projects proposed for federal funding. The final STIP must be approved and adopted by the GDOT Board, the Governor, the Federal Highway Administration (FHWA), and the Federal Transit Administration (FTA). Once a STIP has been approved, the first year of projects constitutes an "agreed to" list for project selection purposes.
GDOT typically updates the STIP annually. The process begins at the end of each calendar year and the proposed STIP is submitted to the Board for approval by September. However, the STIP was not updated in 2016 due to uncertainty regarding federal funding.
MPO Transportation Improvement Program (TIP) Each MPO develops its own program that lists four years of transportation projects proposed for federal funding. According to GDOT, the TIPs are updated annually to be consistent with the annual STIP update process. The projects are drawn from the MPO's long-range planning documents. Once approved by the MPO and the Governor, the TIPs are included as part of the STIP by reference.

Each MPO also develops a Unified Planning Work Program (UPWP) that identifies the planning priorities and activities to be carried out within a metropolitan planning area. At a minimum, the UPWP includes a description of the planning work and resulting products, who will perform the work, timeframes for completing the work, the cost of the work, and the source of funds.

2 Update periods have varied for the SWTP and SSTP. While federal legislation does not specify an update period for the SWTP, GDOT has historically updated the plan every 5 to 10 years. State law requires the SSTP to be updated every two years.
3 GDOT's role in the long-range planning process includes: 1) assisting in projecting future funding levels based on historic and current state and federal revenues; 2) voluntarily developing a travel demand model, with input from the MPO, for use in evaluating future needs and the impact of planned improvements (except in Atlanta); and 3) providing input on the department's priorities for future funding.

Transportation Project Selection and Prioritization

3

Exhibit 1 Key Planning and Programming Documents

Document Purpose/Contents

SWTP/ SSTP

Outlines statewide investment policies and strategies

Minimum

Who

Timespan Develops?

Who

Specific

Approves? Projects?

20 Yrs

Board, Governor,

GDOT

FHWA

No

LRTP STIP

Provides long-range and short-range strategies for an integrated intermodal transportation system within the MPO
Lists federally-funded transportation projects outside of MPOs and includes the TIPs by reference.

20 Yrs 4 Yrs

MPO

MPO

Yes

GDOT

Board, Governor, FHWA, FTA

Yes

Lists federally-funded transportation

TIP

projects within the MPO

4 Yrs

MPO

MPO, Governor

Yes

Source: Agency documents

In addition to the statutorily required documents listed above, planning studies are conducted to inform selection, programming, and prioritization decisions. These studies include the Georgia Statewide Freight & Logistics Action Plan, the Managed Lane System Plan, and the Atlanta Regional Managed Lanes Implementation Plan. For example, the Georgia Statewide Freight and Logistics Action Plan includes multimodal strategies (highway, rail, port, air cargo) for improving freight movement, an evaluation of individual projects, and an economic analysis of freight packages (i.e., groups of projects).
GDOT Organization and Governance
GDOT's Planning Director, Commissioner, and Board, as well as the Governor, all have oversight responsibilities related to planning and programming. GDOT's Planning Director is responsible for developing the statewide long-range transportation plans and programs. These transportation plans and programs must be approved by the Governor and the State Transportation Board. Implementing the plans and programs is the Commissioner's principal responsibility. As discussed below and shown in Exhibit 2, GDOT is unique in both its overall governance structure and the role of the Planning Director.
Overall Governance GDOT is governed by the State Board of Transportation which is comprised of members from each of the state's 14 congressional districts. Board members are elected by each district's state representatives and senators. The Board appoints the GDOT Commissioner. GDOT's governance structure a board selected by the legislature and a department head selected by the board is a distinctive model to Georgia. The most common governance model used in other states consists of a board or commission selected by the governor and a department head selected by the governor with legislative approval.
Planning Director Since the passage of Senate Bill 200 in 2009, the Planning Director is appointed by the Governor and subject to confirmation by the State House of Representatives and Senate Transportation Committees. As shown in Exhibit 2, the Planning Director and the Division of Planning that

Transportation Project Selection and Prioritization

4

he or she oversees are detached from the oversight of the Commissioner and the Board and accountable to the Governor. GDOT also differs from other states by having a Planning Director who reports directly to the Governor. All other state departments of transportation utilize a governance model with a board, commission, or department head that oversees the entire department, including transportation planning.
Exhibit 2 GDOT's Planning Director Reports Directly to the Governor (1)

Governor

State Board of Transportation
Elected by Legislature

Director of Planning Appointed by Governor

Commissioner Appointed by Board
Chief Engineer

Division of Engineering

Division of Permits & Operations

Office of Planning

Office of Bridge Design & Maintenance

Office of Traffic Operations

(1) This is not an agency-wide organization chart. It only includes GDOT offices most directly involved in project selection. Source: Agency Documents and state law

Office of Utilities

Office of Maintenance

The Planning Director provides day-to-day oversight of the Planning Division. The division is staffed by a planning administrator, three assistant planning administrators, and 25 other staff whose responsibilities include gathering and compiling transportation data that is needed for planning and reporting purposes.4 Along with the Planning Division, other GDOT offices assign available federal and state funds to planned projects.

Capacity projects are intended to reduce
congestion. Typical projects include widening, passing lanes, and new location
roadways.

Transportation Project Selection Process
GDOT is responsible for allocating funds among highway projects in a way that maximizes its resources. To accomplish this, the Planning Director first distributes funding among five different project types capacity, pavement, bridges, operational improvements, and safety enhancements. As shown in Exhibit 3, GDOT allocates approximately 50% of highway funding totaling $740 million annually to capacity projects.5 Once funds are allocated according to project type, five different GDOT offices are responsible for selecting projects within each category. The Planning Division's selection process for capacity projects is the focus of this review and is discussed in more detail in the next section. Project selection processes for the other project types is discussed in more detail in Appendix D.

4 Staffing figures are as of April 2016. 5 Based on funding amounts derived from the 2014-2017 STIP.

Transportation Project Selection and Prioritization

5

Exhibit 3 Funds Are Allocated by Project Type and Projects are Then Selected

Funds Allocated by Project Type(1)

$740 Million

$277 Million

$236 Million

$144 Million

$78 Million

Capacity
Widening, new location roadways, express lanes,
etc.

Pavement
Statewide resurfacing, pavement preservation,
statewide restriping

Bridges
Bridge maintenance, bridge replacements, etc.

Safety Enhancements
Pedestrian safety, rumble strips, intersection
improvements, guardrails, railroad crossings etc.

Operational Improvements
Ramp metering, signal coordination, variable speed
limit signs, etc.

Office of Planning
Methodology: Criteria/Scoring

Office of Maintenance
Methodology: Asset-Management Based

Office of Bridge Design & Maintenance
Methodology: Asset-Management Based

Office of Traffic Operations/ Office of Utilities
Methodology: Benefit-Cost Analysis, System-wide Studies, Adjusted Hazard Index

Office of Traffic Operations
Methodology: Operations Improvement
Committee

Selected Projects Programmed in the STIP/TIP(2)
(1) Average annual funding amounts as reported in SWTP/SSTP and derived from the 2014-2017 STIP. GDOT staff indicated that there may be some exceptions to this categorization/ process. For example, some of the larger operational improvement projects are selected/prioritized by the Office of Planning. (2) Some projects may be included in lump sum amounts rather than listed individually. Source: Agency documents
Capacity Project Selection and Prioritization Transportation improvements, including capacity projects, can be proposed by anyone, including GDOT, MPOs, city and county governments, and citizens, as shown in Exhibit 4. MPOs have their own project selection and prioritization processes for project proposals to be included in their long range plans. Generally, potential projects in MPO areas are identified by the MPO and then evaluated against criteria, regional goals and objectives, financial constraints, and Clean Air Act requirements, before being prioritized for inclusion in the long-range transportation plan. GDOT determines the availability of state and federal funds for projects in MPO areas and assists them in prioritizing projects for inclusion in the TIP. Once the long-range plan and the TIP are approved, the TIP is included in the STIP by reference, as previously noted.6
The Planning Division selects projects in non-MPO areas through an informal review process. When a project is proposed to GDOT, the Planning Division may immediately reject the project for reasons including the potential for extensive environmental/residential or business impact. If it is determined that further evaluation is warranted, staff will review and analyze relevant data such as traffic analysis and crash history. If a need for a project is determined, the Planning Division develops a cost estimate and recommends years for implementation based on funding availability and project readiness. The project programming request is signed by the Director of Planning and Chief Engineer and provided to the Office of Financial Management (OFM) for inclusion in GDOT's project management tool, TPRO. OFM
6 The long range transportation plan is approved by the MPO and the TIP is approved by the MPO and the Governor.

Transportation Project Selection and Prioritization

6

assigns a project ID into TPRO and the Office of Planning then includes the project in its prioritization spreadsheet.
Exhibit 4 Project Selection Process for Capacity Projects

Problem is identified by GDOT, MPO, state or local official, general public, etc.

Conduct a study?

YES
GDOT Planners collect and analyze data (e.g., crash history) and make a recommendation, which could be to do nothing

NO

Planning Division

Management reviews the

recommendation

Program the project in the
STIP/TIP?

YES

Projects are programmed in the STIP (rural) and TIP (urban).

NO

(Project remains

in pool to be funded

in future years)

Congressional balancing requirements are reviewed and project selections are

The TIPs are approved by the MPOs.

adjusted to ensure

compliance.

Planning Division evaluates projects based on the score and other factors, such as project support (based on
public comments and meetings with local officials)
Planning Division staff scores projects according to defined
criteria related to safety, congestion, local support, etc.

The final STIP/TIP is approved by the Board, Governor, FHWA, and
FTA.
The project progresses through preliminary engineering, right-of-
way, etc. Public meetings are held to discuss projects. GDOT continues to evaluate the project.

Select the project?

YES

Project is assigned ID and placed in long-range or 10
year program, based on project readiness and funding
availability

NO
Requester Notified (if externally requested)

NO, but maybe In future years

Implement the project?

YES

Project is let for construction

No, not ever

Project is cancelled

(1) The flowchart is intended to provide a general overview; exceptions to this process may occur. It should also be noted that MPOs are involved in identifying, selecting, and prioritizing projects but that role can vary by MPO. Source: Agency documents and staff interviews

Once a project is programmed and included in the prioritization spreadsheet, the Planning Division utilizes a project scoring process to help determine funding order. It should be noted that projects included in MPO TIPs are also subject to the Planning Division's scoring process to determine when they will be funded. Projects are scored according to 13 criteria (five key criteria and eight additional criteria) related to economic development, congestion, safety, and other considerations. Each of these criteria is shown in Exhibit 5 and described in greater detail in Appendix E.

Transportation Project Selection and Prioritization

7

Exhibit 5 Planning Division's 13 Criteria for Capacity Project Scoring

Key Criteria

1. Connectivity: Interchange/Bypass Projects 2. State Route Prioritization (critical, high, medium, low) 3. Project on the Freight Network 4. Project in the Freight Plan
5. Project in top 5 priorities in GRIP corridor (economic development)

Additional Criteria

6. Safety: Crash Data (injuries, fatalities, property damage) 7. Congestion: Level of Service Improvement 8. Current Pavement Condition 9. Project Identified in Planning Study 10. Approved Concept Report (project readiness) 11. Signed PFA (indicates local funding/support) 12. Project Meets SSTP Goal(s)
13. Project Located on RTOP Corridor (traffic signal timing program)

Source: Agency documents
While project scoring is intended to help prioritize projects for funding purposes, the Planning Division's decisions are also influenced by qualitative factors including project momentum, public support, and funding availability. Federal funds are provided to states within funding categories or programs that focus on key areas. According to Planning Division management, funding for capacity projects (and other project types) is contained in 15 different funding programs and subprograms.7 (See Appendix F for more detail.) For example, a portion of GDOT's federal funding is provided through the Surface Transportation Block Grant Program - Areas with Population 5,000 and Under Subprogram. This means that funding under this program/subprogram can only be used on projects in areas meeting the 5,000 or less population requirement. These funding requirements and other factors noted above are considered during Planning Division management meetings where projects are discussed and STIP decisions are made. A draft STIP is released for public comment and then the final STIP, including MPO TIPs incorporated by reference, is approved and adopted.
Once a project is included in the STIP, it generally proceeds through the various project delivery phases scoping, preliminary engineering, right-of-way, utilities, and construction. At each milestone, project cost estimates are updated and GDOT will continue to evaluate the need and feasibility of completing the project. During the annual or four-year update period, projects may be removed from the STIP for various reasons, such as lack of public support. When a project is removed from the STIP, GDOT may either cancel the project or keep the project active and up for consideration in future years.

7 Includes funding programs established under the federal Fixing America's Surface Transportation Act (FAST Act). GDOT also has federal funds remaining in programs established under previous transportation funding legislation (e.g., MAP-21).

Transportation Project Selection and Prioritization

8

As of July 2016, there are approximately 676 capacity projects programmed, including 326 (48%) projects in MPO areas and 302 (45%) projects outside of MPO areas.8

Federal and State Laws and Regulations
Federal and state laws and regulations establish various planning requirements, including the production of certain planning documents, public involvement, performance measures, project prioritization, and funding distributions.
Federal
Federal regulations require GDOT to carry out a continuing, cooperative, and comprehensive transportation planning process, including the development of the STIP and statewide long-range transportation plans. These regulations require that the statewide long-range plans and STIP be developed in consultation with the general public, MPOs, and nonmetropolitan elected officials. Additionally, federal regulations require that each project included in the STIP be consistent with the statewide long-range plans and metropolitan TIPs and full funding be reasonably anticipated.
Federal law also emphasizes a performance-based planning process. The Moving Ahead for Progress in the 21st Century Act (MAP-21), enacted in 2012, requires state DOTs and MPOs to incorporate performance goals, measures, and targets into the process of developing long-range transportation plans. GDOT indicated that national performance goals, measures, and targets are currently under development at the federal level for pavement/bridge and system performance and will be provided to state DOTs. The Fixing America's Surface Transportation Act (FAST), enacted in 2015, maintains the importance of performance-based planning. In addition, the FAST Act requires states to place additional emphasis on transportation projects that benefit the movement of freight.
State
State law does not include specific provisions for project selection and prioritization but does provide some broad requirements pertaining to investment areas and reporting. For example, state law (O.C.G.A. 32-5-27.1) establishes priority for highway maintenance, expansion, and improvement projects in areas most impacted by traffic congestion and areas in need of highway infrastructure for economic development. State law also requires GDOT to produce a 10-Year Strategic plan (separate from the SWTP/SSTP) that shows the percentage of resources to be expended in the following areas: construction of new highway projects; maintenance of existing infrastructure; bridge repairs and replacement; safety enhancements; and administrative expenses. GDOT is also required to annually submit a list of priority projects that reflect 10-20% of the Surface Transportation Plan budget to the Governor for consideration in advance of the legislative session each year. This list is a subset of larger, more noteworthy projects that had already been identified for funding that year, according to Planning Division management. Lastly, state law requires the Planning Director to annually submit the following three reports to the Governor, Lieutenant Governor, Speaker of the House of Representatives and the Transportation Committees: 1) a report detailing the progress of projects and

8 An additional 48 projects (7%) fall in both MPO and non-urban areas.

Transportation Project Selection and Prioritization

9

Congressional balancing requirements were enacted
in 1999 in an effort to address concerns over funding equity. The law requires 80% of total expenditures to be divided equally among the state's congressional districts.

programs in the SSTP; 2) a report detailing the progress of every construction project valued at $10 million or more against the benchmarks; and 3) a report detailing the amount of money saved due to value engineering studies each calendar year.
State law also includes funding allocation requirements that impact project prioritization. O.C.G.A 32-5-30 requires GDOT to budget total state and federal expenditures over two five-year budget periods each decade.9 For each period, 80% of total expenditures must be divided equally among the state's 14 congressional districts. 10 This "congressional balancing" requirement does not apply to expenditures of federal funds specifically earmarked for projects by a member of Congress, any funds for Interstate projects, or any funds for freight corridor projects that have been proposed by the Planning Director and approved by the State Transportation Board. In addition, the Board has the authority to waive the requirement if it conflicts with federal requirements or other circumstances prevent timely project implementation. The congressional balancing requirement has become less restrictive over the years. The original law, implemented in 1999, required that 100% of project funds be divided equally over a three-year period, and there were no exemptions for Interstate and freight network projects.
Best Practices and National Trends
Industry and academic research highlight best practices and national trends in project selection and prioritization, including selection criteria and scoring methodology. Throughout the report, we refer to various selection and prioritization methods, tools, and processes identified and promoted by leaders in the transportation industry as best practices. In addition, our research identified several states implementing one or more best practices. While no one advocates a strictly data-driven decision-making process, these best practices and trends can provide a framework to help ensure transportation officials consistently and reliably select project alternatives that best meet transportation needs in the state. These practices are summarized below.
Performance-based Planning and Programming The importance of performance-based planning and programming (PBPP) is emphasized by American Association of State Highway and Transportation Officials (AASHTO), which has a PBPP Task Force, the Federal Highway Administration (FHWA), as well as federal regulations. PBPP is a data-driven, strategic approach that provides for public and stakeholder involvement and accountability. In addition to providing guidance on funding new capacity projects, a PBPP approach provides supplemental information to help decision makers understand the impact on performance goals (congestion, safety, economic development) if allocations must be reduced due to revenue declines. MAP-21 establishes national goals and calls for the use of performance-based approaches to support those goals. It also requires that state transportation agencies set targets in relation to a set of national performance measures, and calls for coordination of target-setting between states and MPOs to ensure consistency.

9 A new balancing period begins every five years. The current balancing period is 2013-2017.
10 The State Transportation Board has the discretion to determine where the remaining 20% of funds are spent.

Transportation Project Selection and Prioritization

10

Linking Planning and Programming According to FHWA, establishing a strong link between planning and programming is a critical step for state DOTs. The process is complex due to the wide pool of projects to consider and having to balance the needs of rural and urban areas. The FHWA emphasizes that project selection criteria for the STIP should reflect those used to assess priorities in the long-range transportation plan (LRTP).
Prioritization Criteria A peer exchange hosted by North Carolina's Department of Transportation sought to gather innovative examples and best practices for project prioritization. The peer exchange developed the following recommendations for establishing prioritization criteria:
Keeping criteria simple and high-level helps keep decision-making transparent;
Criteria should focus on outcomes rather than outputs; Prioritization criteria should calculate the benefits of proposed projects, and
not simply assess the existing conditions; The content of long-range plans and other multi-modal plans should support
an agency's choice of prioritization criteria; Agencies should choose a manageable number of criteria (i.e., five or six) to
focus on meaningful and comprehensible outcomes; Criteria should focus on impacts to the traveling public rather than impacts
to infrastructure itself (e.g., amount of traffic crossing deficient bridges rather than the number of deficient bridges); Criteria should consider the context of each project (e.g., a rural project should not necessarily lose points for not including sidewalks); and Where possible, criteria should rate projects based on mode-neutral characteristics, such as "asset condition" rather than "pavement condition".
Analytical Tools & Methodology A number of research reports document various decision-making models used to rank transportation projects against multiple criteria. While these models rely on decisionmakers to identify and weight relevant criteria, the actual models themselves perform the rankings. This allows decision-makers to objectively prioritize projects in an automated manner rather than having to rank projects individually.11
A state survey conducted for AASHTO identified four common prioritization approaches benefit-cost analysis, cost effectiveness analysis, process based approach, and goal based approach.12 The report concluded that no single technique is relied on exclusively and that no single approach is "best." A mixed approach may be most useful, where benefit-cost analysis is one factor or criterion among several.13

11 The most common methods include Analytic Hierarchy Process (AHP) and Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS).
12 Gunasekera, K. & Ira Hirschman. (November 2014). Cross Mode Project Prioritization (NCHRP 0836, Task 112). 13 Based on the surveys and case studies, weighting assigned to benefit-cost analysis ranged from 5% to 65% within the categories of projects it was applied to.

Transportation Project Selection and Prioritization

11

In addition, weighting, scaling, and normalization are useful tools to ensure that variables/metrics under consideration are comparable, prioritization criteria is relevant, and project merit is accurately reflected.
GDOT Project Prioritization Study
In 2006, GDOT leadership initiated a project to develop a more data-driven process in accordance with best practices and subsequently hired Cambridge Systematics, Inc. The two main components of the process were performance measures and benefit-cost ratios. The performance measures were tied to long-range goals and weighted according to project type and project location. The benefit-cost ratio compared benefits, including the reduction in fuel and delay costs, and project costs, including right-of-way and construction. Based on the performance ratings and benefit-cost ratios, projects were to be categorized into one of four prioritization tiers, as shown in Exhibit 6. However, the consultant's recommendations were never fully implemented due to GDOT's concerns regarding the lack of flexibility.

Exhibit 6 Cambridge Study Recommended a Tier-Based Prioritization Process

Benefit/Cost Score threshold

Tier 2 Benefit/Cost threshold

Tier 1

Tier 4

Tier 3

Performance Score
Source: Cambridge Systematics 2009 Review
Performance Measures As noted in both the Cambridge study and other best practice research, state DOTs are moving toward the use of performance measures and targets to inform decisions about the amount of funding that will be directed to certain project types (e.g., maintenance, capacity additions). With regard to capacity projects, congestion and safety measures are important indicators of the system's performance and potentially signify areas of need. As discussed on pages 6-7 and later in the finding on page 20, the Planning Division factors these measures into its project prioritization decisions. In addition, MPOs may consider these factors when deciding which projects to include in their plans and programs.

Transportation Project Selection and Prioritization

12

Our review of recent trend data for these measures indicate that GDOT is not currently meeting its targets for congestion or safety, as discussed below.
Congestion As shown in Exhibit 7, both morning and evening peak hour freeway speeds decreased between 2012 and 2016. Currently, the average morning speed is 36.51 and the average evening speed is 33.53, which are below GDOT's target of 40 mph in metro-Atlanta. In addition, GDOT is not meeting its goal of reducing congestion costs per auto commuter in the Atlanta region annually by 10%. While congestion costs decreased significantly between 2006 and 2009, costs have increased or remained stagnant since then.
Exhibit 7 Freeway Speeds Worsen While Congestion Costs Stagnate

Peak-Hour Freeway Speeds in Metro Atlanta

42 40.22
40 39.36
38
36
34
32
30 FY 2012

37.56 37.34

Target = 40 MPH 36.51

FY 2013

FY 2014

Morning

Evening

FY 2015

33.53 FY 2016

$2,000 $1,800

Congestion Costs per Auto Commuter in Metro Atlanta
$1,888

$1,600

$1,400

$1,200 $1,000

$1,092

$1,132

$1,130

$800

2005

2006

Source: Agency documents

2007

2008

2009

2010

2011

2012

2013

2014

Safety Traffic fatalities declined during the eight-year period from 2006 to 2014. As shown in Exhibit 8, the number of annual fatalities decreased from 1,706 to 1,170 (31%) during the period. In the most recent year reported, there was a spike in fatalities, increasing the number by 22% (from 1,170 in 2014 to 1,432 in 2015). The spike also resulted in GDOT not meeting its goal of reducing annual fatalities by 41 in 2015. According to national statistics, the 2015 increase in traffic fatalities was seen nationwide.

Transportation Project Selection and Prioritization

13

Exhibit 8 Fatalities Decrease through 2014 and Then Rise in 2015

1,800 1,706
1,650
1,500
1,350
1,200
1,050

Safety Measure: Number of Annual Fatalities

1,508

1,250

1,199

1,432 1,170

900 2006

2007

2008

Source: Agency documents

2009

2010

2011

2012

2013

2014

2015

Financial Information
GDOT's budget and expenditures have increased significantly in the last year as a result of the Georgia Transportation Funding Act of 2015 (i.e., House Bill 170). As shown in Exhibit 9, GDOT's fiscal year 2017 budget totals $3.4 billion, a 38% increase from its fiscal year 2015 expenditures. The majority of the additional funding can be attributed to motor fuel taxes, which increased from $1.2 billion to $1.68 billion as a result of House Bill 170. House Bill 170 replaced the state gasoline sales tax with a single gasoline excise tax, eliminated tax credits on low/zero-emission vehicles, and created new fees for hotel/motels, heavy vehicles, and alternative fuel vehicles. Motor fuel taxes account for nearly half of GDOT's fiscal year 2017 budget.
The funding increase has resulted in greater investment in various programs, particularly capital maintenance, routine maintenance, and capital construction. Between fiscal years 2015 and 2017, capital maintenance increased from $119 million to $293 million (146%) and routine maintenance increased from $229 million to $454 million (98%). During this same time period, capital construction increased from $1.2 billion to $1.7 billion (36%). According to Planning Division management, the additional state funding will enable the implementation of large-scale projects and allow greater flexibility in project selection decisions. Planning Division management indicated that federal fund sources have more eligibility restrictions, and as a result, can constrain their decision-making. See Appendix F for a listing of federal fund sources and restrictions.

Transportation Project Selection and Prioritization

14

Exhibit 9 Expenditures Increased 38% between Fiscal Years 2015 and 2017

2015 (Actual)

2016 (Actual)

2017 (Budgeted)

Percent Change

By Fund Source Federal Funds Motor Fuel Taxes State General Funds Other

$1,142,613,272 $1,023,458,994
$14,884,378 $286,841,726

$1,640,550,766 $1,440,253,611
$43,316,072 $200,836,492

Total $2,467,798,370 $3,324,956,941

By Program Capital Construction Projects Routine Maintenance Capital Maintenance Projects Debt Service Local Maintenance & Improvement Grants Construction Administration Traffic Management and Control Local Road Assistance Administration Intermodal Departmental Administration Planning Data Collection, Compliance, & Reporting
Total

$1,232,489,892 $229,307,807 $119,045,072 $247,994,653 $127,983,622 $142,525,503 $89,771,827 $94,323,974 $103,105,012 $58,561,190 $15,388,035 $7,301,781
$2,467,798,370

$1,570,043,960 $439,212,984 $431,951,621 $250,041,861 $136,876,373 $130,192,963 $104,454,380 $45,320,296 $109,905,818 $68,378,820 $31,590,940 $6,986,924
$3,324,956,941

Source: Budgetary Compliance Reports and Appropriations Act

$1,593,146,310 $1,660,064,000
$54,479,424 $93,537,703
$3,401,227,437
$1,678,795,154 $454,011,607 $293,168,959 $252,212,858 $165,562,234 $169,799,165 $97,707,637 $96,597,611 $85,562,631 $78,952,804 $16,453,554 $12,403,223
$3,401,227,437

39.4% 62.2% 266.0% -67.4%
37.8%
36.2% 98.0% 146.3%
1.7% 29.4% 19.1%
8.8% 2.4% -17.0% 34.8% 6.9% 69.9% 37.8%

Transportation Project Selection and Prioritization

15

Findings and Recommendations

Selection Criteria and Scoring Methods

The Planning Division should revise its process to ensure projects are formally evaluated against a set of standard criteria before they are selected and programmed.

The Planning Division lacks a formal process and criteria for making project selection and programming decisions. While the Planning Division has procedures for conducting project recommendation planning studies to evaluate potential projects, these studies are not being routinely produced. The Planning Division also has a scoring process for evaluating projects, but does not use it to determine whether to select and program a project. As a result of these issues, we could not review the basis for selecting the 676 capacity projects that are currently programmed. In addition, the Planning Division does not maintain information on projects that never made it to programming and the reasons they were rejected. Without clear criteria/more formalized review processes, there is less assurance that all potential projects were evaluated consistently and objectively and that selected projects demonstrate the greatest need and potential impact.

$2 Billion Truck Lanes Project GDOT programmed a project adding toll-free, truck-only lanes on I-75 between Macon and McDonough without clear indication that the project is a justified investment. A 2008 needs identification study recommended

While the Planning Division has

procedures for evaluating potential

projects,

Planning

Division

management indicated that it is more

against a truck-only lane system in metro-Atlanta for reasons including high of an informal process. As noted on

costs and the concentration of generalized benefits accruing to a limited page 3, Planning Division management

percentage of interstate users during peak periods. While the study provided use planning studies to inform their

indications that the Macon-McDonough project might be a worthwhile project selection and programming

investment, it made no specific recommendation for or against the project. decisions. In addition, the Planning

GDOT programmed the project in an effort to relieve congestion expected to Manual outlines steps for conducting

result from expansion of the Port of Savannah (which GDOT anticipates will project recommendation planning

increase freight traffic and congestion). After GDOT programmed the project, a consultant estimated it would result in a 40% reduction in vehicle hours of delay, but did not monetize the benefits and compare them to the costs. Estimated capital costs for the project exceed $2 billion (plus an additional $4 billion in operations and maintenance costs).

studies, which include meeting with stakeholders, collecting and analyzing data, identifying alternatives, obtaining cost estimates, and making a recommendation, which could be to do

nothing. However, we found that

project recommendation planning studies are not routinely conducted for all proposed

projects. For example, a $2 billion truck-only lanes project was programmed without

a full and complete assessment of the need for the project, evaluation of options and

the prosusaend cons of each, and an explanation for the option selected. A 2008 needs

analysis of statewide truck-only lanes indicated the project "appeared to have

preliminary merit," which seems insufficient justification for a major investment

(more than 2.5 times the $740 million annual investment in capacity projects.) A

subsequent analysis conducted in 2016 estimated the project would reduce travel

Transportation Project Selection and Prioritization

16

times (vehicle hours of delay) by as much as 40%, but the project had already been selected and programmed by that time.14
In addition, there are no standard criteria against which all potential projects are evaluated. Planning Division management indicated that staff analyze relevant data (e.g., crash history), but it is a more informal process, meaning the results may not be formally documented in a study or report. Based on the information gathered (which could vary substantially across projects depending on the amount of evaluation conducted), management decides whether or not to select and program the project. Management also assigns the year for implementation, which could be short-term or long-term (e.g., 35 years out).
After a project is programmed, the Planning Division scores the project according to established criteria if the project is scheduled within a 10-year time frame. However, at this point, the decision to select the project, allocate funding, and assign a programming year have already occurred. Because of the timing of the scoring process, the project scores can help inform implementation order for projects scheduled in the upcoming 10 years but cannot inform programming decisions.
In comparison, Virginia and North Carolina have more formal project selection processes and utilize project scoring to inform programming decisions. Virginia has a multiphase screening process that first determines whether a potential project meets eligibility criteria, satisfies a need identified in the long-range transportation plan, and has a clear and reasonable scope, cost, and schedule. Projects screened out are independently reviewed by a multimodal transportation committee for quality assurance. Projects that pass through the screening process are scored, and the scores then inform programming and funding decisions. North Carolina also uses the scoring process to determine whether a project receives funding and to determine project schedules.
RECOMMENDATIONS 1. The Planning Division should formalize its initial project review process and ensure that studies are conducted as prescribed in the Planning Manual.
2. The Planning Division should develop a process for screening project proposals either by 1) scoring projects prior to programming so the results can more objectively be compared and inform selection decisions or 2) establishing separate screening criteria for use in evaluating projects prior to programming.
Agency Response: GDOT indicated that its "2040 SWTP/SSTP notes investment strategies and the STIP (MPO TIPs) is the formal project selection process."
Auditor's Response: We agree that the STIP (and MPO TIPs) are formal processes. However, this finding relates to the lack of a standard protocol or criteria used to evaluate project proposals prior to programming.

14 Figures represent estimates of capital costs. Because the project is design/build/operate, GDOT anticipates incurring an additional $4 billion in operational and maintenance expenditures.

Transportation Project Selection and Prioritization

17

The Planning Division should establish controls to ensure projects are consistently evaluated against scoring criteria. The Planning Division should also prioritize projects according to their scores to help inform decisions about which projects to select and program.
The Planning Division's project prioritization process lacks the controls necessary to ensure an objective, data-driven project selection process. First, the Planning Division does not require that all projects be scored or objectively evaluated by other means. For those projects that are scored, the Planning Division is lacking data controls to ensure that scores are complete and accurate. Lastly, there are no procedures in place to ensure that project scores or other objective analysis serve as the basis for decisionmaking. An environment where exceptions occur but are not transparent can erode trust with stakeholders and increase the risk of undue political influence.
Our review found a perception existing among stakeholders that the project selection process is subject to political influence. We surveyed MPOs and local government officials in Georgia and found that political influence was perceived to be a moderate or major factor influencing project selection by 10 of 16 (62.5%) MPO officials and 83 of 145 (57.2%) city/county government officials who responded. Whether political influence is real or perceived, the perception is likely to persist if scoring exceptions and weak procedural controls remain.
Controls to Ensure that All Projects Are Scored/Objectively Evaluated
The Planning Division allows certain projects to bypass the initial review process and scoring procedures. Planning Division management indicated that certain projects resulting from economic development initiatives (e.g., Kia Motors manufacturing plant) bypass both the initial project evaluation process and the project scoring process because projects must be implemented when the momentum exists and there may not be enough time for evaluation and scoring. The Planning Division also allows exceptions to the scoring process for projects that: 1) were well underway (e.g., rightof-way acquired) when the scoring process was initially implemented; or 2) involve special circumstances or characteristics and do not fit well within the scoring criteria (e.g., the military base project highlighted on the next page). We reviewed the Planning Division's prioritization spreadsheet and found that 25 of 280 (9%) capacity projects programmed between fiscal years 2016 and 2025 and costing $3.1 billion were unscored.15,16 The Planning Division does not document reasons for not scoring projects.

15 We limited the analysis to projects programmed prior to 2026 because Planning Division management indicated that it scores projects within a 10 year time frame. 16 Includes the $2 billion I-75 truck lanes project described on page 15, which accounts for 67% of the total cost.

Transportation Project Selection and Prioritization

18

Military Base Project Bypassed Scoring
This $17 million project involves widening and a two-lane roundabout for a new access gate at Fort Gordon. The section of Fort Gordon Highway leading to the new gate is expected to experience increased traffic volumes as other gates are scheduled to close. In addition, the increase in traffic volume is expected to be compounded by anticipated growth at Fort Gordon and the surrounding area. Planning Division management indicated that the project

Other states have legislative mandates or procedural controls that impede the ability to bypass scoring. North Carolina and Virginia both have legislative mandates requiring all capacity projects to be scored. Ohio requires all projects to be scored with special procedures applied for projects that do not fit well within the standard scoring criteria (an additional alternative evaluation step

was not scored because it had local support is applied to provide context for low

and was considered a national/cyber security scores). Ohio also accommodates the

concern, as the new gate is for the cyber urgency of certain economic

security wing. Management also noted that it development projects by allowing off-

would have been difficult to apply some of the cycle review of proposals; however,

scoring measures because the base had not all economic development projects

expanded yet.

are scored regardless of urgency. All

of these states, as well as Minnesota

and Oregon, consider economic development factors (e.g., estimated jobs created) for

all projects reviewed as part of their standard evaluation processes, enabling economic

development projects to fit more cleanly/logically within the standard scoring criteria.

Controls to Ensure that All Project Scores are Complete and Accurate
The Planning Division is lacking data controls in the prioritization spreadsheet, and as a result, project scores may not be complete and accurate. As shown in Exhibit 10, 23 of 255 (9%) scored projects had incomplete scores due to missing traffic data. Planning Division staff indicated that the missing data reflects a gap in availability of traffic data from GEARS (an external data source out of GDOT's purview) that had not yet been reconciled. However, there is no clear indication (e.g., flag or comment) in the primary project prioritization spreadsheet that these scores are incomplete. Of the 232 remaining projects with complete scores, 165 (71.1%) had an absence of supporting data for at least one of the scoring criteria. For instance, the field to indicate whether a project was a bypass or interchange (yes or no) was left blank and the criteria defaulted to a zero score, presuming a "no" answer. Planning Division staff indicated that the blanks are intentional (i.e., staff knew it would default to zero and did not enter "no") and not a reflection of missing data. However, it is unclear how one can be certain that each of the missing values was intentional and not an oversight.

Transportation Project Selection and Prioritization

19

Exhibit 10 Projects Programmed with Incomplete Scores or Supporting Data

Scored?

Score Complete?
All Supporting Data Complete?
Source: Agency data

Yes: 67

Projects Programmed: 280 Yes: 255
Yes: 232 No: 165

No: 25 No: 23

Controls to Ensure that Decisions are Based on Project Scores
The Planning Division lacks procedures requiring decisions be based on project scores or other objective analysis. The Planning Division does not produce a formal project rankings document that could serve as an accountability mechanism. Likewise, the Planning Division does not have a formal approval process or documentation requirement for decisions that deviate from the results of the prioritization process. This could include documentation of decisions to fund low-scoring projects ahead of higher-scoring projects, as well as exceptional instances that require bypassing the scoring process altogether.
The 2009 Cambridge Systematics report recommended several strategies for ensuring an objective process. First, the report recommended that projects be clearly categorized into prioritization tiers (1-4). Second, the report recommended defining a certain percent of the budget that must be allocated to Tier 1 projects (highest priority) and establishing rules for when projects in Tiers 2 through 4 could be implemented. Third, the report recommended tracking the consistency between the project priority list and the programming list to ensure that the projects being implemented best support agreed-upon priorities. Lastly, the report acknowledged that some projects may occasionally work their way through the project development cycle outside of the prioritization process but that decisions to implement projects despite a low ranking should be documented.
Similar controls have been implemented in other states to ensure that decisions are based on the results of the prioritization process. North Carolina, Tennessee, and Virginia generate prioritized project lists from their scoring system results, some of which are made publicly available. Ohio narrows applicant projects to a priority subset based on project scoring results that is then organized into project tiers (i.e., construction ready projects, projects under development, and mega-projects) and projects ready to proceed are given priority within each tier. Virginia requires a documentation process for the rare instances where projects are funded out of sequence and a public involvement process when substitutions are made. Oregon also

Transportation Project Selection and Prioritization

20

requires documentation when a lower-ranking project is substituted for a higher ranking project.
RECOMMENDATIONS
1. The Planning Division should ensure that all capacity and economic development projects are scored or objectively evaluated. Specifically the Planning Division could: a. Develop a more formal, objective process to evaluate economic development projects. This could be accomplished through a revision of existing scoring criteria to embody economic evaluation of all projects or development of a separate evaluation process tailored to capacity projects. b. Develop an alternative evaluation process for all projects that do not fit well within the standard criteria.
2. The Planning Division should establish data controls to ensure accuracy and completeness of project scoring data. These could include: a. Indication of scoring status (e.g., not yet scored, partially scored, scoring complete) b. Use of data fields that require a response (e.g., a menu to select yes/no answer). c. Ensuring fields that calculate scores do not default to "0" in absence of data.
3. The Planning Division should develop policies and procedures to ensure that decision-making is based on the results of the prioritization process. Specific controls could include: a. Creating a formal document or report of selection and prioritization process results. b. Comparing the prioritization results to programming decisions. c. Requiring documentation explaining the rationale for decisions that deviate from the prioritization process results (e.g., funding a lowscoring project ahead of higher scoring projects).

The Planning Division should utilize project scoring criteria that most effectively assess a project's need and potential impact.
The Planning Division's project scoring criteria are not consistent with best practices. The Planning Division has 13 scoring criteria related to congestion, safety, economic development, and other factors such as local support (See Appendix E for a complete description). Using defined criteria to score projects is considered a best practice. However, for the scoring process to be effective, the most meaningful criteria must be utilized. Many of the Planning Division's criteria are not well-aligned with long-term goals and are not outcome-focused, as discussed below. As a result, the scoring process may not effectively quantify potential benefits of projects and advance projects most suited to GDOT's goals.
Criteria and Goals Not Well-Aligned Best practice research emphasizes utilizing criteria that are linked to the full-spectrum of long-term goals to ensure that selected projects support priorities. Our review found that 9 of

Transportation Project Selection and Prioritization

21

the Planning Division's 13 project scoring criteria are explicitly linked to one of GDOT's long term goals (see Exhibit 11). The other four criteria include process related issues (e.g., approved concept report, identification in a planning study). In addition, the Planning Division has not established a criterion that supports its goal of improving the environment because the goal is not as applicable for non-transit projects, according to Planning Division management. Three states we interviewed (Ohio, Virginia, and Tennessee) use environmental impact measures such as reduction in emissions or fuel consumption to assess a project's potential to support environmental goals.
Exhibit 11 Goals and Scoring Criteria Do Not Always Align

Goals and Criteria Align

Statewide Transportation Goals Improve Safety
Relieve Congestion
Improve Freight/ Economic Development
Maintain & Preserve
Improve Reliability Improve Environment
Criteria Not Explicitly Linked to a Particular Goal

Project Selection Criteria
Crash Data
Level of Service
Connectivity: Bypass/Interchange Freight Network Freight Plan Governor s Road Improvement Program
Pavement Condition State Route Prioritization
Regional Traffic Operations Program
Goal Without Associated Criteria
Approved Concept Report (project readiness) Project Framework Agreement (local funding) Identification in a Planning Study Statewide Transportation Goal

Goals and Criteria Do Not Align

Source: PAD assessment based on agency documents

Criteria Not Outcome-Focused Best practices also suggest using outcomefocused criteria to most effectively evaluate potential impact. However, only 1 of 13 criteria (level of service) is outcome-focused. The remaining 12 are descriptive in nature and focused on existing conditions, project types, and completion of process steps rather than numeric measures demonstrating potential impact. For example, the freight network criterion is based on whether a project constitutes an improvement to a designated freight network corridor (yes or no) rather than a forecasted measure of the reduction in truck volume/capacity ratio as a result of the potential project. Likewise, the connectivity measure is based solely on project type (all interchange and bypass projects are awarded 30 points) rather than an assessment of how a project improves connectivity (e.g., travel time reduction). Examples of outcome-focused criteria used by other states or recommended in best practice reports include reduction in vehicle hours of travel, crash reductions, and jobs created (see Exhibit 12).

Transportation Project Selection and Prioritization

22

Exhibit 12 Examples of More Outcome-Focused Criteria

Congestion
Traffic volume to capacity ratio Reduction in vehicle hours of travel Reduction in person hours of delay Increase in person throughput

Economic Development/Freight
Estimated jobs created
Estimated gross state product
generated
Improved access to employment
centers
Truck volume to capacity ratio

Safety
Reduction in crashes Reduction in injuries Reduction in fatalities

Source: Review of best practice research and other states documents

In addition to these issues, several of the Planning Division's criteria are redundant. For example, a project on the freight network receives points both for the freight network criterion and the state route prioritization criterion since being on the freight network designates it as a "critical" route. Likewise, projects in the Freight Plan are likely to be located on the freight network. While certain criteria may be interrelated (e.g., reducing congestion could improve economic development), a project selection advisory council in Massachusetts concluded that redundancy can be reduced by using measures focused as much as possible on the specific criterion.
RECOMMENDATIONS
1. The Planning Division should utilize project scoring criteria that are wellaligned with long-term goals, outcome-focused, and non-duplicative.

Agency Response: In its response, GDOT indicated its rationale for the redundancy in criteria, stating that "per both federal and state laws, emphasis is placed on the movement of freight." GDOT also believes that "to comply with developing a robust freight network, projects on these corridors are scored appropriately."

Auditor's Response: While GDOT may wish to utilize a scoring methodology that emphasizes the movement of freight, this could be accomplished without using redundant criteria. For example, GDOT could utilize multiple criteria that evaluate different aspects of a project's potential to improve the movement of freight and/or could give additional weight to a freight-related criterion.

The Planning Division should incorporate benefit-cost analysis as part of its project selection process.
Although best practice research emphasizes the importance of a benefit-cost analysis, the Planning Division does not currently incorporate this aspect into its project selection process. The research supports benefit-cost analysis because it can help: 1) determine if a project is a justified investment; and 2) compare the relative value of projects for ranking/priority purposes. Planning Division management indicated that

Transportation Project Selection and Prioritization

23

it stopped conducting benefit-cost analysis in 2009 because of the difficulty in assigning monetary value to benefits and the uncertainty involved in cost estimates. As a result, there is less assurance that the Planning Division is selecting projects that maximize benefits while minimizing costs.
Benefit-cost analysis involves identifying, measuring, and valuing the benefits and costs of a project. As shown in Exhibit 13, the principal benefits for transportation investments include travel time savings, vehicle operating cost savings, and safety benefits. Typically, benefits are first estimated in physical terms and then valued in economic terms to compare to costs. For example, travel time savings can be assessed, using computer models, as the estimated reduction in vehicle hours traveled. The travel time savings can then be calculated in monetary terms using standardized costper-hour-per-person figures. Benefits are then compared to costs, which are the resources, such as land, labor, and material expended on the project. Costs can include both initial expenses (design, construction, etc.) and on-going expenses (operation, maintenance, etc.).
Exhibit 13 Benefit-Cost Analysis Monetizes Benefits and Compares to Costs

Benefit Examples
Travel time savings reduction in
vehicle hours traveled
Vehicle operating cost savings
reduction in vehicle miles traveled
Safety reduction in number and/
or severity of crashes

Cost Examples
Capital costs cost to implement
the project (engineering, right-ofway, construction, etc.)
Major rehabilitation costs - cost to
maintain the serviceability (e.g. pavement overlays)
Routine maintenance - future
operating and maintenance costs

Source: Review of best practice research and other states documents

Benefit-cost analysis is well-supported by industry research. According to the Transportation Research Board's Transportation Economics Committee, benefit-cost analysis has been widely used to evaluate transportation projects over the last few decades. A study conducted for the American Association of State Highway and Transportation Officials (AASHTO) also concluded that benefit-cost analysis is a key component when prioritizing capacity projects. In addition, FHWA guidance states that economic analysis enables state DOTs to target resources to the best uses and to account for their decisions. Lastly, the 2009 Cambridge Systematics report recommended prioritizing projects according to their benefit-cost ratio and performance ratings.
It should also be noted that the Governor's Congestion Mitigation Task Force, established in 2004 to generate strategies to reduce traffic congestion in metroAtlanta, recommended that GDOT, along with other partner agencies in the metroAtlanta area, establish a benefit-cost methodology to be applied to project selection.

Transportation Project Selection and Prioritization

24

We also found that other state DOTs use benefit-cost analyses or similar types of economic analysis to help prioritize and select projects. Virginia and North Carolina are both required by state law to analyze project benefits relative to project cost. North Carolina's benefit-cost calculation takes into account travel time savings and safety benefits over 10 years and the project cost to North Carolina at the time of submittal. If other funds are committed by locals, then the cost can be lowered and the score increased. Minnesota, Ohio, and Tennessee also conduct benefit-cost analyses or similar types of economic analyses.
RECOMMENDATIONS 1. The Planning Division should incorporate benefit-cost analysis as a criterion in its project selection process.
2. The General Assembly may wish to consider requiring benefit-cost analysis in statute to ensure that resources are maximized.

Agency Response: In its response, GDOT indicated that the "use of a benefit/cost (b/c) analysis often would not be representative of true project need; in urban areas the high cost of right-of-way may skew the b/c lower."
Auditor's Response: The recommendation to incorporate benefit-cost analysis into the project selection process does not mean that it is to be relied on exclusively. As noted earlier, "a mixed-approach is most useful, where benefit-cost analysis is one factor or criterion among several others" that could be more directly related to project need.

The Planning Division should revise its scoring methodology to better ensure that project scores accurately reflect the need for the project and the potential benefit.
The Planning Division's scoring methodology is further limiting the effectiveness of the project scoring process. First, the rationale behind the designation of key criteria and the points assigned to each criteria is unclear. Second, there is no point cap on the safety criterion, which inflates the total score for certain projects. Third, the all-ornothing point allocation for most criteria limits the ability to differentiate merit between similar projects. Lastly, the criteria are not weighted to reflect regional priorities, and scores are not scaled to address statistical issues, such as criteria having a disproportional impact on the project score. These points are discussed below.
Key Criteria Designation The rationale behind the scoring methodology and categorization of the various scoring criteria is unclear. As shown in Exhibit 14, the Planning Division has designated 5 of its 13 scoring criteria as "key criteria". However, three of the five "key criteria" have a maximum point potential of 10 points or less. In comparison, safety has no defined point limit and therefore the highest potential points (e.g., one project had more than 400 safety points), yet is not designated as a "key criteria." In addition, the level of service (congestion) criterion is not designated as a key criteria but accounts for up to 28 points, which is more than the maximum possible points for four of the five key criteria. Furthermore, it is unclear why level of service

Transportation Project Selection and Prioritization

25

is not a key criterion, since congestion reduction is the primary purpose of capacity projects.

Exhibit 14 Project Scores Highlight Issues with Scoring Methodology

Criteria

All or Nothing Maximum

Point

Possible

Application? Points

Key Criteria

Points Awarded Per Project(1)

% of Projects

Min

Max Average Median with Score of 0

Connectivity: Bypass/Interchange

Yes

30

0

30

7.50

0

75%

State Route Prioritization

No

18

0

18

10.64

9

14%

Freight Network

Yes

10

0

10

3.36

0

66%

Freight Plan

No

8

0

8

0.38

0

94%

Governor s Road Improvement Program

Yes

5

0

5

0.37

0

93%

Additional Criteria

Safety: Crash Data

No

No limit

0

413 13.24

6

3%

Congestion: Level of Service

No

28

0

26

7.91

0

60%

Pavement Condition

Yes

5

0

5

0.95

0

81%

Planning Study

Yes

5

0

5

1.03

0

79%

Local Support: Project Framework Agreement

Yes

1

0

1

0.32

0

68%

Project Readiness: Approved Concept Report

Yes

1

0

1

0.31

0

69%

SSTP Goal

Yes

1

1

1

1.00

1

0%

Regional Traffic Operations Program

Yes

1

0

1

0.02

0

98%

(1) Includes capacity projects programmed between fiscal years 2016 and 2026 that were fully scored. Source: GDOT's prioritization spreadsheet

Infinite Point Potential and Safety Criterion Points As noted above, there is no set point range for a project because there is no cap on the number of safety points. This is because the criterion is not outcome-focused, as noted earlier, but based on the number of crashes that occurred on the route to be addressed by the project. However, the lack of a cap on safety points inflates the total project score for certain projects and results in a wide range of total project scores. While the median project score is 41, the scores ranged from 2 to 448. We identified seven managed lane projects that received over 100 points for just the safety criterion, with one project receiving 413 safety points (see the text box on managed lane projects). Other states we interviewed (North Carolina, Virginia, Ohio, and Tennessee) have a set point range such as 1-100 or have a method for redistributing scores on a scale of 1 to 100 after scoring. This allows for a simpler comparison among projects and an easier assessment of which criteria are driving the score within each project.

Transportation Project Selection and Prioritization

26

Managed Lane Projects: Safety Points Inflate Project Scores
Managed lane projects include work related to high-occupancy vehicle (HOV) lanes, high occupancy toll lanes, value priced lanes, or exclusive/special use lanes. GDOT has seven managed lane projects programmed that received scores above 190, almost five times the median project score of 41. The project scores were driven up due to safety points. The safety points awarded per project ranged from 159 to 413 and accounted for 80%-92% of the total project scores. In comparison, the median safety points for all capacity projects is six. Since the safety points awarded are based on existing crash data, the high number of safety points for the seven managed lane projects is more likely reflective of the projects' size and location in metro-Atlanta than the projects' potential to improve safety. As noted earlier, the safety criterion is not outcome-focused, so the extent to which the managed lane projects will reduce crashes is unknown.

Project 1 Project 2 Project 3 Project 4 Project 5 Project 6 Project 7

Total Score
412 255 239 234 193 448 203

Safety Points
378 231 205 200 159 413 162

Safety % of Total Score
92% 91% 86% 86% 82% 92% 80%

All or Nothing Point Application The scoring methodology does not always differentiate the project's degree of merit in a particular scoring criteria. As shown in Exhibit 14, 9 of the 13 criteria are scored in an all-ornothing manner, meaning a project can either receive zero points or all the possible points but nothing in between. As a result, the ability to differentiate merit between projects of similar type (e.g., interchange projects) is compromised. In addition, there is a high prevalence of zero scores. Of the 13 criteria, 10 criteria have more zero scores than non-zero scores, and six criteria have 75% or more of projects with zero scores.
Weighting and Scaling The Planning Division does not weight criteria according to project location to account for varying geographic priorities. Rather, the Planning Division scores all projects the same regardless of location even though congestion is likely a higher priority in metro-Atlanta compared to rural Georgia, where safety or economic development may be a relatively greater concern. In addition, the Planning Division does not utilize scaling, a statistical method of redistributing scores that is essentially grading on a curve. Scaling is used to account for statistical issues including a small range of values, low values, and criteria having a disproportional impact on the total score. North Carolina and Virginia both weight criteria to reflect geographic differences and scale project scores.
RECOMMENDATIONS
1. The Planning Division should re-evaluate its scoring methodology. Specific improvements could include:

Transportation Project Selection and Prioritization

27

a. Modifying its designation of key criteria and assigning potential point values that reflect the relative importance of each criteria;
b. Capping the number of safety points and establishing a total point range (i.e., 1-100);
c. Scoring each criteria on a continuum (when feasible) rather than an all or nothing basis to better differentiate merit;
d. Weighting criteria according to project location to account for differences in regional priorities; and
e. Scaling project scores to account for potential statistical issues.

GDOT, the Planning Division, and the General Assembly should explore alternative methods for considering regional needs.

O.C.G.A. 32-5-30, requires GDOT to balance 80% of
federal and state transportation funding equally across the 14 congressional
districts over a five-year

The congressional balancing requirement may result in projects being advanced or delayed based on their location rather than greatest need. Congressional balancing was implemented to ensure all areas of the state receive their fair share of funding. Best practice research emphasizes the importance of a performance-based process of allocating resources according to

period.

need rather than location. While congressional balancing does

not necessarily preclude a performance-based process, it does

present constraints as GDOT management has to shift priorities to ensure the

requirement is met. The requirement applies to all project types (capacity,

maintenance, bridge, etc.), but GDOT management indicated that capacity projects

are generally impacted by congressional balancing more than other project types.

GDOT management indicated that congressional balancing does not influence whether or not a project is selected, but both GDOT and MPO officials indicated that the requirement results in capacity projects being re-prioritized. Meaning, a higher priority project may be delayed because its cost may not fit within the confines of the balancing amount; thus, another lower priority, less costly project may be advanced. According to Planning Division management, funding decisions are initially based on factors including the project's score, readiness, public support, and momentum. The distribution of funding among congressional districts is then analyzed, and projects are shifted accordingly. While the number of projects affected is difficult to quantify due to limited documentation, GDOT management indicated that the impact is considerable. In addition, our survey of MPO officials found that 10 of 16 officials (63%) perceive the congressional balancing requirement to be a moderate or major impact on the project selection and prioritization process. One MPO official noted that projects that are not current MPO priorities are added to the Transportation Improvement Program (TIP) so districts can spend their allocated amount.

Currently, funding for the calendar year 2013-2017 balancing period is not evenly distributed; consequently, GDOT will need to fund more projects in certain districts in 2017 to ensure compliance. As of November 2016, expenditures for the balancing period, excluding exempted projects, totaled $4.68 billion. Of this amount, $3.74 billion (80%) is subject to congressional balancing. If the $3.74 billion is divided equally among the 14 congressional districts, each district would receive $267 million. However, agency records indicate that the Fourth Congressional District has only received $160 million and the Sixth Congressional District has only received $264 million thus far (see Exhibit 15). This means that in calendar year 2017, GDOT will

Transportation Project Selection and Prioritization

28

have to fund at least $110 million17 worth of projects across these two districts to comply with the congressional balancing requirement, which will likely result in lower priority projects being advanced over higher priority projects.18
Exhibit 15 Funding Shifts Needed to Ensure Compliance with Congressional Balancing Requirement for Calendar Year 2013- 2017

Dade

Catoosa

Walker

Whitfield Murray

Fannin

Union

Gilmer

District 4 ($160 million) and
District 6 ($264 million) will need
a higher proportion of funding in 2017 for the balancing requirement to
be met.

14

Chattooga

Gordon

Pickens

Floyd

11 Bartow

Cherokee

Lumpkin Dawson Forsyth

Polk Haralson

67

Paulding

Cobb

Gwinnett

Douglas

5
Fulton

DeKalb
4

Towns

Rabun

White
9

Habersham Stephens

Banks

Franklin

Hart

Hall

Jackson

Madison

Elbert

Barrow

Clarke

Walton

Oconee

Oglethorpe

Wilkes

Lincoln

Less Than $200 Million $200-299 Million $300-350 Million Over $350 Million

Rockdale

Carroll Heard
Troup

13

Clayton

Fayette Coweta

Henry

3
Meriwether

Spalding

Pike

Lamar

Newton Butts Monroe

Morgan

Greene Taliaferro
10

Columbia McDuffie

Jasper

Putnam

Warren

Hancock

Glascock

Richmond

Jones

Baldwin

Washington

Jefferson

Burke

Harris

Talbot

Muscogee Marion
Chattahoochee

Upson

Crawford

Bibb

Wilkinson Twiggs

Taylor

Peach

Houston

Macon

Bleckley

Laurens

Johnson

Jenkins Emanuel

12

Treutlen

Candler

Screven

Bulloch

Effingham

Montgomery

Schley

Stewart

Webster

Sumter

Quitman Randolph

2
Terrell

Lee

Clay

Calhoun

Dougherty

Early Miller

Baker

Mitchell

Dooly

Pulaski

Crisp

Wilcox

Dodge

Wheeler

Telfair

Toombs

Evans Tattnall

Worth

Turner
8
Tift

Ben Hill Irwin

Colquitt

Berrien Cook

Jeff Davis

Appling

Long

Coffee Atkinson

Bacon Pierce

Wayne
1

Ware

Brantley

Bryan Liberty

Chatham

McIntosh Glynn

Seminole

Decatur

Grady

Thomas

Brooks

Lanier Lowndes

Clinch

Charlton

Camden

Echols

Source: Agency documents

17 Current amount based on expenditures as of November 2016. This amount will increase as expenditures increase throughout the remainder of the balancing period.
18 As previously noted, the Board can vote to waive the requirement in these areas if the balancing requirement cannot realistically be met.

Transportation Project Selection and Prioritization

29

Georgia's congressional balancing requirement does not align with other states' funding methods. We found that other states have methods of considering geographic equity without having a statutory requirement to balance funds by congressional district. For example, an Ohio DOT official indicated that there is no statutory congressional balancing requirement but the DOT does consider regional equity in the distribution of funding and takes into account population, vehicle miles traveled, and lane miles.

RECOMMENDATIONS 1. GDOT and the Planning Division should begin regularly tracking the impact of congressional balancing on project selection and prioritization decisions, such as generating a pre- and post-balancing prioritized list of projects and comparing the results to determine the extent to which priorities shift. Collecting and reporting this information would be helpful in evaluating current requirements and possible alternatives. Alternatives could include:
a. Changing the geographic boundaries to take into account factors such as population, vehicle miles traveled, and lane miles instead of focusing on congressional districts. Congressional district boundaries can shift at any time and, subsequently, planned projects may shift from one district to another. Implementing this recommendation would require a change in state law.
b. Adding geographic-based criteria into the prioritization process.
2. GDOT and the Planning Division should regularly report on the impacts of the congressional balancing requirement to the Board and the General Assembly for further study.
Process Improvements

The Planning Division should streamline and automate the project selection and prioritization process and ensure that all relevant information is tracked and accessible. As part of this effort, the Planning Division should consider the need for decision-making software or tools with greater functionality.

Programmed projects can be scheduled short-term or long-
term (up to 35 years out).

The Planning Division lacks a comprehensive, automated system that tracks all project proposals through each step of the project selection and prioritization process and allows decisions to be documented along the way. As a result, there is less accountability in the process, a risk for inconsistency in decision-making, and inadequate information about how project selection, programming, and prioritization decisions are made.
Currently, the Planning Division does not maintain comprehensive information on proposed projects from inception to completion. For example, the Planning Division cannot report for a given time period the total number of project proposals received and the number/portion of those projects that were initially rejected, rejected after the project review, programmed for the long-term, or programmed in the STIP. Furthermore, it is difficult to track the life cycle of a particular project. While the Planning Division's prioritization spreadsheet is used to prioritize projects for funding, it cannot be used to determine when a project was first proposed, when it

Transportation Project Selection and Prioritization

30

was initially selected and programmed for funding, or when it was scored.19 All changes in project status are also not clearly documented. As a result, it is difficult to determine the extent to which projects have been delayed or advanced or the reasons for the changes. Specific gaps in each step of the process are outlined below and shown in Exhibit 16.

1. Project Identification Potential projects may be requested from numerous sources (local governments, state officials, the general public, etc.), but there is no standard or centralized process for submitting project proposals. Some of these proposals are immediately rejected. Because these projects are not entered into the data, there is no way to easily identify the total number of project proposals, the number of proposals immediately declined, or the reasons that these projects are declined.
2. Project Review Projects that are not initially declined are subject to a preliminary review and study process. Projects that are approved (i.e., programmed) are then entered into the Planning Division's prioritization spreadsheet. Projects that are rejected are not entered into the spreadsheet; consequently, there is no easily accessible tracking of these projects for reporting or review purposes.
3. Project Scoring Projects in the prioritization spreadsheet are scored through a manual data collection process. Planning staff gather the required information for a project, and the data is input into the spreadsheet tool. Once the data is inputted, the spreadsheet automatically calculates the project score for most criteria. As discussed in the finding on page 17, we found that data controls are lacking, resulting in incomplete scores and incomplete supporting data.
4. STIP Development Currently, the prioritization spreadsheet includes 676 capacity projects programmed between fiscal years 2016 and 2051. Although project scores are tracked, the current system does not generate a clear project priority ranking list or grouping that designates relative importance and serves as the basis for decision-making. Instead, Planning Division staff discusses projects one-by-one and matches projects to available funding while ensuring congressional balancing requirements are met. In addition to project scores, less quantifiable factors such as project readiness, political support, and funding restrictions can influence decisions. For example, Planning Division management indicated that a low scoring project may be implemented before higher scoring projects because it is the only project eligible for a particular funding source. Aside from the project scores, the factors that influence the decision to include a project in the STIP are not documented in the prioritization spreadsheet.
5. Implementation Projects are removed from the prioritization spreadsheet after funding for construction is authorized or the project is cancelled, so that only projects currently under consideration remain. This limits the Planning Division's ability to conduct a historical review to compare project scoring and implementation order. In addition, Planning Division management

19 Project information is also maintained in GDOT's project tracking system (TPRO), but the system only includes projects programmed for inclusion in the STIP or programmed as long-range (5-35 years out). It does not include project proposals declined or rejected at the outset.

Transportation Project Selection and Prioritization

31

reported that there are some projects in the prioritization spreadsheet in which the Division has no intentions of implementing but has not removed because the project has not been formally cancelled. However, there is no clear indication of this status in the data.
Exhibit 16 Gaps Exist in the Planning Division's Prioritization Tool

No centralized process for receiving project proposals
No tracking of projects immediately rejected.

Manual data entry process
Lack of data controls

No tracking of projects after authorization
Project status can be unclear

Identification

Initial Review

Scoring

STIP Development

Implementation

No tracking of projects that did not pass through initial review
For projects selected, no indication of the date the project was selected

No clear project ranking list or tiers
Rationale for decision-making not documented

Source: PADs review of GDOT s prioritization spreadsheet and interviews with GDOT staff

A comprehensive prioritization system was recommended in the 2009 Cambridge Systematics report to support accountability and consistency in decision-making and transparency to decision-makers and other stakeholders. Specifically, the report recommended a system that would include: a central location for compiling, reviewing, and updating potential projects; a project screening component for listing criteria and notifying project sponsors of the status; and a programming component for matching funding sources to specific projects and developing a multi-year program.

We found that more advanced prioritization systems have been implemented by other states including North Carolina, Ohio, Tennessee, and Virginia. Generally, these tools help process project proposals, generate prioritized lists or tiers, and produce summary reports with project data and scoring information. For example, North Carolina has "SPOT On!ine", an ARC-GIS web-based software that allows project sponsors to enter project information and relevant data. Virginia produces a project scorecard that provides a project description, cost data, the project score (total and for each criteria), the statewide rank, and the district rank.

RECOMMENDATIONS
1. The Planning Division should streamline and automate the project selection and prioritization process for capacity projects. Specific improvements could include: a. Creating an online application that would allow project sponsors to submit and update project information and track the progress of the project to improve transparency;

Transportation Project Selection and Prioritization

32

b. Centrally compiling and maintaining results from the initial screening process;
c. Automating the input of performance data when feasible for the project scoring process;
d. Adding functions for generating a prioritized list and matching funding sources to specific projects; and
e. Improving reporting capabilities for individual projects and comparisons among projects.
2. The Planning Division should evaluate the extent to which additional software or analytical tools are needed in order to streamline and automate the process.
The Planning Division should establish more specific policies and procedures to guide the project selection and prioritization process.
The Planning Division lacks a comprehensive set of procedures to guide staff through project selection from beginning to end. Currently, the Planning Division has a Planning Manual, but it does not address critical steps, such as project scoring. GDOT also has several standard operating procedures that describe basic planning requirements; however, these also fail to detail key components of the project selection and prioritization process. As a result, there is no assurance that projects are evaluated for selection and scored objectively and consistently. Specific issues are outlined below and shown in Exhibit 17.
Exhibit 17 Policies and Procedures Do Not Encompass All the Key Aspects of the Project Selection Process

Project Identification

GDOT Has:

GDOT Lacks:

Planning Manual indicates that anyone can submit a project request to various entities including GDOT, city & county governments, and MPOs

Written procedures for submitting requests and compiling, tracking, and screening requests

Project Review & Selection

GDOT Has:
Written procedures for conducting the initial review (i.e., project recommendation study)

GDOT Lacks:
Formal criteria for deciding whether or not to select the project

Project Scoring

GDOT Has:
Project scoring criteria and point system

GDOT Lacks:
Detailed procedures for how the scoring should be executed

STIP Development

GDOT Has:
Standard operating procedures for STIP development (meetings, correspondences, etc.)

GDOT Lacks:
Formal procedures or criteria to guide the decision-making process

Source: Agency Documents

Transportation Project Selection and Prioritization

33

Project Identification The Planning Manual indicates that anyone can propose a project and that the request can be submitted to a variety of entities including GDOT, city and county governments, and MPOs. The Planning Division does not have detailed guidelines for how project requests should be compiled and tracked or screened for feasibility. In addition, the Planning Division lacks specific policies regarding the type of information and data that should be submitted by the project requestor to ensure consistency in how project proposals are processed.
Project Review and Selection The Planning Manual prescribes the steps for conducting an initial project review, which is referred to as the project recommendation planning study. The Manual also stipulates that Planning Division management is responsible for reviewing the recommendation and deciding whether or not to select the project. However, project recommendation planning studies are not routinely produced and there are no specific project selection criteria.
Project Scoring As previously discussed, the Planning Division has a project scoring system for prioritizing the order in which selected projects will be funded. However, neither the Manual nor the standard operating procedures contain specific guidance for executing the scoring process. For example, there is no indication of which types of projects should be scored, when the scoring should occur, when scores need to be updated, and if/how exceptions to the scoring process should be handled.
STIP Development GDOT has standard operating procedures for STIP development that outline routine steps such as required meetings and correspondences. However, there are no formal procedures or criteria to guide the decision-making process and inform how various factors (e.g., scores, project readiness, local support/resistance, congressional balancing, etc.) should be taken into account.
Other states have more clearly defined procedures that may help minimize the risk of projects being selected based on subjective factors. For example, Ohio's Director of Transportation is required by state law to develop written guidelines for selecting major new capacity projects that include: a description of how strategic initiatives are advanced by the process; the kinds of projects to which the process applies; criteria used to rank proposed projects; and data that is necessary to apply the ranking criteria. Virginia's DOT has both a policy guide that provides an overall framework for the project selection process and a technical guide that includes detailed procedures for each step in the process. For example, the technical guide stipulates that a project that has been selected for funding must be re-scored and re-evaluated if there are significant changes to scope or costs, with specific thresholds provided.
RECOMMENDATION
1. The Planning Division should develop and/or update policies and procedures to guide all aspects of the project selection and prioritization process. Specifically, the policies and procedures should better address the processes for: proposing projects; compiling, tracking, and screening potential projects; programming projects; scoring projects; and

Transportation Project Selection and Prioritization

34

selecting projects to include in the STIP.
Agency Response: GDOT indicated that the Planning Manual "has been around for many years and is continuously being updated." GDOT further indicated that the Planning Manual is not for the purpose of project scoring or selecting projects. Rather, it is an internal document that serves as a resource for transportation planners.
Auditor's Response: If the Planning Manual is not intended for the purpose of project scoring or selecting projects, then GDOT should develop separate guidelines for these processes.

Transparency
The Planning Division should work with MPOs to clarify its level of input and assistance in the development of the TIPs.
Federal regulations assign MPOs primary responsibility for developing the TIP for their areas, but it is a cooperative process between the MPOs and state DOTs (and public transit operators). The majority of MPOs we surveyed indicated that they are satisfied with their level of influence in project selection decisions. However, several MPOs noted concerns regarding the degree of input into the process and need for greater coordination with GDOT. Our review found that the roles and responsibilities of the Planning Division in MPOs project selection processes are not clearly defined in the Planning Manual or other documents.
In interviews, Planning Division management indicated that the role of the MPO in the TIP development process can vary, as some MPOs have greater resources and expertise. The responses from our MPO survey also indicated that the MPO's role in the project selection process varies. We asked the 16 MPOs which entity (GDOT or the MPO) is primarily responsible for selecting projects. As shown in Exhibit 18, nine respondents indicated that GDOT selects projects, three respondents indicated that the MPO and GDOT have equal input in selecting projects, and three respondents indicated that the MPO selects projects.20 In addition, just over half of respondents (9 of 16) agreed or strongly agreed with the statement that the roles/responsibilities of MPOs and GDOT are clearly defined. In the comments section of the survey, several MPO officials cited issues with the project selection process and their relationship with GDOT. For example, one MPO official noted it would help if GDOT and the MPO staff met to discuss projects at the beginning of the Long Range Transportation Plan and TIP development processes. Several other MPO officials noted similar concerns with coordination and communication between GDOT and the MPO.

20 One respondent indicated "other"

Transportation Project Selection and Prioritization

35

Exhibit 18 The MPO Role in Project Selection Varies

GDOT Selects Projects

GDOT & MPO Select Projects Together

MPO Selects Projects

3

6

GDOT selects projects with little or no input from MPO

GDOT selects projects with moderate or
significant input from MPO

(1) One respondent indicated other Source: PADs survey of MPO officials

3

3

GDOT & MPO selects projects together with
equal input

MPO selects projects with moderate or
significant input from GDOT

Although the majority of the MPOs appear to be satisfied with their level of influence in project selection and GDOT's collaboration efforts, several MPO officials indicated that greater local input is needed. As shown in Exhibit 19, when asked if the MPO has an appropriate level of influence in prioritizing and selecting projects, 11 of the 16 (69%) respondents agreed or strongly agreed. Similarly, 12 of 16 (75%) respondents agreed or strongly agreed that GDOT collaborates effectively with the MPO. However, several MPO officials indicated concerns. For example, one MPO official noted that "since the locals have limited resources, there's a sense that GDOT basically sets the agenda since they control the bulk of funds...Those that hold the gold, set the rules." Another MPO official noted that "it would also help if GDOT had an open mind when it comes to the type of projects that would best work in the MPO area. The locals understand their area and operational improvements might help transportation problems better than widening a road."
Exhibit 19 Majority of MPOs Satisfied with Level of Influence and GDOT's Collaboration Efforts

MPOs have an

appropriate level

4

1

10

of influence.

GDOT effectively

collaborates with

1 1 22

9

MPO staff.

Strongly Disagree

Disagree

Neutral

Agree

Strongly Agree

1 3

Source: PAD s survey of MPO officials

Transportation Project Selection and Prioritization

36

To better communicate MPO roles and ensure sufficient input, other states have developed specific guidelines for MPO staff. These guidelines are intended to clarify the roles and responsibilities of stakeholders, improve efficiency among organizations, and reduce questions and potential conflicts between MPOs and DOTs. For example, Arizona DOT's MPO & COG21 Guidelines & Procedures Manual describes the planning process, including procedures for TIP amendments, guidelines for public participation, and a Regional Transportation Plan checklist. Arizona DOT's manual also provides examples of best practices as well as contact information for each topic so MPOs can work directly with each other for guidance or insight. Ohio DOT's MPO Administration Manual outlines the basic administrative requirements for ODOT and the MPOs and includes a TIP checklist and a planning schedule with the responsible party for each milestone. Maine, Nebraska, and New Mexico DOTs also produce some form of guidelines or manual for MPO and DOT personnel to use during the process.

RECOMMENDATION 1. The Planning Division should work with MPOs to understand their concerns about the level of input and identify solutions. For example, the Planning Division could consider conducting additional planning meetings with those MPOs that may lack staff and resources but still have valuable input due to their knowledge of the local area. In addition, the Planning Division could develop and distribute guidelines for coordination with MPOs to ensure that roles and responsibilities are clearly communicated.
Agency Response: GDOT noted that "funding for a project cannot be authorized in an MPO area unless the project appears in the MPO TIP, which is approved by both the MPO Technical Coordinating Committee and the Policy Committee." GDOT also noted that meetings are conducted with all MPOs and that communication between GDOT and the MPOs is "continuous and ongoing."
Auditor's Response: It is true that MPOs are responsible for developing and submitting their TIPs. Based on the survey results, however, it appears that some MPOs expressed concern about the Planning Division's level of involvement in the TIP creation. We are suggesting that the Planning Division increase communication to understand the reasons for these concerns.

The Planning Division should better communicate its overall project selection process and its criteria and scoring methodology. In addition, the Planning Division should improve its communication on project selection decisions.
The Planning Division does not effectively communicate its overall project selection process, evaluation criteria, scoring methodology, or rationale for selecting, or not selecting, specific projects. Detailed information related to the project selection process and scoring methodology is not made available on GDOT's website or publicly available documents, such as the STIP. While project information is publicly available on GDOT's www.GaRoads.org website (e.g., project schedules, phases by fiscal year, GIS information), the Planning Division does not routinely publish information related to how projects were evaluated and scored or the rationale behind specific project selection decisions, nor does it provide information on projects that were not
21 Council of Governments

Transportation Project Selection and Prioritization

37

selected. Additionally, projects in the STIP are not listed with a priority ranking or with their overall score. Lastly, state-funded projects may be programmed and implemented without being included in the STIP.
Due to the lack of communication about project selection decisions, the Planning Division's project selection process and rationale for decision-making are not always transparent to stakeholders, including MPOs. For example, of the 16 MPO officials we surveyed, seven respondents (44%) strongly disagreed or disagreed with the statement that GDOT's criteria and scoring methodology are transparent (See Exhibit 20). Likewise, seven respondents (44%) strongly disagreed or disagreed that the overall process is transparent. One MPO official noted that they know nothing about state-supported projects until they receive an email from the GDOT planner informing them that the MPO's long-range plan and TIP need to be amended.
Exhibit 20 Nearly Half of MPO Officials Indicated Transparency Issues

GDOT s criteria and

scoring methodology

1

6

is transparent(1)

3

3

GDOT s overall process is transparent.
GDOT effectively communicates reasons for NOT selecting specific projects
GDOT effectively communicates reasons for selecting specific projects

2 3
2

GDOT effectively

communicates reasons

11

3

for amending the TIP

55

33

6

5

4 6 8

5

7

Strongly Disagree

Disagree

Neutral

Strongly

Agree

Agree

2 1 1

(1) Three respondents (19%) indicated that they did not know or were unsure. Source: PAD s survey of MPO officials
We identified four other states that have made information more transparent to stakeholders. These states provide information regarding the overall project selection process and disclose more project-specific data. Three of the four states have statutory requirements to make certain information publicly available. In addition, all four states include or intend to include state-funded projects in the STIP.22 Specific examples are described below.
Virginia (VDOT) promotes transparency and accountability to stakeholders through its Smart Scale website that is dedicated to project selection and includes frequently asked questions about the process, information on "how to read a project scorecard," and a link to its policy guide that describes the criteria and scoring methodology in detail. The website also

22 One state (South Carolina) indicated that past practice was to include state-funded projects in the STIP that were regionally significant or required FHWA action. However, a transportation funding bill has generated additional state funding for projects and the intention is to include these projects in the STIP regardless of funding source.

Transportation Project Selection and Prioritization

38

publishes a list of projects that did not pass the initial screening process, with rationale, and a spreadsheet containing all project scores and ranks, including individual scores for each measure, total project score, and district and statewide ranks.
North Carolina (NCDOT) communicates its overall project selection process and the methodology and scoring criteria used to evaluate each project on its website. The website provides transportation stakeholders access to all project data, scoring, and final selection decisions and publishes a list of prioritized projects. NCDOT also publishes the project scorings associated with the draft and final STIP.
Ohio (ODOT) publishes a list of projects broken into three tiers and the project applications it received on its website. The website also includes the scoring factors, criteria and measures used for each factor, the data source, and the total points for each criteria. Furthermore, the website contains links to the policies and procedures manual used for selecting major new capacity projects. ODOT also reviews applications at public hearings, provides opportunity for sponsor participation in the application review, and allows sponsor feedback subsequent to project scoring.
South Carolina (SCDOT) creates a priority list of projects based on a statutorily created set of nine criteria and publishes the project priority list for each program area and the methodology for the application of those criteria on its website. The published priority lists contain the overall rank, location, and current status of all projects. Additionally, SCDOT's STIP includes the project ranking and program area for selected projects.
It should also be noted that GDOT makes a significant number of STIP amendments (24 to-date in the FY 2015-18 STIP) without providing the rationale for adding, removing, or modifying projects. This does not appear to be a major concern for the MPOs, as most believe GDOT effectively communicates reasons behind the amendments. However, we did identify one state, Missouri (MODOT), that publishes STIP amendments on its website along with its rationale for the amendments (e.g., to add a scoping project, to add right of way, to modify existing project budget). MODOT also publishes a list of reprioritized projects from the previous construction schedule, including the projects that were delayed, rescheduled, or removed with an explanation for each project.
RECOMMENDATIONS
1. The Planning Division should better communicate its project selection process and scoring methodology. For example, the Division could provide a description of the project selection process and scoring methodology on the GDOT website. The Planning Division should also ensure that this information is clearly communicated during public hearings and meetings with MPO officials.
2. The Planning Division should communicate more information regarding its rationale for selecting, or not selecting, specific projects as well as the rationale for adding or removing projects from the approved STIP.
3. In addition, to ensure transparency in the selection of 100% state-funded projects, the Planning Division should include a listing of such projects as part

Transportation Project Selection and Prioritization

39

of the Statewide Strategic Transportation Plan (SSTP). As required by O.C.G.A. 32-2-41.1, the plan is to include a list of projects realistically expected to begin construction within the next four years and should include the cost and sources of funds for each project.
4. To better ensure transparency, the General Assembly may wish to consider establishing a statutory requirement for the Planning Division to publicly disclose more information regarding its process and project selection decisions.

Agency Response: In its response, GDOT indicated that "projects are listed in the STIP with a priority ranking as demonstrated by the year they show funding." In regards to not including statefunded projects in the STIP, GDOT noted that these projects are not required to be in the STIP and that FHWA staff recommended against including these projects in the STIP. In regards to including state-funded projects as part of the SSTP, GDOT indicated that the code section does not stipulate a four-year listing of 100% state-funded projects; rather, it stipulates a four-year listing of projects. GDOT asserts that the STIP satisfies the requirement for a four-year listing of projects.
Auditor's Response: If 100% state-funded projects are not included in the STIP or SSTP, then there is not a complete four-year listing of projects.

Transportation Project Selection and Prioritization

40

Appendix A: Table of Recommendations
The Planning Division should revise its process to ensure projects are formally evaluated against a set of standard criteria before they are selected and programmed (p. 15).
1. The Planning Division should formalize its initial project review process and ensure that studies are conducted as prescribed in the Planning Manual.
2. The Planning Division should develop a process for screening project proposals either by 1) scoring projects prior to programming so the results can more objectively be compared and inform selection decisions or 2) establishing separate screening criteria for use in evaluating projects prior to programming.
The Planning Division should establish controls to ensure projects are consistently evaluated against scoring criteria. The Planning Division should also prioritize projects according to their scores to help inform decisions about which projects to select and program (p. 17).
3. The Planning Division should ensure that all capacity and economic development projects are scored or objectively evaluated. Specifically, the Planning Division could: a. Develop a more formal, objective process to evaluate economic development projects. This could be accomplished through a revision of existing scoring criteria to embody economic evaluation of all projects or development of a separate evaluation process tailored to capacity projects. b. Develop an alternative evaluation process for all projects that do not fit well within the standard criteria.
4. The Planning Division should establish data controls to ensure accuracy and completeness of project scoring data. These could include: a. Indication of scoring status (e.g., not yet scored, partially scored, scoring complete) b. Use of data fields that require a response (e.g., a menu to select yes/no answer). c. Ensuring fields that calculate scores do not default to 0 in absence of data.
5. The Planning Division should develop policies and procedures to ensure that decision-making is based on the results of the prioritization process. Specific controls could include: a. Creating a formal document or report of selection and prioritization process results. b. Comparing the prioritization results to programming decisions. c. Requiring documentation explaining the rationale for decisions that deviate from the prioritization process results (e.g., funding a low-scoring project ahead of higher scoring projects).
The Planning Division should utilize project scoring criteria that most effectively assess a project's need and potential impact (p. 20).
6. The Planning Division should utilize project scoring criteria that are well-aligned with long-term goals, outcomefocused, and non-duplicative.
The Planning Division should incorporate benefit-cost analysis as part of its project selection process (p. 22).
7. The Planning Division should incorporate benefit-cost analysis as a criterion in its project selection process.
8. The General Assembly may wish to consider requiring benefit-cost analysis in statute to ensure that resources are maximized.
The Planning Division should revise its scoring methodology to better ensure that project scores accurately reflect the need for the project and the potential benefit (p. 24).
9. The Planning Division should re-evaluate its scoring methodology. Specific improvements could include: a. modifying its designation of key criteria and assigning potential point values that reflect the relative importance of each criteria; b. capping the number of safety points and establishing a total point range (i.e., 1-100); c. scoring each criteria on a continuum (when feasible) rather than an all or nothing basis to better differentiate merit; d. weighting criteria according to project location to account for differences in regional priorities; and e. scaling project scores to account for potential statistical issues.
GDOT, the Planning Division, and the General Assembly should explore alternative methods for considering regional needs (p. 27).

Transportation Project Selection and Prioritization

41

10. GDOT and the Planning Division should begin regularly tracking the impact of congressional balancing on project selection and prioritization decisions, such as by generating a pre-and post-balancing prioritized list of projects and comparing the results to determine the extent to which priorities shift. Collecting and reporting this information would be helpful in evaluating current requirements and possible alternatives. Alternatives could include: a. Changing the geographic boundaries to take into account factors such as population, vehicle miles traveled, and lane miles instead of focusing on congressional districts. Congressional district boundaries can shift at any time and, subsequently, planned projects may shift from one district to another. Implementing this recommendation would require a change in state law. b. Adding geographic-based criteria into the prioritization process.
11. GDOT and the Planning Division should regularly report on the impacts of the congressional balancing requirement to the Board and the General Assembly for further study.
The Planning Division should streamline and automate the project selection and prioritization process and ensure that all relevant information is tracked and accessible. As part of this effort, the Planning Division should consider the need for decision-making software or tools with greater functionality (p. 29).
12. The Planning Division should streamline and automate the project selection and prioritization process for capacity projects. Specific improvements could include: a. Creating an online application that would allow project sponsors to submit and update project information and track the progress of the project to improve transparency; b. Centrally compiling and maintaining results from the initial screening process; c. Automating the input of performance data when feasible for the project scoring process; d. Adding functions for generating a prioritized list and matching funding sources to specific projects; and e. Improving reporting capabilities for individual projects and comparisons among projects.
13. The Planning Division should evaluate the extent to which additional software or analytical tools are needed in order to streamline and automate the process.
The Planning Division should establish more specific policies and procedures to guide the project selection and prioritization process (p. 32).
14. The Planning Division should develop and/or update policies and procedures to guide all aspects of the project selection and prioritization process. Specifically, the policies and procedures should better address the processes for: proposing projects; compiling, tracking, and screening potential projects; programming projects; scoring projects; and selecting projects to include in the STIP.
The Planning Division should work with MPOs to clarify its level of input and assistance in the development of the TIPs (p. 34).
15. The Planning Division should work with MPOs to understand their concerns about the level of input and identify solutions. For example, the Planning Division could consider conducting additional planning meetings with those MPOs that may lack staff and resources but still have valuable input due to their knowledge of the local area. In addition, the Planning Division could develop and distribute guidelines for coordination with MPOs to ensure that roles and responsibilities are clearly communicated.
The Planning Division should better communicate its overall project selection process and its criteria and scoring methodology. In addition, the Planning Division should improve its communication on project selection decisions (p. 36).
16. The Planning Division should better communicate its project selection process and scoring methodology. For example, the Planning Division could provide a description of the project selection process and scoring methodology on the GDOT website. The Planning Division should also ensure that this information is clearly communicated during public hearings and meetings with MPO officials.
17. The Planning Division should communicate more information regarding its rationale for selecting, or not selecting, specific projects as well as the rationale for adding or removing projects from the approved STIP.
18. In addition, to ensure transparency in the selection of 100% state-funded projects, the Planning Division should include a listing of such projects as part of the Statewide Strategic Transportation Plan (SSTP). As required by O.C.G.A. 322-41.1, the plan is to include a list of projects realistically expected to begin construction within the next four years and should include the cost and sources of funds for each project.
19. To better ensure transparency, the General Assembly may wish to consider establishing a statutory requirement for the Planning Division to publicly disclose more information regarding its process and project selection decisions.

Transportation Project Selection and Prioritization

42

Appendix B: Objectives, Scope, and Methodology
Objectives This report examines the Georgia Department of Transportation's (GDOT) highway project prioritization and selection process. Specifically, our examination set out to determine the following:
1. How does GDOT determine which highway projects it will fund? 2. To what extent does GDOT follow industry standards or best practices for
setting priorities and selecting highway projects? 3. What opportunities exist for making the process for selecting highway
projects more transparent?
Scope This special examination generally covered capacity projects programmed between fiscal years 2016 and 2025, with consideration of earlier or later periods when relevant. Information used in this report was obtained by interviewing program staff, reviewing federal and state laws, and reviewing GDOT and Planning Division policies, procedures, and program documentation. Additionally, transportation planning literature, reports, and research discussing best practices were consulted, as were highway planning personnel in other state DOTs. Surveys of Metropolitan Planning Organizations (MPOs) and other local transportation planning officials were also conducted to gain an understanding of the Planning Division's involvement in local planning efforts, its communication of project selection decisions, and its promotion of transparency in the project selection process.
Data from the Planning Division's project prioritization spreadsheet (the DWT project scoring system) was analyzed to determine if projects were scored and evaluated according to the Planning Division's stated protocols. We assessed the controls over DWT data used for this examination and determined that the DWT data was sufficiently reliable for our analyses.
While the scope focused primarily on capacity projects, we also reviewed basic documentation related to other project types (maintenance, bridge, safety enhancements, and operation improvements).
Methodology To determine how GDOT prioritizes and selects highway projects it will fund, we interviewed GDOT and Planning Division staff regarding each step of the process including project identification, the initial project review, project scoring, and STIP development. We also reviewed state law to identify statutory requirements that may impact project prioritization and selection. In addition, we reviewed various agency documents, including the Planning Manual, standard operating procedures, and congressional balancing reports. We also reviewed the Planning Division's prioritization spreadsheet, as described in greater detail in the next objective. Lastly, we reviewed more detailed project information for a small sample of projects.
To determine the extent to which GDOT follows industry standards or best practices for setting priorities and selecting highway projects, we reviewed transportation planning literature, reports, and research discussing best practices for highway project prioritization and selection. The American Association of State

Transportation Project Selection and Prioritization

43

Highway and Transportation Officials (AASHTO), Federal Highway Administration (FHWA), and the National Academies of Sciences, Engineering, & Medicine's Transportation Research Board were key resources. Reports produced by Cambridge Systematics, Inc., a transportation management and planning consultant, also served as a resource to identify best practices and national trends. Interviews were conducted with Planning Division staff to learn about the methodology, criteria, and scoring system currently used to prioritize and select projects; how the criteria and scoring system was selected; how the results from the prioritization process are used; and what factors outside the prioritization scoring system influence project selection, programming, and prioritization decisions. Interviews were also conducted with Department of Transportation planning officials responsible for highway project prioritization and selection in Michigan, North Carolina, Ohio, Oregon, Tennessee, and Virginia in order to learn about application of methods discussed in the literature and the extent to which they had been successful in implementing best practices. Relevant laws in Georgia and other states were consulted to identify the ways state laws can be a help or hindrance to promoting effective project selection practices. GDOT's 13 project selection criteria and overall methodology was then reviewed and evaluated against best practices identified in the literature and through other states interviews.
Data from the Planning Division's project prioritization spreadsheet was analyzed to: Determine number and characteristics of capacity projects recently considered for funding. Assess whether projects are being scored according to the procedures and protocols indicated by GDOT in interviews and written policy and procedure documents. Identify projects selected that were low-scoring or bypassed the scoring process altogether. Identify projects that may be unfairly advantaged by the existing scoring methodology. Assess the completeness of data.
As noted earlier, we assessed the controls over DWT data used for this examination and determined that the DWT data was sufficiently reliable for our analyses. The suitability of the DWT data for the purposes of this special examination was assessed by examining general completeness of scores, whether illogical entries existed, and whether sufficient supporting data existed to substantiate project criteria scores and total project scores.
To determine what opportunities exist for making the process for selecting highway projects more transparent we reviewed state law to identify statutory requirements related to transparency, including coordination between Planning Division staff and MPOs and local governments. We interviewed Planning Division staff and reviewed policies and procedures to identify methods for involving/coordinating with stakeholders (MPOs, local governments, general public) in each stage of the project selection and prioritization process, and communicating project selection decisions. We reviewed key documents, including the STIP and the SSTP, to assess the level of explanation or description provided regarding project selection, programming, and prioritization decisions. In addition, the audit team surveyed all 16 MPOs in Georgia to determine their perception of the clarity of their roles and responsibilities in project selection, the effectiveness of coordination efforts

Transportation Project Selection and Prioritization

44

with GDOT, their level of influence in project selection, the clarity of GDOT's prioritization method and criteria, and GDOT's communication of project selection decisions. Lastly, we reviewed information from other states, including state laws and DOT websites, to determine the extent to which other states communicate their project scoring methodologies, criteria, project selection decisions, and the roles and responsibilities of all stakeholders during the project selection process. This information was used to identify potential strategies for improving transparency during project selection.
This special examination was not conducted in accordance with generally accepted government auditing standards (GAGAS) given the timeframe in which the report was needed. However, it was conducted in accordance with Performance Audit Division policies and procedures for non-GAGAS engagements. These policies and procedures require that we plan and perform the engagement to obtain sufficient, appropriate evidence to provide a reasonable basis for the information reported and that data limitations be identified for the reader.
Agency Response: In its response, GDOT "acknowledges that the examination was not conducted in accordance with generally accepted government auditing standards [GAGAS], but does recognize that it was performed in accordance with internal policies and procedures [non-GAGAS]." GDOT indicated that conducting the review as non-GAGAS may have impacted the time allowed, scope, and nature of the work conducted.
Auditor's Response: While this engagement was not conducted in accordance with GAGAS standards, it was conducted in accordance with our internal policies and procedures. These policies and procedures ensure that the completeness and accuracy of the work conducted is not compromised. The primary difference between GAGAS and non-GAGAS engagements is the time spent in planning. Planning for a GAGAS engagement involves independently identifying and selecting objectives, conducting an overall risk assessment, and documenting detailed fraud and data reliability assessments. In this special examination, the time spent on these activities was limited to the questions (i.e., objectives) proposed by the requester. We had sufficient time to collect appropriate evidence to support our findings and conclusions within the requested scope of our work, which entailed a comprehensive review of the Planning Division's selection and prioritization of capacity projects.

Transportation Project Selection and Prioritization

45

Appendix C: MPO Map

TN
Chattanooga
Hamilton

NC

Dade

Catoosa

Walker

Whitfield

Dalton

Fannin

Murray

Gilmer

Union

Towns

Rabun

White

Habersham

Chattooga
Rome
Floyd

Gordon
Cartersville Pickens

Bartow

Cherokee

Lumpkin
Dawson Hall
Forsyth

Stephens

Banks

Franklin

Hart

Gainesville

SC

AL

Polk Haralson
Carroll Heard

Paulding

Atlanta
Cobb

Gwinnett

Douglas

Fulton

DeKalb

Jackson

Madison

Elbert

Barrow Walton

Clarke Oconee

Oglethorpe
Athens

Wilkes

Lincoln

Rockdale

Coweta

Clayton

Fayette

Henry

Newton Butts

Morgan

Jasper

Putnam

Greene

Taliaferro Warren

McDuffie

Hancock

Glascock

Spalding

Troup

Meriwether

Pike

Lamar

Monroe

Macon

Jones

Baldwin

Washington

Jefferson

Edgefield
Columbia Richmond
Augusta
Burke

Aiken

Lee Russell

Harris

Talbot

Muscogee

Chattahoochee

Marion

Columbus

Upson Taylor
Schley

Crawford

Bibb

Peach

Macon

Houston

Wilkinson Twiggs

Johnson

Bleckley

Laurens

Warner Robins

Treutlen

Dooly

Pulaski

Dodge

Wheeler

Stewart

Webster

Sumter

Quitman

Randolph

Terrell Lee

Crisp
Albany

Wilcox

Turner

Ben Hill

Telfair

Jeff Davis

Montgomery

Jenkins Emanuel

Screven

Candler

Bulloch

Effingham
Savannah

Toombs

Evans Tattnall

Hinesville

Appling

Long

Bryan Liberty

Chatham

Clay

Calhoun

Dougherty

Early Miller

Baker

Mitchell

Worth

Irwin Tift

Colquitt

Berrien Cook

Coffee Atkinson

Bacon

Wayne

Pierce

Ware

Brantley

McIntosh Glynn

Seminole

Decatur

Grady

Valdosta

Thomas

Brooks

Lanier Lowndes

Clinch

Charlton

Brunswick
Camden

Echols

FL

Source: Agency documents

Transportation Project Selection and Prioritization

46

Appendix D: Project Selection Processes for Maintenance/Pavement, Bridge, Safety Enhancements, and
Operational Improvements
Maintenance/Pavement
GDOT's Office of Maintenance is responsible for preserving and managing the 18,000 miles of state maintained roadway. The Office of Maintenance prioritizes and selects projects based on an asset-management approach, which emphasizes well-defined goals, objectives and targets, quality data and information, and consideration of risk. As discussed below, GDOT's prioritization methodology factors in both asset condition and risk:
Asset Condition - Asset condition for asphalt pavement is evaluated using the Computerized Pavement Condition Evaluation system (COPACES). Every two years, GDOT's local district engineers conduct pavement evaluations of every mile of state-maintained roadway. Each route is assigned to a local DOT Area Maintenance Manager, who is responsible for calculating the initial rating on a 1-100 scale. Routes receiving a rating of 75 or below are then reviewed by the district maintenance engineer and then by a representative from the state maintenance office (general office). Roadways are recommended for resurfacing if they have a rating of 70 or below, which indicates poor or bad condition.
Risk Analysis The COPACES rating is then modified based on a risk analysis. GDOT's risk matrix assigns a risk factor to each route based on criteria that directly affects the condition of the asset. The analysis identifies a total risk factor based on the functional classification of the route, AADT, percent truck traffic, and the population of the county the route is located in. The COPACES rating score is then divided by the total risk factor. This modified COPACES rating prioritizes the needed work.
Each GDOT district is allocated funds based on the number of state route miles in the district by roadway priority type. Using the methodology described above, each district office selects and prioritizes asphalt pavement projects (asphalt accounts for approximately 95% of GDOT's roadways). Concrete pavement projects are selected using an evaluation process that is similar to COPACES but less frequent. In addition, maintenance can include sign and striping projects, which are selected based on visual day and night inspections conducted annually.

Bridges
Bridge projects are prioritized and selected by GDOT's Bridge Maintenance Unit. This Unit inspects bridges and collects data for the Bridge Prioritization Ranking formula. The formula identifies which bridges are candidates for rehabilitation or replacement and where these bridges need to be scheduled in the construction work program. The formula is based on structural capacity (e.g., strength and condition of the structure) and user demand (e.g., amount of traffic crossing the bridge). The specific components of the formula include:

Transportation Project Selection and Prioritization

47

Inventory Rating - an indicator of the bridge's load carrying capacity Average Daily Traffic - the number of vehicles, on average per day, that use the
bridge each year Bypass -the distance, in miles, that a vehicle must travel if the bridge is posted
or closed Bridge Condition a factor that indicates the overall condition of the bridge
deck, substructure and superstructure Risk Factor - used to weigh the risk associated with the various classifications
of roadway systems for which the bridge is a part. Additional weight is also given to bridges with timber components, reduced
weight limits, repairs, substandard vertical or horizontal clearance, fracture, critical and unknown or scour critical foundations.
The Bridge Prioritization Formula generates a project ranking list, which serves as the starting point in the selection process. Other factors considered during project selection include proximity to other relevant work, engineering judgment, and congressional balancing. A final list of projects to include in the STIP is then developed by the State Bridge Engineer, in coordination with the Office of Planning and the Office of Program Delivery.

Safety Enhancements
Safety enhancement projects are intended to reduce the number and severity of lane departure crashes, improve pedestrian safety, or improve the design and operation of an intersection. Typical projects include cable barrier, rumble strips, improved signage and striping, pedestrian safety, corridor improvements, or intersection improvements. These projects are prioritized and selected by the Office of Traffic Operations and the Office of Utilities, for railroad crossing projects. The selection process varies based on project type: site-specific projects (e.g., traffic signals, pedestrian upgrades); system-wide projects (e.g., guardrails); and railroad highway crossings.
Site Improvement - A traffic engineering study is used to select safety improvements for the sites/locations with high crash rates and/or fatalities. First, GDOT identifies the sites with safety improvement potential by analyzing crash data that ranks potential sites. Locations reported by various stakeholders, including citizens, elected officials, and local governments are also considered. Next, district office staff conduct evaluations that include crash data analysis and a field survey, and make a recommendation. Staff also conduct a benefit-cost analysis in which the costs (construction, operations, etc.) are evaluated against the projected benefits (reduced property damages, injuries, and fatalities). The projects are then prioritized based on the benefitcost analysis.
System-Wide System-wide safety improvement studies are conducted to identify low-cost safety improvements to be implemented at a large number of locations with the potential for certain types of crashes. First, GDOT analyzes statewide crash data to identify the types of crashes with high fatalities. Based on GDOT's studies and/or national level data, safety improvements are recommended for selected high-fatality crash types. Each

Transportation Project Selection and Prioritization

48

safety improvement is then allocated a budget based on the damages, injuries, and fatalities that can be reduced by the safety improvement. The allocated budget is then divided by the unit cost of the safety improvement to determine the quantity (e.g., miles to treat). Locations suitable for the safety improvement are selected based on roadway characteristics and crash data. The list of locations is reviewed by the district offices and then finalized.
Railroad Highway Crossings These projects usually entail installing train activated warning devices including gates, lights, and bells. To determine which crossings warrant these devices, crossings are ranked according an Adjusted Hazard Index. The Index is based on the relationship between the number of trains and vehicles at the crossing with adjustments made for factors including crash history, school bus crossings, and passenger rail service.

Operational Improvements
Operational improvement projects are intended to improve efficiency without adding significant capacity. Typical projects include traffic signal coordination, ramp metering, signs, and intersection improvements (e.g., roundabouts). Project requests are submitted in various forms (traffic engineering studies, planning studies, emails and phone calls) from stakeholders including GDOT Operations staff, GDOT district staff, local governments, consultants, elected officials, and citizens. As described below, potential projects undergo an initial screening and then a more in-depth analysis before being voted on by the Operations Committee.
Projects are initially screened by GDOT's District Operations staff and the Office of Traffic Operations staff. The screening process involves reviewing each location using online mapping to evaluate the availability of right of way, potential utility conflicts, possible environmental impacts, programmed projects in the area, and the planninglevel cost of the recommended improvement.
Once a project passes through the screening process, an in-depth analysis is conducted by a consultant to identify a solution. The consultant develops a project synopsis package which includes an evaluation of the existing deficiencies, the description and schematic of the proposed solution, existing and proposed measures of effectiveness (level of service, delay, queue length, etc.), and cost estimate by phase. The consultant also calculates the benefit to cost ratio based on travel time savings for all users at the study location and project costs, including preliminary engineering, utilities, right of way, and construction.
The project synopsis package is reviewed by GDOT Operations staff, and then it is presented to the Operations Committee. The Operations Committee, which is composed of the Chief Engineer, the Director of Operations, the State Traffic Engineer, the Director of Engineering, and the State Planning Administrator, reviews the proposed improvement for its applicability to program goals, FHWA funding stipulations, and benefit to cost ratio. The Committee votes on the project, and if approved, the project is programmed for plan development and construction.

Transportation Project Selection and Prioritization

49

Appendix E: Prioritization Criteria for Capacity Projects

Criteria 1 Connectivity

Description of Criteria
Key Criteria
Bypass and interchange projects that improve access to the highway system receive 30 points. This criteria is intended to help augment scores for new construction projects that are disadvantaged in the safety criteria because no current crash data exists for new projects.

Max Points
30

2

State Route Prioritization

GDOT categorized the state route system into priority levels (critical, high, medium, low). Based on this priority level, projects are awarded a range of 0 to 18 points for this criteria.

18

The state freight network is a system of interstates and freight corridors identified as

3 Freight Network essential to the delivery of goods and freight throughout the state. Projects that make

10

improvements to the freight network receive 10 points for this criteria.

4 Freight Plan

The Freight and Logistics Plan details the current and future transportation needs for moving

freight efficiently within Georgia. Projects included in the Freight Plan are awarded a range of 2-8 points based on the year the project is slated in the Freight Plan timeline (projects earlier

8

in the timeline are considered higher priority and receive more points).

GRIP is a system of proposed economic development highways that was originally adopted

Governor s Road in 1989 to enhance economic development, connectivity, safety, and provide opportunities

5 Improvement

for growth in rural Georgia. Currently, the GRIP system encompasses 3,300 miles across 19

5

Program (GRIP) different corridors. Projects that are identified as one of the top five priorities in a GRIP

corridor receive 5 points.

Additional Criteria

6 Safety

Projects are awarded points based on each injury, fatality, or damage to personal property as a result of traffic accidents in the vicinity of a project. This data is inserted into a tool which assigns weights and calculates a single severity index score that assesses the existing safety condition for a particular project site. There is no maximum point potential for this criteria.

No Max

7

Congestion; Level of Service (LOS)

LOS is a planning level measure of congestion. A scale is used to compare existing LOS with potential increase in LOS as a result of the project using forecasted future travel projections. Projects with an underperforming LOS that will be improved by the project receive points based on the extent of improvement, ranging from 8 to 28 points.

28

8

Pavement Condition

To maximize resources and defer the need for additional maintenance activities, GDOT

considers the existing pavement condition. (Generally, if a roadway is widened all pavement

surfaces are replaced so there would no longer be an immediate need to do maintenance

5

activities.) If a project s pavement score is below the threshold for what is considered good

condition , the project receives 5 points.

9 Planning Study Projects that address a need identified in an existing GDOT planning study receive 5 points.

5

Each project has a concept report that outlines the need and purpose of the project as well

10

Approved Concept Report

as a detailed scope of work. An approved concept report indicates that a project s goals and scope have a consensus amongst stakeholders and is ready to move forward pending funding availability. If a project s concept report has been officially approved, the project

1

receives 1 point.

Project

A PFA is a document that signifies that a local government has funding on hand to contribute

11 Framework

to a transportation project and demonstrates the local government s commitment to a project.

1

Agreement (PFA) Projects with a signed PFA receive 1 point.

12 SSTP Goal

Projects that meet one or more of the Statewide Strategic Transportation Plan (SSTP) goals

of improving safety, improving reliability, reducing congestion, maintaining and preserving

1

infrastructure, and improving the environment receive 1 point.

Regional Traffic 13 Operations
Program (RTOP)

The RTOP is a multi-jurisdictional traffic signal timing program with the goal of improving traffic flow and reducing vehicle emissions. RTOP exists on approximately two dozen corridors in Metro Atlanta. To maximize and supplement previous investments, projects located on a RTOP corridor receive 1 point.

1

Source: Agency documents

Transportation Project Selection and Prioritization

50

Appendix F: FAST Act Federal Funding Programs1, 2

National Highway Performance Program (NHPP)
The National Highway Performance Program provides support for the condition and performance of the National Highway System (NHS), the construction of new facilities on the NHS, and to ensure that investments of federal aid funds in highway construction are directed to support progress toward the achievement of performance targets established in a state s asset management plan for the NHS. The NHS is composed of rural and urban roads serving major population centers and the Interstate system.
Main program: National Highway Performance Program
Surface Transportation Block Grant Program (STBG)
The Surface Transportation Block Grant Program promotes flexibility in state and local transportation decisions and provides flexible funding to best address state and local transportation needs. Funds may be used by the state or local government to preserve or improve conditions and performance on any federal aid highway or bridge projects on any public roads. The FAST Act converted the Surface Transportation Program into the Surface Transportation Block Grant Program acknowledging that this program has the most flexible eligibilities among federal aid highway programs.
Main program: STBG Program Flexible Subprograms: Urbanized Areas with Population Over 200k, Areas with Population Over 5k to 200k, Areas with Population 5k and Under, Off-System Bridge Set-aside, and Transportation Alternatives Set-aside
Highway Safety Improvement Program (HSIP)
The Highway Safety Improvement Program funds projects to achieve a significant reduction in traffic fatalities and serious injuries on all public roads. Safety projects include hazard elimination, railroad crossing, and railroad protective devices.
Main program: Highway Safety Improvement Program Subprogram: High Risk Rural Roads Special Rule Set-aside
Railway-Highway Crossings Program
The Railway-Highway Crossings Program provides funding for safety improvements to reduce the number of fatalities, injuries, and crashes at public railway-highway crossings. Funding is reserved and set aside from HSIP.
Main program: Railway-Highway Crossing Hazard Elimination Subprogram: Railway-Highway Crossing Protective Devices
Congestion Mitigation/Air Quality Program (CMAQ)
The Congestion Mitigation & Air Quality Improvement Program provides a flexible funding source to state and local governments for transportation projects and programs to help meet requirements of the Clean Air Act. These funds are available for projects that reduce congestion and improve air quality for areas that do not meet the National Ambient Air Quality Standards for ozone, carbon monoxide, or particulate matter (nonattainment areas) and for former nonattainment areas that are now in compliance. Projects such as intersection improvements, signal coordination, and ride sharing qualify for these funds. Currently there are 28 Georgia counties in nonattainment or maintenance areas: Barrow, Bartow, Bibb, Carroll, Catoosa, Cherokee, Clayton, Cobb, Coweta, DeKalb, Douglas, Fayette, Floyd, Forsyth, Fulton, Gwinnett, Hall, Heard, Henry, Monroe, Murray, Newton, Paulding, Putnam, Rockdale, Spalding, Walker, and Walton.
Main program: Congestion Mitigation & Air Quality Improvement Subprogram: Projects to Reduce Particulate Matter 2.5 Emissions
Metropolitan Planning Program
The Metropolitan Planning Program establishes a cooperative, continuous, and comprehensive framework for making transportation investment decisions in metropolitan areas.
Main program: Metropolitan Planning Program

Transportation Project Selection and Prioritization

51

National Highway Freight Program (NHFP)
The FAST Act established the NHFP to improve the efficient movement of freight on the National Highway Freight Network and support several goals, including: investing in infrastructure and operational improvements that strengthen economic competitiveness, reduce congestion, reduce the cost of freight transportation, improve reliability, and increase productivity; and improving the safety, security, efficiency, and resiliency of freight transportation in rural and urban areas.
Main program: National Highway Freight Program 1 Some federal funds apportioned to main programs may be allocated to subprograms. 2 2% of funds from five core programs (NHPP, STBG, HSIP, CMAQ, and NHFP) are set-aside for State Planning & Research.
Source: Agency documents

The Performance Audit Division was established in 1971 to conduct in-depth reviews of state-funded programs. Our reviews determine if programs are meeting goals and objectives; measure program results and effectiveness; identify alternate methods to meet goals; evaluate efficiency of resource allocation; assess compliance with laws and regulations; and provide credible management information to decision makers. For more information, contact
us at (404)656-2180 or visit our website at www.audits.ga.gov.