Curve safety improvements using mobile devices and automatic curve sign detection

GEORGIA DOT RESEARCH PROJECT 18-18 Final Report
CURVE SAFETY IMPROVEMENTS USING MOBILE DEVICES AND AUTOMATIC CURVE SIGN DETECTION PHASE I
Office of Performance-based Management and Research 600 West Peachtree St. NW | Atlanta, GA 30308
i

1.Report No.: FHWA-GA-20-1818

2. Government Accession 3. Recipient's Catalog No.:

No.: N/A

N/A

4. Title and Subtitle: Curve Safety Improvements Using Mobile Devices and Automatic Curve Sign Detection Phase I

5. Report Date: November 2019 6. Performing Organization Code: N/A

7. Author(s): Yichang (James) Tsai, Ph.D.; Yiching Wu; Nicolas Six; Cibi Pranav

8. Performing Organ. Report No.: 18-18

9. Performing Organization Name and Address: Georgia Institute of Technology 790 Atlantic Drive Atlanta, GA 30332-0355

10. Work Unit No.: 11. Contract or Grant No.: PI# 0016334

12. Sponsoring Agency Name and Address: Georgia Department of Transportation Office of Performance-based Management and Research 600 West Peachtree St. NW Atlanta, GA, 30308

13. Type of Report and Period Covered: Final; October 2018 November 2019
14. Sponsoring Agency Code:

15. Supplementary Notes: Prepared in cooperation with the U.S. Department of Transportation, Federal Highway Administration.

16. Abstract: A disproportionally high number of serious vehicle crashes (25% of fatal crashes) occur on horizontal curves (FHWA, 2019), even though curves represent only a fraction of the roadway network (5% of

highway miles) (FHWA, 2016). The MUTCD (Manual on the Uniform Traffic Control Devices) (FHWA, 2012) requires various horizontal alignment warning signs (curve signs) to ensure curved

roadway safety. However, current transportation agencies' practices for inventorying the locations and

types of existing curve signs are largely a manual procedure, which is costly, labor-intensive and timeconsuming. This report presents a cost-effective, live curve sign inventory system for meeting MUTCD requirements using intra-agency, low-cost mobile devices (e.g. smartphones), existing vehicles, and deep learning and crowdsourcing technologies with a special focus on 1) critically validating an automatic curve sign detection and classification method and 2) assessing and delivering a smartphonebased data collection module. The outcomes will strongly complement current transportation agencies' curve sign placement operations for meeting MUTCD requirements. A case study, using 13 centerline miles of roadway on State Route 2 and consisting of 471 curve signs in Georgia, was conducted to critically validate the accuracy of detecting and classifying curve signs using the proposed method.

Results show 471 curve signs were correctly detected and it has demonstrated that the developed automatic curve sign detection and classification method using deep learning is very promising for

implementation. Finally, conclusions and recommendation for future research along with a roadmap

for validating, refining, and implementing a cost-effective, live and sustainable curve sign system, are presented.

17. Key Words: Horizontal curves, Curve sign inventory, MUTCD requirements, Automatic detection and classification, Deep learning

18. Distribution Statement: No Restriction

19. Security Classification (of this report): Unclassified

20. Security classification 21. Number 22. Price: (of this page): Unclassified of Pages: 82 Free

ii

Contract Research
GDOT Research Project No. 18-18 Final Report
CURVE SAFETY IMPROVEMENTS USING MOBILE DEVICES AND AUTOMATIC CURVE SIGN DETECTION PHASE I By Yichang (James) Tsai, Ph.D., P.E. Yi-Ching Wu Nicolas Six Cibi Pranav
Georgia Institute of Technology
Contract with Georgia Department of Transportation
In cooperation with U.S. Department of Transportation
Federal Highway Administration
November 2019
The contents of this report reflect the views of the author(s) who is (are) responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the Georgia Department of Transportation or of the Federal Highway Administration. This report does not constitute a standard, specification, or regulation.
i

ii

TABLE OF CONTENTS
LIST OF FIGURES....................................................................................................................... v LIST OF TABLES ....................................................................................................................... vii EXECUTIVE SUMMARY ........................................................................................................ viii CHAPTER 1. INTRODUCTION ............................................................................................. 1
BACKGROUND AND RESEARCH NEED .............................................................................. 1 RESEARCH OBJECTIVES AND TASKS ................................................................................. 2 REPORT ORGANIZATION ....................................................................................................... 4
CHAPTER 2. REVIEW OF GEORGIA DOT'S PRACTICES FOR IDENTIFYING MISSING CURVE SIGNS............................................................................................................ 6
DETERMINATION OF REQUIRED CURVE WARNING SIGNS........................................... 6 ASSESSMENT AND MAINTENANCE OF EXISTING CURVE SIGNS.............................. 13 CHALLENGES ......................................................................................................................... 14 SUMMARY............................................................................................................................... 16
CHAPTER 3. PROPOSED METHODOLOGY ................................................................... 18 DETECTION............................................................................................................................. 22 CLASSIFICATION ................................................................................................................... 26 CLUSTERING .......................................................................................................................... 27
CHAPTER 4. DATA COLLECTION MODULE.................................................................. 28 SMARTPHONE CONFIGURATION ....................................................................................... 28 "AllGather" APP ....................................................................................................................... 33
CHAPTER 5. CASE STUDIES ON AUTOMATIC CURVE SIGN DETECTION AND CLASSIFICATION USING SMARTPHONE DATA .............................................................. 34
iii

CASE STUDY ON STATE ROUTE 2 TEST SITE .................................................................. 34 SUMMARY............................................................................................................................... 42 CHAPTER 6. CASE STUDIES ON AUTOMATIC CURVE SIGN DETECTION AND CLASSIFICATION USING SMARTPHONE DATA .............................................................. 43 ROAD MAP .............................................................................................................................. 43 POTENTIAL BENEFITS.......................................................................................................... 46 CHAPTER 7. CONCLUSIONS AND RECOMMENDATIONS ......................................... 49 ACKNOWLEDGMENTS........................................................................................................... 52 REFERENCES ..............................................................................................53 APPENDIX A. INSTALLATION GUIDE FOR "AllGather" ................................................. 55 APPENDIX B. SMARTPHONE SETUP, DATA COLLECTION, AND DATA TRANSFER PROCEDURES ..............................................................................................64
iv

LIST OF FIGURES
FIGURE 2-1. PHOTO. RIEKER INC.'S BALL-BANK INDICATOR AND THE TABLET DEVICE FITTED INTO A DATA COLLECTION VEHICLE (RIEKER INC.)............................................. 8
FIGURE 2-2. PHOTO. BOUNDING BOX TO ENCOMPASS A CURVE .......................................... 9 FIGURE 2-3. PHOTO. EXAMPLE OF GPS FIT FOR THE TWO PASSES ON THE SAME CURVE ..... 9 FIGURE 2-4. PHOTO. EXAMPLE OF GENERATED CURVE SIGN REPORT ............................... 12 FIGURE 2-5. PHOTOS. AN EXAMPLE OF PANNING AND ZOOMING OF IMAGES BEFORE
ARRIVING AT AN IDEAL BOUNDING BOX ....................................................................... 15 FIGURE 3-1. FLOWCHART. A PROPOSED LOW-COST MOBILE PHONE BASED METHODOLOGY
FOR CURVE SAFETY IMPROVEMENT ............................................................................. 20 FIGURE 3-2. ILLUSTRATION. YOLO V3 NETWORK ARCHITECTURE .................................... 23 FIGURE 3-3. PHOTO. EXAMPLES OF CURVE SINGS WITH DIAMOND AND RECTANGULAR
SHAPES ....................................................................................................................... 23 FIGURE 3-4. PHOTOS. IMAGES FROM HIGH-RESOLUTION IMAGES AND THE SMARTPHONE
IMAGES ....................................................................................................................... 25 FIGURE 3-5. GRAPH. TRAINING EPOCH WITH MODEL ACCURACY ..................................... 26 FIGURE 4-1. ILLUSTRATION. VISUALIZATION OF 3-LANE COVERAGE ............................... 29 FIGURE 4-2. PHOTOS. DIFFERENT VIEWS OF SMARTPHONE INSTALLATION INSIDE A VEHICLE
................................................................................................................................... 30 FIGURE 4-3. PHOTO. AN EXAMPLE OF A 40X40 SIGN IN A 1920X1080 IMAGE................... 32 FIGURE 4-4. PHOTO. UI OF "ALL GATHER" ..................................................................... 33 FIGURE 5-1. MAP. THIRTEEN CENTER LINE MILES OF TEST ROUTE ON STATE ROUTE 2 IN
GEORGIA (MOUNTAIN AREA) ....................................................................................... 35
v

FIGURE 5-2. PHOTOS. GDOT TRUCK AND SMARTPHONE USED FOR FIELD DATA COLLECTION ............................................................................................................... 36
FIGURE 5-3. PHOTOS. EXAMPLES OF TRUE POSITIVES (TP) ............................................... 39 FIGURE 5-4. PHOTOS. EXAMPLES OF FALSE POSITIVES (FP) .............................................. 41 FIGURE 6-1. FLOWCHART. A ROADMAP FOR VALIDATING, REFINING, AND IMPLEMENTING A
COST-EFFECTIVE LIVE CURVE SIGN SYSTEM TO ENSURE ROADWAY SAFETY ................. 45 FIGURE A-1. PHOTO. DOWNLOADED APK FILE .............................................................. 57 FIGURE A-2. PHOTO. APP INSTALLED WINDOW................................................................ 58 FIGURE A-3. PHOTO. PICTURE AND VIDEO PERMISSION POP-UP ....................................... 59 FIGURE A-4. PHOTO. USER ID WINDOW .......................................................................... 59 FIGURE A-5. PHOTO. MAIN WINDOW (RECORDING NOT STARTED) ................................... 60 FIGURE A-6. PHOTO. MAIN WINDOW (RECORDING STARTED) .......................................... 60 FIGURE A-7. PHOTO. "OPEN WITH" WINDOW.................................................................. 63 FIGURE A-8. PHOTO. "INSTALLATION" WINDOW ............................................................. 63 FIGURE B-1. PHOTOS. INCORRECT SMARTPHONE PLACEMENTS ....................................... 65 FIGURE B-2. PHOTOS. PHONE HOLDER USAGE ................................................................. 66 FIGURE B-3. PHOTO. NEAR DASHBOARD PLACEMENT ..................................................... 67 FIGURE B-4. PHOTO. NEAR REAR VIEW PLACEMENT........................................................ 68 FIGURE B-5. PHOTO. ROTATE TO CORRECT ORIENTATION ................................................ 68 FIGURE B-6. PHOTOS. VERTICAL ALIGNMENT.................................................................. 69 FIGURE B-7. PHOTO. VERTICAL ALIGNMENT................................................................... 69
vi

LIST OF TABLES
TABLE 2-1. CURVE PROPERTIES AND ADVISORY SPEED REPORTED IN THE CARS WEB PORTAL ....................................................................................................................................11
TABLE 5-1. DEFINITIONS OF TP, TN, FN, AND FP.................................................................. 37 TABLE 5-2. TEST OUTCOMES ON SR2.................................................................................. 38
vii

EXECUTIVE SUMMARY
A disproportionally high number of serious vehicle crashes (25% of fatal crashes) occur on horizontal curves (FHWA, 2019), even though curves represent only a fraction of the roadway network (5% of highway miles) (FHWA, 2016). The MUTCD (Manual on the Uniform Traffic Control Devices) (FHWA, 2012) requires various horizontal alignment warning signs (curve signs) to ensure curved roadway safety. However, current transportation agencies' practices for inventorying the locations and types of existing curve signs are largely a manual procedure, which is costly, labor-intensive and timeconsuming. Moreover, the locations and types of curve signs cannot always be accurately and reliably collected using visual inspection. Therefore, there is an urgent need to develop a method that improves current manual curve sign inventory methods and supports the new MUTCD-compliance curve sign assessment operation that requires frequent curve sign assessment. An innovative and cost-effective alternative for establishing a live curve sign inventory system for meeting MUTCD requirements has been proposed and validated in the Phases I and II of this research project. A costeffective, live curve sign inventory system is essential for identifying the missing curve signs in a timely manner in support of adequate countermeasures for meeting the MUTCD requirement. A live curve sign inventory system will enable transportation agencies to ensure that they are meeting the MUTCD requirements, including sign types and proper spacing.
Advances in automatic sign detection using deep learning, computing power, and cameras in mobile phones provide a great opportunity to identify and obtain curve sign data cost-effectively. To date, there is no developed method that performs a live curve
viii

sign inventory using the combined capabilities of mobile devices and automatic curve sign detection methods with deep learning. The objective of this research is to develop and critically assess a method that uses low-cost mobile devices and leverages AI to establish a live sustainable curve sign inventory that effectively identifies deficient signs, such as missing signs, in a timely manner to support a proactive curve sign improvement method for meeting the MUTCD requirements.
Curve sign detection is one of the most critical components in the proposed methodology and system. Phase I of this research is to critically validate a developed curve sign detection and classification method that uses low-cost smartphones and deep learning technologies. A case study, using 13 centerline miles of roadway on State Route 2 consisting of 471 curve signs, in Georgia, was conducted to critically validate the accuracy of detecting and classifying curve signs using the proposed method. Results show 471 curve signs were correctly detected. The case study clearly demonstrates that the proposed automatic curve sign detection and classification method using deep learning is very promising.
The field data collection module using smartphones has been developed and is ready for implementation. The detailed implementation manual using Allgather is shown in Appendixes A and B. With the promising outcomes of curve sign detection and classification using deep learning and low-cost mobile devices, the following recommendations are made for future research in Phase II of this research project:
1) Classifying work zone signs into a new "work zone sign" category is recommended.
ix

2) Computing sign locations using various methods, such as camera calibration, is recommended so the location of the detected signs can be computed and visualized on a GIS map.
3) Further study of data reduction and data management is needed. Effectively managing data is a key issue for the implementation of the proposed method. Different possible alternatives, including storing only sign type and sign location or with an additional sign bounding box for confirmation, need to be explored.
4) Link and cross-check the outcomes of automatic curve sign detection and sign location computation with the required MUTCD sign types and spacing to develop a live curve sign condition assessment.
5) An implementation roadmap is needed for deploying the developed low-cost technologies to state DOTs and local transportation agencies (counties and cities).
6) It is recommended to develop an iOS application of the 'AllGather' application for Apple phones, which is predominately used by GDOT engineers.
A roadmap for validating, refining, and implementing a cost-effective live and sustainable curve sign system is also presented. It includes three stages (a proof of concept, a pilot study, and a large-scale implementation). The ultimate goal of this research project, including Phases I and II (a proof of concept) is to critically validate a cost-effective live curve sign inventory system, including methods and methodologies that enable transportation agencies to meet MUTCD curve sign compliance requirement and to reduce current 25% fatalities and crashes on the curved road sections.
x

CHAPTER 1. INTRODUCTION
BACKGROUND AND RESEARCH NEED
According to FHWA (2019), more than 25 percent of fatal crashes occur on horizontal curves that constitute only 5% of highway pavements (FHWA, 2016). The MUTCD (Manual on the Uniform Traffic Control Devices) (FHWA, 2012) requires various horizontal alignment warning signs (curve signs) to ensure curved roadway safety, so an up-to-date live curve sign inventory that enables the Georgia Department of Transportation (GDOT) to manage its curve signs cost-effectively is crucial. However, GDOT's current manual procedure for inventorying the locations and types of existing curve signs is costly, labor-intensive and time-consuming. Moreover, the locations and types of curve signs cannot always be accurately and reliably collected using visual inspection. Therefore, there is an urgent need for a cost-effective method that improves GDOT's current manual curve sign inventory and inspection for meeting MUTCD requirements. Advances in automatic sign detection using artificial intelligence (AI), like deep learning, computing power, and cameras in mobile phones, provide a great opportunity to identify and obtain curve sign data cost-effectively. To date, there is no developed method that performs a live curve sign inventory using the combined capabilities of mobile devices and automatic curve sign detection with AI. The objective of this research is to develop and critically assess a method that uses low-cost mobile devices and leverages AI to establish a live sustainable curve sign inventory that can costeffectively identify deficient signs, such as missing signs, in a timely manner to support proactive curve sign improvement. The proposed method will take advantage of intra-
1

agency, crowdsourced, low-cost mobile devices to ensure that safety improvement needs can be identified and that proactive safety countermeasures can be applied in a timely, cost-effective, and safe manner. This research project will leverage previous research outcomes, including automatic sign detection and curve identification developed by the principal investigator (PI) in previous research projects sponsored by the United States Department of Transportation (US DOT), FHWA, and GDOT.
RESEARCH OBJECTIVES AND TASKS
The objective of this research is to develop and critically assess a method that integrates low-cost mobile devices and artificial intelligence to establish a live sustainable curve sign inventory. The proposed method will cost-effectively improve GDOT's current manual curve sign inventory and enable GDOT to establish a live sustainable curve sign inventory. It will, also enable GDOT to identify and replace curve signs in a proactive, timely, and systematic way that improves Georgia's curved road safety.
The proposed method consists of seven steps: a) adjusting hardware configuration to maximize the automatic curve sign detection; b) collecting sensor data using low-cost mobile devices; c) automatically computing curves, including curve location, radius, point of curve (PC), and point of tangent (PT); d) refining (training) the AI algorithms to better detect curve signs and recognize their MUTCD code; e) computing a sign's location/coordinates using GPS data and camera calibration parameters; f) generating sign inventory with sign type (MUTCD code), location (x-y coordinates), and images; g) reducing images for data storage and management; and h) exploring curve sign applications (e.g., comparing curve sign inventory with the sign requirements to identify
2

sign deficiencies and using different timestamps to identify changes). The proposed method is divided into two phases (Phases I and II) in the research project. This project is a proof of concept research project. Phase I, which is this proposed research, consists of Steps a) to d), focusing on refining the AI algorithms for automatically collecting curve signs using low-cost mobile devices on selected test routes. Phase II consists of Steps e) to h), focusing on enhancing curve sign location accuracy computation, data management, and exploration of potential applications. In addition, a pilot test (in selected counties or districts) will be conducted in Phase II in preparation for statewide implementation. The Phase II proposal can begin in Task 3 after Georgia Tech successfully demonstrates the use of the curve sign detection algorithms using the data collected on State Route 2 to the Office of Traffic Operations.
The major tasks in this research project include the following:
1) Conduct a comprehensive literature review of GDOT's practices for identifying missing curve signs.
2) Determine the optimal configurations for collecting curve sign data in Georgia and conduct data collection on test routes.
3) Calibrate and validate AI algorithm(s) for automatic curve sign detection. 4) Develop a roadmap for implementing curve sign data collection using low-cost
mobile devices. 5) Summarize research findings.
3

REPORT ORGANIZATION
This report is organized as follows:
Chapter 1 introduces the background, the objectives, and the organization of this research project.
Chapter 2 presents the review of state DOTs' practices for identifying missing curve signs.
Chapter 3 presents the development of the automatic curve sign extraction method. This includes a description of the architecture, the preparation for training data, the training process for detection and classification, and the result.
Chapter 4 introduces the proposed data collection method using mobile devices, like smartphones, and GDOT's vehicle fleet. This includes the tests on the smartphone configuration for AllGather app (e.g., camera position and angle, data collection frequency, color, focus, etc.) and the design of the AllGather app. AllGather was successfully deployed by GDOT to collect data on the test routes. Feedback on using smartphones for data collection was also gathered.
Chapter 5 presents a case study to validate the automatic sign extraction method using the images collected by smartphones. The validation process, including the definitions and criteria for false positive, false negative, recall, and precision, are first described.
4

Chapter 6 discusses the Phase 2 implementation plan with potential benefits. A road map is presented for implementing the curve sign detection and classification for a live sustainable curve sign inventory. Chapter 7 presents conclusions and recommendations for future research.
5

CHAPTER 2. REVIEW OF GEORGIA DOT'S PRACTICES FOR IDENTIFYING MISSING CURVE
SIGNS
This chapter reviews how the Georgia Department of Transportation (GDOT) plans for the installation of curve signs that meet MUTCD (FHWA, 2012) requirements; it also reviews GDOT's field assessment practices for identifying missing curve signs. This chapter synthesizes the interviews and inputs provided by the traffic and maintenance engineers at GDOT District 1 and the engineers in GDOT's Office of Traffic Operation. The first section of this chapter describes how GDOT determines the required warning signs at curves through two different processes an office planning process and a field curve safety assessment. The second section of this chapter describes the assessment and maintenance of existing curve signs, including the identification of missing signs. Typically, there are three scenarios that trigger a new curve sign installation. They are road construction/network level sign update, safety/operation improvement on the road sections, and installation of required or safety-related signs that are missing.
DETERMINATION OF REQUIRED CURVE WARNING SIGNS
MUTCD requires that curve warning signs be placed at appropriate locations, be appropriately spaced, and have the most appropriate type of sign at each roadway curve to ensure safe conditions. To install curve warning signs that satisfy the MUTCD requirements, GDOT utilizes two different (but complementary) processes to determine curve warning sign locations, 1) using the Curve Advisory Reporting System (CARS)
6

portal (developed by Reiker Inc.), and 2) conducting curve safety assessment in the field. Since CARS allows data processing from an office location, we call this practice the "office planning" part of GDOT's curve safety determination method. In addition to office planning, each GDOT District uses its maintenance crews to perform a "field assessment" to evaluate curve safety and install appropriate curve warning signs.
Office Planning of Curve Sign Installation
Data collection
To determine curve sign locations and sign types for installation in accordance with MUTCD requirements, GDOT first collects the locations (using GPS) and the ball-bank indicator readings of the curves. The ball-bank indicator measures the inclination of the vehicle as it drives through different road alignments. GDOT equips its vehicles with a Rieker Inc. ball-bank indicator (model: RDS7-GPS-PRO) and tablet (as shown in FIGURE 2-1). The ball-bank indicator is mounted on the vehicle's dash, and the GPS antenna is fixed on the roof of the vehicle. After the devices are connected, powered up, and the digital ball-bank indicator calibrated for zero level, the system (as shown in FIGURE 2-1) is ready to use. Then, GDOT personnel drive the vehicle equipped with the ball-bank indicator, GPS, and the data collection tablet on their allocated GDOT routes. The GPS and ball-bank indicator data is recorded continuously as the drivers travel over their allocated routes. The drivers record the posted speed in the tablet system at the beginning of their run or pull over and record the posted speed limit whenever it changes. After completing the run, the driver connects to a Wi-Fi site and uploads the data collected during their session to the CARS server. More details regarding the data
7

collection set-up can be found in Rieker Inc.'s user manual (Rieker Inc., 2018).
FIGURE 2-1. Photo. Rieker Inc.'s ball-bank indicator and the tablet device fitted into a data collection vehicle (Rieker Inc.)
Curve sign determination using CARS for meeting MUTCD requirements The Curve Advisory Reporting System (CARS) is a web tool developed by Rieker Inc. This tool allows a user to 1) extract curve properties, including point of curve, point of tangent, curve radius, length, vertical grade, super elevation at the apex, etc., 2) determine the curve advisory speed using the curve properties extracted, and 3) determine curve warning sign per the MUTCD.
a) Determination of curves in CARS The user selects the data collection session to load the GPS points of the vehicle trajectory onto a map interface. The user zooms and pans through the vehicle trajectory until they visually identify curves on the GPS trajectory. Users select two points on the map to draw a bounding box that encompasses an identified curve, as shown in FIGURE
8

2-2. The GPS points inside the bounding box are highlighted, and the curve is fit through these points. The goodness of fit is evaluated through a "GPS fit" function that quantifies the percentage of the fit (As shown in FIGURE 2-3). If the fit is 95% or greater, the curve is deemed acceptable; if the curve is less than 95%, the user redraws the bounding box until a 95% fit is achieved. The process is repeated for every curve in the data collection session. As pointed out by GDOT engineers, this curve identification process is very time-consuming.
FIGURE 2-2. Photo. Bounding box to encompass a curve
FIGURE 2-3. Photo. Example of GPS fit for the two passes on the same curve 9

b) Determination of curve advisory speed CARS determines the recommended curve advisory speed based on the physical equation defining the motion of the vehicle on a curved roadway described in AASHTO Green Book (AASHTO 2011). Before computing the curve advisory speed, CARS calculates the superelevation (e) based on the ball-bank indicator angle measured in the field, the radius of the curvature (R) is extracted from the curves determined by the previous step and the test speed. Next, using the radius of curvature (R), the maximum allowable side friction force (fmax) and the calculated superelevation (e), the recommended curve advisory speed (Vcas) is calculated using the following formula:
1. = 15( +) For each data collection session, the users identify multiple curves to extract their properties (radius, length, superelevation, etc.) and determine a curve's recommended advisory speed. For each data session, a list of curves with their associated properties and the recommended advisory speed is stored in the CARS web portal. The summary of the items stored in the portal is shown in TABLE 2-1.
10

TABLE 2-1. Curve properties and advisory speed reported in the CARS web portal

S. No 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Curve Property Name Turn Direction (Left/Right) Travel Direction (East, West, North , South) Point of Curvature (Latitude, Longitude) Point of Tangent (Latitude, Longitude) GPS Fit (%) Avg. test speed (mph) Curve radius (feet) Curve length (feet) Deflection angle (degrees) Curve Classification [class] Super Elevation at apex (%) Vertical Grade [class] Min. calculated advisory speed (mph) Recommended advisory speed (mph)

c) Determination of curve signs
The MUTCD requirements for the appropriate curve signs are automatically recommended by the CARS system. The recommendation system takes into account the curve details (point of curve, point of tangent, posted and recommended advisory speeds) to recommend the signs required on the curve. The output of the signs recommended includes sign type, its location (latitude and longitude), distance from the PC of the curve,
11

direction of sign placement, and the location (which side of the road). It must be noted that the recommendation system only recommends "required" signs for the curve if the speed difference between the posted speed and the recommended advisory speed is greater than or equal to a threshold speed of 10 mph. Otherwise, it recommends only "optional" signs when the speed difference is below that threshold (Table 2C-5 of Horizontal Alignment Sign Selection, MUTCD).
d) Report generation for the required curve signs The final output product is a report listing signs with their details (e.g. sign type, location, roadside, facing direction, etc.) and the curve names to which they belong. An example of the curve sign report is shown in FIGURE 2-4.
FIGURE 2-4. Photo. Example of generated curve sign report
12

Field Curve Safety Assessment for Curve Sign Installation
Field assessment does not rely on the CARS portal to determine the curve signs. District maintenance crews routinely drive over the roadways and make engineering judgments based on their knowledge of roadway guidelines (AASHTO, MUTCD, GDOT Signing and Marking Guide, etc.) to proactively assess safety concerns on curves (such as insufficient sight distance, lack of signs, poor pavement condition, missing striping, insufficient superelevation, etc.). GDOT also responds to citizen concerns (for example, people report frequent crashes on specific curves), investigates the locations, and identifies unsafe curves. After the maintenance crews identify the safety concerns on the curves, they notify the GDOT Office of Traffic Operation. The Office of Traffic Operations investigates the reason/issue and evaluates the suitability of signage to resolve the safety concern. The Office of Traffic Operations then approves new signage and the new signs are installed by the maintenance crews.
ASSESSMENT AND MAINTENANCE OF EXISTING CURVE SIGNS
District maintenance crews routinely drive over the roadways and assess the existing signs. Because they are familiar with the roadways, most of the time, GDOT personnel recognize missing signs on the roadway. They also assess the signs for damage and fading. In addition to physical sign condition, they assess the retro-reflectivity condition of the signs by an annual nighttime visual inspection. The maintenance crews are tasked with the upkeep of signage in their areas (including reflectivity); they are responsible for any signs that are damaged/removed. If a sign is knocked down/missing, damaged, or old/faded, the maintenance office will replace the sign.
13

CHALLENGES
1. Challenges in data collection for office planning of curve installation. a) GPS disruption: The data collection session will have to be re-run when the GPS connection is lost in the middle of a session. b) Incorrect ball-bank indicator (BBI) readings: Driving over poor road conditions (such as potholes) causes sudden movement of the vehicle and affects the BBI reading. Moreover, drivers need to take additional effort to drive smoothly in the center of the driving lane because erratic steering will affect a ball-bank indicator reading.
2. Challenges for using CARS. a) Slow operations: CARS is heavy web-based portal and requires considerable time for loading the map tile for every zoom or pan operation. b) Time-consuming to select good quality curve: Users must pan and zoom several times before landing on an acceptable curve bounding box, which is a time-consuming process. The subfigures A, B, and C in FIGURE 2-5 shows the challenging zoom cases (Zoom 1 and Zomm2) before correctly identifying an ideal bounding box around a curve (Zoom 3).
14

A. Subfigure showing Zoom 1 B. Subfigure showing Zoom 2 C. Subfigure showing Zoom 3 FIGURE 2-5. Photos. An example of panning and zooming of images before arriving
at an ideal bounding box
c) Subjective PC and PT: It is challenging for the user to accurately determine the PC and PT, since the visual perception of the curve can vary at different zoom levels.
d) Time-consuming PC, PT determination: To alleviate the challenge, the CARS system offers GPS fitting and allows a user to nudge the PC and PT points to achieve a higher percent fit for the curve on the GPS trajectory. However, the entire process is tedious and time-consuming. For each curve, it takes about 3-5 minutes to achieve good accuracy.
3. Challenges for installing new curve signs in the field for meeting MUTCD requirements. a) Time-consuming to accurately locate PC and PT in the field: The MUTCD requires curve warning signs to be placed at specific locations with respect to the curve; to achieve that, the point of curve (PC) and point of tangent (PT) must be accurate. Currently, in the field, locating PC and PT requires extensive and time-consuming surveys. 15

4. Challenges in assessing existing curve signs. a) It is difficult to perform accurate and comprehensive MUTCD curve sign requirement assessment: Maintenance crews may be able to identify the missing signs on familiar roadways. However, it is difficult for them to accurately and systematically evaluate the compliance of each curve warning sign on every individual curve in the field (e.g. advisory speed, sign types, sign spacing, etc.). For example, to check the compliance of advanced warning sign placement location with respect to the PC, or chevron spacing, the maintenance crews must physically measure the distance between the signs. b) Lack of an accurate curve signs' location inventory: The current field curve sign inventory records the closest milepost (to 0.01 mile) and is used for sign maintenance and replacement. However, it is not accurate or detailed enough for MUTCD curve sign requirements (e.g. sign types, spacing, and locations).
SUMMARY
1. It is necessary to improve the current practices both in the planning in the office (using CARS to install curve signs) and in the field assessment (for identifying the missing signs and assessing existing signs).
2. There is a need to improve the productivity and efficiency of current office planning for MUTCD required curve sign installation practices, including determining the curves and identifying the required sign types, sign locations, sign spacing, etc.
16

3. The current field assessment for comprehensive and accurate assessment of curve warning signs for compliance with MUTCD requirements (e.g., sign types, spacing, and locations) is performed manually and is very time-consuming. There is a need to develop a low-cost, frequent, and automatic method to digitally compare the in-field curve signs and assess the curve signs' compliance with MUTCD requirements.
4. Missing signs are currently identified by district maintenance crews while on their routine inspections. It is time- consuming and challenging for crews to thoroughly assess each curve sign along a route and identify missing signs. Therefore, there is a need to cost-effectively identify missing signs with a systematic and cost-effective approach, such as using automatic curve sign detection and classification, and digitally comparing them with the required curve signs specified in the MUTCD.
17

CHAPTER 3. PROPOSED METHODOLOGY
An innovative and cost-effective alternative for establishing a live curve sign inventory system for meeting MUTCD requirements has been proposed in this Chapter. A costeffective, live curve sign inventory system is essential for identifying the missing curve signs in a timely manner in support of adequate countermeasures for meeting the MUTCD requirement, including sign types and proper spacing.
The automatic curve sign detection and classification method is the core component for developing a cost-effective live curve sign inventory system. The proposed method takes advantage of intra-agency, crowdsourced, low-cost mobile devices to ensure curved roadway safety in a timely, cost-effective, and safe manner. The proposed method leverages previous research outcomes, including automatic sign detection and curve identification, developed in previous research projects sponsored by the United States Department of Transportation (US DOT), FHWA, and GDOT. The proposed method combines 1) the use of intra-agency fleet vehicles 2) crowdsourced, low-cost mobile devices, and 3) the applications of artificial intelligence, including automated curve sign identification and automatic sign detection, to provide a viable solution for a live sustainable curve sign inventory system to ensure curve signs that meet MUTCD standards.
Advances in automatic sign detection using deep learning, computing power, and cameras in mobile phones provide an excellent opportunity to identify and obtain curve sign data cost-effectively. To date, there is no developed method that performs a live curve sign inventory using the combined capabilities of mobile devices and automatic
18

curve sign detection methods with deep learning. The objective of this research is to develop and critically assess a method that uses low-cost mobile devices and leverages AI to establish a live sustainable curve sign inventory that effectively identifies deficient signs, such as missing signs, in a timely manner to support a proactive curve sign improvement method that meets MUTCD requirements.
MUTCD curve sign compliance regulations are one of the curve safety improvement methods for reducing fatal crashes on curves. Most state DOTs and local transportation agencies are installing curve signs that meet MUTCD requirements; they are eager to find a means that will enable them to cost-effectively identify missing curve signs and check on adequate sign types and spacing so transportation agencies can take proactive action to improve curve safety and meet MUTCD requirements.
Because of a lack of resources, transportation agencies typically survey 100% of the roadway network for curve sign inspection annually or bi-annually. Counties and cities that have limited resources will take longer to inspect their roadways for curve sign safety. Thus, needed curve safety improvements are often not identified until crashes occur and are reported to transportation agencies. Because of the long curve sign inspection duration (annually or bi-annually), it is difficult to take proactive actions on curve sign improvement. Therefore, an innovative and cost-effective method that enables transportation agencies to do more with less is required. FIGURE 3-1 illustrates the proposed methodology for sharp curve safety improvement. First, low-cost devices, like smartphones, can be installed in state DOTs' vehicles, similar to methods used by Uber drivers. This method establishes a new intra-agency, crowdsourced data collection framework by leveraging agencies' existing vehicles and transportation engineers; it uses
19

low-cost mobile devices (e.g., smartphones and/or tablet PCs) for collecting multiple runs of sensing data, including GPS data, acceleration, gyroscope, and image data; it collects mobile device sensing data while transportation engineers are undertaking other tasks.
FIGURE 3-1. Flowchart. A proposed low-cost mobile phone-based methodology for curve safety improvement
As indicated previously, it is difficult for transportation agencies to take proactive action on curve sign improvement in a timely manner because of the long curve sign inspection duration (annually or bi-annually). The proposed methodology provides a low-cost means for transportation agencies to perform the preliminary network-level curve sign assessment in a timely manner (daily and weekly). Once the targeted sections (for example, 5%) of curve safety improvement needs are identified, the detailed curve safety assessment can be conducted only on the targeted section. This enables transportation agencies to focus on the detailed inspection on the targeted sections and take proactive countermeasures in a timely manner (daily, weekly, or monthly, rather than annually or biannually). The proposed methodology is aimed at enhancing the current network-level
20

curve safety assessment method, which is labor-intensive, time-consuming, and often dangerous for the engineers collecting the data.
To optimize the data collection effort, the proposed method creatively utilizes intraagency, crowdsourced, low-cost mobile devices. This means data can be collected using an agency's vehicles while its personnel are conducting existing operations. In this way, data can be collected more frequently (e.g., daily, weekly, monthly, or quarterly), instead of using dedicated data collection vehicles annually or biannually. Thus, multiple runs of data from different drivers at a single curve can be collected and analyzed to eliminate biases that could occur in data collected by only a single run. To ensure data quality and ease the concerns about privacy, we suggest using crowdsourcing data collected from the fleet and employees in a single transportation agency, i.e., intra-agency.
This project leverages previous research outcomes from research projects sponsored by the National Cooperative Highway Research Program (NCHRP) Innovation Deserving Exploratory Analysis (IDEA) on "Using Image Pattern Recognition Algorithms for Processing Video Log Images to Enhance Roadway Infrastructure Data Collection" (Tsai & Wang, 2009) and the United State Department of Transportation (US DOT) on "Remote Sensing and GIS-enabled Asset Management System" (Tsai & Wang, 2013). We have also leveraged our previous studies on development and validation of automatic sign detection (Tsai et. al., 2009; Ai & Tsai, 2014; Tsai & Huang, 2012) and classification (Hu & Tsai, 2011) using traditional computer vision and lidar technologies (Ai & Tsai, 2014) to develop an automatic sign detection and classification using deep learning.
This section presents a proposed automatic curve sign detection and classification method that uses deep learning. The development of the method is divided into three
21

components: detection, classification, and clustering. The following section presents detailed steps. However, an in-depth description of deep learning can be found in other references (Redmon et. al. 2016; Redmon & Farhadi 2017; Redmon & Farhadi, 2018), which not be covered in this report. This report focuses on the method's main points.
DETECTION
Detection is the most challenging component of the proposed method because different issues, including luminosity and poor lighting conditions, reflections on the windshield, and poor image quality (e.g. out of focus images or motion blur), impact the detection accuracy. To effectively tackle these issues, a fine-tuned version of YOLO v3, an object detection and classification neural network, is used in the proposed method. In addition, this neural network processes an image only once, making it faster than many other detection neural network models. FIGURE 3-2shows an overview of the YOLO v3 architecture. This illustration shows different convolution steps used by YOLO v3, from the first convolution layer doing the feature detection to the YOLO layer doing the aggregation of the features and predicting the object class and shape passing by the deconvolution, which allows users to propagate the features to smaller-scale detection. More information about this architecture can be found in YOLO v3: An Incremental Improvement (Redmon & Farhadi, 2018).
22

FIGURE 3-2. Illustration. Yolo v3 network architecture Because of the consideration of computation speed, we decided to use this model (a neural network) with only two classes based on the shape of the sign (diamond and rectangular) for curve sign detection. This decision allows us to run detection processes on lower-resolution images, as we don't need the content of the sign to be clearly visible. This makes the whole process run faster. Examples of two classes of curve signs with diamond and rectangular shapes are shown in FIGURE 3-3.
FIGURE 3-3. Photo. Examples of curve sings with diamond and rectangular shapes 23

While maintaining the YOLO v3 architecture, we made three changes to make it more efficient for traffic sign detection. The first change is to get anchor boxes as close to the actual shapes and sizes of the actual signs as possible. In addition, we resized the input of the neural network model to take full advantage of the aspect ratio of the camera without changing the aspect of the signs. We also made the data augmentation more aggressive in terms of luminosity in order to simulate more diverse cases.
The anchor boxes are a crucial point to update because they give hints to YOLO about the size of the objects to detect. The anchors given by the authors of YOLO provide great anchors for detecting things going from train, so those anchors have a wide range of sizes and shapes. In our particular case, we need to detect traffic signs that are mostly square and never occupy most of the image. So, we computed a set of anchors specific to the actual traffic sign we want to detect. In addition, these new anchors are a better fit to the input resolution we target, as they are directly computed for this resolution.
The first step of YOLO is to resize the given input image (usually at 1920x1080) to the network input size of 704x416. However, during this process, no attention is given to the aspect ratio to ensure that the signs can be detected correctly by the neural network model, so we selected the input size of the network to be 704x416. This allows a good detection rate and a speed of 45 frames per second on a laptop with a Nvidia RTX 2060.
The most important thing in building a robust machine-learning model is to carefully choose good quality data for training. We used data from two different sources. The first data set comes from the high-resolution images from Nashville, as shown in FIGURE 3-4(a); the second data set is the smartphone data collected in Georgia, as shown in
24

FIGURE 3-4(b). The first data set provided us with a high, clear view of signs by using a camera outside the vehicle; the second data set provided us with data close to the target application, but it had reflection on the windshield and blur on the images. There are of 8,719 sign images used from these two different data sets. To make sure that our network learns to generalize to diverse cases, we also relied on data augmentation. As each camera has a different color perception, we decided to utilize all the color parameters: hue, saturation, luminosity. This helped our network to generalize the detection to different luminosity cases, as well as different cameras. To further ensure that our network did not learn the dataset by heart, we randomly cropped and zoomed on images to ensure we obtained a different point of view each time.
FIGURE 3-4. Photos. Images from high-resolution images and the smartphone images 25

CLASSIFICATION
The classification step takes care of assigning a MUTCD code to the detected signs. To effectively classify diverse sign types, we chose a deep learning method (MobileNetV2) to manage this task. This network was trained to classify the cropped version of the detected sign. We chose to run the detection on 40x40 images, as this allowed a clear recognition of the sign class while minimizing the computation overhead. To take into account the imprecision from the detection step, we also added data augmentation using zoom in and out, crop and shift, and color change. This model was trained using the same dataset in the detection described previously in Section 3.1, the difference being that only the cropped bounding box was used here in classification. FIGURE 3-5 shows the epochs (iterations) vs. accuracy. It can be seen that the model's accuracy became very high (close to 100%) for both training and testing cases at 250 300 epochs. This shows the proposed strategy produces very good results during training. Epochs refers to a set of iterations in different classes.
FIGURE 3-5. Graph. Training epoch with model accuracy 26

CLUSTERING
A sign is typically displayed in a consecutive set of images. Clustering was used to group the images with the same sign ID. This allowed us to get not only the number of signs on the images, but also the number of signs on the road. This task is handled by a Kalman filter, as it provides good computation speed and accuracy.
27

CHAPTER 4. DATA COLLECTION MODULE
The proposed method makes use of intra-agency fleet vehicles (such as GDOT's vehicle fleet) and low-cost mobile devices (such as smartphones) as the means for effective data collection. The utilization of GDOT's fleet enables data to be collected while engineers work on other tasks. Smartphones provide a powerful, portable, ubiquitous, and inexpensive system that integrates various necessary sensors, including cameras, GPS devices, accelerometers, magnetometers, gyroscopes with processors, and multiple data transfer options. Because of the current advances in smartphones, the accuracy and frequencies of these sensors have been greatly improved.
SMARTPHONE CONFIGURATION
This section discusses the constraints on the proposed data collection module and helps to identify the smartphone configurations necessary for them to be able to collect the data necessary to meet the proposed method's requirements for implementation. Extensive testing was carried out on data collected in Atlanta, Georgia, to establish requirements for the device position, field of view, video data, location data, and inertial data.
Position and Field of View
The main constraint in a smartphone's configuration was clearly the camera's field of view. To capture image data in front of the vehicle, which would be used for sign detection and classification, the smartphone was mounted on the windshield. Traffic signs can be overhead, on the left side (e.g., right curve), and on the right side (e.g., left curve,) of the vehicle. A requirement was set that signs, as a minimum, on the edges of the lane
28

being driven in and the adjacent lanes on both sides (a total of 3 lanes) of the datagathering vehicle should be captured in the field of view of the camera (as shown in FIGURE 4-1). There can, of course, be highways with more than three lanes in a direction. The assumption for this project was made that curves are not usually on that kind of road.
FIGURE 4-1. Illustration. Visualization of 3-lane coverage The field of view (FOV) of the camera can be increased by using lenses with a larger FOV, which would increase the coverage by any one single vehicle. However, for the same sensor array, this results in objects in the image being represented by a smaller proportion of pixels, which hampers detection and classification. Moreover, larger fields of view have more distortion near the edges. This can be corrected, but at the loss of some information. Increasing the field of view by using multiple devices was tested in the field and found not to be very useful; a single device already has a wide FOV, so adding
29

another device on the windshield only slightly improved the FOV. A smartphone with an FOV of approximately 120 degrees horizontally proved to be ideal. A setup instruction was developed to help position the smartphone on the vehicle. A forward-facing device was positioned horizontally at the center of the windshield, which proved to be ideal for providing equal coverage on both sides of the vehicle. The device was placed at the bottom of the windshield (approximately 165 cm from the ground) to minimize interference with the driver's view. FIGURE 4-2 shows the placement of a phone holder inside a vehicle. The device was affixed using a smartphone holder with suction cups. This view successfully captures the sides of the pavement and the pavement itself, so it can capture road signs, and it can detect and translate pavement markings. FIGURE 4-2 shows the location of the device on the windshield. A detailed smartphone setup manual (Appendix A) was created to aid in the installation of the system.
FIGURE 4-2. Photos. Different views of smartphone installation inside a vehicle
30

Video
The video makes up the largest portion of the data collected. It was necessary to balance video quality, which is required for better accuracy of sign detection and classification and video size. H.264 is a suitable video compression standard. The MPEG-4 container format was used to store video data. The requirement that a sign be detected and correctly classified even if it occupies 0.1% of the image was set. FIGURE 4-3 gives some perspective on the minimum size of a sign within the image that can be accurately detected and classified. Given this approach to detecting and classifying signs, it is recommended that at least a 40x40 pixel color image representation of a sign be used. Thus, to accurately detect and classify signs, a resolution of 1920x1080 was found to be adequate. This provides a target size to a picture size ratio of 0.12% or higher. Signs are easier to detect and classify when they are closer to the vehicle. However, the images with larger signs also move faster when they are closer to sign objects. This adds constraints to the minimum frame rate of video to be captured. At least 10 frames per second is recommended to accurately capture signs as they come closer to the vehicle. Audio was not captured. Autofocus features in the camera API would sometimes focus on the windshield or fluctuate repeatedly. Since the interest was upon objects far away, the autofocus feature was turned off, and the object distance was locked to infinity.
31

FIGURE 4-3. Photo. An example of a 40x40 sign in a 1920x1080 image
Location
Modern smartphones can combine location data from a variety of sources to deliver a precise location estimate. However, only GPS is free from cellular signal and wi-fi availability requirements. GPS points are collected at a frequency of 1Hz. A modern smartphone's GPS can achieve up to 30 cm accuracy in determining location, especially at constant speeds, which is suitable for the proposed application. The location data includes latitude and longitude in the world geographic system (WGS), along with altitude and heading.
Inertia
Accelerometers, magnetometers, and gyroscopes collect linear acceleration, angular orientation, and angular velocity in three directions. Magnetometer data of at least 1Hz are required to assign to the location data. Accelerometer and gyroscope data do not play a direct role in sign detection and classification. However, in the future, they can be used
32

to refine location measurements and to estimate the roughness of roads. In the current implementation, these sensors collect data at 50Hz.
"AllGather" APP Design of "All Gather" App
"All Gather," which was originally developed for other sponsored projects, was refined to be used in this project. "All Gather" is an Android and IOs app that collects smartphone data according to the requirements listed in the previous section. The Android Open Source Project makes it easy to port the implementation to other smartphones or custom devices. The current implementation targets Android 8.0 (Oreo). It is compatible with devices running Android 5.0 (Lollipop) or higher. The app is not published in app stores. It can be installed via manually shared APKs. FIGURE 4-4. Photo. UI of "All Gather" shows the main UI of "All Gather." The green line in the middle is to help users align the phone so that it is straight (no tilt). The remaining storage space and battery life are displayed on the top. An operation manual (Appendix B) for "All Gather" was created to assist users in installing and operating "All Gather" on Android devices.
FIGURE 4-4. Photo. UI of "All Gather"
33

CHAPTER 5. CASE STUDIES ON AUTOMATIC CURVE SIGN DETECTION AND CLASSIFICATION USING SMARTPHONE DATA
This section presents a case study to validate the proposed method. It uses 13 centerline miles of roadway on State Route 2 in Georgia to critically validate the accuracy of detecting and classifying curve signs. The following describes the test site on SR 2, data collection, data processing, and outcomes.
CASE STUDY ON STATE ROUTE 2 TEST SITE
FIGURE 5-1shows the State Route 2 (SR2) test site. The selected portion of SR2 consists of 13 centerline miles of roadway between Hiawassee and Clayton, Georgia. This portion of SR 2 is mountainous and has many curves. In addition, SR2 has already been part of multiple joint studies between Georgia Tech and GDOT, all of them focused on curve safety improvement (including application of high friction surface treatment (HFST)).
34

FIGURE 5-1. Map. Thirteen center line miles of test route on State Route 2 in Georgia (mountain area)
Data collection
The field data collection using smartphones was conducted May 2, 2019, on both directions of the 13 miles of SR 2 (a total of 26 survey miles for each vehicle and smartphone). Data collection was carried out using three different vehicles, including one GDOT vehicle, one Georgia Tech vehicle, and one personal vehicle. Each vehicle was equipped with one smartphone for data collection. FIGURE 5-2 (Subfigure A) shows the GDOT truck. The data collected by the GDOT truck is used for the validation of automatic curve sign detection and classification. This image data was collected by GDOT District 1 engineers. FIGURE 5-2(Subfigure B) shows a smartphone used for testing. The phone is a regular Android smartphone, which can take 30 frames per second
35

(FPS) with an image resolution of 1920 * 1080.
FIGURE 5-2. Photos. GDOT truck and smartphone used for field data collection
Performance measures
Validation was conducted by manually reviewing the collected images to verify whether or not a sign in a particular image was detected by the automatic sign extraction method. The verification outcomes were categorized into true positive (TP), true negative (TF), false positive (FP), or false negative (FN). TABLE 5-1. Definitions of TP, TN, FN, and FPTABLE 5-1 illustrates the meaning of TP, TN, FN, and FP. A TP shows the pixels (i.e., a sign in an image) correctly detected as foreground objects (or signs); this means a sign has been correctly detected in an image. A TN shows pixels with no sign correctly classified as background (i.e., no sign). An FP shows pixels with no sign incorrectly classified as foreground objects; this means the method falsely detected signs. An FN shows pixels with a sign incorrectly classified as background; this means a sign was not detected (completely missed). In practice, transportation agencies are interested in detecting individual signs instead of all images containing signs; thus, the outcomes are reported at the sign level. A sign typically appears in several images; however, it does not need to be detected in every image. It is categorized as TP if it is detected in at least one
36

image. A sign is categorized as FN if it is missed (not detected) in all the images in which it appears. TN is not reported in this project.

TABLE 5-1. Definitions of TP, TN, FN, and FP

Detected Sign

No detected sign

Actual sign

True Positive (TP) False Negative (FN)

No actual sign

False Positive (FP) True Negative (TN)

precision

=

TP TP+FP

---------------------------

(1)

After identifying the TP, FN, and FP cases, recall and precision are computed to evaluate

recall

=

TP (TP+FN)

------------------------------

(2)

the performance of the sign extraction method. Precision, as shown in Equation 1, represents the percentage of signs that are correctly detected among all the detected signs. A high precision rate means the automatic sign extraction method can effectively detect the signs. A recall, as shown in Equation 2, represents the percentage of signs that can be detected among all the individual signs in the images. A high recall rate, like 100%, means all the percentage of signs that can be detected without missing any of them; 100% means all signs were detected.
Test Outcomes
There were 65,340 frames processed at a speed of 28 frames per second. The videos collected on SR2 were processed using the developed curve sign detection and classification method. The videos were manually reviewed by our research team to
37

identify any FN cases (i.e., signs that were not detected).

TABLE 5-2 shows the test outcomes. The outcomes show an overall recall of 100% (with a TP of 471 and a FN of 0). The overall precision is 92% (93% for eastbound and 92% for westbound lanes). Results show 0 FN cases. All visible signs are successfully detected. There were 242 signs detected eastbound (EB), and 229 west-bound (WB). The accuracy was good, having no FN cases and less than 5% FP with 19 cases. There were 8 FP cases for EB data and 11 cases for WB data. Of those signs, 14 were orange work zone signs, 4 were yellow advertisement signs, and one was a part of a yellow digging machine. Figures 5-3 (a) and (b) show the TP examples. One is in a good condition, while the other shows reflection in the windshield. Figures 5-4 (a), (b), and (c) show the FP examples, including diamond work zone signs with orange and yellow colors, and advertisement signs.

Run EB WB Total

TABLE 5-2. Test outcomes on SR2

# of images

TP

FN

FP Recall

32730

242

0

8

100%

32610

229

0

11 100%

65340

471

0

19 100%

Precision 97% 95% 96%

The label on the bounding box shown in Figure 5-3 is defined by the following template: <detected MUTCD code> (<detection confidence>%|<classification confidence>%). Where "detected MUTCD code" is the MUTCD code given by our algorithm during
38

classification, "detection confidence" is a number between 0 and 100 giving the confidence of our algorithm in the computed bounding boxes, and "classification confidence" is a number from 0 to 100 given by our algorithm to show how sure the algorithm is of the result; in both cases, the higher value has better confidence in the results. Different colors of the bounding boxes represent different sign classes.
Figure 5-3. Photos. Examples of True Positives (TP) As can be seen in the above images, work zone signs, which are not curve signs, are detected, and we consider these work zone detections as FP cases. After discussing this issue with GDOT engineers, we concluded that work zone traffic signs should be recognized as another sign category in future data collections. We decided to classify these signs as a "work zone" sign class instead of trying to stop detecting it.
39

FP reduction is the next major improvement in the detection to work on. We will now provide more detailed examples of FP cases. First, the most common case is the work zone signs that are detected and then classified as curve signs. The last example of FP we encountered is an advertising sign. The large yellow areas on sign poles are often detected as curve signs. Even if it's only for a few frames and with low confidence, we are going to add new data into the algorithm to help prevent those kinds of FP cases in the future. Figure 5-4 shows the examples of the three false positive cases described above.
40

Figure 5-4. Photos. Examples of False Positives (FP) The test outcomes have demonstrated that the proposed method is a very promising means to accurately detect curve signs. The research outcomes will enable transportation agencies to develop a cost-effective means to automatically detect curve signs using lowcost mobile devices (smartphones) to establish a live curve sign inventory method.
41

SUMMARY
Through this proof of concept demo, the outcomes show the proposed method is very promising to accurately detect and classify curve signs. The research outcomes will enable transportation agencies to develop a cost-effective means for curve safety improvement in a live and sustainable curve sign inventory using low-cost mobile devices, like smartphones, and other embedded mobile devices.
42

CHAPTER 6. CASE STUDIES ON AUTOMATIC CURVE SIGN DETECTION AND CLASSIFICATION USING SMARTPHONE DATA
This chapter discusses the Phase 2 implementation plan with potential benefits. A roadmap is presented for implementing the curve sign detection and classification for establishing a live sustainable curve sign inventory.
ROAD MAP
The ultimate goal of this research project, including Phases I and II (a proof of concept) is to develop a cost-effective live curve sign inventory system, including methods and methodologies that enable transportation agencies to meet MUTCD curve sign compliance requirement and to reduce current 25% fatalities and crashes on the curved road sections. The proposed live curve sign inventory system will enable transportation agencies 1) to inventory the presence of curve signs, and 2) to identify the missing curve signs, especially the advisory speed limit signs, cost-effectively and timely so the countermeasures (e.g. sign replacement and modification) could be applied before the accidents occur. The curve sign detection and classification are the core components in the proposed live curve sign inventory system.
The proposed method consists of seven steps: a) adjusting hardware configuration to maximize the automatic curve sign detection; b) collecting sensor data using low-cost
43

mobile devices; c) automatically computing curves, including curve location, radius, point of curve (PC), and point of tangent (PT); d) refining (training) the AI algorithms to better detect curve signs and recognizing its MUTCD code; e) computing a sign's location/coordinates using GPS data and camera calibration parameters; f) generating sign inventory with sign type (MUTCD code), location (x-y coordinates), and images; g) reducing images for data storage and management; and h) exploring curve sign applications (e.g., comparing curve sign inventory with the sign requirements to identify sign deficiencies and using different timestamps to identify changes).
The research project is a proof of concept research project and is divided into two phases (Phases I and II). Phase I, which is this proposed research, consists of Steps a) to d), focusing on refining the AI algorithms for automatically collecting curve signs using lowcost mobile devices on selected test routes. Phase II consists of Steps e) to h), focusing on enhancing curve sign location accuracy computation, data management, and exploration of potential applications. In addition, a pilot test (e.g., selected counties or districts) will be conducted in Phase II for preparing statewide implementation.
This section also presents the implementation of Phase I outcomes. First, the preliminary test outcomes on State Route 2 have shown the promising outcomes on curve sign detection and classification. In Phase II, we plan to test the curve sign detection and classification using a large data set, including the city and county streets with poor curve sign conditions. Second, the smartphone data collection module has been developed. It includes smartphone setup manual and installation and data collection manual, shown in Appendixes A and B. They can be tested and implemented by GDOT engineers. We suggest that the smartphone-based data collection module be tested by District 1 engineers.
44

Figure 6-1 shows a roadmap for validating, refining, and implementing a cost-effective live, sustainable curve sign system. It includes three stages. The stage 1 is a proof of concept which include the phases I and II of this research project. They are to complete seven steps of the tasks described above. Stage 2 is a pilot study, which is to test the developed and validated method in a selected county and also in a selected county or city. Stage 3 is a large-scale implementation, which is to deploy the developed solution to one district through Accelerated Innovation Deployment (AID). The key is to make the developed solution scalable on data acquisition, processing, and management.
Figure 6-1. Flowchart. A roadmap for validating, refining, and implementing a cost-effective live curve sign system to ensure roadway safety
45

POTENTIAL BENEFITS
This section presents the potential benefits of implementing the proposed method of establishing a live curve sign inventory for assessing the curve signs for meeting MUTCD requirements.
First, there is no standardized, streamlined, cost-effective operation procedure to check the MUTCD-compliance curve sign requirement periodically (say weekly or monthly), including the missing curve signs, inadequate/substandard curve sign (such as inadequate spacing, etc.) for a timely curve sign replacement and maintenance.
The potential benefits for GDOT include two issues. One is the operational cost savings for GDOT, and the other is the potential reduction of fatal crashes on curves. For MUTCD-compliance curve sign assessment operation, transportation agencies will require more frequent curve sign assessment (e.g. weekly, monthly, quarterly, etc.). The following is the estimated cost and possible saving based on GDOT's current manual operation.
The detailed GDOT practices for identifying missing curve signs are documented in the Chapter 2. As the proposed live curve sign inventory system is able to assess the curve sign condition constantly (e.g. daily, weekly, monthly, quarterly, etc.), the future GDOT operation cost saving is estimated based on the effort needed for a monthly curve signs assessment for identifying the missing signs in a timely manner for meeting the MUTD requirements.
The Maintenance Office is responsible for identifying and replacing the missing signs.
46

The maintenance crews are tasked with the upkeep of signage in their areas; they are responsible for any signs that are damaged/removed. If a sign is knocked down/missing, damaged or old/faded, the maintenance office will replace the sign. The following is the estimation of direct costs (labor and vehicle costs) for assessing the curve signs calculated for an Area Office:
a) Maintenance crew member's remuneration: $18/hr (assumes an average of two payroll levels junior and senior engineers)
b) Number of crew members involved in an assessment: 2 c) Time spent by the crew members on assessing the curve signs: 35 hrs per
assessment in an area (assumes GDOT District 1, Area 4, covering 520 centerline miles). A maintenance crew needs to drive 1040 lane miles for curve sign assessment alone. Driving at 30 miles per hour requires 35 hours on the road for assessment. d) Number of assessments = 12 per year
Total crew member cost: (a) x (b) x (c) x (d) = $18/hr x 2 members x 35 hrs x 12 = $15,120
e) Truck fuel cost: $0.125/ mile (assumes a conservative 20 mpg fuel efficiency and fuel cost = $2.5 per gallon)
f) Total assessment miles: 12,480 miles (assumes 12 times of the year assessment: 1040 x 12 miles)
Total vehicle cost: (d) x (e) = $0.125/ mile x 12,480 miles = $1,560
47

g) Yearly crew member training (especially for the sign assessment): $400 ( assumes there are 8 crew members and $50 per training each person)
h) Truck maintenance: $400 (yearly maintenance, assumes minimal damage) Total cost: $17,580 per Area. Now expanding the same to entire GDOT District 1, which comprises 4 Area Offices, the total cost for sign maintenance alone is $70,320 per year. Extending the same cost to all districts, the total cost incurred to GDOT per year on curve sign maintenance alone = $492,240. Nearly half a million dollars need to be spent on curve sign assessment across Georgia to ensure the curve signs are as per MUTCD requirements.
48

CHAPTER 7. CONCLUSIONS AND RECOMMENDATIONS
A disproportionally high number of serious vehicle crashes (25% of fatal crashes) occur on horizontal curves (FHWA, 2019), even though curves represent only a fraction of the roadway network (5% of highway miles) (FHWA, 2016). To reduce fatal crashes, the MUTCD (Manual on the Uniform Traffic Control Devices) (FHWA, 2012) requires various horizontal alignment warning signs (curve signs) to ensure curved roadway safety. However, current transportation agencies' practices for inventorying the locations and types of existing curve signs are largely a manual procedure, which is costly, laborintensive and time-consuming. A cost-effective live curve sign inventory methodology and system is proposed for meeting MUTCD requirements using intra-agency, low-cost mobile devices (e.g. smartphones), existing vehicles, and deep learning and crowdsourcing technologies.
Curve sign detection is one of the most critical components in the proposed methodology and system. Phase I of this research is to critically validate a developed curve sign detection and classification method that uses low-cost mobile devices (smartphones) and deep learning technologies. A case study, using 13 centerline miles of roadway on State Route 2, consisting of 471 curve signs, in Georgia, was conducted to critically validate the accuracy of detecting and classifying curve signs using the proposed method. Results show 471 curve signs were correctly detected. The case study clearly demonstrates that the proposed automatic curve sign detection and classification method using deep learning is very promising for implementation.
49

The field data collection module using smartphones has been developed and is ready for implementation. The detailed `AllGather' data collection module implementation manual using smartphone shown in Appendixes A and B. A roadmap for validating, refining, and implementing a cost-effective live and sustainable curve sign system is also presented. It includes three stages (a proof of concept, a pilot study and a large-scale implementation). The ultimate goal of this research project, including Phases I and II (a proof of concept) is to critically validate a cost-effective live curve sign inventory system, including methods and methodologies that enable transportation agencies to meet MUTCD curve sign compliance requirement and to reduce current 25% fatalities and crashes on the curved road sections.
With the promising outcomes of curve sign detection and classification using deep learning and low-cost mobile devices, the following recommendations are made for future research in Phase II of this research project:
1) Classifying work zone signs into a new "work zone sign" category is recommended.
2) Computing sign locations using various methods, such as camera calibration, is recommended so the location of the detected signs can be computed and visualized on a GIS map.
3) Further study of data reduction and data management is needed. Effectively managing data is a key issue for the implementation of the proposed method. Different possible alternatives, including storing only sign type and sign location or with an additional sign bounding box for confirmation, need to be explored. 50

4) Link and cross-check the outcomes of automatic curve sign detection and sign location computation with the required MUTCD sign types and spacing to develop a live curve sign condition assessment.
5) An implementation roadmap is needed for deploying the developed low-cost technologies to state DOTs and local transportation agencies (counties and cities).
6) It is recommended to develop an iOS application of the 'AllGather' application for Apple phones which is predominately used by GDOT engineers.
A roadmap for validating, refining, and implementing a cost-effective live and sustainable curve sign system is also presented. It includes three stages (a proof of concept, a pilot study and a large-scale implementation). The ultimate goal of this research project, including Phases I and II (a proof of concept) is to critically validate a cost-effective live curve sign inventory system, including methods and methodologies that enable transportation agencies to meet MUTCD curve sign compliance requirement and to reduce current 25% fatalities and crashes on the curved road sections.
51

ACKNOWLEDGMENTS
We would like to thank the Georgia Department of Transportation (GDOT) for its support. The work conducted in this report was sponsored by the GDOT Office of Performance-Based Management and Research (Research Project 18-18). We would like to thank Mr. Andrew Heath and Mr. Michael Turpeau from the Office of Traffic Operations for their strong support and heavy involvement in the project. We would like to thank Mr. David Jared and Mr. Binh Bui from the Office of Performance-Based Management and Research. We would also like to thank Mr. Carlos Baker and Ms. Kesha Wynn from the Office of Traffic Operations and Mr. Brandon Kirby, Mr. Shane Giles, Mr. Jonathan Peevy, Ms. Natasha Davis, Ms. Jessica Garcia, and Ms. Jessica Suarez from District 1 for their support in the field tests and understanding the curve sign installation and field assessment of missing signs. We would also like to thank the members of the research team at the Georgia Institute of Technology (Georgia Tech), including Anirban Chatterjee, Murali G Sethuraman, Tianqi Liu, and Don Kushan Saminda Wijeratne for their diligent work on data collection, processing, and analysis in this research project.
52

REFERENCES
AASHTO (2011) A Policy on Geometric Design of Highways and Streets. The American Association of State Highway and Transportation Officials, AASHTO Green Book, Washington DC. Ai, C. and Tsai, Y. (2014) "Critical Assessment of an Enhanced Traffic Sign Detection Method Using Mobile LiDAR and INS Technologies." Journal of Transportation Engineering (10.1061/(ASCE)TE.1943-5436.0000760). FHWA (2012). Manual of Uniform Traffic Control Devices (MUTCD). Federal Highway Administration, U.S. Department of Transportation. 2009 Edition. FHWA (2016). Pavement Friction. Accessed December 2019 from https://safety.fhwa.dot.gov/roadway_dept/pavement_friction/ FHWA (2019). Horizontal Curve Safety. Accessed December 2019 from https://safety.fhwa.dot.gov/roadway_dept/countermeasures/horicurves Hu, Z. and Tsai, Y. (2011) "Image Recognition Model for Developing a Sign Inventory" ASCE Journal of Computing in Civil Engineering, Vol. 25(2): 149-158. Rieker Inc (2018) CARS User's Manual. Accessed June 2019 from http://www.riekersolutions.com/NewsPanel/CARS_User's_Manual.pdf Rieker Inc, Curve Advisory Reporting System, Accessed June 2019 from https://www.riekerinc.com/wp-content/uploads/Documents/CARS-PRO-Broch1.pdf
53

Tsai, Y. (2009) "Using Image Pattern Recognition Algorithms for Processing Video Log Images to Enhance Roadway Infrastructure Data Collection", National Academy of Science NCHRP IDEA-121 Final Report. Tsai, Y., Kim*, P., Wang, Z. (2009) "A Generalized Image Detection Model for Developing a Sign Inventory", ASCE Journal of Computing in Civil Engineering, Vol. 23(5):266 276. Tsai, Y., Huang, Y. (2012) "A Generalized Framework for Parallelizing Traffic Sign Detection of Video Log Images Using Multi-Core Processors", Computer-Aided Civil and Infrastructure Engineering, 27(7), 476-493. Tsai, Y. and Wang, Z. (2013) "Remote Sensing and GIS-enabled Asset Management System (RS-GAMS)", Final Report, USDOT RITA Program Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2016) "You only look once: Unified, real-time object detection." Proceedings of the IEEE conference on computer vision and pattern recognition. Redmon, J., and Farhadi, A. (2017) "YOLO9000: better, faster, stronger." Proceedings of the IEEE conference on computer vision and pattern recognition. Redmon, J., and Farhadi, A. (2018) "Yolov3: An incremental improvement." arXiv preprint arXiv:1804.02767.
54

APPENDIX A. INSTALLATION GUIDE FOR "AllGather"
A1 ONLINE INSTALLATION A1-1 Overview This manual explains the following:
Installation of the "All Gather" Data Collection Android app on Android devices Setup of the device on vehicles Operation procedure of the data collection app
A1-2 Requirements Below lists the required hardware:
Android Smartphone. Verified models: Xiaomi and Redmi Note 4, Google Pixel 2, Google Pixel 3a. (a verified and tested Android device that can be expanded to other devices later)
App installer file (currently Allgather-v1-hq.apk or Allgather-v1-lq.apk; updated versions may have different file names)
Mini-USB cable to transfer the installer onto the device SD card (recommended) Xiaomi Redmi Note 4 SD card slot pin (recommended) Car holder 6ft (~2m) Mini-USB charging cable Car Charger SD card reader (recommended) Internet connection through Wi-Fi (for initial setup only)
55

6 ft (~2m) high stick (for initial setup only) A1-3 Software Setup Instructions Note: For different Android Build Version, the UI interface and the setting options may vary. The "online install method" in Appendix C offers a relatively stable way to install "All Gather," but it requires internet connection for downloading the APK file.
1. Transfer Allgather-v3-7d/e.apk or higher to an Android (5.0 or newer) smartphone. The recommended phone is a model with dual band GPS (currently the Xiaomi X8 is the only major phone with this feature). 1. To transfer, connect the smartphone to a PC/Mac via USB cable. 2. A notification will pop up on the phone for the USB that generally says "USB Connected for charging. Tap for Options." Tap the notification and select "File Transfer." The phone will now show up as another drive on a Mac/PC. 3. Copy the .apk and paste it into "<phone's drive>/Internal Storage/Downloads/"
2. The phone will generally have a Files app or a Downloads app. Open it and find the .apk on the phone.
56

FIGURE A-1. Photo. Downloaded APK File 3. Tap the .apk to install it. You may be asked to change a setting to allow the
installation of apps from third-party sources (i.e. not through Google Play Store). Enable that in Settings according to the instructions.
57

4. Once installed, you can open the app called "All Gather."
FIGURE A-2. Photo. App installed window 5. You will be prompted to provide permissions to "All Gather":
1. Storage: to save data 2. Camera: to collect data 3. Location: to collect data 4. Phone: to obtain the IMEI of the phone (useful to identify which device
collected the data when there are multiple devices)
58

FIGURE A-3. Photo. Picture and video permission pop-up 6. Enter a 6-8 digit driver ID to uniquely identify the data collection personnel.
FIGURE A-4. Photo. User ID window 59

7. All Gather is now ready to be used. Tap anywhere on the video feed or on the play button to start recording. Tap again to stop.
FIGURE A-5. Photo. Main window (recording not started)
FIGURE A-6. Photo. Main window (recording started) 8. Data is saved to either of the following:
60

1. <SD card>/Android/data/edu.gatech.ce.allgather/ 2. <internal storage>/Android/data/edu.gatech.ce.allgather/ 9. To transfer data, use EITHER ONE of the following approaches: Using export tool:
1. Connect phone to Windows PC using USB. 2. Tap the notification that appears; choose "Transfer files (FTP)." 3. Run the export tool Windows Desktop application. 4. Choose the drive letter that represents the phone. 5. Click extract. 6. Choose a folder to dump the data into. 7. Wait until you receive the message saying the transfer is complete. Extract data manually: 1. Connect the phone to the computer via USB. 2. Tap the notification that will appear; choose "Transfer files (FTP)." 3. Navigate to the folders listed in Step 8 to find the collected data. 4. Copy and paste the data to your computer.
61

A2: ONLINE INSTALLTION A2-1 Overview This manual explains the following:
1. Installation of the "All Gather" Data Collection Android app on Android devices 2. Setup of the device on vehicles 3. Operation procedure of the data collection app A2-2 Requirements Below lists the required hardware: Android Smartphone, the following models are verified: Xiaomi Red Note 4,
Google Pixel 3a, Google Pixel 2. App installer file downloadable from:
https://drive.google.com/open?id=19wwPtsTZjJxABdd1-L5exCp3LTFVCVv SD card (recommended) Car holder Car Charger SD card reader (recommended) Internet connection through Wi-Fi (for initial setup only) 6 ft (~2m) high stick (for initial setup only) A2-3 Software Setup 1. Uninstall "All Gather" If already installed
Press "All Gather" on the screen and wait for the garbage bin appear. Drag "All Gather" over the garbage bin.
62

2. Enable High Accuracy GPS, Open your device's Settings app. Tap "Security & Location." Then tap "Mode" and pick "High accuracy: Use GPS, Wi-Fi, mobile networks, and sensors to get the most accurate location."
3. To install "All Gather," click or open the link below from your Android smartphone: https://drive.google.com/open?id=19wwPtsTZjJxABdd1-L5exCp-3LTFVCVv
FIGURE A-7. Photo. "Open with" window This will prompt an "Open with" window; select "Package installer." Then tap "install."
FIGURE A-8. Photo. "Installation" window Once installed, open "All Gather" and you will be prompted to provide permissions to "All Gather". The rest installation steps will be the same as the offline installation setup as described in Appendix A1-3 step 5 to 9.
63

APPENDIX B. SMARTPHONE SETUP, DATA COLLECTION, AND DATA TRANSFER PROCEDURES
B-1 Hardware Setup 1. Insert the device into the car holder. 2. During data collection, keep charging the device using a USB cable.
B-2 Operation 1. Unlock the Android device. 2. Open the "All Gather" app. 3. Tap anywhere on the video or the play button to start recording. The interface will turn red when the app is recording. 4. Collect the data. 5. Tap anywhere on the video or the play button to stop recording. The interface will turn back to blue. 6. Once the data collection is complete, use the SD card slot pin to swap the SD card from the device. 7. Transfer the contents of the SD card onto a computer using the SD card reader and the Export tool. The files can also be found manually in "SD card://android/data/edu.gatech.ce.allgather/files/Documents/." 8. Avoid the following situations when installing the phone holder (see FIGURE B-1. Photos. Incorrect smartphone placements). Phone holder in the view (blocks the signs on the left and right) Rear mirror in the view Dashboard in the view
64

Tinted windshield in the view
FIGURE B-1. Photos. Incorrect smartphone placements 65

SMARTPHONE PLACEMENT PROCEDURES: 1. First, clean the lens of the phone's camera; use a microfilm cloth. 2. Second, clean the windshield of the vehicle inside and outside. 3. Clamp the phone onto the phone holder.
FIGURE B-2. Photos. Phone holder usage 4. The clamp at the bottom can be shifted to one side so that the USB cable can be
connected. 5. The phone can now be attached to the windshield. Two placement options have
been shown below:
66

a. Near dashboard
FIGURE B-3. Photo. Near dashboard placement 67

b. Near the rear view mirror
FIGURE B-4. Photo. Near rear view placement
FIGURE B-5. Photo. Rotate to correct orientation Landscape orientation is highly recommended for data collection. To set up the phone in this orientation, open the "All Gather" app and ensure that the lock orientation option is OFF. The green lines can be used to ensure that the phone is vertical: the green lines will
68

line up when the phone is perfectly vertical.
FIGURE B-6. Photos. Vertical alignment The images below can be used for reference for setting up the phone. The suction cup can be locked by turning the tab behind it as shown.
FIGURE B-7. Photo. Vertical Alignment 69

Ensure the following when setting up the phone: i. The dashboard should not be visible. ii. The view should not be obstructed by the phone holder.
iii. The windshield in front of the phone should be clean and free of obstructions.
70