Development of an automated pavement crack sealing system: final report [Dec. 2010]

GEORGIA DOT RESEARCH PROJECT 2047 FINAL REPORT
DEVELOPMENT OF AN AUTOMATED PAVEMENT CRACK SEALING SYSTEM
OFFICE OF MATERIALS AND RESEARCH RESEARCH AND DEVELOPMENT BRANCH

Contract Research GDOT Research Project No. 2047
Final Report DEVELOPMENT OF AN AUTOMATED PAVEMENT CRACK SEALING SYSTEM
By Jonathan Holmes Research Engineer I Georgia Tech Research Institute
of the Georgia Institute of Technology
Contract with Department of Transportation
State of Georgia In cooperation with U.S. Department of Transportation Federal Highway Administration
December, 2010 The contents of this report reflect the views of the author(s) who is (are) responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the Department of Transportation of the State of Georgia or the Federal Highway Administration. This
report does not constitute a standard, specification, or regulation.

TECHNICAL REPORT STANDARD TITLE PAGE

1.Report No.: FHWA-GA-10-2047

2. Government Accession No.:

3. Recipient's Catalog No.:

4. Title and Subtitle: Development of an Automated Pavement Crack Sealing System

5. Report Date: December 2010
6. Performing Organization Code:

7. Author(s): Jonathan Holmes, Wiley Holcombe, Wayne Daley, Colin Usher, Steven Robertson

8. Performing Organization. Report No.: 2047

9. Performing Organization Name and Address: Georgia Tech Research Institute Aerospace, Transportation & Advanced Systems Laboratory 640 Strong St. NW Atlanta, GA 30332-0823
12. Sponsoring Agency Name and Address: Georgia Department of Transportation Office of Materials & Research 15 Kennedy Drive Forest Park, GA 30297-2534

10. Work Unit No.:
11. Contract or Grant No.: SPR00-0005-00(981)
13. Type of Report and Period Covered: Final; November 2003 December 2010
14. Sponsoring Agency Code:

15. Supplementary Notes: Prepared in cooperation with the U.S. Department of Transportation, Federal Highway Administration.

16. Abstract: Pavement crack sealing operations remain predominantly manual due to the challenges associated with automation. The research performed by the Georgia Tech Research Institute in conjunction with the Georgia Department of Transportation has proved in many ways that a commercial-scale automated crack sealing system is viable. Solutions related to the high-speed firing of nozzles, automated crack detection, and navigation in a real-time system have been demonstrated on a limited-scale system.

Additional work remains on the testing of longitudinal crack sealing solutions, and fine tuning of crack detection algorithms. Once these issues have been properly addressed, the remaining tasks will primarily be associated with scaling the system from 12" of width to a full-lane width. The future of automated crack sealing operations is promising as this research has demonstrated that the technical barriers to commercialization have been addressed, thus opening the door for increases in productivity and worker safety.

17. Key Words: Crack sealing, crack detection, automated maintenance

18. Distribution Statement:

19. Security Classification (of this report): Unclassified
Form DOT 1700.7 (8-69)

20. Security Classification 21. Number of Pages:

(of this page):

Unclassified

68

22. Price:

Acknowledgements
We would like to acknowledge and extend our thanks to the following persons who have made the completion of this project possible: Marlon Moses for making the initial presentation to GDOT on the project concept. Buddy Gratton for showing initial interest in the concept and championing the funding of the project. Bill Evans for providing early support and scheduling several road trials with traffic control for the initial pavement image collection sessions. Additional GDOT staff that have supported the crack sealing project including Binh Bui, Rick Deaver, Georgene Geary, and Don Wishon. Their support has been crucial during all phases of the work. The GTRI project team that has included a number of research staff throughout the length of the project: Dr. Victor Breedveld, Dr. Doug Britton, Dr. Jim Clark, Dr. Wayne Daley, Sim Harbert, Wiley Holcombe, Jonathan Holmes, Jake Leverett, Marlon Moses, Steve Robertson, Colin Usher, Robert Wallace, and several student employees. Our Technical Monitor, David Jared, for his frequent assistance with project deliverables and his continued support throughout the entirety of the project. His support persisted through a number of contract modifications and phases.
i

Executive Summary
Pavement crack sealing operations remain predominantly manual due to the challenges associated with automation. The research performed by the Georgia Tech Research Institute in conjunction with the Georgia Department of Transportation has proved in many ways that a commercial-scale automated crack sealing system is viable. Solutions related to the high-speed firing of nozzles, automated crack detection, and navigation in a real-time system have been demonstrated on a 12" wide limited-scale system.
This document describes the initial development of the crack sealing system, which included several iterations of image processing techniques as well as bituminous asphalt dispensing devices. Also, the design of the final prototype system is described in detail from the image capture system, processing algorithms, and dispensing techniques.
Lastly, a number of results were discussed from various trials that took place including several road trials for tuning of image processing techniques as well as evaluation of the prototype system. The final evaluation of the crack detection system, which included grading by a number of judges, resulted in detection of 83% of cracks at a level of 90% or better with 15% of false positive responses where 90% or better refers to capturing 90% or more of the crack in a given crack map. The 83% result was based on a number of images gathered on a particular road trial on GA 100 in March of 2010 where more than 30,000 images were captured. Crack detection performance was found to vary somewhat on different surfaces in later tests, but not formally characterized as the GA 100 tests were. Performance of dispensing was also evaluated in a number of full system trials. Issues with dispensing on the longitudinal servo axis prevented full evaluation of that part of the system, but servo tracking was demonstrated successfully. Dispensing in cracks proved challenging due to issues with pump pressure among other variables yet timing of dispensing actions and performance of the dispensing applicators was demonstrated successfully. Overall, the project team feels as if a successful proof of concept was demonstrated for the automation of crack sealing operations. Once the issues listed in Section 5 of this document are addressed, the technology will be ready for advancement to a full-scale operational system.
Key Words
Crack sealing, automation, sealant, bituminous asphalt, maintenance, dispensing, crack detection
ii

Table of Contents
Acknowledgements.............................................................................................................. i Executive Summary..............................................................................................................ii Key Words ............................................................................................................................ii List of Figures .......................................................................................................................v List of Tables ......................................................................................................................vii Section 1: Introduction ....................................................................................................... 1
Background and Motivation............................................................................................ 1 Overview of GTRI Approach ............................................................................................ 3 Project Objectives ........................................................................................................... 4 Section 2: Interim Report Executive Summaries ................................................................ 8 Phase 1 Interim Report Executive Summary................................................................... 8 Phase 2 Interim Report Executive Summary................................................................. 10 Phase 3 Interim Report Executive Summary................................................................. 11
Recommendation 1 ................................................................................................... 12 Recommendation 2 ................................................................................................... 12 Recommendation 3 ................................................................................................... 12 Section 3: Crack Sealing System Details............................................................................ 14 Crack Sealing Hardware ................................................................................................ 14 Melter/Pumping Units ............................................................................................... 14 Electronics.................................................................................................................. 14 Camera and Lighting Sub-System .............................................................................. 15 Odometers ................................................................................................................. 15 Sealant Applicators .................................................................................................... 15 Dispensing Carriage ................................................................................................... 19 Crack Sealing Software Detail ....................................................................................... 20 Image Processing and Crack Map Generation........................................................... 21 Real-Time Control ...................................................................................................... 24 Section 4: Testing .............................................................................................................. 26
iii

Single Applicator Performance Testing ......................................................................... 26 Single Applicator System Parking Lot Tests .................................................................. 27 Crack Detection Algorithm Assessment ........................................................................ 30 Full System Parking Lot Tests ........................................................................................ 33
Continued Hardware Integration .............................................................................. 35 Full System Road Trials.................................................................................................. 35
December 7 road trial on GA 166.............................................................................. 35 December 20 Road Trial at CCRF................................................................................... 41 Section 5: Conclusions and Recommendations ................................................................ 46 Section 6: List of References ............................................................................................. 49 APPENDIX A Crack Detection Tuning Results...................................................................A-1 APPENDIX B Initial Matlab Crack Detection Algorithm Development............................. B-1 Algorithm Tuning..........................................................................................................B-2 Appendix C Run 6, December 20, 2010, GTRI Cobb County Research Facility ................ C-1
iv

List of Figures

Figure

Page

1. Block Diagram of GTRI Crack Sealing System.................................................................. 4 2. Prototype Crack Sealing Hardware with Detail of Applicators ..................................... 14 3. Single Sealant Applicator .............................................................................................. 18 4. Applicator Manifold ...................................................................................................... 19 5. Crack Detection and Control System Diagram.............................................................. 20 6. User Interface Screenshot ............................................................................................ 21 7. Block Diagram Showing Software Components ........................................................... 22 8. F.O.V. Highlighted ......................................................................................................... 22 9. Image Processing Steps................................................................................................. 23 10. Image Resizing Illustration .......................................................................................... 24 11. Real-Time Control PC Components............................................................................. 25 12. Valve and Piston Displacement and Fluid Pressure Measurements .......................... 27 13. Examples of Sealant Dispensed In Pavement Crack From System Parking Lot Tests. 29 14. Crack Map Passed From Crack Detector To Navigator Showing Applicator Opening
Events.......................................................................................................................... 29 15. Screen Capture of Crack Detection Scoring Software ................................................ 31 16: Dispensing example from GA166 Road Trial Run 7 1. ............................................. 38 17: Dispensing example from GA166 Road Trial Run 7 2. ............................................. 39 18: Dispensing example from GA166 Road Trial Run 7 3. ............................................. 40 19. A set of raw camera images, crack map, and photograph of applied crack sealant
from Run 6 on December 20, 2010............................................................................. 42 20. A set of raw camera images, crack map, and photograph of applied crack sealant
from Run 6 on December 20, 2010............................................................................. 43 21. A crack map along side a plot of the commanded position of the longitudinal
applicator from Run 6 on December 20, 2010 ........................................................... 44

A- 1. Score-Cracktuning-All .............................................................................................A-3 A- 2. Score-CrackTuning-088............................................................................................A-5 A- 3. Raw Image: Run14_Rimage008.bmp ......................................................................A-6 A- 4. Matlab Output: Run14_Rimage008_out.................................................................A-7 A- 5. Test Case 1...............................................................................................................A-8 A- 6. Test Case 2...............................................................................................................A-9 A- 7. Test Case 3.............................................................................................................A-10 A- 8. Test Case 4.............................................................................................................A-11

v

A- 9. Test Case 5.............................................................................................................A-12 A- 10. Test Case 6...........................................................................................................A-13 A- 11. Test Case 7...........................................................................................................A-14 A- 12. Test Case 8...........................................................................................................A-15 A- 13. Test Case 9...........................................................................................................A-16 A- 14. Test Case 10.........................................................................................................A-17 A- 15. Test Case 11.........................................................................................................A-18 A- 16. Test Case 12.........................................................................................................A-19 A- 17. Test Case 13.........................................................................................................A-20 A- 18. Test Case 14.........................................................................................................A-21 A- 19. Test Case 15.........................................................................................................A-22 A- 20. Test Case 16.........................................................................................................A-23 B-1. Original Input Image 005 ......................................................................................... B-3 B-2. Matlab Output 005 .................................................................................................. B-3 B-3. Delivery Platform Output 005..................................................................................B-3 B-4. Original Input Image 008 ......................................................................................... B-3 B-5. Matlab Output 008 .................................................................................................. B-3 B-6. Delivery Platform Output 008..................................................................................B-3 B-7. Original Input Image 039 ......................................................................................... B-3 B-8. Matlab Output 039 .................................................................................................. B-3 B-9. Delivery Platform Output 039..................................................................................B-3 B-10. Original Input Image 041 ........................................................................................ B-4 B-11. Matab Output 041 .................................................................................................. B-4 B-12. Delivery Platform Output 041 ................................................................................ B-4 B-13. Original Input Image 042 ........................................................................................ B-4 B-14. Matlab Output 042 ................................................................................................. B-4 B-15. Delivery Platform Output 042 ................................................................................ B-4 B-16. Original Input Image 108 ........................................................................................ B-4 B-17. Matlab Output 108 ................................................................................................. B-4 B-18. Delivery Platform Output 108 ................................................................................ B-4 B-19. Original Input Image 110 ........................................................................................ B-5 B-20. Matlab Output 110 ................................................................................................. B-5 B-21. Delivery Platform Output 110 ................................................................................ B-5 B-22. Original Input Image 111 ........................................................................................ B-5 B-23. Matlab Output 111 ................................................................................................. B-5 B-24. Delivery Platform Output 111 ................................................................................ B-5 B-25. Original Input Image 132 ........................................................................................ B-5 B-26. Matlab Output 132 ................................................................................................. B-5
vi

B-27. Delivery Platform Output 132 ................................................................................ B-5 B-28. Original Input Image 162 ........................................................................................ B-6 B-29. Matlab Output 162 ................................................................................................. B-6 B-30. Delivery Platform Output 162 ................................................................................ B-6 B-31. Original Input Image 163 ........................................................................................ B-6 B-32. Matlab Output 163 ................................................................................................. B-6 B-33. Delivery Platform Output 163 ................................................................................ B-6 B-34. Original Input Image OR164 ................................................................................... B-6 B-35. Matlab Output OR164 ............................................................................................ B-6 B-36. Delivery Platform Output OR164 ........................................................................... B-6 B-37. Original Input Image 165 ........................................................................................ B-7 B-38. Matlab Output 165 ................................................................................................. B-7 B-39. Delivery Platform Output 165 ................................................................................ B-7

List of Tables

Table

Page

1. Summary of Two Scoring Results From Random Images Taken From GA 100 ....... 33 2. Applicators opened in response to Crack Map
12202010_15_16_11_265_M.bmp ......................................................................... 42 3. Applicators opened in response to Crack Map
12202010_15_16_26_359_M.bmp ......................................................................... 44

vii

Section 1: Introduction
Background and Motivation Crack sealing is an accepted practice in many state Departments of Transportation (DOTs) as this operation is believed to add significant life to roadways. Evidence indicates that it is reasonable to seal cracks to improve overall pavement efficiency [1719]. As a result, there have been several efforts aimed at automating the task of crack sealing. As of the writing of this report, however, no commercially available system exists to address the problem in an automated manner. Several prototype devices have been constructed and tested, however. Examples include the work of Haas [4-8] and Velinsky [9-11]. With advances in technology and the approach proposed, it is believed that it is now possible to build a system to conduct the sealing operation automatically in motion. Research taking place at the Georgia Tech Research Institute (GTRI) has focused on development of an automated crack detection system, which is described in greater detail in this report.
One of the tasks conducted in this effort was to evaluate the current state-of-the-art of crack detection technology. Initial studies focused on the availability of integrated systems that could be used for the detection of cracks; one candidate was what is called pavement management systems (PMS). Several systems exist for the detection of cracks at highway speeds [12], which are representative of a class of systems that are built for assisting in pavement management functions. These systems have a slightly different goal in mind, however, and that is to give an overall description of the pavement condition as opposed to identifying specific cracks. These systems do not run in realtime and are meant to generate statistics that describe the overall condition of the pavement to support decision making on maintenance of these roadways.
One of the next steps was the evaluation of other sensing modes that could be considered for use in detecting cracks. A good source of information was a report generated for the Georgia DOT (GDOT) [12], which identified techniques for the detection of stripping in hot-mix asphalt. This work details how pavement data was collected with a suite of sensors. Eight different sensing modes for detecting stripping in hot-mix asphalt were tested: (1) ground-penetrating radar (GPR), (2) infrared thermography, (3) visible surface condition, (4) visible surface cracking, (5) rutting in the transverse profile, (6) rutting in the longitudinal profile (along the wheel path), (7) falling weight deflection, and (8) seismic measurements. The last two are point analysis methods; the visible sensing was done using a digital pavement imaging system. Stripping is defined as the "the loss of adhesion between asphalt cement and aggregate
1

surface caused by moisture and vapor." It was determined that the visible, thermography, and profilometer data would be candidates for crack detection; however, the resolution on the visible images may not be suitable for crack detection due to a resolution of approximately 1 pixel per millimeter. Additionally, there was significant variability that would have to be addressed due to fluctuations and variability in the lighting.
Additional reviews consisted of studying thermal IR data that was collected using a FLIR Systems SC-1000 that detects thermal energy from 3.4-5 microns. The sensor was set to read a very narrow range of temperatures (6-10C) in order to provide the dynamic range needed for discrimination. There is definitely some improved contrast between the cracks (dark lines) and the road surface. However, because the system is detecting thermal differences, other items that have different thermal properties also appear such as stripes, pavement sealant, concrete, etc. In addition, cooling and/or heating trends along the roadway cause the system to lose contrast and dynamic range forcing the sensor's autogain function to fluctuate. This appeared to be a problem when going under overpasses or beneath overhead signs.
GTRI then reviewed some of the more conventional approaches that have focused on the sensing and automatic detection of cracks for a variety of purposes. Most of the techniques found so far use visible imaging and computer image analysis for the crack detection [13-16]. With the exception of the two other identified systems (UC Davis and University of Texas) [4-8] [9-11], none of the others resulted in systems that were used in automating a crack detection/sealing operation. It is, therefore, difficult to assess their overall effectiveness in identifying cracks under real-world scenarios. All the systems and approaches, however, either required human guidance or had processing times on the order of several (approximately 10) seconds for processing. Based on system timing calculations, a system needs to be able to estimate the position of cracks on the order of 100 ms to be able to seal cracks at about 5 mph.
A detailed literature search did not uncover any crack detection system that exists to operate under the specifications defined for identifying as well as sealing the cracks automatically. Most of the existing systems are geared towards conducting distress surveys to guide maintenance planning.
A synopsis of the state of the art is that the detection of cracks, especially in real-time is challenging. The variability of the environment was noted as a significant confounding element in the development of the algorithms. The approaches that have been attempted include:
2

Laser Range Finding Moir Interferometry (technique that utilizes interference patterns generated on the pavement) Ground Penetrating Radar Thermographic (Thermal Imaging) Visible Imaging Stereoscopic Imaging
Ultimately, in practice, crack sealing operations are predominantly manual. Current practices involve the use of a sealant kettle, pump, and dispensing hardware accompanied by a number of workers. These operations, depending on the severity of cracks in the pavement and the equipment being used, may move at a pace anywhere from 5 to 20 miles per standard work day. There are also a number of safety issues related to worker safety that could be improved greatly by the introduction of automatic crack sealing systems.
Overview of GTRI Approach A prototype was designed and constructed to advance research in automated crack sealing operations. This prototype addressed the previously identified challenge of detecting cracks in real time, identified challenges associated with system integration, and provided a demonstration of the system capability on a limited scale.
The prototype, mounted on a trailer, consisted of a single stereo camera, an applicator system, and a means of providing a continuous supply of sealant to both a longitudinal and a transverse distribution system. The transverse crack distribution system consisted of a bank of 12 discreet nozzles spaced evenly across one foot of travel. For longitudinal cracks, a single dispensing nozzle capable of continuous operation was attached to a linear servo axis for tracking cracks while a towing vehicle was in motion. Servo operation is achieved by controlling the position of the longitudinal dispensing nozzle in real time via a command signal generated from the crack map. Controls were implemented for the prototype to permit automated sealing of identified cracks in a 12" wide band of pavement. This prototype was intended to represent one module that could be replicated and joined together to service a full width lane, when supported with a full-scale sealant melting and distribution system. Figure 1 below provides a highlevel block diagram describing the interaction between all major components in the system. The configuration of this prototype system was designed to meet the primary goals of detecting and filling a 1/16" wide crack at a speed of 5 mph.
3

Figure 1. Block Diagram of GTRI Crack Sealing System
Project Objectives The project objectives and specific objectives for each of five project phases, as stated in the research proposal, are included below. Over the course of the project, modifications to the scope of work have been made. A discussion of the changes to the scope of work follows the list of original project objectives. The objective of the proposed work was to provide improved hardware and methods for pavement crack sealing, enabling a reduction in labor costs, an increase in worker
4

safety, increased pavement repair rates, and reduced material consumption. These objectives were to be addressed by developing the necessary technologies, hardware, and software for automated identification, characterization, and sealing of pavement cracks.
Specific objectives that were identified for each of the five phases of the work plan are shown below:
PHASE 1: Develop and demonstrate components of a sealant application system capable of delivering asphalt cement at a suitable flow rate and with a jet shape and flow duration control to maintain crack overflow at an acceptable level.
PHASE 2: Develop and demonstrate components of a crack detection system capable of identifying pavement cracks of 1/16" or greater, distinguishing cracks from debris and shadows, specifying the location and shape of the crack, and making this determination in a timely manner.
PHASE 3: The development prototype will be capable of demonstrating the ability to identify cracks and apply sealant to the crack with acceptable accuracy at a vehicle speed of five miles per hour.
PHASE 4: The sealant melting system will be capable of melting cold, solid rubberized asphalt cement at a rate to support the highest expected sustained flow rate for sealing cracks at a vehicle speed of five miles per hour. The melting system will be supported by a design for a sealant distribution system capable of supplying sufficient flow to the applicator system. The vehicle displacement measurement system design will be capable of determining the axial and lateral displacement of the vehicle with sufficient precision to permit the control system to determine the timing for each of the applicator valves. The information management and control system will be capable of integrating data from the crack detection system, the vehicle displacement measurement system, and various other sensors, and will be capable of making decisions according to given criteria as to whether an identified crack should be sealed. For those cracks that are to be sealed, the system will be capable of providing proper
5

control commands to each of the valves that will provide appropriate start/stop of the flow in the applicator system. Decisions will be made as to whether it is appropriate to have the automation system control the speed of the vehicle.
PHASE 5: The full-scale prototype will be made up of modules developed in Phase 3 and will be capable of demonstrating all desired characteristics of an integrated automation system. It will provide a basis for subsequent design and manufacture of commercial automation systems.
The project was funded with 5 phases with "Go" or "No-Go" decisions before each next phase. Interim reports were delivered at the end of Phases 1 and 2 and six months after the beginning of Phase 3 that summarized the work completed in that phase [17-19]. At the end of Phase 3, a contract modification revised the objectives of Phases 4 and 5 and authorized the completion of the project in a single phase.
The authorization included slight changes to the objectives and scope of Phases 4 and 5. The authorized changes to the objectives, as stated in the project proposal, are summarized below:
Sealant melting system The Phase 4 work plan addresses alternatives to a conventional melt kettle including the use of an extruder and various technologies for melting the sealant at the nozzle tip. The dispensing technology that has been developed under this project and the automated RPM placement system project requires the sealant to be melted before it reaches the applicator. This dispensing technology can be adequately tested using a conventional, commercial melt kettle. No melting technology will be developed.
Vehicle displacement measurement We have already completed a preliminary search for non-contact vehicle displacement sensors. We have not discovered any attractive alternatives and are proceeding with wheel odometers for vehicle displacement measurement. There will be no further investigation of vehicle displacement sensor technology.
Information management and control system The scope of work described in this section of the Phase 4 work plan will be completed as described.
6

Vehicle speed control The current system design will dispense the material after the vehicle has traveled only 4 or 5 feet from where the camera images were captured. Therefore, the system will not have sufficient information to increase and decrease the travel speed based on an assessment of the amount of cracks per area of pavement. Automated vehicle speed control to maintain a desired setpoint speed will likely be desirable. We plan to identify a suitable low-speed vehicle speed controller for use on the automated RPM placement system. We will apply that solution to the pavement crack sealing system, but will not vary the vehicle speed based on crack frequency. The Phase 5 deliverable is to be a trailer-mounted automated pavement crack sealing system capable of detecting and filling cracks along a 1-foot-wide strip of pavement while in motion at 5 mph utilizing a single nozzle mounted on a servomechanism for filling longitudinal cracks and an array of 12 nozzles mounted one inch on center for filling all other cracks.
7

Section 2: Interim Report Executive Summaries
To trace the development of the crack sealing system, this section is included and comprised of the executive summaries of the three interim reports preceding the final report. The complete interim reports are available from Georgia DOT. The remainder of this report covers the accomplishments and findings since the third interim report was delivered.
Phase 1 Interim Report Executive Summary Cracks in the road surface are conventionally sealed manually, using workers operating in the traffic lane. Automation of crack sealing offers potential benefits in reduced labor requirements, improved worker safety, increased pavement repair rates, and reduced material consumption through improved application accuracy. The objective of this fivephase, 51 month project is to develop a system capable of filling pavement cracks automatically while in motion at 5 miles per hour in the traffic lane. The objective of the first phase of the project is to establish the capability for dispensing the asphalt-based crack sealant at a rate sufficient to make automation of this operation feasible. This report covers the Phase 1 efforts to develop the sealant application system capabilities.
The Phase 1 objective as stated in the research proposal is to develop and demonstrate components of a sealant application system capable of delivering asphalt cement at a suitable flow rate and with a jet shape and flow duration control to maintain crack overflow at an acceptable level. The suggested flow rate target range is discussed below beginning on page 4. The target range is based on providing a nozzle to cover every oneinch width of traffic lane or to cover every -inch width of traffic lane. The target range is based on a penetration depth of the asphalt crack sealant of either one inch or inch. The target range is based on an application speed of 5 miles per hour. At 5 miles per hour, the nozzle will be above a 1/16-inch-wide transverse crack for 0.71 ms. The suggested flow rate target range is between 360 ml/s and 1442 ml/s and the suggested flow volume target range is between 0.26 ml and 1.02 ml.
An initial flow test was conducted using hot-melt adhesive and a commercial hot-melt adhesive application system borrowed from Spraymation, a manufacturer of hot-melt adhesive application systems. Discussions on the flow requirements for this application were held with the lead engineer at Spraymation. Spraymation proposed to furnish one of their standard, large-orifice, pneumatically operated applicator valves and a custom, piston-spring accumulator to provide the short-duration, high-flow-rate material dispensing required for this application.
8

The proposed applicator/accumulator was purchased and the system was assembled. The flow performance of the system was tested extensively using three different hotmelt adhesives applied at a range of temperatures. The use of hot-melt adhesives offered a number of advantages for the initial testing. For the hot-melt adhesive flow tests, no ventilation was required, no mixing was required, and the borrowed equipment could be utilized. Rheological measurements were made on both the hotmelt adhesives and the asphalt crack sealants so that the fluid properties of the hotmelt adhesives could be compared to those of the asphalt crack sealants. Those measurements suggest that the hot-melt adhesive flow test results should be representative of asphalt crack sealant flow tests.
Analysis of the hot-melt adhesive flow test results showed two unexpected characteristics, 1) there does not appear to be a correlation between viscosity and flow rate and 2) if there is a correlation between average pressure and average flow rate it is that higher average pressure correlates to lower average flow rate. The first result suggests that viscosity is not the dominant phenomenon governing the flow rate in these experiments. Modeling of the accumulator was begun to try to explain the second of these unexpected results. The model of the accumulator was then used to evaluate modifications to improve the flow performance of the system.
The borrowed hot-melt adhesive application system was replaced with purchased equipment so that a short series of asphalt crack sealant flow tests could be run. At the same time, the accumulator was modified based on predictions made with the model of the accumulator. A short series of asphalt crack sealant flow tests was conducted using one asphalt crack sealant applied at the recommended application temperature and using the modified accumulator and the newly purchased hot-melt adhesive application system. The performance of this modified sealant application system meets the Phase 1 objective and is within the suggested flow rate target and flow volume target ranges listed above.
The initial sealant application system developed under the Phase 1 effort has demonstrated the ability to deliver asphalt cement through a nozzle at a flow rate that would be sufficient to fill cracks to a depth of inch over inch of lane width while in motion at 5 miles per hour, that is at an average flow rate in excess of 360 ml/s. The initial sealant application system has demonstrated the ability to deliver asphalt cement at this average flow rate for approximately 10 ms, delivering between 4 and 5 ml of material. The initial sealant application system has demonstrated the flow start/stop control of approximately 10 ms. At 5 miles per hour, the nozzle will travel approximately 7/8" in 10 ms. The initial sealant application system fitted with a nozzle with a
9

rectangular orifice has demonstrated the ability to deliver the material in a rectangular pattern. Therefore, the performance of the initial sealant application system developed under the Phase 1 effort has met the Phase 1 objectives.
While the performance of the initial sealant application system developed under the Phase 1 effort has met the Phase 1 objectives, the results of the Phase 1 effort have led to the identification of a number of opportunities for improvement and questions to be addressed. Opportunities for improvement include increasing the flow rate of the system and increasing the responsiveness of the valve. Questions to be answered include characterization of the material flow rate profile. Recommendations for further investigation and improvement of the sealant application system are discussed. Since the proposed Phase 2 effort is focused entirely on development of the crack detection system, it may be desirable to next undertake selected tasks from both Phase 2 and Phase 3 in order to minimize the risks associated with this development.
Phase 2 Interim Report Executive Summary Cracks in the road surface are conventionally sealed manually using workers operating in the traffic lane. Automation of crack sealing offers potential benefits in reduced labor requirements, improved worker safety, increased pavement repair rates, and reduced material consumption through improved application accuracy. The objective of this fivephase, 51-month project is to develop a system capable of filling pavement cracks automatically while in motion at 5 miles per hour in the traffic lane. The objective of the first phase of the project was to establish the capability for dispensing the asphalt-based crack sealant at a rate sufficient to make automation of this operation feasible. This report covers the Phase 2 efforts to develop the crack detection system capabilities.
In the Phase I, effort we demonstrated the ability to deliver sealant material at a rate sufficient to fill an average crack while in motion at 5 mph. The goal of the Phase 2 effort was to demonstrate the ability to find cracks at the desired rates so to be able to provide a signal to the crack sealing device to allow for triggering of the crack sealing operation.
In the work conducted in Phase 2, we evaluated several concepts for sensing cracks in the pavement, which would be the first operation required before sealing. Several options were considered including thermal infrared (IR) imaging, profilometers, near IR imaging, visible imaging, and imaging with structured light. After an initial investigation we chose three for further analysis and development. These were the visible imaging, the near IR imaging, and the imaging with structured lighting approaches.
10

Experiments were designed and data collected both at Georgia Tech and also along two sections of actual roadway that were targeted for sealing along with data from the Department of Public Safety track in Forsyth, Georgia. Based on this work, we chose a lighting and imaging configuration that was a combination of the structured lighting and visible imaging. This was accomplished by using directed LEDs and a stereo camera, which allowed us to capture two images simultaneously with lighting from orthogonal directions. The spectra of the illumination along with filtering enabled us to isolate the illumination from each of the two directions. We selected the orthogonal directions to be parallel to the road and perpendicular to the road to address the majority of the cracks, which are either longitudinal or transverse.
Algorithms were also developed to process the data generated by the imaging cell. We demonstrated the ability to process this data for crack identification at speeds of approximately 700 ms, but also identified a path to reducing these operations to speeds of about 100 ms, which is our estimate of what would be required for the prototype sealing device under consideration. We gave a demonstration to our technical monitors and a representative of GDOT Maintenance on September 29, 2006.
Phase 3 Interim Report Executive Summary While the performance of the initial sealant application system developed under the Phase 1 effort met the Phase 1 objectives, the results of the Phase 1 effort led to the identification of a number of opportunities for improvement and questions to be addressed as discussed beginning on Page 8 of the Phase 1 report. Since several people at the Georgia Department of Transportation felt that addressing these recommendations as soon as possible would minimize the risks associated with this project, we proposed to begin a portion of the Phase 3 effort simultaneously with the Phase 2 effort. We received authorization to proceed with Phase 2 and the limited Phase 3 effort on April 26, 2005. The Phase 1 Recommendations for further investigation and improvement of the sealant application system, provided on Page 3 of this report, constitute the scope of work of the limited Phase 3 effort.
The limited Phase 3 effort has shown that effective mixing can be achieved; that the repeatability of the flow rate, flow quantity delivered, activation delay, and valve open time is satisfactory for the proposed concept; and that the system can place sealant in representative cracks with little or no splash. Spraymation, with our input, has developed a design for a coaxial accumulator/applicator that should provide significant cost and performance improvements over the existing prototype system. They provided a budgetary cost estimate that suggests that the proposed system will be economically
11

feasible. The specific recommendations are grouped into three groups identified below as Recommendations 1, 2, and 3.
Recommendation 1 Tests were conducted using the asphalt application system that was developed during Phase 1. The system includes a heater/melt tank/high-pressure pump unit, a precise timer unit, a spring-piston accumulator, and a pneumatic-operated sealant valve; all from Spraymation. We were able to achieve satisfactory mixing of the asphalt-based crack sealer using a simple, paddlewheel mixer. We used density measurements of samples taken as the melt tank was emptied to evaluate mixing. We compared all combinations of paddlewheel on and off and recirculating loop on and off, including no mixing.
We investigated the repeatability of the flow rate, flow quantity delivered, activation delay, and valve open time. Sample mass dispensed and initial pressure were measured for about 90 asphalt-dispensing shots. The sample mass dispensed was very repeatable. Our investigation indicates that most of the variation in mass dispensed is due to the variation in the initial pressure. For example, the average amount of sealant dispensed was 4.60 g for the 63 shots with an initial pressure between 900 and 940 psi. The standard deviation of the amount of sealant dispensed was 0.155 g.
Recommendation 2 Trials have been conducted to investigate the capability of the sealant application system for delivering asphalt crack sealant into stationary, artificial cracks. The trials have been run at a range of target heights and offsets between the nozzle center and the crack center. A micrometer stage is used to position the nozzle in the desired relative position over the artificial crack. Trials have been conducted with a 1-inch by 1/16-inch rectangular nozzle. Evaluation of performance has included the depth of penetration, appearance, amount of material deposited in the crack, amount of material deposited outside the crack, and amount of material departing as a splash. The results of the crack filling trials are consistent with our expectations. There is no observable splash. The sealant penetration is deeper for a wider crack. Reducing the nozzle height increases the sealant penetration.
Recommendation 3 A two-way non-disclosure agreement was executed between Spraymation, Inc. and the Georgia Tech Research Corporation in June, 2005, to permit the exchange of confidential information related to the development of a concept for an
12

accumulator/valve module for use in a production automated pavement crack sealing system. Spraymation has provided a proprietary drawing of the conceptual design for an accumulator/valve module and the associated manifold... Each applicator module includes a pneumatic control valve, a pneumatic actuator for the sealant valve, a valve needle and seat, a spring-piston accumulator, and an inlet check valve. Twenty-five applicator modules will be mounted in a manifold on four-inch centers. Four manifolds will be mounted on the system, staggered by one inch to provide 100 applicators on one-inch centers, thereby covering a 100-inch lane width. (Prior to implementation, the manifold layout will be adjusted to provide coverage for the desired lane width.) The manifolds will include passages for recirculating heat-transfer oil to heat the manifold and the applicator modules. The manifolds will include passages to provide 1000 psi, 380F crack sealant to the applicator module inlets. The mounting design for the applicator modules will provide for quick replacement in the field. Spraymation provided a budgetary cost estimate in November. Their budgetary price estimate of $240,000 is for 100 applicators and the associated manifolds; an average price of $2400 per applicator.
13

Section 3: Crack Sealing System Details
Crack Sealing Hardware The crack sealing system consisted of a number of components described in Section 1. The demonstration prototype used consisted of a tow vehicle pulling a custom trailer constructed from a T-slot aluminum framing system. The pickup bed had a large generator and a melter-pumping system installed, and a custom trailer housed all other components of the system. The entire system is shown in Figure 2 below. Each of the major subsystems is described in detail in the following subsections.
Figure 2. Prototype Crack Sealing Hardware with Detail of Applicators Melter/Pumping Units The integral melter/pumping units were standard items available from hot-melt adhesive dispensing suppliers. These units were sized to provide a continuous flow for the longitudinal crack filling subsystem and the intermittent flow associated with the transverse crack filling subsystem. Both melter/pumping units were outfitted with an additional mixing unit. These small melter/pumping units removed the need of having to operate a large crack sealant kettle in the demonstration unit; however, a much larger sealant unit would be needed to operate a system for the entire lane width. Electronics Besides housing general wiring, safety circuits, and voltage distribution, the main electronics cabinet was home to the core image processing computer as well as the realtime operating system. Real-time processing is handled via QNX and controls all real-
14

time functions such as the firing of individual nozzles and navigation and operation of the control signal to the linear servo axis. These functions are guided by crack detection algorithms running on a separate computer.
Camera and Lighting Sub-System This area of research required unique solutions to allow the system to perform the task of identifying cracks in 100ms. The design of the imaging cell is driven by the need to create two identical images that differ in the direction of lighting required to illuminate both transverse and longitudinal cracks. Two colors of light emitting diodes (LEDs) are projected onto the camera field of view at differing angles to better highlight the respective features of the two primary types of cracks. The two colors are filtered separately and captured by a calibrated stereo camera mounted above a 12x12" field of view. These two differing images are then used for crack detection algorithms. What is achieved by this approach is that each stereo image set already has some features identified, thus simplifying crack detection routines in order to speed up the overall process. The entire system is enclosed in a series of thick rubber sheets that shroud the imaging cell from light in the environment.
Odometers For navigation of the crack sealing system, it is imperative that the position of the trailer is always accurately known. An encoder assembly was attached to each trailer wheel to monitor wheel position on each of the wheels to not only track overall distance traveled, but to also carefully track the two wheel positions relative to one another when the trailer is turning. Even slight variations in the angle of the trailer can contribute to error in timing of the transverse crack filling subsystem, which drives the need for an elaborate odometer system.
Sealant Applicators
Background During Phase 1, an initial flow test was conducted using hot-melt adhesive and a commercial hot-melt adhesive application system borrowed from Spraymation, a manufacturer of hot-melt adhesive application systems. Discussions on the flow requirements for this application were held with the lead engineer at Spraymation. Spraymation proposed to furnish one of their standard, large-orifice, pneumatically operated applicator valves and a custom, piston-spring accumulator to provide the short-duration, high-flow-rate material dispensing required for this application. The proposed applicator/accumulator was purchased and the system was assembled. The
15

flow performance of the system was tested extensively using three different hot-melt adhesives applied at a range of temperatures.
GTRI developed an automated raised-pavement-marker placement system concurrently with this pavement crack sealing system. High-performance dispensing equipment was also needed for placing markers in motion at 5 mph. The design of a first hot-meltadhesive applicator with a coaxial accumulator and valve was based in part on the valve and accumulator acquired during Phase 1 of the pavement crack sealing project. Marker placement road trials were completed using that applicator. Later, it was determined that significantly more adhesive volume would be required. Another applicator was designed that included a pair of 28 g accumulators. Simple performance modeling was done on the spring-piston accumulator, testing was done to characterize the performance of the applicator; experiments were conducted to develop a suitable discharge nozzle; and a series of marker placement road trials were conducted. It was concluded that the highest likelihood of success in the design of the array applicators for the pavement crack sealing system would be achieved by using key elements of the marker placement applicators. For example, the accumulator bore and stroke, accumulator piston design, and accumulator spring were identical in both applicators. In addition, the valve seat and valve needle geometry were the same.
There were, however, many significant differences given the large differences in the system requirements. It was anticipated that a full-lane-width pavement crack sealing system might include as many as 144 accumulator-valve pairs on one-inch centers. It would be undesirable to provide individual heaters and temperature control circuits for each accumulator-valve pair. It was decided that a heated manifold would be used to heat the valves and accumulators and to provide them with high-pressure crack sealant. It was decided that building both the valves and the accumulators as modular units would likely reduce the overall cost and simplify the system maintenance. The requirement to locate the accumulator applicators in a close array precluded the use of the same pneumatic valve actuator that was used on the marker placement applicator. A design was developed that permitted placing the accumulator-valve pairs on two-inch centers; this set the requirement for a pneumatic valve actuator that was approximately 1.875 inches in diameter. Two rows of manifolds containing the accumulator-valve pairs can be used with a one-inch lateral stagger to give an array of applicators that are one inch on center.
Development of sealant applicator array During the first week in July, we resumed discussions with Spraymation about the design and construction of the single-applicator unit. This included discussion about our
16

project schedule and their production schedule for both the single-applicator unit and the pair of six-applicator units. Spraymation proposed a relationship that was different from that under which the adhesive applicators were supplied for the RPM placement system. GTRI was to be responsible for the applicator design, preparation of the shop drawings, and suitability of the design for the application. Spraymation was to fabricate, assemble, and test the units. Spraymation's quotation reads in part:
"If the applicator does not operate as expected and the root cause is Spraymation's fabrication or assembly Spraymation will rectify it. If the problem is a design problem, parts don't fit, parts fail, or just does not work, the problem will need to be handled on a time and materials basis. It must also be understood that if problems arise delivery may be delayed. The price will include making/purchasing parts to GTRI specification, assembly of those parts based on your drawing and bill of materials and functional testing." Spraymation proposed a preliminary review of our shop drawings prior to submittal of the purchase order and final shop drawings. They proposed a 4-week schedule for building the single-applicator unit followed by a one-week period for our test and approval of the single-applicator unit and a 4-week schedule for building the pair of sixapplicator units. Based on the proposal from Spraymation, we began generating the shop drawings for the single applicator unit. During the process, we also made design changes to address known issues and issues identified during the process. We held informal design reviews as needed. A formal, internal design review was held on July 24, 2008.
17

Figure 3. Single Sealant Applicator
The unit shown in the figure above was used as a basis for the snap open testing described in the Single Applicator Performance Testing section of this document. Observations from these tests led to decisions made about timing of shots in the initial parking lot tests. These initials tests also lead to modifications to reduce excess fluid from being fired in shots by reducing the volume of the crack sealant passage.
For the full system, two sets of sealant applicators were developed to cover the 12" wide section of pavement. Working in collaboration with Spraymation, makers of the previous single applicator manifold, a new design was created based on an existing sixapplicator design. The primary objective for the design revision was to reduce the volume of the passage between the valve seat and the nozzle plate. This was achieved by eliminating the offset passage with change of direction downstream of the valve seat such that the nozzle plate is adjacent to the end of the valve module. The valve module and accumulator modules were lowered .75 inches deeper into the manifold. The fluid
18

passages, heater bores, RTD bore, electrical cavity, etc. were also moved to accommodate this change. Shop drawings were developed for the six-applicator manifold and for five other parts that were either new or different from the parts of the single-applicator module.
Figure 4. Applicator Manifold Dispensing Carriage The dispensing carriage consists of two manifolds with a total of 12 individually addressable nozzles for the purpose of filling transverse cracks while in motion. This carriage also carries a single applicator mounted to a linear servo axis for the purpose of filling longitudinal cracks. Each dispenser in the transverse crack dispensing system is comprised of a pneumatic-operated valve and a spring-loaded accumulator and is housed in a heated manifold. This allows each of the nozzles to be fired individually when commanded by the Navigator as it passes over a crack. The timing of this firing is crucial, and resulted in a detailed timing study, which is discussed in the Testing section.
19

The longitudinal crack system employs one nozzle attached to a high-torque linear servo axis that can be commanded to follow a longitudinal crack as the system is towed. Each of these dispensing elements is supplied with a continuous supply of crack sealant from their respective melter/pumping systems via heated hoses. The dispensing equipment is mounted to a single structure on casters that is supported by a four-bar linkage tied to the axle of the trailer. This mounting approach allows for the crack filling hardware to follow the surface of the road closely without being damaged by variations on the pavement surface. There is a lift cylinder attached to the entire four-bar mechanism to allow the system to be stowed while the crack sealing system is in tow at highway speeds, but not in operation. The dispensing carriage is shown in the detailed view within Figure 2. Crack Sealing Software Detail The software for the crack detection and control system consists of two sub-systems: a vision processing sub-system and a real-time controls sub-system. The vision processing sub-system consists of a camera and a Windows based processing computer. The control sub-system consists of a real-time QNX machine interfaced to wheel encoders and a dispensing device. The overall system configuration is shown in Figure 1. Both machines are Pentium Core-2 Duo 2.4Ghz machines with 2 GB of system memory. The design allows the QNX PC to control and query all of the hardware as shown in Figure 5 below.
Figure 5. Crack Detection and Control System Diagram
20

Image Processing and Crack Map Generation The vision processing sub-system is written entirely in Microsoft Visual C++. In addition to running the crack detection routines, this sub-system also provides a user interface for control and monitoring of the overall system during operation. The interface allows the user to start and stop the system, as well as view the images and crack maps as they are processed and generated. Figure 6 shows a screen capture of the interface.
Figure 6. User Interface Screenshot The software contains a class for the user interface, acquiring images from the camera, identifying crack segments in images, performing coordinate transformations, and sending the map to the real-time controls QNX PC. Figure 7 shows a diagram of the code layout as described.
21

Figure 7. Block Diagram Showing Software Components A Point-Grey Research Bumblebee camera is used for image acquisition. This camera acquires two images from separate sensors in a single housing. During operation, the image processing software will wait for an image to arrive from the camera. The camera is set to hardware trigger mode, acquiring images only when the QNX Control PC sends a trigger signal. This allows the two systems to remain synchronized. Due to the stereo effect of the two images, the field of view processed is reduced in both images to only the area of the images visible to both sensors. This shared field of view is illustrated in Figure 8.
Figure 8. F.O.V. Highlighted
22

The crack detection algorithm is contained in the image processor class. This class processes both left and right images independently. The raw images (1) are flat-fielded to normalize the light distribution across the images (2). A threshold is applied to the flat-fielded image to find candidate crack segments (3). Finally, a series of filters is run on the candidate crack segments and a final resulting crack map is generated (4). Figure 9 illustrates this process. Once the crack map is generated for each image, they are sent to the coordinate transformation class.
Figure 9. Image Processing Steps Once in the coordinate transformation class, a mapping is made for both images from image space to a world coordinate frame representing the location on the road surface of the cracks. The full resolution images are first transformed into a 192x192 pixel image, with each pixel representing a 1/16" area on the road surface. This area is associated with the bank of sealant applicator nozzles physically located behind the imaging area. Both left and right images are combined, creating a single crack map. The next step reduces the resolution from 1/16" x 1/16" to 1/16"x 1/8" (192x192 to 192x96 pixels) as shown in Figure 10. The reduced resolution is perpendicular to the nozzles, which each covering a 1 inch wide area. The final step rotates the map 90 degrees clockwise to match the orientation of the map expected by the QNX control PC.
23

Figure 10. Image Resizing Illustration
The final step for the image processing component is to send the crack map to the QNX control PC. The crack map is sent via UDP protocol via a physical Ethernet connection between the image processing PC and the control PC.
Several iterations of the crack detection software have been created throughout the project including a study to identify the range of various thresholds and parameters used in crack detection algorithms. A crack detection tuning task is captured in Appendix A and the initial Matlab crack detection algorithm development is captured in Appendix B.
Real-Time Control The controls sub-system operates on a real-time QNX operating system. This allows the processes such as actuating the applicator nozzles, or sending a hardware trigger signal to the camera, to occur in a timely fashion. At 5 mph, the applicators are moving at 88 inches a second. At this speed, a 2 millisecond delay could cause the applicators to miss the crack altogether (assuming a 1/16" wide crack). Therefore, a real-time system is required.
The control PC has physical connections to the camera for image capture, two encoders for pose information, and the sealant applicators for actuation. This allows the system to control all of the hardware aspects of the crack sealing system. The two encoders (one on each side of the system) allow for tracking of the travel direction and distance with respect to the captured and processed images. See Figure 11 for an illustration of this operation. The system will acquire an image approximately every 11 inches of travel, allowing for a small overlap between subsequent captured images. When an image capture is requested, the control system records the time and current encoder values, and generates a hardware trigger signal for the camera.
24

Figure 11. Real-Time Control PC Components The control PC then loops in a wait-state for a crack map to arrive from the image processing PC via the Ethernet connection. Since the control PC generated the hardware trigger to capture the image, the exact time the image was taken, and position of the applicators relative to the image captured is known. When a map is received, the control system will then add this map to the current map queue being processed. This map queue represents all the identified cracks on the road surface between the current camera field of view and the actual applicator nozzles. During the time the system is waiting to receive the new map, it is also concurrently updating the applicator location relative to the current crack maps already in memory using the values reported from the left and right encoders. The system calculates the location of the identified cracks on the road surface relative to the applicator nozzle array. When the applicator approaches a crack on the road surface, the control PC will actuate the nozzle, propelling sealant into the crack.
25

Section 4: Testing
Single Applicator Performance Testing A series of performance tests was conducted on the single applicator unit dispensing hot-melt adhesive to properly characterize the response of the physical dispensing system. The performance tests utilized several features incorporated into the single applicator design specifically for performance testing. Openings were machined in the accumulator and valve bodies allow observation of the position of the sealant valve and accumulator piston with a high-speed video camera. A pressure transducer port is provided downstream of the sealant check valve so that a high-temperature, highfrequency-response pressure transducer can be used to measure the sealant pressure in the passage between the accumulator and the sealant valve. A high-speed video system was used to make displacement-versus-time measurements on the accumulator piston and the sealant valve needle. High-speed video images were also captured of the adhesive flow out of the nozzle. The high-speed video system also captured data from the pressure transducer. A three-position pneumatic control valve was used to open the sealant valve.
Individual tests were conducted to measure the sealant valve response time, sealant valve displacement as a function of time, accumulator piston displacement as a function of time, and time required for fluid flow to exit the valve and be visible at the nozzle.
The sealant valve response time is defined as the time difference from illumination of the LED on the pneumatic control valve to detection of a drop in sealant pressure by the pressure transducer. The high-speed camera recorded the LED on the pneumatic control valve; the associated data acquisition system recorded the sealant pressure. The sealant valve response time was measured twice with values of 0.024 seconds and 0.025 seconds.
For measurement of the sealant valve displacement as a function of time, the highspeed camera recorded the motion of the coupler between the sealant valve and the pneumatic cylinder. The associated data acquisition system recorded the sealant pressure. For measurement of the accumulator piston displacement as a function of time, the high-speed camera recorded the motion of a flag mounted between the accumulator spring and the accumulator piston. The associated data acquisition system recorded the sealant pressure. Figure 12 shows the displacement vs. time for both the valve and the accumulator. The individual runs are aligned at time zero defined as when the fluid pressure drops from its steady state value which was generally around 1000 psi. Fluid pressure had decreased to less than 50 psi by 0.009 to 0.010 seconds.
26

Valve & Piston Displacement vs. Time Pressure vs. Time

Displacement [inch]] Pressure [psi]

1.6

1200.0

1.4

1000.0

1.2

1

800.0

0.8

600.0

0.6

400.0

0.4

0.2

200.0

0

0.0

0.000 0.002 0.004 0.006 0.008 0.010 0.012

Time [sec]

Valve 001 Valve 002 Valve 003 Piston 004 Piston 005 Piston 006 Pressure 001 Pressure 002 Pressure 003 Pressure 004 Pressure 005 Pressure 006

Figure 12. Valve and Piston Displacement and Fluid Pressure Measurements

For measurement of the time required for fluid flow to exit the valve and be visible at the nozzle, the high-speed camera recorded the fluid flow exiting the applicator nozzle. The associated data acquisition system recorded the sealant pressure.
Time zero is defined as when the fluid pressure drops from its steady-state value. Four measurements were taken of the times required for fluid to appear at the nozzle exit. The average of these results was 0.005-0.006 seconds. This measurement is defined as the fluid flow response time. This test also showed that while the pressure in the applicator drops below 50 psi after about 0.008 seconds, the flow continues for more than 0.020 seconds.
Results of this testing is used by the Navigator software. The flow response time is the combination of the sealant valve response time and the fluid flow response time. Base on the performance testing, the flow response time is 0.0245 + 0.0055 = 0.030 seconds. This value is entered into the Navigator configuration file.
Single Applicator System Parking Lot Tests Initial integration tests were required to validate the function of all major system components using a single applicator. Several trial runs were made by towing the crack sealing trailer a short distance across 5 or 6 transverse cracks at speeds between 2 and 3 mph. In every case, dispensing by the 11 missing applicators was simulated by LEDs that were at the location of each missing applicator and that were connected to the Navigator's control outputs.

27

During the runs, data was collected by the Crack Detector and the Navigator computers. The Crack Detector could be set up to save left and right raw camera images as well as crack maps in both binary image files and comma-separated-value files. After each run, Navigator log files could be saved to the hard disk drive. The Navigator log files include a variety of time-stamped data: configuration parameters, image capture time, crack map generation time, and data on each applicator opening event. For each applicator opening event, recorded data included: applicator number, image number, axle center location (x, y, ), the position of the applicator in both inches and crack map units, distance since image snap, and vehicle speed.
A high-speed video camera was positioned above the applicator carriage aimed at the applicator and LEDs below acquiring images at 1000 frames/second. Lamps were placed on the carriage to illuminate the pavement. Evaluation consisted of reviewing the highspeed video after each run to determine whether or not the LEDs that simulated dispensing were illuminated as they passed over the pavement cracks. This data was used to adjust time offsets in 1 ms increments between the camera and applicator systems.
Recent modifications to the system included improving the skirts that exclude sunlight from the camera field of view and raising the height of the image capture system mounting plate. Since these changes affect the image of the pavement, the initial session was focused on evaluating the image quality and selecting the intensity threshold levels used by the crack detection algorithms. The objective of the subsequent tests was to verify that the system would dispense sealant in the pavement cracks by evaluating the simulated dispensing events using the high-speed video system. Over the course of several sessions, several issues were identified and resolved. Three errors in the distance offset settings that the Navigator uses to determine where to open the nozzles were discovered and corrected.
Given that the correction of the offsets did not result in dispensing events over the pavement cracks, the accuracy of the odometer calibrations was investigated. It was discovered that one odometer wheel was smaller than the other one. The Navigator code was modified to allow each odometer wheel circumference to be set independently. However, the use of carefully measured odometer wheel circumferences did not solve the problem. An effective odometer wheel circumference was calculated from measuring the distance traveled by the trailer using a distance measuring wheel and the corresponding angular rotation of each odometer wheel in encoder counts over a distance of approximately 230 feet. These effective odometer wheel circumferences were significantly different than the measured circumferences,
28

but their use resulted in the Navigator illuminating the LEDs as they passed over the pavement cracks. The discrepancy between the measured odometer wheel circumference and the effective odometer wheel circumference could be a result of either the behavior at the interface between the trailer tire and the odometer tire or the behavior at the interface between the trailer tire and the pavement.

Figure 13. Examples of Sealant Dispensed In Pavement Crack From System Parking Lot Tests

Once the offsets and odometer wheel circumferences were corrected such that the Navigator was illuminating the LEDs as they passed over the cracks, the first system trial that included dispensing was conducted. The system was able to dispense material in pavement cracks in motion at around 2.8 mph. An example of the shots is shown in Figure 13.

12

11

10

9

8

7

6

5

4

3

2

1

0

0

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 3

Figure 14. Crack Map Passed From Crack Detector To Navigator Showing Applicator Opening Events.

29

Figure 14 shows the results from Run 1 on October 21, 2009. The solid line on the graph represents the edges of the crack map extracted from the comma-separated-value file, which can be compared to the crack map image at the right. The axes are in inches on the pavement. Aspect ratio has been set to match the crack map that has twice the resolution longitudinally as it does laterally. The points marked on the graph represent applicator opening events. The applicators are numbered 0 through 11 from left to right with the even applicators on the forward row and the odd applicators on the aft row. The Navigator fired applicators 1, 2, 3, 4, 5, and 6 to fill the crack.
Applicators 2, 3, and 5 were opened twice over the crack. The mechanical characteristics of the accumulators and sealant valves of the applicators are such that once the sealant valve is opened, the accumulator will fully discharge before the sealant valve can be closed. It then takes some time for the accumulator to be recharged. The recharge time will depend on the capacity of the high-pressure pump that feeds the applicators from the melt kettle. In this system, it takes several seconds for the pump to recharge a single applicator. Consequently, the Navigator's second opening of an applicator will have no impact on the amount of sealant dispensed in the crack.
Crack Detection Algorithm Assessment A road trial was completed near the border of Heard and Carroll counties on Georgia Highway 100. The objective of the road trial was to collect a series of crack images with the current system configuration needed to fine tune the crack detection algorithm parameters. An auxiliary speedometer based on the trailer-wheel odometer was installed in the cab to provide low-speed speed measurement to the driver during the trial.
During the road trial, approximately 30,000 raw camera images and crack maps were captured along with Navigator log files, which equates to about 5 miles of roadway. The pavement was imaged as sequential 12-inch-by-12-inch images. During the initial runs, specific pavement regions were targeted including pavement containing longitudinal cracks, spider cracks, and road markings. Then, longer runs were made to collect a large number of images from both of the traffic lanes. The data was used to assess the current performance of the crack detection algorithms and improve their performance. In order to objectively evaluate the performance of the detection algorithms, scoring software was created. With this software package, the crack sealing team generated a tool that allowed researchers to go through large amounts of images and apply a score for the image set in order to create an objective measure of the crack detection algorithm performance.
30

In February, work began on the crack scoring software. With this package, the crack sealing team was generating a tool that allowed researchers to go through large amounts of images and apply a score for the image set in order to create an objective measure of the crack detection algorithm performance. The current layout of the software can be seen in Figure 15 below. The first revision of the software was completed in April.
Figure 15. Screen Capture of Crack Detection Scoring Software The purpose of the test is to determine the effectiveness, measured as the power and specificity, of the crack detection algorithm, which is used to detect road cracks. A sample set of two hundred randomly selected images (n = 200) were independently rated by three experts. Interrater reliability was measured by the percentage of exact agreement (Pearson's product-moment correlation, etc,). The average interrater reliability is 0.88. The degree of consistency within each subject was measured similarly. The specificity of the software is 0.86 and the power 0.84. The average results from three judges for two different scoring sessions are shown in
31

Table 1. The key finding found from these preliminary scoring sessions is that adjustments need to be made to better account for road markings and "spider cracks." Improvements have been demonstrated in three revisions of the crack detection software. The overall effectiveness was defined as the sum of the number of images with no cracks and the number of cracks with 90 percent or greater detection minus the number of false positive identifications. The effectiveness was at 52% for the initial scores, 69% for the second round of scoring, and 83% for the final round. These improvements were the result of changes made in the identification of cracks within crack detection routines and are suitable due to the fact that the same image set was used for both rounds of scoring. One of the goals of this exercise was to find a suitable method for scoring images, but the primary goal was to create an objective way to evaluate crack detection results so that improvements could be made. Some other points about the image set are the variables considered in the scoring software. For instance, a parameter was used called "spider cracks," which refers to any single ~12x12" image that has multiple horizontal or vertical cracks. These "spider cracks" are characterized differently than an image with only vertical or horizontal cracks. The other variables are related to DOT-applied road markings and discolorations on the pavement surface. A detailed tutorial was created to limit any error that could arise from misinterpretation. The primary findings from Table 1 indicate that crack detection software correctly identified 83% of cracks at a level of 90% or better with 15% of false positive responses. Although the number of false positive responses increases, the overall improvement in effectiveness is substantial throughout the three trials of the software.
32

Table 1. Summary of Two Scoring Results From Random Images Taken From GA 100
Full System Parking Lot Tests In November 2010, as the full system neared completion, a series of trials were conducted in a parking lot on the Georgia Tech campus to verify functionality and to aid in completion of the hardware integration. These trials were intended to prepare for a series of road trials to follow. On November 1, 2010, a brief parking lot trial of the crack detection system was conducted. It was confirmed that the Crack Detector was generating the 3-value crack maps that classify each pixel of the crack map as not-crack, array-crack, or longitudinalcrack. On November 8, 2010 we conducted a checkout of the dispensing system using asphalt-based crack sealant. A full block of crack sealant was melted in the melt tank, the entire system was heated to operating temperature, and sealant was dispensed from each of the 12 array applicators. For this checkout, the solenoid valves were actuated manually. In addition, the mixer pump motor frequency drive was turned on, but motor was not able to turn the pump. Just prior to the next parking lot trial, another test was conducted on the mixer pump with the melt tank at the set point temperature. The motor did turn the pump for a few seconds, but was unable to turn the pump on all other attempts. It was confirmed that the frequency drive was supplying well more that the rated stall current of the motor.
33

On November 12, 2010, a parking lot trial of the complete system was conducted. The Crack Detector was generating the 3-value crack maps. Two or more versions of the real-time Navigator were tried. Six of the accumulators were set for of full stroke and six of the accumulators were set for full stroke. We confirmed that the applicators would dispense the set amount of sealant when actuated either manually or by program LEDtest. The system made a series of 16 short runs across the parking lot. Initially only one applicator was enabled. Later, either six or all 12 applicators were enabled. There were perhaps a dozen times when asphalt was dispensed as a result of passing over a crack. Those shots were consistently 25 inches beyond the crack. Very little material was dispensed during those shots. Open time was initially set at 15 ms and later changed to 40 ms. It seemed that much more material was dispensed by LEDtest and by manual actuation than by Navigator.
The next parking lot trial was run on November 17. During that trial, it was discovered that the recently installed cableway on the left side of the trailer frame was keeping the left wheel odometer from firmly contacting the left trailer tire. This problem was corrected by shifting the wheel odometer shaft position axially. This problem was the source of the apparent 25-inch-offset observed in the November 12 trial. Since the left wheel odometer was not always in contact with the trailer tire, the Navigator was calculating a curved trajectory when the trailer was traveling in a straight line. Once the odometer problem was corrected, the system began dispensing crack sealant at the location of the detected crack.
An indoor system checkout was conducted on November 30. During that checkout, the following list of issues was identified.
The longitudinal servo axis is only operating from the center of the range of travel to the right end; it collides at right end. The axis should be homed to the left end of the range of travel. Installation of servo setup software on a laptop PC needed to recover from loss of encoder position and servo axis errors. The Navigator is logging so much data about the operation of the longitudinal axis that it is overwriting the configuration file and much of the needed data from operation of the array nozzles. Applicators 8 and 9 never open or close. LEDs on those optically-isolated output modules never come on. There is a loose panel on the face of the real-time-operating-system PC chassis. There might be three screws on the loose in the PC chassis.
34

The next parking lot trial was run on December 1. A new test routine was available to test simultaneous opening of multiple sealant applicators. Earlier observations suggested that the system was not always successfully opening multiple nozzles simultaneously. Prior to that trial, a compressed air reservoir had been added to the compressed air circuit near the applicator carriage. Tests during this trial confirmed that multiple sealant applicators could be opened simultaneously given the presence of the compressed air reservoir.
Continued Hardware Integration During October and November 2010, a large number of small hardware tasks were completed. Tasks included replacing circuit breakers; connecting the power leads for the two melters from the main cabinet; correcting the wiring of the command digital outputs to the longitudinal servo drive; running the sealant hose, compressed air line, and heater cables along the left side of the trailer; and connecting the solenoid valve for opening the longitudinal nozzle. The servo drive was set up such that upon power up, the servo drive loads the program to position the longitudinal nozzle based on inputs from the Navigator digital outputs. The heater bars were installed in the melt tank extension and the wiring to the cartridge heaters and RTDs in the heater bars was completed.
Issues discovered during the parking lot trials described above generated new hardware action items. The motor driving the melt-tank mixing pump was replaced with a gear motor with sufficient output torque to drive the mixing pump. It was discovered that improper switch settings on the output module chassis had disabled applicators 8 and 9; this was corrected. The home position of the longitudinal axis was corrected such that the longitudinal nozzles homes to the left end of the range of travel rather than to the center of the range of travel. Once these hardware issues were resolved, the system was prepared for the first of two planned road trials.
Full System Road Trials
December 7 road trial on GA 166 The initial full-system road trial was conducted on December 7, 2010. Arrangements were made with Kevin Law for Georgia DOT to provide traffic control in conjunction with a crack sealing operation that they were conducting at the same time. They provided traffic control for the full system road trial on GA166 in Carroll County between mile posts 14 and 15.
35

A number of problems were experienced during the course of the initial road trial including:
A stainless steel washer was accidentally dropped into the large melter the night before the trial. On the morning of the trial, prior to departure, the mixer unit was removed from the melter-pump unit then replaced to resolve the issue and prevent a potential catastrophic pump failure. This delayed arrival to the site by approximately 45 minutes. This issue highlighted the need for a filter or screen in the system. The 120VAC plug dropped out of the generator receptacle in route to the trial site and dragged along the pavement for part of the trip. The plug had to be repaired prior to the start of the trial. For much of the one-hour travel time, heat was not provided to the system components. The lack of heating power meant that equipment dropped to the low ambient temperatures. The temperatures eventually reached set points but this was about 2 hours after arriving at the site. During the course of the trial, one caster wheel on the carriage was damaged beyond repair due to an issue with losing air pressure. A temporary replacement was purchased at a nearby store. Cold temperatures were introducing new behaviors to the dispensing system. An action item was generated to add insulation to key components to limit this effect in future runs.
Despite these obstacles, the system was operated for a few long runs. These runs provided a good feel for how the system performed in the field. For example, the Crack Detector generated false positives in some specific areas of the pavement but not in others.
The operation of the system was broken up into seven runs. At intervals, the system was stopped; data collected by the Navigator in log files was written to hard disk; and images and crack maps saved by the Crack Detector were collected in a folder for that run. Three example sets of images from Run 7, the last run, are included in Figure 16, Figure 17 and Figure 18. The sets include the left and right raw camera images, the resulting crack map, and a digital camera image of the pavement after the system had dispensed sealant into the crack. Figure 16 and Figure 18 include four raw camera images and 2 crack maps. In these examples, the same crack appears in two consecutive images due to the planned overlap of images that guarantees that all of the pavement is imaged for crack detection. In the image set, the crack maps contain pixels in three shades, black, gray, and white. Black pixels are not crack. White pixels are cracks
36

assigned to the array nozzles. Gray pixels are cracks assigned to the longitudinal crack nozzle. The Navigator log files contain data on the actions carried out by the Navigator in response to the crack map inputs. For example, during Run 7, 193 individual applicator opening events were carried out in response to cracks identified in 30 different crack maps.
37

Figure 16: Dispensing example from GA166 Road Trial Run 7 1. 38

Figure 17: Dispensing example from GA166 Road Trial Run 7 2. 39

Figure 18: Dispensing example from GA166 Road Trial Run 7 3. 40

December 20 Road Trial at CCRF A second full-system road trial was conducted on December 20, 2010. Due to issues scheduling Georgia-DOT-provided traffic control for a second road trial prior to the contract end date on December 24, 2010, it was decided to conduct the second road trial on the roads around the GTRI Cobb County Research Facility (CCRF). A system demonstration for Georgia DOT personnel, originally scheduled for December 17, 2010, was rescheduled for the afternoon of December 20 because of weather meaning the final trial of the system was conducted with GDOT personnel present. A few preliminary runs were made at CCRF on December 19 in preparation for the road trial the following day.
During the day of December 20th, data was collected on 9 runs. For each of the runs, the Navigator logged data on array applicator opening events and on positioning of the longitudinal applicator. For each of the runs, the Crack Detector saved all raw camera images and crack maps. For three of the runs, photographs were made after the run of the applied crack sealant. For those photographs, a distance measurement was made with a measuring wheel to facilitate correlation with the raw camera images and crack maps. Video was collected on two of the runs.
An example application of sealant made during Run 6 on December 20 is shown in Figure 19. The figure includes the left and right raw camera images, the resulting crack map, and a photograph of crack sealant applied to this crack. The transverse crack is more pronounced in the left camera image because the lighting for that camera emphasizes the transverse cracks. A very small longitudinal crack can be seen in the right camera image and in the photograph of applied crack sealant. This longitudinal crack was not identified by the Crack Detector.
Table 2 shows that all 12 applicators were opened in response to Crack Map 12202010_15_16_11_265_M.bmp and shows the opening sequence. The applicators are numbered 0 to 11. The even numbered applicators are on the forward row; the odd numbered applicators are on the aft row. Applicator 0 is the left most applicator. From Table 2 it can be observed that the applicators on the forward row were opened before the applicators on the aft row and that, given the shape of the crack, the applicators on the right were opened before the applicators on the left.
41

12202010_15_16_11_265_L.bmp

12202010_15_16_11_265_R.bmp

12202010_15_16_11_265_M.bmp

CCRF101220 105.jpg

Figure 19. A set of raw camera images, crack map, and photograph of applied crack sealant from Run 6 on December 20, 2010

Table 2. Applicators opened in response to Crack Map 12202010_15_16_11_265_M.bmp

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

30

12 10 8 6 4 0 2 11 9 7 5 3 1

42

Figure 20 shows an example of a crack that the Crack Detector has assigned to the longitudinal applicator as indicated by the gray pixels in the crack map. At the bottom of the crack, white pixels indicate that a small portion of the crack was assigned to the array applicators. As a result, the Navigator opened Applicators 4 and 3. The photograph shows the longitudinal crack and the crack sealant applied by the array applicators. No crack sealant was dispensed by the longitudinal applicator; the longitudinal applicator temperature setpoint was too low. Figure 21 shows the crack map alongside a plot of the commanded position of the longitudinal applicator.

12202010_15_16_26_359_L.bmp

12202010_15_16_26_359_R.bmp

12202010_15_16_26_359_M.bmp

CCRF101220 108.jpg

Figure 20. A set of raw camera images, crack map, and photograph of applied crack sealant from Run 6 on December 20, 2010

43

Table 3. Applicators opened in response to Crack Map 12202010_15_16_26_359_M.bmp

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

56

24 3

38

37

12202010_15_16_26_359_M.bmp

36

0

50

100

150

200

250

Figure 21. A crack map alongside a plot of the commanded position of the longitudinal applicator from Run 6 on December 20, 2010

44

Observations from December 20 included: The array applicators were reliably placing crack sealant in the assigned cracks. The Crack Detector did generate some false positive cracks likely resulting from dark regions of the pavement. The Navigator was commanding the longitudinal applicator to track identified longitudinal cracks. Sealant would only flow from the longitudinal applicator after the temperature setpoint was raised to almost 400 F. An additional step of discharging material from the pump would also be required to ensure flow from this applicator during tests. The larger sealant melter-pump unit would not supply sealant at the target pressure of 1000 psi. Sealant pressures ranged from 500 to 800 psi. The sealant supply pressure determines the amount of sealant dispensed and the velocity at which it is dispensed. The polyethylene bags that the sealant blocks are cast in are not melting in the sealant melter-pump unit. On December 20, a polyethylene bag wrapped around the mixer pump shaft, jammed the shaft, and caused a shaft coupling failure. Although inhibiting mixing, this did not stop operation of the melter or pump for delivery to the applicators.
45

Section 5: Conclusions and Recommendations
The research performed by GTRI in conjunction with the Georgia DOT has proved in many ways that a commercial-scale automated crack sealing system is viable. During this research, the following major challenges were demonstrated:
High-speed firing of nozzles (within 20ms) in real-time operation Overall system integration completed on limited-scale system Crack detection algorithms executed within 100 ms Real-time navigation occurring across several stages of coordinate transformations
Despite demonstrating these key performance measures of the system, there are several issues that must be discussed in consideration of expanding the system to be suitable for a full lane width. The following issues must be considered or addressed prior to successfully implementing a full scale system:
1. Obtain melter-pump unit with mixer capable of consistently supplying bituminous-asphalt-based crack sealant at the design pressure. Currently, the design pressure is set to 1000 psi, which was chosen because it was the highest pressure recommended during initial development of the applicators. A full-scale system could use a lower pressure if different springs were incorporated into the accumulator design.
2. Existing Crack Detector uses an image processing approach on camera images of the pavement illuminated in the visible wavelength range. Image processing algorithms are based on contrast as a result of the structured light provided by the illumination system. Experience to date suggests that this method is confounded by regions of high contrast caused by features other than pavement cracks. Examples of confounding regions include dark stains in the pavement, lane stripes, raised-pavement markers, and crack sealant. The 83% efficiency of the current crack detection algorithms are based upon a visible imaging system and this efficiency will not be sufficient for a full-scale system. Future developments may require a fusion of multiple sensors such as a 3D laser scanning system. The output from this type of sensor would be a depth profile. If this depth profile is provided or assembled into depth images, the crack detection algorithms currently in use could be used to find pavement cracks. For example, algorithms designed to identify features that are raised above the mean surface of the pavement could be used to help identify raised features
46

such as crack sealant, lane stripes, and raised-pavement markers eliminating a number of false positives caused by paint markings or dark spots on the pavement. 3. Very limited testing was conducted on the longitudinal crack applicator. Additional testing is required to eliminate remaining software errors, to verify and refine calibration, and to optimize control parameters. Furthermore, the longitudinal crack filling system would require a considerable increase in flow rate of crack sealant to be effective. 4. During the short runs conducted to date, significant amounts of crack sealant were deposited on the skid plate aft of the applicators. Eventually, the buildup would contact the pavement and act as a squeegee. It will be desirable to investigate the causes of this buildup and to reduce or eliminate it. It may be possible to observe the buildup using high-speed vision and a periscope perhaps with a mirror and a coaxial light source. Possible remedies might include raising the height of the applicator carriage or adding a heated shield aft of the applicator array. 5. An investigation of applicator timing was conducted on the single applicator using hot-melt adhesive. No timing tests have been conducted on the array of 12 applicators, even though changes were made to the sealant flow passage design in the manifolds for the array of 12 applicators. No timing tests have been conducted using bituminous-asphalt-based crack sealant. Conduct tests to determine the delay time between energizing the solenoid of the pneumatic valve and initiation of material flow and to determine the length of time required to discharge the accumulator for different accumulator stroke settings. This information may not be necessary as the current demonstration was completed by determining this delay empirically. 6. It is believed that the compressed air circuit as originally implemented was not adequate to permit the opening of multiple applicators simultaneously. A compressed air reservoir was added at the aft end of the trailer to increase the supply pressure at the flow rates required to open multiple applicators simultaneously. Application of sealant during the trials suggests that the system is now capable of opening multiple applicators simultaneously. Some experimental investigation is warranted to verify that material valve performance is identical when 1 material valve is opened and when 12 material valves are opened. 7. The initial goal of the system was to seal cracks at a speed of 5 mph. The goal of 5 mph was not reached during demonstrations because researchers felt the
47

dispensing of sealant into cracks at speeds above 2-3 mph resulted in poor performance on the transverse crack sealing operation. In summary, a number of results were discussed from various trials that took place including several road trials for tuning of image processing techniques as well as evaluation of the prototype system. The final evaluation of the crack detection system, which included grading by a number of judges, resulted in detection of 83% of cracks at a level of 90% or better with 15% of false positive responses where 90% or better refers to capturing 90% or more of the crack in a given crack map. The 83% result was based on a number of images gathered on a particular road trial on GA 100 in March of 2010 where more than 30,000 images were captured. Crack detection performance was found to vary somewhat on different surfaces in later tests, but not formally characterized as the GA 100 tests were. Performance of dispensing was also evaluated in a number of full system trials. Issues with dispensing on the longitudinal servo axis prevented full evaluation of that part of the system, but servo tracking was demonstrated successfully. Dispensing in cracks proved challenging due to issues with pump pressure among other variables yet timing of dispensing actions and performance of the dispensing applicators was demonstrated successfully. Overall, the project team feels as if a successful proof of concept was demonstrated for the automation of crack sealing operations. Once the issues listed above are addressed, the technology will be ready for advancement to a full-scale operational system.
48

Section 6: List of References
[1] L. Galehouse and J. S. Moulthrop, "Roadway Pavement Preservation 2005," in First National Conference on Roadway Pavement Preservation, Kansas City, Missouri, 2005, pp. 120-131.
[2] A. Hand, J., P.E., et al., "Cost-Effectiveness of Joint and Crack Sealing: Synthesis of Practice," Journal of Transportation Engineering, vol. 126, pp. 521-529, November/December 2000.
[3] W. Bekheet, et al., "Transporation Research Circular E-C078," in Roadway Pavement Preservation 2005, Kansas City, Missouri, 2005, pp. 87-98.
[4] C. Haas, et al., "A Design for Automated Pavement Crack Sealing," in Construction Congress 2nd 1991, Cambridge, MA, 1991, pp. 222-227.
[5] C. Haas, et al., "Implelmentation of an Automated Road Maintenance Machine (ARRM)." Project Summary Report for the Center for Transportation Research, Bureau of Engineering Research, University of Text as Austin, 1999.
[6] Y.-S. Kim, et al., "Man-Machine Balanced Crack Sealing Process for UT Automated Road Maintenance Machine," in Proceedings of the Fifth International Conference, Newport Beach, California, 1988.
[7] Y.-S. Kim, et al., "Path Planning for a Machine Vision Assisted, Teleoperated Pavement Crack Sealer," Journal of Transportation Engineering, vol. 124, pp. 137-143, March/April 1998 1998.
[8] Y.-S. Kim and C. T. Haas, "A Man-Machine Balanced Rapid Object Model for Automation of Pavement Crack Sealing and Maintenance," Canadian Journal of Civil Engineering, vol. 29, pp. 459-474, June 2002.
[9] S. A. Velinsky, "Heavy vehicle system for automated pavement crack sealing," Heavy Vehicle Systems, vol. 1, pp. 114-128, 1993.
[10] S. A. Velinsky and K. R. Kirschke, "Design consideration for automated pavement crack sealing machinery," in Proceedings of the 2nd International Conference on Application of Advanced Technologies in Transportation Engineering, Minneapolis, MN, USA, 1991, pp. 76-80.
[11] W. Wei, et al., "A biniary image enhancement and recognition approach in crack detection using exploring agents," in Electronic Imaging and Multimedia Technology IV, Beijing, China, 2005, pp. 287-296.
[12] P. D. Michael I. Hammons, et al., "Detection of stripping in hot mix asphalt," Office of Material and Research, Georgia Department of Transportation, Gainesville, FL 32606 ERES Project No. 16355, August 11, 2004.
[13] L. Bursanescu, et al., "Three-dimensional infared laser vision system for road surface features analysis," in ROMOPTO 2000: Sixth Conference on Optics, Bucharest, Romania, 2001, pp. 801-8.
49

[14] H.-C. Chung and M. Shinozuka, "Highway Surface Distress Inspection using Remote Sensing," in Engineering, Construction and Operations in Challenging Environments Earth and Space 2004: Proceedings of the Ninth Biennial ASCE Aerospace Division International Conference, League City/Houston, TX, United States, 2004, pp. 231-238.
[15] S. A. Guralnick, et al., "Highway pavement surfaces reconstruction by Moire interferometry," in Proceedings of the 2nd International Conference on Applications of Advanced Technologies in Transportation Engineering, Minneapolis, MN, USA, 1991, pp. 71-75.
[16] M. S. Kaseko and S. G. Ritchie, "Pavement Image Processing Using Neural Networks," in Proceedings of the 2nd International Conference on Applications of Advanced Technologies in Transportation Engineering, Minneapolis, MN, USA, 1991, pp. 238-242.
[17] Wiley Holcombe, Jim Clark, Steven Robertson, Marlon Moses, Victor Breedveld, and Wayne Daley, Phase 1 Research Report - Development of an Automated Pavement Crack Sealing System (A-7318), Georgia Institute of Technology, December 3, 2004.
[18] Wayne Daley, Wiley Holcombe, Steven Robertson, Jonathan Holmes, and Colin Usher, Phase 3 Research Report, Report on limited, six-month, Phase 3 effort Development of an Automated Pavement Crack Sealing System (A-7318), Georgia Institute of Technology, December 12, 2005.
[19] Wayne Daley, Wiley Holcombe, Doug Britton, Jonathan Holmes, and Colin Usher, Phase 2 Research Report - Development of an Automated Pavement Crack Sealing System (A-7318), Georgia Institute of Technology, December 6, 2006.
50

APPENDIX A Crack Detection Tuning Results
GOAL: Determine acceptable values for the seven image analysis parameters. The parameters are:
threshold 1 threshold 2 major axis length minor axis length major/minor ratio maximum reasonable crack size orientation angle
The crack detection algorithm did not use the major/minor ratio. The "maximum reasonable crack size" parameter seems misnamed. The area of the crack needs to be greater than this value. A more descriptive name might be "minimum reasonable crack size".
A number of tests were run to find a reasonable range of values for these parameters. It was inconclusive except for the thresholds. A threshold range of -21 to -24 gives reasonable results. A test was run on the fourteen images in the sample set. The thresholds varied from -21 to -24, the major axis was 75, minor axis was 30 and the maximum crack size was 0. With four values for each threshold, a full factorial experiment is 16 tests per filename. The results are shown in the graph below. The chart is sorted by filename, threshold 1 and then threshold 2. The table below shows the association between filenames and data points.
The chart shows that the ratio of pixels inside the crack and the ratio of pixels outside the crack decreased with decreasing values of threshold 1. The value of threshold two was irrelevant except for image file Run14_Rimage008.bmp, points 129-144. In this image, for a constant threshold 1 value, if threshold 2 was greater than or equal to it, the ratios inside and outside the crack decreased compared to when threshold 2 was greater than threshold 1. The one exception was when T1=T2=-23, point 139. In this case the ratios are equal to the ratios when T1=-23 and T2 = -24.
A - 1

Data Points 1-16 17-32 33-48 49-64 65-80 81-96 97-112 113-128 129-144 145-160 161-176 177-192 193-208 209-224

Table A- 1 Filename Run14_Rimage165.bmp Run14_Rimage164.bmp Run14_Rimage162.bmp Run14_Rimage132.bmp Run14_Rimage111.bmp Run14_Rimage110.bmp Run14_Rimage042.bmp Run14_Rimage041.bmp Run14_Rimage008.bmp Run13_Rimage164.bmp Run13_Rimage163.bmp Run13_Rimage108.bmp Run13_Rimage039.bmp Run13_Rimage005.bmp

A - 2

Score-Cracktuning-All-001.xls

14 Files, Maj. Axis Len=75, Minor Axis Len=30, MaxCrackSize=0

-19

Sort: filename, threshold 1, threshold 2

1

1

17

33

49

65

81

97

113 129 145 161 177 193 209

0.9

-20

0.8

0.7

-21

0.6

threshold 1

threshold 2

0.5

ratioPixelsInside

ratioPixelsOutside

-22

0.4

0.3

-23

0.2

0.1

-24

0

Figure A- 1. Score-Cracktuning-All

A - 3

The uniqueness of image "Run14_Rimage008.bmp" was interesting. It was used it to create a color image showing the reference crack map from Matlab, the crack map from the OpenCV code and the pixels outside the crack map from the OpenCV code. The pixels in the OpenCV crack map that are also in the Matlab crack map are shown in blue. The pixels in the Matlab crack map that are not in the OpenCV crack map are shown in green. The pixels in the OpenCV crack map that are not in the Matlab crack map are shown in red. Sixteen color images were created using four values for threshold1 and four for threshold2. The major axis length was 75, the minor axis length was 30 and maximum crack size was 0. These are the same parameters used in the full factorial experiment above for all the image files. A surprising result was that the pixels outside the crack map shown in red tend to be on the boundary of the crack. The chart below shows the results of the 16 tests. The sixteen color crack maps are below the chart. The crack maps are ordered from the lowest ratio of pixels inside to the highest.
A - 4

Score-CrackTuning-008-010.xls Maj. Axis Len=75, Minor Axis Len=30, MaxCrackSize=0
sort: ratioPixelsInside, threshold1, threshold2

-19

1

1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16

0.9

-20

0.8

0.7

-21

0.6

0.5

-22

0.4

0.3

-23

0.2

0.1

-24

0

threshold 1 threshold 2 ratioPixelsInside ratioPixelsOutside

Figure A- 2. Score-CrackTuning-088

A - 5

Figure A- 3. Raw Image: Run14_Rimage008.bmp A - 6

Figure A- 4. Matlab Output: Run14_Rimage008_out A - 7

Ratio Inside = 62% Ratio Outside = 4% T1 = -24 T2 = -21 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 5. Test Case 1

A - 8

Ratio Inside = 62% Ratio Outside = 4% T1 = -24 T2 = -22 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 6. Test Case 2

A - 9

Ratio Inside = 62% Ratio Outside = 4% T1 = -24 T2 = -23 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 7. Test Case 3

A - 10

Ratio Inside = 62% Ratio Outside = 4% T1 = -24 T2 = -24 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 8. Test Case 4

A - 11

Ratio Inside = 68% Ratio Outside = 6% T1 = -23 T2 = -21 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 9. Test Case 5

A - 12

Ratio Inside = 68% Ratio Outside = 6% T1 = -23 T2 = -22 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 10. Test Case 6

A - 13

Ratio Inside = 73% Ratio Outside = 8% T1 = -22 T2 = -21 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 11. Test Case 7

A - 14

Ratio Inside = 73% Ratio Outside = 8% T1 = -22 T2 = -22 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 12. Test Case 8

A - 15

Ratio Inside = 77% Ratio Outside = 11% T1 = -21 T2 = -21 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 13. Test Case 9

A - 16

Ratio Inside = 78% Ratio Outside = 7% T1 = -23 T2 = -23 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 14. Test Case 10

A - 17

Ratio Inside = 78% Ratio Outside = 7% T1 = -23 T2 = -24 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 15. Test Case 11

A - 18

Ratio Inside = 85% Ratio Outside = 10% T1 = -22 T2 = -23 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 16. Test Case 12

A - 19

Ratio Inside = 85% Ratio Outside = 10% T1 = -22 T2 = -24 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 17. Test Case 13

A - 20

Ratio Inside = 89% Ratio Outside = 13% T1 = -21 T2 = -22 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 18. Test Case 14

A - 21

Ratio Inside = 89% Ratio Outside = 13% T1 = -21 T2 = -23 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 19. Test Case 15

A - 22

Ratio Inside = 89% Ratio Outside = 13% T1 = -21 T2 = -24 Major Axis Length = 75 Minor Axis Length = 30 Maximum Crack Size = 0 Angle = 0.78539 Radians

Figure A- 20. Test Case 16

A - 23

APPENDIX B Initial Matlab Crack Detection Algorithm Development
Activities this past quarter supported the porting of the crack detection code from the Matlab development and analysis environment to the implementation platform. The functions did not map one to one so it was necessary to identify a sequence of operations and features to provide equivalent functionality. This was done using the right-hand images. (In our terminology, right-hand image of the image pair is the image that is tuned to find transverse cracks.) In addition we worked on looking at techniques to improve the performance of the code in terms of processing speed. Additional work included activities to support preparation for a July 1 demonstration of the system for sponsor.
An analysis of the potential features that could be used to aid crack identification while reducing the effect of noise (image components that are not cracks) was conducted. The potential features expected to be significant were:
High threshold (selected empirically to maximize the area [extent] of crack found) Low threshold (selected empirically as a filter for noise) Major and Minor axis lengths Ratio of major and minor axis lengths Eccentricity (Related to above ratio) Size of suspected crack Angle of crack ( this was chosen 45 for right images) Euler number
The image processing environment (OpenCV) on the delivery platform did not support all of these features inherently without additional coding so we initially used a subset as shown below
High threshold (selected empirically to maximize the area [extent] of crack found) Low threshold (selected empirically as a filter for noise) Major and Minor axis lengths Size of suspected crack Angle of crack (this was chosen 45 for right images)
The resulting outputs for a test set of images are shown in Figure B-1 through Figure B39. They are presented as a sequence of three images in which the first is the raw image
B - 1

the second is the Matlab output and the last is that generated on the delivery platform. The cracks appear to be well represented in the outputs on the delivery platform. The image in Figure B-28 is not detected as the analysis is tuned to find horizontally oriented cracks. It is anticipated that we will need to adjust the thresholds or possibly modify the feature set as we get more data from road testing. Algorithm Tuning A number of experiments were run on an existing pavement image set to find a reasonable range of values for the features or parameters listed above. The experiments showed no impact on performance over the range explored for all parameters except the thresholds. A threshold range of -21 to -24 gives reasonable results. A test was run on the fourteen images in the sample set. The thresholds varied from -21 to -24, the major axis was 75, minor axis was 30 and the maximum crack size was 0. With four values for each threshold, a full factorial experiment is 16 tests per filename. The complete results are given in the Appendix of this report.
B - 2

Figure B-1. Original Input Image 005

Figure B-2. Matlab Output 005

Figure B-3. Delivery Platform Output 005

Figure B-4. Original Input Image 008

Figure B-5. Matlab Output 008

Figure B-6. Delivery Platform Output 008

Figure B-7. Original Input Image 039

Figure B-8. Matlab Output 039

Figure B-9. Delivery Platform Output 039

B - 3

Figure B-10. Original Input Image 041

Figure B-11. Matlab Output Figure B-12. Delivery Platform

041

Output 041

Figure B-13. Original Input Image 042

Figure B-14. Matlab Output Figure B-15. Delivery Platform

042

Output 042

Figure B-16. Original Input Image 108

Figure B-17. Matlab Output Figure B-18. Delivery Platform

108

Output 108

B - 4

Figure B-19. Original Input Image 110

Figure B-20. Matlab Output Figure B-21. Delivery Platform

110

Output 110

Figure B-22. Original Input Image 111

Figure B-23. Matlab Output Figure B-24. Delivery Platform

111

Output 111

Figure B-25. Original Input Image 132

Figure B-26. Matlab Output Figure B-27. Delivery Platform

132

Output 132

B - 5

Figure B-28. Original Input Image 162

Figure B-29. Matlab Output Figure B-30. Delivery Platform

162

Output 162

Figure B-31. Original Input Image 163

Figure B-32. Matlab Output Figure B-33. Delivery Platform

163

Output 163

Figure B-34. Original Input Image OR164

Figure B-35. Matlab Output Figure B-36. Delivery Platform

OR164

Output OR164

B - 6

Figure B-37. Original Input Image 165

Figure B-38. Matlab Output Figure B-39. Delivery Platform

165

Output 165

B - 7

Appendix C Run 6, December 20, 2010, GTRI Cobb County Research Facility
C - 1

Summary of applicator discharges during Run 6 on December 20, 2010

Imagefilename 12202010_15_15_55_312_M.csv 12202010_15_16_01_328_M.csv 12202010_15_16_11_265_M.csv 12202010_15_16_17_562_M.csv 12202010_15_16_22_937_M.csv 12202010_15_16_24_359_M.csv 12202010_15_16_24_734_M.csv 12202010_15_16_26_359_M.csv 12202010_15_16_32_296_M.csv 12202010_15_16_32_765_M.csv 12202010_15_16_46_500_M.csv 12202010_15_16_46_968_M.csv 12202010_15_17_21_765_M.csv 12202010_15_17_36_625_M.csv 12202010_15_17_37_203_M.csv 12202010_15_18_09_437_M.csv 12202010_15_18_24_906_M.csv 12202010_15_18_41_890_M.csv 12202010_15_18_42_343_M.csv 12202010_15_18_49_078_M.csv 12202010_15_18_55_187_M.csv 12202010_15_18_57_281_M.csv 12202010_15_18_58_218_M.csv 12202010_15_19_00_250_M.csv 12202010_15_19_00_750_M.csv 12202010_15_19_02_687_M.csv 12202010_15_19_05_015_M.csv 12202010_15_19_10_562_M.csv 12202010_15_19_11_906_M.csv 12202010_15_19_13_187_M.csv 12202010_15_19_15_484_M.csv 12202010_15_19_18_328_M.csv 12202010_15_19_18_953_M.csv 12202010_15_19_19_593_M.csv 12202010_15_19_22_015_M.csv 12202010_15_19_24_546_M.csv 12202010_15_19_32_515_M.csv 12202010_15_19_33_093_M.csv 12202010_15_19_49_437_M.csv

PCtime

deltatime distance IO_NozzleOpenTime

imageNum total & list of nozzles 1 2 3 4 5 6 7

12/20/10 2:15:55.312 PM 01:24.960

2.67

12/20/10 2:14:30.352 PM

4

4 8 6 9 7

12/20/10 2:16:01.328 PM 01:26.663

8.67

12/20/10 2:14:34.665 PM

13

6 2 10 8 3 11 7

12/20/10 2:16:11.265 PM 01:26.320

20.00

12/20/10 2:14:44.945 PM

30

12 10 8 6 4 0 2 11

12/20/10 2:16:17.562 PM 01:26.821

26.00

12/20/10 2:14:50.741 PM

39

3 4 2 3

12/20/10 2:16:22.937 PM 01:27.014

31.33

12/20/10 2:14:55.923 PM

47

2 4 3

12/20/10 2:16:24.359 PM 01:27.510

33.33

12/20/10 2:14:56.849 PM

50

12/20/10 2:16:24.734 PM 01:27.882

34.00

12/20/10 2:14:56.852 PM

51

8 8 0 6 2 9 7 1

12/20/10 2:16:26.359 PM 01:27.904

37.33

12/20/10 2:14:58.455 PM

56

2 4 3

12/20/10 2:16:32.296 PM 01:26.593

46.67

12/20/10 2:15:05.703 PM

70

12/20/10 2:16:32.765 PM 01:26.985

47.33

12/20/10 2:15:05.780 PM

71

10 2 8 4 0 6 1 3

12/20/10 2:16:46.500 PM 01:26.196

61.33

12/20/10 2:15:20.304 PM

92

12/20/10 2:16:46.968 PM 01:26.650

62.00

12/20/10 2:15:20.318 PM

93

12 8 10 6 4 2 0 11

12/20/10 2:17:21.765 PM 01:26.033

99.33

12/20/10 2:15:55.732 PM

149

12 8 10 6 4 2 0 9

12/20/10 2:17:36.625 PM 01:26.627

118.67

12/20/10 2:16:09.998 PM

178

12/20/10 2:17:37.203 PM 01:27.196

119.33

12/20/10 2:16:10.007 PM

179

12 2 0 4 6 8 10 1

12/20/10 2:18:09.437 PM 01:27.232

151.33

12/20/10 2:16:42.205 PM

227

12 10 8 4 6 2 0 11

12/20/10 2:18:24.906 PM 01:27.519

174.00

12/20/10 2:16:57.387 PM

261

12 10 8 6 4 2 0 11

12/20/10 2:18:41.890 PM 01:26.947

196.00

12/20/10 2:17:14.943 PM

294

12/20/10 2:18:42.343 PM 01:27.385

196.67

12/20/10 2:17:14.958 PM

295

12 2 0 4 8 10 6 1

12/20/10 2:18:49.078 PM 01:27.514

206.00

12/20/10 2:17:21.564 PM

309

1 10

12/20/10 2:18:55.187 PM 01:27.514

215.33

12/20/10 2:17:27.673 PM

323

4 0 10 1 9

12/20/10 2:18:57.281 PM 01:26.694

218.67

12/20/10 2:17:30.587 PM

328

2 0 1

12/20/10 2:18:58.218 PM 01:26.960

220.00

12/20/10 2:17:31.258 PM

330

10 10 8 6 2 4 11 9

12/20/10 2:19:00.250 PM 01:26.890

222.67

12/20/10 2:17:33.360 PM

334

12/20/10 2:19:00.750 PM 01:27.376

223.33

12/20/10 2:17:33.374 PM

335

9 0 6 8 10 2 1 11

12/20/10 2:19:02.687 PM 01:26.747

226.00

12/20/10 2:17:35.940 PM

339

2 6 7

12/20/10 2:19:05.015 PM 01:26.965

229.33

12/20/10 2:17:38.050 PM

344

4 4 6 3 7

12/20/10 2:19:10.562 PM 01:27.193

237.33

12/20/10 2:17:43.369 PM

356

2 4 3

12/20/10 2:19:11.906 PM 01:27.122

239.33

12/20/10 2:17:44.784 PM

359

2 0 1

12/20/10 2:19:13.187 PM 01:27.387

241.33

12/20/10 2:17:45.800 PM

362

5 6 4 10 5 9

12/20/10 2:19:15.484 PM 01:27.629

245.33

12/20/10 2:17:47.855 PM

368

10 0 6 8 2 4 7 3

12/20/10 2:19:18.328 PM 01:27.300

250.67

12/20/10 2:17:51.028 PM

376

2 4 3

12/20/10 2:19:18.953 PM 01:27.752

252.00

12/20/10 2:17:51.201 PM

378

2 10 9

12/20/10 2:19:19.593 PM 01:27.658

253.33

12/20/10 2:17:51.935 PM

380

3 8 6 7

12/20/10 2:19:22.015 PM 01:27.432

258.00

12/20/10 2:17:54.583 PM

387

2 2 1

12/20/10 2:19:24.546 PM 01:27.624

262.00

12/20/10 2:17:56.922 PM

393

4 0 6 1 5

12/20/10 2:19:32.515 PM 01:27.668

278.00

12/20/10 2:18:04.847 PM

417

2 2 1

12/20/10 2:19:33.093 PM 01:27.825

279.33

12/20/10 2:18:05.268 PM

419

10 10 8 6 0 4 11 9

12/20/10 2:19:49.437 PM 01:27.770

316.67

12/20/10 2:18:21.667 PM

475

3 2 0 1

8 9 10 11 12 9 7 5 3 1
5 7 9 5 7 9 5 3 1 7 11 5 3 1 3 5 7 9 11 9 7 5 3 1 9 7 5 3 1 3 5 9 11 7
7 5 3 7 9
1 11 5
7 3 5

C - 2

2202010_15_15_55_312_L.bmp

12202010_15_15_55_312_R.bmp

12202010_15_15_55_312_M.bmp

CCRF101220 102.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

4

48 6 9 7

C - 3

12202010_15_16_01_328_L.bmp

12202010_15_16_01_328_R.bmp

12202010_15_16_01_328_M.bmp

CCRF101220 104.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

13

6 2 10 8 3 11 7

C - 4

12202010_15_16_11_265_L.bmp

12202010_15_16_11_265_R.bmp

12202010_15_16_11_265_M.bmp

CCRF101220 105.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

30

12 10 8 6 4 0 2 11 9 7 5 3 1

C - 5

2202010_15_16_17_562_L.bmp

12202010_15_16_17_562_R.bmp

12202010_15_16_17_562_M.bmp

CCRF101220 106.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

39

34 23

C - 6

12202010_15_16_22_937_L.bmp

12202010_15_16_22_937_R.bmp

12202010_15_16_22_937_M.bmp

CCRF101220 107.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

47

24 3

C - 7

12202010_15_16_24_359_L.bmp

12202010_15_16_24_359_R.bmp

12202010_15_16_24_359_M.bmp

CCRF101220 107.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

50

0

C - 8

12202010_15_16_26_359_L.bmp

12202010_15_16_26_359_R.bmp

12202010_15_16_26_359_M.bmp

CCRF101220 108.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

56

24 3

C - 9

12202010_15_16_32_765_L.bmp

12202010_15_16_32_765_R.bmp

12202010_15_16_32_765_M.bmp

CCRF101220 109.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

71

10 2 8 4 0 6 1 3 7 9 5

C - 10

12202010_15_16_46_968_L.bmp

12202010_15_16_46_968_R.bmp

12202010_15_16_46_968_M.bmp

CCRF101220 110.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

93

12 8 10 6 4 2 0 11 7 9 5

C - 11

C - 12

12202010_15_17_21_765_L.bmp

12202010_15_17_21_765_R.bmp

12202010_15_17_21_765_M.bmp

CCRF101220 111.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12

149

12 8 10 6 4 2 0 9 7 11 5

C - 13

12202010_15_17_36_625_L.bmp

12202010_15_17_36_625_R.bmp

12202010_15_17_36_625_M.bmp

CCRF101220 112.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 178

C - 14

12202010_15_18_09_437_L.bmp

12202010_15_18_09_437_R.bmp

12202010_15_18_09_437_M.bmp

CCRF101220 113.jpg

imageNum 227

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 12 10 8 4 6 2 0 11 9 7 5 3 1

C - 15

12202010_15_18_24_906_L.bmp

12202010_15_18_24_906_R.bmp

12202010_15_18_24_906_M.bmp

CCRF101220 114.jpg

imageNum 261

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 12 10 8 4 6 2 0 11 9 7 5 3 1

C - 16

12202010_15_18_41_890_L.bmp

12202010_15_18_41_890_R.bmp

12202010_15_18_41_890_M.bmp

CCRF101220 115.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 294

C - 17

12202010_15_18_49_078_L.bmp

12202010_15_18_49_078_R.bmp

12202010_15_18_49_078_M.bmp

CCRF101220 116.jpg

imageNum 309

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 1 10

C - 18

12202010_15_18_57_281_L.bmp

12202010_15_18_57_281_R.bmp

12202010_15_18_57_281_M.bmp

CCRF101220 117.jpg

imageNum 328

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 20 1

C - 19

12202010_15_18_58_218_L.bmp

12202010_15_18_58_218_R.bmp

12202010_15_18_58_218_M.bmp

CCRF101220 118.jpg

imageNum 330

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 10 10 8 6 2 4 11 9 7 5 3

C - 20

12202010_15_19_00_250_L.bmp

12202010_15_19_00_250_R.bmp

12202010_15_19_00_250_M.bmp

CCRF101220 119.jpg

imageNum total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 334

C - 21

12202010_15_19_02_687_L.bmp

12202010_15_19_02_687_R.bmp

12202010_15_19_02_687_M.bmp

CCRF101220 120.jpg

imageNum 339

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 26 7

C - 22

12202010_15_19_05_015_L.bmp

12202010_15_19_05_015_R.bmp

12202010_15_19_05_015_M.bmp

CCRF101220 121.jpg

imageNum 334

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 44 63 7

C - 23

12202010_15_19_10_562_L.bmp

12202010_15_19_10_562_R.bmp

12202010_15_19_10_562_M.bmp

CCRF101220 122.jpg

imageNum 356

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 24 3

C - 24

12202010_15_19_11_906_L.bmp

12202010_15_19_11_906_R.bmp

12202010_15_19_11_906_M.bmp

CCRF101220 123.jpg

imageNum 359

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 20 1

C - 25

12202010_15_19_13_187_L.bmp

12202010_15_19_13_187_R.bmp

12202010_15_19_13_187_M.bmp

CCRF101220 123.jpg

imageNum 362

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 5 6 4 10 5 9

C - 26

12202010_15_19_15_484_L.bmp

12202010_15_19_15_484_R.bmp

12202010_15_19_15_484_M.bmp

CCRF101220 125.jpg

imageNum 368

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 10 0 6 8 2 4 7 3 1 11 5

C - 27

12202010_15_19_18_328_L.bmp

12202010_15_19_18_328_R.bmp

12202010_15_19_18_328_M.bmp

CCRF101220 126.jpg

imageNum 376

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 24 3

C - 28

12202010_15_19_18_953_L.bmp

12202010_15_19_18_953_R.bmp

12202010_15_19_18_953_M.bmp

CCRF101220 126.jpg

imageNum 378

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 2 10 9

C - 29

12202010_15_19_19_593_L.bmp

12202010_15_19_19_593_R.bmp

12202010_15_19_19_593_M.bmp

CCRF101220 126.jpg

imageNum 380

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 38 6 7

C - 30

12202010_15_19_22_015_L.bmp

12202010_15_19_22_015_R.bmp

12202010_15_19_22_015_M.bmp

CCRF101220 127.jpg

imageNum 387

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 22 1 1

C - 31

12202010_15_19_24_546_L.bmp

12202010_15_19_24_546_R.bmp

12202010_15_19_24_546_M.bmp

CCRF101220 128.jpg

imageNum 393

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 40 6 1 5

C - 32

12202010_15_19_32_515_L.bmp

12202010_15_19_32_515_R.bmp

12202010_15_19_32_515_M.bmp

CCRF101220 129.jpg

imageNum 417

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 22 1

C - 33

12202010_15_19_33_093_L.bmp

12202010_15_19_33_093_R.bmp

12202010_15_19_33_093_M.bmp

CCRF101220 129.jpg

imageNum 419

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 10 10 8 6 0 4 11 9 7 3 5

C - 34

12202010_15_19_49_437_L.bmp

12202010_15_19_49_437_R.bmp

12202010_15_19_49_437_M.bmp

CCRF101220 130.jpg

imageNum 475

total & list of nozzles 1 2 3 4 5 6 7 8 9 10 11 12 32 0 1

C - 35