FOLLOW-UP REVIEW Prepared for the Budgetary Responsibility Oversight Committee Russell W. Hinton, State Auditor An Assessment of the Georgia Police Academy's Results-Based Budget Goals and Desired Results October 2003 Performance Audit Operations Division 254 Washington St. S.W. Department of Audits and Accounts Atlanta, GA 30334-8400 This is a Follow-Up Review of the August 2001 Program Evaluation on the Assessment of the Georgia Police Academy's Results-Based Budget Goals and Desired Results conducted by the Performance Audit Operations Division. Purpose of the Georgia Police Academy The purpose of the Georgia Police Academy (GPA) is to improve law enforcement personnel's knowledge, skills, and performance in the execution of their duties. The GPA is primarily responsible for providing advanced and specialized training to state and local law enforcement personnel and also has the responsibility for training County Coroners. Issues Addressed Specific objectives of the original program evaluation were: To comment on the validity of the Results-Based Budget (RBB) goals and desired results; and To evaluate the reliability of the RBB data. Background Created in 1962 by the Georgia Police Academy Act (OCGA 35-4), the GPA operated as an independent state entity until a fiscal year 1993 reorganization by the Board of Public Safety that consolidated the GPA, Georgia Fire Academy, and the Georgia Public Safety Training Center (GPSTC). Since legislation has not been enacted to consolidate the entities, the GPA and the Georgia Fire Academy are independent entities for budget purposes. For operational purposes, they function as divisions of the GPSTC. The GPA is located at the GPSTC in Forsyth. Activity Data and Funding In fiscal year 2002, the GPA offered over 400 courses attended by approximately 9,800 students. GPA reported fiscal year 2003 expenditures that totaled approximately $2.1 million, with $1.2 million in state funds, $840,665 in federal funds, and $101,063 in funds from other sources. Summary of the Program Evaluation While the program evaluation found that GPA's RBB goals and desired results were valid indicators of its effectiveness, the evaluation noted that the desired results could be improved Follow-Up Review of the Georgia Police Academy 1 and that several factors had an adverse impact on the validity of the RBB data. In addition, the program evaluation found that several factors had an adverse impact on the reliability of the RBB data. Results-Based Budgeting The Budgetary Accountability and Planning Act of 1993 requires that programs provide data in their Results-Based Budget to indicate how well they are solving the problems they were created and funded to address. The intent for capturing and reporting such data is to reflect the progress the program has made toward achieving its purpose for existing. After the program evaluation was released, the GPA, with the approval of the Office of Planning and Budget, changed their RBB. According to the GPA, RBB was changed to better reflect its mission through measuring the availability and quality of the training it provides. The GPA's goals and desired results measures, as detailed in the Governor's Budget Report for fiscal year 2002, are shown in Exhibit 1. The GPA's goals and desired results measures, as detailed in the Governor's Budget Report for fiscal year 2004, are shown in Exhibit 2 and reflect GPA's revised RBB. EXHIBIT 1 GEORGIA POLICE ACADEMY Fiscal Year 2002 Purpose: To improve law enforcement personnel's knowledge, skills, and performance in the execution of their duties. Goal 1: Improve the ability of law enforcement in Georgia to protect lives and property through quality training. - The percentage of Police Academy students who attend Police Academy FY 2000 Desired FY 2000 Actual courses and pass that show a minimum of 10% improvement in subject matter knowledge, skills, and abilities based on pre-and post-testing. 64% 79% - In a six-month follow-up survey, 50% of employers will state that the course- work at the Police Academy was relevant and helpful. 50% N/A Source: Governor's Budget Report, Fiscal Year 2002, p. 419 EXHIBIT 2 GEORGIA POLICE ACADEMY Fiscal Year 2004 Purpose: Develop and deliver training that is consistent with the contemporary needs of Georgia's law enforcement personnel. Goal 1: The availability of quality training programs that result in competent on-the-job performance. Desired Result 1: Annually the Georgia Police Academy satisfies 100% of all training requests received. Desired Result Actual Result FY 2001 100% 97.22% FY 2002 100% 97.85% FY 2003 100% FY 2004 100% Goal 2: The delivery of quality training programs that result in competent on-the-job performance. Desired Result 1a: In an annual follow-up survey, 100% of employers will state that the coursework at the Police Academy was relevant and helpful. FY 2001 FY 2002 FY 2003 FY 2004 Desired Result 100% 100% 100% 100% Actual Result 98.06% 95.69% Note 1: Agencies send numerous students throughout the year. Conducting individual follow-up interviews is not cost effective. Source: Governor's Budget Report, Fiscal Year 2004, p. 631 2 Follow-Up Review of the Georgia Police Academy Status of Findings and Recommendations Finding (as reported in 2001): The GPA's fiscal year 2002 Results-Based Budgeting (RBB) goal and desired results are valid indicators of its effectiveness. However, the desired results could be improved. In addition, there are several factors that adversely impact the validity of the RBB data. Current Status: As previously stated, the GPA has changed their RBB, and therefore, the recommended potential improvements stated in the program evaluation are no longer applicable. The revised goal and desired results are valid indicators of GPA's effectiveness. However, the audit team did note problems with the actual survey used to measure progress toward the second goal. The survey used to obtain satisfaction data regarding fiscal year 2002 training at GPA questioned law enforcement agencies about training received at the regional police academies, not GPA. This information was reported as GPA's RBB measure in the Governor's Budget Report for fiscal year 2004. Also, the survey is sent to all law enforcement agencies rather than only to those agencies that have undergone training at the GPA during that year. Finding (as reported in 2001): The evaluation found that there are several factors that adversely impact the reliability of the RBB data. Current Status: The original report found that the reliability of the RBB data was adversely impacted by the correctness of the data (instructors' calculations of course averages were not verified), the completeness of the data (GPA did not ensure that all the courses that should be in the RBB database were, in fact, in the database), and the security of the RBB database. With the changes to GPA's RBB goals, the issues related to the correctness and completeness of data, and to the security of the database have been corrected. Follow-Up Review of the Georgia Police Academy 3 For additional information or to request a copy of the program evaluation, contact Paul E. Bernard, Director, Performance Audit Operations Division, at 404-657-5220 or go to our website: www.audits.state.ga.us/internet/pao/index.html. 4 Follow-Up Review of the Georgia Police Academy