Follow-Up Review Report No. 17-18 May 2018 Georgia Department of Audits and Accounts Performance Audit Division Greg S. Griffin, State Auditor Leslie McGuire, Director Why we did this review This follow-up review was conducted to determine the extent to which the Georgia Department of Education, Georgia Professional Standards Commission, and the General Assembly addressed the recommendations presented in our December 2015 special examination (Report #15-14). The 2015 special examination was conducted at the request of the House Appropriations Committee. The Committee requested that we determine if the salary incentives are working as intended. About Salary Incentives House Bill 280 (effective July 1, 2009) established two types of math and science salary incentives. The salary incentive for grades K-5 teachers is intended to increase math and science content knowledge for teachers who earn a math or science endorsement. A teacher may receive $1,000 per endorsement per year for up to five years. The salary incentive for early career teachers certified to teach grades 6-12 math and science is intended to increase recruitment and retention in these subject areas. The incentive amount is equal to the difference between an early career teacher's pay, which is based on his/her actual years of experience, and the salary of a teacher with six years of experience. As a result, incentive payments range from approximately $2,500 to $4,500 per year for the first five years of teaching. Follow-Up Review Math and Science Salary Incentives for Teachers Design and implementation problems still exist What we found Since our 2015 special examination, the General Assembly, Professional Standards Commission (PSC), and the Georgia Department of Education (GaDOE) have taken few steps to address report recommendations. The original examination noted the math and science salary incentives are not working as intended. Their impact on content knowledge of Georgia's grades K-5 teachers and on recruitment and retention of early career middle and high school math and science teachers has been limited by various design and implementation problems. The total annual cost of the incentives is approximately $15 million, for a total of $120 million expended since the incentives' inception. Grades K-5 Salary Incentive The number of K-5 teachers attaining math or science endorsement remains low. As shown in Exhibit 1, 79 of 180 (44%) school systems have a teacher with a K-5 math and/or science endorsement. Since our evaluation, the number of teachers with an endorsement has increased from 572 to 1,000, with five school systems accounting for 49% of the growth. These 1,000 kindergarten and elementary school teachers who hold a math and/or science endorsement represent approximately 6.2% of the 16,219 teachers GaDOE estimates are eligible for the endorsement. Funding for the endorsement has been available since fiscal year 2013. 270 Washington Street, SW, Suite 1-156 Atlanta, Georgia 30334 Phone: (404)656-2180 www.audits.ga.gov Math and Science Salary Incentives for Teachers 2 The original examination noted that the out-of-pocket cost for K-5 teachers to attain the math and science endorsements is high when compared to the incentive payment they receive from attaining those endorsements. The $1,000 annual salary incentive per endorsement has not changed since the original examination was published, nor have there been any efforts made to decrease the costs of attaining these endorsements. A survey conducted by PSC indicated that improving the cost-benefit of attaining an endorsement could improve the likelihood that educators would complete a K-5 math or science endorsement. The cost of coursework to obtain the math or science endorsement ranges from $576 to $4500. Exhibit 1 As of October 2017, 44% (79 of 180) of School Systems Had Teachers Who Hold a K-5 Math and/or Science Endorsement with Eight School Systems Accounting for 497 Out of 1,000 Teachers with an Endorsement. Dade Catoosa Dalton Whitfield Chickamauga Walker Murray Fannin Gilmer Towns Union Rabun White Habersham 0 Teachers 1-5 Teachers Trion Chattooga Floyd Rome Polk Haralson Bremen Gordon Calhoun Pickens Lumpkin Hall Stephens Banks Franklin Hart Bartow Cartersville Cherokee Forsyth Gainesville Commerce Jackson Jefferson Madison Elbert Marietta Paulding Cobb Buford Gwinnett Dekalb Barrow Clarke Oconee Oglethorpe Atlanta Decatur City Walton Wilkes Douglas Lincoln 6-20 Teachers 21-40 Teachers 41-80 Teachers Carroll Carrolton Fulton Clayton Fayette Henry Newton Morgan Greene Taliaferro Warren Heard Coweta Spalding Jasper Butts Putnam Hancock Glascock Columbia Richmond Troup Meriwether Pike Lamar Monroe Jones Baldwin Jefferson Washington Burke Upson Harris Talbot Muscogee Crawford Bibb Taylor Peach Wilkinson Twiggs Dublin Johnson Emanuel Jenkins Screven Marion Macon Houston Bleckley Laurens Treutlen Candler Bulloch Effingham Schley Stewart Webster Sumter Quitman Randolph Terrell Lee Clay Pulaski Dooly Dodge Wheeler Crisp Wilcox Telfair Turner Ben Hill Jeff Davis Vidalia Evans Toombs Tattnall Bryan Appling Liberty Long Chatham Calhoun Dougherty Early Miller Baker Mitchell Pelham Worth Irwin Tift Coffee Colquitt Berrien Atkinson Cook Bacon Wayne Pierce Ware Brantley McIntosh Glynn Seminole Decatur Grady Thomas Brooks Lanier Lowndes Clinch Charlton Camden Thomasville Valdosta Echols Source: Georgia Department of Education Math and Science Salary Incentives for Teachers 3 In addition, the geographic availability of the coursework still exists as a barrier to K-5 teachers attaining a math or science endorsement. A survey conducted by PSC indicated that better availability of required coursework would improve the likelihood that educators would complete a K-5 math or science endorsement. The number of locations a teacher could attend an endorsement program has increased by three since the original report was published, and there are two regions of the state where no endorsement program is offered. PSC stated that it cannot require an entity to provide the coursework required to earn an endorsement; however, it does encourage Regional Education Service Agencies (RESAs) to offer such coursework. PSC has not conducted any studies to determine if the attainment of a K-5 math or science endorsement results in improved outcomes for K-5 math and science students. PSC stated that it is not currently possible to conduct such an evaluation because the measures used to track student achievement growth by GaDOE have not remained consistent. PSC stated a study can be conducted if and when the GaDOE student achievement growth becomes measurable year-to-year in a consistent, uniform way. Grades 6-12 Salary Incentive As shown in Exhibit 2, 174 of the 180 school systems in Georgia have at least one teacher who received a grades 6-12 salary incentive. However, the number of teachers receiving a grades 6-12 salary incentive has declined since the original examination was published in 2015, from 3,693 to 3,519. PSC did conduct a survey of school system HR personnel to get feedback on the impact of the salary incentives on retention and recruitment. A majority of respondents (58%) stated that the grades 6-12 salary incentive does not improve recruitment, but 51% believed the salary incentive did improve retention of early career teachers. PSC has not increased its communications to colleges and universities regarding the availability of the grades 6-12 salary incentives for math and science teachers. However, staff currently meet with the deans of each public higher education institution on a regular basis and have stated that they will provide information on the availability of the grades 6-12 salary incentive at future meetings. The grades 6-12 salary incentive continues to be paid in a lump sum after the completion of the applicable school year. Therefore, it continues to be unclear what influence the incentive has as a retention tool for math or science teachers. By the time the teacher receives the grades 6-12 salary incentive, they may have already decided to not teach math or science. GaDOE continues to utilize CPI1 data to determine a teacher's years of experience for qualification for the grades 6-12 teacher salary incentive. This data does not track actual years of experience, but rather years of creditable experience. The original examination found there are a number of scenarios in which a teacher's actual years of teaching experience do not match the number of years of experience reflected in CPI data. GaDOE has not corrected these CPI data issues. 1 Classified/Certified Personnel Information database. This GaDOE database collects data points from school systems including teacher count, years of experience, classes taught, and certification level. Math and Science Salary Incentives for Teachers 4 Exhibit 2 As of October 2017, 174 of 180 School Systems Have at Least One Teacher Receiving the Grades 6-12 Salary Incentive Dade Catoosa Da lton Ch ickamauga Whitfield Murray Walker Fannin Gilmer Union Towns Rabun White Habersham Tri on Chattooga Gordon Calhoun Pickens Lumpkin Stephens Floyd Rome Polk Haralson Bremen Hall Banks Franklin Hart Bartow Cherokee Forsyth Gai ne sville Commerce Cartersville Marietta Paulding Cobb Buford Gwinn ett Dekalb Jackson Madison Jefferson Elb ert Barrow Clarke Walton Ocon ee Oglethorpe Doug las Atlanta Decatur City Social Circle Wilkes Carroll Carrolton Fulton Clayto n Fayette Henry Newton Morgan Greene Taliaferro Warren Heard Coweta Spalding Butts Jasper Putnam Hanco ck Lincoln 0 Teachers1 1-5 Teachers 6-10 Teachers 11-50 Teachers 51-100 Teachers 101-450 Teachers Columbia Richmond Troup Meriwether Pike Lamar Mon ro e Jones Baldwin Jefferson Washin gton Upson Harris Talb ot Muscogee Crawford Bibb Taylor Peach Wilkinson Twiggs Du bl in Johnso n Emanuel Burke Jenkins Screven Marion Ma co n Houston Bleckley Laurens Treutlen Candler Bulloch Effingham Schley Stewart Webster Sumter Quitman Clay Randolph Terrell Lee Dooly Pulaski Dodg e Wh eeler Crisp Wilcox Telfair Turner Ben Hill Jeff Davis Vidalia Toombs Evans Tattnall Appling Long Bryan Liberty Chatham Calhoun Doug herty Wo rt h Irwin Tift Coffee Bacon Wayne McIntosh Early Miller Baker Mitchell Pelham Colquitt Berrien Cook Atkinson Pierce Ware Brantley Glynn Seminole Decatur Grady Thomas Brooks Th om asvil le Lanier Lowndes Clinch Valdosta Echols Charlton Camden 1 Additionally, 55 teachers at 16 charter schools, four RESAs, and the Department of Juvenile Justice, received a salary incentive. Source: Georgia Department of Education What we recommend Given the ongoing design and implementation problems identified with the math and science salary incentives, the General Assembly may wish to revisit the continuing need for these incentives. Research shows the benefits of targeted teacher training; however, the majority of school systems do not have teachers participating in these K-5 math and science endorsement programs. If the General Assembly determines there is a continuing need for the K-5 incentive, the endorsement program requires a redesign to ensure the endorsements are effective in recruiting teachers to get additional training that improves their ability to teach math and science. Math and Science Salary Incentives for Teachers 5 If the General Assembly determines there is a continuing need for the grades 6-12 salary incentive, it should be redesigned to ensure the salary incentive effectively recruits grades 6-12 math and science teachers. PSC's Response: PSC agreed with the findings in the follow-up review. PSC noted that any study of student achievement should be conducted by GaDOE. PSC reports that it is currently providing better information to colleges and universities regarding the 6-12 incentive, and that it will continue to work with potential providers to increase the number of programs available to teachers. GaDOE's Response: GaDOE agreed with the findings of the follow-up review. GaDOE stated that conversations have occurred with executive and legislative budget offices regarding the timing of incentive payments, but it has not received permission to alter the schedule or process for disbursement of these payments. As GaDOE does not control the appropriations process, GaDOE believes this finding is applicable to a wider body than just GaDOE. In addition the CPI dataset is designed to calculate the state's Training and Experience program within the QBE system, and as such its structure cannot be fundamentally altered. The following table summarizes the findings and recommendations in our 2015 report and actions taken to address them. A copy of the 2015 special examination report 15-14 may be accessed at http://www.audits.ga.gov/rsaAudits. Math and Science Salary Incentives for Teachers 6 Math and Science Salary Incentives for Teachers Follow-Up Review, May 2018 Original Findings/Recommendations The cost to attain a K-5 math or science endorsement is high compared to the monetary benefit. We recommended that the cost benefit of earning an endorsement be improved by either increasing the financial benefit of the K-5 salary incentive or decreasing the cost of earning an endorsement. Current Status Not Addressed The salary incentive has not increased. The cost of earning an endorsement has not decreased. The salary incentive for K-5 math and science endorsement remains unchanged, at $1,000 per year for each endorsement. No actions have been taken to lower the cost to teachers of earning either endorsement. Local school system human resources personnel have reported that cost range is the largest factor that prevents educators from obtaining a K-5 math or science endorsement. In the original examinations the cost range for required coursework was $576-$4,500. Some school systems provide courses free of charge to their teachers. The geographic availability of K-5 math or science endorsement programs presents a barrier to teachers obtaining endorsements. We recommended increased geographic availability so that teachers in all regions of the state can readily obtain an endorsement. Not Addressed There has been no significant increase in the geographic availability of endorsement programs. Since our review the number of locations offering an endorsement program increased from 21 to 24. Educators reported that increased availability of endorsement programs would improve the likelihood that teachers would attempt a K-5 math or science endorsement. PSC and GaDOE do not evaluate K-5 student math and science outcomes to determine if students with teachers that have that endorsement have improved outcomes. We recommended that the PSC and/or GaDOE evaluate whether the attainment of endorsements by teachers results in improved outcomes for kindergarten and elementary students. Not Addressed There have been no studies conducted to evaluate the impact of K-5 teacher endorsements on student math and science outcomes. PSC stated that it is currently not possible to conduct a longitudinal study to measure student outcomes because there is not a consistent, uniform method to determine student achievement growth over time. The state has changed the basis for student achievement numbers. PSC should provide better information to colleges and universities regarding the availability of the grades 6-12 salary incentive for math and science teachers. We recommended that the PSC provide better information to colleges and universities regarding the availability of the grades 6-12 salary incentive for math and science teachers to help recruit future teachers. Not Addressed PSC stated that they have not provided additional information to colleges and universities regarding the availability of the 6-12 salary incentive. However, staff regularly meets with the deans of the public institutions of higher education and stated that they discuss the incentives with the deans during these regular meetings. The General Assembly should improve certainty of funding. We recommended that the funding be stabilized for both grades K-5 and 6-12 salary incentives. Not Addressed The funding for the salary incentives has not changed. The General Assembly appropriates funds to the GaDOE annually. Math and Science Salary Incentives for Teachers 7 Math and Science Salary Incentives for Teachers Follow-Up Review, May 2018 Original Findings/Recommendations Current Status GaDOE pays teacher salary incentives in one lump sum at the end of the year, rather than throughout the year. We recommended changing the timing of payments to teachers. At a minimum, informing teachers that they will receive a payment prior to making the decision to leave the profession or switch subjects. Consideration could also be given to paying teachers the incentive throughout the year rather than in a lump sum after the end of the applicable year. Not Addressed GaDOE has not addressed the timing of payments to teachers. Also, GaDOE does not inform teachers in advance they will receive math and science salary incentives. PSC has not considered whether limiting the grades 6-12 salary incentive to early career teachers is advantageous. We recommended that PSC consider whether it would be advantageous to expand the salary incentives beyond early career teachers or to other teaching subjects. Partially Addressed PSC has stated that the primary purpose of the salary incentives is to attract new teachers to those subjects, not retain existing math and science teachers. PSC conducted a survey of HR personnel in an effort to determine the impact of the salary incentives on recruitment and retention of teachers, but not specifically on whether it would be advantageous to expand the salary incentives beyond early career teachers or to other teaching subjects. GaDOE uses CPI2 data to determine a teacher's years of experience. However, CPI tracks creditable years of service, not years of teaching experience. We recommended that GaDOE correct CPI data issues to ensure that teachers with greater than five years of actual experience do not receive the grades 6-12 salary incentive. Not Addressed GaDOE has not addressed the CPI data issues. 8 Findings 0 Fully Addressed 1 Partially Addressed 7 Not Addressed 2 Classified/Certified Personnel Information database. This GaDOE database collects data points from school systems including teacher count, years of experience, classes taught, and certification level. The Performance Audit Division was established in 1971 to conduct in-depth reviews of state-funded programs. Our reviews determine if programs are meeting goals and objectives; measure program results and effectiveness; identify alternate methods to meet goals; evaluate efficiency of resource allocation; assess compliance with laws and regulations; and provide credible management information to decision makers. For more information, contact us at (404)656-2180 or visit our website at www.audits.ga.gov.