Follow-Up Review Report No. 15-04

June 2015

Georgia Department of Audits and Accounts
Performance Audit Division
Greg S. Griffin, State Auditor Leslie McGuire, Director

Why we did this review
This follow-up review was conducted to determine the extent to which the Roosevelt Warm Springs Institute for Rehabilitation (the Institute) addressed the recommendations presented in our June 2013 performance audit (Report #12-08).
The 2013 performance audit was conducted at the request of the House Appropriations Committee. The Committee asked that we examine the efficacy of Institute programs and the use of its facilities. As a result, we reviewed 1) whether the Medical Rehabilitation Unit was selfsustaining, 2) the effectiveness of the Vocational Rehabilitation Unit, 3) facility utilization and management, and 4) whether auxiliary programs were self-sustaining.

Follow-Up Review
Roosevelt Warm Springs Institute for
Rehabilitation
Action taken to improve Institute's
financial viability and effectiveness
What we found The Roosevelt Warm Springs Institute for Rehabilitation (the Institute) has taken steps to address all of the findings noted in our original audit. Initial steps have improved the financial stability of its medical operations, expanded evaluation of vocational program effectiveness, and addressed facility issues. The original audit noted the Institute's core activities faced a number of strategic and operational challenges. Due to the scale of these challenges, significant time and resources are necessary to fully implement the recommendations noted in the original audit.

About the Institute
The Institute is a state-operated comprehensive rehabilitation center. The Institute prepares individuals with disabilities for employment in a 24-hour residential setting. Also located at the Institute, the state operates inpatient rehabilitation and long-term acute care hospitals, available to the general public.
The 940-acre campus in Warm Springs, Georgia, also includes historical cottages, a camp, conference center, recreation center, and golf course. The Institute, which receives both state and federal funding, is overseen by the Georgia Vocational Rehabilitation Agency.

Medical Services
In fiscal year 2014, the Institute signed an interagency agreement for Georgia Regents Health (GRHealth)1 to manage the Institute's inpatient hospitals. GRHealth has experience in medical operations, and the transfer was intended to improve the medical operations at the Institute, resulting in increased financial viability. During the period reviewed for the original audit, the majority of the Institute's state appropriations were used to cover losses in the inpatient hospitals and outpatient clinics.
GRHealth is making a number of operational changes to improve the financial viability of the medical operations. For example, GRHealth updated hospital charges and streamlined billing. Additionally, GRHealth has more closely aligned compensation
1 GRHealth is an organization that oversees the medical operations associated with Georgia Regents University, which houses the Medical College of Georgia. Georgia Regents University is part of the University System of Georgia.

270 Washington Street, SW, Suite 1-156

Atlanta, Georgia 30334

Phone: (404) 657-5220

www.audits.ga.gov

Roosevelt Warm Springs Institute for Rehabilitation

2

and benefits with industry standards to improve staff retention. However, the net result of these changes on profitability is still unknown. Consequently, hospital management is unable to determine when the hospitals will no longer require annual state appropriations to cover their losses. As shown in Exhibit 1, the hospitals would need to increase patient discharges by 42% over fiscal year 2015 projections to break even.2 It is important to note that continued operational changes may improve the hospitals' financial position, which would lower the patient utilization break even point. While these operational changes may improve hospital finances, they will not affect location or competition, two key factors identified in the original report as impacting utilization.

Exhibit 1 Increase in Patient Utilization Needed to Break Even Fiscal Years 2009-2016
800

700

600

500

400

300

200

100

0 2009

2010

2011

2012

2013

2014

2015 1 2016 1

Total Discharges

Below Break Even

Break Even2

1Fiscal year 2015 and 2016 figures are projections, as of April 2015. 2The break even point does not take into account operational changes being implemented that have the potential to increase revenues and decrease costs. The net effect of these changes is unknown.

Source: 12-08 report and Roosevelt Warm Springs Rehabilitation & Specialty Hospitals records

The original audit noted that state appropriations were used to subsidize the hospitals, and the subsidy had impacted the state's ability to maximize federal vocational rehabilitation funds. The transfer of the medical operations was intended to improve the hospitals' financial viability, which would decrease the state's subsidization. Institute staff indicated that vocational funding increased as a result of the transfer of the medical operations. Staff estimated that the Institute will spend an additional $900,000 in state funds on vocational rehabilitation in fiscal year 2016, which would generate more than $3 million in additional federal funds. As shown in Exhibit 2, state appropriations for the medical operations fluctuated between fiscal years 2010 and 2015, with a range of $3.4 million to $6.5 million and an average of $4.7 million. From 2014 to 2015, state funding decreased by 47%. A significant reason for the decline was the closure of the outpatient clinics in fiscal year 2013. Our original audit determined that the outpatient clinics had sustained losses of more than $1 million in fiscal years 2010 and 2011 and that state appropriations were used to offset these losses. Additionally, in fiscal year 2014, appropriations were used for expenditures related to the transition of the medical operations to GRHealth. For example, the state paid approximately $600,000 for annual leave payouts for employees transitioning from the Institute to a nonprofit associated

2 This analysis does not take into account the $25.9 million in bond funds authorized for the medical facilities.

Roosevelt Warm Springs Institute for Rehabilitation

3

with GRHealth. The fiscal year 2016 budget provides $2.1 million in state appropriations for medical operations. However, in the past, the medical operations have received additional funding during the amended budget.

Exhibit 2 Decrease in State Appropriations for Medical Operations Fiscal Years 2010-2016
$7,000,000

$6,000,000

$5,000,000

$4,000,000

$3,000,000

$2,000,000

$1,000,000

$0 2010

2011

2012

2013

2014

2015

2016

Source: 12-08 report, Roosevelt Warm Springs Rehabilitation & Specialty Hospitals records, and appropriations bills

In fiscal years 2015 and 2016, the General Assembly authorized a total of $25.9 million in state bond funds to address needed repairs, update facilities to modern standards, and expand patient capacity at the two inpatient hospitals. The estimated completion date is fiscal year 2017. The purpose of the renovation is to improve utilization and help the hospitals to generate sufficient revenue to become profitable. Staff indicated that no additional state bond funding requests are anticipated; the hospitals intend to fund future facility projects from hospital revenue. However, this outcome will depend on the hospitals' ability to generate sufficient revenue to fund needed projects.
Vocational Services
The Institute has made progress in improving the vocational program since the original report. It implemented a new program model to deliver more individualized vocational services to its students. It also initiated new short-term services available to high school students and other eligible clients. To improve overall management, the Institute has expanded the management information it collects in order to better assess the effectiveness of its services.
To evaluate outcomes of students and align services with student needs, the Institute has established a process for collecting information about student outcomes after discharge from the program. In fiscal year 2015, staff began compiling employment information using the Vocational Rehabilitation (VR) program's client management database GRACI and the Department of Labor's unemployment wage database. Staff have instituted a process to regularly survey students who have completed the program to obtain information related to employment and independent living.
To monitor students' progress through the program, the Institute has implemented a new student information system to track student services, schedules, and program progress. The Institute has identified competencies for various services and training courses and created numeric scales to assess students'

Roosevelt Warm Springs Institute for Rehabilitation

4

achievements in course-related competencies. However, the Institute has not yet implemented program completion criteria or a standard method for staff to assess students' readiness to enter the workforce. Staff indicated that a new assessment tool will be implemented beginning in fiscal year 2016 to assess student readiness for employment.
Facilities At the time of the original report, a number of buildings on the Institute's campus exhibited significant disrepair and deterioration. The Institute has undertaken several facility projects to begin to address these issues and to reverse the trend of deferred maintenance. For example, the Education and Training building was renovated to better support the vocational program. After the closure of the outpatient clinics, the Institute repurposed the outpatient building to provide vocational services to its students. However, the Institute has not created a master facility plan that includes future usage and repair costs for all of its buildings. The Georgia Vocational Rehabilitation Agency (GVRA), which oversees the Institute, plans to develop a "master long-term plan" for the campus using an external consultant.
GVRA Response: GVRA generally agreed with our assessment of Institute actions. GVRA noted that the Institute "has worked diligently over the past two years implementing corrective actions to address all previous findings" and is "committed to ensuring that all of the original findings are fully addressed."
University System of Georgia Response: The University System of Georgia noted that additional improvements had been made to hospital operations other than those noted in the report. Additionally, "GRHealth believes that the Medical Services of [Roosevelt Warm Springs Hospitals] are viable and beneficial." With the facility improvements noted in the report, GRHealth believes that the hospitals will be able to increase utilization and improve the level of care to support sustainable operations.
The following table summarizes the findings and recommendations in our 2013 report and actions taken to address them. A copy of the 2013 performance audit report (#12-08) may be accessed at www.audits.ga.gov/rsaAudits/download/15983.

Roosevelt Warm Springs Institute for Rehabilitation

5

Roosevelt Warm Springs Institute for Rehabilitation Follow-Up Review, June 2015

Original Findings/Recommendations
The practice of using state appropriations to subsidize the Institute's Medical Rehabilitation Unit is questionable and has impacted the state's ability to maximize federal vocational rehabilitation funds. We recommended that the General Assembly and GVRA review the long-term viability of the Institute's medical unit. We also recommended an evaluation to determine whether any proposed options to improve the medical unit's financial circumstances would generate the revenue necessary to meet the desired level of profitability on a long-term basis.

Current Status
Partially Addressed In fiscal year 2015, to improve financial viability, management of the medical operations was transferred to GRHealth and an associated nonprofit. Operational improvements have been made; however, it is unclear when the hospitals will become self-sustaining.
In April 2013, the Institute closed its outpatient clinics, which had sustained losses of more than $1 million in fiscal years 2010 and 2011. The Institute then entered into an agreement for GRHealth to manage the inpatient hospitals for fiscal year 2014. In fiscal year 2015, management of the medical operations was transferred to the newly formed nonprofit Roosevelt Warm Springs Rehabilitation & Specialty Hospitals. The nonprofit has taken steps to address the financial viability of the hospitals, with the assistance of GRHealth. For example, staff indicated that the hospitals updated their charges and are renegotiating insurance contracts to increase revenue. Additionally, a new website, under GRHealth, was created to market the hospitals to potential patients.
State appropriations used to subsidize the hospitals decreased by approximately $3 million (47%) in fiscal year 2015 from $6.5 million in fiscal year 2014. However, it is unclear when state appropriations will no longer be needed. The nonprofit and GRHealth have analyzed the number of patients necessary for the hospitals to break even. As shown in Exhibit 1 on page 2, under current financial conditions, the hospitals need to increase patient discharges by 42% over fiscal year 2015 projections. Staff stated that this analysis does not consider operational changes currently being implemented, which should positively impact the hospitals' finances. As a result, there is not sufficient information at this time to determine if or when the hospitals will be able to attract the number of patients necessary to become self-sustaining.
Furthermore, in fiscal years 2015 and 2016, the state committed a total of $25.9 million in bond funds to renovate the hospital facilities. Annual bond payments of approximately $2.4 million will commit state funding for the 20-year bond terms. This funding is not considered in the hospitals' break even analysis.

The Institute does not track the outcomes of its vocational students to determine how well it is meeting program objectives of preparing students for employment and independent living.
To monitor program effectiveness, we recommended that the Institute implement a process to analyze the employment and independent living outcomes of its vocational students after discharge.

Fully Addressed The Institute has established a process to track employment and independent living outcomes of its students after discharge. Staff compile information from three sources: VR's GRACI database, the Department of Labor's unemployment wage database, and post-discharge surveys of students completing the program. The Institute plans to conduct analysis on the information collected to improve program services.

Roosevelt Warm Springs Institute for Rehabilitation

6

Roosevelt Warm Springs Institute for Rehabilitation Follow-Up Review, June 2015

Original Findings/Recommendations

Current Status

The Institute should develop consistent, measureable assessment criteria to determine when students have successfully completed the vocational program and a performance goal for a reasonable level of vocational services after discharge.
To ensure students' preparation for employment is properly assessed prior to discharge, we recommended that the Institute clearly define completion criteria and standardize the method used to determine and document students' readiness to enter the workforce.
We also recommended that the Institute track what VR services are required after discharge and develop a performance goal regarding a reasonable level of services for the student population.

Partially Addressed Within individual program components, the Institute has implemented competencies with quantifiable metrics to measure student achievement. However, the Institute has not implemented a standard assessment to determine when a student has successfully completed the overall program and is ready to enter the workforce. The Institute has not defined completion criteria in written policies or procedures. Staff indicated that a pilot for a new assessment tool was conducted in May 2015, and the Institute plans to implement the new tool in fiscal year 2016.
Additionally, the Institute has not begun tracking students' postdischarge VR services, although it plans to do so. Staff stated that problems with VR's GRACI database have prevented the Institute from accurately tracking these services.

The Institute should evaluate the effectiveness of individual vocational services and admission criteria.
We recommended that the Institute expand its analysis of the vocational program and implement a more comprehensive tracking and reporting system to improve program performance. Specifically, we recommended the Institute track which services students utilize, monitor student improvement and outcomes in each program area, research the labor market to identify occupations with the strongest employment potential, and evaluate admission criteria based on actual student outcomes.

Partially Addressed The Institute has implemented a student information system with the capability to track student services and achievements. Also, staff used labor market research to update its career and technical education (CTE) programs of study. However, the Institute has not yet completed an evaluation of its admission criteria.
In fiscal year 2015, the Institute implemented a new information system that will allow for significantly improved evaluation of program services. Staff can more easily track which services students receive and their achievement in various courses. The Institute has implemented pre- and post- assessments to allow staff to quantify student improvements. The Institute plans to evaluate its services using data from this system in coordination with outcome data discussed in finding two on page 5.
Additionally, the Institute utilized the 2014 Governor's High Demand Career Initiative Report to evaluate its CTE programs. Most of the 26 current or planned programs of study are discussed in this report. The Institute plans to continue monitoring statewide employment trends and change CTE programs as necessary to ensure their relevancy.
However, the Institute has not completed an evaluation of its admission criteria. The Institute recently hired an Admissions Director, who has started the evaluation process. The evaluation will consider student characteristics and outcomes to determine which individuals are most likely to benefit from Institute services. The Institute plans to complete the evaluation process in fiscal year 2016 when historical outcome data is available.

Roosevelt Warm Springs Institute for Rehabilitation

7

Roosevelt Warm Springs Institute for Rehabilitation Follow-Up Review, June 2015

Original Findings/Recommendations
Several buildings on the Institute's campus have deteriorated to the point of requiring extensive renovation, repair, or demolition.
We recommended that the Institute create a master facility plan, including future usage plans and repair costs for its uninhabitable and deteriorating buildings. We also recommended that the Institute determine whether it should demolish some of the 31 historical cottages rather than continue to maintain them all. Finally, we recommended that the Institute consider "moth-balling" buildings not chosen for renovation or repair to prevent further deterioration. This would involve securing the building and its components and providing adequate ventilation.

Current Status
Partially Addressed The Institute has undertaken numerous vocational facility projects using both state bond funding and operational funds. It is using $3.4 million in state bond funds to upgrade fire alarm, sprinkler, and electrical systems in several buildings. Also, the Institute used operational funds for various facility projects, including renovating its Education and Training Building to meet current program needs. However, the Institute has not yet created a master facility plan that includes future plans and costs for all of its buildings.
A number of cottages and the old vocational dorm are currently unusable, and no plans have been made regarding their repair or usage. According to staff, the Institute did consider "mothballing" the unused cottages but decided to direct funding to program initiatives and facility upgrades instead. The Institute has made plans to repair some cottages in the campus core in partnership with several nonprofit groups.
For the medical facilities, GRHealth has assumed responsibility for the inpatient hospital buildings and one cottage. GRHealth has conducted a facility assessment to identify needed projects and related costs. In fiscal years 2015 and 2016, the General Assembly authorized a total of $25.9 million in state bonds to fund the identified facility projects.

The Institute should monitor the profitability of each of its revenuegenerating activities on an on-going basis.
We recommended that the Institute develop complete profit and loss statements for each of its revenue-generating activities to better monitor financial performance and to make necessary changes to improve each activity's financial performance.
To ensure proper handling of program income, we recommended that, if the Institute wishes to enter into business arrangements with the Roosevelt Warm Springs Development Fund, it consult with the Georgia Department of Law to verify the arrangement complies with state laws and regulations.

Partially Addressed The Institute has segregated the finances of most auxiliary programs in order to monitor associated revenues and expenses.
In addition to providing vocational rehabilitation services, the Institute conducts several revenue-generating activities, including a camp, recreation center, conference center, and golf course. The Institute has segregated the finances of the camp and recreation center, and staff indicated that revenues and expenses are reviewed to monitor profitability and set prices. However, expenditures from the conference center's facility rentals are still recorded with other activities, such as staff education. As a result, staff cannot review revenues and expenses for facility rentals separately.
Additionally, the golf course has been closed since March 2011, so no revenue has been generated. The Institute indicated that it has stopped maintaining the course, so no expenditures are incurred. The Institute has signed an agreement with a third party nonprofit to manage the course, beginning in fiscal year 2016. The course will be used for vocational training.
Finally, the Institute indicated that it has not resumed its previous practice of sending program income from the Institute to the Development Fund and has no plans to do so.

6 Findings

1 Fully Addressed 5 Partially Addressed 0 Not Addressed

The Performance Audit Division was established in 1971 to conduct in-depth reviews of state-funded programs. Our reviews determine if programs are meeting goals and objectives; measure program results and effectiveness; identify alternate methods to meet goals; evaluate efficiency of resource allocation; assess compliance with laws and regulations; and provide credible management information to decision-makers. For more information, contact
us at (404) 657-5220 or visit our website at www.audits.ga.gov.