FY 2018 Quality Management Annual Report Table of Contents Quality Management Annual Report FY 2018 TABLE LISTING ........................................................................................................................................................4 FIGURE LISTING......................................................................................................................................................5 SECTION 1: EXECUTIVE SUMMARY ........................................................................................................................7 FY 2018 Provider Performance ..................................................................................................................... 8 FY 2018 Quality Management Activities ...................................................................................................... 9 FY 2018 Key Findings .................................................................................................................................. 11 FY 2019 Areas of Focus............................................................................................................................... 16 In Conclusion .............................................................................................................................................. 18 SECTION 2: INTRODUCTION.................................................................................................................................19 SECTION 3: BEHAVIORAL HEALTH QUALITY REVIEWS..........................................................................................21 BACKGROUND ......................................................................................................................................................21 SAMPLING METHOD .............................................................................................................................................22 Individual Records and Billing Review ........................................................................................................ 22 Individual and Staff Interviews ................................................................................................................... 24 BHQR OVERALL REVIEW SCORES............................................................................................................................25 BHQR Overall Scores................................................................................................................................... 26 BHQR Overall Scores by Tier and Provider Size .......................................................................................... 27 BHQR Billing Validation............................................................................................................................... 30 Figure 6. FY 2018 Percent of Discrepancy Reasons .................................................................................... 35 BHQR Assessment and Planning................................................................................................................. 38 BHQR Service Guidelines ............................................................................................................................ 39 Figure 7. FY 2018 Percent of Dollars Reviewed .......................................................................................... 40 by Non-Intensive Outpatient Service ......................................................................................................... 40 Figure 8. FY 2018 Percent of Dollars Reviewed by Specialty Service.......................................................... 41 BHQR Focused Outcome Areas (FOA) ........................................................................................................ 46 Figure 9. BHQR Focused Outcome Area(s) Scores by Year ......................................................................... 47 Figure 10. BHQR Focused Outcome Area by Year ...................................................................................... 48 Figure 11. BHQR Focused Outcome Area by Year ...................................................................................... 48 Figure 12. BHQR Focused Outcome Area by Year ...................................................................................... 49 Figure 13. BHQR Focused Outcome Area by Year ...................................................................................... 50 Figure 14. BHQR Focused Outcome Area by Year ...................................................................................... 50 Figure 15. BHQR Focused Outcome Area by Year ...................................................................................... 51 BHQR Staff and Individual Interviews......................................................................................................... 51 ASSERTIVE COMMUNITY TREATMENT (ACT) .............................................................................................................55 1|P a g e Quality Management Annual Report FY 2018 ACT Quality Review Overall Scores............................................................................................................. 56 ACT Billing Validation.................................................................................................................................. 57 ACT Assessment and Planning.................................................................................................................... 60 ACT Service Guidelines ............................................................................................................................... 61 ACT Focused Outcome Areas ..................................................................................................................... 63 CRISIS STABILIZATION UNIT QUALITY REVIEWS...........................................................................................................65 Crisis Stabilization Unit Sample Method .................................................................................................... 65 Crisis Stabilization Unit Quality Review ...................................................................................................... 66 CSUQR Individual Record Review (IRR) ...................................................................................................... 67 CSUQR Service Guidelines .......................................................................................................................... 71 CSU Focused Outcome Area ....................................................................................................................... 73 CSUQR Staff Interview and Individual Interview ........................................................................................ 77 REASSESSMENT FREQUENCY REVIEWS ......................................................................................................................80 BHQR Reassessment Review Findings ........................................................................................................ 80 CSU Reassessment Review Findings ........................................................................................................... 84 TECHNICAL ASSISTANCE/EXIT CONFERENCE...............................................................................................................86 SUMMARY OF FINDINGS AND RECOMMENDATIONS FOR BEHAVIORAL HEALTH PROVIDERS.................................................87 Provider Performance ................................................................................................................................ 87 FY 2018 BH Accomplishments .................................................................................................................... 89 BH Systems Strengths and Recommendations for Improvement .............................................................. 91 SECTION 4: INTELLECTUAL AND DEVELOPMENTAL DISABILITIES ........................................................................95 BACKGROUND ......................................................................................................................................................95 SAMPLING METHOD .............................................................................................................................................97 Person-Centered Review (PCR)................................................................................................................... 97 Quality Enhancement Provider Review (QEPR).......................................................................................... 97 REVIEW PROCESSES...............................................................................................................................................98 PCR and QEPR............................................................................................................................................. 98 Quality Technical Assistance Consultation (QTAC) ................................................................................... 100 PERSON CENTERED REVIEW..................................................................................................................................100 PCR Scores by Tool.................................................................................................................................... 100 PCR Scores by Focused Outcome Area (FOA) ........................................................................................... 103 PCR Scores by Tool and Focused Outcome Area (FOA) ............................................................................ 104 FOAs by Demographics............................................................................................................................. 105 Opportunities for Growth by FOA ............................................................................................................ 108 Individual Service Plan Quality Assurance Checklist (ISP QA) .................................................................. 119 ISP Expectations ....................................................................................................................................... 121 Results Using Original ISP QA Checklist .................................................................................................... 121 Results Using New ISP QA Checklist ......................................................................................................... 123 2|P a g e Quality Management Annual Report FY 2018 PCR Results by Service.............................................................................................................................. 124 PCR Strengths and Recommendations ..................................................................................................... 126 QUALITY ENHANCEMENT PROVIDER REVIEW (QEPR) ...............................................................................................127 QEPR Scores by Size.................................................................................................................................. 127 Overall QEPR Score by Tool and Year........................................................................................................ 128 Overall Crisis Provider Scores ................................................................................................................... 129 Qualifications and Training (Q & T) and Service Specific (DDSS) .............................................................. 130 Qualifications & Training Opportunities for Improvement ...................................................................... 131 QEPR Scores by Provider Size ................................................................................................................... 133 QEPR Scores by Tool and FOA................................................................................................................... 135 QEPR Scores by FOA and Provider Size..................................................................................................... 137 QUALITY AND TECHNICAL ASSISTANCE CONSULTATION (QTAC)...................................................................................138 INTELLECTUAL AND DEVELOPMENTAL DISABILITY SUMMARY OF FINDINGS AND RECOMMENDATIONS ................................139 FY 2018 IDD Accomplishments................................................................................................................. 140 IDD System Strengths and Recommendations ......................................................................................... 141 SECTION 5: BEHAVIORAL HEALTH AND INTELLECTUAL DEVELOPMENTAL DISABILITIES: OVERALL RESULTS BY FOCUSED OUTCOME AREA ................................................................................................................................149 SECTION 6: FEEDBACK SURVEY RESULTS ...........................................................................................................151 SECTION 7: CONCLUSION...................................................................................................................................154 APPENDIX A: ABBREVIATIONS AND ACRONYMS ...............................................................................................157 APPENDIX B: SCORE DISTRIBUTIONS .................................................................................................................159 3|P a g e Table Listing (Links available by ctrl + clicking on table name) Quality Management Annual Report FY 2018 Reviews Completed by Fiscal Year Number of Completed Reviews by Type FY 2018 Overview of BHQR and CSUQR Review Data BHQR and ACT Results by Category by Year BHQR and CSUQR Results by Category by Year FY 2018 Overview of IDD Review Data FY 2018 Summary by Tool and Review Type Table 1. BHQR Overall Scores by Tier Table 2. BHQR Category Scores by Provider Size Table 3. Amount Reviewed by Funding Source Table 4. FY 2018 Non-Intensive Outpatient Services Percent of Funds Justified Table 5. FY 2018 Specialty Services Percent of Funds Justified Table 6. BHQR Billing Discrepancy Reasons Table 7. BHQR Assessment & Planning Indicator Scores by Year Table 8. BHQR Service Guidelines Scores by Service Type Table 9. FY 2018 Improvement Results by Service Table 10. FY 2018 Service Type Declined Results Table 11. FY 2018 Interview Comparison Table 12. BHQR and ACT Results by Category by Year Table 13. ACT BHQR Billing Discrepancy Reasons by Year Table 14. ACT Service Guidelines Scores Table 15. Technical Assistance/Exit Conference Details for FY 2018 Table 16. BHQR, ACT, CSUQR Overall Averages by Year Table 17. BHQR and CSUQR FOA Scores Table 18. Reassessment Review Results Table 19. PCR Sample by Region Table 20. QEPR Provider Sample by Size Table 21. Number of Records by Review Tool and Review Type Table 22. Low Scoring Whole Health Indicators (PCR) Table 23. Low Scoring Safety Indicators (PCR) Table 24. Low Scoring Person Centered Practices Indicators (PCR) Table 25. Low Scoring Community Life Interview Indicators (PCR) Table 26. Low Scoring Community Life Record Review Indicators (PCR) Table 27. Low Scoring Choice Indicators (PCR) Table 28. Low Scoring Rights Indicators (PCR) 4|P a g e Quality Management Annual Report FY 2018 Table 29. ISP QA Life Indicator by Region and Year Table 30. ISP QA Checklist Ratings by Expectation (FY 2018) Table 31. Average Rating by Expectation (0 - 4) Table 32. PCR PRR Results by Service and Year Table 33. FY 2018 PCR Record Review Results by FOA and Service Table 34. Top Strengths Identified During a PCR Table 35. Top Recommendations Identified During a PCR Table 36. QTACs by Referral Source and Type Table 37. QTAC Referral Reasons: FY 2018 Table 38. FY 2017 Summary by Tool and Review Type Table 39. The Collaborative Provider Feedback Surveys Figure Listing (Links available by ctrl + clicking on figure name) Figure 1. FY 2018 Diagnostic Categories Reviewed Figure 2. BHQR Fiscal Year Results by Category Figure 3. Fiscal Year 2018 Results by Provider Size Figure 4. BHQR Billing Validation Amount Reviewed by Fiscal Year Figure 5. BHQR Amount Reviewed by Funding Source and Fiscal Year Figure 6. FY 2018 Percent of Discrepancy Reasons Figure 7. FY 2018 Percent of Dollars Reviewed by Non-Intensive Outpatient Service Figure 8. FY 2018 Percent of Dollars Reviewed by Specialty Service Figure 9. BHQR Focused Outcome Areas Scores by Year Figure 10. BHQR Focused Outcome Area by Year Whole Health Figure 11. BHQR Focused Outcome Area by Year Safety Figure 12. BHQR Focused Outcome Area by Year Person Centered Practices Figure 13. BHQR Focused Outcome Area by Year Community Life Figure 14. BHQR Focused Outcome Area by Year - Choice Figure 15. BHQR Focused Outcome Area by Year - Rights Figure 16. BHQR Individual Interview Data by Fiscal Year Figure 17. BHQR Staff Interview Data by Fiscal Year Figure 18. ACT Fiscal Year Results by Category Figure 19. ACT Billing Validation by Fiscal Year Figure 20. ACT Assessment and Treatment Planning Scores by Year Figure 21. ACT FOA Scores by Category by Year Figure 22. CSU Overall Scores by Fiscal Year Figure 23. Individual Record Review Category Scores Figure 24. CSU Service Guidelines Staffing and Curriculum Results by Indicator Figure 25. CSUQR Service Guidelines Policy Adherence Indicators by Fiscal Year 5|P a g e Quality Management Annual Report FY 2018 Figure 26. CSUQR Service Guidelines Medication Policy Adherence Results by Year Figure 27. CSUQR FOA Results by Year Figure 28. CSU Individual and Staff Interview Results by Fiscal Year Figure 29. CSU Individual Interview Subcategory Data by Year Figure 30. CSU Staff Interview Subcategory Data by Year Figure 31. BHQR FY 2018 Overall Score Distribution of Reassessed Providers Figure 32. BHQR FY 2018 Billing Score Distribution of Reassessed Providers Figure 33. CSU #1 Category Score by Reassessment Figure 34. CSU #2 Category Score by Reassessment Figure 35. PCR Scores by Tool and Year Figure 36. PCR Scores by FOA and Year Figure 37. FY 2018 PCR Scores by Tool and FOA Figure 38. FY 2018 PCR Scores by FOA and Region Figure 39. FY 2018 PCR Scores by FOA and Residential Setting Figure 40. FY 2018 PCR Scores by FOA and Funding Source Figure 41. FY 2018 PCR Scores by FOA Whole Health Figure 42. FY 2018 PCR Scores by FOA - Safety Figure 43. FY 2018 PCR Scores by FOA - Person Centered Practices Figure 44. FY 2018 PCR Scores by FOA - Community Life Figure 45. FY 2018 PCR Scores by FOA - Choice Figure 46. FY 2018 PCR Scores by FOA - Rights Figure 47. PCR ISP QA Life Indicator by Year Figure 48. "New" ISP QA Checklist Ratings Figure 49. FY 2016 - FY 2018 QEPR Samples by Provider Size and Type Figure 50. Overall QEPR Scores by Tool and Year Figure 51. Overall Crisis Provider Scores by Tool and Year Figure 52. QEPR Qualifications and Training and DDSS Results by Year Figure 53. Overall and Qualifications and Training Score by Provider Size and Review Year Figure 54. Overall QEPR Scores by FOA and Year Figure 55. FY 2018 QEPR Scores by Tool and Focus Outcome Area Figure 56. FY 2018 QEPR Provider Scores by FOA and Size Figure 57. FY 2018 QTAC Technical Assistance Provided Figure 58. Focused Outcome Areas by Review Type 6|P a g e Section 1: Executive Summary Quality Management Annual Report FY 2018 Since 2016, the Georgia Collaborative Administrative Services Organization (the Collaborative) has assisted the Georgia Department of Behavioral Health & Developmental Disabilities (DBHDD) in its management of services and supports for individuals receiving Community Behavioral Health and Rehabilitation Services (CBHRS), New Options Waiver (NOW), Comprehensive Supports Waiver (COMP), and state funded behavioral health (BH) and intellectual and developmental disabilities (IDD) services. DBHDD delegates BH, Crisis Stabilization Unit (CSU), and IDD quality reviews to the Collaborative. On June 30, 2018, the Collaborative's Quality Management Department completed the third year of the contract with DBHDD. See the table below for the number of BH and IDD quality reviews and interviews conducted. Reviews Completed by Fiscal Year Fiscal Year Number of Reviews 2016 2017 BH/CSU* 177 208 IDD** 584 581 2018 197 584 Total 582 1,749 Individuals Interviewed Staff Interviewed BH/CSU 643 735 IDD 1,151 921 BH/CSU 657 774 IDD 1,186 702 616 920 652 759 1,994 2,992 2,083 2,647 *The additional subset of Assertive Community Treatment (ACT) Quality Reviews are included in the BH/CSU totals (FY16: 17; FY17: 18; FY18: 20). Calculations specific to the ACT Quality Reviews are in a separate section of the report. ** Crisis providers are included in the total number of reviews but are not included in the figures/graphs below (FY16: 1; FY17: 4; FY18: 1). Separate calculations of Quality Enhancement Provider Reviews (QEPR) and Person Centered Reviews (PCR) are included below. A separate QEPR calculation for crisis providers can be found throughout the report. 7|P a g e FY 2018 Provider Performance Behavioral Health Quality Reviews Quality Management Annual Report FY 2018 Crisis Stabilization Unit Quality Reviews IDD: Quality Enhancement Provider Reviews1 1 The QEPR sample is designed to ensure all providers are reviewed at least once every three to four years. Comparisons across years are not appropriate. 8|P a g e Quality Management Annual Report FY 2018 IDD: Person Centered Reviews by Focused Outcome Area and Year 100% 75% 93.6% 87.4% 84.7% 96.5% 91.6% 89.1% 84.8% 90.3% 82.6% 80.6% 76.9% 71.1% 90.7% 82.3% 95.4% 92.0% 90.2% 78.7% 50% 25% 0% Whole Health Safety Person Centered Practices Community Life Choice Rights FY 2016 (N = 484) FY 2017 (N = 481) FY 2018 (N = 484) FY 2018 Quality Management Activities Based on FY 2017 analysis and dialogue with DBHDD, the following were the areas of focus for FY 2018: Behavioral Health To improve provider billing practices and the associated supporting documentation, the Billing Validation threshold for the Behavioral Health Quality Reviews (BHQR) was increased by DBHDD to 80 percent. The billing scores' average increased to 85 percent; reflecting improved provider performance and a decrease of funds at risk for recoupment. Behavioral health quality assessors continued to identify billing practices that put providers and the larger system at risk. Quality Management made recommendations for increased oversight of providers with uncorrected or ongoing billing and documentation practices. 9|P a g e Quality Management Annual Report FY 2018 A new quality of care process was piloted to collect, monitor, and report various issues to DBHDD specific to providers' physical environments, staffing, and quality of services which are outside of the scope of the quality reviews. To standardize and mirror the sampling processes with IDD reviews, Quality Management grouped BH providers into three categories (Small, Medium, Large) based on the number of unique individuals served in the six months preceding a review. Subsequent revisions to BHQR sample sizes reflect provider groupings. There was implementation of an oversampling process for both BHQRs and Crisis Stabilization Unit Quality Reviews (CSUQR) to ensure appropriate sample sizes. To increase engagement of individuals with complex care needs and reduce readmission rates, Quality Management educated Crisis Stabilization Unit providers about the benefits of the Collaborative's Care Coordination team. On July 1, 2018, CSUQR process and tools were incorporated into the electronic system (Disabilities Quality Management System) to enhance accuracy, standardization, and formality through electronic reporting. Quality Management revised and updated the quality section on the Collaborative's website to increase ease of use and access to resources available. Intellectual and Developmental Disabilities The Collaborative's continuous improvement efforts for the IDD system focused on three key areas: training for stakeholders, the new IDD Case Management system, and tool revisions for the review processes. Following are the initiatives for each of these areas. Throughout the year, the Collaborative's Quality Management team and DBHDD met to discuss and review findings from FY 2017, current fiscal year findings, and other identified areas of need across the provider network. Training sessions on Focused Outcome Areas were developed and presented throughout the year based on this analysis and discussion. The following is a list of trainings provided for stakeholders in the 2018 fiscal year: 10 | P a g e Quality Management Annual Report FY 2018 o Whole Health: Empowering Individuals to Maintain their Healthcare; Techniques on How to Help Educate People on Medications o Person Centered: Person Centered Documentation Parts 1 and 2; What is Person Centered Language Part 1 and 2; Staff Matching, Why It Works o Community: How to Support Community Inclusion o Rights and Choice: The importance of supporting concepts of Choice and Rights from the perspectives of people receiving services and providers In FY 2018, a training plan was developed for the new IDD Case Management System project. The plan was developed for internal (the Collaborative and designated DBHDD staff) and external (individuals applying for services, providers of IDD services, and Support Coordination staff) users and other stakeholders to provide guidance on navigating and using the integrated case management system. Through workgroups with the DBHDD Quality Improvement Office, IDD program staff and other stakeholders, implementation of tool revisions occurred over the course of FY 2018 to align with the DBHDD IDD Provider Manual, policies and procedures. The following tools were revised in FY 2018: Individual Interview, Observation, Staff Interview, Provider Record Review, ISP QA Checklist, and Support Coordination Interview. FY 2018 Key Findings Based on data collected from the past three years, the following have been identified as strengths and areas for improvement: Behavioral Health Strengths The BHQR Service Guidelines was the highest scoring category of the four in FY 2016, with an average of 90 percent. Although it declined slightly in FY 2017 by two points, it has since returned to a 90 percent result. Moreover, nearly three quarters of the Service Guidelines scores within the BHQR met or exceeded 90 percent and nine reviews received a perfect score; three of which were providers reviewed for the first time. 11 | P a g e Quality Management Annual Report FY 2018 The BHQR Assessment and Planning, a category comprised of 10 indicators, had significant increases from the previous year. Documentation reviewed during the BHQR clearly demonstrated evidence that the "Individual had been provided supports and service options" (98%) as well as having their "Known preferences followed to the extent possible by the service delivery provider" (98%). Through ongoing technical assistance and assessment of practices that support whole-person treatment, reviewed documentation indicated providers incorporated techniques to address the whole health and wellness of individuals more frequently in FY 2018. Significant improvement in the indicators within Assessment and Planning and the Focused Outcome Area related to Whole Health demonstrated increased awareness, assessment, and service planning by providers with individuals. All child and adolescent (C&A) CSU records demonstrated that the individual was included in the modification of their plans, providers honored youths' preferences and differences, and providers offered individuals options of supports and services. When comparing scores to FY 2017, there were specific areas that show statistically significant improvement (p < .01). These included the following: o FY 2018 demonstrated even higher levels of documentation specific to the BHQR Choice FOA subcategory than in FY 2017. While results of all indicators exceeded 90 percent in FY 2017 for the FOA Choice subcategory, increases in documentation of the providers supplying alternatives when barriers had been identified rose from 92 percent to 95 percent. o Specific to the CSUQR Individual Record Review (IRR) category, all areas demonstrated significant improvement in scoring from the previous year with the exception of Course of Stay and Treatment Planning. Furthermore, in analysis of C&A populations versus Adult populations, several FOA indicators specifically reviewed for C&A individuals met or exceeded 90 percent, some specifically reaching 100 percent. o In Crisis Stabilization Unit records, the score for "Discharge summary/note included the necessary criteria" improved from the previous fiscal year by 30 points (92%). 12 | P a g e Quality Management Annual Report FY 2018 Intellectual and Developmental Disabilities Strengths Most providers and staff were aware of unique safety needs and achievements of individuals they serve (average Safety FOA of 95.6%). Most individuals were aware of how to selfpreserve (88.6%) and felt safe in their work and living environments (99.6%). Most direct service providers have a clear understanding of the unique communication styles and skills for each individual they serve (94.7%). Based on observations, staff and providers took advantage of opportunities to provide education on health and rights, and with one exception, all individuals expressed that they were free from abuse, neglect, and exploitation. When comparing scores to FY 2017, there were specific areas that showed statistically significant improvement regarding documentation practices by providers. These included the following: o Providers showed improvement in ensuring documentation for age and genderspecific preventative healthcare: Prostate-Specific Antigen tests increased 25.3 points to 56.6 percent met. Bone density tests increased 24.3 points to 60.7 percent met. Mammogram screening increased 11.2 points to 51.0 percent met. o Providers showed improvement in ensuring documentation for specialty healthcare reports, as applicable: Neurological evaluation/supports increased 15.6 points to 71.7 percent met. Gastrointestinal evaluation/supports increased 13.2 points to 86.2 percent met. Psychiatric evaluation/supports increased 10.3 points to 78.8 percent met. o Significant improvement was found in documentation on how staff provided education to individuals and families (if approved by the individual) on all prescribed medications (increase of 22.6 percentage points) and to the individual on risks and benefits of medication (increase of 18.2 percentage points). 13 | P a g e Quality Management Annual Report FY 2018 o Documentation showed providers were much more likely to ensure critical incident reports were completed and follow-up had occurred at an increase of 18.2 percentage points. o Based on information in the medical record, providers were more likely to document evidence of identifying, addressing, and seeking prevention of abuse, neglect, and exploitation; addressing rights education; and that individual rights and responsibilities were presented in an understandable language/format than in FY 2017. Behavioral Health Assessed Areas of Need Although there was an increase in the inclusion of medication consent forms in medical records for FY 2018, this FOA Safety indicator remained one of the lowest scores for both BHQRs (78%) and CSUQRs (82%). Providers did not address all identified/assessed areas of need in the individualized recovery or resiliency plan (IRP) specific to BHQR (67%). Co-occurring health conditions were addressed within the IRPs reviewed in both the BHQR (60%) and CSUQR (54%). This is a slight increase from FY 2017. Although Crisis Stabilization Units (CSU) demonstrated improvement in their compliance with medication storage and safe handling practices, this continued to be the lowest-scored indicator within CSUQR Compliance with Service Guidelines with a score of 67 percent met. Discharge planning with individuals served continued to be a challenging area for providers. At 66 percent, it was the second-lowest scored indicator in the area of the BHQR Assessment & Planning; particularly, in the required area that staff assist individuals in identifying clinical benchmarks that indicate progress in their treatment and recovery. Because it is important for individuals entering services to have the expectation of improvement, discharge planning is a critical and useful clinical process and can aid in advancing the expectation of recovery. Quality Management will continue to provide technical assistance on discharge planning during quality reviews. 14 | P a g e Quality Management Annual Report FY 2018 Performance on ACT Assessment and Planning indicators improved in FY 2018; however, some scores remained low to include the indicators: o "IRP addressed co-occurring health conditions" (70%) o "Assessed needs were addressed on the IRP" (68%) o "Discharge planning criteria were specified" (66%) While the Service Guidelines score improved from the previous fiscal year in the BHQR, documentation of coordination and collaboration with either family/significant others (59%) or additional healthcare providers (70%) remained an area of need. Intellectual and Developmental Disabilities Assessed Areas of Need Provider documentation indicated most providers of IDD services did not offer education on prescribed medications, including the risks and side effects, for individuals receiving services. Over the previous three years, individuals receiving services appeared to be less likely to know what medications they are taking, why they are taking them, what the side effects may be, or to have an awareness of the color and shape of medication. The following crisis and safety issues were identified: o Approximately 30 percent of individuals interviewed did not know how to respond to an emergency or crisis situation if supports/providers were incapacitated. o Forty percent of staff did not provide education to the person on the use of the Crisis Hotline. o Close to 58 percent of employee records reviewed this year (43.4% in FY 2017) indicated staff/provider training on the Georgia Crisis Response System had not been completed as required. o Based on record reviews, in F8 2018 providers were more likely than in FY 2017 to document evidence of "Identifying, addressing, and seeking prevention of abuse, neglect, and exploitation; addressing rights education" (up 16.6 points to 39.5%); and to have "Individual rights and responsibilities written in an understandable language/format" (up 11.5 points to 87.3%). 15 | P a g e Quality Management Annual Report FY 2018 o Support Coordinators were often unaware of needed crisis plans (47.7%), behavior support plans (57.2%), or the individualized techniques needed to follow a plan for the individual (53.3%). Support Coordinators' awareness of the triggers that may be the catalyst to a crisis scored below seventy percent: relapse (60.6%), decomposition (57.3%), or behavioral issues (69.7%). Community Life remains the lowest scoring area in the PCR and QEPR: o Twenty-five to 30 percent of individuals receiving services indicated they had not been given the opportunity to learn about and develop new social roles in the community or experience new community activities. o Provider and Support Coordinator documentation often did not show support for individuals to be engaged in the community, develop social roles, explore new activities, or participate in activities the same as other citizens. FY 2019 Areas of Focus Behavioral Health As the lowest scoring indicator of CSUQR Service Guidelines (67%), the safe handling and storage of medications remains an area of focus for improvement. Quality Management will conduct a statewide provider training on the development of and adherence to policies/procedures related to the safe storage and handling of medications in accordance with state requirements in FY 2019. Due to increased national and local suicide rates, Quality Management will continue a focus on safety planning and risk reduction at the provider level through the following efforts: o Continue ongoing partnership with the Collaborative's Care Coordination department to provide training and education to providers on safety and crisis planning. o In conjunction with DBHDD, maintain ongoing tracking, trending, analysis, and identification of areas for improvement, with continued suggestions of appropriate interventions that address the needs of and support for individuals identified with suicide risk when served at crisis stabilization units. 16 | P a g e Quality Management Annual Report FY 2018 The nationally recognized opioid epidemic continues to claim thousands of lives; therefore, the Quality Management department will continue to support DBHDD in addressing this disease by participating in training and development of inclusion of Medication Assisted Treatment (MAT) standards, conducting analysis on findings, and reporting to DBHDD with recommendations. Quality Management will continue to conduct sample size analysis to ensure records sampled are representative of the provider and statewide utilization levels for BHQR. Additional inclusion of "Immediate Action Items/Recommendations" at exit conferences will occur to offer actions that providers should take for immediate quality enhancement, risk reduction and for those items that will be submitted to the Collaborative's Compliance Department for further investigation. Lastly, to further enhance the review of billing practices and assure all providers are billing from approved Medicaid sites, the Collaborative, at the Direction of DBHDD, Quality Management will propose a pilot project to establish baseline data on billing locations for both BH and IDD providers. Intellectual and Developmental Disabilities Quality Management conducted training in June of 2018 to help providers and Support Coordinators learn how to provide education to individuals on regarding medications and promote independence towards self-directing health. The Collaborative recommends DBHDD consider revising policies to provide more emphasis on delivering education in these areas to individuals and families. The Collaborative, in conjunction with DBHDD, will track indicators related to medication education and self-directing health to determine if additional training would be beneficial. Quality Management will collaborate with DBHDD to develop specific training for Support Coordination as it relates to behavior supports and how to evaluate this service. DBHDD may want to consider requiring providers to attend annual training on developing community connections for individuals receiving services. Training and related resources could include practical application of ideas and examples to help staff and Support 17 | P a g e Quality Management Annual Report FY 2018 Coordinators understand different and realistic ways to connect an individual to the community. DBHDD may want to consider conducting a targeted safety campaign that includes resources, and web-based and face-to-face training throughout the state. The DBHDD monthly newsletter could include a new tip or resource that providers, individuals, and families can use to promote independence in the area of safety; for example, including web resources such as Classroom, which provides education about stranger awareness. In Conclusion During FY 2018, the continued collaboration between DBHDD and Quality Management resulted in increased quality related to justification of billing (BHQR), service provision (BHQR/CSUQR), and education on health and rights (QEPR); however, both IDD and BH quality reviews indicate areas of focus such as safety/crisis planning and medication education. It has been a goal for DBHDD and the Collaborative to integrate quality measurement processes for both behavioral health and intellectual and developmental disability services. Side-by-side comparisons are difficult to evaluate due to the use of different types of sampling methods for each discipline, as well as differences in the tools and standards used. During FY 2019, Quality Management, in conjunction with the DBHDD Office of Quality and divisional subject matter experts, will review the behavioral health and intellectual and developmental disability record review tools to identify modifications and enhance the ability to make these types of comparisons. The above analysis and recommendations are discussed in more detail throughout this report. A summary of review processes and sampling procedures are included. The report provides an indepth analysis of data from the behavioral health, intellectual and developmental disability, and CSU quality reviews; as well as tabular and graphic displays of findings. 18 | P a g e Section 2: Introduction Quality Management Annual Report FY 2018 The Georgia Collaborative Administrative Services Organization (the Collaborative), contracted by Georgia's Department of Behavioral Health and Developmental Disabilities (DBHDD), employed 30 quality assessors to complete more than 2,300 quality reviews and over 9,600 individual and staff interviews in the past three fiscal years across Georgia. The mission of Quality Management within the Collaborative is to provide DBHDD with valuable and beneficial data for use in decision-making regarding quality of services, funding, and development of programs (among other quality improvement initiatives). This annual report is the result of assessing, gathering, compiling, analyzing, and measuring the quality of the service-delivery system through assessment of and technical assistance provided to DBHDD's behavioral health and intellectual and developmental disability providers. Furthermore, this report contains suggestions for modifications in some processes, methods, approaches, and tools to measure the quality and impact of services. Quality Management recognizes that quality assessment or measurement processes must have a quality review periodically to examine and determine if we are measuring the things most important and meaningful to individuals receiving services within Georgia. This third annual report includes behavioral health, intellectual and developmental disability, and crisis stabilization unit (CSU) findings as well as a detailed explanation of the review processes, analysis, and comparisons of network performance across providers and information from individuals about the services received. A difference in proportions test was used to determine statistical significance and, where applicable, determine change from year to year.2 Areas highlighted in blue within tables reflect significance at a p < .01 or have the p-value defined. While there are similarities in the review processes, there are also distinct differences; therefore, behavioral health, intellectual and developmental disability, and CSU results are reported separately. The report is divided into the following sections: 2 Bohrnstedt, George W. & Knoke, David. (1988). Statistics for Social Data Analysis, 2nd Edition. Itasca, Illinois. F.E. Peacock Publishers, Inc., pgs. 198-200. 19 | P a g e Quality Management Annual Report FY 2018 Behavioral Health Quality Reviews o Assertive Community Treatment o Crisis Stabilization Unit Quality Reviews Intellectual and Developmental Disabilities o Person Centered Reviews o Quality Enhancement Provider Reviews o Quality Technical Assistance Consultations Behavioral Health, and Intellectual and Developmental Disability Discussion Provider and Individual Feedback Surveys Two appendices are attached o Acronym and abbreviation list, Appendix A o Distribution graphs for all tools used in the BHQR, CSUQR, and intellectual and developmental disability reviews, Appendix B Summaries of findings and recommendations after each service delivery system section to address areas needing improvement or training/education programs to help improve the quality of services provided to Georgians 20 | P a g e Quality Management Annual Report FY 2018 Section 3: Behavioral Health Quality Reviews Background The purpose of the Behavioral Health Quality Review (BHQR) is to determine providers' adherence to DBHDD's standards and to assess the quality of the service delivery system through individual record and claims reviews.3 Review indicators are based on DBHDD and Medicaid requirements, and are organized into four categories: Billing Validation, Assessment and Planning, Service Guidelines, and Focused Outcome Areas (FOA). The score for each category represents the percent of relevant indicators met or present. The BHQR Overall score is calculated by averaging the scores for the four categories.4 Each category accounts for 25 percent of the overall rating. In collaboration with DBHDD, minor alterations were made before and throughout the FY 2018 review process to adhere to DBHDD and Medicaid requirement changes, as well as language changes specific to tools, providing more clarity for assessors and providers. Consistent with previous year reviews; when a BHQR is completed, the lead assessor convenes the team to analyze data gathered and plan the exit conference. The exit conference provides immediate, preliminary feedback of the BHQR findings to the provider. A report of these preliminary findings is left with the provider in the form of an exit conference report outlining the provider's identified strengths and any opportunities for improvement from the four primary categories of the review. Assessors also include any items falling outside the parameters of the review determined to be an area of concern or risk. Technical assistance is also provided during the review and exit conference. Within 30 days of completion of a BHQR, a final assessment report is posted on the Collaborative's website and the provider is notified via email of the posting, along with the final scores identified as the final assessment. Like the exit conference report, the final assessment identifies strengths and 3 Please refer to the following link to access a full description of the review process and review tools. The Georgia Collaborative - Quality Management BH 4 The FOA subcategories are individually scored and are not averaged for the final overall FOA result at the review level. The final overall FOA result is calculated by adding all "yes" or "present" responses of all FOA subcategories and dividing by the total "yes" or "no" responses of the combined subcategories for each review. 21 | P a g e Quality Management Annual Report FY 2018 opportunities for growth in the four categories of FOA, Billing, Service Guidelines, and Assessment and Planning. The final assessment also highlights areas of concern falling outside the scope of scoring. The final assessment also includes recommendations for improvement and comparisons to a statewide average based on the previous year's results. Providers are offered an opportunity to appeal the BHQR findings. Appeal information, including timeframes for submission, is provided upon notification of the final report and is made available to providers by visiting the Collaborative's website or reviewing the Georgia Collaborative Provider Handbook. Sampling Method Individual Records and Billing Review During the 2018 fiscal year, 136 providers were eligible for review, as determined by DBHDD, and received a BHQR. This is a reduction from the implementation year (FY 2016) of 141 providers and an increase of 4 providers from FY 2017. Sixteen providers (12%) were reviewed for the first time in FY 2018. Provider eligibility is based on multiple factors including volume of individuals served, claim volume, and type of services provided. Additionally, while efforts to review all providers within the BH network are made, some providers may be classified as ineligible due to site closure, changes to location or site, or at the direction of DBHDD. A sample of individuals was selected for each of the record reviews, and a sample of those individuals' claims was used for the billing review. To be eligible for the sample, each individual selected must have had at least three claims/encounters in the three months (or longer, if necessary) preceding the BHQR. Providers deemed ineligible because of minimal claim availability or individuals served (< 5) within the timeframe are reviewed periodically for reevaluation of eligibility. To select the individuals as part of the record review, an unduplicated list of individuals receiving services from the provider was stratified by service and payer source: Medicaid, state-contract, and fee-for-service paid claims. The sample was selected proportionate to the providers' ratio of individuals served by payer source, ensuring each service provided and approved for review by DBHDD was represented. The sample for each provider consisted of up to 30 records for non- 22 | P a g e Quality Management Annual Report FY 2018 intensive and specialty service providers, with an additional 15 individuals for providers who also offered assertive community treatment (ACT). The number of individuals selected per provider was based on the number of individuals served during the six months prior to review. Small Providers (serving 5-50 individuals) = 5-10 records selected Medium Providers (serving 51-100 individuals) = 20 records Large Providers (serving 101+ individuals) = 30 records Crisis Stabilization Unit Quality Reviews = 15 records Assertive Community Treatment Providers = 15 additional records Providers serving less than five individuals are monitored and added to the review process once at least 5 individuals have been served; however, these providers can be selected for an ad hoc review at the direction of DBHDD. Oversampling of individuals at each of the provider sites began in the second quarter of FY 2018 to provide for full sample completion respective to the number of records reviewed. The oversample includes additional, unique individuals following the above outlined sampling method. Providers receive the listing of individuals (including the oversample) at the start of the review process. Assessors ensure that records included in the sample adhere to the guidelines outlined above. Oversampled individuals are reviewed only at the time the original sample individuals are found to be ineligible for review based on criteria mentioned previously. Individuals listed on the oversample but not reviewed are deleted from the review to not skew BHQR outcomes. During FY 2018, 3,070 individuals were sampled for record reviews, averaging approximately 20 records per provider. This number of records is nearly 800 fewer than the previous year and averages approximately nine fewer records per provider. This is most likely due to the changes in sampling methods specific to provider size, as well as decline in overall reviews conducted. Less overall reviews may have been conducted because of provider eligibility requirements, improved provider scores compared to previous reviews, and/or at the direction of DBHDD. Furthermore, declines in reassessments also occurred from 35 in FY 2017 to 20 in FY 2018 due to increased scores on previous reviews; thereby reducing the number of records reviewed. See Reassessment Frequency Review section for more information. FY 2018 data also provided an additional level of detail specific to type of diagnosis reviewed for each individual, where 86 percent of records reviewed had a confirmed mental health diagnosis. 23 | P a g e Quality Management Annual Report FY 2018 Although individuals may have more than one diagnosis, the following information shown in Figure 1, was made available demonstrating co-occurring reviews with IDD, physical diagnoses, as well as substance use diagnoses. Figure 1. FY 2018 Diagnostic Categories Reviewed IDD Substance Use 7% 30% Mental Health 86% For each record, a random sample of up to 10 paid claims was selected for a billing review for both BHQR and ACT reviews. The number of claims reviewed per individual was based on services provided and claim submissions. The total number of claims reviewed for FY 2018 was 25,598 compared to 29,602 in FY 2017. The number of records reviewed directly affects the total number of claims reviewed. Thus, as there was a decrease in the number of records reviewed, there was also a reduction in number of claims reviewed. When providers did not have adequate claim submissions in the three months preceding their review claims selection timeframe was extended but did not precede the provider's previous BHQR timeframe (when applicable). BHQR Billing Validation and claim(s) reviews focused on specific services. The services included are listed in Table 4 and Table 5. All eligible providers were reviewed at least once during the fiscal year and had at least one claim per billed service included in the claims review, ensuring the complete array of services provided were included in the BHQR. Individual and Staff Interviews Samples used for the Individual and Staff Interviews were selected by the provider and quality assessors conducting the BHQR; services received or provided were not considered in the selection of interviewees. Interview sampling methods remained the same for FY 2018 with no changes since inception. Quality assessors attempted to complete a minimum of five Individual and five Staff 24 | P a g e Quality Management Annual Report FY 2018 Interviews per BHQR; however, the actual number interviewed fluctuated based on individual and staff availability, their agreement to participate in the interview process, the number of employees, and the number of individuals the provider served at the time of the review. If an individual or staff declined an interview, assessors selected an additional individual or staff to be interviewed, when possible. In FY 2018, 616 Individual Interviews and 652 Staff Interviews were completed. Providers who served individuals for both non-intensive outpatient services as well as ACT had at least one individual receiving ACT services and one ACT team member interviewed when possible. This does not include interviews conducted during the crisis stabilization unit (CSU) reviews that are discussed later in the report. There was a decrease in both Individual and Staff Interview participation by approximately 120 interviews in each category from the previous year. Analysis has shown that fewer interviews were conducted at smaller provider locations (serving 5-50 individuals). Several Small providers had only three Individual/Staff Interviews conducted due to the availability of individuals served and staff available with some individuals/staff being previously interviewed. The Collaborative makes attempts to avoid duplication of individuals/staff interviewed. Additionally, in FY 2016 and FY 2017, oversampling of interviews occurred with some providers having six to seven Individual/Staff Interviews conducted at time of review, increasing the total interview counts for those years. Results from interviews conducted are not included in the BHQR Overall scores; however, results provide valuable, qualitative feedback to promote quality improvement activities. The results of interviews were shared with providers and DBHDD to provide direct communication and perception of individuals receiving services and staff providing services as an additional quality initiative. BHQR Overall Review Scores In this report, data is aggregated and presented by overall provider scores as well as by category (Billing Validation, Assessment and Planning, Service Guidelines, Focused Outcome Areas [FOA], and Individual and Staff Interview). The four main categories, Billing Validation, Assessment and Planning, Service Guidelines and Focused Outcome Areas each account for 25 percent of the Overall 25 | P a g e Quality Management Annual Report FY 2018 score. Each FOA (Whole Health, Safety, Person Centered Practices, Community Life, Choice, and Rights) also has scores displayed and discussed. BHQR Overall Scores Figure 2 shows a graphical representation of the average Overall score for the 156 reviews and the scores for each category compared to the previous two fiscal years to demonstrate areas of improvement and decline. The mean of Overall scores for FY 2018 was 88 percent. This is an increase of four points over FY 2017, affected by the increases identified in all of the four categories. The FOA category demonstrated the most consistent increases across fiscal years with a FY 2018 result of 92 percent. Service Guidelines, the highest category in FY 2016 at 90 percent decreased in FY 2017 by two points to 88 percent yet returned to the baseline result for FY 2018 with significant improvement (p < .01). The lowest scoring category remained Assessment and Planning for the third year, previously averaging 79 percent in FY 2016, 77 percent for FY 2017, and significantly increased (p <.01) to 84 percent in FY 2018. The following are key findings of the overall category scores: Assessment and Planning had the greatest improvement from the previous fiscal year by ten points, compared to other BHQR categories. However, Assessment and Planning scores remain lower than other categories for the third consecutive year and continue to be driven by the lack of documentation to support incorporation of whole health and wellness goals into Individual Recovery/Resiliency Plans (IRPs), as well as lack of addressing co-occurring health conditions, discharge planning criteria, and addressing all assessed needs. Billing Validation steadily trended upward since contract inception with scores of 81 percent, 84 percent, and 85 percent respectively across the past three fiscal year as depicted in Figure 2 on the next page. The FOA category continued to demonstrate an upward trend in results through FY 2018. Providers have been receptive to education and technical assistance specific to this category and, therefore, have increased documentation and service treatment needs related to this area. 26 | P a g e Quality Management Annual Report FY 2018 Figure 2. BHQR Fiscal Year Results by Category Billing Validation FY16: 81% FY17: 84% FY18: 85% Service Guidelines FY16: 90% FY17: 88% FY18: 90% Overall FY16: 84% (N = 141) FY17: 84% (N = 167) FY18: 88% (N = 156) Focused Outcome Areas FY16: 85% FY17: 89% FY18: 92% Assessment & Planning FY16: 79% FY17: 74% FY18: 84% BHQR Overall Scores by Tier and Provider Size In July 2014, DBHDD implemented a community behavioral health provider network structure in which providers were classified using a four-tiered structure.5 Tiers are defined as follows: Tier 1: Comprehensive Community Providers Tier 2: Community Medicaid Providers Tier 2+: Community State Funded Providers Tier 3: Specialty Providers Table 1 provides a snapshot of the BHQR Overall scores by tier and a distribution of scores by each tier and presented in detail in Appendix B. Results increased from FY 2017 to FY 2018 across tier levels 1, 2, and 2+. Tier 1 and 2 improved by four points and exceeded the statewide average of 88 5 Policies regarding the implementation and definition of the DBHDD Community Behavioral Health Provider Network Structure can be found at DBHDD's PolicyStat website: https://gadbhdd.policystat.com 27 | P a g e Quality Management Annual Report FY 2018 percent. Tier 2+ providers reviewed (N = 6) previously had the highest score (compared to other tiers) increased by one percentage point yet fell behind Tiers 1 and 2. Consistent with results of previous years, scores remained generally lower for Tier 3 providers (83%) compared to the other tiers. Table 1. BHQR Overall Scores by Tier* Fiscal Year Tier 1 Tier 2 Tier 2+ Tier 3 Overall Average FY 2016 86% 85% 88% 78% 84% (N = 24) (N = 90) (N = 5) (N = 22) (N = 141) FY 2017 85% 85% 86% 83% 84% (N = 28) (N = 115) (N = 6) (N = 18) (N = 167) FY 2018 90% 89% 87% 83% 88% (N = 29) (N = 93) (N = 6) (N = 28) (N = 156) *N represents the number of reviews conducted per Tier level. Some providers may have had a reassessment review thereby increasing the number as compared to previous years. FY 2018 sampling methods were altered based on a defined provider size procedure, established in collaboration with DBHDD. Provider sizes were determined based on number of unique individuals served within a six-month period and stratified into three categories by size: "Small," "Medium," and "Large." Providers serving less than five individuals are evaluated for review eligibility throughout the year and may receive an ad hoc quality review at the direction of DBHDD. Provider sizes are outlined below: Small Providers (serving 5-50 individuals) Medium Providers (serving 51-100 individuals) Large Providers (serving 101+ individuals) Results of both the Overall and Billing score based on provider size are identified in Figure 3. 28 | P a g e Quality Management Annual Report FY 2018 Figure 3. Fiscal Year 2018 Results by Provider Size 85% 91% 89% 88% 80% 90% 90% 85% Overall Score Small (N = 71) Medium (N = 29) Billing Score Large (N = 56) Overall (N = 156) Based on FY 2018 findings, Medium providers had the highest overall result at 91 percent and exceeded the statewide average of 88 percent. Specific to Billing scores, Medium and Large providers exceeded the 85 percent billing average. Small providers had an average Billing score of 80 percent while Medium/Large providers had an average 90 percent. Additionally, Small providers scored lower in all categories, including Overall score. Table 2 includes category scores by provider size. Table 2. BHQR Category Scores by Provider Size FY 2018 Provider Size Overall Billing Service Guidelines Assessment & Planning FOA Small 85% 80% 89% Medium 91% 90% 92% Large 89% 90% 91% Statewide 88% 85% 90% 82% 91% 88% 95% 85% 92% 84% 92% 29 | P a g e Quality Management Annual Report FY 2018 While Small providers make up 45 percent of the reviews conducted (N = 71, or 61 unique providers), they have fewer individuals reviewed (5-10); thereby, increasing the impact of missing elements or standards assessed negatively. BHQR Billing Validation The Billing Validation score for each BHQR is the percent of justified paid dollars divided by the total paid dollars for the reviewed claims. Billing scores are averaged across the network of annual reviews to obtain an annual statewide average. This category continued to trend upward from FY 2016. Improvement at the provider level is evident as in FY 2018; 19 providers scored a 99 percent or above whereas 15 providers scored such in FY 2017. One provider, not previously reviewed, scored a zero in FY 2018. See the distribution of BHQR Billing scores in Appendix B. Since an increase in Billing scores continued in FY 2018, the billing threshold will raise to 90 percent for FY 2019 to align with DBHDD's KPI (Key Performance Indicators). See Reassessment Frequency Review section for more details. Figure 4 shows the total dollar amount reviewed through an analysis of paid claims, for all providers reviewed during the BHQRs for FY 2018 ($2,430,951.50) as compared to FY 2017 ($2,934,560.52) and FY 2016 ($3,417,902.28) as well as the dollar amount found to be unjustified, $294,743.82 (12%), $463,049.93 (16%), and $807,050.16 (24%), respectively. In FY 2018, fewer funds reviewed may be due to several factors, including but not limited to: provider sample size, limited claims availability, or the types of service reviewed. Although the total funds reviewed for FY 2018 were fewer than reviewed in previous years, the amounts unjustified and susceptible to recoupment continued to positively trend downwards from 24 to 16 to 12 percent. 30 | P a g e Quality Management Annual Report FY 2018 Figure 4. BHQR Billing Validation Amount Reviewed by Fiscal Year6 Total Amount Paid Claims Reviewed in FY 2016: $3,417,902.28 Total Amount Paid Claims Reviewed in FY 2017: $2,934,560.52 Total Amount Paid Claims Reviewed in FY 2018: $2,430,951.50 $3,500,000.00 $3,000,000.00 $2,500,000.00 $2,000,000.00 $1,500,000.00 $1,000,000.00 $500,000.00 $0.00 $807,050.16 24% $463,049.93 16% $2,610,852.12 76% $2,471,510.59 84% $294,743.82 12% $2,136,207.68 88% Total $ Justified Total $ Unjustified FY 2016 FY 2017 FY 2018 Figure 5 provides the percent of reviewed claim amounts by funding source. In FY 2018, $1,930,310.97 in Medicaid funds were reviewed compared to the FY 2018 state funded amount of $500,640.53. 6 The percent of justified and unjustified dollar amounts, in Figure 4, are total dollar amounts justified or unjustified divided by the total amount of funds reviewed in total for the year. The annual statewide average Billing score is not equivalent to the percent justified/unjustified depicted in the figure; statewide Billing score is calculated based on averaging all provider scores at year-end. 31 | P a g e Quality Management Annual Report FY 2018 Figure 5. FY 2018 BHQR Amount Reviewed by Funding Source 21% State Funds* Total Amount Reviewed $2,430,951.50 79% Medicaid *State funds are fee for service and state funded encounters combined. Eighty-seven percent of Medicaid claims reviewed were found justified compared to 90 percent of state funded claims/encounters in FY 2018, see Table 3. Table 3. Amount Reviewed by Funding Source State Funds Medicaid FY 2018 FY 2018 Amount Justified $450,758.86 $1,685,448.82 (90%) (87%) Amount Unjustified $49,881.67 $244,862.15 (10%) (13%) Total Amount Reviewed $500,640.53 $1,930,310.97 Although some services have substantially fewer state funds reviewed for FY 2018 compared to Medicaid funds, 100 percent of state funds for Community Support, Individual, and Psychological Testing were identified as justified. However, Family Skills Training and Development claims had only 35 percent of state-funded billing funds justified compared to 80 percent of Medicaid funds. Statefunded justified dollars also exceeded Medicaid justified dollars in terms of ACT, Community Support Team (CST), Intensive Case Management, and Intensive Family Intervention. Only 49 percent of state funded dollars reviewed for Mental Health (MH) Peer Support Individual were justified compared to 32 | P a g e Quality Management Annual Report FY 2018 76 percent of reviewed Medicaid dollars. In FY 2019, Quality Management will further analyze the billing discrepancies between the funding sources for any trends. Table 4 and Table 5 provide the amount and percentages of funds justified by service, non-intensive outpatient and specialty services of all funds combined, respectively. Table 4. FY 2018 Non-Intensive Outpatient Services Percent of Funds Justified Service Amount Justified Amount Reviewed Justified Diagnostic Assessment $37,270.98 $39,762.22 94% Community Transition Planning $3,103.01 $3,312.21 94% Individual Outpatient Services $380,030.45 $409,683.46 93% Behavioral Health Assessment $75,895.66 $83,962.92 90% Medication Administration $3,494.80 $3,876.51 90% Psychiatric Treatment $67,178.02 $75,386.25 89% Group Outpatient Services $107,754.77 $121,168.70 89% Community Support Individual $128,832.09 $18,089.04 88% Service Plan Development $51,843.00 $59,530.24 87% Nursing Assessment & Health Services $44,389.22 $51,337.63 86% Family Counseling/Therapy $140,012.07 $166,185.24 84% Case Management $109,814.09 $134,652.93 82% Psychological Testing $29,166.90 $35,869.25 81% Crisis Intervention $9,301.10 $12,123.05 77% Total $1,188,086.16 $1,343,771.74 88% 33 | P a g e Quality Management Annual Report FY 2018 Table 5. FY 2018 Specialty Services Percent of Funds Justified Service Amount Justified Amount Reviewed Peer Support Whole Health & Wellness - Individual $5,606.28 $5,606.28 Peer Support Whole Health & Wellness - Group $10,082.92 $10,225.74 MH Peer Support Program $83,508.23 $90,261.59 Assertive Community Treatment (ACT) $295,649.45 $325,851.60 Psychosocial Rehabilitation - Individual $151,633.45 $172,628.40 Intensive Case Management $16,751.46 $19,101.80 Community Support Team (CST) $13,042.33 $14,985.88 Opioid Maintenance $9,430.70 $11,063.50 Psychosocial Rehabilitation Program $66,408.84 $78,011.70 Addictive Disease Support Services (ADSS) $54,703.17 $64,477.32 Intensive Family Intervention $221,099.15 $267,624.22 MH Peer Support - Individual $19,806.24 $26,289.03 AD Peer Support - Individual $399.30 $1,052.70 Grand Total $948,121.52 $1,087,179.76 Justified 100% 99% 93% 91% 88% 88% 87% 85% 85% 85% 83% 75% 38% 87% When a claim was found to be unjustified, assessors selected all applicable reasons a reviewed claim was identified as a discrepancy; therefore, one claim may have multiple reasons. Nearly 4,300 fewer discrepancies were found in FY 2018 than in the baseline year of FY 2016. Figure 6 provides the percent of discrepancies within the fiscal year. 34 | P a g e Quality Management Annual Report FY 2018 Figure 6. FY 2018 Percent of Discrepancy Reasons Content not support units billed (11.3%) Content not match service definition (10.6%) Content not support code billed (10.3%) Progress note is missing (8.3%) Missing/incomplete order (7.3%) Does not meet admission criteria (6.5%) Code is missing/different than code billed (6.2%) Staff credential missing (6.0%) Credential not supported by documentation (5.4%) Signature missing (4.7%) Date of entry missing (4.1%) Other* (3.8%) Content is not unique to the individual (3.6%) Intervention outside staff's scope/practice (3.1%) Diversionary activities billed (2.8%) Location missing (out-of-clinic) (2.7%) Units billed exceed time/units documented (2.3%) 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% *Other instances in Figure 6 include the following discrepancy reasons: Intervention outside staff's scope/practice (3.1%) Diversionary activities billed (2.8%) Location missing (out-of-clinic) (2.7%) Units billed exceed time/units documented (2.3%) Multiple services billed at the same time (1.2%) Date of service incorrect/missing (< 1%) Non-billable activity (< 1%) Consistency requirements missing (< 1%) Time in/time out missing (< 1%) Mutually exclusive services billed (< 1%) Record not submitted within timeframe (< 1%) 35 | P a g e Quality Management Annual Report FY 2018 Printed name missing (< 1%) The most prevalent discrepancy reasons FY 2018 differed from previous years: Content not supporting units billed For example, billed eight units for a medication check with no further documentation of interventions provided. Content not supporting code billed For example, out-of-clinic modifier billed but session took place in office. Content of note not matching service definition For example, billed CSI but documentation indicated the practitioner provided counseling. In both FY 2016 and FY 2017, the most common discrepancy reasons found were missing/incomplete service orders, individuals not meeting admission criteria, and missing progress notes. Individuals not meeting admission criteria declined by 61 percent; an improvement. Instances of missing progress notes declined in FY 2018 by 44 percent or 326 instances; also an improvement. Two reasons for unjustified claims, "Progress note missing" and "Individual did not meet admission criteria," became two of the most improved areas in FY 2018, in addition to "Time in/out missing," which declined in instances by 76 percent. Table 6 provides the number of all discrepancy reasons, in ranking order for FY 2018, and the change in percent from FY 2016 to FY 2018. A negative difference demonstrates improvement from year to year with highlighted areas demonstrating a statistically significant change. Overall, 19 (83%) of the observed indicators (N = 23) improved from the previous fiscal year. 36 | P a g e Quality Management Annual Report FY 2018 Table 6. BHQR Billing Discrepancy Reasons* FY 2016 FY 2017 FY 2018 Instances Instances Instances Claims Reviewed Missing/incomplete order Does not meet admission criteria Quantitative 31,213 1,612 (5.2%) 1,132 (3.6%) Instances 29, 602 496 (1.7%) 748 (2.5%) Instances 25,598 285 (1.1%) 254 (1.0%) Instances Percent Change FY16 FY17 -5.16% -67.5% -30.2% Change Progress note is missing Code is missing/different than code billed 779 (2.5%) 381 (1.2%) 676 (2.3%) 379 (1.3%) 326 (1.3%) 244 (1.0%) -8.6% 4.9% Staff credential missing 299 (1.0%) 295 (1.0%) 236 (0.9%) 4.4% Signature missing Date of entry missing 275 (0.9%) 263 (0.8%) 194 (0.7%) 185 (0.6%) 182 (0.7%) -25.1% 160 (0.6%) -26.4% Units billed exceed time/units documented Consistency requirements missing 236 (0.8%) 202 (0.7%) 112 (0.4%) 57 (0.2%) 90 (0.4%) 31 (0.1%) -49.7% -70.6% Credential not supported by documentation 199 (0.6%) 276 (0.9%) 211 (0.8%) 45.9% Record not submitted within timeframe Location missing (out-of-clinic) Time in / time out missing 197 (0.6%) 166 (0.5%) 92 (0.3%) 19 (0.1%) 155 (0.5%) 68 (0.2%) 10 (< 0.1%) 106 (0.4%) 14 (0.1%) -90.5% -2.2% -22.0% Date of service incorrect / missing Printed name missing Performance Standards Content does not match service definition Content does not support code billed 63 (0.2%) 42 (0.1%) FY 2016 Instances 735 (2.4%) 428 (1.4%) 42 (0.1%) 2 (<0.1%) FY 2017 Instances 489 (1.7%) 397 (1.3%) 34 (0.1%) 8 (<0.1%) FY 2018 Instances 413 (1.6%) 403 (1.6%) -30.6% -92.6% Percent Change -29.9% -2.3% Content does not support units billed Intervention outside staff's scope/practice Content is not unique to the Individual 375 (1.2%) 348 (1.1%) 137 (0.4%) 518 (1.8%) 79 (0.3%) 243 (0.8%) 441 (1.7%) 120 (0.5%) 140 (0.5%) 45.7% -75.8% 86.8% Multiple services billed at the same time 95 (0.3%) 79 (0.3%) 45 (0.2%) -11.3% Non-billable activity 76 (0.2%) 59 (0.2%) 34 (0.1%) -17.9% Diversionary activities billed 48 (0.2%) 26 (0.1%) 111 (0.4%) -41.5% *Areas highlighted in blue within table reflect significance at a p < .01 and indicators with n < 90 were not tested due to low volume. Percent Change FY17 FY18 -13.5% -33.7% -60.8% Change -44.1% -25.5% -7.8% 7.7% 0.8% -7.5% -36.3% -11.4% -34.9% -20.4% -76.2% -5.1% >100% Percent Change -2.2% 17.5% -1.6% 73.6% -33.3% -34.9% -33.6% >100% 37 | P a g e Quality Management Annual Report FY 2018 BHQR Assessment and Planning Assessment and Planning consisted of ten indicators answered once per record reviewed. Table 7 contains the indicators and percent "yes" in ranking order. All indicators increased or remained equal to FY 2017 results, with areas of growth continuing to include the following: Similar to FY 2016 and FY 2017, the lowest-scoring indicator remains "Co-occurring health conditions have been addressed in the IRP" with FY 2018 results of 60 percent. When all other needs were identified (housing, employment, childcare, higher education, etc.), they were only addressed in 59 percent of the plans in FY 2016, 48 percent in FY 2017, and 67 percent in FY 2018. In FY 2016, 80 percent of records were scored as having a discharge plan present. In collaboration with DBHDD, the indicator was re-written for FY 2017 to include that discharge plans meet requirements of having a specific discharge date, specific step-down service, and criteria to measure progress. This seemingly small shift in this indicator caused an overall decline in this indicator score to 58 percent in FY 2017, however, scores increased in FY 2018 to 66 percent; an improvement by eight points. In order to gather additional data, FY 2019 review tool revisions will require Assessors to document specifically the missing elements through a drop-down menu. Table 7. BHQR Assessment & Planning Indicator Scores by Year Indicator FY 2016 FY 2017 Current medical screening is present 98% 97% Individual meets admission criteria 95% 97% Goals/objectives honor hopes, choice, preferences, outcomes 91% 89% Current behavioral health assessment N/A 94% IRP is individualized in personalized language 78% 87% Interventions/objectives are goal-linked & service-consistent 96% 91% Whole health & wellness in IRP 59% 56% All assessed needs are addressed 59% 48% Discharge plan defines criteria 80% 58% Co-occurring health conditions addressed in IRP 36% 34% Assessment & Planning Result 79% 77% *Areas highlighted in blue within table reflect significance at a p < .01 as tested from the previous year. FY 2018 98% 97% 94% 94% 94% 92% 69% 67% 66% 60% 84% 38 | P a g e Quality Management Annual Report FY 2018 Comparisons in scoring continue to reflect a deficit in a comprehensive, whole-person, whole-health approach to recovery/resiliency planning, as it remains the lowest of all BHQR categories. However, the average score for Assessment and Planning increased significantly (p < .01) from 77 to 84 percent for FY 2018. All outcomes increased significantly, except "Individual meets admission criteria" (97% for FY 2017 and FY 2018). The greatest improvement was among "All assessed needs are addressed" (48% in FY 2017, 67% in FY 2018). Data suggests that although records contained required assessment documentation, such as medical screenings and current behavioral health assessments, a vast percentage of recovery/resiliency plans lacked the documentation of discharge planning, co-occurring health conditions, or a plan for increased whole health and wellness. BHQR Service Guidelines Compliance with service guideline indicators were answered once per record reviewed. The number of indicators answered varied, specific to the service. The service reviewed for each record was dependent upon on the services contained in the billing claims sample for the respective individual; therefore, multiple services could be reviewed within one individual's record. FY 2018 reviews contained 16 non-intensive outpatient services and 14 specialty services, as shown in Figure 7 and Figure 8. In FY 2018, Community Transition Planning was added as a scored service and included in the billing sample of 17 reviews (18 records). Furthermore, Peer Support Whole Health and Wellness differentiated between individual and group services in the latter part of FY 2018. 39 | P a g e Quality Management Annual Report FY 2018 Figure 7. FY 2018 Percent of Dollars Reviewed by Non-Intensive Outpatient Service Individual Counseling (27%) Psychosocial Rehabilitation - Individual (11%) Community Support Individual (10%) Case Management (9%) Family Counseling/ Therapy (8%) Group Outpatient Services (8%) Psychiatric Treatment (5%) Individual Counseling (27%) Behavioral Health Assessment (6%) Psychiatric Treatment (5%) Behavioral Health Assessment (6%) Service Plan Development (4%) Group Outpatient Psychosocial Nursing Assessment & Health Services (3%) Services (8%) Rehabilitation - Individual Family Skills Training and Development (3%) Diagnostic Assessment (3%) Psychological Testing (2%) Family (11%) Counseling/ Community Therapy (8%) Case Support Management Individual (10%) Crisis Intervention (1%) (9%) Medication Administration (<1%) 40 | P a g e Quality Management Annual Report FY 2018 Figure 8. FY 2018 Percent of Dollars Reviewed by Specialty Service Assertive Community Treatment (36%) Intensive Family Intervention (29%) MH Peer Support Program (10%) Psychosocial Rehabilitation Program (8%) Addictive Disease Support Services (7%) MH Peer Support - Individual (3%) Intensive Case Management (2%) Addictive Disease Support Services (7%) Psychosocial Rehabilitation Program (8%) Assertive Community Treatment (36%) Community Support Team (2%) Opioid Maintenance (1%) MH Peer Support Program (10%) Peer Support Whole Health & Wellness (1%) Peer Support Whole Health & Wellness - Individual (1%) Psychosocial Rehabilitation-Group (<1%) Intensive Family Intervention (29%) AD Peer Support - Individual (<1%) Peer Support Whole Health & Wellness - Group (<1%) Table 8 provides the scores by service type and year of review and represents the number of reviews assessed for the service, based on the claims sample. Additionally, in Table 8 (r) references the number of reviews, and (n), the number of records. For FY 2016, each provider was reviewed only once; thus, the (r) also represents the number of providers reviewed for each service for that specific fiscal year. However, in FY 2017 and again in FY 2018 multiple providers were reviewed more than once; thus, the (r) representation has been changed to reflect the number of reviews, which allows for consistency in the year-to-year comparisons. 41 | P a g e Quality Management Annual Report FY 2018 Table 8. BHQR Service Guidelines Scores by Service Type (r) represents the number of reviews and (n) the number of records) Service Type AD Peer Support - Individual Addictive Disease Support Services Assertive Community Treatment Case Management Community Support Community Support Team Community Transition Planning Family Counseling/ Training Group Counseling/ Training Individual Counseling Intensive Case Management Intensive Family Intervention MH Peer Support Individual MH Peer Support Program Nursing Assessment & Health Opioid Maintenance Treatment FY 2016 79% (r = 1, n = 7) 81% (r = 48, n = 224) 85% (r = 19, n = 314) 85% (r = 69, n = 557) 84% (r = 84, n = 484) 94% (r = 7, n = 23) N/A 96% (r = 92, n = 667) 93% (r = 63, n = 418) 97% (r = 113, n = 1288) 95% (r = 10, n = 55) 86% (r = 30, n = 221) 87% (r = 7, n = 119) 95% (r = 33, n = 234) 85% (r = 85, n = 884) N/A FY 2017 48% (r = 1, n = 13) 85% (r = 69, n = 293) 88% (r = 20, n = 334) 84% (r = 90, n = 746) 83% (r = 114, n = 637) 92% (r = 7, n = 13) N/A 91% (r = 129, n = 904) 90% (r = 78, n = 561) 94% (r = 150, n = 1980) 94% (r = 13, n = 53) 85% (r = 37, n = 185) 95% (r = 6, n = 38) 86% (r = 25, n = 199) 80% (r = 116, n = 897) 98% (r = 3, n = 69) FY 2018 80% (r = 2, n = 9) Percent Percent Change Change FY16- FY17- FY17 FY18 -64.6% 66.7% 80% (r = 61, n = 252) 4.7% 2.4% 84% (r = 21, n = 336) 3.4% -4.5% 88% (r = 85, n = 644) -1.2% 4.8% 86% (r = 102, n = 446) -1.1% 3.6% 99% (r = 10, n = 22) -2.2% 7.6% 100% (r = 17, n = 18) N/A N/A 94% (r = 109, n = 625) -6.2% 3.3% 95% (r = 75, n = 535) -3.2% 5.6% 96% (r = 132, n = 1390) -3.3% 2.1% 94% (r = 11, n = 46) -1.2% 0.0% 90% (r = 33, n = 149) -1.2% 5.9% 90% (r = 20, n = 92) 8.4% -5.3% 91% (r = 36, n = 210) -10.5% 5.8% 87% (r = 104, n = 649) -5.5% 8.7% 92% (r = 6, n = 52) N/A -6.1% Peer Support Whole Health & Wellness Individual Group Psychiatric Treatment 92% (r = 7, n = 45) N/A N/A 95% (r = 91, n = 843) Psychosocial Rehabilitation Program 90% (r = 75, n = 196) Psychosocial Rehabilitation - Individual 96% (r = 25, n = 649) 68% (r = 12, n = 67) N/A N/A 92% (r = 128, n = 950) 93% (r = 26, n = 179) 85% (r = 98, n = 904) 86% (r = 7, n = 28) 93% (r = 2, n = 28) 80% (r = 2, n = 3) 97% (r = 111, n = 717) 96% (r = 26, n = 153) 90% (r = 83, n = 679) -35.3% N/A N/A -3.3% 3.2% -12.9% 26.5% N/A N/A 5.4% 3.2% 5.9% Note: Areas highlighted in blue within table reflect significance at a p < .01 from the previous year. AD Peer Support Individual service was not statistically tested due to low response volume in FY 2017. 42 | P a g e Quality Management Annual Report FY 2018 Service Guidelines scored highest of the four categories in FY 2016 with an average of 90 percent. Although it declined slightly in FY 2017 by two points, it has since returned to 90 percent. Moreover, nearly three quarters of the reviews (71.8%, N = 112) met or exceeded 90 percent with nine reviews receiving a perfect score; three of whom were providers reviewed for the first time. Psychiatric Treatment, Psychosocial Rehabilitation Program, and Individual Counseling demonstrated the highest rate of compliance for FY 2018 compared to all services measured, exceeding 95 percent. Community Support Team was reviewed using 13 indicators and scored 100 percent on all but one indicator (91%) related to development of a crisis plan including both the provider and the individual. Furthermore, Community Transition Planning, a newly reviewed service for FY 2018, resulted in 100 percent compliance of records meeting the Compliance with Service Guidelines necessary standards. All services reviewed in FY 2018 were scored in relation to documentation contained within progress notes. Three indicators were scored for all services and the results indicate significant improvement from the previous fiscal year (p < .01) and included the following: Progress Notes document progress (or lack of) toward goals/objectives identified on the IRP: o FY 2018: 90 percent o FY 2017: 86 percent The staff interventions reflected in the progress notes are related to the staff interventions listed on the IRP: o FY 2018: 92 percent o FY 2017: 88 percent The progress notes document individual response to the staff intervention provided: o FY 2018: 99 percent o FY 2017: 98 percent The most-improved services from FY 2017 to FY 2018 are outlined in Table 9 with (r) referencing the number of reviews, and (n), the number of records. Although these services were above the statewide average, specific indicators at the service level can lend to growth and opportunity. Table 9 provides for the lowest scoring indicator(s) by service. 43 | P a g e Quality Management Annual Report FY 2018 Table 9. FY 2018 Improvement Results by Service (r) represents the number of reviews and (n) the number of records Service Name and FY 2018 lowest scoring indicator(s): FY 2016 FY 2017 FY 2018 AD Peer Support - Individual 79% (r = 1) 48% (r = 1) 80% (r = 2) The providing practitioner is a Georgia-Certified Peer Specialist-AD (CPS-AD) Peer Support Whole Health & Wellness There is a minimum of one contact weekly either face-toface or by phone (Review authorization period) Service is provided by a Whole Health and Wellness Coach (CPS) Collaboration with other healthcare providers to assure individual has access to needed services is documented (1 x per authorization) Nursing Assessment & Health Services 0% (n = 7) 92% (r = 7) 89% (n = 45) 100% (n = 45) 77% (n = 45) 85% (r = 85) 100% (n = 10) 68% (r = 12) 66% (n = 67) 71% (n = 66) 51% (n = 67) 80% (r = 116) 22% (n = 9) 86% (r = 7) 50% (n = 28) 68% (n = 28) 68% (n = 28) 87% (r = 104) Nursing goals and objectives are individualized and address health issues to include (but not limited to) medical, physical, nutritional, and behavioral needs Community Support Team 67% (n = 883) 60% (n = 800) 75% (n = 648) 94% (r = 7) 92% (r = 7) 99% (r = 10) There is a joint development of a crisis plan to include the provider and individual. The Provider is listed as primarily 87% (n = 23) responsible Note: Areas highlighted in blue within table reflect significance at a p < .01 from the previous year. 92% (n = 12) 91% (n = 22) The BHQR data also suggest additional areas of focus noted through declines in scores from FY 2017 to FY 2018 by service type. Table 10 contains detailed results, with further analysis and trending included at the indicator level and supplemented by the lowest scoring indicators provided by service. 44 | P a g e Quality Management Annual Report FY 2018 Table 10. FY 2018 Service Type Declined Results (r) represents the number of reviews and (n) the number of records Service type and FY 2018 lowest scoring FY 2016 FY 2017 indicator(s): FY 2018 Opioid Maintenance Therapy N/A 98% (r = 3) 92% (r = 6) The staff interventions reflected in the progress notes are related to the staff interventions listed on the treatment plan MH Peer Support - Individual N/A 87% (r = 7) Progress notes contain documentation of the individual's progress (or lack of) toward specific goals/objectives on the treatment plan Assertive Community Treatment 77% (n = 119) 85% (r = 19) The ACT Team completes a Treatment Plan Review with the staff, the individual, and his/her family/informal supports prior to the reauthorization of services 58% (n = 314) There is evidence the ACT Team is working with informal support systems/collateral contacts at least 2-4 times per month with or without the individual present (and it is documented) to provide support and skills training to assist the 43% (n = 314) Individual in his/her recovery (Review authorization period) Note: Areas highlighted in blue within table reflect significance at a p < .01 from the previous year. 99% (n = 69) 95% (r = 6) 93% (n = 27) 88% (r = 20) 59% (n = 307) 39% (n = 283) 83% (n = 52) 90% (r = 20) 77% (n = 92) 84% (r = 21) 28% (n = 321) 33% (n = 306) Minimal reductions in scores for Service Guidelines occurred in FY 2018 related to Opioid Maintenance Therapy, MH Peer Support Individual, and ACT. Specific ACT results are found within this report as a subset of the BHQR process, as requested by DBHDD. However, even in higher scoring services, indicators specific to the following fell below 80 percent: "Appropriately credentialed practitioners" o AD Peer Support Individual, 22 percent o Assertive Community Treatment, 72 percent "Coordination with family, services and resources, or other health care providers" 45 | P a g e Quality Management Annual Report FY 2018 o Addictive Disease Support Services, 59 percent o Community Support, 63 percent o Peer Support Whole Health & Wellness - Group, 68 percent o Peer Support Whole Health & Wellness Individual, 71 percent This demonstrates that while overall Compliance with Service Guideline results are promising with improvement from the previous year, there are still areas of improvement available depending upon the service. Lastly, Substance Use Intensive Outpatient and Medication Assisted Treatment are to be included with data collection for FY 2019. BHQR Focused Outcome Areas (FOA) Focused outcome areas indicators are answered once per record reviewed. Each FOA has a different number of indicators for a total of 22 indicators. Figure 9 outlines results of overall FOA results of FY 2018 compared to previous years. Additionally, each respective section contains a figure specific to the subcategory FY 2018 results as compared to previous years. 46 | P a g e Quality Management Annual Report FY 2018 Figure 9. BHQR Focused Outcome Area(s) Scores by Year 84% 74% 63% 92%91%95% 96% 93% 83% 87% 80% 78% 96%97% 92% 90%93% 93% 89%92% 85% Whole Health Safety Person Centered Practices Community Life Choice Rights FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) Overall Score Whole Health indicators address whether the records reviewed demonstrated individuals were treated holistically, with their physical health needs being assessed, documented, and monitored. Of the six FOAs, Whole Health with results shown in Figure 10 (historically the lowest scoring) has surpassed the subcategory of Safety (lowest scoring category in FY 2018) by six points. Whole Health for FY 2017 scored 74 percent compared to 63 percent in FY 2016 and has since increased significantly (p < .01) to 84 percent. All four Whole Health indicators improved by approximately ten points each. 47 | P a g e Quality Management Annual Report FY 2018 Figure 10. BHQR Focused Outcome Area by Year Whole Health 63% 74% 84% FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) Safety indicators address whether providers were attending to certain risk factors for individuals, such as their tendency to experience a crisis, have suicidal or homicidal thoughts or actions, and whether individuals have received information and education about the risks and benefits of prescribed medications. Safety represents the lowest-scoring FY 2018 FOA and significantly declined (p < .01) from 83 percent to 78 percent; see Figure 11. For FY 2018, the indicator most often scored "no" was "Individuals (or their legal guardians) has been educated on the risk/benefits of all medication prescribed and there is a signed consent form". However, results of this indicator have trended upward (p < .01) since contract inception to a result of 68 percent (58% in FY 2017). Results of documentation as it relates to how providers "Work with individuals to develop, document, and implement safety/crisis plans" remained consistent from FY 2017 to FY 2018 with a score of 88 percent. Figure 11. BHQR Focused Outcome Area by Year Safety 80% 83% 78% FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) 48 | P a g e Quality Management Annual Report FY 2018 Person Centered Practices measures whether documentation indicates individuals actively participated in creating/modifying the care plans, as needed and desired. While the score of Person Centered Practices declined slightly in FY 2017, there was a significant improvement in FY 2018 from 91 percent to 95 percent. Figure 12 provides results for fiscal years 2016 through 2018. Scores for all indicators improved (p < .01) in FY 2018; "Individuals were active participants in the planning and receiving of services" scored 99 percent. The indicator, "Plan reassessed based upon any changing needs, circumstances and/or response by the individual" had the lowest score at 87 percent. Figure 12. BHQR Focused Outcome Area by Year Person 92% Centered 91% Practices 95% FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) Community Life indicators address how individuals were engaged in their communities of choice and whether they held valued social roles. Figure 13 graphically presents the annual fiscal year result for the Community Life FOA. Only one of the five Community Life indicators declined from the previous year used to measure whether "Individuals were assessed for their need to make changes in their living, learning, working, and/or social environments" (99% in FY 2017, 98% in FY 2017). Results increased across the remaining four indicators for the second year since inception (p < .01), with each indicator at or above 95 percent. 49 | P a g e Quality Management Annual Report FY 2018 Figure 13. BHQR Focused Outcome Area by Year Community Life 87% 93% 96% FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) Choice indicators address how, and if, providers offered individuals options of services and encouraged individuals to make educated choices concerning supports and services provided. Figure 14 provides results of the Choice FOA by fiscal year. In 95 percent of records, "If a barrier was identified, documentation demonstrated alternatives were explored" were scored "yes". Ninetyeight percent of all records reviewed for FY 2018 demonstrated "Individual's known preferences and differences were followed". All improvements were significant per indicator at p < .01. The indicator "Documentation demonstrated how the individual was provided with options of supports and services" scored 98 percent in FY 2018. Figure 14. BHQR Focused Outcome Area by Year 92% Choice 96% 97% FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) Rights indicators address whether individuals were apprised of their rights while receiving services, at the point of entry with the provider and on an annual basis thereafter, and whether they had been informed of their rights under Federal HIPAA laws. There was significant improvement (p < .01) from FY 2016 (90%) to FY 2017 (93%), but minimal change occurred from FY 2017 to FY 2018 with an 50 | P a g e Quality Management Annual Report FY 2018 overall subcategory score of 93 percent, see Figure 15. There was significant improvement (p <.01) for FY 2018 among the indicators "Individuals informed of rights at onset of service delivery" and "HIPAA Privacy and Security Rules were reviewed with individual". Yet, attention is still needed specific to "Individuals informed of rights at least annually", as only 73 percent of records reflected annual updates. However, this was an increase (p < .01) of nearly 13 percent for FY 2018. Figure 15. BHQR Focused Outcome Area by Year Rights 90% 93% 93% FY 2016 (N = 141) FY 2017 (N = 167) FY 2018 (N = 156) BHQR Staff and Individual Interviews Six FOA categories comprise the Staff and Individual Interviews. Individual Interviews served to assess the individual's perception of care with the provider, services rendered, and support in working toward personal goals. Staff Interviews helped determine whether a person-centered approach was used in providing services and empowering individuals. Data in Figure 16 represent the average scores for the individual FY 2018 interviews. Figure 17 represents average scores of Staff Interviews. 51 | P a g e Quality Management Annual Report FY 2018 Figure 16. BHQR Individual Interview Data by Fiscal Year Whole Health 93% 94% 95% Safety 96% 96% 97% Person Centered Practices 96% 97% 99% Community Life 95% 95% 97% Choice 97% 98% 99% Rights 98% 98% 99% Overall FY 2016 (N = 737) FY 2017 (N = 735) 96% 96% 98% FY 2018 (N = 616) 52 | P a g e Quality Management Annual Report FY 2018 Figure 17. BHQR Staff Interview Data by Fiscal Year Whole Health Safety Person Centered Practices Community Life Choice Rights Overall FY 2016 (N = 753) FY 2017 (N = 774) 98% 97% 98% 98% 98% 98% 98% 98% 99% 99% 99% 99% 99% 99% 99% 99% 100% 100% 99% 98% 99% FY 2018 (N = 652) It is notable that all areas scored in the 90th percentile for the third year in a row, indicating a high level of satisfaction for individuals served by the providers. Furthermore, very little difference was identified between scores of similar indicators asked of both individuals and staff. In some cases, results of Staff Interviews were elevated above responses made by individuals on indicators similar in nature. Examples of this include indicators outlined in Table 11. 53 | P a g e Quality Management Annual Report FY 2018 Table 11. FY 2018 Interview Comparison Indicator Individual Staff Choice Individual: Individual's options for supports and services are offered. Staff: Staff is presenting options of services and supports. Please provide a specific example of how an individual in services has a choice of services, supports or staff. Community Life 99% (N = 609) 99.8% (N = 640) Individual: Individual actively participates in the community. Staff: Individual actively participates in the community. Whole Health & Wellness 97% (N = 2,813) 99% (N = 1,853) Individual: Individual is offered needed education and resources/tools to help them manage his/her own health. Staff: Staff are communicating with other providers of care to ensure good communication in coordination of services. Person Centered Planning 98% (N = 540) 97% (N = 615) Individual is educated and/or offered support in developing a WRAP plan as requested. Staff: Staff is aware of and supports the individual as requested to develop a WRAP plan. Rights 93% (N = 371) 91% (N = 396) Individual: Individual feels they can say "no" to recommended treatment, services or supports without retaliation. Staff: Staff is aware of the right to refuse treatment, services or supports without retaliation. Safety 99% (N = 603) 100% (N = 648) Individual: Individual is aware of or recognizes what constitutes: Abuse, Neglect, Exploitation Staff: Staff is aware of what constitutes abuse, neglect and 99% (N = 594) exploitation. *Areas highlighted in blue within table reflect significance at a p < .01 as tested between Individual to staff results. 100% (N = 648) Notably, the most apparent discrepancy between Individual Interviews and Staff Interviews for FY 2017 was specific to the indicator of health-related needs addressed. Of the individuals interviewed, 90 percent felt their health needs were addressed, while staff perceived this to be the case in 97 54 | P a g e Quality Management Annual Report FY 2018 percent of the interviews conducted. Moreover, 93 percent of individuals had seen a primary care physician, 72 percent had seen a dentist, and 93 percent received a routine preventative screening within the past 12 months. In 92 percent of FY 2018 interviews, staff indicated they had received training in providing whole-health informed services and 100 percent of staff felt they supported individuals in overcoming barriers. Assertive Community Treatment (ACT) Quality Management conducted 21 ACT provider reviews in FY 2018 specific to 18 unique providers. FY 2018 included one ACT only provider with thirty ACT-specific individuals sampled. Of providers who offered both non-intensive outpatient services, as well as ACT services, sample selection included up to 15 additional individuals. Three ACT providers required a reassessment within the fiscal year due to low initial scores in FY 2017 or upon initial review in FY 2018. The ACT providers' reviews contained 334 records and 3,334 billing claims. While 88 percent of ACT records reviewed documented a mental health diagnosis, there was a greater percentage of physical and substance use diagnoses than in the BHQRs. Forty-nine percent of ACT records reviewed documented a co-occurring physical diagnosis while BHQR records documented 30 percent and ACT records documented 47 percent co-occurring substance use diagnoses while BHQR records documented 41 percent. The data in the following section represents findings from reviews of ACT services only. 55 | P a g e Quality Management Annual Report FY 2018 ACT Quality Review Overall Scores Figure 18 provides ACT statewide averages by category for the past three years of quality reviews. The Overall score for FY 2018 was 89 percent for all ACT reviews. Figure 18. ACT Fiscal Year Results by Category Billing Validation FY16: 92% FY17: 90% FY18: 91% Service Guidelines FY16: 85% FY17: 88% FY18: 84% Overall FY16: 88% (N = 19)* FY17: 87% (N = 20)* FY18: 89% (N = 21)* Assessment & Planning FY16: 85% FY17: 80% FY18: 87% Focused Outcome Areas FY16: 88% FY17: 90% FY18: 93% *ACT only providers: FY16: 2; FY17: 2; FY18: 1 Table 127 provides the result by category of ACT reviews for FY 2016 through FY 2018 as compared to the overall BHQRs. Of note is that all of the category scores for ACT reviews, except for Service Guidelines, are higher than the corresponding BHQR scores for FY 2018. While these three areas 7 ACT services are reviewed as a part of the BHQR process. For the purpose of this annual report, ACT scores have been extracted for evaluation, monitoring, and analysis. 56 | P a g e Quality Management Annual Report FY 2018 increased from the previous year, Service Guidelines declined in FY 2018 for ACT services by four points to 84 percent. Table 12. BHQR and ACT Results by Category by Year Billing Validation FOA BHQR FY 2016 81% 85% BHQR FY 2017 84% 89% BHQR FY 2018 85% 92% Assessment & Planning 79% 77% 84% Service Guidelines 90% 88% 90% Overall 84% 84% 88% ACT FY 2016 ACT FY 2017 ACT FY 2018 92% 88% 90% 90% 91% 93% 85% 85% 88% 80% 88% 87% 87% 84% 89% ACT Billing Validation Figure 19 shows the total dollar amount reviewed through ACT claims analysis since inception, as well as the dollar amount found to be unjustified and justified across the years. While reviews contained a greater dollar amount in FY 2017, 90 percent were validated as justified which is less compared to the other years. The rate of justified amounts has since increased in FY 2018 by one percentage point ($296,038.97). 57 | P a g e Quality Management Annual Report FY 2018 Figure 19. ACT Billing Validation by Fiscal Year Total Amount Reviewed in FY 2016: $306,628.32 Total Amount Reviewed in FY 2017: $340,428.04 Total Amount Reviewed in FY 2018: $325,845.60 $300,000.00 $250,000.00 $200,000.00 $150,000.00 $100,000.00 $50,000.00 $0.00 $23,422.57 8% $283,205.75 92% $32,707.800 10% $307,720.24 90% $29,806.63 9% $296,038.97 91% FY 2016 Total Amount Justified FY 2017 FY 2018 Total Amount Unjustified Information in Table 13 indicates the specific billing discrepancy reasons found during the ACT reviews with 3,334 claims reviewed for FY 2018, more than any other year. Assessors select all applicable discrepancy reasons for all reviewed claims; therefore, one claim may have multiple discrepancy reasons. While the prevalence of identified common discrepancies continued to decline from the baseline year to FY 2018, such as staff credential missing (2.3%, 1.2%, to 0.4%) and code missing/different than code billed (0.8%, 0.6%, to 0.4%) other reasons for discrepancies became more prevalent. These include intervention outside staff's scope of practice (42 instances in FY 2018 [1.2%]) and content not unique to the individual (56 instances in FY 2018, [1.6%]). While there was continued improvement in the number of instances from the past fiscal year's discrepancies, the following discrepancy reasons increased from the previous year: Content does not support code billed Intervention outside staff's scope of practice Multiple services billed at the same time Content is not unique to the individual 58 | P a g e Quality Management Annual Report FY 2018 Diversionary, non-billable and non-therapeutic in nature, activities billed (for example, watching a movie, playing bingo, etc.) The following discrepancy reasons were not identified in FY 2018: Not meeting admission criteria for service Missing/incomplete order Time in/time out missing Table 13. ACT BHQR Billing Discrepancy Reasons by Year Claims Reviewed Not meeting admission criteria for service Missing/incomplete order Quantitative Staff credential missing Location missing (out-of-clinic) Code is missing / different than code billed Units billed exceed time / units documented FY 2016 Instances 3029 3 (0.1%) 0 (0.0%) 69 (2.3%) 39 (1.3%) 25 (0.8%) 17 0.6%) FY 2017 Instances 3221 20 (0.6%) 12 (0.4%) Instances 40 (1.2%) 32 (1.0%) 19 (0.6%) 24 (0.7%) Percent FY 2018 Change Instances FY16- FY17 3427 192 0 (0.0%) >100% 0 (0.0%) N/A Change 15 (0.4%) -45.5% 32 (1.0%) -22.8% 13 (0.4%) -28.5% 10 (0.3%) 32.8% Progress note is missing 16 (0.5%) 17 (0.5%) 35 (1.0%) -0.1% Date of entry missing 1 (0.03%) 22 (0.7%) 4 (0.1%) >100% Signature missing 1 (0.03%) 25 (0.8%) 4 (0.1%) >100% Time in / time out missing Date of service incorrect/missing Performance Standards 0 (0.0%) 0 (0.0%) 10 (0.3%) 3 (0.1%) Instances 0 (0.0%) N/A 1 (<0.1%) N/A Change Content does not support units billed Content does not support code billed Intervention outside staff's scope of practice 27 (0.9%) 17 (0.6%) 12 (0.4%) 51 (1.6%) 24 (0.8%) 0 (0.0%) 32 (0.9%) 34 (1.0%) 42 (1.2%) -77.6% -32.8% 100% Non-billable activity Multiple services billed at the same time 9 (0.3%) 7 (0.2%) 8 (0.2%) 2 (0.1%) 8 (0.2%) 16.4% 6 (0.2%) 73.1% Content does not match service definition 5 (0.2%) 5 (0.2%) 3 (0.1%) Content is not unique to the individual 4 (0.1%) 5 (0.2%) 56 (1.6%) * Due to the low response volume, Quality did not conduct statistical difference in proportions testing on the above table. 6.0% -17.5% Percent Change FY17 FY18 206 -100% -100% Change -66.7% 0.0% -33.3% -57.1% -80.0% -85.7% -87.5% -100% -100% Change -43.8% 25.0% 100% 0.0% 100% -50.0% > 100% 59 | P a g e Quality Management Annual Report FY 2018 The Billing Validation score is the percent of justified billed dollars divided by the total paid/billed dollars for the reviewed claims. The statewide average ACT Billing score of 91 percent was higher than the BHQR statewide average of 85 percent in FY 2018. This remained consistent with the FY 2017 and baseline year result demonstrating ACT providers had fewer funds identified as unjustified compared to BHQR reviews overall. More than 75 percent of reviews conducted in FY 2018 resulted in a Billing score of 90 percent or greater. The remaining five reviews fell below 85 percent. Targeted technical assistance is conducted in cases where scores continually decline or fall below the threshold established. Alignment with the BHQR process, ongoing education and frequency of reviews for those providers who fall below the 90 percent threshold will continue at two intervals in the upcoming year. ACT Assessment and Planning Figure 20 provides the ACT Assessment and Planning score results by specific indicator annually since FY 2016. Similar to BHQR results, indicators assessing whether co-occurring health conditions, whole health and wellness, assessed needs are addressed on the IRP, and discharge planning criteria remain the lower scoring indicators, however, with noted statistical improvement (p < .01) from FY 2017. Whole health and wellness improved by ten points, co-occurring condition documentation increased by 21 points, while both assessed needs and discharge criteria improved by 17 and 20 points respectively. 60 | P a g e Quality Management Annual Report FY 2018 Figure 20. ACT Assessment and Treatment Planning Scores by Year Current medical screening is present Individual meets admission criteria Interventions/objectives are goal-linked & service-consistent Goals/objectives honor hopes, choice, preferences, outcomes IRP is individualized in personalized language N/A Current behavioral health assessment Whole health & wellness in IRP Co-occurring health conditions addressed in IRP All assessed needs are addressed Discharge plan defines criteria 100% 99% 100% 100% 99% 100% 99% 97% 97% 94% 94% 97% 92% 97% 96% 91% 89% 75% 76% 86% 47% 49% 70% 59% 51% 68% 90% 46% 66% FY 2016 (N = 19) FY 2017 (N = 20) FY 2018 (N = 21) ACT Service Guidelines The ACT service guidelines score contained 14 indicators. Table 14 shows the item-level detail to illustrate both the percentage point and percent change across indicators and comparing fiscal year results within the ACT Service Guidelines, ranked in descending order by FY 2018 results. The ACT Service Guidelines category score decreased significantly (p < .01) from FY 2017 (88%) to FY 2018 (84%). Three indicators scored at or near 100 percent for all three fiscal years: "Individual meets admission or continuing stay criteria" 61 | P a g e Quality Management Annual Report FY 2018 "One of the contacts per month addresses the symptom assessment and management of medications (once a month)" "For discharged individuals, there are multiple documented attempts to locate and make contact with the individual prior to discharge (over a 45-day period)" While success is evident with 11 of the 14 indicators exceeding 90 percent, the ACT team having all required staff substantially declined (p < .01) in FY 2018 by 21 percent. Additionally, and consistent with previous years, continued improvement is needed specific to the reauthorization of ACT services. "ACT team completes a treatment plan review with the staff, the individual, and his/her family/informal supports prior to the reauthorization of services" significantly declined (p < .01) by approximately 53 percent in FY 2018 to a result of 28 percent. Furthermore, evidence the "ACT team is working with informal support/contacts at least two to four times per month prior to reauthorization" decreased (p < .01) by five points to a result of 34 percent. The indicator "ACT team completed a treatment plan review with the staff, the individual, and his/her family/informal supports" significantly declined in FY 2018 to 28 percent. Table 14. ACT Service Guidelines Scores FY 2016 FY 2017 Documentation shows the individual meets admission or continuing stay criteria. For discharged individuals, there are multiple documented attempts to locate and make contact with the individual prior to discharge (over a 45-day period). One of the contacts per month addresses the symptom assessment and management of medications (once a month). The progress notes document individual response to the staff intervention provided. The staff interventions reflected in the progress notes are related to the staff interventions listed on the treatment plan. 100% 100% 100% 99% 98% 99% 100% 99.7% 99% 97% FY 2018 Percent Change FY16FY17 Percent Change FY17FY18 100% -1.0% 1.0% 100% 0.0% 0.0% 100% -0.3% 0.3% 98% 0.0% -1.0% 96% -1.0% -1.0% 62 | P a g e Quality Management Annual Report FY 2018 Table 14. ACT Service Guidelines Scores FY 2016 FY 2017 There is documentation of individual's responses and a discussion to the agreement of services identified in the treatment planning/individual recovery planning. There is documentation of individual's involvement in transition planning. Progress notes contain documentation of the individual's progress (or lack of) toward specific goals/objectives on the treatment plan. The ACT team is working with the individual toward educational or vocational needs, interests, per IRP (once per authorization). There is documentation to support when substance use services are needed and are integrated into the treatment plan. Following admission to a psychiatric facility, the ACT team is involved in each individual's discharge planning. The ACT team has all required staff. There is evidence the ACT team is working with informal support systems/collateral contacts at least 2-4 times per month with or without the individual present (and it is documented) to provide support and skills training to assist the individual in his/her recovery. (Review specific to authorization period.) The ACT team completes a treatment plan review with the staff, the individual, and his/her family/informal supports prior to the reauthorization of services. ACT Service Guidelines Result N/A 82% 91% 95% 94% 87% 76% 43% 58% 85% (N = 19) 92% 90% 99% 97% 93% 95% 91% 39% 59% 88% (N = 20) FY 2018 Percent Change FY16FY17 Percent Change FY17FY18 96% N/A 4.3% 95% 8.9% 5.6% 94% 8.1% -5.1% 94% 2.1% -3.1% 92% -1.1% -1.1% 91% 8.4% -4.2% 72% 16.5% -20.9% 34% -10.3% -12.8% 28% 1.7% -52.5% 84% (N = 21) 3.4% -4.5% ACT Focused Outcome Areas Focused Outcome Area (FOA) indicators are answered once per record reviewed. Each FOA has a different number of indicators for a combined total of 22 indicators. The Overall score for ACT FOA increased (p < .01) from FY 2017 by three points (93%). Four of the six subcategories met or 63 | P a g e Quality Management Annual Report FY 2018 exceeded 95 percent and Choice remained consistent across all three years at 97 percent, see Figure 21. Identified declines for FY 2018 were specific to the Safety and Rights subcategory with only 70 percent of records having "Annual updates to rights" and "Documentation of medication education and consent". However, both individual indicators increased from 58 and 62 percent respectively. Figure 21 provides the ACT FOA results for each category by fiscal year. The greatest improvement in the FOA subcategory scores specific to ACT reviews, for the second consecutive year, was Whole Health, which increased from 76 percent to 90 percent in 2017 and significantly (p < .01) to 95 percent in FY 2018. Furthermore, the Whole Health indicator "Documented safeguards utilized for medication known to have substantial risk" improved in FY 2018. "Ongoing assessment to determine external referrals for health services, supports and treatment" significantly increased (p < .01) in FY 2018. 95% 90% 76% Figure 21. ACT FOA Scores by Category by Year 82% 80% 78% 99% 96% 91% 97% 97% 94% 97% 97% 97% 89% 87% 86% 93% 90% 90% Whole Health Safety Person Centered Practices FY 2016 (N = 19) Community Life Choice Rights FY 2017 (N = 20) FY 2018 (N = 21) Overall 64 | P a g e Crisis Stabilization Unit Quality Reviews Quality Management Annual Report FY 2018 At DBHDD's request, implementation of the Crisis Stabilization Unit Quality Reviews (CSUQR) began in January of FY 2016. The purpose of the CSUQR is to assess the provider's overall practices, quality of service delivery, and to determine adherence to DBHDD standards through individual record reviews.8 When possible, Quality Management conducts the CSUQR and BHQR simultaneously; CSU providers received separate CSUQR scores and final assessment reports. The CSUQR indicators are based on the DBHDD policies and Provider Manual for Community Behavioral Health Providers and are organized into three review categories: Individual Record Review (IRR), Service Guidelines, and FOAs. The score for each category represents the percent of applicable indicators met or present. The CSUQR Overall score is calculated by averaging the three categories, with each category accounting for 33.3 percent of the Overall score. During FY 2018, Quality Management conducted reviews at 19 eligible CSU providers. Like FY 2016, two were freestanding CSUs, and 17 occurred in conjunction with BHQRs. In FY 2017 four CSUs had a second review due to scores falling below the threshold (80 percent Overall score), this reduced to only two CSUs that required a reassessment in FY 2018. Results for the reassessment reviews were included within the overall results incorporating results of 21 reviews. Five of the 19 providers served child and adolescent (C&A) as well as adults, the remaining CSU providers served adults only. Crisis Stabilization Unit Sample Method A random sample of 15 individuals who had received services within the three months preceding the review was selected for record review. When providers did not have an adequate number of individuals served in the three months (a minimum of three days/nights admission) preceding the review, samples were pulled from individuals served up to six months preceding the review. CSUs serving both populations of adults and C&A have a sample of up to ten adults and five C&A selected. 8 Please refer to the following link to access a full description of the review process and review tools. http://georgiacollaborative.com/providers/prv-BH.html 65 | P a g e Quality Management Annual Report FY 2018 FY 2018 CSUQRs consisted of 315 individual records, 290 adult records, and 25 C&A records. All CSUQRs had the maximum 15 records reviewed. The CSUQR mirrors the BHQR scoring and samples for the Individual Interviews and Staff Interviews (selected by the provider and quality assessors). Results from the interviews were not included in calculating the provider's overall CSUQR score. Quality assessors attempted to complete a minimum of five Individual and five Staff Interviews per CSUQR; however, the actual number fluctuated based on individual and staff availability, their agreement to participate in the interview process, the number of staff, and the number of individuals the provider served at the time of the review. Individuals selected for interviews were currently at the CSU, and the staff selected were providing services on the CSU. If an individual or staff declined an interview, assessors selected a different individual or staff. Crisis Stabilization Unit Quality Review Figure 22 shows the Overall and category scores for reviews completed for the past three years (FY 2016, FY 2017, and FY 2018). Areas to highlight based on FY 2018 findings specific to CSUQRs include: The Overall average score for CSUs has steadily trended upward to a FY 2018 result of 88 percent. This is a two-point increase from the previous fiscal year. Nineteen reviews (90%) scored within the 81 to 100 percent range compared to the FY 2016 total of thirteen (68%) and FY 2017 total of eighteen (78%). See distribution graphs in Appendix B. Both the Individual Record Review (IRR) and Service Guideline category increased for FY 2018, with the IRR demonstrating significant improvement (p < .01). The category of Focused Outcome Areas (FOA) remained at 91 percent in FY 2018. Individual Record Review scores of CSUs serving both adults and C&A scored two points higher (85%) than CSUs serving an adult only population; while Service Guideline scores for these CSU providers were three points lower (88%). No difference in FOA scores occured 66 | P a g e Quality Management Annual Report FY 2018 between the populations (91%). Category sections below contain further details pertaining to the C&A CSU record reviews. Figure 22. CSU Overall Scores by Fiscal Year Service Guidelines FY16: 82% FY17: 87% FY18: 91% Individual Record Review FY16: 79% FY17: 80% FY18: 83% Overall Score FY16: 83% (N = 19) FY17: 86% (N = 23) FY18: 88% (N = 21) Focused Outcome Areas FY16: 88% FY17: 91% FY18: 91% Two CSU providers had a second review during FY 2018 due to Overall scores that were below the threshold. The results of these two specific CSUQR reassessment reviews are contained within the reassessment review section of this report in Figures 33 and 34. CSUQR Individual Record Review (IRR) Individual Record Review (IRR) indicators were answered once per record reviewed. The IRR is comprised of six subcategories: Assessment and Planning, Admission, Initial Evaluation, CSU Course of Stay, Documentation, Transition/Discharge Summary, and CSU Planning. Each of the six subcategories had a unique number of indicators, with 30 total scored within the IRR category. Significant improvement (p < .01) or change was identified for all subcategories except Course of Stay and Treatment Planning. Figure 23 identifies the subcategories and results for each by fiscal year. 67 | P a g e Quality Management Annual Report FY 2018 Figure 23. Individual Record Review Category Scores Assessment & Planning Admission/ Initial Evaluation/ Screening for Risk Crisis Stabilization Specific Planning Crisis Stabilization Course of Stay Documentation Transition/ Discharge Planning FY 2016 (N = 19) 86% 85% 88% 69% 69% 78% 70% 72% 74% 83% 86% 85% 87% 89% 85% 63% 62% 92% FY 2017 (N = 23) FY 2018 (N = 21) * The IRR subcategories were individually scored and are not averaged for the final overall IRR score. The final overall IRR score is calculated by adding all "yes" or "present" responses of all IRR indicators and dividing by the total "yes" or "no" responses combined for each record review. The IRR score in FY 2016 was 79 percent and has since increased over the past two years to 83 percent in FY 2018. There is room for improvement when reviewing specific indicators or indicators of the CSUQR IRR. The following indicators received the lowest score for FY 2018 and were also low scoring indicators for FY 2017. These indicators are also evaluated by population, adult or C&A. Documentation: Community transition plan present if individual is readmitted 19 percent o C&A N/A due to no readmissions, Adult 19 percent Assessment & Planning: Co-occurring health conditions included in IRP/NCP 54 percent o C&A 25 percent, Adult 55 percent 68 | P a g e Quality Management Annual Report FY 2018 CSU Planning: IRP/NCP addresses safety issues 55 percent o C&A 88 percent, Adult 51 percent Non-Scored: Vital signs every eight hours 62 percent o C&A 18 percent, Adult 66 percent Non-Scored: Evidence of follow-up and continuing care connection 27 percent o C&A 46 percent, Adult 25 percent Assessment and Planning section had a three-point increase from the previous year with noted improvement as 100 percent of records contained a current medical screening and documentation supporting the individual met admission criteria. "Orders for admission to CSU" were present (97%), as were "Orders for withdrawal management regimens" (94%). Moreover, despite significant increases (p < .01) for both "Discharge plan(s) defining criteria" (69%) and "Co-occurring conditions included in the IRP/NCP" (54%), further growth remains obtainable. Specific to CSU Course of Stay, indicators "Individual is offered groups as needed" and "Individual participated in training/therapy" showed a decline overall by four or five points; however, 100 percent of C&A records documented evidence that the "Individual is offered groups as needed" and that the "Individual participated in training/therapy". Adult record reviews resulted in only 87 percent documenting the offering of group services and 82 percent participation in training/therapy. The third question in this IRR subcategory, "Co-occurring conditions assessed and addressed simultaneously", demonstrated an increase (p < .01) from 69 percent in FY 2017 to 79 percent in FY 2018. When comparing the adult and C&A population, a difference was identified specific to cooccurring disorders; 40 percent of C&A records and 80 percent of adult records "Co-occurring conditions assessed and addressed simultaneously"". Only 55 percent of IRP/NCPs addressed all assessed safety issues, which is an increase from the previous fiscal year's results of 36 percent. A difference between C&A and adult was evident in that 88 percent of C&A records addressed safety issues compared to only 51 percent of adult records. Quality Management will conduct a safety planning training specific to CSUs in FY 2019. 69 | P a g e Quality Management Annual Report FY 2018 Documentation, previously the highest scoring subcategory of the IRR, declined from 89 percent in FY 2017 to 85 percent in FY 2018. The indicator assessing "Community transition plan when the individual was readmitted within 30 days" decreased to 19 percent, and "Medication Administration Record (MAR) having a documented legend" significantly decreased (p < .01) to 65 percent. Medical progress notes (90%) and non-medical progress notes (83%) supplying evidence of progress toward goals and/or objectives remained consistent from FY 2017 to FY 2018 regardless of adult or C&A population. One hundred percent of all C&A records provided documentation specific to both medical and non-medical progress notes displaying evidence of progress compared to 89 percent (medical progress notes) and 82 percent (non-medical progress notes) of adult records reviewed. Although transition/discharge planning historically resulted in a relatively low score for FY 2017 and FY 2016, the indicator of transition/discharge plans containing the needed documentation has significantly increased (p < .01) from the previous year's result from 62 percent to 92 percent. 70 | P a g e Quality Management Annual Report FY 2018 CSUQR Service Guidelines The 16 CSU Service Guidelines indicators (14 scored and 2 non-scored) were answered once per review to assess the CSU program. See Figure 24 and Figure 25. Figure 24. CSUQR Service Guidelines Staffing and Curriculum Results by Indicator by Year CSU Staffing Requirements Met (FY16 n = 19, FY17 n = 23, FY18 = 21) C&A Minimum Staff Present (FY16 n = 3, FY17 n = 5, FY18 = 5) C&A Staff Ratio Met (FY16 n = 3, FY17 n = 5, FY18 = 5) C&A Nursing Staff Ratio (FY16 n = 3, FY17 n = 5, FY18 = 6) Access to Addictionologist (FY16 n = 18, FY17 n = 23, FY18 = 21) 100% 96% 95% 100% 100% 100% 100% 100% 100% 100% 100% 100% 67% 96% 86% Psychiatrist Avilable for Consultation (FY16 n = 0, FY17 n = 18, FY18 = 21) Physician Availability (FY16 n = 19, FY17 n = 22, FY18 = 21) C&A Psychiatrist (Non-scored) (FY16 n = 3, FY17 n = 6, FY18 = 5) Model/Curriculum for SU treatment (Non-scored) (FY16 n = 16, FY17 n = 22, FY18 = 21) FY 2016 (N = 19) FY 2017 (N = 23) 100% 100% 100% 100% 100% 67% 83% 100% 69% 91% 90% FY 2018 (N = 21) 71 | P a g e Quality Management Annual Report FY 2018 CSU staffing requirements were met in 95 percent of the reviews conducted. Of the five CSUs serving children and adolescents, all met the staffing requirements and ratios for FY 2017 and FY 2018. An additional indicator was added to the FY 2017 review process specific to the "Availability of a psychiatrist for consultation in cases in which the CSU physician(s) is/are not specialized in psychiatry", which scored 100 percent both years since indicator inception. Figure 25. CSUQR Service Guidelines Policy Adherence Indicators by Fiscal Year Deaf, Deaf-Blind, Hard of Hearing Policies (FY16 n = 19, FY17 n = 23, FY18 = 17) Infection Control Plan Adherence (FY16 n = 19, FY17 n = 23, FY18 = 21) Theraputic Blood Level Monitoring (FY16 n = 19, FY17 n = 23, FY18 = 21) Seclusion & Restraint Policy Adherence (FY16 n = 19, FY17 n = 23, FY18 = 20) Medication Storage Policy Adherence (FY16 n = 19, FY17 n = 23, FY18 = 21) Adherence to Medication Notification Policy (FY16 n = 19, FY17 n = 23, FY18 = 21) Protocols for Handling Drugs (FY16 n = 19, FY17 n = 23, FY18 = 21) 53% 65% 100% 84% 87% 86% 89% 83% 90% 95% 91% 85% 53% 57% 67% 79% 83% 86% 95% 87% 95% FY 2016 (N = 19) FY 2017 (N = 23) FY 2018 (N = 21) 72 | P a g e Quality Management Annual Report FY 2018 While the majority of policies reviewed improved from the previous year, see Figure 25, "Provider's adherence to their seclusion and restraint policy" declined in FY 2018 to a rate of 85 percent. However, improvement was evident in both "Medication notification policy adherence" and "Medication storage policy adherence". See Figure 26 for fiscal year results of three indicators related to medication policy adherence reviewed by assessors. Figure 26. CSUQR Service Guidelines Medication Policy Adherence Indicators by Fiscal Year Medication Notification Policy Adherence FY 2016 79% FY 2017 83% FY 2018 86% Medication Storage Policy Adherence FY 2016 53% FY 2017 57% FY 2018 67% Protocols for Handling Drugs FY 2016 95% FY 2017 87% FY 2018 95% From FY 2016 to FY 2017, scores increased across the CSU providers specific to Service Guidelines, with more than half of reviews scoring above 90 percent. Now with the Collaborative in its third year of the contract, two-thirds of providers are scoring above 90 percent for this CSUQR category. The median score in FY 2017 was 91 percent compared to the median score in FY 2016 at 80 percent. FY 2018 median provider score is 100 percent. Overall, the statewide average for Service Guidelines increased by four points (91%) in FY 2018. CSU Focused Outcome Area Focused Outcome Area (FOA) indicators were answered once per record reviewed. Each FOA had a unique number of indicators for a total of 23 indicators assessed overall. Please refer to the BHQR FOA section for a definition of the six FOAs. 73 | P a g e Quality Management Annual Report FY 2018 The Overall FOA score for CSU providers increased from 88 percent to 91 percent in FY 2017 and remained at 91 percent for FY 2018. Consistent with both previous years' results for CSUQR FOAs, the highest subcategories were Choice (97%), Rights (97%), and Community (92%). Although both Choice and Community decreased somewhat from the previous year's review by one to two percentage points, respectively, Rights significantly increased (p < .01) from 92 percent to 97 percent. Whole Health (88%) and Safety (82%) decreased from FY 2017, while Person Centered Practices (88%) remained unchanged. See Figure 27 for FOA results by year. Figure 27. CSUQR FOA Results by Year Whole Health Rights FY16: 97% FY17: 92% FY18: 97% Choice FY16: 99% FY17: 98% FY18: 97% FY16: 83% FY17: 89% FY18: 88% FY16: 88% FY17: 91% FY18: 91% Community FY16: 92% FY17: 94% FY18: 92% Safety FY16: 76% FY17: 85% FY18: 82% Person Centered Practices FY16: 77% FY17: 88% FY18: 88% * The FOAs were individually scored and are not averaged for the final overall FOA result at the review level. The final overall FOA result is calculated by adding all "yes" or "present" responses of all FOAs and dividing by the total "yes" or "no" responses combined for each review. The following were findings based on each FOA for FY 2018: Whole Health demonstrated consistent findings in line with what is noted in the CSUQR IRR category. Documentation has improved compared to baseline results regarding evidence of whole health and wellness incorporated into the individual's length of stay. "Documentation of medication safeguards" declined from 85 percent in FY 2017 to 83 percent in FY 2018 as 74 | P a g e Quality Management Annual Report FY 2018 did "Documentation regarding ongoing assessment for determination of need for external referrals" (95 percent in FY 2017 to 91 percent in FY 2018). o One hundred percent of all C&A records reviewed met the criteria of "Demonstrating ongoing assessment to determine external referrals" compared to 91 percent of adult records reviewed. Safety documentation demonstrated significant declines (p < .01) in both "Services offered in a safe environment" (96 to 87%) and "Documentation of providers' assistance with development of safety/crisis planning when needed" (95 to 86%). Improvement was noted in the "Documentation of medication consent and education" compared to FY 2017, with a nearly 10-point increase to 72 percent. o Important to note is the difference in the C&A population versus the adult population specific to "Services being offered in an environment that ensures the individual's safety", as this indicator resulted in a score of only 44 percent for C&A records reviewed compared to 91 percent of adult records documenting such information. o Person Centered Practices documentation in FY 2016 did not support "Individual was an active participant in CSU treatment planning"; however, the result for FY 2017 (90%) and FY 2018 (91%) depicts higher scores. o Specific to the C&A population, 88 percent of records documented evidence of participation in planning of services, which may include guardian participation as well. Additionally, findings remained consistent from the previous year with 83 percent of records documenting "Individuals actively participated in the modification of treatment plan". For FY 2018, 100 percent of C&A records indicate inclusion in the modification of plans. o However, "Treatment plan is reassessed based on changing needs or circumstances" remained a prevalent area for improvement with only 76 percent of total records providing such (91% of C&A records compared to 74% adult records). Community documentation supported a two-point improvement in "Individuals assisted in identifying aftercare placement" (97%). There was a decline in "Transition planning 75 | P a g e Quality Management Annual Report FY 2018 throughout service delivery that involved the individual and/or their natural supports" (89%). No change was found in individual's informed choice driving selection of housing options (95%). o No significant differences were found between the C&A population when compared to the indicators of the adult population on this FOA. o One hundred percent of C&A records received a score of "yes" on all Community FOA indicators with the exception of "Individual and provider have discussed aftercare placement" which was 95 percent. Choice reflected consistent findings with FY 2017 with the "Individual's preferences for treatment were followed" while at the CSU (99%) and when "Barriers to treatment were identified" documentation demonstrated that alternatives were explored to address the barriers (96%). o No significant difference was found between the indicator scores from FY 2017 to FY 2018 within the Choice FOA nor was a signficant difference found between the C&A and adult record scores. o However, it is important to note that 100 percent of C&A records reviewed demonstrated following the individual's preferences and providing the individual with options of supports and services compared to 99 percent records documented following individual preferences and 97 percent documenting providing options within the adult records. Rights supported "HIPAA Privacy and Security Rules were reviewed with the individual" (98%) and "Individuals were informed of rights at onset of service delivery" (92%) both increased in FY 2018 with the latter indicator demonstrating significant improvement in total sample of records reviewed (p < .01). o The C&A records reviewed (n = 25) demonstrated 100 percent compliance in all areas of Rights, demonstrating higher results than the adult record reviews, especially regarding being informed of rights at onset of services (92%, n = 265). 76 | P a g e CSUQR Staff Interview and Individual Interview Quality Management Annual Report FY 2018 The CSUQR interview indicators were similar to the BHQR interview indicators and were divided into the six FOAs. Individual Interviews were used to assess the individual's perception of care with the provider, services rendered, and support in working toward personal goals. Staff Interviews helped determine if a person centered approach was used in providing services and empowering individuals. The data and anecdotal information gathered from the Individual and Staff Interviews supplemented the record review and enhanced the review process by providing a tailored perspective to illustrate the quality of care the agency provided using both a quantitative and qualitative approach. Data in Figure 28 illustrates the results of Staff and Individual Interviews by fiscal year. Individual Interview scores declined slightly from the baseline year to FY 2017 yet increased to 95 percent for FY 2018. Staff Interview scores increased to 99.2 percent for FY 2018. Figure 28. CSUQR Individual and Staff Interview Results by Fiscal Year INDIVIDUAL INTERVIEW FY 2016 94.9% (N = 94) FY 2017 93.0% (N = 114) FY 2018 95.1% (N = 102) STAFF INTERVIEW FY 2016 98.6% (N = 96) FY 2017 98.7% (N = 115) FY 2018 99.2% (N = 105) Figure 29 represents the average FOA scores for Individual Interviews while Figure 30 provides results of Staff Interviews for FY 2016, FY 2017, and FY 2018. Individual Interview results increased in FY 2018 to 95 percent. Again, all but one FOA scored in the 90th percentile. The lowest-scoring FOA 77 | P a g e Quality Management Annual Report FY 2018 for Individual Interviews based on indicator level data remained Whole Health (88%), which may indicate an increased need for CSUs to attend to and plan for individuals' co-occurring health conditions or health needs such as preventive services, dental services, or primary care. This issue was also highlighted in the IRR and FOA categories both in FY 2018 and FY 2017. Figure 29. CSUQR Individual Interview Subcategory Data by Year Whole Health 85% 83% 88% Safety 95% 96% 97% Person-Centered Planning 96% 94% 95% Community Life 99% 96% 96% Choice 97% 98% 98% Rights 100% 99% 98% Overall 95% 93% 95% FY 2016 (N = 94) FY 2017 (N = 114) FY 2018 (N = 102) All Staff Interview indicators answered for FY 2018 exceeded 90 percent. However, the indicator specific to staff being aware of special diets for individuals with access to a nutritionist scored lower than all the others at 92 percent. Additionally, CSUQR and BHQR Staff Interview scores indicated 78 | P a g e Quality Management Annual Report FY 2018 knowledge regarding providing services aligns with standards assessed by the FOA indicators, as well as knowledge of the individuals they serve. Figure 30. CSU Staff Interview Subcategory Data by Year Whole Health Safety Person-Centered Planning Community Life Choice Rights Overall 97% 98% 99% 99% 98% 98% 98% 100% 99% 99% 99% 100% 100% 100% 100% 97% 100% 100% 99% 99% 99% FY 2016 (N = 96) FY 2017 (N = 115) FY 2018 (N = 105) 79 | P a g e Quality Management Annual Report FY 2018 Reassessment Frequency Reviews BHQR Reassessment Review Findings Beginning in FY 2017, the frequency of a BHQR and CSUQR was based on minimum scoring thresholds: less than 80 percent Overall score or less than 70 percent Billing Validation. Providers scoring above the minimum threshold received one BHQR/CSUQR per fiscal year; providers falling below the threshold received two reviews. Providers remain on a frequency schedule of approximately every six months until two subsequent review scores meet threshold requirements. The threshold process continued throughout FY 2018. However, based on FY 2017 data, the threshold was increased for FY 2018 to 80 percent for both Overall score and Billing score. Twentyseven percent of providers required a reassessment in FY 2017 (N = 35), 15 percent (N = 20) required a reassessment in FY 2018, a 12 point decrease. Two of these providers had not been reviewed in FY 2017 due to either being a new provider for FY 2018 or had insufficient claims. Providers whose scores fell below the threshold of 80 percent for either Overall or Billing were scheduled for a repeat review approximately six months following their initial FY 2017 review, allowing ample time for claims submission and documentation to reflect any changes made by the provider based on previous review findings. There was an average of 184.6 days between reviews. Most reassessed providers in FY 2018 were Tier 2 (N = 14). Figure 31 displays the distribution of provider scores for the reassessed providers for scores obtained during the first and second review in FY 2018. Improvement in scores is illustrated when comparing providers' first and second reviews. For the first review of FY 2018, 55 percent of providers scored in the 81 to 90 percent range. Compared to previous years, this same percentage of providers scored in the 71 to 80 percent range for their first review. Additionally, some providers scored between 51 and 79 percent at the first review, and later progressed to the 71 to 80 percent category. 80 | P a g e Quality Management Annual Report FY 2018 Figure 31. BHQR FY 2018 Overall Score Distribution of Reassessed Providers 70% 60% 60% 55% 50% 40% 35% 30% 20% 20% 15% 10% 5% 5% 5% 0% First Review FY 2018 (N = 20; Mean = 83%; Median = 84%) Second Review FY 2018 (N = 20; Mean = 91%; Median = 91%) Overall Score: The average result for the reassessed providers at first review of FY 2018 was 83 percent compared to 91 percent later in the year. Although 83 percent exceeds the threshold requirement for Overall score, 15 providers were reassessed due to the Billing score falling below the threshold. See section related to Billing score in the reassessment section on the next page. Further results show the following: Nine providers exceeded 85 percent at initial review for Overall score compared to 18 at time of second review. Eighty-five percent (N = 17) of providers who had a second review increased an Overall score by end of the fiscal year. The greatest increase in score for a provider was a 32-point change from initial to subsequent review. Two providers had a decline in Overall score from time of initial review to subsequent FY 2018 review, with an average decrease of two percent in their Overall score. Distributions of reassessed providers based on each scored category can be found in Appendix B. Those providers who fell below the established threshold at either of the two reviews during FY 2018 81 | P a g e Quality Management Annual Report FY 2018 will be required to have an initial and subsequent review scheduled for the 2019 fiscal year regardless of improvement in one or both categories (Overall and Billing score). Additionally, due to the increased levels of scores across the network as well as the increased statewide average, the threshold for FY 2019 is anticipated to increase to 90 percent for both Overall and Billing. Assessment and Planning was the lowest category for the reassessed providers in FY 2017 and continued as such in FY 2018 with a result at initial review of 79 percent and an increase of ten points (89%) at subsequent review when averaged across providers. Similar to providers reviewed only once annually, the lowest scored indicators included "Co-occurring condition documentation and assessment, whole health and wellness within the IRP", "Discharge plan defines criteria", as well as "All assessed needs addressed on the IRP". Of the 85 percent of providers who demonstrated improvement in Assessment and Planning, half increased by 10 points or more. One provider had an increase of 47 points increasing from 37 percent at initial review to 84 percent by year-end. Improvement was also evident against the statewide average of 88 percent as 10 (50%) providers scored at or above this result at the time of their second FY 2018 review, compared to only three (15%) providers at the time of their first FY 2018 review. Billing Score: More than two-thirds (80%, N = 16) of providers reviewed for a second time in FY 2018 had an increase in the Billing score (average increase of 14 points). One provider had a substantial increase of 61 points, going from 28 percent to 89 percent. The average across reassessed providers at the time of their initial review was 75 percent compared to 89 percent following all providers' second reviews. Only, two of the twenty reassessed providers scored 100 percent for this category. See Figure 32 for the distribution scores of reassessed providers. Nearly 24 percent of claims were found unjustified at initial review and decreased to only 10 percent unjustified claims at the time of the providers' second review. The most prevalent instances of billing discrepancies identified for reassessed providers remained consistent with those found in FY 2017: meeting admission criteria, missing and/or incomplete service orders, as well as missing progress notes. 82 | P a g e Quality Management Annual Report FY 2018 Figure 32. BHQR FY 2018 Billing Score Distribution of Reassessed Providers 60% 55% 50% 40% 40% 35% 30% 25% 20% 10% 10% 10% 10% 5% 5% 5% 0% First Review FY 2018 (N = 20; Mean = 75%; Median = 78%) Second Review FY 2018 (N = 20; Mean = 89%; Median = 92%) Service Guidelines: The BHQR Service Guidelines score increased (88 percent in FY 2017 to 90 percent in FY 2018) as well as the reassessed providers and scores. The average score for the reassessed providers at the time of initial review was 87 percent, just slightly below the statewide average, yet improved to 92 percent at second review thereby exceeding the statewide average of 90 percent. Seventeen services were reviewed throughout the reassessments. Reassessments did not include review of Opioid Maintenance, AD Peer Support-Individual, or Community Transition Planning as providers requiring reassessments either did not provide these services or claims were not available in the time period sampled. Most services improved by an average of two to four points with substantial improvement noted in the area of Intensive Family Intervention, which showed a 14-point increase from initial review (74%) to subsequent review (88%). Improvement was noted in progress note documentation. This included "Staff interventions related to the IRP", "Progress towards goals and objectives documented", as well as the "Individual's response to interventions contained in the progress notes". The two exceptions of the above-mentioned improvement were found in Nursing Assessment and Health Services and Psychiatric Treatment; both had a decline from first to second review. 83 | P a g e Quality Management Annual Report FY 2018 Focused Outcome Areas: Providers who were reviewed for a second time in FY 2018 demonstrated improvement in FOA scores, from 88 percent to 93 percent from first to second FY 2018 review. Fifteen providers, or 75 percent, demonstrated improvement in this area, while four had a decline in score, at an average of five points. Of all FOAs reviewed during a reassessment in FY 2018, Safety was the lowest scoring category both at first review (69%) and at second review (78%). Two providers scored a zero in Safety at onset or initial FY 2018 review. One of these providers remained at zero percent for the subsequent review while the other provider increased to a perfect 100 percent score. The Safety FOA remained an area for growth and improvement among not only reassessed providers but also the entire network. Furthermore, Whole Health remained one of the lower scoring FOAs for reassessed providers, both at initial as well as subsequent reviews, 76 percent compared to 83 percent, respectively. Documentation consistently lacked evidence of addressing medical conditions and safeguards of medications. CSU Reassessment Review Findings Two of the 19 CSU providers had a reassessment during FY 2018 compared to four providers in FY 2017. Figure 33 and Figure 34 provides the Overall score result and category scores by each of the two CSU providers at the time of initial FY 2017 review and subsequent FY 2018 reviews. Both CSUs were slightly below the statewide Overall score average by one and four points in FY 2017, respectively. CSU #1 declined by 15 points in their first FY 2018 review with a decrease in all individual categories. This CSU did demonstrate progress from the first FY 2018 review to the second FY 2018 review yet will remain on the frequency schedule until two consecutive overall review scores meet the threshold. 84 | P a g e 100% 80% 60% 40% 20% 0% Quality Management Annual Report FY 2018 Figure 33. CSU #1 Category Score by Reassessment 79% 72% 64% 72% 57% 62% 70% 64% 55% 95% 91% 81% OVERALL Individual Record Service Guidelines Review CSU 1 First FY 2018 Score CSU 1 Second FY 2018 Score Focused Outcome Areas CSU 1 FY 2017 Figure 34 outlines CSU #2 progress and improvement in each of the three categories as well as Overall score results. An initial result of 76 percent Overall is evident with an extremely low FY 2017 score specific to Service Guidelines (57%). While FOAs had a minor decline from FY 2017 to FY 2018. CSU #2 has been removed from the reassessment schedule due to completing two consecutive Overall score goals of 80 percent. Figure 34. CSU #2 Category Score by Reassessment 100% 80% 60% 40% 76% 83% 85% 78% 88% 86% 79% 57% 71% 93% 89% 90% 20% 0% OVERALL Individual Record Service Guidelines Focused Outcome Review Areas CSU 2 First FY 2018 Score CSU 2 Second FY 2018 Score CSU 2 FY 2017 85 | P a g e Quality Management Annual Report FY 2018 Technical Assistance/Exit Conference Upon completion of all BHQRs and CSUQRs, the lead assessor completed a formal exit conference. The exit conference supplied providers with tentative scores, provider strengths, and opportunities for growth. Providers received immediate, preliminary feedback of the BHQR and CSUQR findings at the time of the exit conference. Technical assistance was also provided during the exit conference and throughout the review process pertaining to opportunities for growth and areas of risk. The table below (Table 15) provides details on the technical assistance/exit conferences completed during FY 2018. Table 15. Technical Assistance/Exit Conference Details for FY 2018 Fiscal Year 2018 Quarter 1 Quarter 2 Quarter 3 Quarter 4 Total Year End Total BHQR Exits 41 40 44 31 156 Total Minutes 2362 2360 2660 1922 9304 Total Attendees 397 302 357 326 1382 Total CSUQR Exit Total Minutes Total Attendees 2 6 3 10 21 140 370 210 700 1420 37 32 59 183 311 Total Exits 43 46 47 41 177 In FY 2018, 177 exit conferences were completed. The total number of attendees was 200 more than previous years at over 1,600 (an average of ten individuals per exit conference). The CSUQR average attendance was 15 and BHQR average attendance was nine. Total time of exit conferences was over 10,700 minutes for the year and the average exit length was approximately 60 minutes. The length of exit conferences varies by type of exit conference and may include ACT reviews, which included an additional 15 records per review and details specific to ACT services discussed and reported. Exit conferences and technical assistance will continue throughout FY 2019 as through the provider feedback survey, providers identified them as helpful assistance from the Collaborative. 86 | P a g e Quality Management Annual Report FY 2018 Summary of Findings and Recommendations for Behavioral Health Providers Quality Management continually reassesses processes to ensure we are capturing results and analyzing outcomes that give the best information and truest picture of service quality in Georgia. This includes reviewing feedback received from both providers and individuals, as well as revision of review tools as needed to ensure accurate and measurable reporting of results. June 2018 marked the completion of the third year of the Quality Management contract. Quality assessors completed 156 BHQRs and 21 CSUQRs in FY 2018 with 177 exit conferences conducted across all review types. Exit conferences may have occurred jointly across the BHQR, CSUQR, and ACT reviews depending on whether the provider rendered more than one of those services. Provider Performance Table 16 summarizes the Overall Averages by fiscal year: The Overall score for all three review types met or exceeded 88 percent for the third year: BHQR 88 percent, ACT 89 percent, and CSU 88 percent. Twenty providers were reviewed for a second time in FY 2018 due to initial FY 2017 low scores in Overall, Billing, or both with nearly two-thirds demonstrating improvement across all categories. Although improvements were evident across all review types and categories, Service Guidelines within ACT reviews significantly declined (p<.01) from 88 percent to 84 percent in FY 2018. Table 16. BHQR, ACT, CSUQR Overall Averages by Year Billing Validation FOA BHQR FY 2016 81% 85% Assessment/ Planning 79% Service Guidelines 90% BHQR FY 2017 84% 89% 77% 88% BHQR FY 2018 85% 92% 84% 90% Overall 84% 84% 88% ACT FY 2016 92% 88% 85% 85% 88% 87 | P a g e Quality Management Annual Report FY 2018 Table 16. BHQR, ACT, CSUQR Overall Averages by Year Billing Validation FOA ACT FY 2017 90% 90% Assessment/ Planning 80% Service Guidelines 88% ACT FY 2018 91% 93% 87% 84% Overall 87% 89% CSUQR FY 2016 NA 88% IRR 79% 82% 83% CSUQR FY 2017 NA 91% IRR 80% 87% 86% CSUQR FY 2018 NA 91% IRR 83% 91% 88% Table 17 summarizes the FOAs by review type and year. The Overall score for the FOAs met or exceeded 91 percent in FY 2018 for both BHQR and CSUQRs. Whole Health, continually the lowest score for BHQRs, has continued to improve from FY 2016. However, it remains an area for improvement across all providers. Safety increased for both BHQRs and CSUQRs from FY 2016 to FY 2017, declined in FY 2018 for both review types. In summary, the BHQR and CSUQR Overall FOA scores increased or remained static since inception; however, the subcategory of Safety suffered a decrease from FY 2017 to FY 2018 for both BHQRs (83 to 78%) and CSUQRs (85 to 82%). Table 17. BHQR and CSUQR FOA Scores Focused Outcome BHQR BHQR BHQR CSUQR Areas FY 2016 FY 2017 FY 2018 FY 2016 Whole Health 63% 74% 84% 83% Safety 81% 83% 78% 76% Person Centered Practices 92% 91% 95% 76% Community 87% 93% 96% 92% Choice 92% 96% 97% 98% Rights 90% 93% 93% 97% Overall Score 85% 89% 92% 88% CSUQR FY 2017 89% 85% 88% 94% 98% 92% 91% CSUQR FY 2018 88% 82% 88% 92% 97% 97% 91% Specific to reassessed providers who fell below the minimum threshold for Billing and Overall scores, Table 18 reflects the number of providers who improved from the first FY 2018 review to the second 88 | P a g e Quality Management Annual Report FY 2018 review within the year. While all categories for the statewide BHQR average increased from FY 2017 to FY 2018, reassessed providers all fell below the statewide average at first review, but exceeded the statewide average after second review. Providers who obtained low FY 2017 Overall or Billing scores were scheduled for review and reassessment in FY 2018. While many of the reassessed providers demonstrated improvements on an individual level from first FY 2018 review to second FY 2018 review, those that continued to fall below the established thresholds are to be scheduled for an additional two reviews in FY 2019. Additionally, providers were supplied with additional technical assistance during reviews to assist with performance improvement. Table 18. Reassessment Review Results Number of Providers Who Increased in Score from FY 2018 Review #1 to FY 2018 Review #2 BHQR CSU Category (N = 20) (N = 2) Billing 16 (80%) NA IRR NA 1 (50%) Service Guidelines 12 (60%) 2 (100%) Focused Outcome Areas 15 (75%) 2 (100%) Assessment & Planning 17 (85%) NA Overall Scores 17 (85%) 2 (100%) FY 2018 BH Accomplishments Throughout the year, the Quality Management Department partners with DBHDD Quality Improvement, DBHDD subject matter experts, and others to discuss and review findings, concerns, and areas of need across the provider network based on monthly, quarterly, and annual review results. It is through this partnership the Collaborative derives recommendations of revisions to the review process. Below is a listing of accomplishments achieved by the Collaborative that have occurred throughout the 2018 fiscal year. BHQR and CSUQR tool revisions for clarity, alignment, and formalized criteria including; 89 | P a g e Quality Management Annual Report FY 2018 o Assessment and Planning discharge criteria and diagnosis verification o Defining requirements for what constitutes a medical assessment and a behavioral health assessment. o Addition of Opioid Maintenance Therapy as a measurable service o Revision Staff Interview questions Quality Training Series o Safety Planning 101 o Intensive Family Intervention (IFI) Training o Residential Services Documentation o Documentation 101 o Progress Notes 101 o DBHDD Provider Manual: Your Guide to Success On July 1, 2018, CSUQR process and tools were incorporated into the electronic system (Delmarva Quality Management System) to ensure accuracy, standardization, and formality through electronic reporting. A new quality-of-care process was piloted to collect, monitor, and report various issues to DBHDD specific to providers' physical environments, staffing, and quality of services which are outside of the scope of the quality reviews. To standardize and mirror the sampling processes with IDD reviews, Quality Management grouped BH providers into three categories (Small, Medium, Large) based on the number of unique individuals served in the six months preceding a review. Subsequent revisions to BHQR sample sizes continue to reflect these provider groupings. To increase engagement of individuals with complex care needs and reduce readmission rates, Quality Management educated Crisis Stabilization Unit staff about the benefits of the Collaborative's Care Coordination team. Increased billing threshold for reassessed providers from 70 to 80 percent based on annual results and network success. 90 | P a g e Quality Management Annual Report FY 2018 Annual review of all BHQR and CSUQR tools in preparation for new fiscal year. Quality Management revised and updated the quality section on the Collaborative's website to increase ease of use and access to resources available. BH Systems Strengths and Recommendations for Improvement In FY 2018, both BHQR and CSUQR scores increased, with both review types statewide scores resulting in Overall score of 88 percent. Findings from both BHQR and CSUQR show a result equal to or exceeding 90 percent for both Service Guidelines (BHQR: 90%, CSUQR 91%), and FOAs (BHQR: 92%; CSUQR 91%). When comparing FY 2018 scores to FY 2017, there were specific areas that show statistically significant improvement (p = <.01). These included the following: The BHQR Assessment and Planning, a category comprised of 10 indicators, had significant increases from the previous year with the exception of the individual meeting admission criteria. However, this specific indicator had the same result of 97 percent for both fiscal years. While sixteen of the services reviewed with the BHQR Service Guidelines demonstrated improvement from FY 2017 to FY 2018, both Individual Counseling and Psychosocial Rehabilitation Program has consistently documented significant improvement (p < .01) year to year since FY 2016. For FY 2018, of the three BHQR FOA Safety indicators, the one most often scored "no" was "Individuals (or their legal guardians) signed medication consent forms along with the prescriber". However, results of this indicator have trended upward since contract inception and increased by ten points from FY 2017 to a result of 68 percent for FY 2018. All BHQR Person Centered Practice FOA indicators improved significantly demonstrating the individual is receiving individualized services while being an active participant in the planning, modification, or receiving of services. Additionally, records more consistently documented that a plan reassessment had been completed when needed in FY 2018 (87%) than in FY 2017 (82%). 91 | P a g e Quality Management Annual Report FY 2018 FY 2018 demonstrated even higher levels of documentation specific to the BHQR Choice FOA than in FY 2017. While results of all indicators exceeded 90 percent in FY 2017 for this category, increases in "Documentation of the providers supplying alternatives when barriers had been identified" rose from 92 percent to 95 percent. Documentation reviewed during the BHQR clearly demonstrated evidence that the "Individual had been provided supports and service options" (98%) as well as "Individuals had their known preferences and differences followed to the extent possible" by the service delivery provider (98%). Specific to the CSUQR IRR, all areas demonstrated significant improvement in scoring from the previous year with the exception of Course of Stay and Treatment Planning. Furthermore, when analyzing C&A records within the CSUQR, several FOA indicators met or exceeded 90 percent with some reaching 100 percent. There was signficant improvement (p < .01) in documentation reviewed from FY 2017 to FY 2018 that Individuals at the CSU had rights information readily available written in a language accessible to the individual (FY 2018 98%). In CSUQR records, the discharge summary/note included the necessary criteria improved from the previous year fiscal year by 30 points (92%). Quality Management found the following additional strengths in the system: Through ongoing technical assistance and assessment of practices that support whole-person treatment, reviewed documentation indicated both behavioral health, as well as crisis stabilization unit providers, incorporated techniques to address the whole health and wellness of individuals more frequently in FY 2018. While this occurred in both BHQR and CSUQR scores, significant improvement in the BHQR indicators within Assessment and Planning and the FOA of Whole Health, demonstrated increased awareness, assessment, and service planning by providers with individuals. The BHQR Service Guidelines was the highest scoring category of the four in FY 2016, with an average of 90 percent. Although it declined slightly in 2017 by two points, it has since returned to a 90 percent result. Moreover, nearly three quarters of the Service Guidelines 92 | P a g e Quality Management Annual Report FY 2018 scores within the BHQR met or exceeded 90 percent and nine reviews received a perfect score; three of which were providers reviewed for the first time. All child and adolescent CSU records demonstrated that the individual was included in the modification of their plans, providers honored youths' preferences and differences, and providers offered individuals options of supports and services. Although improved each fiscal year, the billing category in the BHQR remains second lowest at 85 percent. Thus, keeping in line with standard requirements, DBHDD provider manual guidelines, and KPI metrics, it is recommended to increase the threshold for reassessment to 90 percent from 85 percent for both Overall and Billing score for FY 2019. While several providers remained consistent or demonstrated improvements in their results from FY 2017 to FY 2018, opportunities for improvement remain. Several targeted training efforts have occurred over FY 2018 to address low scoring indicators and or service specific trainings resulting from BHQR or CSUQRs scoring and reviews. Additionally, Quality Management has devised several recommendations identified below based on results supplied within this annual report. Recommendation I: Continue annual review and evaluation of existing BHQR and CSUQR tools and requirements, as well as inclusion of specific reasons not met, to align with current state requirements and DBHDD recommendations in accordance with the DBHDD Provider Manual. Recommendation II: BHQR services additions or changes to include: Addition of Substance Abuse Intensive Outpatient Program (SAIOP) to be inline with Provider Manual Peer Support Whole Health & Wellness divided to reflect Individual versus Group Medication Assisted Treatment as a measurable service Recommendation III: Continue sample size analysis to ensure records sampled at the individual provider level are representative of the provider and statewide utilization levels. Recommendation IV: Continue the Quality Training Series in FY 2019 based on analysis of FY 2018 data, for all BH and CSU agencies to include, but not limited to, the following: 93 | P a g e Quality Management Annual Report FY 2018 Utilization of BHQR & CSUQR results at the agency level Crisis and Safety planning for Crisis Stabilization Units Documentation specific to paraprofessional staff Safe Handling and storage of Medications (CSUQR specific) Recommendation V: Continue inclusion of Immediate Actions/Recommendations at BHQR and CSUQR exit conferences to provide immediate feedback to providers. Recommendation VI: Increase Overall score and Billing score thresholds from 80 to 90 percent to align with DBHDD Key Performance Indicators for FY 2019. Recommendation VII: Complete analysis of Intensive Family Intervention (IFI) services to determine specific areas for improvement. Recommendation VIII: Initiate baseline data collection conducted at each BHQR regarding Overall Programmatic indicators, such as services provided at Medicaid approved sites, agency-wide staffing requirements and policies/procedures. Recommendation IX: Include a scored indicator regarding whether or not CSU discharges are submitted within 48 hours to align with DBHDD requirements. Recommendation X: As IDD QEPR tools are revised for FY 2020, continued alignment and collaboration between indicators asked for both BHQR and QEPR. 94 | P a g e Quality Management Annual Report FY 2018 Section 4: Intellectual and Developmental Disabilities Background Person Centered Reviews (PCR) and Quality Enhancement Provider Reviews (QEPR) assess the extent to which individuals with intellectual and developmental disabilities are supported with the services they receive and achieve outcomes important to them, and to evaluate provider systems.9 The purpose of the PCR is to assess the quality of life as well as the effectiveness of and the satisfaction individuals receiving services have with the service delivery system. The NCI Adult InPerson Survey (formally the Adult Consumer Survey) is conducted as part of the PCR process. The data collected for the survey is entered in the ODESA system [web-based application developed and maintained by Human Services Research Institute (HSRI)]. HSRI analyzes and generates annual reports comparing Georgia's results to other participating states and the national average. The purpose of the QEPR is to review providers' systems and practices to ensure they meet requirements set forth by the Medicaid waiver and DBHDD, and to evaluate the effectiveness of their service delivery system. Follow-up review activities provide technical assistance to help providers improve service delivery systems through Quality Technical Assistance Consultation (QTAC). Quality assessors use various tools to collect data from interviews, observations, and record reviews to compile a well-rounded picture of individuals receiving services: their circle of supports, how involved they are in the decisions and plans developed for them, as well as the quality of services provided. Individuals sampled for the PCR or QEPR participate in the Individual Interview (II) and Individual Service Plan Quality Assurance (ISP QA) Checklist. Both review processes include a Provider Record Review (PRR) and the Developmental Disability Service Specific (DDSS) requirements for each service received, a Staff Interview (SI) with a sample of direct support providers, and on-site observation of day or residential programs. 9 Please refer to the following link to access a full description of the review process and review tools. The Georgia Collaborative: Quality Management IDD 95 | P a g e Quality Management Annual Report FY 2018 During the PCR, the Support Coordinator Record Review (SCRR) and Support Coordinator Interview (SCI) tools are completed for the Support Coordinator working with the individual receiving services. During the QEPR, each provider organization receives one administrative review to monitor compliance with requirements through the Qualifications and Training (Q&T) component of the review. The Q&T includes review of a sample of personnel/staff records to determine if staff has the necessary qualifications specific to services rendered, and whether required training was completed within specified timeframes. In this section of the report, each review tool presents results for both the PCR and QEPR. Aggregate scores of IDD quality reviews are a weighted average, based on the total number of standards scored [total met / (total met + total not met)]. Except for the Q&T and DDSS tools, indicators within each tool are grouped into six FOAs, which are areas of the individual's life important to achieve and maintain: Whole Health--individuals receiving services are healthy, aware of their health-related needs, and direct their own health care regimen Safety-- individuals receiving services are safe in their home and work environments and in their communities; they understand or are learning how to self-preserve in all environments Person Centered Practices--supports and services are provided based on personal preferences and direction Community Life--individuals receiving services are actively participating and developing social roles in their communities as desired Choice--information needed to make informed choices on life decisions, such as where to live, where to work, and which supports, services and providers to use, is available Rights--rights are upheld and information and education is provided to ensure understanding of rights 96 | P a g e Quality Management Annual Report FY 2018 Sampling Method Person-Centered Review (PCR) The PCR sample used a random sample of 484 eligible adults, age 18 and over, who had not received a PCR during the previous year and were receiving services reviewed through this contract. The PCR sample was stratified and sampled proportionate to each of the six DBHDD regions; therefore, the number of PCRs per region are proportionate to the number of individuals receiving services within the region. Table 19 shows the number and percent for the population of individuals receiving services across the state, the sampling frame (eligible for a PCR this year), as well as the number of PCRs completed within each region. Region 1 2 3 4 5 6 Total Table 19. FY 2018 PCR Sample by Region Population Population eligible for a PCR10 PCR Sample N % N % N % 2,406 19.9% 1,489 19.6% 89 18.4% 2,090 17.3% 1,154 15.2% 78 16.1% 3,007 24.9% 2,241 29.5% 140 28.9% 1,353 11.2% 987 13.0% 63 13.0% 1,560 12.9% 790 10.4% 56 11.6% 1,674 13.8% 934 12.3% 58 12.0% 12,090 100.0% 7,595 100.0% 484 100% Quality Enhancement Provider Review (QEPR) Those providers not reviewed in FY 2016 or FY 2017 were eligible for a QEPR in FY 2018.11 QEPReligible providers rendering services to anyone selected for the PCR sample were automatically included in the QEPR sample. Additional providers were randomly selected from the remaining QEPR-eligible providers until a sample size of 93 was met. Finally, two randomly selected Support Coordinator (SC) Agencies and four providers selected by DBHDD completed the QEPR sample of 99 10 Providers who participated in a QEPR in FY17 and individuals who only received services from those providers were excluded. 11 Except for Community Service Boards (CSBs), which are eligible for review every other year. 97 | P a g e Quality Management Annual Report FY 2018 providers. Quality Management conducted an additional QEPR of one crisis services provider, to bring the total to 100 QEPRs for the year. Based on the number of eligible individuals served, providers were stratified into three categories by size: "Small," "Medium," and "Large." The number within each size category is shown by year in Table 20. Table 20. QEPR Provider Sample by Size Number Provider Size FY 2016 FY 2017 Small (caseload 30) 46 52 Medium (30 < caseload < 100) 36 19 Large (caseload 100) 16 24 Support Coordination Agency Crisis Stabilization Unit 1 1 1 4 Total 100 100 FY 2018 72 12 13 2 1 100 Review Processes PCR and QEPR The focus of the PCR is on quality of life and quality of services received. The focus of the QEPR is on the provider's overall practices, quality of services offered to all individuals served by the provider, and level of compliance with Medicaid waiver and state requirements. Both the PCR and QEPR use the Individual Observation Staff Assessment (IOSA), which includes an interview with individuals receiving services, their staff, and on-site observation(s) (OBS) at residential and day programs, as applicable. In addition to the IOSA, the PCR and QEPR include an evaluation of the Individual Service Plan using a quality assurance checklist (ISP QA), a review of the provider's records, including documentation of service delivery requirements using the Developmental Disabilities Service Specific (DDSS) review tool. The number of Provider Record Reviews (PRR) and DDSS reviews completed for each process depends upon the number of services received by the individuals in the sample. While the PCR and QEPR share most of the same tools, there are a few exceptions. The PCR includes an interview with the Support Coordinator and a review of the record in the Consumer Information System (CIS) maintained by the Support Coordinator for the individual. The QEPR has an additional 98 | P a g e Quality Management Annual Report FY 2018 review tool, Qualifications and Training, which is used to review a sample of records from all staff ensuring required training and other state requirements are current and documented (e.g., background screenings, level of education). The total number of records reviewed or interviews completed in FY 2018 for the PCR and QEPR is listed in Table 21, for each review tool. Table 21. FY 2018 Number of Reviews by Tool and Review Type Review Tool PCR QEPR Total IOSA - Individual Interview (II) 484 436 920 IOSA Observation (OBS) 353 285 638 IOSA - Staff Interview (SI) 443 316 759 ISPQA Checklist 484 440 924 Support Coordinator Interview (SCI) 484 8 492 Support Coordinator Record Review (SCRR) 484 40 524 Provider Record Review (PRR) 856 561 1,417 Staff Qualifications and Training (Q&T) N/A 780 780 DDSS - Behavioral Supports Consultation 7 7 14 DDSS - Community Access (Group) 227 327 554 DDSS - Community Access (Individual) 5 126 131 DDSS - Community Living Support 49 141 190 DDSS - Community Residential Alternative 107 298 405 DDSS - Crisis 0 1 1 DDSS - Occupational Therapy 0 26 26 DDSS - Physical Therapy 0 6 6 DDSS - Prevocational 32 54 86 DDSS - Respite 4 14 18 DDSS - Support Coordination 484 43 527 DDSS - Supported Employment DDSS - Speech Therapy DDSS - Transportation Total Number of PCRs/QEPRs 13 49 62 0 6 6 1 3 4 484 100 584 When a PCR is completed, a report is given to the provider that includes the strengths of the support team. It includes an evaluation of the supports and services provided and recommendations for the individual interviewed and the support team (including the Support Coordinator, provider, and 99 | P a g e Quality Management Annual Report FY 2018 family). A provider who participates in a QEPR receives a comprehensive report that identifies strengths of the service delivery systems, recommendations for improvement, and several performance scores. These scores include the Overall score, Q&T score, and DDSS score. Quality Technical Assistance Consultation (QTAC) The QTAC is an additional review that is conducted 90 days after completion of the QEPR. This review is based on any service concerns identified during the PCR or QEPR, or if the provider requests technical assistance. Using findings from the QEPR, technical assistance is provided to support providers and to offer suggestions and guidance to help improve their service delivery systems. The process uses a consultative approach to address specific issues and concerns related to someone receiving services or systems and practices that need improvement. The QTAC supplements the PCR and QEPR processes by affording contracted providers the opportunity to solicit technical assistance for specific needs within the service delivery milieu. Person Centered Review PCR Scores by Tool Figure 35 shows the average score for each tool used during the PCR, comparing FY 2016 through FY 2018. Findings each year show a similar pattern, with scores for the Individual Interview, Staff Interview and Support Coordinator Interview higher than scores for provider or Support Coordinator documentation (record reviews). Scores across all tools decreased from FY 2016 to FY 2017, and in FY 2018, the scores increased on all of the tools. The FY 2018 scores for Observations, PRRs, and SCRRs were significantly higher than FY 2017 (p < .01); however, it should be noted the percentages are calculated using results from all indicators scored within each FOA and the denominators were quite large, increasing the likelihood of statistical significance. 100 | P a g e Quality Management Annual Report FY 2018 Figure 35. PCR Scores by Tool and Year 100% 90% 80% 70% 95.1% 98.4%96.8%98.2% 91.9%92.2% 96.4% 94.3% 95.6% 79.0% 74.8% 70.2% 79.9% 77.4% 73.7% 90.8% 83.5% 83.3% 60% 50% Individual Observation Staff Interview Provider Record Support Support Interview Review Coordinator Coordinator Record Review Interview FY 2016 (N = 484) FY 2017 (N = 481) FY 2018 (N = 484) Several indicators may have driven the increases in the Observation, PRR, and SCRR tools. Indicators listed below were included if the sample size each year was at least 50 and the increase from FY 2017 to FY 2018 was at least 10 percentage points and significant at p < .01: Observations Opportunities were used to provide health and rights education to the individual Staff took advantage of opportunities to provide experiences to support informed choice Staff took advantage of opportunities to provide education to the individual on responsibilities regarding rights 101 | P a g e Quality Management Annual Report FY 2018 Provider Record Review Documentation included the following copies of the individual's current preventative healthcare reports based upon gender, age and need (if the individual refused the treatment, this is documented): PSA test, bone density, mammogram Documentation included the following copies of the individual's specialty healthcare reports based upon diagnoses, assessments and referrals (if the individual refused the treatment, this is documented): Neurological evaluation/supports, wheelchair evaluation/supports, GI evaluation/supports, and psychiatric evaluation/supports Documentation demonstrated the results of monitoring (DDP and/or Behavior Specialist) and analysis of the Safety Plan or Positive Behavior Support Plan, and staff had been trained on the individual's Positive Behavior Support Plan or Safety Plan for challenging behaviors Documentation demonstrated education is provided to the individual and family (if approved by the individual) on all prescribed medications, and to the individual on risks and benefits of medication Critical incident reports were completed, and follow-up/remediation occurred When a Positive Behavior Support Plan was used to reduce challenging behaviors, there must be evidence the following had been addressed: The provider monitored plans for reviews, analyzed trends, and summarized the effectiveness of the plan and termination criteria Ongoing evidence of identifying, addressing, and seeking prevention of abuse, neglect, and exploitation was documented Documentation showed evidence of exploring ways for the individual to participate in community activities and types of employment the same as other citizens Documentation showed evidence that meaningful choices were offered to individuals for providers and services, and the manner in which services were provided Documentation demonstrated evidence of ongoing rights education and that individual rights and responsibilities were written in an understandable language/format Documentation contained records of day to day living expense agreements 102 | P a g e Quality Management Annual Report FY 2018 Support Coordinator Record Review Emergency preparedness plans were in place HIPAA Privacy and Security Rules, as outlined at 45 CFR Parts 160 and 164, were specifically reviewed with individuals Exercising meaningful choices regarding the manner by which services were provided was reflected in the documentation PCR Scores by Focused Outcome Area (FOA) All the PCR tools (II, SI, OBS, PRR, SCI, and SCRR) are designed to measure the six FOAs, with many indicators within each FOA. Results for each FOA, based on all six tools by fiscal year are shown in Figure 36. While scores for the FOAs had shown a decrease from the first to the second year of the contract, in FY 2018 (third contract year), scores have increased on each FOA. The increases range from 1.3 percentage points for Safety to 5.8 points in Community Life. These all represent a statistically significant increase; however, the denominators are quite large and small differences may reflect significance at p < .01. 103 | P a g e Quality Management Annual Report FY 2018 100% 90% 80% Figure 36. PCR Scores by FOA and Year 93.6% 87.4% 84.7% 96.5% 91.6% 90.3% 89.1% 84.8% 82.6% 80.6% 76.9% 90.7% 95.4% 92.0% 90.2% 82.3% 78.7% 71.1% 70% 60% 50% Whole Health Safety Person Centered Practices Community Life Choice Rights FY 2016 (N = 484) FY 2017 (N = 481) FY 2018 (N = 484) PCR Scores by Tool and Focused Outcome Area (FOA) In this section, PCR results for FY 2018 are presented by FOA and tool (Figure 37). Findings for the review components varied across each FOA and indicated the following: Similar to FY 2017, documentation reviewed within the provider record review was the lowest scoring component across all FOAs, except for Rights (84.4%). Support Coordinators' documentation was the lowest scoring component in Rights (57.3%), for which the interview with the Support Coordinator was also relatively low when compared to other tools (78.5%). FY 2018 Staff and Individual Interview findings were at least 86 percent or higher, across all FOAs. 104 | P a g e Quality Management Annual Report FY 2018 Figure 37. FY 2018 PCR Scores by Tool and FOA Whole Health 88.2% 94.4% 86.2% 97.7% 95.5% 75.7% Community Life 86.6% 89.0% 79.6% 95.0% 67.6% 67.0% Safety 92.3% 97.1% 81.7% 99.9% 97.2% 80.5% Choice 96.3% 93.8% 85.3% 97.9% 70.6% 58.5% Person Centered Practices 0% 25% Individual Interview (N = 484) Observation (N = 353) 87.9% 95.3% 84.8% 95.5% 80.3% 77.6% Rights 57.3% 97.6% 98.4% 78.5% 99.0% 84.4% 50% 75% 100% 0% 25% 50% 75% 100% Staff Interview (N = 443) Support Coordinator Record Review (N = 484) Support Coordinator Interview (N = 484) Provider Record Review (N = 856) FOAs by Demographics For this section, analysis was completed comparing FOA results across regions, residential settings, and funding source. Results for FOA by region, residence and funding source are presented (Figures 38 through 40). To enhance the readability of the side-by side demographics, the start point of the graphs' axis is 50 percent. Please note that some of the categories have a relatively small sample. The sample for host homes is 37 and for state funded services (SFS) is 52; however, some findings may be worth further analysis. Findings indicate the following: With the exception of Community Life, Region 1 showed the lowest scores of the FOAs, particularly in Whole Health and Person Centered Practices, for which the score was 10 points lower than the highest scoring region. 105 | P a g e Quality Management Annual Report FY 2018 Individuals receiving services and living with a parent scored lower in Whole Health than individuals in any other residential setting (p < .01). Individuals receiving services through the COMP waiver were more likely to have Whole Health indicators met than for services rendered through NOW or SFS (p < .01). Individuals receiving services through SFS were less likely to have Safety indicators met than services provider through the COMP waiver or NOW waiver (p < .01). Figure 38. FY 2018 PCR Scores by FOA and Region Whole Health 82.5% 92.0% 86.4% 89.7% 88.2% 86.8% Community Life 75.9% 79.2% 77.6% 78.0% 75.6% 73.9% Safety 88.5% 94.8% 89.9% 93.9% 93.4% 91.4% Choice 78.7% 85.5% 81.4% 85.7% 83.4% 80.6% Person Centered Practices 79.2% 89.7% 82.4% 89.4% 87.9% 84.1% 50% 60% 70% 80% 90% 100% Rights 88.8% 94.2% 91.0% 94.0% 95.1% 91.3% 50% 60% 70% 80% 90% 100% Region 1 (n = 89) Region 4 (n = 63) Region 2 (n = 78) Region 5 (n = 56) Region 3 (n = 140) Region 6 (n = 58) 106 | P a g e Quality Management Annual Report FY 2018 Figure 39. FY 2018 PCR Scores by FOA and Residential Setting Whole Health 89.6% 89.8% 88.1% 84.5% Community Life 76.2% 80.3% 79.5% 76.6% Safety 92.6% 93.2% 92.2% 90.2% Person Centered Practices 84.9% 87.3% 87.8% 83.7% 50% 60% 70% 80% 90% 100% Group Home (n = 161) Own Place (n = 48) Choice 82.2% 83.6% 85.0% 81.9% Rights 91.4% 92.4% 92.2% 92.7% 50% 60% 70% 80% 90% 100% Host Home (n = 37) With Parents (n = 236) Figure 40. FY 2018 PCR Scores by FOA and Funding Source Whole Health 84.7% 88.4% 85.0% Community Life 78.6% 76.5% 76.5% Safety 91.2% 92.2% 87.8% Person Centered Practices 85.4% 84.8% 83.4% 50% 60% 70% 80% 90% 100% Choice 83.3% 82.3% 80.4% Rights 93.0% 91.9% 91.4% 50% 60% 70% 80% 90% 100% NOW (n = 113) COMP (n = 319) SFS (n = 52) 107 | P a g e Quality Management Annual Report FY 2018 Opportunities for Growth by FOA Each PCR tool is organized around the FOAs, Figures 36 through 40, and each FOA uses several standards, or indicators, to measure how well the area is being addressed. As indicated in this report, most findings showed relatively high scores across all perspectives of the PCR, i.e., interviews, on-site observations and documentation areas all averaging over 75 percent; and overall improvement shown for each FOA. However, there are specific areas within each FOA where indicator level scoring was relatively low, has remained low over the past two or three years, or has decreased since FY 2017. These are presented in this section by FOA. The p values for difference of proportions between the FY 2017 and FY 2018 results are included. Statistical significance for this report has been determined to be p < .01 (highlighted in blue), indicating the difference in the scores is statistically significant. Whole Health The average Whole Health score in FY 2018 was 87.4 percent, an increase from 84.7 percent in FY 2017. Close to 97 percent of Support Coordinators monitor the health and welfare of individuals receiving services, which is an increase from 93.5 percent in FY 2017. Furthermore, almost all individuals interviewed had supports and service to address whole health needs (99.4%) and access to medications (99.8%). The following table (Table 22) indicates a continued decline in several areas, based on results from the Individual Interview. Each year since 2016, individuals receiving services have been less likely to be aware of what medications they are taking and why, the side effects of the medications, and what the medication should look like. They were also less likely each year to be aware of their diagnosis. Staff support for the individual to learn about medications and their side effects remained relatively low in FY 2018, as did provider documentation of a Pap test/pelvic exam and hearing evaluations. Documentation indicated an increase in the degree to which providers offer education on medication risks and side effects, and on all prescribed medications. These are, however, still among the lower scoring PRR Whole Health standards in FY 2018. In addition to low scoring indicators listed in the table, one area with a higher score in FY 2018 has shown a significant decline since FY 2017 (p 108 | P a g e Quality Management Annual Report FY 2018 < .01). Although still scoring relatively high (90.1%), individuals receiving services were less likely to self-manage health according to preferences (down 7.5 points). Table 22. Low Scoring Whole Health Indicators (PCR) Individual Interview Individual receiving services was aware of: The side effects of the medications Why medications are prescribed What medications he/she is taking The medication's color, shape, when it is taken His or her diagnoses Staff Interview Staff could describe: How to support the individual to learn about medications The side effects of medications taken Provider Record Review Documentation demonstrated how/include: Copies of the individual's current preventative healthcare reports for a Pap test or Pelvic exam Copies of the individual's current preventative healthcare reports for a hearing evaluation or supports Individuals and families are provided education on all prescribed medications Education is provided to the individual on the risks and side effects of the medication FY 2016 FY 2017 74.3% (n = 350) 86.0% (n = 351) 78.3% (n = 428) 84.9% (n = 351) 81.9% (n = 481) 52.4% (n = 410) 68.5% (n = 410) 68.3% (n = 435) 80.4% (n = 408) 71.9% (n = 473) 81.0% (n = 357) 86.3% (n = 388) 67.7% (n = 195) 78.7% (n = 211) 48.0% 31.7% (n = 200) (n = 316) 48.8% 38.3% (n = 447) (n = 862) 56.6% (n = 327) 60.5% (n = 332) 25.7% (n = 723) 33.9% (n = 610) FY 2018 43.6% (n = 433) 62.5% (n = 435) 63.5% (n = 458) 72.6% (n = 431) 64.1% (n = 482) 68.4% (n = 307) 75.4% (n = 353) 33.1% (n = 356) 37.5% (n = 814) 48.3% (n = 621) 52.1% (n = 511) p value FY 17/18 p = .01 p = .07 p = .01 p = .01 p = .01 p = .87 p = .24 p = .68 p = .73 p = 0 p = 0 109 | P a g e Quality Management Annual Report FY 2018 Safety Safety scores had an average of 91.6 percent in FY 2018, compared to 90.3 percent in FY 2017. Almost every provider and Support Coordinator reviewed (> 90%) had emergency preparedness plans in place, had documented specific contact information, and documented risks or safety issues for the individual. Support Coordinators advocated to ensure follow-up occurred for safety (97.0%) or critical incidents (96.6). Almost every individual interviewed felt safe in all environments and free from all types of abuse (> 99%). Table 23 includes the low scoring Safety indicators for FY 2018, indicating some had significantly decreased (p < .01) since FY 2017. Findings suggest a decrease each year in an understanding of what to do if support staff or other supports become incapacitated. At the same time, many staff are not providing education to individuals receiving services on how to use the crisis hotline and most providers were not documenting how they address abuse, neglect and exploitation or offer education on how individuals can self-preserve. Support Coordinators were not always aware of restrictive interventions, needed behavior or crisis plans, and some triggers related to behavior health. Table 23. Low Scoring Safety Indicators (PCR) Individual Interview Individual is aware of or recognizes what constitutes exploitation Individual is aware of how to respond in an emergency/safety situation if supports are incapacitated Staff Interview Staff is providing education on how to use the crisis hotline Provider Record Review Documentation demonstrated: FY 2016 FY 2017 FY 2018 90.4% 76.6% 70.5% (n = 480) (n = 480) (n = 482) 87.3% 78.7% 71.5% (n = 473) (n = 468) (n = 481) 88.4% 52.8% 59.9% (n = 392) (n = 91) (n = 352) p value FY 17/18 p = .03 p = .01 p =.30 110 | P a g e Quality Management Annual Report FY 2018 Table 23. Low Scoring Safety Indicators (PCR) Ongoing evidence of identifying, addressing, and seeking prevention of abuse, neglect, and exploitation How education is offered to self-preserve or develop effective resiliency skills according to the individual's learning style Evidence of a written order by the physician including the rationale and instructions for the use of adaptive supportive devices or medical protective equipment Support Coordinator Interview Support Coordinator was aware: Of interventions/plans in place Of the individualized techniques for following the behavior plans: Behavior Emergency Safety Of specific triggers related to behavior health issues: Decomposition Relapse Behavioral Issues FY 2016 53.7% (n = 499) 52.5% (n = 549) 97.8% (n = 93) 87.8% (n = 181) 79.5% (n = 132) 92.2% (n = 258) 91.6% (n = 237) 87.4% (n = 174) 88.9% (n = 180) 88.8% (n = 180) FY 2017 22.9% (n = 893) 41.1% (n = 893) 72.3% (n = 101) 47.8% (n = 92) 55.8% (n = 104) 70.6% (n = 163) 71.5% (n = 172) 58.2% (n = 117) 60.5% (n = 114) 74.6% (n = 189) FY 2018 39.5% (n = 838) 41.5% (n = 843) 66.1% (n = 165) 47.7% (n = 107) 53.3% (n = 122) 66.5% (n = 179) 69.3% (n = 202) 57.3% n = 131) 60.6% (n = 142) 69.7% (n = 188) p value FY 17/18 p = 0 p = .86 p = .29 p = .39 p = .71 p = .42 p = .48 p = .89 p = .91 p = .01 111 | P a g e Quality Management Annual Report FY 2018 Person Centered Practices Person Centered Practices showed an average score of 84.8 percent, an increase from 82.6 percent in FY 2017. Almost all Support Coordinators had the ISP in the record (98.6%). Support Coordinators (96.9%) and service providers (98.5%) documented how supports/services changed when needed. Findings each year (see Table 24) suggest a steady decline in the percent of individuals receiving services who understood how to make progress on their goals or how goals could be changed, and many staff, providers, and Support Coordinators were not reviewing progress on goals with the person. Many individuals receiving services were unable to describe something new experienced or learned in the past six months. Table 24. Low Scoring Person Centered Practices Indicators (PCR) Individual Interview FY 2016 FY 2017 FY 2018 p value FY 17/18 Individual determines when progress on goals is achieved or goals are met 87.7% 81.8% 75.3% (n = 480) (n = 479) (n = 482) p = .01 Individual describes progress on goals 90.4% 84.8% 78.8% (n = 478) (n = 474) (n = 482) p = .02 The individual understands changes can be made to goals, supports, and services 97.3% 89.6% 81.1% (n = 479) (n = 479) (n = 482) p = 0 In the past 6 months, the Individual was able to identify something: New or experienced 70.4% 60.3% 62.6% (n = 479) (n = 471) (n = 479) p = .46 Had been learned 78.8% 65.8% 67.2% (n = 480) (n = 474) (n = 478) p = .46 Staff Interview Staff formally reviews progress on goals/objectives with the individual 75.8% 72.2% 68.4% (n = 826) (n = 302) (n = 434) p = .28 Provider Record Review Documentation reflected: The individual's talents 48.9% 38.9% (n = 552) (n = 894) NA NA The individual's strengths 57.0% 48.9% (n = 553) (n = 894) NA NA 112 | P a g e Quality Management Annual Report FY 2018 Table 24. Low Scoring Person Centered Practices Indicators (PCR) FY 2016 FY 2017 FY 2018 p value FY 17/18 The individual's talents/strengths12 NA NA 44.2% (n = 847) NA The individual's hopes and dreams 53.3% 42.0% 31.5% (n = 552) (n = 893) (n = 847) p = 0 Documentation demonstrates review of progress and benefit of goals occurs regularly with the individual. Progress notes or learning logs describe progress toward goals including the individual's response to an intervention or activity, based on data Support Coordinator Record Review Documentation demonstrated: The individual's talents The individual's strengths Progress notes describe progress toward goals including response to the intervention or activity, based on data 51.4% (n = 552) 67.7% (n = 551) 49.2% (n = 480) 56.4% (n = 479) 63.9% (n = 483) 40.4% (n = 373) 72.9% (n = 893) 28.8% (n = 479) 41.0% (n = 480) 57.8% (n = 481) 44.0% (n = 848) 68.1% (n = 850) 35.3% (n = 484) 44.2% (n = 484) 62.7% (n = 483) p = .31 p = 0 p = .03 p = .32 p = .05 Community Life Community Life, interaction with and integration in the surrounding community, showed the lowest average score among the FOAs for each year, 80.6 percent (FY 2016), 71.1 percent (FY 2017) and 76.9 percent (FY 2018) respectively. Most providers (96.5%) showed evidence that community services were provided in the community, instead of being provided at the day service location. Almost all individuals interviewed indicated they go out in the community (99.2%) and are able to choose the type of activity in which to participate (96.2%). Information from the face-to-face interviews are shown in Table 25. Findings indicated from the three different interview perspectives (individual, support coordinator, staff) that, individuals 12 The two separate indicators were combined into one for FY 2018. 113 | P a g e Quality Management Annual Report FY 2018 receiving services were often not developing, or supported to develop, valued social roles or exploring community employment options. However, while still among the lowest scoring interview indicators, improvement has been shown in several of these areas from all three perspectives. Table 25. Low Scoring Community Life Interview Indicators (PCR) FY 2016 FY 2017 FY 2018 p value FY 17/18 Individual Interview The individual was: Exposed to new community activities ( in the past 6 months) 71.4% 63.2% 70.1% (n = 472) (n = 473) (n = 478) p = .02 Actively pursuing preferences related to goals of employment (not a readiness model) 82.4% 70.4% 71.5% (n = 335) (n = 338) (n = 312) p = .76 Provided opportunities to develop new social roles 75.4% 69.4% 74.8% (n = 476) (n = 477) (n = 477) p = .06 Provided opportunities to learn about social 81.1% 71.5% 75.8% roles in the community (n = 476) (n = 478) (n = 476) p = .13 Staff Interview Staff was able to describe how the individual is provided opportunities to: Develop community employment 71.4% 58.3% 60.7% (n = 795) (n = 235) (n = 338) p = .57 Develop new social roles 59.3% 60.4% 70.6% (n = 791) (n = 298) (n = 429) p = 0 Support Coordinator Interview The Support Coordinator was: Aware of the individual's new community experiences 73.0% 62.0% 68.9% (n = 467) (n = 471) (n = 476) p = .03 Aware of how the individual is provided opportunities to develop new valued social roles in the community 71.2% (n = 463) 66.9% (n = 475) 70.7% (n = 481) p = .01 Able to define or explain how the individual 79.5% 72.6% 79.3% is supported to uphold valued social roles (n = 472) (n = 475) (n = 484) p = .02 Documentation from Support Coordinator and Provider Record Reviews indicated that several areas showed improvement since FY 2017; however, these areas remain among the lowest scoring 114 | P a g e Quality Management Annual Report FY 2018 indicators (Table 26). Many service providers and Support Coordinators were not documenting how they supported individuals to develop social roles, seek employment, or participate in the community. Table 26. Low Scoring Community Life Record Review Indicators (PCR) FY 2016 FY 2017 FY 2018 p value FY 17/18 Documentation demonstrated: Opportunities to seek employment in competitive integrated settings Provider Record Review 55.2% 24.9% 24.3% (n = 364) (n = 794) (n = 729) p = .77 Support Coordinator Record Review 48.2% 41.9% 43.7% (n = 363) (n = 394) (n = 373) p = .61 Development of social roles and natural supports that reflect the individual's interests Provider Record Review 42.9% 25.4% 28.9% (n = 532) (n = 881) (n = 823) p = .10 Support Coordinator Record Review 48.5% 46.4% 52.9% (n = 480) (n = 481) (n = 482) p = .04 How the individual is supported to learn about, explore and experience the community Provider Record Review 44.3% 39.5% 44.6% (n = 532) (n = 881) (n = 827) p = .03 Support Coordinator Record Review 48.7% 39.0% 45.7% (n = 476) (n = 480) (n = 481) p = .03 How the individual is supported to have or has responsibilities in the community as desired Provider Record Review 62.8% 42.8% 50.7% (n = 530) (n = 883) (n = 825) p = 0 Support Coordinator Record Review 58.5% 59.5% 63.5% (n = 480) (n = 477) (n = 479) p = .21 How the individual is supported to/able to participate in community activities and employment the same as Individuals without disabilities Provider Record Review 79.1% 40.4% 55.6% (n = 460) (n = 854) (n = 806) p = 0 Support Coordinator Record Review 76.9% 62.2% 62.6% (n = 455) (n = 458) (n = 462) p = .92 Engagement in community life Provider Record Review 75.6% 39.5% 44.6% (n = 533) (n = 881) (n = 827) p = .49 115 | P a g e Quality Management Annual Report FY 2018 Choice Choice is the second-lowest scoring FOA showing an average score of 82.3 percent for FY 2018, an increase from FY 2017. Most Support Coordinators (95.6%) demonstrated the individual was living in the most integrated community setting appropriate to the individual's needs, preferences and level of independence. Additionally, most service providers (90.7%) showed evidence the individual was offered choice. As indicated in Table 27, staff were often not able to describe how they provided the individual options for competitive employment or integrated living settings. Many Support Coordinator and provider record review indicators showed significant decreases from FY 2016 to FY 2017. Of these, some showed improvement but are still relatively low scoring. Findings indicated a lack of documentation for how individuals receiving services were making meaningful and informed choices in different aspects of their lives, including living environments, living situations, competitive employment and community participation. Table 27. Low Scoring Choice Indicators (PCR) Staff Interview Staff is presenting options of: Competitive/supported employment Living situations are integrated into local community (i.e., full continuum of housing options, roommate) Support Coordinator Interview FY 2016 FY 2017 74.7% (n = 517) 93.0% (n = 341) 50.5% (n = 220) 65.5% (n = 905) Support Coordinator offers choices related to supported employment and/or competitive employment 72.4% (n = 381) 62.0% (n = 413) Provider Record Review Providers demonstrated through documentation how individuals: Are provided a choice of living situations 74.9% 26.4% (n = 267) (n = 708) Exercise meaningful choices about living environments 70.7% 21.1% (n = 259) (n = 715) FY 2018 49.2% (n = 313) 65.5% (n = 284) 64.7% (n = 434) 26.0% (n = 739) 26.3% (n = 759) p value FY 17/18 p = .78 p = 1.0 p = .41 p = .86 p = .02 116 | P a g e Quality Management Annual Report FY 2018 Table 27. Low Scoring Choice Indicators (PCR) Make informed choices about competitive 54.6% 30.8% 26.5% or supported employment options (n = 379) (n = 827) (n = 759) Are provided with information to make informed choices (education, exploration and experiences) 40.7% 27.4% 27.8% (n = 548) (n = 891) (n = 845) Make informed choices about community participation and social interaction 54.8% 42.8% 47.5% (n = 540) (n = 888) (n = 832) Support Coordinator Record Review Support Coordinators demonstrated through documentation how individuals: Are provided with education, exploration and experiences 41.9% 31.2% 35.8% (n = 482) (n = 481) (n = 483) Are offered employment or educational options 55.4% 49.3% 52.7% (n = 401) (n = 423) (n = 438) Make meaningful choices about the living environment 77.2% 50.3% 53.6% (n = 351) (n = 441) (n = 476) Are provided choices of living situations 75.0% 47.1% 55.6% (n = 324) (n = 433) (n = 466) Make meaningful choices about community 56.5% 50.3% 66.3% participation (n = 480) (n = 441) (n = 483) p = .06 p = .85 p = .05 p = .13 p = .31 p = .33 p = .01 p = .01 Rights Scores surrounding Rights have remained relatively high each year since FY 2016, showing an average score of 92.0 percent in FY 2018, an increase of approximately two points since FY 2017 (90.2%). Providers showed evidence for all rights restrictions that a fading plan was in place, and only one record reviewed (PRR) gave an indication of co-mingled funds between the individual and the provider the provider was given technical assistance on how to rectify. Almost every individual interviewed felt they had privacy (99.2%), did not have rights restrictions (99.9%), were treated with respect (99.9%), and almost all staff were aware of individual's rights and preferences for exercising those rights (99.3%). Findings in Table 28 show the lowest scoring indicators for the Rights FOA. Many Support Coordinators were not aware of restrictive interventions that were in place and many were not addressing rights restrictions. Support Coordinator documentation did not always show evidence of a signed informed consent form for psychotropic medications or of a signature on the notification of 117 | P a g e Quality Management Annual Report FY 2018 rights and responsibilities. Documentation often did not show evidence the complaints/grievance policy was shared in a manner accommodating the individual's learning style. Table 28. Low Scoring Rights Indicators (PCR) FY 2016 FY 2017 Support Coordinator Interview Support Coordinators were: Aware of restrictive interventions/plans in place 87.8% 47.8% (n = 181) (n = 92) Addressing identified rights restrictions 90.6% 62.0% (n = 223) (n = 129) Support Coordinator Record Review Support Coordinator documentation demonstrated evidence of: The individual or legal guardian's signature on notification the individual has been informed about rights and responsibilities, at least annually 58.4% 22.7% (n = 473) (n = 476) Informed consent for taking psychotropic medications prescribed by a psychiatrist or psychiatric nurse practitioner 49.1% 35.5% (n = 281) (n = 251) The complaints/grievance policy is shared in a manner accommodating the individual's learning style 61.5% (n = 478) 34.7% (n = 479) HIPAA Privacy and Security Rules are specifically reviewed with individuals 72.1% 55.2% (n = 480) (n = 478) Consent provided by the individual or legal guardian for the implementation of the Positive Behavior Support Plan 79.1% (n = 86) 55.2% (n = 49) Provider Record Review Provider records demonstrated evidence of: Ongoing education about rights 53.8% 32.6% (n = 548) (n = 891) For Provider-Owned or Controlled Residential Settings, there was documentation demonstrating the individual has a lease or other legally enforceable agreement to protect from eviction 64.6% 44.3% (n = 113) (n = 201) FY 2018 56.9% (n = 130) 68.9% (n = 180) 23.6% (n = 461) 41.1% (n = 248) 42.0% (n = 483) 66.9% (n = 484) 66.9% (n = 61) 43.0% (n = 849) 49.8% (n = 849) p value FY 17/18 p = .18 p = .21 p = .72 p = .19 p = .02 p = 0 p = .56 p = 0 p = .25 118 | P a g e Quality Management Annual Report FY 2018 Individual Service Plan Quality Assurance Checklist (ISP QA) During the PCR, assessors reviewed the content of the Individual Service Plan (ISP) to assess adherence to requirements and the extent to which the plan specifically addressed specific needs, goals, and desires. ISP Written to Support a Meaningful Life The ISP QA checklist provided an overall rating for each service plan, based upon the degree to which the ISP was written to provide a meaningful life for the individual receiving services. Each ISP was identified as providing one of the following life-styles: Service Life: The ISP supports a life with basic paid services and paid supports. Needs "important for" the individual were addressed, such as health and safety. However, there is not an organized effort to provide support in obtaining other expressed desires "important to" the individual, such as getting a driver's license, having a home, or acting in a play. The individual receiving services was not connected to the community and has not developed social roles but expresses a desire to do so. Good but Paid Life: The ISP supports a life with connections to various supports and services (paid and non-paid). Expressed goals "important to" the person are present, indicating goals and desires were obtained beyond basic health and safety needs. Individuals may go out into the community but with only limited integration into community activities. For example, someone may go to church but not have the opportunity to participate in Sunday school or sing in the choir. Community connections are lacking, and there is an indication of a desire to achieve more. Community Life: The ISP supports a life with the desired level of integration in the community and in various preferred settings. Friends and support beyond providers and family members is demonstrated, as is the development of meaningful social roles, such as belonging to a Red Hat club or a book club or having employment in a competitive rather than segregated environment. Rather than just going to church, the Individual receiving services may be an usher at the church or sing in the choir. Relationships developed in the community are reciprocal. The ISP is written with goals that help support moving toward a Community Life, as she or he chooses. 119 | P a g e Quality Management Annual Report FY 2018 Overall findings are presented in Figure 47. A majority of ISPs reviewed each year was written to support a "Good but Paid Life." Findings in FY 2018 may suggest a shift from Good but Paid Life category (decreasing) to both Community Life and Service Life (increasing). However, these differences were quite small and the most prominent results is the continued majority of ISPs rated as a Good but Paid Life. 100% 75% Figure 47. PCR ISP QA Life Indicator by Year 73.8% 70.8% 66.7% 50% 25% 9.0% 14.9% 11.9% 16.6%17.3%18.4% 0% Service Life A Good but Paid Life Community Life FY 2016 (N = 484) FY 2017 (N = 481) FY 2018 (N = 484) Findings by region and year are presented in Table 29. While N sizes are relatively small, each region had over 50 ISPs reviewed. Findings for Region 2 appear to indicate a downward trend in the percent of ISPs written to support a Service Life or Good but Paid Life, and a concurrent upward trend in Community Life. In Region 6, the percent of ISPs supporting a Good but Paid Life has decreased over the three-year timeframe and the percent supporting a Community Life has increased. Each year, Region 1 has shown the highest, or close to the highest, percent of ISPs supporting a Community Life, compared to all other regions. 120 | P a g e Quality Management Annual Report FY 2018 Region 1 2 3 4 5 6 Table 29. ISP QA Life Indicator by Region and Year Service Life Good but Paid Life Community Life FY 2016 FY 2017 FY 2018 FY 2016 FY 2017 FY 2018 FY 2016 FY 2017 FY 2018 5.3% 6.3% 15.7% 58.5% 55.8% 58.4% 36.2% 37.9% 25.8% 12.6% 10.0% 5.1% 82.8% 75.6% 67.9% 4.6% 14.4% 26.9% 9.1% 8.2% 17.1% 73.6% 74.6% 67.9% 17.4% 17.2% 15.0% 5.6% 9.8% 15.9% 83.3% 86.3% 77.8% 11.1% 3.9% 6.3% 5.3% 20.6% 8.9% 78.9% 68.3% 75.0% 15.8% 11.1% 16.1% 19.1% 23.7% 25.9% 72.1% 69.5% 55.2% 8.8% 6.8% 19.0% ISP Expectations In December 2017, DBHDD completed modifications to the checklist template and expectations, and the new/revised form went into effect in January 2018. Therefore, 397 ISP QA Checklists were completed using the original form and 78 were completed using the new ISP form. All the expectations and indicators have been modified, with the exception of the Overall Rating of how the ISP is written, shown for all 484 checklists in Figure 47 and Table 29 above. For the remaining results, data are presented for the original and new ISPs separately. Results Using Original ISP QA Checklist Quality assessors reviewed 12 different indicators to measure the quality of the ISP. Each indicator is listed in Table 30 and each has four requirements that must be met by the content in the ISP. The indicator is rated 0 - 4; "0" meaning the content of the ISP did not meet the requirements and "4" meaning all four of the requirements were addressed in the ISP. Data in Table 30 shows the percent of ISPs at each rating on the 12 different indicators. For example, 46.0 percent of ISPs reviewed in FY 2018 had all four requirements present for the indicator regarding the communication chart. Data indicates: Fifty-seven point one percent of all ISPs reviewed addressed all elements in each section. Expectations regarding the rights, psychotropic medications, or behavior supports and the health and safety sections were most likely to have all four expectations met. 121 | P a g e Quality Management Annual Report FY 2018 The ISP indicators measuring how well the individual's hopes and dreams were addressed and whether goals were person-centered were most likely to have a rating of "0" or "1", 21.1 percent and 18.0 percent, respectively. Table 30. ISP QA Checklist Ratings by Expectation (FY 2018) N = 397 Ratings ISP QA Checklist Description 0 1 2 3 4 Average Rights, Psychotropic Medications, Behavior Supports Section 0.6% 0.0% 0.0% 3.1% 96.3% 3.9 Health and Safety Review Section completed accurately and thoroughly 0.6% 0.0% 0.6% 8.7% 90.1% 3.9 Person-centered Important to/For 0.6% 0.0% 1.9% 28.6% 68.9% 3.7 SIS completed and support needs are addressed in the ISP 0.6% 0.0% 3.1% 28.6% 67.7% 3.6 Communication Chart 0.6% 0.0% 6.2% 47.2% 46.0% 3.4 Service Summary 2.5% 2.5% 14.3% 28.0% 52.8% 3.4 Relationship Map & discussion on ways to develop relationships 0.6% 4.3% 14.3% 33.5% 47.2% 3.3 Meeting Minutes 1.9% 7.5% 16.8% 21.1% 52.8% 3.3 Training Goal Action Plan 0.6% 3.7% 11.2% 54.7% 29.8% 3.2 Hopes and Dreams 13.0% 8.1% 9.3% 16.1% 53.4% 3.1 Action Plans/Objectives 0.6% 2.5% 20.5% 39.1% 37.3% 3.1 Goals are Person-centered 4.3% 13.7% 12.4% 28.6% 41.0% 3.0 Total 2.2% 3.6% 9.3% 28.3% 57.1% 3.4 Table 31 shows the average rating (0 - 4) by expectation for FY 2016 through FY 2018. The average rating for all three years was 3.4 and showed very little change by year for each expectation. Expectations measuring how well the individual's hopes and dreams were addressed and whether goals were person-centered were the lowest rated expectations all three years. 122 | P a g e Quality Management Annual Report FY 2018 Table 31. Average Rating by Expectation (0 - 4) ISP QA Checklist FY 2016 FY 2017 (N = 484) (N = 481) Rights, Psychotropic Medications, Behavior Supports Section 3.9 3.9 Health and Safety Review Section completed accurately and thoroughly (HRST information should be noted 3.9 3.9 throughout ISP) Person-centered Important To/For 3.7 3.6 Supports Intensity Scale (SIS) completed and support needs are addressed in the ISP 3.6 3.6 Communication Chart Service Summary Relationship Map & discussion on ways to develop relationships 3.5 3.4 3.2 3.3 3.4 3.3 Meeting Minutes 3.2 3.2 Training Goal Action Plan 3.2 3.2 Hopes and Dreams 3.0 3.0 Action Plans/Objectives Goals are Person-centered 3.2 3.1 3.0 2.9 Overall Average 3.4 3.4 FY 2018 (N = 397) 3.9 3.9 3.7 3.6 3.4 3.4 3.3 3.3 3.2 3.1 3.1 3.0 3.4 Results Using New ISP QA Checklist The "New" ISP QA Checklist (effective date 1/1/2018) contains seven Expectations that use a Likert scale to score the result. Scoring options are Needs Improvement, Emerging, Achieving and Exceeding. Likert scale ratings are presented for the state in Figure 48. The Expectations "Summary reflects the individual's community life" and "[Relationship] Map demonstrates the individual is connected to the community" showed the greatest need for improvement, 20.5 percent and 10.3 percent, respectively. Consistent with ratings from the original ISP QA, over 75 percent of all new ISP QA's were rated either "Achieving" or "Exceeding" on the following expectations: Goals and objectives were person centered (75.6%) Health and Safety section included how all current issues, needs and/or risks were addressed (97.4%) 123 | P a g e Quality Management Annual Report FY 2018 Profile was detailed and included person centered information (79.5%) Communication Chart provided meaningful information staff can use to best support the individual (76.6%) Figure 48. "New" ISP QA Checklist Ratings Map demonstrates the person is connected to the 10.3% community. 34.6% 16.7% Communication Chart provides meaningful information staff can use to best support the person. 2.6% 20.8% 41.6% Profile is detailed and includes person centered information. 2.6% 17.9% 28.2% Summary describes changes in the person's life in the last 6.5% year and the supports/plans needed to address the 22.1% change with the person. 36.4% 38.5% 35.1% 51.3% 35.1% Summary reflects the person's community life. 20.5% 39.7% 29.5% 10.3% Health and Safety section includes how all current issues, 1.3%1.3% needs and/or risks are addressed. 47.4% 6.4% Goals and objectives are person centered. 17.9% 21.8% 50.0% 53.8% Needs Improvement 0% Emerging 25% 50% 75% Achieving Exceeding 100% PCR Results by Service During the PCR, Quality Management conducted a record review with every provider who provided services to the individual and whose services are eligible for review by the Collaborative. Information in Table 32 shows results for record reviews completed during the PCRs, by service and year. The N is the number of records reviewed for each service and the percent met is based on the total number of indicators reviewed. When reviewing the findings, it is important to note several services had 124 | P a g e Quality Management Annual Report FY 2018 eight or fewer records reviewed. Among services with 30 or more records, Supported Employment reflected the highest score in FY 2018, an increase from last year and close to the FY 2016 rate. Table 32. PCR PRR Results by Service and Year FY 2016 FY 2017 Service N % Met N % Met Provider Record Review 558 79.0% 894 70.2% Behavioral Supports - - 2 91.7% Community Access (Group) 233 77.1% 378 67.9% Community Access (Individual) 60 79.7% 95 72.9% Community Living 47 83.8% 72 66.8% Community Residential 91 80.4% 195 73.4% Nursing Services - - 1 87.8% Occupational Therapy - - - - Physical Therapy Prevocational - - - - 70 75.9% 74 64.9% Respite 1 87.5% 6 61.8% Speech/Language Therapy Supported Employment - - - - 54 82.3% 71 77.8% Transportation 2 84.7% - - Support Coordination Record Review 484 79.9% 481 73.6% FY 2018 N % Met 856 74.8% 8 83.9% 361 74.6% 92 74.4% 104 70.1% 195 77.1% 2 68.8% 1 83.3% 1 82.8% 51 71.6% 8 67.6% 1 82.8% 32 81.0% - - 484 77.5% Table 33 displays scores by FOA for the services reviewed during the PCR for FY 2018. Among providers with more than 30 records, results indicated the following: Providers of Community Access, Prevocational, and Supported Employment showed the lowest scores in Whole Health. Support Coordinators reflected the lowest Rights score and providers of Community Living Supports reflected the lowest Community Life score. The average statewide scores for FOAs ranged from a high of 84.4 percent for Rights to a low of 67.0 percent for Community Life. Within the reviewed services, the greatest variance was for Support Coordination, from a high of 97.2 percent for Safety to a low of 57.3 percent for Rights. 125 | P a g e Quality Management Annual Report FY 2018 Table 33. FY 2018 PCR Record Review Results by FOA and Service Service Whole Comm Health Safety PCP Life Choice Rights Behavioral Supports (N = 8) 81.2% 93.3% 87.7% 41.7% 79.4% 86.8% Community Access (Group) (N = 361) 69.3% 81.0% 79.3% 70.1% 60.7% 86.2% Community Access (Individual) (N = 92) 70.5% 80.9% 78.6% 69.9% 59.3% 85.8% Community Living Support (N = 104) 71.4% 75.8% 73.5% 56.0% 54.0% 84.4% Community Residential Alternative (N = 195) 87.6% 80.9% 75.0% 62.9% 55.2% 82.3% Prevocational (N = 51) 69.1% 81.4% 76.5% 62.3% 50.0% 86.2% Respite (N = 8) 81.3% 70.8% 64.2% 49.4% 53.4% 79.3% Supported Employment (N = 32) 61.9% 80.8% 88.0% 93.9% 78.7% 84.2% Registered Nurse (N = 2) 77.8% 71.4% 50.0% N/A 41.7% 83.3% Support Coordination (N = 484) 95.5% 97.2% 80.3% 67.6% 70.6% 57.3% State PRR Average (N = 856) 75.7% 80.5% 77.6% 67.0% 58.5% 84.4% Average 83.9% 74.6% 74.4% 70.1% 77.1% 71.6% 67.6% 81.0% 68.8% 77.4% 74.8% PCR Strengths and Recommendations During each PCR, assessors identified strengths about services offered and provided recommendations to help improve services and overall quality of life for individuals receiving services. Table 34 and Table 35 display strengths and recommendations identified in PCRs completed in FY 2018. Staff strengths most often cited included an awareness of unique safety needs, achievements, and communication of everyday decisions. Staff identified ways to help the individual experience new things in the community, explored other potential dreams and promoted independence. Table 34. Top Strengths Identified During a PCR Strength Instances Staff is aware of the unique safety needs of the individual. 374 Staff has a clear understanding how the individual communicates choice making in everyday decisions. 335 Individual feels valued. 316 Staff acknowledges the individual's achievements. 311 Percent of PCRs 77.8% 69.6% 65.7% 64.7% 126 | P a g e Quality Management Annual Report FY 2018 Table 34. Top Strengths Identified During a PCR Strength Instances Staff is knowledgeable of and provided examples of how preferences for exercising rights are actively being supported. 261 Staff consistently promotes independence. 243 Staff demonstrates the use of person centered values and approaches in everyday interactions with the individual. 225 Percent of PCRs 54.3% 50.5% 46.8% Table 35. Top Recommendations Identified During a PCR Recommendation Instances Conduct "what if" scenarios to determine the individual's skills in various safety situations. 254 Identify ways to expose the individual receiving services to new experiences in his/her community. 201 Ensure daily schedules and activities promote exposure to new things ('new places and new faces') and are not stagnant in nature. 191 Support exploration of other potential dreams. This can be done by using the 3 Es; Education, Exposure, and Experience. 185 Percent of PCRs 52.8% 41.8% 39.7% 38.5% Quality Enhancement Provider Review (QEPR) QEPR Scores by Size The QEPR Overall score is based on findings from the Individual Interviews, Staff Interviews, Observations and Provider Record Reviews (PRR). The Administrative Qualifications and Training (Q&T), based on a sample of staff rendering services, and the Developmental Disability Service Specific (DDSS) reviews that monitor compliance specific to each service, are reviewed and scores are supplied to the provider, but are not included in the provider's Overall score. During FY 2018, one crisis provider received a QEPR. Because crisis services are very different than typical services, the results for crisis providers are presented separately. 127 | P a g e Quality Management Annual Report FY 2018 In FY 2018, 99 providers who were not reviewed in FY 2016 or FY 2017 were selected for a QEPR: 72 Small, 12 Medium, and 15 Large. One provider offered crisis services and the provider of this service was reviewed separately for the service, bringing the total to 100 QEPRs completed. Comparisons by year are not an appropriate measure as the FY 2017 and FY 2018 samples were not selected to be representative of the state the proportion of Small providers increased each year. See Figure 49 for distribution of QEPRs by provider size for FY 2016, FY 2017 and FY 2018. Figure 49. FY 2016 - FY 2018 QEPR Samples by Provider Size and Type FY 2016 47% 36% 16% 1% FY 2017 52% 19% 25% 4% FY 2018 72% 12% 15% 1% 0% 20% 40% 60% 80% Small (caseload 30) Large (caseload 100) Medium (30 < caseload > 100) Crisis Provider 100% Overall QEPR Score by Tool and Year Figure 50 shows the average Overall scores for providers reviewed in FY 2016, FY 2017 and FY 2018 and scores for each tool used in the review process. The Overall score for FY 2018 was 84.8 percent, with the Observation and Staff Interviews showing the highest scores of just over 95 percent. As in FY 2016 and FY 2017, the PRR reflected the lowest-scoring area (72.4%). 128 | P a g e Quality Management Annual Report FY 2018 Figure 50. Overall QEPR Scores by Tool and Year Individual Interview FY16: 95.1% FY17: 90.7% FY18: 89.6% Provider Record Review FY16: 73.6% FY17: 69.6% FY18: 72.4% Overall Score FY16: 88.4% (N = 99) FY17: 83.7% (N = 96) FY18: 84.8% (N = 99) Staff Interview FY16: 96.9% FY17: 95.2% FY18: 95.3% Observation FY16: 98.2% FY17: 95.7% FY18: 96.0% Overall Crisis Provider Scores Six crisis providers have received a QEPR, one in FY 2016, four in FY 2017, and one in FY 2018. Figure 51 shows the Overall scores for these providers and scores by tool and year. Compared to other providers reviewed with the QEPR, crisis providers show relatively higher scores across all tools, but especially higher scores for the PRR (91.1% versus 72.4%). 129 | P a g e FY16: N = 1 FY17: N = 4 FY18: N = 1 Quality Management Annual Report FY 2018 Figure 51. Overall Crisis Provider Scores by Tool and Year Individual Interview FY16: 83.4% FY17: 86.0% FY18: 90.2% Provider Record Review FY16: 88.0% FY17: 84.1% FY18: 91.1% Overall Score FY16: 85.1% FY17: 86.8% FY18: 93.1% Staff Interview FY16: 81.3% FY17: 89.1% FY18: 97.8% Observation FY16: 87.7% FY17: 92.5% FY18: 97.9% Qualifications and Training (Q & T) and Service Specific (DDSS) Every QEPR included a review of staff qualifications and training. In FY 2018, a sample of 780 staff records was reviewed. The sample was stratified by service to ensure all services offered by each provider were included in the review process. The primary purpose of the Q&T record review was to confirm relevant staff information was accurate and up to date (e.g., driver's license, performance evaluations, background screening) and staff had received all required trainings specific to services provided. DDSS reviews are completed to ensure services are provided as specified by DBHDD (Figure 52). 130 | P a g e Quality Management Annual Report FY 2018 Figure 52. QEPR Qualifications and Training and DDSS Results by Year QEPR Providers Crisis Providers Q & T FY 2016: 80.6% (N = 1,003) FY 2017: 82.6% (N = 947) FY 2018: 84.6% (N = 780) DDSS FY 2016: 99.9% (N = 1,389) FY 2017: 99.8% (N = 1,667) FY 2018: 99.8% (N = 1,101) Q & T FY 2016: 92.8% (N = 11) FY 2016: 91.8% (N = 51) FY 2018: 94.5% (N = 5) DDSS FY 2016: 100% (N = 8) FY 2017: 100% (N = 48) FY 2018: 100% (N = 6) Qualifications & Training Opportunities for Improvement Provider staff, particularly for crisis providers, appeared to do relatively well maintaining compliance with required training and qualifications with overall Q&T scores of 84.6 percent and 94.5 percent, respectively. Most staff (97.4%) followed DBHDD's policy 04-104 for Criminal Records Checks and were properly licensed (96.5%). One hundred percent of crisis providers ensured crisis response system staff (mobile team members and intensive support staff) had participated in training and passed an examination demonstrating competence in all crisis protocols and requirements. For providers reviewed this year, several areas present opportunities for improvement in staff training, including the following: Within the first 60 days of hire, many staff did not have training on: o Use of the Georgia Crisis Response System (42.1% met) o How to work with individuals with co-occurring diagnoses (47.4% met) o Suicide prevention skills (65.3% met) Required training for Developmental Disability Professionals did not always include: 131 | P a g e Quality Management Annual Report FY 2018 o Supports Intensity Scale overview (52.0% met) o Individual service planning training (58.4% met) The following required topics were not always included in the annual training: o Specific individual medications and their side effects (64.8% met) o Emergency and disaster plan procedures (68.3% met) The Q&T review component uses a sample of employees from each provider to determine compliance with standards. The number of employee records sampled per provider varies based upon the total number of staff and services rendered. For this report, the results are presented by indicator and by provider for all employee records reviewed. Therefore, while an indicator for most employee records may support a "met", fewer providers may have all employees in compliance with the standard. For example: The aggregate score for staff records shows 77.5 percent had training on person-centered values, principles and approaches. o Only 66.3 percent of QEPR providers had this training documented for the employee records selected. Approximately 97 percent of staff records had documented evidence of all background screening requirements. o However, only about 87 percent of QEPR providers demonstrated all staff records reviewed have all required background screening requirements in place. Accurate and up-to-date annual work performance evaluations were present for 72.9 percent of employee records reviewed. o Only 49.5 percent of providers showed all staff records reviewed had met this requirement. Most staff (86.2%) were current on annual tuberculosis testing. 132 | P a g e Quality Management Annual Report FY 2018 o Only 61.2 percent of providers had documentation that all staff records reviewed contained current tuberculosis testing. QEPR Scores by Provider Size13 Figure 53 displays the distribution of the QEPR Overall and Q&T scores by size of the organization and year.14 In FY 2018, Overall scores ranged from 82.0 percent for Medium providers to 86.3 percent for Small providers. Qualifications and Training scores ranged from a low of 81.1 percent for Medium providers to 85.8 percent for large providers. On average, Medium providers scored lower than Small or Large providers on both the Overall score and Q&T. All providers, regardless of size, scored above 98 percent on the DDSS tool (not shown in the figure). 13 Crisis provider scores were excluded. There was one Small crisis provider reviewed in FY 2018, one large, one medium and two Small crisis providers reviewed in FY 2017 and one Small crisis provider reviewed in FY 2016. 14 See Figure 49 for details regarding provider size categories. 133 | P a g e Quality Management Annual Report FY 2018 Figure 53. Overall and Qualifications and Training Score by Provider Size and Review Year Overall Score Qualifications and Training 100% 100% 90% 90.4% 89.0% 86.3% 86.1% 84.4% 82.0% 86.1% 80% 83.2% 82.7% 70% 90% 85.3% 85.8% 80.0% 80% 82.2% 81.1% 86.4% 79.8% 81.5% 70% 76.0% 60% 60% 50% Small Medium Large 50% Small Medium FY 2016 (N = 99) FY 2017 (N = 96) FY 2018 (N = 99) Large QEPR Scores by Focused Outcome Area (FOA) The Overall score for each review was divided into six FOAs. Each FOA score was calculated with the combined results from the Individual Interview, Staff Interview, Observation (IOSA), and Provider Record Review.15 Results for QEPRs (except for crisis providers) are shown by FOA and year in Figure 54. With the exception of Person Centered Practices, scores within each FOA increased slightly from FY 2018. The lowest scoring areas each year were Community Life, Choice and Person Centered Practices, while measures surrounding Safety and Rights remained relatively high showing scores over 90 percent each year. 15 FOAs calculated for the PCR also included the Support Coordinator Interview and SCRR, which are not part of the QEPR process. Comparisons between the PCR and QPER are not appropriate. 134 | P a g e Quality Management Annual Report FY 2018 Figure 54. Overall QEPR Scores by FOA and Year Rights FY16: 92.4% FY17: 91.2% FY18: 92.4% Choice FY16: 86.2% FY17: 75.9% FY18: 78.2% Whole Health FY16: 89.1% FY17: 83.2% FY18: 84.9% Overall Score FY16: 88.4% FY17: 83.7% FY18: 84.8% Community FY16: 78.0% FY17: 70.4% FY18: 72.0% Safety FY16: 93.9% FY17: 90.9% FY18: 91.0% Person Centered Practices FY16: 82.6% FY17: 81.6% FY18: 80.4% QEPR Scores by Tool and FOA In this section, results by FOA for the QEPR are presented for each tool used in calculating the Overall score for the QEPR process Individual Interview, Observation, Staff Interview and PRR (see Figure 55).16 Findings for the providers reviewed this year indicate the following: Provider documentation (PRR) was the lowest scoring tool across all the FOAs, ranging from a low of 52.5 percent for measures surrounding Choice to 83.8 percent for Rights. 16 See Table 21 for the number of interviews and records completed for each QEPR component. 135 | P a g e Quality Management Annual Report FY 2018 Observations and Staff Interviews reflected relatively high Community Life scores, 93.1 percent and 85.6 percent respectively; however, the Individual Interview results in this FOA was somewhat lower (79.0%) and providers did not document this as well (64.3%). Community Life and Person Centered Practices were the lowest scoring FOAs (79.0% and 91.5% respectively) from the Individual Interviews. Observation scores were approximately 93 percent or higher in all the FOAs with the exception of Person Centered Practices (90.4%). Figure 55. FY 2018 QEPR Scores by Tool and Focus Outcome Area Whole Health Safety Person Centered Practices Community Life Choice Rights 86.1% 95.4% 94.9% 75.2% 78.7% 91.7% 96.9% 98.8% 81.5% 93.9% 90.4% 74.3% 64.3% 79.0% 85.6% 93.1% 52.5% 94.8% 93.6% 96.6% 96.4% 97.5% 96.3% 83.8% 0% 25% Individual Interview (N = 436) Observations (N = 285) 50% 75% Staff Interview (N = 316) Record Review (N = 955) 100% 136 | P a g e Quality Management Annual Report FY 2018 QEPR Scores by FOA and Provider Size Provider scores by size and FOA are presented in Figure 56. Compared to Small and Large providers, Medium providers scored relatively lower across all FOAs. With the exception of Whole Health and Person Centered Practices, there was little variation across FOAs for Small and Large providers in FY 2018. Small providers performed relatively better on Whole Health and Large providers performed relatively better on Person Centered Practices. Figure 56. FY 2018 QEPR Provider Scores by FOA and Size Whole Health Safety Person Centered Practices Community Life Choice Rights 50% 88.7% 81.4% 82.0% 91.5% 90.2% 90.8% 78.8% 78.2% 82.9% 72.5% 69.7% 72.8% 80.9% 72.7% 78.1% 92.5% 90.5% 93.3% 60% 70% 80% 90% 100% Small (N = 72) Medium (N = 12) Large (N = 15) 137 | P a g e Quality Management Annual Report FY 2018 Quality and Technical Assistance Consultation (QTAC) A total of 161 QTACs were completed, of which 136 were at the provider level and 25 at the individual level. A provider level QTAC focuses on the overall provider practices and processes versus an individual level QTAC that only focuses on services and supports for that one person. Due to the ability to check more than one referral type, totals do not sum to 161. Table 36 shows the number of QTACs by referral source and type for FY 2018. Most provider referrals were internally initiated but the provider may have also requested additional technical assistance once on site. Most QTACs were completed at the provider level (84.5%) and most had an internal referral source (85.3%). The majority of QTACs (64.6%) were generated from the QEPR. Table 36. QTACs by Referral Source and Type Provider (N = 136) Individual (N = 25) Referral Source Number Percent Number Percent Internal 116 85.3% 25 100.0% Provider 20 14.7% 0 0.0% Referral Type: Quality of Care (PCR/QEPR) CAP Review Provider Request QEPR Follow Up 9 6.5% 24 96% - - 1 4% 26 18.8% - - 103 74.6% - - Table 37 provides a list of detailed reasons for the QTACs completed in FY 2018. The most frequently cited reasons were related to the QEPR and alert follow-ups. More than one reason can be identified for a QTAC. Table 37. QTAC Referral Reasons: FY 2018 Reason QEPR follow up Quality of care follow up Review of person-centered documentation Request for 2 day Person-Centered Thinking Training Person-centered training needed Lack of person-centered documentation Other N % 103 62.7% 33 20.5% 13 8.1% 5 3.1% 3 1.9% 2 1.2% 2 1.2% 138 | P a g e Quality Management Annual Report FY 2018 Table 37. QTAC Referral Reasons: FY 2018 Reason Assistance with criminal background checks Social inclusion training needed Total N % 1 0.6% 1 0.6% 161 100.0% Technical assistance is provided at every QTAC. Approximately 45 to 48 percent of provider level QTACs offered brainstorming, group or individual discussion, and resources (Figure 57) to help address the areas needing improvement. The most common type of technical assistance offered at the individual level was individual discussion (80.0%) with staff. Figure 57. FY 2018 QTAC Technical Assistance Provided 1:1Training 6.6% 8.0% Brainstorming 24.0% Group Discussion Group Training 16.0% 18.4% 0.0% Individual Discussion Resources 8.0% Role Play 10.3% 0.0% 0% 20% Provider (N = 136) 47.1% 47.8% 44.9% 47.8% 80.0% 40% 60% 80% Individual (N = 25) 100% Intellectual and Developmental Disability Summary of Findings and Recommendations 139 | P a g e Quality Management Annual Report FY 2018 June 2018 marked the completion of the third year of the Collaborative contract. Quality assessors completed 100 QEPRs, 484 PCRs, and 161 QTACs. Beginning in FY 2017, the Collaborative implemented tool revisions to evaluate the quality of intellectual and developmental disability services and supports in both the QEPR and PCR. The six FOAs identified throughout this report and indicators were used to address a wide range of requirements and promising practices within each FOA. FY 2018 IDD Accomplishments Several opportunities were provided throughout the year for the IDD Quality Management team to work in partnership with DBHDD. This collaboration focused on three key initiatives: training for stakeholders, the new IDD Case Management system and tool revisions for the review processes. Throughout the year, the Collaborative's Quality Management team and DBHDD met to discuss and review findings from FY 2017, current fiscal year findings, and other identified areas of need across the provider network. Training sessions on four Focused Outcome Areas were developed and presented throughout the year based on this analysis and discussion. The following is a list of trainings provided for stakeholders in the 2018 fiscal year: Whole Health Person Centered Empowering Individuals to Maintain their Healthcare Techniques on How to Help Educate People on Medications Person Centered Documentation: Part 1 & Part 2 What is Person Centered Language? Part 1 & Part 2 Staff Matching, Why it Works Community How to Support Community Inclusion Rights & Choice The importance of supporting the concepts of Choice and Rights from the perspective of people receiving services and providers 140 | P a g e Quality Management Annual Report FY 2018 Additionally, in collaboration with DBHDD, a training plan was developed for the new IDD Case Management System project. The plan was developed for internal (the Collaborative and designated DBHDD staff) and external (individuals applying for services, providers of IDD services, and Support Coordination staff) users and other stakeholders to provide guidance on navigating and using the web-based application. Through workgroups with the DBHDD Quality Improvement Office, IDD program staff, and other stakeholders, implementation of tool revisions occurred over the course of FY 2018 to align with the DBHDD IDD Provider Manual, policies and procedures. The tool revisions included: Tool Updates Tool Redesign Individual Interview Observation Provider Record Review Staff Interview ISP QA Checklist Support Coordination Interview (to be implemented July 2018) IDD System Strengths and Recommendations Most providers and staff were aware of unique safety needs and achievements of individuals they serve. Most individuals were aware of how to self-preserve (88.6%) and felt safe in their work and living environments (99.6%). Based on observations, staff and providers took advantage of opportunities to provide education on health and rights, and with one exception, all individuals expressed that they were free from abuse, neglect, and exploitation. Findings on many of the tools for both the PCR and QEPR showed scores of over 90 percent (see Table 38). Similar to FY 2017, documentation is the lowest scoring area from record reviews during both the PCR and QEPR. 141 | P a g e Quality Management Annual Report FY 2018 Table 38. FY 2018 Summary by Tool and Review Type Tool PCR QEPR (N = 484) (N = 99) Crisis (N = 1) IOSA - Individual Interview 92.2% 89.6% 90.2% IOSA - Observation 98.2% 96.0% 97.9% IOSA - Staff Interview 95.6% 95.3% 97.8% Provider Record Review 74.8% 72.4% 91.1% SC Record Review 77.4% NA NA SC Interview 83.5% NA NA Admin Q&T NA 84.6% 91.8% DDSS 99.9% 99.8% 100% FOA scores from both review types (PCR and QEPR) suggest the service delivery system appears to do well across the six critical areas, showing scores of at least 70 percent each year. Scores from both review types were over 90 percent for Safety and Rights. Providers have effective systems and practices in place to help individuals be safe in their environments and exercise their rights, and individuals have indicated they do feel safe and their rights are upheld. Whole Health scores for both the PCR (87.4%) and for providers reviewed this year with a QEPR (84.9%) indicated most health needs are likely identified and addressed. Support Coordinators did very well on the health-related sections of the ISP in accurately completing all components of rights, the psychotropic medications and behavior supports section, and the health and safety review section. Opportunities for Improvement While overall FOA scores remained relatively high, drilling down to the standards/indicators measuring each FOA provides insight into opportunities for developing quality improvement initiatives or training programs. 142 | P a g e Quality Management Annual Report FY 2018 Whole Health Findings for Whole Health show, from the perspective of individuals who were interviewed, a consistent decline in several health-related areas. Each year they appear to be less likely to know what medications they are taking, why, and what the side effects are. Because individuals with intellectual and developmental disabilities may depend on color and shape of medication to ensure the correct pill/tablet is taken, having decreased awareness of this each year could be problematic and lead to an increase in medication errors. These trends appear to be supported through the PRR, for which findings indicate most providers did not offer education to individuals receiving services on their prescribed medications (48.3%) or including the risks and side effects (52.1%). Furthermore, for the PCR, because about half (N = 236) of the individuals sampled live in a family home and Whole Health was the lowest scoring area, this may be an area that warrants additional analysis to determine how living in a family home impacts an individual's health indicators. Recommendation XI: Currently the DBHDD Provider Manual for Community Developmental Disability Providers requires providers to educate individuals on the risks and benefits of antipsychotic medications and all other types of prescribed medication and this must be documented in the clinical record. Training in this area was conducted by the Collaborative in June of 2018. It is recommended this indicator continue to be tracked and evaluated in FY 2019 to determine if additional training would be beneficial. Recommendation XII: The Collaborative could complete an ad hoc report to compare the FOA scores at the indicator level across different resident type, specifically for individuals living in their family home. Safety Approximately 30 percent of individuals interviewed did not know how to respond to an emergency or crisis situation if supports were incapacitated. Over 52 percent of recommendations for the PCR 143 | P a g e Quality Management Annual Report FY 2018 were related to helping individuals receiving services learn how to manage safety situations by conducting "what if" scenarios. At the same time, 40 percent of staff were not providing education on the use of the Crisis Hotline. In addition, only 42.1 percent of staff records reviewed this year, and 43.4 percent reviewed in FY 2017, documented staff/provider training on the Georgia Crisis Response System. Information from Support Coordinators appears to point to a need for a better understanding of issues surrounding behavioral health, including a better awareness of intervention plans and specific triggers an individual may have. Support Coordinators were not likely to be aware of needed crisis plans (47.4% met) or behavior support plans (57.2% met). Half of the Support Coordinators were not aware of the individualized techniques needed to follow a plan for the individual (53.3%). Recommendation XIII: The Collaborative and DBHDD could consider providing specific training to Support Coordination as it relates to behavior supports and how to evaluate this service. Recommendation XIV: Almost every provider and Support Coordinator reviewed during safety situations is a fundamental support of the service delivery system. DBHDD could consider conducting a targeted safety campaign that includes resources, as well as web-based and face-to-face training throughout the state. The DBHDD monthly newsletter could include a new tip or resource that providers, individuals, and families can use to promote independence in the area of safety. Person Centered Practices A key area of focus for the Centers for Medicare and Medicaid Services (CMS) and DBHDD is to ensure services and supports are provided using person centered practices and planning. To the extent possible, the individual receiving services should be at the center of all decisions, plans, and goals. It is important for providers to understand who each individual is and what he or she may want, hope for, and dream. Unfortunately, many ISPs did not ensure all the goals were person centered (74.5%) or their hopes and dreams were addressed throughout the plan (80.8%). Most individuals receiving services seemed to understand their own talents, strengths and goals (82.6%) 144 | P a g e Quality Management Annual Report FY 2018 but most providers did not document those talents/strengths (42.2%) or how they review progress toward achieving goals with each individual (44.0%). Recommendation XV: DBHDD and the Collaborative will be providing training on the changes to the ISP being implemented in FY 2019. It is recommended this training include a section on writing person centered goals. Recommendation XVI: One of the key components to implementing a person centered approach to planning is to ensure individuals receiving services have the opportunity to review their goals and make changes as deemed necessary. With the implementation of the IDD Case Management System (new electronic record and ISP) it will be important to monitor how well the new system supports person centered planning. In the meantime, DBHDD could consider including a standard in the provider manual that goals should be reviewed with the individual more frequently and documented within the record. Community Life According to CMS expectations, individuals with intellectual and developmental disabilities should be able to participate in their communities in the same manner as individuals who do not have a disability. Community Life remains the lowest scoring area in the PCR and for providers reviewed through the QEPR. Information from individual interviews suggests about 25 to 30 percent of individuals receiving services had not been given the opportunity to learn about and develop new social roles in the community, or experience new community activities. In addition, provider and Support Coordinator documentation often does not show support for individuals to be engaged in the community, develop social roles, explore new activities or participate in activities the same as other citizens. The ISP should be written to help ensure integration into the community in various settings, as desired. However, only 18 percent of ISPs reflected full participation in the community. Recommendation XVII: An annual training could be offered on developing community connections for individuals receiving services. DBHDD could consider making this annual training mandatory for all providers. Training, and related resources, could include practical application of ideas and 145 | P a g e Quality Management Annual Report FY 2018 examples to help staff and Support Coordinators understand different and realistic ways to connect an individual to the community. Recommendation XIII: As part of the training on the revised ISP, DBHDD could consider providing training to include specific approaches to assist Support Coordinators in documentation in the Supports Intensity Scale (SIS) related to the support needs of the individual while in the community and developing ISPs and goals consistent with an integrated life in the community. Choice Informed choice is the cornerstone of helping anyone understand and achieve meaningful goals and direct supports and services. Information from documentation indicates most provider records lacked evidence informed choice was provided for competitive or supported employment (26.5%), living situations (26.0%), environments (26.3%), community participation/social interactions (47.5%), or educational opportunities (27.8%). Over half of staff interviewed had not provided options for competitive employment; however, provider and Support Coordinator documentation appears to be what is primarily affecting the low Choice FOA score. Recommendation XIX: Quality Management can develop a training to address how to document choices offered and how providers can support individuals to make informed decisions. This training would target support staff and Support Coordinators. Rights The proper use of restrictive interventions is strictly monitored and enforced by DBHDD. Individuals interviewed during the review processes did not indicate any violations regarding the improper use of any type of restrictive intervention or any unauthorized restrictions in their home, community, work, or day program. They generally feel their rights are upheld. However, records maintained by providers and Support Coordinators, as well as interviews with the Support Coordinators, indicate there may be some issues regarding proper documentation of addressing rights. 146 | P a g e Quality Management Annual Report FY 2018 Signed consent forms are often missing for individuals prescribed psychotropic medications (26.5%), thus indicating the individual had been informed of rights and responsibilities. Additionally, less than half of the providers documented how they provided rights education (43.0%). Recommendation XX: DBHDD could consider requiring Support Coordinators to provide education to individuals and families on the importance of giving and receiving copies of signed consent forms for psychotropic medications. Through this, Support Coordinators could ensure consent forms are being completed and subsequently document this in their own support notes. Provider Documentation Throughout the findings in this report, documentation by providers and Support Coordinators has shown consistently lower scores when compared across review tools or within the FOAs. Support Coordinators documented areas of health (95.5%) and safety (97.2%), but did not document rights (57.3%), the only area in which service providers scored higher. Provider documentation reviewed this year was lowest in the area of choice, the same as in FY 2017. Recommendation XXI: The Collaborative and DBHDD could consider developing develop a stakeholder workgroup to review the Support Coordination Outcome Review template and determine if there are needed improvements or changes that can better support providers. Recommendation XXII: Quality Management could develop a training to specifically address how Support Coordinators can improve support notes and documentation specific to areas identified throughout the intellectual and developmental disability section of this report. Additional Recommendations Quality Enhancement Provider Reviews (QEPR) One of the Collaborative initiatives for the 2019 fiscal year is to increase the frequency of QEPR data to analyze reviews from a three-year to a two-year cycle. This would provide information that is 147 | P a g e Quality Management Annual Report FY 2018 more current for all providers on the quality of services across the state, and increase the ability to examine more timely comparison of provider scores across the IDD Provider Network. Recommendation XXIII: The Georgia Collaborative Quality Department will continue to collaborate with DBHDD on revisions to the IDD review tools to more efficiently assess the quality of specific services according to the state guidelines. Collectively, DBHDD and the Collaborative could evaluate and develop the tools based on new policies and the DBHDD IDD Provider Manual. The QEPR report was originally designed to provide detailed data information to the provider regarding services rendered to all individuals receiving services. The report displays findings from the administrative review component for staff qualifications and training and quality of care concerns, including protected health information (PHI). The QEPR reports are not publicly posted at this time. Having the ability to post the QEPR providers reports on the Georgia Collaborative website would allow individuals and family members easy access to information on provider's quality performance. Additionally, providers would be able to see how their peers are conducting business in relation to the review process. In order to post the QEPR reports, they need to be restructured with all PHI removed. Recommendation XXIV: The Collaborative proposes to redesign the QEPR final report for public posting of a summary of findings on the Collaborative website, similar to BH. The report will remove all PHI; additionally, at the exit conference the provider will receive a detailed preliminary report to contain overall findings, recommendations and specific Individual Record Review findings. Quality Technical Assistance Consultation (QTAC) Each provider selected for a QEPR receives a QEPR follow up QTAC, regardless of score or opportunities for improvement. However, some providers need the technical assistance and additional on-site time from assessors more than others. By reducing the number of QEPR follow up QTACs completed each year, the Collaborative could focus technical assistance efforts on providers who most need support. 148 | P a g e Quality Management Annual Report FY 2018 Section 5: Behavioral Health and Intellectual Developmental Disabilities: Overall Results by Focused Outcome Area Quality Management analyzed the Focused Outcome Areas (FOA), as described throughout this report, in all four major review processes: BHQR, CSUQR, QEPR and PCR. The following graphic (Figure 58) displays results for each FOA by these review processes. While the same concepts are measured, it is important to note most direct comparisons across the processes are not appropriate and should be made with caution. The following caveats should be taken into consideration when reviewing the data: Sampling processes vary for different review processes. The total scores for each FOA for BHQR and CSUQR results are based only on record reviews. For QEPR and PCR, total scores for each FOA are based on indicators from four different tools including individual/staff interviews, observations and record reviews. A total of 22 indicators are used for all FOAs measured during the BH and CSU reviews, three to four per area. During the QEPR and PCR, a total of 115 indicators are used, 16 to 39 per area. Behavioral health and intellectual and developmental disability providers are required to record information using different provider manuals, requirements by waiver program, and national standards. 149 | P a g e Quality Management Annual Report FY 2018 Figure 58. FY 2018 Focused Outcome Areas by Review Type Whole Health Safety Person Centered Community Choice Rights BHQR FOA Overall: 92% 84% 78% 95% 97% 97% 93% CSUQR FOA Overall: 91% 88% 82% 88% 92% 97% 97% QEPR FOA Overall: 72.4% 75.2% 78.7% 74.3% 64.3% 52.5% 83.8% PCR FOA Overall: 74.8% 75.7% 80.5% 77.6% 67.0% 58.5% 84.4% In review of specific indicators for both the behavioral health and intellectual and developmental disability reviews, direct comparisons could not be identified. Although both reviews measure the same areas, behavioral health and intellectual and developmental disability indicators were not designed to be compared. As tools are revised and updated, effort will be made to develop more comparative indicators between BH and IDD in upcoming fiscal years. 150 | P a g e Section 6: Feedback Survey Results Quality Management Annual Report FY 2018 Following completion of quality reviews for both behavioral health and intellectual and developmental disabilities, providers are offered the opportunity to complete a feedback survey. For intellectual and developmental disability services, individuals who participate in the interview are also offered the opportunity to provide feedback about the process. Surveys are optional and may be completed by the same agency more than once in a fiscal year. In addition, it is not known who completes the survey, as individuals and staff are not required to submit their name or provider agency. A five-point Likert scale is used: strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly disagree. Table 39 provides information from providers for data entered into the system during FY 2018. Response rates were generally low, but findings were overwhelmingly positive across all the review types and from both providers and individuals. The percent was calculated as follows: (Strongly Agree + Somewhat Agree) / Total Responses. Additionally, two surveys were linked to the joint QEPR and BHQR process through the provider submission of name with the survey responses. The results for both systems, BH and IDD, using the calculation mentioned previously, was 99.6 percent. Results of indicators for each of the two systems are identified below for annual comparison. Table 39. FY 2018 Collaborative Provider Feedback Surveys Percent: Strongly Agree + Somewhat Agree/Total Responses IDD Survey Indicators (N = 66) Overall, you are satisfied with the review/consultation process. 98.5% The Collaborative staff interacted with you and your staff in a professional manner. 100% The Collaborative staff interacted with the individuals you support in a professional manner. 100% The Collaborative staff answered your indicators and concerns clearly and consistent with DBHDD manual. (If you disagree, please explain at the end 98.5% of the survey.) The Collaborative staff facilitated an environment which was collaborative and positive. 100% You would contact the Collaborative staff for technical assistance, training, and resource support, if needed. 100% BH (N = 23) 100% 100% 100% 95.1% 100% 100% 151 | P a g e Quality Management Annual Report FY 2018 Table 39. FY 2018 Collaborative Provider Feedback Surveys Percent: Strongly Agree + Somewhat Agree/Total Responses IDD Survey Indicators (N = 66) The process provided constructive feedback. 100% The process helped identify the strengths of your supports and services. 100% The feedback you received will help provide supports and services that meet the desired outcomes of the individuals you support. 100% The recommendations generated from this process can be used to make a positive contribution to the individuals served. 98.4% The recommendations generated from this process can be used to make a positive contribution to your organization overall. 100% The feedback provided will assist your organization with making quality improvements to systems and practices. 100% BH (N = 23) 100% 100% 100% 100% 100% 100% Overall Survey Result 99.6% 99.6% Only five responses were received specific to the individual survey completed following an IDD quality review, all responses resulted in "strongly agree," a 100 percent score. Indicators asked to individuals include: Overall, I am pleased with how the interview went. The purpose of the interview was explained to me. I was treated with respect. The person who interviewed me seemed interested in what I said. The person who interviewed me was pleasant. My indicators were answered. The length of the interview was good. Quality Management will continue to review its processes, including information obtained from these surveys, to ensure providers are equipped with the necessary tools and opportunities to best serve individuals across the state. Quality Management will also tailor its quality training both to its 152 | P a g e Quality Management Annual Report FY 2018 internal staff as well as for Georgia providers to promote a cooperative and constructive partnership in the quality of services provided in the state of Georgia specific to the individual's needs. 153 | P a g e Quality Management Annual Report FY 2018 Section 7: Conclusion FY 2018 marked the third complete year of the Collaborative Quality Management review process. During FY 2018, Quality Management completed 781 reviews for behavioral health and intellectual and developmental disability providers. The BHQR Overall score for FY 2018 was 88 percent, a fourpoint increase from both FY 2017 and the baseline year of FY 2016. Additionally, all four BHQR categories increased from the previous year specific to BHQRs with Assessment and Planning improving by ten percentage points. While there is demonstrated success noted in all of the different BHQRs categories, there are opportunities for growth similar to findings of previous years related to addressing all assessed needs, co-occurring issues being assessed and addressed, as well as appropriate discharge planning criteria documentation. Furthermore, ACT Service Guidelines score declined from FY 2017 to FY 2018 by four points. Several declines in ACT indicators brought the Service Guidelines category from 88 percent to 84 percent while the remaining categories of billing, FOAs, and Assessment and Planning increased. Additionally, FY 2018 marked the second year reassessments were completed for providers who fell below the threshold criteria established in FY 2017. Twenty providers (15%) were reviewed for a second time during the fiscal year compared to 35 (27%) in the year prior. In review of scores for these specific providers at the time of their first FY 2018 review compared to their second FY 2018 review, providers increased their scores in all areas. For example, the average Overall score went from 83 percent to 91 percent. While it is anticipated the reassessment of providers will lead to elevated results from year-to-year, some providers continue to decline in scoring from first review to second review. Two reassessed providers in FY 2018 declined rather than improved in Overall score. However, FY 2018 annual findings (with increases statewide) promote that ongoing review processes, provider education, and technical assistance have improved documentation specific to care of individuals and promoted improved quality of service to the individuals served. Consequently, any provider reviewed after July 1, 2018 having an Overall score or Billing score of less than 90 percent will be required to have a second review conducted. 154 | P a g e Quality Management Annual Report FY 2018 Specific to IDD reviews, Quality assessors completed 100 QEPRs, 484 PCRs, and 161 QTACs. Scores for all of the components of the PCR, i.e., interviews, observations and documentation, averaged over 75 percent, an overall improvement for each FOA. The average QEPR score for FY2018 has increased to 84.8 percent with the provider record review as the lowest scoring area. With the exception of the individual interview, all tools for the QEPR increased in score from the previous year. The QEPR FOA Overall scores all increased from FY 2017 except for Person Centered Practices that declined somewhat each fiscal year (FY 2016 82.6% to FY 2018 80.4%). Overall data results showed strengths in the area of Safety and Person Centered Practices. Where possible, Quality Management conducted joint BHQR, CSUQR and QEPRs to reduce administrative burden to providers. Ten joint reviews were conducted during FY 2018, all at Community Service Board locations. Using such a method promotes additional education to the network and among assessors in the field, further strengthening expertise and collaboration. Each joint review lends valuable information and feedback through joint discussions and exit conferences for providers and assessors alike. Additionally, providers are supplied an additional opportunity to complete a satisfaction survey following the process. Feedback obtained from the satisfaction surveys provides a qualitative approach to individual and staff perceptions related to review processes and yielded positive results in FY 2018. Quality Management will continue to use feedback from the surveys to review processes and provide professional, effective, and constructive approaches to ensure providers are equipped with the necessary tools and opportunities they need to best support individuals. Provider trainings in FY 2019 are to occur both in person as well as via webinar through the formalized Quality Training Program based on collaboration and partnership with DBHDD. Quality Management shall use findings from the behavioral health, CSU, and intellectual and developmental disability reviews to recommend topics for trainings. Such trainings will be prioritized based on identified system needs. Additionally, as the collaboration and partnership continues, changes to 155 | P a g e Quality Management Annual Report FY 2018 measurement tools to further align indicators is both recommended and required for ongoing smooth review processes. 156 | P a g e Appendix A: Abbreviations and Acronyms Quality Management Annual Report FY 2018 Acronym ACT AD BH BHQR C&A CIS CMS CST CSU CSUQR DBHDD DDSS FOA FY II IDD IFI IOP IOSA IRR IRP ISP QA ISP KPI MAR MH N NA NCI NCP OBS PCP PCR PHI PRR QEPR Definition Assertive Community Treatment Addictive Diseases Behavioral Health Behavioral Health Quality Review Child and Adolescent Consumer Information System Centers for Medicaid and Medicare Services Community Support Team Crisis Stabilization Unit Crisis Stabilization Unit Quality Review Department of Behavioral Health and Developmental Disabilities Developmental Disability Service Specific Focused Outcome Area(s) Fiscal Year Individual Interview Intellectual and Developmental Disability Intensive Family Intervention Intensive Outpatient Individual Observation Staff Assessment Individual Record Review Individual Recovery / Resiliency Plan Individual Service Plan Quality Assurance Checklist Individual Service Plan Key Performance Indicator Medication Administration Record Mental Health Number in sample Not Applicable National Core Indicators Nursing Care Plan On-site Observations Person Centered Practices Person Centered Review Protected Health Information Provider Record Review Quality Enhancement Provider Review 157 | P a g e Q&T QTAC SC SCI SCRR SFS SI SIS SU TA Qualifications and Training Quality Technical Assistance Consultation Support Coordinator Support Coordinator Interview Support Coordinator Record Review State Funded Services Staff Interview Supports Intensity Scale Substance Use Technical Assistance Quality Management Annual Report FY 2018 158 | P a g e Appendix B: Score Distributions Quality Management Annual Report FY 2018 Graphical distributions for each category show the percent of providers who scored within the specified range of scores, as indicated on the horizontal/x-axis. With each distribution, the median, mean, and mode may be provided. The median represents the score that falls in the middle of distribution (50th percentile) and the mean represents the average score. Standard deviation refers to the amount of variation or dispersion there is in a distribution of scores, or how much scores tend to spread-out from the mean. Standard deviations are sometimes used to determine significant differences between scores within a distribution; however, this application requires a distribution to be normally distributed (similar to a bell curve). Most of the distributions presented in this report do not meet the requirements of a normal distribution, meaning they do not look like a normal bell curve; therefore, standard deviations have been excluded from the report at this time. Table of Contents (Links available by clicking on Figure Name) Behavioral Health Quality Review Distributions BHQR Overall Score Distribution BHQR Tier 1 Overall Score Distribution BHQR Tier 2 Overall Score Distribution BHQR Tier 2+ Overall Score Distribution BHQR Tier 3 Overall Score Distribution BHQR Billing Score Distribution BHQR Assessment & Treatment Score Planning Distribution BHQR Service Guidelines Score Distribution BHQR Focused Outcome Areas Score Distribution Assertive Community Treatment (ACT) Quality Review Distributions ACT Overall Score Distribution ACT Billing Score Distribution ACT Assessment & Planning Distribution ACT Service Guidelines Score Distribution ACT Focused Outcome Areas Score Distribution 159 | P a g e CSU Quality Review Distributions CSU Overall Score Distribution CSU Service Guidelines Score Distribution CSU Focused Outcome Areas Score Distribution CSU Individual Record Review Score Distribution Quality Management Annual Report FY 2018 Reassessment BHQR Distributions BHQR Overall Score Distribution of Reassessed Providers BHQR Billing Score Distribution of Reassessed Providers BHQR Assessment & Treatment Planning Distribution of Reassessed Providers BHQR Service Guidelines Score Distribution of Reassessed Providers BHQR Focused Outcome Areas Score Distribution of Reassessed Providers PCR Distributions: Scores by Tool and Year PCR Individual Interview Scores by Year PCR Observation Scores by Year PCR Staff Interview Scores by Year PCR Provider Record Review Scores by Year PCR Support Coordinator Review Scores by Year PCR Support Coordinator Interview Scores by Year QEPR Distributions: Scores by Tool and Year FY 2018 Overall QEPR Score FY 2018 QEPR Individual Interview Scores FY 2018 QEPR Observation Scores FY 2018 QEPR Staff Interview Scores FY 2018 QEPR Provider Record Review Scores FY 2018 QEPR Qualifications and Training Scores FY 2018 QEPR Service Discrepancy Scores FY 2018 QEPR Overall Scores by Provider Size FY 2018 QEPR Qualifications and Training Scores by Provider Size 160 | P a g e Quality Management Annual Report FY 2018 BHQR Overall Score Distribution by Year 60% 50% 51%49% 47% 42% 40% 30% 26% 25% 20% 16%19% 10% 5% 1% 0% 1%1%1%2%2%2% 3% 3% 6% 0% FY 2016 (N = 141; Mean = 84%; Median = 87%) FY 2017 (N = 167; Mean = 84%; Median = 86%) FY 2018 (N = 156; Mean = 88%; Median = 90%) BHQR Tier 1 Overall Score Distribution by Year 80% 70% 68% 60% 58% 50% 48% 48% 40% 30% 21% 25% 20% 17% 11% 10% 3% 0% FY 2016 (N = 24; Mean = 86%; Median = 88% ) FY 2017 (N = 28; Mean = 85%; Median = 85%) FY 2018 (N = 29; Mean = 88%; Median = 90%) 161 | P a g e 60% 50% 40% 30% 20% 10% 0% 100% 80% 60% 40% 20% 0% Quality Management Annual Report FY 2018 BHQR Tier 2 Overall Score Distribution by Year 51%50% 49% 41% 19%19% 24%24% 1% 2%1% 4%5%2% 6% FY 2016 (N = 90; Mean = 85%; Median = 87%) FY 2017 (N = 115; Mean = 85%; Median = 86%) FY 2018 (N = 93; Mean = 89%; Median = 90%) BHQR Tier 2+ Overall Score Distribution by Year 83%83% 80% 20% 17%17% FY 2016 (N = 5; Mean = 88%; Median = 87%) FY 2017 (N = 6; Mean = 86%; Median = 85%) FY 2018 (N = 6; Mean = 87%; Median = 86%) 162 | P a g e Quality Management Annual Report FY 2018 BHQR Tier 3 Overall Score Distribution by Year 60% 56% 50% 41% 43% 40% 30% 29%27% 20% 10% 5% 14% 5%6%7% 6%7% 11% 7% 9%11%7% 11% 0% FY 2016 (N = 22; Mean = 78%; Median = 83%) FY 2017 (N = 18; Mean = 83%; Median = 91%) FY 2018 (N = 28; Mean = 83%; Median = 89%) BHQR Billing Score Distribution by Year 70% 60% 53% 50% 48% 42% 40% 33% 30% 28%26% 20% 10% 4%1%1% 1%2%2% 1%1%1% 1%1%1% 2%2%1% 1%4%1% 5%5%3%10%8%11% 0% FY 2016 (N = 141; Mean = 81%; Median = 88%) FY 2017 (N = 167; Mean = 84%; Median = 90%) FY 2018 (N = 156; Mean = 85%; Median = 92%) 163 | P a g e Quality Management Annual Report FY 2018 BHQR Assessment & Treatment Planning Score Distribution by Year 50% 40% 40% 30% 31% 31%31% 30% 29% 23% 21% 20% 16% 17% 10% 4%5% 7% 9% 1%1%1% 1%1%1% 2% 0% FY 2016 (N = 141; Mean = 79%; Median = 79%) FY 2017 (N = 167; Mean = 77%; Median = 78%) FY 2018 (N = 156; Mean = 84%; Median = 86% ) BHQR Compliance with Service Guidelines Score Distribution by Year 70% 60% 58% 62% 53% 50% 40% 30% 34%31%29% 20% 10% 1% 1% 1%1% 1%1% 1%7%4% 6%7%3% 0% FY 2016 (N = 140; Mean = 90%; Median = 92%) FY 2017 (N = 167; Mean = 88%; Median = 91%) FY 2018 (N = 156 ;Mean = 90%; Median = 93%) 164 | P a g e Quality Management Annual Report FY 2018 BHQR Focused Outcome Areas Score Distribution by Year 80% 67% 60% 47% 50% 41% 40% 30% 28% 20% 17% 2% 4%2%2% 8%3% 0% FY 2016 (N = 141; Mean = 85%; Median = 87%) FY 2017 (N = 167; Mean = 89%; Median = 90%) FY 2018 (N = 156 ; Mean = 92%; Median = 93%) ACT Overall Score Distribution by Year 60% 50% 47% 48% 50%48% 42% 40% 30% 26% 25% 20% 11% 10% 5% 0% FY 2016 (N = 19; Mean = 88%; Median = 91%) FY 2017 (N = 20; Mean = 87%; Median = 91%) FY 2018 (N = 21; Mean = 89%; Median = 90%) 165 | P a g e Quality Management Annual Report FY 2018 ACT Billing Score Distribution by Year 70% 67% 60% 60% 58% 50% 40% 37% 30% 20% 20% 10% 15% 19% 5%5%5% 10% 0% FY 2016 (N= 19; Mean = 92%; Median = 96%) FY 2017 (N = 20; Mean = 90%; Median = 96%) FY 2018 (N = 21; Mean = 91%; Median = 95%) ACT Assessment & Planning Score Distribution by Year 70% 60% 58% 50% 40% 35% 404%3% 38% 30% 26% 20% 20% 19% 16% 10% 5% 0% FY 2016 (N = 19; Mean = 85%; Median = 80%) FY 2017 (N = 20; Mean = 80%; Median = 80%) FY 2018 (N = 21; Mean = 87%; Median = 87%) 166 | P a g e 60% 50% 40% 30% 20% 10% 0% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Quality Management Annual Report FY 2018 ACT Service Guidelines Score Distribution by Year 50% 33%32% 383%7%35% 29% 21% 11% 10% 5% FY 2016 (N = 19; Mean = 85%; Median = 89%) FY 2017 (N = 20; Mean = 88%; Median = 89%) FY 2018 (N = 21; Mean = 84%; Median = 86%) ACT FOAs Score Distribution by Year 81% 55% 47% 37%35% 16% 19% 10% FY 2016 (N = 19; Mean = 88%; Median = 91%) FY 2017 (N = 20; Mean = 90%; Median = 91%) FY 2018 (N = 21; Mean = 93%; Median = 95%) 167 | P a g e Quality Management Annual Report FY 2018 CSU Overall Score Distribution by Year 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% 47% 43%48% 43% 35% 26% 17% 21% 5%4%5% 5% FY 2016 (N = 19; Mean = 83%; Median = 83%) FY 2017 (N = 23; Mean = 86%; Median = 88%) FY 2018 (N = 21; Mean = 88%; Median = 90%) CSU Compliance with Service Guidelines Score Distribution by Year 80% 76% 70% 61% 60% 50% 40% 32% 32% 30% 20% 10% 4% 11% 9% 11% 5% 4%5% 14% 4% 17% 16% 0% FY 2016 (N = 19; Mean = 82%; Median = 80%) FY 2017 (N = 23; Mean = 87%; Median = 91%) FY 2018 (N = 21; Mean = 91%; Median = 100%) 168 | P a g e Quality Management Annual Report FY 2018 CSU Focused Outcome Areas Overall Distribution by Year 60% 57%52% 50% 47% 48% 40% 35% 37% 30% 20% 16% 10% 9% 0% FY 2016 (N = 19; Mean = 88%; Median = 88%) FY 2017 (N = 23; Mean = 91%; Median = 92%) FY 2018 (N = 21; Mean = 91%; Median = 91%) CSU Individual Record Review Score Distribution by Year 70% 65% 60% 50% 42% 40% 30% 37% 29% 26% 33% 29% 20% 11% 10% 5% 4%5% 5%4%5% 0% FY 2016 (N = 19; Mean = 79%; Median = 77%) FY 2017 (N = 23; Mean = 82%; Median = 82%) FY 2018 (N = 21; Mean = 83%; Median = 83%) 169 | P a g e Quality Management Annual Report FY 2018 FY 2018 BHQR Overall Score Distribution of Reassessed Providers 70% 60% 60% 55% 50% 40% 35% 30% 20% 20% 15% 10% 5% 5% 5% 0% First Review FY 2018 (N = 20; Mean = 83%; Median = 84%) Second Review FY 2018 (N = 20; Mean = 91%; Median = 91%) FY 2018 BHQR Billing Score Distribution of Reassessed Providers 60% 55% 50% 40% 40% 35% 30% 25% 20% 10% 10% 10% 10% 5% 5% 5% 0% First Review FY 2018 (N = 20; Mean = 75%; Median = 78%) Second Review FY 2018 (N = 20; Mean = 89%; Median = 92%) 170 | P a g e Quality Management Annual Report FY 2018 FY 2018 BHQR Assessment & Planning Score Distribution of Reassessed Providers 70% 60% 60% 50% 45% 45% 40% 30% 20% 10% 5% 15% 10%10% 10% 0% First Review (N = 20; Mean = 79%; Median = 83%) Second Review (N = 20; Mean = 89%; Median = 89%) FY 2018 BHQR Service Guidelines Score Distribution of Reassessed Providers 70% 65% 60% 50% 45% 40% 40% 35% 30% 20% 10% 5% 5% 5% 0% First Review (N = 20; Mean = 87%; Median = 89%) Second Review (N = 20; Mean = 89%; Median = 89%) 171 | P a g e Quality Management Annual Report FY 2018 FY 2018 BHQR FOAs Score Distribution of Reassessed Providers 70% 65% 60% 50% 50% 40% 40% 35% 30% 20% 10% 10% 0% First Review (N = 20; Mean = 88%; Median = 91%) Second Review (N = 20; Mean = 93%; Median = 95%) PCR Individual Interview Scores by Year 80% 67.8% 70% 66.1% 62.0% 60% 50% 40% 30% 21.6% 20% 5.4% 19.6% 11.0% 18.2% 10.7% 10% 0% 0.2%0.4% 0.8%1.9%3.15%.8% 5.4% FY 2016 (N = 484; Mean =95.1% ; Median = 97.1%) FY 2017 (N = 481; Mean = 91.9%; Median = 93.7%) FY 2018 (N = 484; Mean = 92.2%; Median = 95.1%) 172 | P a g e Quality Management Annual Report FY 2018 PCR Observation Scores by Year 60% 56.5% 51.9% 50% 48.5% 48.8% 41.7% 40% 38.1% 30% 20% 10% 3.90%.42%.76%.11%.4% 0% FY 2016 (N = 412; Mean = 98.4%; Median = 99.5%) FY 2017 (N = 181; Mean =96.8%; Median = 98.7%) FY 2018 (N = 276; Mean =98.2%; Median = 100%) PCR Staff Interview Scores by Year 70% 69.3% 67.8% 67.3% 60% 50% 40% 30% 24.5% 24.4% 20% 10% 0% 0.4% 2.2% 11.1% 6.8% 5.8% 0.9% 0.8% 2.6% 16.0% FY 2016 (N = 482; Mean = 96.4%; Median = 97.7%) FY 2017 (N = 224; Mean = 94.3%; Median = 96.0%) FY 2018 (N = 312; Mean = 95.6%; Median = 97.1%) 173 | P a g e 35% 30% 25% 20% 15% 10% 5% 0% Quality Management Annual Report FY 2018 PCR Provider Record Review Scores by Year 0.2% 26.6% 29.3% 28.4% 29.2% 23.9% 22.5% 18.3% 16.4% 14.1% 9.3% 5.0% 6.2% 1.2% 3.4% 3.9% 0.3% 0.6% 17.7% 18.3% 10.6% 7.3% 5.6% 1.7% 0.2% FY 2016 (N = 356; Mean = 79.0%; Median = 79.1%) FY 2017 (N = 478; Mean = 70.2%; Median = 70.5%) FY 2018 (N = 483; Mean = 74.8%; Median = 75.9%) PCR Support Coordinator Record Review Scores by Year 30% 25% 20% 15% 10% 5% 0% 25.0% 24.3% 24.4% 22.9% 20.2% 21.1% 19.4% 15.8% 15.3% 13.0% 9.6% 11.4% 17.0% 14.5% 9.1% 5.4% 6.8% 5.0% 0.8%0.20%.8%0.40%.8%0.4%1.7%1.5%0.8%0.2%2.7%2.1% 2.3% 2.7% 2.5% FY 2016 (N = 484; Mean = 79.9%; Median = 84.1%) FY 2017 (N = 481; Mean = 73.7%; Median = 76.4%) FY 2018 (N = 484; Mean = 77.4%; Median = 75.9%) 174 | P a g e Quality Management Annual Report FY 2018 PCR Support Coordinator Interview Scores by Year 60% 50% 40% 30% 45.5% 45.1% 39.6% 41.5% 39.0% 33.1% 20% 12.0% 10.6% 10% 5.0% 0.4% 0.4% 0% 0.2% 7.1% 2.5% 7.3% 5.2% 1.0%0.2%1.5% 1.7% FY 2016 (N = 477; Mean = 90.8%; Median = 98.4%) FY 2017 (N = 479; Mean = 83.3%; Median = 96.7%) FY 2018 (N = 484; Mean = 83.5%; Median = 98.4%) FY 2018 Overall QEPR Score 60% 54.5% 50% 40% 32.3% 30% 20% 13.1% 10% 0% (N = 99; Mean = 84.8%; Median = 86.4%) 175 | P a g e Quality Management Annual Report FY 2018 FY 2018 QEPR Individual Interview Scores 80% 70% 68.7% 60% 50% 40% 30% 24.2% 20% 10% 5.1% 1.0% 1.0% 0% (N = 99; Mean = 89.6%; Median = 91.9%) FY 2018 QEPR Observation Scores 80% 75.9% 70% 60% 50% 40% 30% 20% 13.8% 10% 8.0% 2.3% 0% (N = 87; Mean = 96.0%; Median = 97.2%) 176 | P a g e Quality Management Annual Report FY 2018 FY 2018 QEPR Staff Interview Scores 90% 81.6% 80% 70% 60% 50% 40% 30% 20% 10% 10.2% 7.1% 1.0% 0% (N = 98; Mean = 95.3%; Median = 96.3%) FY 2018 QEPR Record Review Scores 40% 30.3% 30% 27.3% 20.2% 20% 14.1% 10% 5.1% 3.0% 0% (N = 99; Mean = 72.4%; Median = 73.4%) 177 | P a g e 50% 40% 30% 20% 10% 0% 100% 80% 60% 40% 20% 0% Quality Management Annual Report FY 2018 FY 2018 QEPR Qualifications and Training Scores 45.5% 1.0% 18.2% 15.2% 5.1% 6.1% 1.0% 8.1% (N = 99; Mean = 84.6%; Median = 90.7%) FY 2018 QEPR Service Discrepancy Scores 91.9% 8.1% (N = 99; Mean = 99.8%; Median = 100.0%) 178 | P a g e Quality Management Annual Report FY 2018 FY 2018 QEPR Overall Scores by Provider Size 80% 73.3% 70% 60% 51.4% 50% 50.0% 40% 37.5% 33.3% 30% 20% 16.7% 13.3% 11.1% 13.3% 10% 0% Small (N = 72; Mean = 86.3%; Median = 87.2%) Medium (N = 12; Mean = 82.0%; Median = 84.6%) Large (N = 15; Mean = 84.4%; Median = 83.5%) 179 | P a g e Quality Management Annual Report FY 2018 FY 2018 QEPR Qualifications and Training Scores by Provider Size 70% 66.7% 60% 50% 44.4% 40% 33.3% 30% 25.0% 25.0% 20.0% 20% 16.7% 16.7% 10% 8.3% 6.7% 6.7% 5.6% 4.2% 1.4% 1.4% 0% 9.7% 8.3% Small (N = 72; Mean = 85.3%; Median = 90.9%) Medium (N = 12; Mean = 81.1%; Median = 83.9%) Large (N = 15; Mean = 85.8%; Median = 83.5%) 180 | P a g e