Annual quality management report, 2014 January 2014-December

Georgia Department of Behavioral Health & Developmental Disabilities
ANNUAL QUALITY MANAGEMENT REPORT January 2014 December 2014
Prepared by the DBHDD Office of Quality Management February 2015

Contents
Introduction..................................................................................................................................... 3 Activities of the Quality Councils................................................................................................... 3
Executive Quality Council.......................................................................................................... 3 Hospital System Program Quality Council................................................................................. 4 Community Behavioral Health Program Quality Council.......................................................... 5 Developmental Disabilities Program Quality Council ............................................................... 5 Status of Quality Management Work Plan Goals ........................................................................... 7 DBHDD QM Work Plan............................................................................................................. 7 Hospital System QM Work Plan................................................................................................. 8 CBH QM Work Plan................................................................................................................... 8 DD QM Work Plan ..................................................................................................................... 8 Key Performance Indicators and Outcomes ................................................................................... 9 Data Collection Plan/Data Definition Document ....................................................................... 9 Dashboards.................................................................................................................................. 9 Hospital System Key Performance Indicators ............................................................................ 9 Community Behavioral Health Program Key Performance Indicators .................................... 10 Developmental Disability Programs Key Performance Indicators ........................................... 14
Administrative Services Organization (ASO) and DD ......................................................... 17 Quality Monitoring Activities....................................................................................................... 18
Complaints and Grievances .................................................................................................. 18 Hospital and Community Incident Data ............................................................................... 20 Hospital Peer Review and Credentialing .............................................................................. 23 Hospital Utilization Review.................................................................................................. 23 Adult Mental Health Fidelity Reviews ................................................................................. 23 Mobile Crisis Response System Performance and Quality Monitoring ............................... 25 QM Audits: Quality Service Reviews of Adult Behavioral Health Community Providers . 25 Child and Adolescent Community Mental Health Programs (CAMH)................................ 27 Division of Addictive Diseases (AD) Quality Management Activities................................ 28 Mental Health Coalition Meetings........................................................................................ 29 Behavioral Health Contracted External Review Organization (ERO) ................................. 30 Provider Network Analysis................................................................................................... 32 Implementation and Results of Best Practice Guidelines: .................................................... 33 Division of Developmental Disability Quality Management Reviews................................. 40
1

Quality Enhancement Provider Reviews (QEPR) ................................................................ 47 DD Transition Quality Review Analysis .............................................................................. 51 Pioneer Project ...................................................................................................................... 51 2014 Specialized DD Quality Improvement Study: Provider Systems and Driver Outcomes ............................................................................................................................................... 51 DBHDD Quality Management Training Program ........................................................................ 52 Data Reliability Process ................................................................................................................ 53 Hospital System KPI Data Integrity ......................................................................................... 53 Community BH Key Performance Indicator Data Integrity ..................................................... 53 DD KPI Data Integrity .............................................................................................................. 54 Summary ....................................................................................................................................... 54 Appendix A DBHDD Quality Management Work Plan............................................................... 55 Appendix B Hospital System Quality Management Work Plan................................................... 59 Appendix C Community Behavioral Health Quality Management Work Plan............................ 63 Appendix D Developmental Disabilties Quality Management Work Plan .................................. 65 Appendix E Hospital System KPI Dashboards............................................................................. 71 Appendix F CBH System KPI Dashboards .................................................................................. 75 Appendix G Developmental Disabilities System KPI Dashboards .............................................. 99
2

Introduction
The Georgia Department of Behavioral Health and Developmental Disabilities (DBHDD) serves as the single state authority for the provision of direct services, administration, and monitoring of all facets of the state publicly funded behavioral health & developmental disabilities service system. DBHDD's role as a direct service provider is limited to the operation of five state hospital campuses. Outpatient services are delivered by a network of private and public providers with whom DBHDD contracts. DBHDD Contractors are community-based organizations which administer behavioral health & developmental disabilities services throughout the state and are responsible for the provision of comprehensive services for children and adults with substance abuse disorders, serious and persistent mental illness (SPMI) and intellectual/developmental disabilities.
This report is DBHDD's 2014 Annual Quality Management (QM) System Report, the purpose of which is to provide a summary of the quality-related activities that have taken place across DBHDD's hospital, community behavioral health and developmental disabilities systems of care during 2014. This is DBHDD's third annual review of its Quality Management system. Because there is a lag time associated with the availability of some data, the analysis and discussion contained within this report will vary somewhat by date range, but generally focuses on activities between January 2014 and December 2014. This report is made available to Department staff and other stakeholders.
The Department's Quality Management Plan was last revised in April 2013 and provided detailed information about the current organizational structure of the Quality Management Program, a description of the Executive and Program Quality Councils and the goals and objectives of each council. This QM plan can be found at: http://dbhdd.georgia.gov/sites/dbhdd.georgia.gov/files/related_files/document/QM%20PlanApril%202013%20rev.pdf The next scheduled update of the DBHDD Quality Management Plan is in the spring of 2015.
Activities of the Quality Councils
Executive Quality Council The Executive Quality Council (EQC) meets six times per year, acts as the governing body for the QM program and is the ultimate authority for all DBHDD QM activities. During 2014 the EQC met in January, March, May, July, September and December.
During those meetings the EQC: Performed its annual review of the QM system. Specified the information that should be reported to the EQC. Participated in the planning for the re-engineering of the DBHDD I/DD service system. Reviewed and monitored the Office of Incident Management and Investigation's (OIMI) trends and patterns. Received updates from the Hospital, Community Behavioral Health and I/DD Program Quality Councils (PQCs) regarding the quality management-related work that each
3

functional area prioritized and reviewed trends/patterns from their Key Performance Indicators (KPIs). Received an update and discussed the setting of CHB PQC recovery oriented KPIs. Received an update and discussed the Hospital System CRIPA Transition Plan. Received updates from the Settlement Director regarding DBHDDs compliance with the Settlement Agreement. Prioritized the development of and received updates regarding the progress of a PI project related to corrective action plans, performance improvement and remedies for poorly performing and non-compliant community providers. Received updates and monitored the regulatory compliance at the East Central/ Gracewood campus. Discussed the role and integration of the Administrative Services Organization (ASO) into the DBHDD system of care. Reviewed and approved a proposed revision of the DBHDD QM framework in 2015 to align with the National Quality Strategy (http://www.ahrq.gov/workingforquality/.
Hospital System Program Quality Council The Hospital System PQC was on a quarterly meeting schedule during 2014, and held three meetings between January 2014 and December 2014 (one meeting was canceled). In addition to those quarterly meetings, the Hospital System held monthly Hospital System-wide Quality Managers meetings (joint meetings of the hospital quality management staff) to monitor and address patient safety and other performance measures. During those meetings this PQC:
Continued the review/monitoring of PI initiatives focused on management of aggression, restraint and seclusion, polypharmacy, consumer satisfaction and other performance measures.
Continued the review/monitoring of and modified strategies being utilized by hospitalbased PI teams to improve patient safety.
Addressed data collection methodologies and data integrity issues that affected reporting timeliness and quality.
Reviewed and discussed the Triggers and Thresholds report data, the Hospital System Dashboard measures and specific hospital system KPI trends and patterns and made suggestions/recommendations for program/service changes. Reviewed existing KPI measures and developed a list of proposed KPIs for consideration by the Hospital System Program Quality Council.
Worked to improve corrective action plans and assure better cause identification and descriptions of methodologies for improving the effectiveness of corrective actions.
Collaborated with the Office of Incident Management and Investigations to improve investigations and reports so that there is more consistent consideration of root causes of incidents, and to link any process or systemic issues identified into the Quality Management System.
Established a Hospital System Information Management Committee. Established a Health Information Management Committee.
For 2015 the Hospital System PQC will move to a bi-monthly schedule and continue to hold the monthly Quality Managers' meeting to support the quality management program.
4

Community Behavioral Health Program Quality Council The Community Behavioral Health PQC was scheduled to meet monthly and held nine meetings between January 2014 and December 2014. During those meetings the CBH PQC:
Reviewed and discussed the selected incident trends and patterns for community based providers.
Reviewed and discussed the results, trends and/or patterns of the CBH KPIs and as a result of those reviews: o modified some of the target thresholds o determined additional KPIs that needed to be developed and/or revised o made suggestions/recommendations for program/service changes
Discussed and recommended recovery-oriented and suicide prevention KPIs. Reviewed and discussed the results of a statewide Deaf Services' survey. Received an update/overview of the Child and Adolescent program's quality
management system. Received regular updates regarding the findings of the fidelity reviews (for Supported
Employment and Assertive Community Treatment). Received an update regarding the work of the Suicide Prevention Program. Discussed and recommended solutions to assist with improving the integrity of the data
submitted to DBHDD by community BH providers. Reviewed and discussed transition reports received from the Office of Transition
Services Reviewed and discussed the 2013 Adult and Youth Consumer Satisfaction Survey
Reports. Discussed the preparation and integration of CBH quality related components into the
DBHDD ASO.
Developmental Disabilities Program Quality Council The Developmental Disabilities PQC met quarterly during 2014. Outcomes of those meetings include:
A review of trends from Person Centered Reviews (PCR) and Quality Enhancement Provider Reviews (QEPR).
Advised DD staff on the development of protocols and guidelines for obtaining proper Informed Consent.
Developed a project to educate individuals and families on "choice" and making informed choices.
Initiated a project to develop a DD QI Council Communication Plan to share information between State and Regional QI Councils and the community at large.
DD Quality Improvement Councils and DD Advisory Council The Division of DD has six regional and one statewide quality improvement councils. The role of the Quality Improvement (QI) Councils is to review and analyze data for developing service improvement targets and tracking progress. Data sources that are available to the QI Councils include data collected by the DD ERO (Delmarva), such as, the National Core Indicator (NCI) surveys, Person Centered Reviews (PCR), Quality Enhancement Provider Reviews (QEPR), and
5

other data sets. Because of their unique positions within the system, members of the QI Councils are in a position to identify gaps and problems with existing services and most importantly, then use this data, and what it identifies, to make system changes at local, regional and state levels. The QI Councils are active partners in quality improvement efforts of the Division of DD.
The Regional and Statewide QI Councils met at least quarterly during 2014. All the Councils convened in October for their annual joint conference. Data from the FY14 Quality Assurance Report was shared and discussed with the Councils. Each Council had a chance to begin developing their 2015 work plans based on their respective regional data. Additionally, each Council presented on the quality improvement projects that they completed in FY14. Examples of those presentations can be found at: http://www.dfmcgeorgia.org/quality_improvement_council/project_plan_presentations/index.html.
The Statewide Quality Council met quarterly during 2014. In partnership with the Division of DD, the Statewide Quality Council began work on the development of a QI Council Communication Plan. This plan will improve the dissemination of information regarding the activities of the Statewide and Regional Councils. The plan will also improve communication between the Councils themselves. An invitation was extended to all Regional QI Council CoChairs to become members of the Statewide QI Council. The Regional Co-Chairs attended the December 2014 Statewide Quality Council meeting.
The Statewide QI Council continued to provide support to the Division concerning the Transition Plan for the Home and Community Based Waivers. Support included education of community stakeholders and providers concerning the plan and how the Division of DD would be collecting data for the plan.
In 2014, the Division of DD also implemented the DD Advisory Council. The purpose of the DD Advisory Council is to advise the Department on matters related to the care and service of people with intellectual/developmental disabilities served by the Department. The Council has been tasked:
To assist the Division of DD in assuring the Department's services to people with developmental disabilities reflect adherence to the standard of "best practice."
To assist the Division in assuring the Department's programs for people with developmental disabilities provide quality services in a cost effective manner.
To recommend improvements to the Division for existing programs serving people with developmental disabilities.
To recommend development and implementation of additional programs for people with developmental disabilities in Georgia.
To review the Department's policy, policy revisions, and make recommendations regarding the adherence to the Department's mission and the cost of proposed policies and amendments.
To facilitate communication among Department staff, providers of services, service recipients, parents/guardians/advocates of people with developmental disabilities, and
6

other public and private entities involved in delivering services to people with developmental disabilities.
The Advisory Council met bi-monthly with Division staff and other stakeholders. Please see Attachment 1: DD Advisory Council Year-End Report for a summary of the Council's 2014 accomplishments. The DD Advisory Council continues to meet bi-monthly.
Status of Quality Management Work Plan Goals
Each Program Quality Council maintains a work plan to guide the quality management activities within its area of responsibility. The EQC oversees the development of the DBHDD QM work plan, and then the Program Quality Councils develop program-specific work plans for the hospital system, the community behavioral health, and developmental disabilities service delivery systems.
Below are descriptions of the status of each functional area's work plan and the progress toward achieving the work plan goals for each Quality Council:
DBHDD QM Work Plan As of December 2014 the DBHDD QM Plan and work plans were in the process of review and revision. During 2014 the 2013 QM Plan (with some revisions) and work plans have continued to be used.
Goal 1. The first goal related to developing accurate, effective and meaningful performance measures has been met and will continue to be reviewed and updated on an annual basis. The next review is scheduled to start in January of 2015. The second task of the first goal requires obtaining input from stakeholders to develop the KPIs. This was addressed during quality management-related discussions at the community based consortium meetings, regular meetings with the Georgia CSB Association's Benchmarking Committee and through DD quality management meetings.
Goal 2. The second goal is related to the education of stakeholders regarding QM. As of August 2014, the DBHDD QM Learning Plan was being updated and when finalized will be included in a revised QM Plan. In May of 2014, the second in the series of QM web-based training modules was released to all DBHDD staff and was completed in July 2014. New training materials will be developed during 2015 for internal and external stakeholders to provide education about alignment of DBHDD's quality management system to the National Quality Strategy.
Goal 3. The third goal is a multi-year goal and is related to assessing and improving the effectiveness of the QM system and its components. A new framework based, upon the National Quality Strategy has been proposed and approved by the EQC. At the time of this report implementation strategies for the new framework were being discussed.
Goal 4. The fourth goal, related to the QM Data Systems is again a multi-year goal. The completion of a data management needs assessment was begun in June of 2014 is on target and will have a comprehensive statement of work. The second task of this goal was to develop data sharing partnerships with other state agencies and has been completed. The third task, related to
7

the creation of an Enterprise IT and Information Systems Improvement Plan, is on target. All other tasks have either been met or on target for completion. A joint DBHDD - Georgia Collaborative ASO workgroup was created and work started in the fall to define the requirements for collection of quality indicators and the content of quality review tools. This work will continue throughout 2015 as the new ASO is implemented and data collection and reporting begins.
The following are summaries of the activities related to each PQC's QM work plan which support the goals of the DBHDD's QM Work Plan. See Appendix A.
Hospital System QM Work Plan The Hospital System QM Work Plan (see Appendix B) represents a high level set of goals focused on the Quality Management infrastructure needed to maintain an effective quality management system. The overarching purpose of these goals is to refine the quality management system so that there is greater consistency, accuracy, data integrity and accountability. These goals reflect the Hospital System's dedication to developing and maintaining the capacity to improve quality and do so efficiently, effectively, and in a way that maximizes the utilization of its resources.
The Hospital System is working to maintain and improve quality as it assists in DBHDD's strategic direction toward building community-based services while reducing its dependence on state hospitals. As the System's hospitals are reduced in size, closed and/or repurposed, it is essential that an effective quality management system is maintained so that those transitions are managed in a way that assures the consumers receive the quality of service that they deserve. At the time of this report, the first goal related to developing accurate, effective and meaningful performance measures has been met. Modifications have been made to the second goal secondary to a change in strategy. The focus of the third goal was shifted in response to developing system-wide data collection plans as opposed to each hospital creating their own. Additionally there have been target completion date revisions to the fourth goal as a result of a new proposal and statement of work being drafted.
CBH QM Work Plan Although there were some delays due to competing priorities, the majority of the tasks that were to be completed have been completed. The progress towards the remainder of the goals is consistent with the plan. See Appendix C for the CBH QM Work Plan.
DD QM Work Plan Many tasks were accomplished by their initial completion dates; however, some timelines required adjustment. The adjustments allow additional time for more thorough planning and development of an updated DD quality management system. In 2014, the Division of DD began a project to re-engineer how I/DD functions at both a systemic and support provision level. The Division formed the following four workgroups:
Support Coordination: reviewed present support coordination responsibilities, and developed a "Pioneer Project" to improve the quality of transitions from State Hospitals to the Community. More information on the Pioneer Project can be found under "DD Transition Quality Review Analysis". Additionally, a program was developed for the improvement of Support Coordination which was entitled "Enhanced Support
8

Coordination." This was to include pre-transition and post-transition activities. A key component of this was early engagement by Support Coordination and also included broader service delivery post-transition. A new model of monitoring identified as "Recognize and Refer" was used to encourage collaboration and improvement of service delivery versus punitive ratings. Referrals could be of the clinical or nonclinical nature. Continuous Quality Improvement: Reviewed current QM practices, developed I/DD Performance Indicators with the input from external and internal stakeholders; developed a Mortality Review Process and Report that will be disseminated in 2015; assessed current data collection protocols. Competency-Based Training: Reviewed current training practices; assessed training needs; provided training supports to the five workgroups, plus regional and state staff. Individual and Community Supports: Conducted quarterly sample reviews of transitions that have occurred utilizing standardized performance assessment tools; develop an efficient process to ensure funding transfers for community placements; analyze trends in provider data are used to determine key courses of action to be taken by Performance Management Unit or other relevant units.
See Appendix D for the DD work plan.
Key Performance Indicators and Outcomes
Data Collection Plan/Data Definition Document The DBHDD data definition document was developed for the KPIs, for use by each of the three functional QM areas within the Department. The data definition document which was developed in 2013 provides guidance on how each element and attribute should be used. It gives details about the structure of the elements and format of the data. Additionally, this document was used as the basis to develop a tool (called the Performance Measure Evaluation Tool) which provides guidance on developing new and evaluating existing KPIs.
Dashboards The KPI dashboard format incorporates the KPI data in table and graph form and includes measure definition & explanation, numerator & denominator explanation and an analysis of the KPI for the time period. The KPI dashboards can be found in Appendices E, F and G.
Hospital System Key Performance Indicators The KPIs utilized by the Hospital System are a combination of quality measures that support the System's value of three priority areas:
1. The use of consumer feedback to reflect the quality of our services. a. Client Perception of Outcome of Care i. Summary comments and analysis: The DBHDD Hospital System facilities have consistently scored higher than the baseline established on the basis of the national averages for the same survey tool. The Quality Management departments at each facility explored ways to improve the consistency and timeliness of reporting and the consistency and quality of the methods of administration of the survey instruments. As evidence, during 2014, a new process of submitting
9

data was implemented, reducing the lag time in reporting. Although the rate is observed to vary from month-to-month, this is not abnormal when compared to national rate averages. b. Client Perception of Empowerment i. Summary comments and analysis: The DBHDD Hospital System facilities have consistently scored higher than the baseline established on the basis of the national averages for the same survey tool. The Quality Management departments are exploring ways to improve the consistency and timeliness of reporting and the consistency and quality of the methods of administration of the survey instruments. As evidence, during 2014, a new process of submitting data was implemented, reducing the lag time in reporting. The 12 month trend for this KPI is nearly flat, but DBHDD scores remain consistently above the standard set. 2. The importance of continuity of care with regard to the transition of consumers between hospital and community services. a. Continuing Care Plan Created (Overall) i. Summary comments and analysis: The hospital system, as a whole, continues to perform well above The Joint Commission target rate. Several instances of non-compliance in 2014 were attributed to clients that have been discharged while on Conditional Release. These discharges comply with court orders, but often lack the required paperwork needed to achieve a completed Continuing Care Plan. These clients were typically on Conditional Release for several years prior to discharge, and standards to comply with an approved Continuing Care Plan have changed since being placed on Conditional Release. The Joint Commission is aware of this issue and it only affects a fraction of the discharged population. 3. The importance of supporting the recovery of individuals receiving hospital services. a. Individual Recovery Plan Audit - Quality Measure i. Summary comments and analysis: Rates continued to improve during the first 8 months of the year, but dropped slightly from August to December due to scores at a single hospital. That hospital is in the process of researching and retraining staff to ensure scores improve.
Summary and Recommendations: Hospital System Results from Hospital System KPIs have consistently remained high. The Hospital System plans to continue to monitor and improve the quality of care measured by these KPIs and has done considerable work on developing new measures that will present further opportunities for improvement. The hospital system dashboard can be found in Appendix E.
Community Behavioral Health Program Key Performance Indicators The KPIs utilized by the CBH Programs are a combination of quality measures that support the Department's vision and measure quality for each program/service outlined below.
10

Summary and Recommendations for the current CBH KPIs: 1. Georgia Housing Voucher Program adult individuals with serous and persistent mental illness (SPMI) in stable housing. Summary comments and analysis: The number of individuals receiving Georgia Housing Vouchers who are in stable housing has significantly exceeded the HUD standard of six months and DBHDD's target of 77% for the January 2014 to December 2014 time period, and appears to be stable at approximately 92%. 2. Georgia Housing Voucher Program adult individuals with SPMI who left stable housing under unfavorable circumstances and have been reengaged and reassigned vouchers. Summary comments and analysis: DBHDD tracks Georgia Housing Voucher individuals who left stable housing under unfavorable circumstance and were reengaged in services. The target, set at 17% has been met for this time period and will continue to be monitored. 3. Adult Mental Health supported employment providers that met a caseload average on the last day of the calendar month of employment specialist staff to consumer (between 1:15 to 1:20) . Summary comments and analysis: Although the target of 85% or more was not met during this reporting period, analysis reveals that several providers had lower ratios than 1:15. This means that those providers had smaller caseloads per staff member. The CBH PQC discussed this indicator and determined that if providers have a smaller ratio, that is not detrimental to the consumer, therefore this measure ended on 6/30/14 and was replaced with a target ratio not to exceed 1:20 starting on 7/1/14. Once revised, the target threshold was met for July then exhibited some sub-threshold variability during August through November. 4. Percent of supported employment consumers who were employed on the last day of the calendar month. Summary Comments and analysis: This KPI was initiated in July of 2014 with a threshold set at 43% which was met between July and November 2014. 5. Individuals who had a first contact with a competitive employer within 30 days of enrollment. Summary comments and analysis: The overall percentage of consumers who had first contact has continued to increase and exceeded its target of 75% during the July-September 2014 quarter by 5%. This measure is analyzed on a 30 day lag and October 2014 December 2014 data was not available for analysis as of the date of this report. 6. Assertive Community Treatment consumers who are received into services within 3 days of referral. Summary comments and analysis: The target of 70% was met during the months of May, July, October and November but the data displayed varying percentages. Overall there appears to be an upward trend. This KPI will continue to be monitored. 7. Assertive Community Treatment consumers with a Psychiatric Inpatient admission within the past month. Summary comments and analysis: The target of 7% or less was not met for this reporting period and hospital utilization appears to be holding steady. This may be
11

due to the fact that some teams have reported that unstable housing has been contributing to the psychiatric admissions. 8. Average number of jail/prison days utilized per enrolled Assertive Community Treatment consumer. Summary comments and analysis: Overall the target of 1 day or less was met for all months during this reporting period except for March and November 2014 which minimally exceeded the threshold. 9. Intensive Case Management consumers with a Psychiatric Inpatient admission within the past month. Summary comments and analysis: For this reporting period overall the target of 5% or less was met for the months of February and May. There is some variability exhibited for the other months but the percentages generally appear to be consistent with previous quarters. 10. Intensive Case Management consumers housed (non-homeless) within the past month. Summary comments and analysis: Overall the target of 90% or more was met during this reporting period. 11. Average number of jail/prison days utilized per enrolled Intensive Case Management consumer. Summary comments and analysis: Except for April of 2014 the overall target of .25 days or less was not met for this reporting period. This KPI will continue to be monitored. 12. Community Support Teams with a Psychiatric Inpatient admission within the past month. Summary comments and analysis: Overall the target of 10% or less was met during this reporting period. 13. Community Support Team consumers housed (non-homeless) within the past month Summary comments and analysis: Overall the target of 90% or more was met during this reporting period. 14. Average number of jail/prison days utilized per enrolled Community Support Team consumer. Summary comments and analysis: Overall the target of 0.75 days or less was met during this reporting period. The exception is the month of May 2014 which shows a slight upward trend. 15. Case Management consumers with a Psychiatric Inpatient admission within the past month. Summary comments and analysis: Overall the target of 5% or less was met during this reporting period. 16. Case Management consumers housed (non-homeless) within the past month Summary comments and analysis: Overall the target of 90% or more was met during this reporting period. 17. Average number of jail/prison days utilized per enrolled Case Management consumer Summary comments and analysis: Overall there continues to be some variability in the average number of jail/prison days utilized during this time; which was met during the months of April, July, August, September and October 2014. The overall average by quarter appears to be consistent or slightly better with previous quarters.
12

18. Adult Addictive Disease consumers active in AD treatment 90 days after beginning noncrisis stabilization services. Summary comments and analysis: This KPI became effective in July 2013 and is collected on an annual basis. The target of 25% was met for 2014.
19. Adult Addictive Disease consumers discharged from crisis or detoxification programs who receive follow-up behavioral health services within 14 days. Summary comments and analysis: This KPI became effective in July 2013 and is collected on an annual basis. The target of 35% was almost met with the 2014 annual percent at 34.6.
20. Individuals meeting Settlement Agreement criteria who are enrolled in settlement funded services who state they are satisfied with the services they are receiving. Summary comments and analysis: Data collection was put on hold during this reporting period secondary to the QM audit team performing a follow-up quality review of a sample of individuals with repeated inpatient hospital re-admissions and high utilizers of crisis services. Conclusions could not be drawn from the few surveys that were completed during this reporting period.
21. Individuals meeting Settlement Agreement criteria who are enrolled in settlement funded series who feel their quality of life has improved as a result of receiving services. Summary comments and analysis: Data collection was put on hold during this reporting period secondary to the QM audit team performing a follow-up quality review of inpatient hospital re-admissions and high utilizers of crisis service. Conclusions could not be drawn from the few surveys that were completed during this reporting period.
22. Percent of youth with an increase in functioning as determined by a standardized tool. Summary comments and analysis: The Department is transitioning from the Child and Adolescent Functional Assessment Scale (CAFAS) to the Child and Adolescent Needs and Strengths (CANS). The implementation of the CANS is scheduled for April 2015. Data collection for this KPI will begin in FY16.
23. Percent of families of youth satisfied with services as determined by a standardized tool. Summary comments and analysis: This data is collected and analyzed on an annual basis. In 2014, 84% of families of youth were satisfied with the community mental health services they received which exceeded the target of 80%.
Summary and Recommendations: Community Behavioral Health During 2014 development and refinement of the quality management program continued for the Community Behavioral Health Programs. This included review and modification of existing KPIs, development of new KPI's, didactic communication with providers through the coalition meetings regarding KPIs & quality, and continued collaboration with the Georgia Association of Community Service Boards. Also the KPIs were used by community behavioral health leadership to systemically review the services being provided by the behavioral health provider network and identify opportunities for change and modification. Additionally there has been significant work with regard to developing quality related transition plans for the implementation of the DBHDD ASO (Georgia Collaborative). The Community Behavioral Health dashboard can be found in Appendix F.
13

Developmental Disability Programs Key Performance Indicators The time period for data collection and analysis presented below was January 1, 2014 through November 30, 2014. Data collected in December 2014 was not available at the time of the writing of this report; but will be included in the 2015 Interim Report.
The current key performance indicators are used to help the Division of DD determine: The level at which individuals are receiving person centered supports and services The level of community integration The quality of transitions from State Hospitals to the Community Whether individuals are healthy and safe The efficiency of specific DD services
(See Appendix G for the DD Programs dashboards).
In July 2014, the Division of DD convened a stakeholder work group to develop quality Outcome and Performance Indicators. The indicators focus on the quality of services provided by DD Providers and the Division itself. DD will use some of these indicators as KPIs for Providers and the DD system itself. At the time of this report, the indicators were being finalized. Examples of draft outcomes and indicators include:
Outcome: People have timely access to needed services Performance Indicator: Average number of days between approval of a Prior
Authorization and services beginning Outcome: People are Connected to their Community Performance Indicator: Proportion of individuals who have established at least one non-
paid/non-family community relationship.
The finalized indicators will be discussed in the 2015 Interim Report.
Person Centered Supports Please refer to the Section entitled: "DD Reviews of Individuals Served" for additional information on Person Centered Supports, Individual Support Plan Quality Assurance, and DD Transitions of Individuals into the Community.
Implementation of New Individual Support Plan Process and Template Implementation of the new ISP process and template was placed on hold until the Georgia Collaborative ASO was secured. Division of DD staff are working with Collaborative staff to finalize the new ISP. The template will be built-in to the new DBHDD data system. Implementation is scheduled to begin July 1, 2015.
The new ISP will assure a more person-centered approach to developing supports for an individual, and should lead to improved community integration. A training curriculum and ISP Guide has been developed, and training will begin in the spring of 2015.
14

Health and Safety The Division of DD utilizes the National Core Indicator Survey to gather, directly from individuals and their families, information about their level of satisfaction with the services and supports they are receiving; and to gather additional data on the health and safety of the those individuals. Additional health and safety information is gathered from the independent reviewer as well as reviews performed by the Regional Offices.

Key indicators that have been reviewed include vaccines, dental examinations, annual physicals, and the perception of safety and dignity.

The National Core Indicators are organized by "domains" or topics. These domains are further broken down into sub-domains, each of which has a statement that indicates the concerns being measured. Each sub-domain includes one or more "indicators" of how the state performs in this area.

Domain

Sub-Domain

Concern Statement

Individual Outcomes
Health, Welfare, and Rights

Work

People have support to find and maintain community integrated employment.

Community Inclusion
Choice and Decision-Making

People have support to participate in everyday community activities.
People make choices about their lives and are actively engaged in planning their services and supports.

Self Determination
Relationships Satisfaction Safety Health Medications Wellness

People have authority and are supported to direct and manage their own services.
People have friends and relationships. People are satisfied with the services and supports they receive. People are safe from abuse, neglect, and injury. People secure needed health services. Medications are managed effectively and appropriately. People are supported to maintain healthy habits.

15

System Performance

Respect/Rights Service Coordination

People receive the same respect and protections as others in the community.
Service coordinators are accessible, responsive, and support the person's participation in service planning.

Access

Publicly-funded services are readily available to individuals who need and qualify for them.

Georgia is performing at or above the National Average in most of the sub-domains.
For example: Georgia is above National Average in Choice (94% vs 86%), Work (89% vs 81% reporting having a community job), and Safety (94% vs 87 %) individuals reporting rarely feeling afraid or scared at their work or day program and (87% vs 81%) individuals reporting they rarely feel afraid or scared in their home.
Georgia is performing below the National Average in the areas of individuals who reported that they are self-directing their services (2% vs 11%); had Vision-Hearing Screenings in the last year (49% vs 57% Vision and 51% vs 56% Hearing); and always have a way to get places when they want to go somewhere (75% vs 83%)
The Division and the Division's QI Councils (regional and state) consistently use NCI data to drive quality improvement initiatives. For example; regional QI councils have used NCI data to develop a staff training curriculum on Community Inclusion; and educational materials on selfpreservation for individuals and families. More examples of these initiatives can be found at:
http://www.dfmc-georgia.org/quality_improvement_council/project_plan_presentations/index.html
The latest Georgia NCI data (2012-2013) was reported in the 2014 Interim Report. Georgia's 2012-2013 NCI reports can be found at: http://www.nationalcoreindicators.org/states/GA/
2013-2014 Georgia NCI data should be available in July 2015 and will be reported in the 2015 QM Interim Report.
Efficiency of Services (Georgia Crisis Response System for Developmental Disabilities) The goal the Georgia Crisis Response System for Developmental Disabilities is to provide timelimited home and community based crisis services that support individuals with developmental disabilities in the community, and provide alternatives to institutional placement, emergency room care, and/or law enforcement involvement (including incarceration).

16

Two main components of the Georgia Crisis Response System (GCRS) are Intensive In-Home Supports and Intensive Out-of-Home Supports.
The intent of Intensive In-Home Support is to stabilize the individual through behavioral intervention strategies provided under the recommendations of the DD Mobile Crisis Team. The services are provided in the individual's home and may be provided 24/7 for a limited period of time. In 2014, 6% of crisis incidents resulted in the need for intensive in-home supports. This is a significant decrease in utilization from 2013 (14%). Additional analysis of the data is needed and will be reported on in the 2015 Interim Report.
The intent of Intensive Out-of-Home Supports is to stabilize the individual through nursing and behavioral supports, on a time-limited basis. Intensive Out-of-Home Supports are provided in Crisis Support Homes. Georgia currently operates eleven Crisis Support Homes that are strategically located across the state. In 2014, 11% of crisis incidents resulted in the need for intensive Out-of-home supports. This is a significant decrease in utilization from 2013 (20%). Additional analysis of the data is needed and will be reported on in the 2015 Interim Report.
Individuals under the age of 18 years cannot be served in an adult Crisis Support Home. Those individuals were served in the Division's Temporary and Immediate Support (TIS) Home. In June 2014, the contract for the TIS home was not renewed. The Division is developing a Request for Proposal to procure a provider(s) to create and operate two new Child and Adolescent Crisis Support Homes. One home will be support children and adolescents in North Georgia, and the other home will support children and adolescents in Southern Georgia. The RFP will be released in early 2015.
Crisis data shows that the system is operating as it should, with the individual receiving crisis supports in the least restrictive environment as possible. The Division of DD has experienced, however, an ongoing issue when attempting to support dually diagnosed individuals. Behavioral Health has implemented its own Mobile Crisis Response System, and the Division of DD is partnering with Behavioral Health to address this shared population. An example of this partnership is the establishment of a Co-Occurring Case Review Committee. The Committee reviews cases that have presented challenges for community providers to a team of clinical leaders in DBHDD and from Georgia Regents University. The Committee conducts focused discussions to identify possible gaps/barriers in care, practice issues (e.g. medication regimens, polypharmacy), workforce training issues, and any other circumstances that will assist in developing strategies to assure that individuals are receiving high quality care; with expertise who can be consulted with when problems arise and how DBHDD can use what is learned to improve the transition and discharge planning process for individuals leaving State institutions.
Administrative Services Organization (ASO) and DD A key goal of the Georgia Department of Behavioral Health and Developmental Disabilities is to improve access to high-quality and effective services for individuals with intellectual/ developmental disabilities. To help achieve this goal, the Department signed a contract in September 2014 with ValueOptions (now Beacon Health Options) to function as an Administrative Services Organization (ASO) for DBHDD. The services provided will support both the DD and BH community service delivery systems. The ASO is now known as the Georgia Collaborative ASO. In the latter half of 2014, DBHDD staff began working with
17

Collaborative staff to plan and implement the functions of the Georgia Collaborative. Some highlights of the ASO functions include:
Maintaining a 24/7 crisis and access line for behavioral health and developmental disability services.
Creating a single information technology system for behavioral health and developmental disability services.
Using state-of-the-art technologies to create efficiencies and improve the quality of care. Providing an integrated and effective platform for monitoring the department's quality
management plan. Providing focused utilization management and review services for intensive BH services
and a streamlined process for less intensive BH services.
More information of the Georgia Collaborative ASO can be found at: http://dbhdd.georgia.gov/press-releases/2014-09-25/dbhdd-awards-contract-administrativeservices-organization
Summary and Recommendations: Division of Developmental Disabilities The Division of DD continues its efforts to improve the quality of supports and services through the use of its key performance indicators and system evaluation. The Division uses this data as a driver for an improved transition process. National Core Indicator data showed that Georgia has areas of improvement around its health indicators. DBHDD also recognizes the need for a more comprehensive, robust and systematic analysis (gathered from multiple sources such as the independent reviewer and the ROs) of consumer transitions. This is currently being addressed through the DD Re-Engineering Project. The Crisis Response System for DD has provided quality behavioral crisis service to individuals with DD which has resulted in less involvement of law enforcement and hospitalization. Because dually diagnosed individuals still present a challenge not only to the Crisis Response System, but the DD/BH community as whole, the Division has and continues to take steps to evaluate how to better serve these individuals.
Quality Monitoring Activities
Complaints and Grievances The Office of Public Relations (OPR) Constituent Services and Legislative Affairs received a total of one hundred sixty-two (162) complaints, grievances and inquiries resulting in opened casework from January 1 thru June 30, 2014. All 162 cases have been addressed by staff in the state office, regional office or regional hospital.
A total of 162 cases are inactive/closed. All cases were triaged to the appropriate office and responded to within 5 to 7 business days depending on the nature of the complaint or inquiry. Of the 162 complaints, grievances and inquiries received there were a total of 37 issue categories. Some of the issue categories cited include addictive diseases adult services, developmental disabilities planning list, developmental disabilities exceptional rate, developmental disabilities self-directed services, host homes, mental health outpatient and inpatient treatment and services, mental health crisis stabilization unit, mental health residential, mental health housing, provider enrollment and certification, personnel concerns, and issues that
18

were referred to other state agencies. Within the six month period, Region 3 received 39% of regional cases and 28.4% of all complaints/grievances and inquires. This is consistent with Region 3 having the highest population density of all regions.

The OPR recognized the following four most frequent issues and inquiries received from January 1, 2014 thru June 30, 2014:
1. Developmental Disabilities NOW/COMP Waiver eligibility for the New Options Wavier (NOW) and the Comprehensive Supports (COMP) wavier (Approved for waiver but funding is limited.)
2. Developmental Disabilities provider services, individual waiver budget 3. Mental Health need for assistance accessing community-based services 4. Mental Health need for residential long term placement and treatment

A large percentage of complaints/grievances and inquires originated from the Governor's office, legislative offices, family and friends of a consumer, or the consumer themselves.

The following table illustrates the location to which cases were triaged for resolution:

Assignment

Location

Disabilities

Percentages

DD

Regional Hospitals

GRH- Atlanta

0

ECRH- Augusta

0

WCRH- Columbus 0

CSH- Milledgeville 0

GRH- Savannah

0

SWSH- Thomasville 0

Totals

0

MH AD

Facilities Offices All Complaints

10

0

10

0

0

0

2

0

2

0

0

0

0

0

0

0

0

0

12

0

12

83.3% 0.0% 16.7% 0.0% 0.0% 0.0% 100.0%

6.2% 0.0% 1.2% 0.0% 0.0% 0.0% 7.4%

Regional Offices DD MH AD

Region 1

17

8

0

25

Region 2

8

7

0

15

Region 3

29

15

2

46

Region 4

4

3

0

7

Region 5

5

7

0

12

Region 6

10

3

0

13

Totals

73

43

2

118

21.2% 12.7% 39.0% 5.9% 10.2% 11.0% 100.0%

15.4% 9.3% 28.4% 4.3% 7.4% 8.0% 72.8%

State Office 2 Peachtree Addictive Diseases Mental Health
Development

4

12.5%

4

12.5%

1

3.1%

2.5% 2.5% 0.6%
19

Disabilities Legal
Public Relations
Human Resources Legal Provider Network Management Other Agency
Investigations Totals

3

9.4%

5

15.6%

3

9.4%

3

9.4%

4

12.5%

4

12.5%

1

3.1%

32

100.0%

1.9% 3.1% 1.9% 1.9%
2.5% 2.5% 0.6% 19.8%

Total Cases

162

100.0%

The Office of Public Relations Constituent Services section currently has data regarding inquiries, complaints and grievances received from January 2014 through June 2014. During a data migration event conducted in recent months, constituent service staff lost data as well as access to archived data for the time period of July 2014 through December 2014. The Office of Public Relations is currently working with the Department's IT division to recover the lost data. However, the data was not available at the time this report was published.

As a result of the Department's recent transition to a cloud-based data storage platform using Microsoft SharePoint, the Office of Public Relations has requested the design and build of a customer relationship management type platform to manage and process cases. It is felt that this cloud-based, custom platform will not only eliminate the possibility for data loss in the future, but vastly improve constituent interactions, as well as the overall case management process.
Hospital and Community Incident Data The following incident review covers death reports and critical incident reports received in the Office of Incident Management and Investigations from January 1, 2014, through December 31, 2014. The total incidents received by month for hospitals and community providers are included in Tables 1 and 3 below. The tables also provide a comparison for the current report period (CY 2014) with the prior calendar year (January 1, 2013 December 31, 2013).
Hospital Incident Data As Table 1 indicates, the total number of hospital incidents for Calendar Year (CY) 2013 was 8,081 compared to the current report period of 6,824. Overall a 16 % reduction occurred. This reduction is in part due to the closing of Southwestern State Hospital at the end of December 2013 along with increased quality improvement efforts to reduce incidents in the hospital system. When calculating a rate for comparison the rate for CY 2013 is 17.3 and CY 2014 is 16.5. Both a reduction in occupied bed days (OBD) and overall incidents contributed to the lower rate. (Note: Rate is calculated by Total Incidents/Occupied Bed Days x 1000.)

Table 1: Total Incidents by Quarter:

Hospital

CY-2013

Qtr1

Qtr2

Qtr3

Qtr4

Total

20

CY-2014

2075 1700

2028 1751

2149 1621

1829 1752

8081 6824

The five most frequent hospital incidents reported during this review period (CY 2014) are listed below in Table 2. Incident types A04 and A03, Aggressive act to staff, Aggressive act to another individual-Physical occurred more often than all others and account for 51% of the total number of incidents reported. This percentage did not change from the prior 12 months. However, actual number of incidents of Aggressive act to another individual-Physical decreased 21% and Aggressive act to staff-Physical decreased 10%. A01 Accidental Injury, A30 Property Damage and A25 Falls round out the most frequently reported hospital incidents. These five incident types account for 76% of the total number of incidents reported.

Table 2: Most Frequently Reported Hospital Incidents (CY-2014) Hospital Incident Type A04-Aggressive act to staff-Physical A03-Aggressive act to another individual-Physical A01-Accidental Injury A30-Property Damage A25-Fall Total

Total 1801 1668 638 548 528 5183

Community Incident Data The total community incidents for the current report period (CY 2014) were 3,974 compared to CY 2013 report period of 3,842 reflecting an increase of 3.4%. In October 2013 the Department reminded providers of the requirement to report incidents and provided technical assistance to several providers regarding the incident reporting process. After that reminder, the number of incidents reported increased for three consecutive quarters. However, other factors may also have contributed to the increase in reported incidents.

Table 3: Total Incidents by Quarter:

Community

CY-2013

Qtr1

Qtr2

951

1068

CY-2014

1009

1100

Qtr3 902
943

Qtr4 941
922

Total 3842
3974

The most frequently reported community incident type is Hospitalization of an Individual in a community residential program. See Table 4 below for the five most frequently reported community incidents.

Table 4: Most Frequently Reported Community Incidents (CY 2014) Community Incident Type C-Hospitalization of an Individual in a community residential program C-Incident occurring in the presence of staff which required intervention of law enforcement services

Total 1327
376

21

C-Individual injury requiring treatment beyond first aid C-Individual who is unexpectedly absent from a community residential program or day program C-Alleged Individual Abuse-Physical Total

326
293 258 2580

Hospitalization of an individual in a community residential program occurred more frequently than all other community incident types combined and increased 8.5% from the prior 12 month period. This incident type includes hospitalizations for any reason including medical and psychiatric hospitalizations as well as transfers from crisis stabilization units for continued mental health treatment at a state hospital. Reporting of incidents occurring in the presence of staff which required intervention of law enforcement services increased 15.7%. Individual injury requiring treatment beyond first aid decreased 6.3 %; Individual who is unexpectedly absent from a community residential program or day program decreased 5.8%, and Alleged Individual Abuse-Physical increased 22.9%. It is likely that increases in reporting of incidents is due in part to more accurate reporting of incidents that were previously under-reported. Additional analysis will be performed on community incidents related to the five most frequently reported community incidents in CY 2014 and the results will be reported to the CBH PQC in early 2015.
Patterns and Trends During this report period the Office of Incident Management and Investigation compiled, analyzed and provided information regarding incident patterns and trends to the Community Behavioral Health Program Quality Council (CBH PQC), the DBHDD Executive Quality Council (EQC), the Division of Developmental Disabilities, the Division of Addictive Diseases, the Division of Community Mental Health, the Suicide Prevention Coordinator, and the Regional Hospital Administrators, Risk Managers and Incident Managers. Based on a review of the data, additional data needs were identified and provided in subsequent meetings. The trended information has been used for quality improvement purposes to identify providers who may require technical assistance and/or training.
Community Incident Data Behavioral Health Services Community incident data can be further categorized by disability type. Community behavioral health providers reported 1,185 critical incidents during this report period or 30% of the total number of community incidents. The incident types requiring an investigation and reported most frequently for Behavioral Health were: Hospitalization of an Individual in a community residential program, Incident occurring in the presence of staff which requires intervention of law enforcement services, Individual who is unexpectedly absent from a community residential or day program, Individual Injury requiring treatment beyond first aid and Criminal Conduct by Individual.
Community Incident Data Developmental Disability Services Community developmental disability providers reported 2,789 critical incidents or 70% of all incidents during this report period. The incident types requiring an investigation and reported most frequently for developmental disabilities were Hospitalization of an Individual in a community residential program, Individual injury requiring treatment beyond first aid, Incident occurring in the presence of staff which requires intervention of law enforcement services, Alleged Individual Abuse-Physical and Alleged Neglect.
22

Community Mortality Reviews During this review period the Community Mortality Review Committee met nine times to review the unexpected deaths of individuals receiving DBHDD services. (Note: Category II expected deaths and Category III deaths that require no investigation per policy were not reviewed.) A total of 141 unexpected deaths were reviewed during this period with 30 reviewed in the 1st quarter, 43 in the 2nd quarter, 39 in the 3rd Quarter, and 29 in the 4th Quarter of 2014.
In addition, the Department entered into contracts for external mortality reviews of all deaths of I/DD individuals who were transitioned from hospitals to the community during the Settlement Agreement as well as all persons who died by suicide who met the definition of the target population of the Agreement. Information from these mortality reviews as well as the reports from the Community Mortality Review Committee will be included in the first annual mortality report for FY 2015 which is anticipated to be distributed in the first quarter of FY 2016.
Hospital Peer Review and Credentialing During this report period no changes have been made in the credentialing process. The mentoring system has been modified so that discipline chiefs have more direct responsibility for managing these functions.
Hospital Utilization Review The Hospital System and Regions continue to monitor and address issues related to rapid readmissions (less than 30 days), people with 3 or more admissions in a year, and people with 10 or more admissions in a lifetime. These indicators are monitored via the Triggers and Thresholds report. Additionally, each Hospital maintains its own utilization review processes and functions.
Adult Mental Health Fidelity Reviews
Assertive Community Treatment Assertive Community Treatment Fidelity Reviews are conducted once per DBHDD fiscal year for all twenty-two state contracted ACT teams. Between January 2014 and December 2014 the DBHDD ACT & CST Services unit conducted a total of 28 DACTS (Dartmouth Assertive Community Treatment Scale) fidelity reviews on all 22 State Contracted ACT Teams. Six teams received two reviews during the stated time frame due to the review cycle following the fiscal calendar year. The review typically takes 2-3 days with individualized on-site technical assistance provided to each staff member of the ACT team. Once the DBHDD ACT & CST Services Unit completes the Fidelity review, results of the Fidelity Review are given to the ACT team, leadership within the agency, the regional office in which the team operates, and the DBHDD Adult Mental Health Director and other departmental leadership. Results are also provided to the ACT Subject Matter expert hired as part of the Independent Reviewer's review of the DOJ Settlement. This is followed by a detailed discussion of the report inclusive of each scale and the rating for each scale along with any explanation or recommendation for the rating. This occurs during the exit interview which is attended by the ACT provider, regional and state office staff.
Review items that are found to be below the acceptable scoring range: a score of 1 or 2 results in a Corrective Action Plan (CAP) which each team develops with the assistance of the DBHDD regional and state office staff to ensure inclusion of all areas that scored below fidelity. The CAP
23

is then submitted to both the regional and state office for approval and monitoring. ACT teams are contractually required to obtain a DACTS mean score of 4.0 and total score of 112. Of the twenty two state contracted teams fifteen teams achieved a score within this range of fidelity, indicating that they are serving the appropriate population, maintaining an acceptable caseload, delivering the service with intended frequency and intensity, providing crisis response, conducting effective daily team meeting discussion of consumers, engaging formal and informal supports, being involved in hospital admissions and/or discharges and delivering 80% of the teams services in the community. Seven teams scored below a mean of 4.0 on fidelity. Some of those areas that needed attention were: increasing team involvement in hospital admissions and discharges, strengthening delivery and documentation of contacts with consumer's informal support system, increasing the stability of staffing and reducing turnover and increasing cooccurring disorders treatment. All seven teams submitted a CAP and received technical assistance from the DBHDD ACT & CST services Unit, the regional staff and specific area trainers. All seven teams have demonstrated improvements in low scoring areas with 86% of items originally scoring a 1 or 2 raised to a 3 or higher following the completion of the corrective actions in the plan.
Supported Employment Supported Employment (SE) Fidelity Reviews are conducted annually for all twenty-one state contracted SE providers. In 2014 from January-December 2014 a total of twenty-two IPS Fidelity Reviews were completed using the 25-item Individual Placement and Support (IPS) model for supported employment. During this time frame four SE Providers received another IPS Fidelity Review as part of the annual review cycle. Once the SE Fidelity Review was complete, results were given to the SE provider, the Regional office in which the team operates the DBHDD Adult Mental Health Director, and other Departmental leadership. Results were also provided to the SE Subject Matter expert hired as part of the Settlement and were posted on DBHDDs website. This was followed by an exit interview inclusive of the provider and, Regional and State staff with a detailed discussion of the review outcome and report. Outcomes were also discussed with the CBH PQC. Review items that were found to be below the acceptable scoring range a score of 1 or 2, resulted in a Quality Improvement Plan (QIP) which each team developed and submitted for acceptance to the Regional and State office. SE providers are contractually expected to minimally obtain an IPS total score of 74.
Of the twenty-two providers who have received a Fidelity Review, twenty-one achieved a score within the acceptable range of fidelity, indicating that they were effectively integrating SE and mental health, maintaining collaboration with GVRA, demonstrating clearly defined employment duties for SE staff, implementing zero exclusion, rapidly engaging consumers in competitive job search, assessing consumer's interests and making job placements based on identified interests and skills. At the time of the review, 1 provider scored below the acceptable range of fidelity.
Some of the areas of needed attention were, increasing collaboration with GVRA, connecting consumers with work incentives planning, integration of SE and mental health treatment team, engaging in sufficient employer contacts, and frequent employer contact. All twenty-two providers have submitted or are in the process of submitting QIP's and each provider received on-site technical assistance from DBHDD SE Services Unit, the regional staff, and subject matter experts in order to improve operations in areas of deficiency. All four teams that received
24

another IPS Fidelity Review as part of the annual review cycle had a decreased mean score from 3.9 to 3.8. The decreased mean score is reflective of agency staffing changes. During January 2015- June 2015 there are 13 IPS Fidelity Reviews tentatively scheduled.
Mobile Crisis Response System Performance and Quality Monitoring In March 2013 the DBHDD procured mobile crisis response services (MCRS) in all 6 of its regions. MCRS began in 100 counties in June 2013. As of July 1, 2014, MCRS are available statewide in all 159 counties.

Two vendors were chosen to cover the state and have been participating in the MCRS Quality Management System since the beginning of their contracts. There are 20 data points that the vendors report on monthly to the regions. This data is reviewed monthly by a State MCRS committee, as well as quarterly at a MCRS Quality Consortium. Through these meetings, a quarterly data template has been created, barriers to implementation have been resolved, and processes have been put into place to improve the quality of the service.

Between January and November 2014, 16,697 calls were received. The below table shows the average (mean) response time for mobile crisis teams Response time is defined as the amount of time in between being dispatched to a location where the individual is located until the time of arrival at that location.

Month
January 2014 February 2014 March 2014
April 2014 May 2014 June 2014 July 2014 August 2014 September 2014 October 2014 November 2014

Average Response Time (in Minutes) 53 49 48 50 49 47 49 54 58 52 52

QM Audits: Quality Service Reviews of Adult Behavioral Health Community Providers In October 2013, the DBHDD redirected the focus of the QM Department's audit work as a result of findings provided by an external consultant (Dr. Nancy Ray) regarding data collected and reported from quality audits for repeat admissions. The QM Audit Team selected 24 individuals for review that met the criteria of having three or more admissions to a State Hospital within the last 12 months, with the last discharge date being no later than January 16, 2014 and the most recent admission occurring between November 14, 2013 and January14, 2014. The review (January 2014-June 2014) included a completion of record reviews similar to the consultant's methodology. In addition, the members of the audit team visited three hospitals (GRH-ATL, GRH-SAV, and ECRH) and interviewed staff to follow up on trends identified through the chart audit and resolve any questions associated with the medical record review.
25

Based upon the results of this audit as compared to the results of the audit performed by the consultant in 2011 some of the findings included:
The re-admission rate (three or more within 12 months) to State Hospitals decreased statewide from 9.88%, July 1, 2011-June 30, 2012, to 9.15%, January 1, 2013 December 31, 2013.
The majority of individuals were discharged with an appointment to a mental health provider in their community.
Medical issues were identified and addressed consistently for individuals in the State Hospital, but linkage to community medical providers at time of discharge was not found in documentation.
IRPs were completed within policy designated timeframes: 90 % for 24 hour IRPs, 96% for 72 Hour IRPs, and 94% for 15 Day IRPs.
Behavior Guidelines in all cases were written in an understandable language, individualized to the individual's issues, and based on positive behavioral supports.
Individuals are not consistently assessed, treated or followed-up with regarding Substance Abuse issues.
Cognitive disabilities need to be better assessed and treatments adjusted to the level needed by an individual based upon their deficits.
Documentation did not support that IRPs are being implemented as written. Documentation of attendance at the Treatment Mall was inconsistent and did not clearly
illustrate an individual's progress towards goals or discharge criteria. Discharge linkages did not always meet an individual's identified needs. The factors influencing an individual's repeat admission were not consistently identified
and included in an individual's IRP.
Upon completion of the review of individuals who had repeat admissions to a State Hospital, the Department began a pilot review of individuals who meet settlement criteria and frequently utilize crisis services in both the community and through the State Hospital System. The reviews combine a focus of the State Hospital services along with community-based crisis and therapeutic services, beyond those listed in the Settlement Agreement, allowing for a comprehensive look at the services individuals receive. The project focuses on an individual's treatment, level of satisfaction, and unmet needs or barriers to successful treatment, and follows the individual through their continuum of care, including their transition process into the community. In keeping with past quality audit/service reviews conducted by the QM Department, records are reviewed and individuals & staff are interviewed. The initial project focused on Regions 1 and 3 (July 2014-November 2014). The remaining regions will be reviewed beginning in January 2015. Based on the information gathered as of the date of this report in Regions 1 and 3, the following has been identified:
Homelessness is a recurrent factor impacting recidivism. Multiple factors appear to be impacting the ability to place individuals in appropriate housing, including consumer choice to be discharged to shelters. Additional data needs to be collected to determine whether current housing options are appropriate and sufficient to meet the needs of this population.
Individuals with only SA disorders were highly represented in the sample of the highest utilizers of crisis stabilization units and, despite the availability of SA treatment services, these highest utilizers were not participating in these services.
26

The majority of individuals cycling through crisis services with a primary diagnosis of SA are not connecting to outpatient services due to several factors, such as: the individual relapses soon after discharge; housing issues; a lack of wraparound services upon discharge; or the individual refuses follow-up appointments.
This information was shared with the AD program lead and consideration was given to how more aggressive post-discharge community engagement may be needed. A pilot program was approved at one of the crisis stabilization units in Region 1 to implement such strategies and assess the effectiveness of such interventions on reducing repeated use of crisis stabilization units by these individuals.
As the reviews continue throughout the State, the QM team will continue to identify any trends and patterns and areas for improvement and communicate those to the behavioral health programmatic leads for consideration in planning services and supports.
Child and Adolescent Community Mental Health Programs (CAMH) Monthly or quarterly reports related to Quality Improvement data were produced for all CAMH programs (Psychiatric Residential Treatment Facilities, Care Management Entities/Community Based Alternatives for Youth, Crisis Stabilization Units, and Resiliency Support Clubhouses) by the Georgia State University Center of Excellence for Child and Adolescent Behavioral Health. The data and formats of the reports were reviewed by the applicable program quality consortium. All quality improvement consortiums agreed to move toward a provider report card instead of the extensive report and have finalized data collection measures. All quality improvement consortiums also reviewed their respective quality improvement plans and made the necessary changes to match the standardized data collection measures for CY2014. These standardized processes will increase the reliability and validity of the data being entered into the data tool by the providers and will therefore produce better data reports in the future, allowing for accurate review and process improvement activities. All quality consortiums have also standardized their agendas allowing a large amount of time to be spent on reviewing data and looking for opportunities to improve programming. It is anticipated that the standardized quality improvement processes will improve the services being delivered to children and adolescents in Georgia.
In June 2014, the Office of Children, Youth, and Families, along with the Georgia Interagency Director's Team, a state-level interagency collaboration which is a subgroup of the Department's Behavioral Health Coordinating Council, hosted the 7th Annual System of Care Academy. This three day training event was held in Stone Mountain, Georgia. All Child and Adolescent Providers, as well as youth, parents, managed care organization staff, child welfare staff, juvenile justice staff, and other state agency staff were invited and participated. Topics were varied and included, but were not limited to: leadership, best practice for treatment of ADHD, and youth engagement. Approximately 350 people participated in this training. The next academy will be held in 2015.
In August 2014, DBHDD's Division of Community Mental Health held a training and technical assistance symposium in Macon, Georgia. All child & adolescent and adult providers were invited to participate and receive training on how to increase and improve the quality of the service(s) they provide. Topics were varied and included, but were not limited to: Co-Occurring
27

Mental Health/Developmental Disabilities Treatment for Youth, Innovations in Georgia Children Mental Health System, Georgia System of Care, Military Families & Youth, and Best Practices for Engagement of Transition Age Youth and Young Adults. Approximately 350 people participated in this training. The next symposium will be held in the summer of 2015.
Division of Addictive Diseases (AD) Quality Management Activities The Division of Addictive Diseases provides leadership for adult and adolescent substance abuse treatment services. The Division's responsibilities include: program oversight; grants management; ensuring compliance with federal and state funding requirements; maintaining collaborative relationships with advocacy groups and other stakeholders; providing data and information at the regional and local levels to impact policy decisions; statewide technical assistance to providers and the six BHDD Regional Offices; developing and maintaining collaboration among private and public sector providers and stakeholders; providing training and information on best practices for substance abuse treatment; coordinating collaborative efforts in increasing best practices models; assisting community and faith-based groups in developing capacity and training; overseeing HIV Early Intervention Services among substance abusers and their families and significant others; overseeing men's residential treatment services throughout Georgia and the Ready for Work women's programs.

Program staff assigned to the Division's state office is responsible for conducting provider site reviews to ensure fidelity/compliance to service guidelines and federal block grant requirements. Listed below is a table that provides an overview of each program area and the QM activities conducted by staff along with the frequency:

AD Service/ Description RFW Residential
RFW Outpatient Programs
RFW Transitional housing supports Clubhouses

QM Activities/On-site reviews

Frequency

Site visits are currently conducted by Women's Treatment Coordinator. APS does not audit these programs. Staff use tool to review provider compliance with standards and overall performance in providing gender specific substance abuse treatment services. In addition, TCC vendor conducts review of all Therapeutic Childcare programs offering services to children. Clinical reviews of these programs against requirements are conducted by addiction credentialed staff with gender specific training and historical context of programs and interaction with child welfare agencies. Site visits are currently conducted by Women's Treatment Coordinator. APS does not audit these programs. Staff use tool to review provider compliance with standards and overall performance in providing gender specific substance abuse treatment services. Site visits are currently conducted by Women's Treatment Coordinator.
Site visits conducted by C&A program staff to ensure program design and requirements are being followed. Staff person is 7 Challenges trained.

1x every 2 years
1x every 2 years
As needed basis if monthly reports indicate an issue 1x every 2 years

Recovery Centers
IRT (Intense Residential Treatment) Programs

Site visits conducted by Adult program staff to ensure program design and requirements are being followed. Clinical review of these programs against requirements are conducted by addiction credentialed staff Site visits conducted by C&A program staff to ensure program design and requirements are being followed. Staff person is 7 Challenges trained.

1x every 2 years 1x every 2 years

Transitional/IOP Site visits conducted by Adult program staff to ensure program design and As needed basis if

28

HIV EIS
AD Treatment Courts Opioid Maintenance

requirements are being followed. Clinical review of these programs against requirements are conducted by addiction credentialed staff Site visits conducted by vendor to ensure program design and requirements are being followed.
None currently as program serves more of an administrative function.
Site visits conducted by State Opioid Maintenance Treatment Authority.

monthly reports indicate an issue 1x every 2 years
N/A
1x every 2 years

Adult Residential Site visits conducted by Adult program staff to ensure program design and 1x every 2 years Treatment Services requirements are being followed. Clinical reviews of these programs
against requirements are conducted by addiction credentialed staff.

Providers who are not in substantial compliance with Federal requirements are provided an indepth review of those requirements and additional training if needed to ensure future compliance and when needed, corrective action plans.

In addition to site reviews, program staff process contract payments and programmatic reports received monthly from providers to ensure service guidelines are being met from a contractual standpoint. Once reviews are completed, the results are shared with the regions and providers to review performance/progress and identify any areas in need of improvement.

Division of Addictive Diseases Training The Division of Addictive Diseases also ensures that training is offered to providers to improve quality of services. Trainings initiated by the Division this year include the following;

Title of Training
Strategic Trauma and Abuse Recovery: A Source-Focused Model for Healing
Advanced Clinician Training for DUI Treatment Providers
Advanced Clinician Training for DUI Clinical Evaluators
The Anti-Reward System of the Adolescent Brain: "The Neurobiology and Pharmacology of Addiction, Anxiety, and Depression of the Adolescent"
Star Behavioral Health Providers (Tier One) STAR Behavioral Health Providers: Military Culture Training (Tier 3) Prolonged Exposure Therapy (PE) for PTSD
STAR Behavioral Health Providers: Military Culture Training (Tier 2) Introduction to Trauma Informed Care for Youth

Estimated # of attendees
97 85 96
26 454
103
202 120

Mental Health Coalition Meetings Adult Mental Health specialty service providers meet either monthly or bi-monthly, these include individual Coalition meetings for all Supported Employment providers, a Coalition meeting for all Assertive Community Treatment providers, a Coalition meeting for all Community Support Team providers, a combined Coalition meeting for all Case Management
29

and Intensive Case Management providers, and a Coalition meeting for all providers of Crisis Stabilization Unit services.
These Coalition meetings are vehicles for disseminating and gathering information, maintaining open communication, promoting provider collaboration and fostering the partnership between the Department and provider agencies. This forum allows for discussion of programmatic operations and performance (including key performance indicators), informal presentations/inservice, discussion of Departmental policies and any other matters of relevance for these evidence-based practices.
During calendar year 2014, there were 5 ACT and CST coalition meetings and one combined AMH services coalition meeting held in Macon; for ease of access for all providers across the state these meetings alternate between Macon and Atlanta. Participation is required from all providers either in person or via a conference line that is made available for all participants. Representation at each coalition may include but is not limited to all ACT and CST providers, DBHDD regional and state office state offices, the BH ERO, and a Respect Institute speaker. Coalition meeting agenda topics for 2014 included: Office of Recovery Transformation presentation-definition of Recovery and the Recovery
guiding principles and values Benefits counseling information and DBHDD Medicaid Eligibility Specialists Georgia Vocational Rehabilitation Agency presentation Transitioning individuals from jails/prison into the community Effective usage of CTP for serving individuals while in an institution Preventing burnout Integration of services with SE, ICM, housing, Supported Housing, CST & Mobile Crisis Overall Outcomes of Fidelity Reviews by ACT and CST Services Unit
Behavioral Health Contracted External Review Organization (ERO) APS Healthcare is the External Review Organization (ERO) for DBHDD's behavioral health service. Many of the functions and products provided by this vendor contribute to the Department's quality management of the Provider Network. These elements include training, technical assistance, prior authorization for services, provider audits, and provider billing and service provision data. Several notable outcomes occurred during the time period of this report regarding provider network management, training opportunities, and authorization processes.
Audits: The ERO conducted 326 audits in 2014. In an effort to develop a systematic review and response to audit findings, DBHDD implemented Policy 01-113, Noncompliance with Audit Performance, Staffing, and Accreditation Requirements for Community Behavioral Health Providers, in September 2012. This policy provides a protocol for DBHDD to respond to providers who receive failing audit scores, do not meet minimum staffing requirements, or fail to achieve or maintain accreditation. DHBDD made improvements to tracking and communicating audit scores both internally and with the Department of Community Health (DCH). Staff at DBHDD has worked to collaborate with DCH to develop procedures regarding consistent management of providers which fail to achieve compliance with DBHDD standards as evidenced by failing audits. As a result of this collaboration, protocols and
30

dialogue have been strengthened between the two Departments to ensure a consistent and efficient process of responding to provider deficiencies via corrective or adverse action. In 2014, DBHDD executed Policy 01-113 and related DBHDD-DCH protocols for approximately 50 providers who had failing audit scores. Additionally, DBHDD has collaborated with DCH/Program Integrity Unit to identify providers with failing audit scores (especially billing audit scores); in response to these issues, approximately 40 providers have received intensive technical assistance to assist them during the individual's review of all billing claims and clinical documentation prior to payment. DBHDD also has used ERO audit findings regarding providers' compliance with DBHDD Provider Manual as a primary basis for termination of agreements with fourteen providers. DBHDD currently is engaging several other providers via Policy 01-113 to address other concerns identified in ERO audits. This implementation has resulted in a refinement of the network based on provider performance.
DBHDD and the ERO completed the annual evaluation of the ERO audit tool. The audit tool was modified to reflect updates in the DBHDD Provider Manual on a quarterly basis. The current audit tools can be found on the APS Knowledgebase page at www.apsero.com.
Training The ERO (APS Healthcare) has provided many training opportunities to the network during the report period. In addition to the onsite technical assistance provided at each audit exit interview, the ERO has also offered both broad and targeted information to the provider network:
Expanded prior authorization reviews to include private psychiatric hospitals in Regions 4 and 6 to support least restrictive and appropriate treatment and service for those in need of acute services options. ERO outreach and training to Region 4 and 6 private hospitals emphasized that successful admission, treatment delivery, and discharge planning are best accomplished when both the hospital and communitybased providers are actively engaged in the process. This promotes the opportunity to improve the quality of life post discharge through facilitation of stable housing, identification of chronic medical conditions and referrals for coordinated care through Assertive Community Treatment (ACT), Intensive Case Management (ICM) and other community-based programs.
The ERO also provided several onsite training sessions during the annual DBHDD Community Mental Health Symposium, in addition to five statewide trainings.
Participation and training as an element of the Georgia Certified Peer Specialist training, including CPS-Parenting Documentation Training, the first of its kind. This included providing four CPS Training sessions, three CPS-Parenting Documentation Training sessions, and two CARES Documentation Training sessions.
Continued offering of the Ambassador Program for new providers and providers' new staff members.
In coordination with DBHDD and to support of the full implementation and ongoing support of Task Oriented Recovery Services (TORS), the ERO provided training to agencies that deliver this service.
Participated in the ongoing implementation and support for provider training of the Case Management/Intensive Case Management Toolkit and Community Support Teams, as these services were supported to providers across the state.
31

Provided eight onsite trainings for the Georgia Medicaid Fraud Control Unit (MFCU) on APS Audit Protocol and Procedures.
The ERO also provided technical assistance to 10 providers in order to support them in the implementation and support of systems implementation.
In addition, the ERO has been instrumental in assisting the Department with additional training opportunities related to ACT and Community Support Team (CST). Following feedback received from providers, DBHDD and the ERO partnered to provide training regarding ACT and CST services in multiple venues. In addition to the ERO's regular attendance at ACT and CST coalition meetings, the ERO provided technical assistance specific to ACT via:
Targeted feedback to DBHDD regarding ACT authorization and audit processes and evaluation of inter-rater reliability.
Provided ongoing assistance to providers in group and individual trainings regarding how best to utilize and coordinate ACT, CST, and other intensive services in conjunction with community resources and individual strengths to meet consumer needs to increase effectiveness of outcomes.
Service Utilization & Authorization: During the report period, licensed clinicians at the ERO manually reviewed 61,843 authorization requests for community services. Of those, 3,746 authorization requests were specific to ACT services.
In the spring of 2013, DBHDD used utilization data to perform a review of units authorized for several service packages and to identify trends. This review was conducted by a panel of experienced clinicians and operational experts using a zero-based methodology that examined each service individually and in the context of other services available. The review resulted in a recommendation and subsequent changes to selected authorization packages. While there was some reduction in the number of units authorized in each package, the changes did not equate to a reduction or limit to services.
The primary aim of the initiative was to support services at levels sufficient to treat and support individuals at all levels of care. The changes to the authorization array promoted recovery and resiliency through the use of a comprehensive and robust array of case management/skills development services combined with appropriate psychiatric treatment, individual, group, and family therapy services rather than relying heavily on one or two isolated service modalities for individuals with complex needs. DBHDD continues to monitor utilization trends for continuous quality improvement activities. Claims information provided by the ERO also informed key decisions related to validation and continuation of the content of service authorization packages.
Provider Network Analysis
The Department engages in community behavioral health and developmental disability service planning that encompasses an array of services that will assist individuals in living a life in the community. This service array provides levels of care for individuals who are identified as the target population as well as those who meet eligibility criteria for state supported services.
32

Service planning is unique to the needs of each community and includes significant input from community members and service recipients. An annual network analysis is conducted through DBHDDs Regional Offices and seeks to identify the impact of state and federal resources on the consumers who received services from State contracted treatment providers.
The Regional Network Analysis (RNA) concluded in 2014 looked at services for the SFY 13 and the first quarter of SFY 14. The next RNA will capture the remaining quarters of SFY 14 and the first quarter of SFY 15.
Each region compiles information regarding demographics, prevalence data, descriptions of individuals served, funding resources, service delivery areas, services provided, pilot projects and/or grants awarded, collaborative efforts with other state/local agencies and stakeholders, and need identification. While there are common elements in each report, there are also unique features given the different characteristics of each region.
The Regional Network Analysis, in conjunction with the Regional Planning Board Annual Plan, informs the planning processes for funding and service delivery for the upcoming year. The Report tracks the implementation of new services implemented as a result of the Settlement Agreement in addition to the services that have long been available in the region. The RNA is reviewed by DBHDD state and regional leadership. This review better helps the Department understand the gaps in its service delivery system. For example, the analysis from Region 4 last year identified a need for an increase in addictive disease services both outpatient and inpatient based. The Region then began to work with existing providers on developing a more robust continuum of services and helped providers develop alternative funding options.
The RNA serves as a resource to both state and regional office staff. It is referenced as part of the federally required state block grant application and is used by regional planning boards in the development of the Community Plan, which is a planning document submitted annually to DBHDD. The RNA has become an important document for the Department in its effort to widen its partnerships at the state, county and city levels.
Implementation and Results of Best Practice Guidelines:
Beck Initiative The Beck Initiative is a collaborative clinical, educational, and administrative partnership between the Aaron T. Beck Psychopathology Research Center of the University of Pennsylvania and DBHDD to implement recovery-oriented Cognitive Therapy (CT-R) training and consultation throughout the DBHDD network. Fusing the recovery movement's spirit and cognitive therapy's evidence base, CT-R is a collaborative treatment approach that prioritizes attainment of patient-directed goals, removal of obstacles to the goals, and engagement of withdrawn patients in their own psychiatric rehabilitation. Through intensive workshops and ongoing consultation, tangible tools to help remove roadblocks to recovery of people with severe mental illness are placed in the hands of care providers across the network. CT-R provides the fabric for promoting continuity of care with the goal of helping affected individuals achieve a sustained integration in the community.
33

Broad Project Goals
To promote hope, autonomy, and engagement in constructive activity, for individuals served by agencies in the DBHDD network;
To establish CT-R as a standard practice of care for people served within DBHDD agencies;
To promote the sustained implementation of CT-R into the DBHDD network; To improve the professional lives of therapists in the DBHDD system; To conduct program evaluation to examine outcomes such as client attrition, service use,
recidivism, therapist turnover, and the sustainability of high-quality CT in DBHDD settings; To utilize the evidence-based practice of CT-R in the Department as a roadmap for delivering recovery-oriented care; and To serve as a model for other large mental health systems.
FY: 14 - Project Plan Providers in Region 6 received this training between August and December 2013. Regions 1 and 3 were trained and received consultation/supervision between February 2014 - August 2014. The CT-R Training Program consists of workshops (Phase 1), 6-month consultation (Phase 2) and sustainability (Phase 3). The training sites and providers receiving the training will be the State Hospital (key providers), the community (ACT teams, Community Support Teams and Community Service Boards), and supervisors.
Project Plan progress for Region 6 providers: Supervisor Training o 10 professionals trained
Hospital Training o August 8, 2014: 53 professionals trained o August 9, 2014 : 32 professionals trained o August 15, 2104: 37 professionals trained o August 16, 2014: 34 professionals trained
Week one of community providers training: August 19-23, 2014 o 17 professionals trained
Week two of community providers training: August 26-30, 2014 o 37 professionals trained
Trainings for Regions 1 & 3 began in February (Hospital Trainings) and March 2014 (Community Providers). Region 4 was completed in June 2013.
Progress Made during this period: 34 individuals trained in the hospital workshops (4 days of training) 55 individuals trained in the outpatient week long workshops
34

FY: 15 - Project Plan DBHDD will finalize training and consultation in the remaining Region, which is Region 5. Dates and logistics are currently being planned, but will mirror the format in Region 2.
Suicide Prevention Program DBHDD recognizes suicide as a significant public health issue in the State of Georgia and has developed a suicide prevention program. The program's goals include:
preventing suicide deaths, reducing other suicidal behaviors including attempts, reducing the harmful after-effects associated with suicidal behaviors, and improving the mental health of Georgians through primary prevention activities,
access to care, early intervention, crisis treatment and continuing care.
A foundation of suicide prevention is providing awareness to communities and groups about the crisis of suicide and engaging citizens to work in their communities. In 2014 over 20 awareness events were held in Georgia throughout the entire state with group sizes ranging from 20 to 450 to community groups such as faith based groups. In 2014 there were 12 active suicide prevention coalitions and at least 10 new communities interested in forming coalitions.
In 2014 the Suicide Prevention Program adopted the focus of Suicide Safer Communities to encourage multiple activities and multiple community partners in suicide prevention. In September 2014 the third annual Suicide Prevention Coalitions' Conference, Joining Hands Across Georgia was held in Kingsland, GA with an attendance of about 80 people from active and developing coalitions. The fifth annual suicide prevention conference for Georgia's colleges and universities was held in May 2014 at Middle Georgia State College in Macon with over 200 participants from over 40 colleges and universities. The keynotes for each of these conferences focused on Building Suicide Safer Communities.
The Georgia Suicide Prevention Information Network (GSPIN) website www.gspin.org supports awareness, coalitions, survivors groups and the interested public. During 2014 the website had over a million hits (1,408,116) and the traffic to the GSPIN website was higher than 2013 in each month of 2014 except December. Also, this year GSPIN developed two password protected communities to serve the suicide prevention coalitions and for Georgia's colleges and universities. These online communities were developed for the sharing of information and mutual support.
With a more aware general public, there is a need to identify people at high risk of suicide in the general public and assist them in accessing care. In order to address the access to care issue, the Suicide Prevention Program supported two evidence based gatekeeper trainings. Gatekeepers act as outreach liaisons to provide their community with information about how to identify someone at high risk of suicide, how to encourage the person to get help, and how to access behavioral health and crisis services. The programs are called: Question, Persuade, and Refer (QPR) and Mental Health First Aid (MHFA) and are for both adults and youth. These programs teach community members to recognize the signs of suicidal behavior and direct individuals to assistance. Between January 1, 2014 and December 31, 2014, DBHDD trained at least 500 Georgia citizens in QPR and 500 citizens in mental health first aid. The training was provided throughout the State and included 25 QPR trainings, 26 adult Mental Health First Aid trainings
35

and 4 Youth Mental Health First Aid trainings in counties to community members in churches, schools, libraries and other community settings. There was a reduction in Youth Mental Health First Aid courses given through DBHDD. Additionally, there was a SAMSHA grant opportunity to support Youth Mental Health through the school systems that was awarded to the Department of Education to support a total of five counties.
To expand the use of QPR in Georgia communities and support its sustainability, the Suicide Prevention Program supported eight QPR Instructor Training events in each of the Department's Regions and added 119 new certified trainers to the existing group of certified QPR trainers throughout the state. These new trainers were recruited from our coalitions, colleges and universities, the schools and agencies that serve the jails, older adults and foster children. In October 2012, the Suicide Prevention Program and the federal CHIPRA program collaborated to sponsor a Train the Trainer for Youth Mental Health First Aid and 17 individuals were certified. Fourteen of these individuals provided three trainings each during 2013 and continue to be in the trainer pool for Georgia for 2014.
The Suicide Prevention Program, through its contractor, The Suicide Prevention Action Network of Georgia (SPAN-G), revised the suicide prevention training segments in the Crisis Intervention Team (CIT) trainings coordinated by the National Alliance on Mental Illness (NAMI) that is given to law enforcement and first responders throughout Georgia. In addition to identification of suicide, the program now contains information about supporting and managing suicide survivors at the scene of a death and on self-care. This module has been expanded into two modules, the first on suicide and the second on self-help and peer to peer support. During 2014 SPAN-GA gave 28 trainings in the revised Suicide module during CIT trainings to approximately 1,000 personnel from The Georgia Bureau of Investigation (GBI), Sheriff's Offices, Police Departments, High School Security, Pardons and Parole, Emergency Medical Service (EMS) and Fire Departments.
Once there is awareness and training to the general public and agencies that deal with individuals at high risk of suicide that referral for care is needed, the behavioral health network needs to be trained to further screen, assess and treat individuals at risk of suicide. The program staff worked with experts from the New York State Psychiatric Institute consisting of Dr. Barbara Stanley from the Suicide Intervention Center and Dr. Kelly Posner from the Center for Suicide Risk Assessment in order to address provider needs for screening, intervention and follow up which were identified as a result of death reviews. Additionally the program staff worked with Dr. Doreen Marshall, Associate Dean of Counseling at Argosy University, to design an evidence-based program for the Department's providers. By the end of 2012 the Suicide Prevention Evidence-Based Practice Initiative (SPEBP) had begun. Level 1 of the SPEBP Initiative involves:
Using the CDC's (Center for Disease Control) Self-Directed Violence Surveillance: Uniform Definitions and Data Elements to address lack of common definitions in reporting suicidal behavior,
Using The Columbia Suicide Severity Rating Scale (C-SSRS) to address lack of an effective process to identify people at risk of suicide,
36

Using Drs. Barbara Stanley and Greg Brown's Safety Planning and Follow-up Tool (brief interventions) to address lack of immediate interventions for those at risk of suicide but who don't need to be hospitalized.
Providing training to our provider leadership in the current best practices in Assessing and Managing Suicide Risk with a focus on basic competencies.
Taken together, the elements above form DBHDD's Suicide Prevention Evidence Based Practice (SPEBP) Initiative called A.I.M. (Assessment, Intervention, and Monitoring) with the outcome of identification, brief intervention and monitoring of consumers who are at high risk of suicide move toward the goal of helping them become securely situated in services and more empowered to act in their own self-interest.
During 2014 the Suicide Prevention Program staff continued to provide a variety of A.I.M process training activities. Monthly one hour "Introduction to A.I.M." webinars were held in January, February and March 2014. Over 100 individuals participated in these introductory webinars during 2014. Another 100 participants attended 4 A.I.M. skill building days for DBHDD providers and another 132 participants attended the 2 A.I.M. trainings for crisis providers in February 2014 in Lawrenceville and Macon.
To further address the need for information about assessment skills, one Assessing and Managing Suicide Risk for Mental Health Professionals training provided by the SAMHSA funded Suicide Prevention Resource Center was taught by Maureen Underwood to clinical leadership in DBHDD provider organizations (19 attendees). Together with the 140 clinical leaders trained in 2013 this provides Georgia with a group of at least 150 professionals who can be further trained to deliver this basic clinical course in their own behavioral health agencies. Plans for a train the trainer are under way for FY 2015.
During 2014 concern has grown among school systems about the number of students who respond that they have seriously considered suicide or made a suicide attempt in Georgia's Student Health Survey II given in all of Georgia's middle and high schools. In the last reported Student Health Survey II given in the 2013/2014 school year 54,859 students in grades 6 through 12 said they had seriously considered suicide in the last year and 31,346 said that they had attempted suicide in the last year. Since these numbers represent over 9% and 5% respectively of middle and high school students on average from each school system the Suicide Prevention Program has had many requests for assistance in training school personnel and developing protocol relating to suicidal students. In response to the demand the Suicide Prevention Program again provided training in the LIFELINES: Intervention Program. During 2014 10 LIFELINES: Intervention Programs were given to over 650 school personnel. Notable among these trainings was the partnered effort with Gwinnett County Schools where over 500 social workers, psychologists, counselors, school nurses, and school safety officers were trained over 5 days in preparation for the newly revised county protocol.
Postvention, intervening when there has been a suicide death, is becoming more and more a focus of the Suicide Prevention Program. Working with Those Bereaved by Suicide for Mental Health Providers was developed by Dr. Doreen Marshall to help behavioral health providers understand how to help those bereaved by suicide in behavioral health settings, including how to
37

help professionals bereaved by suicide. In 2014, Dr. Marshall taught 4 workshops in Working with the Bereaved, one in four of DBHDD's six regions to over 150 mental health providers.
The Suicide Prevention Program also provided ongoing postvention suicide training to the schools through its LIFELINES: Postvention programs. Three LIFELINES: Postvention trainings were provided to teams of school personnel and community professionals who work with school staff after a suicide death of a young person. This program trained over 100 school and behavioral health personnel to respond effectively to suicide deaths in the schools.
Additionally, DBHDD provides training to teams of survivors of suicide and other committed individuals and technical assistance to these teams in developing and running groups. During 2014 there were 27 Survivors of Suicide Groups (SOS) groups operating in Georgia covering all 6 DBHDD regions. Training was held to prepare new SOS group leaders in February 2014 and 13 new group leaders were trained. During this year groups were established in Albany and Gwinnett County. Groups are currently being developed in Rabun and Cobb Counties. Leaders were also trained to join current teams for sustainability in Gwinnett, Columbus and Kennesaw. Additionally, 13 people were trained or retrained to deliver the family survivor program for communities called Starfish. Again in 2014, Camp SOS, a weekend camp for families, was held for fifteen families of children, parents, and grandparents ages 6 to 70. This represents a tripling of the attendance from 2013.
Educational and outreach materials (purple packets) were designed that included materials from the Link Counseling Center, the American Association of Suicidology, identification of crisis service providers and crisis telephone numbers. Purple packets are disseminated to survivors of suicide by first responders, mental health professionals, funeral directors, clergy and others who encounter survivors of suicide death. Purple packets were provided to DBHDD providers who attended gatekeeper and A.I.M. trainings and supplies of purple packets were given out at the Coalition. In 2014 over 4,000 purple packets were disseminated throughout the state.
The DBHDDs Suicide Prevention staff continues to provide on-site and telephone consultation with providers who have experienced the death of a consumer by suicide, participate in meetings of the Executive Quality Council, the Community Behavioral Health Program Quality Council, the DD Program Quality Council and the Community Mortality Review Committee. Consultation to providers included introduction to the EBP Initiative and A.I.M program. As part of its consultation to other agencies in Georgia there were three on-site visits with school systems experiencing a large number of deaths, including suicide deaths.
There have been coordinated efforts with the Georgia Department of Human Services and Georgia Divisions of Aging and Family & Children's Services in order to assist with planning for future suicide prevention initiatives. Suicide Safer Communities was developed for state agency personnel and other community members to introduce the core principles of providing prevention, identification, intervention, and postvention. These trainings disseminated the core principles to people who work with the elderly as well as those who work in the schools and higher education settings.
Additionally, DBHDD and the Garrett Lee Smith Youth Suicide Prevention Program contracted with the University of Rochester's Dr. Peter Wyman to provide resources and technical
38

assistance for selected communities in Georgia to implement the Sources of Strength Youth Suicide Prevention Program with a high degree of fidelity in middle and high schools in the project's target communities from 2010 through 2014. The overall project objectives are: (a) to increase healthy coping practices to reduce the numbers of youth who become suicidal. (b) to connect potentially suicidal youth with capable adults. During 2014 DBHDD had contracts with two local agencies (CETPA which serves the Latino community and The Southern Jewish Resource Network) and five school systems.
During 2015, the Suicide Prevention Program staff anticipate researching, developing the infrastructure and implementing Key Performance Indicators for the Suicide Prevention Program.
Office of Deaf Services In April 2014, the Office of Deaf Services (DS) began the process of obtaining the information needed to ensure quality provision of behavioral health & developmental disabilities services to individuals with hearing loss and developing policies and practices to implement new standards of care.
Goals of Deaf Services for 2014 included: gathering information and developing a baseline array of statewide community based behavioral health services for deaf individuals promoting best practices in behavioral health American Sign Language (ASL) interpreting
An initial standard/performance indicator was developed in July 2014 and included in the Comprehensive Community Provider (CCP) requirements. The intent of this standard is to require that community based providers offer accessible services to deaf and hard of hearing individuals. The first task of this standard requires providers to notify DS at intake of all newly enrolled individuals with any level of hearing loss. In response, the DS provides a brief communication screening and if necessary, a full communication assessment and provides a report of the results to be incorporated within the individual's treatment plan. The second task requires that providers and DS work together to gather data to develop further performance indicators and to establish, provide, and oversee the quality of accessible services.
To promote best practices in ASL interpreting services for individuals with behavioral health conditions, DS has created a credential for those individuals who provide interpreter services to deaf individuals with BH issues receiving services from DBHDD providers. Beginning in August of 2014, specialty practicum training was initiated for those who have already earned the generalist certification as an ASL interpreter (as awarded by the Registry of Interpreters for the Deaf, Inc.). Those successfully completing the intensive three-pronged process (including the practicum) will earn the credential of Georgia Behavioral Health Interpreter (GaBHI). As the credentialed workforce grows, the DBHDD will first prioritize and then require the use of GaBHIs for direct behavioral health services. DBHDD has hired ten (10) part-time interpreters in the process of earning the GaBHI credential. As a result, from September to November, the hours provided by these qualified staff interpreters increased from 26% to 43% of the total interpretation services provided directly by DBHDD to its constituents.
39

DS continues to develop and refine a work plan to guide the quality management activities within its area of responsibility. This work plan encompasses a statewide review of said services and is based on an interdepartmental effort and guided by stakeholder and provider input.
Division of Developmental Disability Quality Management Reviews The purpose of the Person Centered Review (PCR) is to assess the effectiveness of and the satisfaction individuals have with the service delivery system. The Division of DD's External Quality Review Organization (Delmarva) utilizes interviews, observations and record reviews to compile a well-rounded picture of the individual's circle of supports and how involved the person was in the decisions and plans laid out for that person. The Division of DD also conducts PCRs with Individuals who have Recently Transitioned to the Community (IRTC). This allows DD to compare and evaluate the success of the transition.

The time period for DD data reported here is December 2013 through November 30, 2014. December 2014 data was not available at the time of the writing of this report, but will be included in the 2015 Interim QM Report.

Below, are results for: Individuals who recently transitioned from an institution to the community (IRTC) and participated in a Person Centered Review (PCR); A group of randomly selected individuals who were receiving waiver services, already established in the community (Established) and participated in a PCR; The previous year's IRTC interviews; Quality Enhancement Provider Reviews (QEPR), including the Qualification and Training, as well as provider Strengths and Barriers; Follow up with Technical Assistance (Follow Up w/TA) and the Follow Up with Technical Assistance Consultation (FUTAC).
Between December 20131 and November 2014 a total of 87 new IRTC interviews and 449 Established individual interviews have been completed. The following tables display results for IRTC individuals compared to the Established individuals, as well as the previous year's IRTC results when appropriate.

While individuals in both groups, IRTC and Established, were more likely to be male, there are some large demographic differences between the groups. Individuals who had recently transitioned to the community were:
More likely to be older, age 45 and over (76% v 40%); Much more likely to live in a group home (89% v 28%); Much more likely to have a profound intellectual disability (67% v 10%); More likely to receive services through the COMP waiver (100% v 62%).

Region

Table 1. Demographic Characteristics Dec 2013 - Nov 2014 IRTC

Established

1 The last Annual Quality Management Report reported PCR data collected through November 2013.
40

1 2 3 4 5 6 Gender Female Male Age Group 18-25 26-44 45-54 55-64 65+ Home Type Group home Host home Other Own place With parent Disability Autism Cerebral Palsy and Other Intellectual Disability Profound Intellectual Disability Waiver GIA NOW COMP
Total

N

%

5

5.7%

18 20.7%

13 14.9%

29 33.3%

10 11.5%

12 13.8%

N % 65 14.5% 75 16.7% 137 30.5% 64 14.3% 56 12.5% 52 11.6%

31 35.6% 56 64.4%

163 36.3% 286 63.7%

5

5.7%

16 18.4%

27 31.0%

31 35.6%

8

9.2%

58 12.9% 213 47.4% 98 21.8% 53 11.8% 27 6.0%

77 88.5%

9 10.3%

0

-

1

1.1%

0

-

127 28.3% 67 14.9% 3 0.7% 65 14.5% 187 41.6%

0

-

0

-

29 33.3%

58 66.7%

9 2.0% 1 0.2% 393 87.5% 46 10.2%

0

-

0

-

87 100.0%

87

44 9.8% 128 28.5% 277 61.7% 449

Table 2 displays information from the face to face interviews with individuals (Individual Interview Instrument or III), providing their perspective on the outcomes measured. Results are positive, with an average rate of 85 percent of outcomes present for the IRTC group and 91% for the Established group.
Compared to the Established population, IRTC results were similar except on the following Standards:
41

Person is developing desired social roles (IRTC group is 27.5 percentage points lower) Person is involved in the design of the service plan (IRTC group is 13.7 points lower) Person is afforded choice of services and supports (IRTC group is 11.6 points lower)

Compared to last year's IRTC results, eight out of the fifteen standards have had various degrees of improvements, and the other eight standards had slight declines. On average, this year's results are similar to last year's.

Table 2: Individual Interview Instrument

Results by Standard

Dec 2013 - Nov 2014

Standard

IRTC Established

N=87 N=449

1. The person is afforded choice of services and supports.

81.4%

94.4%

2. The person is involved in the design of the service plan.

75.6%

89.3%

Jan-Nov 2013 IRTC
N=165
89.1% 78.8%

3. The service plan is reviewed with the person, who can make changes.

75.9%

81.5%

67.9%

4. The person's goals and dreams are reflected in supports and services.

86.2%

91.8%

90.3%

5. The person is achieving desired outcomes/goals

97.7%

96.7%

97.0%

6. The person actively participates in decisions concerning his or her life.

82.8%

93.1%

82.4%

7. The person is satisfied with the supports and services received.

98.9%

94.7%

96.4%

8. The person is free from abuse, neglect and exploitation. 9. The person is healthy. 10. The person is safe or has self-preservation skills.

97.7% 95.4% 94.3%

98.7% 92.9% 94.7%

95.8% 94.5% 95.8%

11. The person is educated and assisted to learn about and exercise rights.

75.9%

87.8%

78.8%

12. The person is treated with dignity/respect. 13. The person's preferences related to privacy are upheld.

100.0% 98.9%

98.7% 98.9%

99.4% 98.8%

14. The person has opportunities to access and participate in community activities.

75.6%

83.5%

85.5%

15. The person is developing desired social roles.

38.4%

65.9%

39.9%

Average

85.0% 90.8%

86.0%

Delmarva Quality Improvement Consultants (QIC) review each person's Individual Support Plan with a Quality Checklist (ISP QA) to determine an overall rating for each individual reviewed, based upon the degree to which the ISP is written to provide a meaningful life for the individual receiving services. There are three different categories for each ISP.

42

1. Service Life: The ISP supports a life with basic paid services and paid supports. The person's needs that are "important for" the person are addressed, such as health and safety. However, there is not an organized effort to support a person in obtaining other expressed desires that are "important to" the person, such as getting a driver's license, having a home, or acting in a play. The individual is not connected to the community and has not developed social roles, but expresses a desire to do so.
2. Good but Paid Life: The ISP supports a life with connections to various supports and services (paid and non-paid). Expressed goals that are "important to" the person are present, indicating the person is obtaining goals and desires beyond basic health and safety needs. The person may go out into the community but with only limited integration into community activities. For example, the person may go to church or participate in Special Olympics. However, real community connections are lacking, such as singing in the church choir or being part of an organized team, and the person indicates he or she wants to achieve more.
3. Community Life: The ISP supports a life with the desired level of integration in the community and in various settings preferred by the person. The person has friends and support beyond providers and family members. The person has developed social roles that are meaningful to that person, such as belonging to a Red Hat club or a book club or having employment in a competitive rather than segregated environment. Rather than just going to church the person may be an usher at the church or sing in the choir. Relationships developed in the community are reciprocal. The ISP is written with goals that help support people in moving toward a Community Life, as the person chooses.
The distribution of the ISP rating from this year and last year is presented in Figure 1. For individuals who transitioned from an institution in 2014, 36 percent of the ISPs were written to support a Service Life, which is greater than the established population (17%) but less than last year's IRTC results (44%). Only one percent of ISPs in this year's IRTC group were written to support a Community Life, which is lower than the established population but the same as last year's IRTC results.
43

Figure 1: ISP QA Checklist Results by How ISP is Written
100%

75%

50%

44%

36%

75%
63% 55%

25%

17%

0% Service Life

Good But Paid Life

8%

1%

1%

Community Life

IRTC CY2014 (87) Established CY2014 (448) IRTC CY2013 (165)

During the Person Centered Review process, a record review is completed for all providers offering services to the individual at the time of the review. Therefore, provider documentation is examined for each service the individual receives. For the 87 individuals who transitioned from an institution, 145 provider records were reviewed. Results for each standard reviewed are presented in Table 3. On average, IRTC results are slightly lower than for individuals established in the community (60% v 64%), particularly on the following standards, where IRTC results were approximately 15 percentage points lower:
Personal funds are managed by individual and protected (14.9 points lower) Potential risk to individuals/staff/others is managed (15.1 points lower) Individual is afforded choices of services &supports (15.8 points lower) Individual chooses community services/supports (15.3 points lower)
But on the following standards, IRTC results were better than the established group: Means to identify health status and safety needs (15.6 points higher) Positive behavior support plans are in place (24 points higher)
The other low scoring standards for the IRTC group are: person centered focus is supported in the documentation (18.6% met); and documenting how the individual directs supports and services (26.3% met). These two standards are also among the lowest scoring standards for the Established group. When comparing this year's IRTC results with last year's, this year's average is slightly better (59.9% vs 58.6%), with decreases on some standards but improvements others.

44

Table 3: Provider Record Review

Results by Standard

Dec 2013 - Nov 2014

Standard

IRTC

Established

N=145

N=763

1. Person centered focus supported in documentation.

18.6%

32.4%

2. Human and civil rights are maintained.

62.8%

60.3%

3. Personal funds managed by individual and protected.

44.1%

59.0%

4. Clear description of services/supports/care/treatment. 73.8%

70.9%

5. The provider maintains a central record for individual.

98.6%

97.9%

6. Potential risk to individuals/staff/others is managed.

61.4%

76.5%

7. Information is protected, organized and confidential.

67.6%

79.4%

8. Medication oversight/administration.

95.4%

81.5%

9. Individual is afforded choices of services &supports.

46.2%

62.0%

10. Means to identify health status and safety needs

42.1%

26.5%

11. Means to evaluate quality/satisfaction of services.

89.7%

97.2%

12. Meets NOW/COMP documentation requirements.

95.2%

94.9%

13. Individual is making progress/achieving desired goals. 59.3%

71.0%

14. Individual directs supports and services.

26.3%

27.4%

15. Individual chooses community services/supports.

12.6%

29.9%

16. Positive behavior support plans are in place. *

77.4%

53.4%

Average

59.9%

63.9%

* New question effective Feb 2014

Jan-Nov 2013 IRTC
N=281
24.2% 62.1% 64.9% 63.7% 94.7% 78.9% 72.9% 85.1% 46.6% 33.2% 85.7% 87.1% 56.2% 18.5% 14.6%
-
58.6%

Every individual has a Support Coordinator who helps ensure the person receives needed services, delivered as prescribed in the ISP. Documentation maintained by the Support Coordinator for the person is reviewed during the Person Centered Review process. Results for the Support Coordinator Record Review (SCRR) are shown in Table 4. Overall, this year's results are lower than last year (54.8% vs 64.4%). The results for IRTC are lower than for individuals already established in the community (54.8% vs 61.7% on average), especially on these standards:

Person-centered focus shown in the documentation (12.8 points lower); Human and civil rights are maintained (14.6 points lower); Individuals are afforded choices of services and supports (25.2 points lower); Individuals are included into larger community (17.1 points lower)
When compared to last year's IRTC results, this year shows a substantial decrease in the following areas:
Person-centered focus shown in documentation (11.6%) Human and civil rights are maintained (36.4%); Documentation describes available services, supports and care of individual (16.3%)

45

Support coordinator monitors services/supports according to the ISP (19.5%) Individuals are afforded choices of services and supports (11.1%)
The results below also show some increase in the scores but they all are less than ten percent.

Table 4. Support Coordinator Record Review

Results by Standard

Dec 2013 - Nov 2014

Standard

IRTC Established

N=87 N=449

1. Person-centered focus shown in the documentation

30.2%

43.0%

2. Human and civil rights are maintained

44.2%

58.8%

Jan-Nov 2013 IRTC
N=165
41.8% 80.6%

3. Documentation describes available services, supports & care of individual

43.7%

46.1%

60.0%

4. Support coordinator monitors services/supports according to the ISP

59.3%

65.0%

78.8%

5. Support coordinator continuously evaluates supports and services

71.8%

68.4%

65.5%

6. Effective approach to assessing/making recommendations related to risk management

90.8%

83.8%

87.9%

7. Confidentiality of the individual's information is protected 8. Individuals are afforded choices of services and supports 9. Individuals are included into larger community.

100.0% 36.8% 15.3%

96.9% 61.9% 32.4%

98.2% 47.9% 17.9%

Average

54.8%

61.7%

64.4%

To help complete a well-rounded description of provider services, relevant providers/staff are interviewed. Results for the Staff Provider Interview are presented in Table 5. Findings are generally quite positive. IRTC results are slightly lower than for individuals already established in the community.

Table 5: Staff Provider Interview

Results by Standard

Dec 2013 - Nov 2014

Standard

IRTC

Established

N=145

N=763

1. Implementation of individual centered/directed supports and services.

88.6%

92.4%

2. Health

91.3%

94.5%

3. Safety

94.0%

95.4%

4. Rights Upheld

87.1%

94.4%

5. Privacy and Confidentiality

98.6%

99.3%

6. Respect and Dignity

99.3%

99.9%

Jan-Nov 2013 IRTC
N=281
87.3%
96.1% 84.0% 93.3% 99.5% 99.6%

46

7. Implementation of the plan's identified supports and services
Average

91.0% 91.2%

95.0% 94.7%

94.2% 91.5%

Observations are conducted for residential services (if not a family or own home) and day services programs. This year's results are similar to previous years on the standards measuring Health, Safety, Rights and Self Advocacy. However, IRTC results for the current year are much higher than last year, but still slightly lower than the Established group, on the standards measuring Community Life, Choice, and Celebrating Achievements.

Standard
1. Health 2. Safety 3. Rights and Self Advocacy 4. Community Life 5. My Life and My Choice 6. Celebrating Achievements
Average

Table 6. Observation Results by Standard

Dec 2013 - Nov 2014

IRTC N=140

Established N=631

93.3%

96.6%

94.8%

98.7%

98.4%

97.9%

87.5%

88.0%

96.6%

96.8%

94.2%

97.9%

95.4%

97.5%

Jan-Nov 2013 IRTC
N=275
97.4% 97.0% 97.3% 56.8% 89.6% 89.2%
93.5%

Quality Enhancement Provider Reviews (QEPR) The purpose of the Quality Enhancement Provider Reviews is to monitor providers to ensure they meet requirements set forth by the Medicaid waiver and Division of DD and to evaluate the effectiveness of their service delivery system.
Between October 20132 and November 2014, the Quality Enhancement Provider Review (QEPR) was completed for 46 service providers and each of the QEPR included an Administrative Review of Qualifications and Training.
The average compliance score for the 46 providers reviewed was 56.8%, lower than the previous reporting period (69.0% from Jan-Sep 2013).
Providers continue to score relatively low in the area of completing a minimum of 16 hours of annual training (42.7%), job descriptions are in place for all personnel (50.0%), and receiving training within 60 days after hire and then annually (54.0%).
Some areas that had a large decrease from last year's scores relate to providers having a current certification from DBHDD (23.1%) and Proxy Caregivers with the necessary training (26.6%)
2 The last Annual QM report provided QEPR data collected through September 2013.
47

Number Questions
4 2 6 2
4
15 7
1
1 1
2 45

Table 7. Administrative Qualifications and Training Elements in Quality Enhancement Provider Reviews

Expectations

Oct 2013Nov 2014
N=46

The type and number of professional staff attached to the organization are properly Trained, Licensed, Credentialed, Experienced and Competent.

70.0%

The type and number of all other staff attached to the organization are properly Trained, Licensed, Credentialed, Experienced and Competent.
Job descriptions are in place for all personnel. There is evidence that a national criminal records check (NCIC) is completed for all employees. Orientation requirements are specified for all staff. Prior to direct contact with consumers, all staff and volunteer staff shall be trained and show evidence of competence.

72.7% 50.0% 66.3%
66.0%

Within the first sixty days, and annually thereafter, all staff having direct contact with consumers shall have all required annual training. Provider ensures that staff receives a minimum of 16 hours of annual training. Organizations having oversight for medication or that administer medication follow federal and state laws, rules, regulations and best practices.

54.0% 42.7% 64.3%

Provider has a current certification from DBHDD (receives less than $250,000 waiver dollars per year).

76.9%

Provider has the required current accreditation if required (receives $250,000 or more waiver dollars per year). DD providers using Proxy Caregivers must receive training that includes knowledge and skills to perform any identified specialized health maintenance activity.

82.4% 66.7%

Average

56.8%

Jan-Sep 2013 N=26 79.7% 76.0% 67.9% 82.4%
78.6%
61.3% 60.3%
70.8%
100.0% 89.5%
93.3% 69.0%

During the QEPR, Delmarva works with each provider to identify strengths and best practices as well as barriers providers face in developing optimal service delivery systems. A total of 614 strengths were identified, and a total of 368 barriers were documented during the reviews completed between October 2013 and November 2014. Providers may have identified more than one strength or barrier, but each will be recorded only one time per provider.

Many of the strengths identified reflected areas of satisfaction with supports and services, receptiveness to improving quality, accessibility, flexibility and respect.
Barriers noted by many of the providers include excessive paperwork and lack of financial resources (cost of doing business vs. reimbursement rates), conflicting messages (regulation versus person centered approach) and not having the support plan driven by the person.
Using findings from the QEPR, a follow up review takes place to determine if the provider made corrections and implemented recommendations to improve their service delivery. Technical
48

assistance is also offered to providers to support continued quality improvement. Two technical assistance processes are implemented: the Follow up with Technical Assistance (Follow Up w/TA) and the Follow Up with Technical Assistance Consultation (FUTAC). The Follow Up w/ TA is conducted approximately ninety days after completion of the QEPR. From October 2013 to November 2014, 26 Follow Up w/ TA reviews and 431 FUTAC were completed.
Table 8 shows the types, referral sources and reasons of the FUTACs. Most of the reviews were onsite (92.3%), referred at the individual level (88.9%), the source of the referral was from one of the Regional Office HQMs (92.6%), with the Support Coordinator monthly score of a 3 or 4 as the primary reason for the referral (91.2%).

Table 8. Follow Up With Technical Assistance (FUTAC) Oct 2013 - Nov 2014

Type

Desk Review

33

Onsite Review

397

Referral Source

Division

2

HQM

399

Internal

10

Other Regional Office Staff

6

Provider

14

Referral Reason Level

Individual

383

Provider

48

Table 8 Continued. Follow Up With Technical Assistance (FUTAC) Oct 2013 - Nov 2014

Referral Reason Level

SC Monthly Monitoring Scores of 3 & 4s

393

Corrective Action Plan (CAP)/Critical Incident

2

Provider Self Request

25

Complaints/Grievance

7

QEPR Alert

0

PCR Alert

4

Compliance Review

0

Support Plan Needing Improvement

0

Level of Care Registered Nurse (LOC RN) Review

0

Table 9 shows the Focused Outcome Area addressed and technical assistance provided. Health, Safety and Provider Record Review documentation were most often the Focused Outcome Area addressed. Technical assistance most often included discussion with the provider and brainstorming.

49

Table 9. Follow Up With Technical Assistance (FUTAC) Oct 2013 - Nov 2014
Focus Outcome Area

Health

290

Safety

194

Rights

74

Choice

8

Community Life

25

Person Centered

55

Administrative P&P

4

Administrative Q&T

10

Documentation PRRG

356

Documentation ISPQA

3

Technical Assistance Provided

1:1 training

100

Brainstorming

238

Group Training

31

Individual Discussion

340

Strategic Planning

32

CAP Development

12

Resources-Hard Copy

80

Group Discussion

54

Resources-web-based

132

Role Play

8

Skill Building

68

Provider Strength

1. Most Frequently Noted Provider Strengths October 2013 - November 2014

Customer's satisfaction with supports and services Receptiveness to improving their quality of supports and services Attitude of putting the persons served first Respect for individuals served Provider is flexible People served have direct access to management and leadership staff Dependability Longevity with the individuals served Responsiveness to individuals' needs

Times Noted
33 31 27 27 23 21 18 18 17

Percent N=614
5% 5% 4% 4% 4% 3% 3% 3% 3%

50

2. Most Frequently Noted Provider Barriers October 2013 - November 2014
Provider Barrier
Cost of doing business vs. reimbursement rates Excessive paperwork requirements Documentation not reflective of person centered approach Lack of implementation of Person Centered Tools (i.e. Important To/For; Good Day/Bad Day) Support plan not driven by the person Shortage of internal self-assessment and quality assurance practices Multiple oversight organizations with differing agendas Ineffective or lack of training for provider/staff

Times Noted
22 15 11
11
11 10 10 10

Percent N=368
6% 4% 3%
3%
3% 3% 3% 3%

DD Transition Quality Review Analysis During 2013 six Regional Quality Review Teams were developed to review and approve all consumer transitions to the community prior to the transition. The goal of the Regional Quality Review Teams was to ensure individuals with DD who transitioned from state hospitals received adequate services and supports in a safe environment. This quality review analysis continued during 2014.

Pioneer Project The Pioneer Project was developed to assess and develop stability of placements of waiver participants based on the Settlement Agreement in Region 2. A "Core Team "was developed which included members from a team of consultants (CRA), DBHDD Regional Office/State Office staff, and grew to include participants from support coordination and others. The charge of the Core Team was to establish a way to determine stability and develop processes to improve stability. This included work focused on providers and improving Support Coordination.

To do this, all of the providers (residential 21, total provider base-36, the 36 includes residential and "day" services) met with the Core Team to discuss approaches they would use around person centered service delivery, to discuss a "success story," and to determine what DBHDD could do to better support them in serving all their participants as well as ADA participants.

In an effort to address the issues found in the Georgia State analysis, DBHDD is re-evaluating the current transition process, is developing CAPS, and will be taking additional steps to increase the quality of those transitions. The outcome of DBHDD's transition quality improvement efforts will be reported in the 2014 Interim Report.
2014 Specialized DD Quality Improvement Study: Provider Systems and Driver Outcomes

Each year, the Division of DD conducts a Quality Improvement Study. The topic of the study is data driven based on trends that have been seen over the previous year(s). In 2014, the Division decided to look how current Provider systems of operation act as drivers of positive outcomes.

51

Several studies have identified "driver" outcomes. When present in someone's life, there is a greater likelihood other outcomes will also be present. In this study we use data from Delmarva reviews completed between July 2010 and March 2014 to identify Driver Outcomes for individuals receiving services through Home and Community Based Waiver services as part of the Georgia Quality Management System. Multivariate analytic techniques were used to generate two specific Driver Outcomes: Person Centered Planning (PCP) and Community Integration and Rights (CIR). Logistic Regression models were developed to examine the net impact of several different explanatory or independent variables on each of these outcomes. Including demographic characteristics, we examine the impact of provider performance in documenting the implementation of various policies and organizational procedures (Provider Record Reviews).
Results indicate that when controlling for other factors in the model, the type of residence, disability, and services received are associated with the person's likelihood of having outcomes related to input into services, community integration, decision making and rights present. Several different aspects of the provider's systems were the strongest predictors of outcomes, including documentation that individuals had a choice of services and supports, were given a choice of community services and supports, and were able to direct their own services and supports.
Recommendations were developed based upon the evidence presented in the study. The Division of DD is reviewing those recommendations, and will report on the outcomes in the 2015 Interim Report. Please see Attachment 2 for the full Quality Improvement Study.
DBHDD Quality Management Training Program
During 2014, the second QM web based training module was released for completion by DBHDD. The target date for completion of the first module was July 31, 2014 which was met with good compliance.
In the Division of Developmental Disabilities the initial Training and Education (T&E) plan, was to develop training materials based upon the Division's new Individual Service Plan process and Electronic Individual Service Plan (eISP).
A stakeholder workgroup consisting of stakeholder members from the redesign workgroup for the ISP process and eISP, Division staff and Delmarva, planned for the development and roll out of the eISP training statewide. A 150 page step by step manual Train the Trainer Manual and slide presentation were developed for Regional Office and Support Coordination representatives designated as "Master Trainers", with Delmarva providing the Train the Trainer sessions. Master Trainers can then use the manual and PowerPoint presentation to train providers in their region on the new process and the eISP. In January, the Master Trainers were trained on the curriculum.
Based upon feedback from the Master Trainers, the manual, curriculum and slide presentation were modified. Delmarva staff also worked with Columbus Community Services' IT to create the new eISP in the Consumer Information System (CIS). Provider training on the new ISP and eISP was to be scheduled from March April. However, in February, the Division decided, based
52

on the implementation timeline for the new ASO, to postpone the implementation of the new ISP process and eISP until after July 2014.
A new T&E Plan was developed for approval on two different training sessions to be completed prior to the end of the year. Based upon data and stakeholder feedback and anecdotal information, curriculum was developed to support individuals' health and wellbeing and value individuals' choices in life: Quality Health and Safety Management for Nursing and Developmental Disabilities Professionals and Valued Visions. Training modules were approved and 30 sessions were completed. Over 470 individuals attended these sessions. .
The Quality Health and Safety Management for Nursing and Developmental Disabilities Professional training focused on duties and responsibilities for RNs and DDPs including quality risk management, based upon CMS requirements and state standards. Feedback from participants indicated the training was useful, concise, and gave a better understanding of the RN and DDPs' roles in health and safety management for the individuals served. The Valued Vision training focused on how individuals receiving supports and services and direct support staff can develop valued decision making skills. Included in the presentation was the reintroduction of the Vision Workbook developed in collaboration with HSRI.
Data Reliability Process
Accurate and reliable data is imperative for the success of the DBHDD QM Program. Some of the DBHDDs data integrity activities include:
Hospital System KPI Data Integrity The Hospital System Quality Management office has utilized the Performance Measure Evaluation Tool (PMET) to identify and assess those KPIs that need additional work in order to assure data integrity. The Hospital System PQC has prioritized data integrity as an important issue and the Assistant Director of Hospital System Quality Management is working with the Hospital Quality Managers Committee to make the needed improvements.
Beginning with the reporting period of January 2014, a report tool was developed that gives hospitals the ability to drill down directly to reported data failures and make needed corrections to data that is reported to The Joint Commission (commonly known as the HBIPs measures). Use of that tool resulted in several data-collection methodology changes, which improved the reliability of the data and timeliness of reporting.
In addition, beginning in December 2013, DBHDD's EMR system was improved to capture needed data directly from the physician electronic record. This improved data collection by eliminating interpretation and data re-entry of the reported data.
Community BH Key Performance Indicator Data Integrity The majority of the data that comprises the CBH KPIs is received from providers via a webbased monthly programmatic report. Once the data is received by DBHDD, the data must pass a logic safeguard validation and is reviewed by staff with programmatic oversight of each specific program and regional DBHDD office staff before it is accepted. Feedback is given to providers
53

when errors or omissions occur and they are required to re-complete and re-send their data once corrected. Technical assistance is provided as needed. DD KPI Data Integrity Every two weeks, the analyst working with the ERO (Delmarva) runs a report to identify any incorrect or missing data from the database. This process generates a report from data collected as part of the PCR and QEPR processes which is reviewed by managers, who correct any identified errors. In order to ensure proper handling of possible missing data or data errors, a Data Correction Protocol has been developed to track data errors and necessary correction. For approved reviews or reports, all changes in the data are documented in the "Reopen Review Log". This information is reviewed periodically by the quality improvement regional manager for possible trends. After the data in the report have been corrected, a new report is generated and distributed as necessary.
Summary
The sections above reference the multitude of quality related activities taking place across DBHDD. Key activities that have taken place between January 2014 and December 2014 include an annual review of the QM system review; the review and revision of KPIs; review and re-structuring in DD based upon a comprehensive system wide review of the DD QM system by an external contractor (DD Re-engineering project); the release of an RFP and the procurement of an ASO (currently known as the Georgia Collaborative ASO), the continuation of QM web based training, and significant communication with and training of providers on cognitive therapy (Beck Initiative), and suicide prevention. During 2015 the DBHDD QM Plan and QM work plans will be revised and it is anticipated that there will be significant changes to the DBHDD QM system as the Georgia Collaborative is integrated into the functioning of the Department.
54

Appendix A DBHDD Quality Management Work Plan

Goal 1: Develop accurate, effective and meaningful performance indicators.

Tasks

Responsible Person

Target Completion

Date

Review and assess current key Program QC chairs with

Jan 2015

performance indicators for

assist from Carol Zafiratos,

continued value and applicability Steve Holton, Eddie Towson

Collaborate with stakeholders

Program Quality Councils

December 2013

using the identified performance

measure evaluation tool (PMET)

criteria to develop key

performance indicators

Status
Met and ongoing

Goal: 2 Educate stakeholders regarding QM (includes staff, providers and ultimately individuals

and families).

Tasks

Responsible Person

Target Completion

Status

Date

Update the current QM Training Carol Zafiratos and Training

August 2014,

Plan and ensure inclusion of

Department

delayed to April

training for hospitals, CBH and

2015

DD

Complete development of two Carol Zafiratos and Training

April 2015

Second web

additional modules to DBHDD- Department

based training

wide web-based training

program

materials.

completed

and

distributed to

all DBHDD

staff.

Completed in

July 2014

Develop a pilot project to assess Carol Zafiratos, Eddie

June 2015

feasibility of using web based

Towson and Training

training modules for training

Department

community based providers

55

Goal: 3 Assess and improve the effectiveness of the QM system and its various components. This is a multi-year goal.

Tasks
Evaluate the utility of the PQC approved outcomes framework for use in the DD system and Hospital System. Each PQC evaluates their levels of achievement of work on their respective KPIs, utilizing, at a minimum, the Performance Measure evaluation tool. PQCs present their assessments of progress in meeting quality goals and thresholds established for KPIs to the EQC EQC evaluates progress of PQCs and makes recommendations or takes other action as appropriate. Modify QM system and/or components as needed

Responsible Person
Program Quality Council Chairpersons
Program Quality Council Chairpersons
Program Quality Council Chairpersons
Program Quality Council Chairpersons and EQC
Program Quality Council Chairpersons

Target Completion
Date September 2014 revised to December 2014 Aug 2015
Sept 2015
Sept 2015
Oct 2015

Status
Framework revised and new framework proposed and approved at the December 2014 EQC meeting

Goal 4: Integrate QM Data Systems (have access to the data needed that is compatible with the hospital, community BH and community DD systems and which follows an individual and the services they receive across their lifetime, as applicable). This is a multi-year goal.

Tasks
Perform a comprehensive QM data management needs assessment, Collect and inventory all Hospitals, Regional Offices, and Central

Responsible Person
CIO Business Analyst/consultant designee and Carol Zafiratos, Steve Holton and Eddie Towson

Target Completion Date
Phase 1 Business analysis and requirements-January 2015. Revised to May 2015. Phase 2 Implement

Status
Started June 1, 2014. Will have new Statement of Work (SOW) for a more comprehensive effort. One person was not

56

Office management reports, data used for decision making and processes used to collect and update data. Inventory how and who uses reports to make decision.

and integrate into SharePoint June 30 2015

Define and develop data sharing partnerships/agreements with other agencies.
Create a 5 year DBHDD Enterprise Information Technology and Information Systems Improvement Plan that emphasizes collaboration, communications, accountability, decision making, standardization, measures outcomes and quality Information.

DBHDD Leadership representative(s) [COO & Director of IT] Director of IT

December 2014 December 2014

Implement Business intelligence (BI) Analytics technologies like SharePoint that enable quicker turnaround or deployment of Information Management Systems Integrate ASO information Systems, DBHDD EMR (Hospital System AVATAR), CSB information systems using our Data Warehouse and technology like SharePoint to capture more information direct

Director of IT Director of IT

October 2014 2015

enough. The SOW for performing assessment will be reviewed and approved by Chief Medical Officer, Director Hospital Operation, and Director RHA's by 31 January 2015 Complete
Complete, Windows 7, Internet Explorer 11, e-mail migration from Groupwise to Microsoft Exchange, Novell to Windows network, Laptop encryption all completed over last 8 months for DBHDD. Office 365 e-mail in cloud will be completed by end of February 2015 Completed
On target for a July 2015 completion
57

care events surrounding patient. Systems need to capture all events that occur around a patient and integrate that data for better patient care management and decision making (I.E, patient transitions as a result of settlement, incidents, critical events that impact patient, outcomes, etc.) Evaluate the effectiveness and efficiency of the newly created system

Director of IT, Carol Zafiratos, Steve Holton and Eddie Towson

2016

On target

58

Appendix B Hospital System Quality Management Work Plan

Goal 1: Develop accurate, effective and meaningful performance indicators.

Tasks

Responsible Person

Target

Completion Date

Determine the criteria for

Carol Zafiratos

June 2013

developing the key performance

indicators

Identify and assess current

Steve Holton, Dr. Risby,

June 2013

performance indicators for value Carol Zafiratos

and applicability

Modify KPIs, as appropriate

Hospital System Quality

July 2013

Council

Develop and implement data

Steve Holton and Carol

collection plans for KPIs (identify Zafiratos

August 2013

responsible persons for data

entry, collection, reporting, etc.)

Status Completed
Completed
Completed Completed

Goal 2: Educate stakeholders regarding QM (includes staff, providers and ultimately

individuals and families).

Tasks

Responsible Person

Target

Status

Completion Date

Update the current QM Training Carol Zafiratos, Steve Holton June 2013 - Delayed In process

Plan and ensure inclusion of

and Training Department

until January 2014.

training for hospitals see

Revised March 2015

Appendix J for current plan

Identify desired knowledge,

Director of Hospital System August 2013

Completed

skills, abilities and behaviors for Quality Management

Delayed to

Hospital Quality Managers

December 2013

Assess training needs of QMs. Director of Hospital System September 15, 2013 Strategy was

Quality Management

which was delayed modified to

to February 2014. accommodate

New target date is DBHDD QM

July 2015

training plan

for PI team

facilitators

and leaders.

Collaborative

individual

assessment &

planning will

be done after

that training

59

Develop training plans and methodology for QMs.

Director of Hospital System Quality Management ,Carol Zafiratos and Training Department

November 1, 2013, which was delayed to March 2014. New target date is July 2015

program has been completed Strategy was modified to accommodate DBHDD QM training plan for PI team facilitators and leaders. Collaborative individual assessment & planning will be done after that training program has been completed

60

Goal 3: Assess and improve the effectiveness of the QM system and its various

components.

Tasks

Responsible Person

Target

Completion Date

Status

Set target values for Hospital System KPIs.

Dr. Emile Risby Chair Hospital System Program Quality Council

June 2013

Completed

Each hospital creates their data definition/collection plans

Program Quality Council Chairpersons

Each hospital identifies and

Program Quality Council

submits their KPIs (hospital level) Chairpersons

and PI goals to the HSPQC

Hospitals update analyses and begin to prepare reports for Hospital System QC (Quality Management effectiveness review meeting scheduled for March 2014)

Program Quality Council Chairpersons

March 2014 Task amended
March 2014. New dates to be added in next work plan revision. March 2014. New dates to be added in next work plan revision.

The focus of this goal was shifted in response to new leadership's desire to develop a new set of KPIs, with data collection plans, that all hospitals would utilize. Work continues on developing those new indicators. Next work plan will be modified to reflect those changes See above
See above

61

Goal 4: Integrate QM Data Systems (have access to the data needed that is compatible

with the hospital, community BH and community DD systems and which follows an

individual and the services they receive across their lifetime, as applicable).

Tasks

Responsible Person

Target

Status

Completion Date

Organize a Hospital System information management committee

Director of Hospital System Quality Management

Develop methodology for performing IM needs assessment

Chair of Information Management Committee & Director of Hospital System Quality Management

Perform needs assessment in hospitals and analyze results

Chair of Information Management Committee & Director of Hospital System Quality Management

Set priorities for IM needs and communicate priorities to OIT, as appropriate.
Develop Hospital System IM plan

Chair of Information Management Committee & Director of Hospital System Quality Management Chair of Information Management Committee & Director of Hospital System Quality Management

July 15, 2013
September 1, 2013, then revised with a new target date of April 2014. New proposal and statement of work should be completed by March 2015 November 1, 2013 then revised to April 2014. Revised target date: August 2015
December 1, 2013 revised to July 2014. Revised target date:
Oct 2015 November 2014 revised to November
2015

Completed a committee has been selected. Will initiate activities when the consultant has been hired Statement of work is being created by CIO as part of proposal to fund consultants

62

Appendix C Community Behavioral Health Quality Management Work Plan

Goal 1: Develop accurate, effective and meaningful performance indicators.

Tasks

Responsible Person

Target Completion

Date

Distribute Performance Measure Carol Zafiratos

July 2013

Evaluation Tool (PMET) to CBH

committee members

Utilize criteria (from PMET) to Chris Gault and CBH

September 2013

assess current KPI's

Program Staff

delayed but

completed in

December 2013

Use PMET and develop new

Chris Gault and CBH

October 2013

KPI's as indicated

Program Staff

Make recommendations regarding Chris Gault and CBH

October 2013

the infrastructure that is needed to Program Staff

delayed but

ensure data integrity and follow

completed in

up for new KPIs

December 2013

Collaborate with stakeholders to Chris Gault and CBH

October 2013

review and provide feedback on Program Staff

new KPI's

Develop data collection plans for Chris Gault and CBH

November 2013

new KPIs (identify responsible Program Staff

persons for data entry, collection,

reporting, etc.)

Implement data collection plans Chris Gault and CBH

January 2014

for new KPIs

Program Staff

Initiate provider based data

Resources need to be

March 2014 delayed

integrity reviews

identified

to May 2015

Status Completed Completed
Completed Completed
Completed Completed
Completed

Goal: 2 Educate stakeholders regarding QM (includes staff, providers and ultimately individuals

and families).

Tasks

Responsible Person

Target Completion

Status

Date

Develop and implement

CBH PQC and Carol

Start Date =

Completed

recommendations for the first

Zafiratos

September 2013

three quality management related

training modules for State and

Completion Date =

Regional Office BH staff

January 2014

Once approved implement the CBH Program Managers

Start Date = October Completed

training recommendations and

2013. Target

monitor compliance for state staff

completion February

2014

63

Develop a QM training plan for providers Develop a QM training plan for individuals served and families

CBH PQC, Chris Gault and Carol Zafiratos
CBH PQC, Chris Gault and Carol Zafiratos

January 2014, delayed to May 2015 March 2014, delayed
August 2015

Goal: 3 Assess and improve the effectiveness of the QM system and its various components.

This is a multi-year goal.

Tasks

Responsible Person

Target Completion

Status

Date

Using the PMT, annually review all KPI's for efficiency and effectiveness

CBH PQC

January 2015 In process

Goal 4: Integrate QM Data Systems (have access to the data needed that is compatible with the

hospital, community BH and community DD systems and which follows an individual and the

services they receive across their lifetime, as applicable). This is a multi-year goal.

Tasks

Responsible Person

Target Completion

Status

Date

Make recommendations based CBH PQC through Chris upon KPI selection for future data Gault needs

December 2013 Completed delayed until March and ongoing
2014

64

Appendix D Developmental Disabilties Quality Management Work Plan

Goal 1: Assess and improve the effectiveness of the QM System and its various components that assures quality person-centered supports and services for individuals with developmental disabilities. Goal 2: Develop accurate and meaningful performance indicators.

Tasks

Responsible Person

Documentation review (i.e. relevant policies and procedures, recent CMS Waiver changes, DOJ Settlement Agreement, etc.) Assessment of current data collection methods
Assessment of current data utilization
Interview Central and Regional Office staff to identify capabilities of quality practitioners Conduct Stakeholder interviews to determine capabilities of quality practitioners Conduct Focus Groups with targeted stakeholders to collect information on strengths, benefits and opportunities for improvement
Conduct Interviews with service provider and service coordination staff Conduct comparison of requirements generated by DBHDD to CMS and DOJ requirements Establish QI Council workgroup to design new

Director of DD Quality Management and Contractor
Director of DD Quality Management and ASO Director of DD Quality Management and Contractor Director of DD Quality Management and Contractor Director of DD Quality Management and Contractor Director of DD Quality Management and Contractor
Director of DD Quality Management and Contractor Director of DD Quality Management and Contractor Director of DD Quality

Target Completion
Date 06/30/13
07/31/13 Revised to 07/01/15 07/31/13
07/31/13
07/31/13
07/31/13
07/31/13
07/31/13
07/31/13 Revised to

Status Completed
Completed Completed Completed Completed
Completed Completed Planning timeline for design of new system

65

QM system with participation from DD Advisory Council

Management and Contractor

Develop report describing the status of the "as is" system
Develop recommendations for improvements to Georgia's quality system
As part of Goal 1 DD will establish accurate, effective, and meaningful performance indicators for DD Services and DD Providers Finalize measurements

Director of DD Quality Management and Contractor Director of DD Quality Management and Contractor Director of DD Quality Management and Contractor
Director of DD Quality Management and Contractor

Develop comprehensive description of redesign for statewide DD QM system

Director of DD Quality Management and Contractor

02/01/14 08/01/13

has been extended to allow for more thorough planning and development Completed

08/01/13 Revised to 02/01/14
08/15/13 Revised to 03/01/14

Completed Completed

09/15/30/13 Revised to 03/01/14 then to 12/31/14 and revised again to 03/01/15 10/01/13 Revised to 03/01/14

Planning timeline for design of new system has been extended to allow for more thorough planning and development

Goal 3: Educate Stakeholders regarding QM (including staff, providers, and individuals

and families

Tasks

Responsible

Target

Status

Person

Completion

Date

08/31/13.

Completed

Director of DD Revised to

Identify core knowledge and Quality

05.01.15

skill requirements for each Management and

quality role identified.

Dept Director of

QM

Review and analyze the instructional

Director of DD Quality

08/31/13 Revised to

Planning timeline for design of new system

66

system/knowledge and basic Management and

skill topics with DBHDD

Dept Director of

Staff and quality councils. QM

03/01/14 then again to 05/01/15

Develop materials and methods for learning management and curriculum development

Director of DD Quality Management and Dept Director of QM

Create DD training program draft and review with DBHDD Staff and Quality Councils

Director DD Quality Management

Finalize training program with input from Quality Councils and Advisory Council

Director DD Quality Management

Train staff and stakeholders on new DD QM System

Director DD Quality Management and Contractor

Draft a manual which includes the following sections:

Director of DD Quality Management and Contractor

09/30/13 Revised to 03/01/14 then again to 05/01/15 10/31/13 Revised to 04/01/14 then again to 05/01/15 11/15/13 Revised to 05/01/14 then again to 05/01/15
12/15/13 Revised to 08/01/14 then again to 05/01/15
12/15/13 Revised to 03/01/14 then again to 05/01/15

has been extended to allow for more thorough planning and development Development timeline has been extended to allow for more thorough planning and development Timeline has been adjusted as a result of extended planning and development period
Timeline has been adjusted as a result of extended planning and development period
Timeline has been adjusted as a result of extended planning and development period
Timeline has been adjusted as a result of extended planning and development period

QM and improvement requirements section
Roles and responsibilities section
Guidance on joint agency collaboration
Reporting requirements
Tools for data collection and
67

analysis

Review drafts of each section with DBHDD staff and QI Councils and Advisory Council

Director of DD Quality Management

12/31/13 Revised to 04/01/14 then again to 05/01/15

Timeline has been adjusted as a result of extended planning and development period

Goal 4: Ensure that individuals with DD transitioned out of state hospitals to receive high

quality services and to achieve life goals in community.

Tasks

Responsible

Target

Status

Person

Completion

Date

Develop the follow-up and Joseph Coleman, 04/01/13

Completed

monitoring process

Director of

6/5/13

Revisions completed to

Transitions DD

incorporate full review of

findings/reports by Central

Office

Finalize the audit tool

Joseph Coleman, 04/01/13

Completed

Director of

6/5/13

Revisions completed to

Transitions DD

utilize full monitoring tool

developed by DOJ

Identify the

Joseph Coleman, 04/01/13

Completed

reviewers/auditors

Director of

Transitions DD

Create, hire, train

Joseph Coleman, 7/1/13

Completed

Regional DD Transition Director of

Quality Review Team

Transitions DD,

and Rose Wilcox.

Director of

Training and

Education DD

Decide the process of data Joseph Coleman, 6/10/13

Completed

collection, reporting, and Director of

correcting problems

Transitions DD

identified

Review quality of

Joseph Coleman, 06/20/13

Completed. Results sent to

transition for 79

Director of

GSU for analysis

individuals who have

Transitions DD

Provider CAPs generated by

transitioned out of state

reviews submitted by

hospitals as of July 1,

Providers and

2012

reviewed/approved by

Region Office and Transition

Fidelity Committee

68

Pre-transition review of Provider capacity to ensure quality care for 40 individuals whose planned May/June transitions were postponed until after July 1, 2013 Review and revise the current transition process to develop a comprehensive process / plan

Joseph Coleman, Director of Transitions DD
Joseph Coleman, Director of Transitions DD

06/25/13 7/1/13

Completed Provider CAPs generated by reviews submitted by Providers and reviewed/approved by Region Office and Transition Fidelity Committee Work ongoing. Final revisions to transition process to be completed February, 2014

Goal 5: Integrate QM Data Systems in a matter which is compatible with Department data

systems (Hospital, Community BH and Community DD) which will allow Division to follow

an individual and their services across their lifetime. This is a multi-year goal.

Tasks

Responsible

Target

Status

Person

Completion

Date

08/01/13.

ASO (Georgia

Revised to

Collaborative) has been

05/01/15

procured and

implementation is

Director of DD

underway. There are two

Quality

teams of DBHDD and

Management

Collaborative staff that

are responsible for this

Develop Division DD

work. The Collaborative

information management

QM Team and the

committee

Collaborative IT Team

Ongoing

ASO (Georgia

Collaborative) has been

procured and

08/01/13

implementation is

Revised to

underway. DD staff are

Director of DD 07/1/15

working with Business

Assessment current

Quality

Analyst to develop work

information management Management and

flows for collection and

systems methods for

Division Data

utilization

collection and utilization Manager

Set priorities for IM needs Director of DD

Completed and ongoing.

and work with OIT to

Quality

10/01/13

ASO (Georgia

address those needs as

Management and

Collaborative) has been

appropriate.

Division Data

procured and

69

Manager

Include development of new DD case management system in the Department's RFP for an Administrative Service Organization (ASO). Revised to: Develop new ISP for inclusion in the Georgia Collaborative Case Management System Work with ASO to develop and test new system

Director of DD Quality Management
Director of DD Quality Management and Vendor

10/01/13 Revised to 07/01/15
08/01/14 Revised to 07/01/2015

Train end users on new system
Transition data from old case management system to new system

Director of DD Quality Management and Vendor Director of DD Quality Management and Vendor

10/01/14 Revised to 07/01/2015
12/31/14 Revised to 07/01/2015

implementation is underway. There are two teams of DBHDD and Collaborative staff that are responsible for this work. The Collaborative QM Team and the Collaborative IT Team Completed
Timeline adjusted to match ASO implementation timeline
Timeline adjusted to match ASO implementation timeline
Timeline adjusted to match ASO implementation timeline

70

Appendix E Hospital System KPI Dashboards
71

Client Perception of Empowerment

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30%

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Numerator Denominator Rate
Quarterly Average

Jan-14 96
131 73%

Feb-14 100 136 74%
74%

Mar-14 124 165 75%

Apr-14 110 160 69%

May-14 137 184 74%
72%

Jun-14 103 142 73%

Jul-14 Aug-14 Sep-14

74

109

136

120

140

182

62%

78%

75%

72%

Oct-14 129 164 79%

Nov-14 106 138 77%
77%

Dec-14 99
129 77%

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of clients at discharge or at annual review who respond positively to the empowerment domain on the Inpatient Consumer Survey.
Measure explanation: This measure shows client responses to the following questions: *I had a choice of treatment options. *My contact with my doctor was helpful. *My contact with nurses and therapist was helpful. (Source: NRI) The determination of the line where the red/yellow areas of the graph meet is based on the national average published by NRI for December 2013 through November 2014, less one standard deviation. (Data collection for surveys were started state-wide in February 2012.)

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of clients who respond positively to the Denominator: Number of clients completing at least 2 items in the

empowerment domain

empowerment domain Included populations: Clients who were

discharged during the period and completed at least 2 questions

in the domain. Only clients served in programs associated with

Adult Mental Health are surveyed.

COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
Scores have continued to fall well within the target range during this period, with an overall slight improment trend over the course of the year.
July-September 2014 Analysis
Despite the expected seasonal drop in rate, respondents still rated DBHDD's service higher than the national average.
April-June 2014 Analysis
Respondents continued to score above the average this quarter.
January-March 2014 Analysis
The rate continues its established upward trend. National average of this data continues to display relatively large fluctuations, but DBHDD continues to score above the average this quarter.
72

Continuing Care Plan Created (Overall)
100%

95%

90%

85%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Numerator Denominator Rate
Quarterly Average

Jan-14 Feb-14 Mar-14

293

261

301

296

264

303

99%

99%

99%

99%

Apr-14 May-14 Jun-14

339

320

316

344

332

322

99%

96%

98%

98%

Jul-14 Aug-14 Sep-14

293

278

310

296

282

312

99%

99%

99%

99%

Oct-14 Nov-14 Dec-14

294

224

275

300

225

280

98% 100%

98%

99%

MEASURE DEFINITION AND EXPLANATION Measure definition: Patients discharged from a hospital-based inpatient psychiatric setting with a continuing care plan that contains all of the following: reason for hospitalization, principal discharge diagnosis, discharge medications and next level of care recommendations.

Measure explanation: This measure is a nationally standardized performance measure for behavioral health organizations, reported to The Joint Commission through our partner, NRI, on a quarterly basis. The data are for people who were treated in adult mental health inpatient programs only. The colored bands represent ranges that indicate level of acceptibility of scores and are based The Joint Comission "Target Rates" published quarterly, 4 to 5 months after the quarter ends. The most recent rates published are used as guides for current data. The red area of the graph indicates the area that is below The Joint Commission's Target Range. The Joint Commission changed the target range in October 2012 from 93.4% to 94.4%.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Psychiatric inpatients for whom the post

Denominator: Psychiatric inpatient discharges. Included

discharge continuing care plan is created and contains all of Populations: Patients referred for next level of care with ICD-9-CM

the following: reason for hospitalization, principal discharge Principal or Other Diagnosis Codes for Mental Disorders.

diagnosis, discharge medications and next level of care

Excluded Populations: The following cases are excluded:

recommendations.

Patients who expired Patients with an unplanned departure

Included Populations: NA

resulting in discharge due to elopement or failing to return from

Excluded Populations: None

leave Patients or guardians who refused aftercare Patients or

guardians who refused to sign authorization to release information

Patients discharged to another unit within the same hospital

COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
Scores have continued to fall well within the target range during this period.

July-September 2014 Analysis
Rate remains above The Joint Commission guidelines. Continued monitoring is recommended.

April-June 2014 Analysis
Several individuals discharged this quarter were done so off Conditional Release. Due to the fact that the Conditional Release process may take several years and include court-orders, The Joint Commission guidelines are not officially met. However, the overall rate remains above The Joint Commission guidelines.
January-March 2014 Analysis
Data continues to trend towards 100%. Certain factors, such as clients discharged directly off of on long-term Conditional Release programs, will hinder reporting at 100%. In such cases, some semblance of a Continuing Care Plan was created at the time of release, but does not meet current The Joint Commission guidelines.

73

Individual Recovery Plan Audit - Quality Measure

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Numerator Denominator Rate
Quarterly Average

Jan-14 Feb-14 Mar-14

1892 2101 90%

1800 1971 91%

2007 2181 92%

91%

Apr-14 May-14 Jun-14

1793 1950 92%

1975 2158 92%

1819 1967 92%

92%

Jul-14 Aug-14 Sep-14

1570 1706 92%

1717 1851 93%

1627 1782 91%

92%

Oct-14 Nov-14 Dec-14

1466 1631 90%

1582 1781 89%

1381 1554 89%

89%

MEASURE DEFINITION AND EXPLANATION Measure definition: Percent of positive responses to the Individualized Recovery Plan audit's questions on "Quality."
Measure explanation: Chart audit focusing on the quality and internal-consistency of the Individualized Recovery Plan. Audit began January 2012.

COMPONENTS OF NUMERATOR AND DENOMINATOR Numerator: Total number of "Yes" responses to questions 2-Denominator: Total number IRP audits conducted. 20 on the IRP audit

COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
Scores have continued to fall well within the target range during this period. Possible causes for a slight down turn in the current quarter are being addressed and should be reflected in the next quarter's scores.
July-September 2014 Analysis
Rate is holding relatively steady during this quarter. Continued emphasis on the IRP process has kept this rate above the threshold.
April-June 2014 Analysis
The emphasis on IRP Quality continues to drive steady improvements on results. Continuing statistically significant upward trend in rate shows improving quality of the IRP is evidence of systematic processes.
January-March 2014 Analysis
The emphasis on IRP Quality continues to drive steady improvements on audit results. Continuing statistically significant upward trend in rate shows improving quality of the IRP is evidence of systematic processes.
74

Appendix F CBH System KPI Dashboards
Percent of Georgia Housing Voucher Program adult MH individuals in stable housing (greater than 6 months) Target 77%
100%

95%

90%

85%

80%

75%

70%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14

Sep-14

Oct-14

Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

890

905

929

967 1026 1044 1089 1134 1159 1251

0

0

972

989 1014 1058 1117 1141 1194 1247 1274 1369

0

0

91.6% 91.5% 91.6% 91.4% 91.9% 91.5% 91.2% 90.9% 91.0% 91.4% #N/A #N/A

91.6%

91.6%

91.0%

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: A measure of stable housing based on nationally accepted HUD standard.
Measure explanation: An initial indication of the program's ability to prevent homelessness and re-institutionalization.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of individuals leaving the program less than 6 Denominator: Number of individuals in the program greater

months.

than 6 months.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
As a rolling average, this measure continues to remain stable as new individuals are added on to the program.

April-June 2014 Analysis
As a rolling average, this measure continues to remain stable as new individuals are added on to the program.

January-March 2014 Analysis
As a rolling average, this measure continues to remain stable as new individuals are added on to the program.

75

Percent of Georgia Housing Voucher Program adult MH individuals who left stable housing under unfavorable circumstances and have been reengaged and reassigned vouchers Target 10%

50% 45% 40% 35% 30% 25% 20% 15% 10%
5% 0%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

40

44

47

47

47

45

220

225

237

247

259

273

18.2% 19.6% 19.8% 19.0% 18.1% 16.5%

19.2%

17.8%

Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

49

54

53

63

0

0

290

307

314

333

0

0

16.9% 17.6% 16.9% 18.9% #N/A #N/A

17.1%

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: A measure to determine negative program leavers in order to divert them from homelessness or other more expensive systems of care.

Measure explanation: Reinforces the notion that recovery is not a straight line and that reengagement after initial failure is an important program component.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of individuals that left the program under Denominator: Number of individuals that left the program

negative circumstances that reentered the program.

under negative circumstances.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Over the course of the program this measure appears to be stable hovering between highs of 19% and lows at 16%.

April-June 2014 Analysis
Although lower than the historic rate of close to 20%, the target threshold of 10% has been exceeded by over 60%.

January-March 2014 Analysis
This measure continues to remain stable as 1 in 5 negative discharges are reengaged and reenter stable housing.

76

Percent of adult MH supported employment providers that meet a caseload on the last day of the month of employment specialist staff to consumer ratio
(between 1:15 to 1:20) Target 85% or more
*Key Performance Indicator Deactivated July 2014

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40%

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Numerator Denominator Rate
Quarterly Rate

Jan-14 17 27
63.0%

Feb-14 16 27
59.3% 59.3%

Mar-14 15 27
55.6%

Apr-14 14 27
51.9%

May-14 18 27
66.7% 60.5%

Jun-14 17 27
63.0%

Jul-14 Aug-14 Sep-14

0

0

0

0

0

0

#N/A

#N/A

#N/A

KPI Inactive

Oct-14 Nov-14 Dec-14

0

0

0

0

0

0

#N/A

#N/A

#N/A

KPI Inactive

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of adult MH contracted supported employment providers that met a mental health caseload on the last day of the month average between 1 to 15 and 1 to 20 on the last day of the calendar month.
Measure explanation: To examine the proportion of mental health contracted Supported Employment agencies, that devote the appropriate staffing the Dartmouth model indicates is necessary for obtaining and maintaining
COMPONENTS OF NUMERATOR AND DENOMINATOR Numerator: Number of contracted providers with a Denominator: Number of contracts DBHDD Community consumer to staff ratio between 1:15 and 1:20 on the Mental Health holds for Supported Employment. last day of the month.
COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
This measure is no longer active.

July-September 2014 Analysis
This measure is no longer active.

April-June 2014 Analysis
Percentages appear to be similar to previous months.

January-March 2014 Analysis
There were a few providers who were over ratio during the quarter, however, there were more that were under ratio. This means that many providers had smaller caseloads per staff member. Program Quality Council discussed this indicator and determined that if providers serve a smaller ratio, that it is not detrimental to the consumer, therefore this measure will end on 6/30/14 and be replaced with a measure that examines 20:1 and under starting on 7/1/14.

77

Percent of providers that meet a caseload average consumer to staff ratio 20:1 and under (Target 85% or more)
*KPI activated July 2014*

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

0

0

0

0

0

0

23

22

21

20

22

0

0

0

0

0

0

0

27

27

27

27

27

0

#N/A #N/A #N/A #N/A #N/A #N/A 85.2% 81.5% 77.8% 74.1% 81.5% #N/A

KPI Inactive

KPI Inactive

81.5%

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of SE consumers who were employed on the last day of the calendar month or who were discharged during the month while employed.
Measure explanation: To examine the percentage of consumers were are able to obtain employment while utilizing Supported Employment services.
COMPONENTS OF NUMERATOR AND DENOMINATOR Numerator: Number of consumers competitively employed at Denominator: Number of consumers served that month. end of month plus the number of consumers competitively employed at discharge that month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Target met this quarter. There appears to be a slight upward trend in 2014.

April-June 2014 Analysis
KPI inactive this quarter.

January-March 2014 Analysis
KPI inactive this quarter.

78

Percent of Supported Employment consumers who were employed on the last day of the calendar month or who were discharged during the month while employed Target (43%) or more *KPI activated July 2014*

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30% 25% 20%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14

Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

0

0

0

0

0

0

581

609

604

603

604

0

0

0

0

0

0

0 1228 1232 1207 1220 1194

0

#N/A #N/A #N/A #N/A #N/A #N/A 47.3% 49.4% 50.0% 49.4% 50.6% #N/A

KPI Inactive

KPI Inactive

48.9%

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of SE consumers who were employed on the last day of the calendar month or who were discharged during the month while employed.
Measure explanation: To examine the percentage of consumers were are able to obtain employment while utilizing Supported Employment services.
COMPONENTS OF NUMERATOR AND DENOMINATOR Numerator: Number of consumers competitively employed at Denominator: Number of consumers served that month. end of month plus the number of consumers competitively employed at discharge that month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
There appeared to be a slight upward trend this quarter. At the end of the quarter there was a focus on discharging consumers who have been steady in employment, need minimal supports, and could maintain their employment with a step-down service.
April-June 2014 Analysis
Data collection not initiated during this quarter.

January-March 2014 Analysis
Data collection not initiated during this quarter.

79

Percent of unduplicated individuals who had 1st contact with a competitive employer within 30 days of enrollment Target (75%) or more

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40%

Oct-Dec 2013

Jan-Mar 2014

Apr-Jun 2014

Jul-Sep 2014

Oct-Dec 2013 Jan-Mar 2014 Apr-Jun 2014 Jul-Sep 2014

Numerator

70

100

101

109

Denominator

108

132

141

137

Rate

65%

76%

72%

80%

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of individuals meeting settlement criteria that were enrolled during the quarter that had contact with a potential employer in the open job market within 30 days of enrolling in supported employment services.

Measure explanation: To examine the percentage of settlement criteria consumers who are able to have rapid job placement opportunities. Note: Measure is taken on a 30-day lag.
COMPONENTS OF NUMERATOR AND DENOMINATOR Numerator: Number of settlement criteria consumers who Denominator: Number of settlement criteria consumers who started Supported Employment services during the quarter started Supported Employment services during the quarter. and who had first contact with a competitive employer within 30 days.

COMMENTS AND/OR ANALYSIS PER QUARTER
July-September 2014 Analysis
Target met this quarter. There appears to be a slight upward trend in 2014.

April-June 2014 Analysis
A similar number of consumers had first contact within thirty days as compared to Quarter 3. However, there were nine more consumers who began services in Quarter 4 in comparson to Quarter 3. Therefore, the overall percentage of consumers went down in comparison to Quarter 3.
January-March 2014 Analysis
Threshold was met this quarter.
October-December 2013 Analysis
The target was not met this quarter. The percentage appears to be similar to last quarter. Two providers indicated that the Fidelity Reviews related to Supported Employment completed by DBHDD State Office has enhanced their understanding of this key performance indicator. Both indicated that they believe they will have better percentages moving forward.

80

Percent of Assertive Community Treatment consumers who are received into services within 3 days of referral Target (70%) or more
*Key Performance Indicator activated July 2013*

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50%

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Numerator Denominator Rate
Quarterly Rate

Jan-14 92
138 66.7%

Feb-14 82
140 58.6% 63.7%

Mar-14 80
121 66.1%

Apr-14 76
112 67.9%

May-14 75
104 72.1% 69.4%

Jun-14 58 85
68.2%

Jul-14 Aug-14 Sep-14

72

78

78

86

116

112

83.7% 67.2% 69.6%

72.6%

Oct-14 Nov-14 Dec-14

94

102

0

119

129

0

79.0% 79.1% #N/A

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of ACT consumers who began services during the month that waited three days or less since their date of referral to ACT services.

Measure explanation: To examine the percentage of consumers who are able to access ACT services in a rapid

manner.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers received into

Denominator: Total number of consumers received into

services within 24 hours of referral date plus number services.

of consumers received into services within 3 days of
f ld COMMENTS AND/OR ANALYSIS PER QUARTER

October-December 2014 Analysis
Data for this quarter is incomplete.

July-September 2014 Analysis
The target was met one month during the quarter. Some providers indentified that some referral sources do not include all the referral information. It requires additional time to follow up with the referral sources before the individual can be received into services. Providers identified the need to continue to educate referral sources on all the inforamtion that is needed.
April-June 2014 Analysis
The target was met one month during the quarter.

January-March 2014 Analysis
Some barriers that the ACT Teams identified included: receiving incorrect contact information for the referred consumers which increases the amount of time it takes the team to locate and make contact with the consumer, receiving an increase of referrals for homeless consumers which increases the amount of time to locate the consumer, and consumers that move directly after the referral is made which increases the time it takes the team to identify new contact information.
81

Percent of Assertive Community Treatment consumers admitted to a Psychiatric Hospital within the past month Target (7%) or less
15%

10%

5%

0% Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator

152 142 169 167 181 178 168 168 168 147 152

0

Denom inator

1654 1568 1581 1621 1563 1696 1664 1642 1656 1650 1675

0

Percent

9.2% 9.1% 10.7% 10.3% 11.6% 10.5% 10.1% 10.2% 10.1% 8.9% 9.1% #N/A

Quarterly Average

N/A due to monthly unduplicated counts

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in ACT services for over thirty days that were admitted to a psychiatric hospital during the month.

Measure explanation: To examine the percentage of consumers who are utilizing psychiatric hospitals for stabilization.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers admitted to Psychiatric Denominator: Census on the last day of the month minus

Inpatient.

number of enrollments during the month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
The target was not met during any month this quarter. Some teams report that unstable housing has been contributing to the psychiatric admissions.
April-June 2014 Analysis
Target was not met during any month this quarter. Some Teams cited that when their consumers go into a hospital for a medical issue, the hospital automatically transfers them over to the psychiatric unit simply due to the mental health diagnosis listed in the record. These Teams are working on outreach and education towards these hospitals. One provider indicated their percentages were higher the last two months this quarter due to a few number of consumers who repeatedly return to the hospital. Another team indicated they are looking for different ITR homes to better support their consumers.
January-March 2014 Analysis
Some providers indicate that consumers are sometimes discharged from hospitals prior to achieving stability, which may lead to decompensation in the community and rehospitalization. Some ACT teams reported that sometimes consumers are discharged from the hospital withut their knowledge, preventing them from assisting with supportive discharge planning. Many teams indicated that their hospital days are typically from a small number of consumers with long length of stays in hospitals.
82

Average # of jail/prison days utilized (per enrolled Assertive Community Treatment consumer)
Target (1.0 day) or less
1.50

1.25

1.00

0.75

0.50

0.25

0.00

Jan-14

Feb-14

Mar-14

Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Rate Quarterly Rate

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

1525 1534 1873 1614 1574 1525

1826 1833 1852 1875 1826 1841

0.835 0.837 1.011 0.861 0.862 0.828

0.633

0.566

Jul-14 Aug-14 Sep-14 1418 1708 1731 1807 1840 1812 0.785 0.928 0.955 0.808

Oct-14 Nov-14 Dec-14

1482 1820

0

1811 1809

0

0.818 1.006 #N/A

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The average number of days consumers in ACT services for over thirty days spent in jail/prison during the month.

Measure explanation: To examine the amount of time consumers spend in jail.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of jail days utilized for consumers in Denominator: Number of discharges plus census on the

services 30 plus days.

last day of month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Target was met each month this quarter. One provider cited strong relationship with jails and the ability to advocate for consumers.

April-June 2014 Analysis
Target was met each month during the quarter.

January-March 2014 Analysis
Slight upward trend during the quarter. Some teams reported that some consumers where choosing to not take their prescribed medications which may have impacted this measure this quarter.

83

Percent of Intensive Case Management consumers with a Psychiatric Inpatient Admission within the past month Target (5%) or less
15%

10%

5%

0% Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

34

28

52

40

37

44

63

63

59

70

0

0

627 641 663 729 776 802 903 1053 1140 1159

0

0

5.4% 4.4% 7.8% 5.5% 4.8% 5.5% 7.0% 6.0% 5.2% 6.0% #N/A #N/A

N/A due to monthly unduplicated counts

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in ICM services for over thirty days that were admitted to a psychiatric hospital during the month.

Measure explanation: To examine the percentage of consumers who are utilizing psychiatric hospitals for stabilization.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers admitted to Psychiatric

Denominator: The census on the last day of the month

Inpatient.

minus number of enrollments during the month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Performance measure not met during any month this quarter.

April-June 2014 Analysis
Percentages appeared to be consistent with previous quarters. One provider indicated that they are working to improve their psychiatric inpatient admissions by working with consumers who specifically utilize psychiatric inpatient facilities as a coping mechanism.
January-March 2014 Analysis
Percentage appeared to be consistent with previous quarters.

84

Percent of Intensive Case Management consumers housed (non homeless) within the past month Target (90%) or more
100%

95%

90%

85% Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator

635

655

643

711

743

776

997 1033 1138 1171

0

0

Denom inator

657

682

667

741

776

806 1023 1067 1175 1210

0

0

Percent

96.7% 96.0% 96.4% 96.0% 95.7% 96.3% 97.5% 96.8% 96.9% 96.8% #N/A #N/A

Quarterly Average

N/A due to monthly unduplicated counts

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in ICM services on the last day of the month that were not homeless.

Measure explanation: To examine the percentage of consumers who are not living in homeless shelters or on streets at a single point in time.
COMPONENTS OF NUMERATOR AND DENOMINATOR Numerator: Number of consumers by living arrangement on Denominator: Number of consumers by living the last day of the month minus number of homeless: street, arrangement on the last day of the month. homeless shelter.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Measure met every month this quarter.

April-June 2014 Analysis
Percentage remained above the target through the quarter. One provider noted success with working with local shelters for temporary housing supports while seeking housing vouchers.
January-March 2014 Analysis
Percentage remained above the target through the quarter. Providers did not report any barriers.

85

Average # of jail/prison days utilized (per enrolled Intensive Case Management consumer)
Target (0.25 days) or less
0.75

0.50

0.25

0.00 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denominator Rate Quarterly Rate

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

217 344 315 202 259 316

705 756 816 872 911 930

0.308 0.455 0.386 0.232 0.284 0.340

0.385

0.286

Jul-14 Aug-14 Sep-14 458 592 590
1186 1287 1346 0.386 0.460 0.438
0.370

Oct-14 Nov-14 Dec-14

627

0

0

1460

0

0

0.429 #N/A #N/A

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The average number of days consumers (who have been in ICM services for over thirty days) spent in jail/prison during the month.

Measure explanation: To examine the amount of time consumers spend in jail.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of jail days utilized for consumers Denominator: Number of discharges plus census on last

in ICM services 30 plus days.

day of month.

COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
Data for this quarter is incomplete.

July-September 2014 Analysis
Performance measure not met during any month this quarter.

April-June 2014 Analysis
Average was below target in April but above target during May and June. One provider who had high utilization indicated that their numbers were due to a small number of consumers who were staying in jail a long time. Another provider indicated that the Mental Health Court in their area has helped to keep several of their consumers from going to jail which had aided in their ability to keep consumers in the community.
January-March 2014 Analysis
Percentage appeared to be consistent with previous quarters.

86

Percent of Community Support Team consumers with a Psychiatric Inpatient Admission within the past month Target (10%) or less
30%

25%

20%

15%

10%

5%

0% Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

19

14

20

21

9

15

20

25

13

23

15

0

220 231 239 245 245 242 247 258 256 265 266

0

8.6% 6.1% 8.4% 8.6% 3.7% 6.2% 8.1% 9.7% 5.1% 8.7% 5.6% #N/A

N/A due to monthly unduplicated counts

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in CST services for over thirty days that were admitted to a psychiatric hospital during the month.

Measure explanation: To examine the percentage of consumers who are utilizing psychiatric hospitals for stabilization.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers admitted to Psychiatric

Denominator: Census on last day of month minus the

Inpatient.

number of enrollments during month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
There appears to be a decrease in the month of September. One provider indicated that they are using more proactive coping mechanisms with consumers to reduce admissions. Another provider indicated that the DBHDD funded Beck Initiative training has been very helpful to assist staff in engaging with clients early in treatment.
April-June 2014 Analysis
The percentages were below target through the quarter.

January-March 2014 Analysis
The percentages appeared higher overall this quarter in comparison to previous quarters. Some teams indicated that the holiday season may have contributed to the increase in hospitalizations. One team reported that they were down a staff member, which may have not made their team as effective at preventing hospitalizations. Another team reported that they were able to proactively engage families during the second quarter but that family involvement waned in the third quarter, which may have impacted the increase in hospitalizations.
87

Percent of Community Support Team consumers housed (non homeless) within the past month Target (90%) or more
100%

95%

90%

85%

80% Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

235 243 236 239 242 236 244 257 262 269 278

0

238 248 239 246 245 240 247 259 267 272 279

0

98.7% 98.0% 98.7% 97.2% 98.8% 98.3% 98.8% 99.2% 98.1% 98.9% 99.6% #N/A

N/A due to monthly unduplicated counts

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in CST services on the last day of the month that were not homeless.

Measure explanation: To examine the percentage of consumers who are not living in homeless shelters or on streets at a single point in time.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers by living arrangement on Denominator: Number of consumers by living

last day of month minus number of homeless: street,

arrangement on last day of month.

homeless shelter.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Consistent with previous quarters, the percentage appeared to remain consistent during the quarter. Teams did not report any barriers. Several teams reported that they find family members are willing to take in consumers.
April-June 2014 Analysis
Percentages remained above target through the quarter.

January-March 2014 Analysis
Consistent with previous quarters, the percentage appeared to remain consistent during the quarter. Teams did not report any barriers.

88

Average # of jail/prison days utilized (per enrolled Community Support Team consumer)
Target (0.75 days) or less

2.00

1.75

1.50

1.25

1.00

0.75

0.50

0.25

0.00 Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Rate Quarterly Rate

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14

47 104 121 180 248 100 154 135 166

262 265 285 295 288 287 294 300 298

0.179 0.392 0.425 0.610 0.861 0.348 0.524 0.450 0.557

0.335

0.607

0.510

Oct-14 Nov-14 Dec-14

154 158

0

301 307

0

0.512 0.515 #N/A

Quarterly data not complete

BOTTOM

0.600

0.600

0.600

0.600

0.600

0.600

0.600

0.600

0.600

0.600

0.600

0.600

MEASURE DEFINITION AND EXPLANATION Measure definition: The average number of days consumers (who have been in CST services for over thirty days) spent in jail/prison during the month.

Measure explanation: To examine the amount of time consumers spend in jail.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of jail days utilized for consumers in Denominator: Number of discharges plus census on the

CST services 30 plus days.

last day of the month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
The percentages were below target through the quarter.

April-June 2014 Analysis
Target was met on two of the three months this quarter.

January-March 2014 Analysis
Slight upward trend through the quarter. One team reported that they were down a staff member, which may have not made their team as effective at preventing jail days or reducing length of stay in jail. Another team reported that they were able to proactively engage families during the second quarter but that family involvement waned in the third quarter, which may have impacted the jail utilization.

89

Percent of Case Management consumers with a Psychiatric Inpatient Admission within the past month
Target (5%) or less
10%

5%

0% Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

21

17

32

26

19

24

673

703

727

695

688

665

3.1% 2.4% 4.4% 3.7% 2.8% 3.6%

3.3%

3.4%

Jul-14 Aug-14 Sep-14

21

30

20

689

753

833

3.0% 4.0% 2.4%

3.1%

Oct-14 Nov-14 Dec-14

23

26

0

914

928

0

2.5% 2.8% #N/A

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in CM services for over thirty days that were admitted to a psychiatric hospital during the month.

Measure explanation: To examine the percentage of consumers who are utilizing psychiatric hospitals for stabilization.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers admitted to Psychiatric Denominator: Census on last day of month minus the

Inpatient.

number of enrollments during month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Measure met each month this quarter.

April-June 2014 Analysis
Percentage remained below the target through the quarter. Providers did not report any barriers.

January-March 2014 Analysis
Percentage remained below the target through the quarter. Providers did not report any barriers.

90

Percent of Case Management consumers housed (non homeless) within the past month Target (90%) or more
100%

95%

90%

85%

Jan-14

Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Percent Quarterly Average

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14

680

730

724

712

681

689

718

800

875

724

757

745

726

696

708

733

814

897

93.9% 96.4% 97.2% 98.1% 97.8% 97.3% 98.0% 98.3% 97.5%

N/A due to monthly unduplicated counts

Oct-14 Nov-14 Dec-14

913

929

0

933

948

0

97.9% 98.0% #N/A

MEASURE DEFINITION AND EXPLANATION Measure definition: The percent of consumers in CM services on the last day of the month that were not homeless.

Measure explanation: To examine the percentage of consumers who are not living in homeless shelters or on streets at a single point in time.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of consumers by living arrangement on Denominator: Number of consumers by living arrangement

last day of month minus the number of homeless: street,

on last day of month.

homeless shelter.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Measure met each month this quarter.

April-June 2014 Analysis
Percentage remained above the target through the quarter. Providers indicated that they have been able to help monitor and assist with consumer medication adherence. Providers indicated that they believed this impacted the consumers' ability to remain stable which enabled them to maintain their housing. Providers also indicated that seeking out natural supports have given many consumers safe temporary housing while seeking more permanent housing arrangements.
January-March 2014 Analysis
Percentage remained above the target through the quarter. Providers did not report any barriers.

91

Average # of jail/prison days utilized (per enrolled Case Management consumer)
Target (0.25 days) or less

0.50

0.45

0.40

0.35

0.30

0.25

0.20

0.15

0.10

0.05

0.00 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

Jul-14

Aug-14 Sep-14 Oct-14 Nov-14 Dec-14

Num erator Denom inator Rate Quarterly Rate

Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14

246 336 253 204 234 248

805 825 838 842 846 831

0.306 0.407 0.302 0.242 0.277 0.298

0.338

0.272

Jul-14 Aug-14 Sep-14 193 145 173 876 982 1068
0.220 0.148 0.162 0.175

Oct-14 Nov-14 Dec-14

273 319

0

1120 1100

0

0.244 0.290 #N/A

Quarterly data not complete

MEASURE DEFINITION AND EXPLANATION Measure definition: The average number of days consumers (who have been in CM services for over thirty days) spent in jail/prison during the month.

Measure explanation: To examine the amount of time consumers spend in jail.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of jail days utilized for consumers in Denominator: Number of discharges plus census on the

CM services 30 plus days.

last day of month.

October-December 2014 Analysis
Data for this quarter is incomplete.

COMMENTS AND/OR ANALYSIS PER QUARTER

July-September 2014 Analysis
Measure met each month this quarter.

April-June 2014 Analysis
Average was below target in April but above target during May and June. One provider indicated that violations of probation have impacted their jail utilization. Two providers indicated that they have been able to contact and communicate with local District Attorneys regarding some of their consumers in jail. This has yielded a few conditional releases.
January-March 2014 Analysis
Percentage appeared to be consistent with previous quarters.

92

Percent of adult clients active in AD treatment 90 days after beginning non-crisis stabilization services. Target 25%
50%

45%

40%

35%

30%

25%

20%

15%

2011

2012

2013

2014

Num erator Denom inator Percent

2011 0 0
#N/A

2012 0 0
#N/A

2013 22
100 22.0%

2014 3714
11784 31.5%

MEASURE DEFINITION AND EXPLANATION Measure definition: This measure captures how many individuals in AD services remained engaged in treatment 90 days after beginning community based treatment services.

Measure explanation: The purpose of this measure is to determine level of engagement and retention of individuals involved in AD community based treatment.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The unduplicated count of individuals Denominator: The unduplicated count of individuals

entering non-crisis stabilization services identified by who received Community Based Treatment services

having a Registration or New Episode MICP who

where the authorization (MICP) for service had Adult

had Medicaid claims or State Encounters for

Addictive Diseases selected as the Primary Diagnostic

community Based Treatment services, excluding

Category.

Crisis Stabilization and Detoxification (Residential

and Ambulatory) between 90 - 120 days after entry

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
This KPI became effective in July 2013, is collected on an annual basis and is considered a more critical quality issue. The target threshold of 25% was not met in 2013. It is anticipated that updated data will become available in October 2014.

Annually 2013
The previous KPI, Percent of adult AD consumers who abstain from use or experience reduction in use (while in treatment) Target (40%), was replaced with this KPI.

93

Percent of clients discharged from crisis or detoxification programs who receive follow-up behavioral health services within 14 days. Target 35%

65% 60% 55% 50% 45% 40% 35% 30% 25%

2011

2012

2013

2014

Num erator Denom inator Percent

2011 2012 2013 2014

0

0 34 2427

0

0 100 7014

#N/A #N/A 34.0% 34.6%

MEASURE DEFINITION AND EXPLANATION
Measure definition: This measure captures how many individuals who were discharged from detox and/or crisis received follow-up services in the community within 14 days.

Measure explanation: The purpose of this measure is to determine if those served in these higher levels of care were provided follow-up services in community based treatment.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The unduplicated count of

Denominator: The unduplicated count of individuals

individuals who had Medicaid Claims or State who received Crisis Stabilization services where the

Encounters for any Community Based

authorization (MICP) for service had Adult Addictive

Treatment service excluding Crisis

Diseases selected as the Primary Diagnostic Category.

Stabilization and Detoxification (Residential

and Ambulatory) within 14 days of the last

Crisis encounter.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
This KPI became effective in July 2013, is collected on an annual basis and is considered a more critical quality issue. It is anticipated that 2014 data will become available in October 2014.

Annually 2013
The previous KPI was inactivated after FY2012 and replaced with the current KPI. The threshold of 35% was not met in 2013.

94

Percent of individuals meeting community settlement agreement criteria who are enrolled in settlement funded services who state they are satisfied with the services they are receiving Target 90% or more
100%

90%

80%

70% October 2012 - March 2013

April 2013 - September 2013

October 2013 - March 2014

April 2014 - September 2014

October April 2013 - October April 2014 -

2012 - September 2013 - September

March 2013 2013 March 2014 2014

Numerator

32

40

7

9

Denominator
Rate

34

52

8

9

94%

77%

88%

100%

MEASURE DEFINITION AND EXPLANATION Measure definition: Those individuals who meet Settlement Agreement Criteria, were chosen by the QM Audit Team to receive an audit, and who agreed to be interviewed who stated they are satisfied with the ADA service they are receiving.

Measure explanation: The purpose of this measure is to provide the Department with a snapshot of the level of satisfaction of individuals involved in settlement agreement services.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The number of individuals who answered yes.

Denominator: The total number of individuals responding to the question.

COMMENTS AND/OR ANALYSIS PER PERIOD
April 2014 - September 2014
Conclusions could not be drawn from the few surveys that were completed during this reporting period.
October 2013 - March 2014
Data collection was put on hold between April 2013 June 2014 secondary to the QM audit team performing a follow-up audit of inpatient hospital re-admissions.
April 2013 - September 2013
Data collection was put on hold between April 2013 June 2014 secondary to the QM audit team performing a follow-up audit of inpatient hospital re-admissions.
October 2012 - March 2013
ADA services have been in place for a longer period of time and providers have been improving their quality of service via agency specific PI indicators. It is hypothesized that these quality improvement processes may have impacted individuals' satisfaction with services.

95

Percent of individuals meeting community settlement agreement criteria who are enrolled in settlement funded services who feel their quality of life
has improved as a result of receiving services Target 90% or more
100%

90%

80%

70% October 2012 - March 2013

April 2013 - September 2013

October 2013 - March 2014

April 2014 - September 2014

October April 2013 - October April 2014 -

2012 - September 2013 - September

March 2013 2013 March 2014 2014

Numerator

30

37

8

7

Denominator

30

45

8

8

Rate

100%

82%

100%

88%

MEASURE DEFINITION AND EXPLANATION Measure definition: Those individuals who meet Settlement Agreement Criteria, were chosen by the QM Audit Team to receive an audit, and who agreed to be interviewed who stated their quality of life has improved since receiving ADA services.

Measure explanation: The purpose of this measure is to determine one of the impacts settlement services may have on the target population.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The number of individuals who answered yes.

Denominator: The total number of individuals responding to the question.

COMMENTS AND/OR ANALYSIS PER PERIOD
April 2014 - September 2014
Conclusions could not be drawn from the few surveys that were completed during this reporting period.
October 2013 - March 2014
Data collection was put on hold between April 2013 June 2014 secondary to the QM audit team performing a follow-up audit of inpatient hospital re-admissions.
April 2013 - September 2013
Data collection was put on hold between April 2013 June 2014 secondary to the QM audit team performing a follow-up audit of inpatient hospital re-admissions.
October 2012 - March 2013
While there is an upward trend towards overall improvement in quality of life, the benchmark may be difficult to reach due to the nature of SPMI and its impact on the individual. Because individuals are continuously enrolled in services, there is a subset of individuals interviewed who may not have been enrolled in services for a sufficient amount of time to realize the impact on their quality of life. The trend should continue to improve as providers continue to improve their quality of service.
96

97

Percent of families of youth served by CMEs who are satisfied with services as determined by their parent or legal guardian using a standardized survey tool (YSS-F)
Target 80%
100%

90%

80%

70%

60%

50%

40% 2011

2012

2013

2014

Num erator Denom inator Rate

2011 0 0
#N/A

2012 0 0
#N/A

2013 491 585 84%

2014 468 557 84%

MEASURE DEFINITION AND EXPLANATION Measure definition: This measure identifies families of youth who are being served by the Care Management Entities who respond to satisfaction questions on the YSS-F standardized survey instrument.

Measure explanation: To examine the general satisfaction with services received while being served by a CME.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Percentage of respondents with an agverage score

Denominator: Number of respondents to YSS-F questions

>3.5 from the statisfaction questions.

related to general satisfaction.

COMMENTS AND/OR ANALYSIS PER QUARTER
Annually 2014
Responses to the satisfaction survey surpassed the target rate.
Annually 2013
Responses to the satisfaction survey surpassed the target rate.

98

Appendix G Developmental Disabilities System KPI Dashboards

Percentage of Individuals Who Have Had a Flu Vaccine in Past Year Target 75%

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30%

2010

2011

2012

2013

BOTTOM

2010 2011 2012 2013

Num erator

261 270 260 358

Denominator 414 416 412 497

0.750

Rate
0.750 0.750

63.0% 64.9% 63.1% 72.0%
0.750

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage on individuals surveyed through the National Core Indicator Survey who report having a flu shot in past year.

Measure explanation: Allows for additional monitoring of the health of individuals.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The numerator is the number of

Denominator: The Denominator is the number of

individuals who reported that they have had a flu shot individuals who were able to answer this question.

in the last year. NCI data management and analysis Not all individuals were capable or we aware is they

is coordinated by Human Services Research Institute had a flu shot or not. NCI data management and

(HSRI). Georgia enters data in the ODESA database analysis is coordinated by Human Services Research

which HSRI uses for analysis.

Institute (HSRI). Georgia enters data in the ODESA

database which HSRI uses for analysis.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
2014 NCI data will not be available until spring of 2015. This data will be reported in the 2015 Interim Report.
Annually 2013
72% of respondents from Georgia and 76% of respondents across NCI States were reported to have had a flu vaccine in the past year. This is up significantly from 63% last year; however Georgia remains within the average range of NCI States.
Annually 2012
63% of respondents from Georgia were reported to have had a flu vaccine in the past year. This is slightly down from 65% for the previous year. 63% is significantly below the national average (77%) of all NCI States.

99

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50%

Percentage of Individuals Who Have Had a Dental Examine in Past Year Target 80%

2010

2011

2012

2013

2010 2011 2012 2013

Num erator

306 326 312 391

Denominator 431 418 445 514

Rate

71.0% 78.0% 70.1% 76.1%

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage on individuals surveyed through the National Core Indicator Survey who report having a dental exam.

Measure explanation: Allows for additional monitoring of the health of individuals.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The numerator is the number of individuals Denominator: The Denominator is the number of

who reported that they have had a dental examination in individuals who were able to answer this question.

the last year. NCI data management and analysis is

Not all individuals were capable or were aware if they

coordinated by Human Services Research Institute

had a dental exam or not. NCI data management

(HSRI). Georgia enters data in the ODESA database

and analysis is coordinated by Human Services

which HSRI uses for analysis.

Research Institute (HSRI). Georgia enters data in

the ODESA database which HSRI uses for analysis.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
2014 NCI data will not be available until spring of 2015. This data will be reported in the 2015 Interim Report.

Annually 2013
76% of respondents from Georgia and 80% of respondents across NCI States were reported to have had a dental exam in the past year. This is up significantly from 70% last year; however Georgia still remains within the average range of NCI States

Annually 2012
70% of respondents reported having a dental exam in the past year. This is down significant from 78% the previous year. 70% is also significantly lower that the national average (80%) for all other NCI States. This KPI has been given Departmental priority and solutions to improve this KPI are being reviewed.

100

Percentage of Individuals Who Have Had an Annual Physical in Past Year Target 92%
100%

95%

90%

85%

80%

75%

70% 2010

2011

2012

2013

Num erator Denom inator Rate

2010 414 465 89%

2011 373 451 83%

2012 466 518 90%

2013 448 520 86%

2014

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage on individuals surveyed through the National Core Indicator Survey who report having a physical exam.

Measure explanation: Allows for additional monitoring of the health of individuals.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The numerator is the number of individuals who Denominator: The Denominator is the number of

reported that they have had an annual physical examination individuals who were able to answer this question. Not all

in the last year. NCI data management and analysis is

individuals were capable or we aware is they had a

coordinated by Human Services Research Institute (HSRI). physical exam or not. NCI data management and analysis

Georgia enters data in the ODESA database which HSRI is coordinated by Human Services Research Institute

uses for analysis.

(HSRI). Georgia enters data in the ODESA database

which HSRI uses for analysis.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
2014 NCI data will not be available until spring of 2015. This data will be reported in the 2015 Interim Report.
Annually 2013
86% of respondents from Georgia and 89% of respondents across NCI States were reported to have had a physical exam in the past year. This is down slightly from 90% last year; however Georgia still remains within the average range of NCI States
Annually 2012
90% of respondents reported having had a physical exam in this past year. This is slightly down from the previous year which as reported at 91%. 90% is in line with the national average (90%) for all other NCI States.

101

Percentage of Individuals Who Feel Safe in Their Home Target 90%
100%

95%

90%

85%

80%

75%

70% 2010

2011

2012

2013

Num erator Denom inator Rate

2010 297 326 91%

2011 291 338 86%

2012 342 384 89%

2013 336 386 87%

2014

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage on individuals surveyed through the National Core Indicator Survey who report feeling safe in their residential environment.

Measure explanation: Allows for additional monitoring of the safety of individuals

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The numerator is the number of individuals Denominator: The Denominator is the number of

who reported that they either feel safe in their home or never individuals who were able to answer this question. Not all

feel afraid in their home. NCI data management and

individuals were capable or were willing to answer this

analysis is coordinated by Human Services Research

question. NCI data management and analysis is

Institute (HSRI). Georgia enters data in the ODESA

coordinated by Human Services Research Institute

database which HSRI uses for analysis.

(HSRI). Georgia enters data in the ODESA database

which HSRI uses for analysis.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
2014 NCI data will not be available until spring of 2015. This data will be reported in the 2015 Interim Report.

Annually 2013
87% of respondents from Georgia and 81% of respondents across NCI States reported they never feel scared at home. This is down slightly from 89% last year; however Georgia's average is significantly about the average range of NCI States.

Annually 2012
89% of respondents reported they never feel scared at home. This is an improvement from the previous year which was reported at 86%. 89% is in line with the national average (82%) for all other NCI States.

102

Percentage of Individuals Who Report They are Treated with Dignity and Respect Target 90%
100%

95%

90%

85%

80%

75%

70% 2010

2011

2012

2013

Num erator Denom inator Rate

2010 157 164 96%

2011 194 200 97%

2012 170 177 96%

2013 302 311 97%

2014

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage on individuals surveyed through the National Core Indicator Survey who report staff and family treat them with respect.

Measure explanation: Allows for additional monitoring of the safety of individuals.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The numerator is the number of individuals who Denominator: The Denominator is the number of

reported that their staff treat them with dignity and respect. individuals who were able to answer this question. Not all

NCI data management and analysis is coordinated by

individuals were capable or were willing to answer this

Human Services Research Institute (HSRI). Georgia enters question. NCI data management and analysis is

data in the ODESA database which HSRI uses for analysis. coordinated by Human Services Research Institute

(HSRI). Georgia enters data in the ODESA database

which HSRI uses for analysis.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
2014 NCI data will not be available until spring of 2015. This data will be reported in the 2015 Interim Report.
Annually 2013
97% of respondents from Georgia and 93% of respondents across NCI States reported they are treated with dignity and respect. This is up slightly from 96% last year, and Georgia ranks top among the NCI States.
Annually 2012
96% of respondents reported that they are treated with dignity and respect. This is slightly down from the previous year when 97% reported they felt that they were treated with dignity and respect. 96% is in line with the national average (94%) of all other NCI States.

103

Percentage of Individuals Who Report They have a Choice of Supports and Services Target 95%

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50%

2010

2011

2012

2013

Num erator Denom inator Rate

2010 265 441 60%

2011 297 457 65%

2012 349 521 67%

2013 432 600 72%

2014

MEASURE DEFINITION AND EXPLANATION Measure definition: Individuals report that they have choice in the supports they receive.

Measure explanation: Division of DD strives to support individuals to move choice in all supports and services.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: The numerator is the number of individuals who

Denominator: The Denominator is the number of individuals who

reported that they had a choice in the supports and services they were able to answer this question. Not all individuals were

receive.NCI data management and analysis is coordinated by

capable or were willing to answer this question. NCI data

Human Services Research Institute (HSRI). Georgia enters data in management and analysis is coordinated by Human Services

the ODESA database which HSRI uses for analysis.

Research Institute (HSRI). Georgia enters data in the ODESA

database which HSRI uses for analysis.

COMMENTS AND/OR ANALYSIS PER YEAR
Annually 2014
2014 NCI data will not be available until spring of 2015. This data will be reported in the 2015 Interim Report.
Annually 2013
72% of respondents from Georgia and 52% of respondents across NIC States reported that they have a choice of support and services. This is up significantly from 67% last year, and Georgia ranks top among the NCI States
Annually 2012
67% of respondents reported that they have a choice of supports and services which is 2% improvement from the previous year. 67% is significantly above the national average (54%) of all other NCI States.

104

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0%

Percentage of Crisis Incidents that Resulted in Intensive In-Home Supports

Jan-Mar 2014

Apr-Jun 2014

Jul-Sep 2014

Oct-Dec 2014

Jan-M ar 2014 Apr-Jun 2014 Jul-Sep 2014 Oct-Dec 2014

Numerator

45

34

41

41

De nominator

683

703

665

650

Percentage

7%

5%

6%

6%

CY2014 Totals
161 2701 6%

BOTTOM

0.000

0.000

0.000

0.000

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage of crisis incidents that could warrant additional in-home supports for the individual or family in crisis.

Measure explanation: Most crisis episodes can be sufficiently addressed by a Mobile Crisis Team at the time of the crisis. Some crisis episodes, however, may need additional supports or training for the individual or family that will hopefully lessen or eliminate the chance of such a crisis happening again. These supports or trainings may be provided in the person's home for up to 24 hours a day and 7 days a week.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of crisis episodes statewide that

Denominator: Total number of crisis episodes statewide.

resulted in the need for additional intensive in-home

supports.

COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
Provision of intensive in-home supports remained the same for the last two quarters of 2014, and relatively the same all year long.
July-September 2014 Analysis
Utilization of intensive in-home supports increased 1.2% during this quarter. The Temporary Intensive Supports (TIS) home that supported children and adolescents, was closed in July. The closing could have contributed to the slight increase of in home supports. Additional analysis is needed.
April-June 2014 Analysis
Utilization of intensive in-home supports decreased slightly during this quarter; however, the decrease was not significant.
January-March 2014 Analysis
Utilization of intensive in-home supports dropped by slightly more than half during this quarter (5% as compared to 12% last quarter); however, the number of crisis call episodes doubled (682 compared to 318). Further investigation is needed into the cause for the rise in calls, and the main disposition categories. Receditivism and dually-diagnosed individuals continue to chanllenge DBHDD. DBHDD has been seeking possible providers of emergency respite and planned respite as alternatives to intensive in-home crisis sypports once the crisis has been stabilized.
105

Percentage of Crisis Incidents that Resulted in Placement of the Individual in a Crisis Support Home

100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0%

Jan-Mar 2014

Apr-Jun 2014

Jul-Sep 2014

Oct-Dec 2014

Jan-M ar 2014 Apr-Jun 2014 Jul-Sep 2014 Oct-Dec 2014

Numerator

70

71

63

75

De nominator
Rate

683 10%

703

665

650

10%

9% 12%

CY2014 Totals
279 2701 10%

MEASURE DEFINITION AND EXPLANATION Measure definition: Percentage of crisis incidents that could warrant placement in a crisis support home while the crisis was addressed.

Measure explanation: Most crisis episodes can be sufficiently addressed by a Mobile Crisis Team at the time of the crisis. Some crisis episodes, however, may need additional supports or training for the individual or family that will hopefully lessen or eliminate the chance of such a crisis happening again. From time to time it may be in the best interest of the individual and family that these supports and trainings be provided out of the individuals home and in a crisis support home. Placement in a crisis home should be the option of last resort for dealing with a crisis episode.

COMPONENTS OF NUMERATOR AND DENOMINATOR

Numerator: Number of crisis episodes statewide that

Denominator: Total number of crisis episodes statewide.

resulted in the need for an individual to be removed from

their home and place in a crisis support home.

COMMENTS AND/OR ANALYSIS PER QUARTER
October-December 2014 Analysis
Provision of in-home supports remained the same during this quarter, but utilization of the crisis homes increased by 3 percentage points. The increase was not significant however when compared to the first three quarters of 2014. DBHDD has continued in its efforts to recruit additional respite and emergency respite providers to reduce the to use the crisis homes.
July-September 2014 Analysis
Utilization of the crisis homes remained basically the same as last quarter. There again only a 1 percentage point drop in utilization from last quarter. DBHDD has been seeking possible providers of emergency respite as an alternative to out of home crisis placement.
April-June 2014 Analysis
Utilization of the crisis homes remained basically the same as last quarter. There was only a 1 percentage point drop in utilization from last quarter. DBHDD has been seeking possible providers of emergency respite as an alternative to out of home crisis placement.
January-March 2014 Analysis
Utilization of the crisis homes dropped by half during this quarter (11% as compared to 22% last quarter); however, the number of crisis call episodes doubled (682 compared to 318). Further investigation is needed into the cause for the rise in calls, and the main disposition categories. Receditivism and dually-diagnosed individuals continue to challenge DBHDD. DBHDD has been seeking possible providers of emergency respite as an alternative to out of home crisis placement.
106