State of Georgia Department of Community Health Georgia Families Program AMERIGROUP Community Care PERFORMANCE IMPROVEMENT PROJECTS REPORT SFY 2014 November 2013 TABLE OF CONTENTS 1. Background ........................................................................................................................ 1-1 CMO Overview .................................................................................................................................. 1-3 Study Rationale ................................................................................................................................. 1-3 Study Summary ................................................................................................................................. 1-3 Validation Overview........................................................................................................................... 1-5 HSAG's Validation Scoring Methodology .......................................................................................... 1-7 2. Findings .............................................................................................................................. 2-1 Aggregate Validation Findings .......................................................................................................... 2-1 Design ........................................................................................................................................... 2-3 Implementation .............................................................................................................................. 2-3 Outcomes ...................................................................................................................................... 2-4 PIP-Specific Outcomes ..................................................................................................................... 2-4 Analysis of Results ........................................................................................................................ 2-4 Adolescent Well-Care.................................................................................................................... 2-5 Annual Dental Visits ...................................................................................................................... 2-6 Appropriate Use of ADHD Medications ......................................................................................... 2-7 Childhood Immunizations--Combo 10 .......................................................................................... 2-9 Childhood Obesity ....................................................................................................................... 2-10 Comprehensive Diabetes Care ................................................................................................... 2-12 Avoidable Emergency Room Visits ............................................................................................. 2-13 Member Satisfaction.................................................................................................................... 2-16 Provider Satisfaction ................................................................................................................... 2-17 3.Conclusions and Recommendations ...............................................................................3-1 Conclusions ....................................................................................................................................... 3-1 Recommendations ............................................................................................................................ 3-1 APPENDIX A. PIP-SPECIFIC VALIDATION SCORES....................................................................... A-1 AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Health Services Advisory Group, Inc. Page i ACKNOWLEDGMENTS AND COPYRIGHTS CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA). AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Health Services Advisory Group, Inc. Page ii Performance Improvement Project Validation Report AMERIGROUP Community Care 1. BACKGROUND The Georgia Department of Community Health (DCH) through its Division of Medical Assistance Plans is responsible for administering the Medicaid program and the Children's Health Insurance Program (CHIP) for the State of Georgia and overseeing quality improvement activities. The State refers to its Medicaid managed care program as Georgia Families and to its CHIP program as PeachCare for Kids. For the purposes of this report, "Georgia Families" refers to all Medicaid and PeachCare for Kids members enrolled in managed care. The Georgia Families program serves the majority of Georgia's Medicaid and CHIP populations. The DCH requires its contracted Care Management Organizations (CMOs) serving members under Georgia Families to conduct performance improvement projects (PIPs) as set forth in 42 CFR 438.240 to assess and improve the quality of a targeted area of clinical or nonclinical care or service provided to members, and to report the status and results of each PIP annually. AMERIGROUP Community Care (AMERIGROUP) is one of the Georgia Families CMOs. The validation of PIPs is one of three federally-mandated activities for state Medicaid managed care programs. The other two required activities include the evaluation of CMO compliance with State and federal regulations and the validation of CMO performance measures. These three mandatory activities work together to assess the CMOs' performance with providing appropriate access to high-quality care for their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measure rates provide a barometer of the quality and effectiveness of the care. The DCH requires the CMOs to initiate PIPs to improve the quality of health care in targeted areas of low performance, or in areas identified as State priorities or health care issues of greatest concern. The DCH required its CMOs to conduct nine PIP studies during the 2012 calendar year and submit them for validation in 2013. PIPs are key tools in helping DCH achieve goals and objectives outlined in its quality strategy; they provide the framework for monitoring, measuring, and improving the delivery of health care. The primary objective of PIP validation is to determine each CMO's compliance with requirements set forth in 42 CFR 438.240(b)(1), including: Measurement of performance using objective quality indicators. Implementation of system interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities to increase or sustain improvement. To meet the federal requirement for the validation of PIPs, DCH contracted with Health Services Advisory Group, Inc. (HSAG), the State's external quality review organization (EQRO), to conduct the validation of AMERIGROUP's PIPs. AMERIGROUP submitted PIPs to HSAG AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-1 Health Services Advisory Group, Inc. BACKGROUND between June 30, 2013, and August 1, 2013, and HSAG validated the PIPs between July 1, 2013, and August 8, 2013. The validated data represent varying measurement time periods as described in Table 2-3 through Table 2-11. For PIPs initiated prior to January 1, 2012 (Annual Dental Visits and Childhood Obesity), HSAG reviewed the PIPs using the Centers for Medicare & Medicaid Services (CMS) validation protocols.1-1 For PIPs initiated on or after January 1, 2012 (Adolescent Well-Care Visits, Appropriate Use of ADHD Medications, Avoidable Emergency Room Visits [Collaborative], Childhood Immunizations--Combo 10, Comprehensive Diabetes Care, Member Satisfaction and Provider Satisfaction), HSAG used CMS' updated validation protocols.1-2 Compared to the 2002 CMS PIP protocols, the changes made to the 2012 protocols consisted of reversing the order of Activities III and IV, and Activities VII and VIII. These changes did not impact HSAG's validation process. Table 1-1--CMS Protocol Changes PIP Activity Clearly Defined Study Indicator(s) Correctly Identified Study Population Appropriate Improvement Strategies Sufficient Data Analysis and Interpretation of Results CMS 2002 Protocol Activity III Activity IV Activity VII Activity VIII CMS 2012 Protocol Activity IV Activity III Activity VIII Activity VII HSAG evaluated two key components of the quality improvement process, as follows: 1. HSAG evaluated the technical structure of the PIPs to ensure AMERIGROUP designed, conducted, and reported PIPs using sound methodology consistent with the CMS protocol for conducting PIPs. HSAG's review determined whether a PIP could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring real and sustained improvement. 2. HSAG evaluated the outcomes of the PIPs. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. Evaluation of each PIP's outcomes determined whether AMERIGROUP improved its rates through the implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results) and, through these processes, achieved statistically significant improvement over the baseline rate. Once statistically significant improvement is achieved across all study indicators, HSAG evaluates whether AMERIGROUP was successful in sustaining the improvement. A primary goal of HSAG's PIP validation is to ensure that DCH and key stakeholders can have confidence that 1-1 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Managed Care Organization Protocol. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, Final Protocol, Version 1.0, May 2002. 1-2 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Protocol 3: Validating Performance Improvement Projects (PIPs): A Mandatory Protocol for External Quality Review (EQR), Version 2.0, September 2012. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-2 Health Services Advisory Group, Inc. BACKGROUND reported improvement in study indicator outcomes is supported by statistically significant change and the CMO's improvement strategies. CMO Overview The DCH contracted with AMERIGROUP beginning in 2006 to provide services to the Georgia Families program population. Prior to 2012, AMERIGROUP served the eligible population in the Atlanta, North, East, and Southeast geographic regions of Georgia. In early 2012, the CMO expanded coverage statewide and added the Central and Southwest regions. The HEDIS technical specifications that AMERIGROUP used for its PIP indicators require a member to be continuously enrolled with the CMO. While the new population was included in the PIPs' interventions, the measurement of the PIPs' effectiveness (i.e., the indicator results) excluded members who did not meet the indicators' continuous enrollment criteria. Study Rationale The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas. Although HSAG has validated AMERIGROUP's PIPs for six years, the number of PIPs, study topics, and study methods has evolved over time. AMERIGROUP submitted nine PIPs for validation. The PIP topics included: Adolescent Well-Care Visits Annual Dental Visits Appropriate Use of ADHD Medications Avoidable Emergency Room Visits (Collaborative) Childhood Immunizations--Combo 10 Childhood Obesity Comprehensive Diabetes Care Member Satisfaction Provider Satisfaction Study Summary As noted in its Quality Strategic Plan Update (November 2011), DCH identified the improvement and enhancement of the quality of patient care provided through ongoing, objective, and systematic measurement, analysis, and improvement of performance as one of its four performance-driven goals. The goals are designed to demonstrate success or identify challenges in achieving intended outcomes related to providing quality, accessible, and timely services. AMERIGROUP's June 30, 2013, through August 1, 2013, PIP submissions included six clinical HEDIS-based PIPs: Adolescent Well-Care Visits, Annual Dental Visits, Appropriate AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-3 Health Services Advisory Group, Inc. BACKGROUND Use of ADHD Medications, Childhood Immunizations--Combo 10, Childhood Obesity, and Comprehensive Diabetes Care; two nonclinical PIPs: Member Satisfaction and Provider Satisfaction; and one collaborative PIP: Avoidable Emergency Room Visits. Table 1-2 outlines the key study indicators incorporated for the six clinical HEDIS-based PIPs. Table 1-2--PIP Study Topics and Indicator Descriptions PIP Study Topic Adolescent WellCare Visits Annual Dental Visits Appropriate Use of ADHD Medications Childhood Immunizations-- Combo 10 Childhood Obesity Comprehensive Diabetes Care PIP Study Indicator Description The percentage of members 1221 years of age who had at least one comprehensive well-care visit with a PCP or an OB/GYN practitioner during the measurement year. The percentage of members 23 years of age and 221 years of age who had at least one dental visit during the measurement year. 1. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who had one follow-up visit with a practitioner with prescribing authority during the 30-day Initiation Phase. 2. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who remained on the medication for at least 210 days and who, in addition to the visit in the Initiation Phase, had at least two follow-up visits with a practitioner from 31300 days following the IPSD. One of the two visits (during days 31300) may be a telephone visit with a practitioner. The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis (DTaP); three polio (IPV); one measles, mumps, rubella (MMR); three H influenza type B (HiB); three hepatitis B (HepB); one chicken pox (VZV); four pneumococcal conjugate (PCV); one hepatitis A (HepA); two or three rotavirus (RV); and two influenza (flu) vaccines by their second birthday. The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation, nutrition counseling and physical activity counseling. The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had a HbA1c control < 7.0%, LDL-C control < 100mg/ml, and BP control < 140/90 mmHg. Table 1-3 outlines the key study indicators for the collaborative Avoidable Emergency Room Visits PIP. Table 1-3--Collaborative PIP Study Topic and Indicator Descriptions PIP Study Topic Avoidable Emergency Room Visits PIP Study Indicator Description 1. The percentage of practices that provide the same day appointments for routine and urgent care. 2. The percentage of practices that provide routine and urgent care appointments after hours. 3. The percentage of practices that provide appointments for routine and urgent care after hours and have the ability to document after hours clinical advice in the patient's record. 4. The percentage of practices that have access to and utilize electronic health records. 5. The percentage of practices that receive information regarding ER visits from the study hospitals. 6. The percentage of ER visits for `avoidable' diagnoses (dx382Acute Suppurative otitis:382.9Unspecified otitis:462Acute pharyngitis:465.9Acute upper respiratory infection:466Acute bronchitis:786.2Cough) among members under 21 years of age who had a visit to the ED in three selected Children's Healthcare of Atlanta facilities in the Atlanta region. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-4 Health Services Advisory Group, Inc. BACKGROUND Table 1-4 outlines the key study indicators incorporated for the two satisfaction-based PIPs. The effectiveness of the Member Satisfaction PIP was measured using the Consumer Assessment of Healthcare Providers and Systems (CAHPS) 5.0H, Medicaid Child Survey. This survey provided information on parents' experiences with their child's provider and CMO. The final AMERIGROUP PIP topic was Provider Satisfaction. AMERIGROUP contracted with a vendor to produce and administer a survey to document the effectiveness of this performance improvement project. Survey Type Member Provider Table 1-4--Satisfaction-Based PIP Study Indicators Question #36 #48 Survey Question The percentage of respondents who rate the health plan an 8, 9, or 10 in response to the question "Using any number from 0-10, where 0 is the worst health plan and 10 is the best, what number would you use to rate your child's health plan?" The percentage of providers who respond "very satisfied" or "somewhat satisfied" to the question "Please rate your overall satisfaction with Amerigroup." Validation Overview HSAG obtained the data needed to conduct the PIP validation from AMERIGROUP's PIP Summary Forms. These forms provided detailed information about AMERIGROUP's completed PIP activities. Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be scored Met. Given the importance of critical elements to the scoring methodology, any critical element that received a Not Met score resulted in an overall validation status for the PIP of Not Met. A CMO would be given a Partially Met validation status if 60 percent to 79 percent of all evaluation elements were scored Met or one or more critical elements were scored Partially Met. HSAG provided a Point of Clarification when the CMO fully met the evaluation element criteria and only minor documentation edits not critical to the validity of the PIP were recommended to the CMO. In addition to the overall validation status (e.g., Met), HSAG provided an overall percentage for all evaluation elements (including critical elements) scored Met. HSAG calculated the overall percentage by dividing the total number of elements scored Met by the total number of elements scored Met, Partially Met, and Not Met. HSAG also calculated a critical element overall percentage by dividing the total number of critical elements scored Met by the sum of the critical elements scored Met, Partially Met, and Not Met. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-5 Health Services Advisory Group, Inc. BACKGROUND Figure 1-1 illustrates the three study stages of the PIP process: Design, Implementation, and Outcomes. The Design stage establishes the methodological framework for the PIP. The activities in this stage include development and documentation of the study topic, question, indicators, population, sampling, and data collection. A sound study design is necessary for the successful implementation of improvement strategies. Once the study design is established, the PIP process moves into the Implementation stage. This stage includes data analysis and implementation of improvement strategies. During the Implementation stage, CMOs should incorporate a continuous or rapid cycle improvement model such as the Plan-Do-Study-Act (PDSA) Cycle. Figure 1-1--PIP Study Stages Incorporating the PDSA Cycle Outcomes Design The PDSA cycle includes the following actions: Plan--conduct barrier analyses; prioritize barriers; develop targeted intervention(s) to address barriers; and develop an intervention evaluation plan for each intervention Do--implement intervention; track and monitor the intervention; and record the data Study--analyze the data; compare results; and evaluate the intervention's effectiveness Act--based on the evaluation results, standardize, modify, or discontinue the intervention The PDSA cycle is repeated throughout each measurement period. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final Outcomes stage evaluates for statistically significant and sustained improvement of the project outcomes. Once statistically significant improvement in the outcomes is achieved, the improvement must be sustained in a subsequent measurement period. If the study outcomes do not improve, the CMO's responsibility is to continue the PDSA cycle until statistically significant improvement is achieved and sustained. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-6 Health Services Advisory Group, Inc. BACKGROUND HSAG's Validation Scoring Methodology The scoring methodology evaluates whether or not the CMO met all the documentation requirements according to the CMS protocols, as well as evaluates whether or not all study indicators have achieved statistically significant improvement over the baseline rate. In Activity IX (real improvement achieved), the CMO must achieve statistically significant improvement across all study indicator(s) between the baseline and a subsequent measurement period to receive a Met score. For Activity X (sustained improvement achieved), HSAG assesses for sustained improvement once all study indicators achieve statistically significant improvement over the baseline and the CMO reports a subsequent measurement period. All study indicators must achieve statistically significant improvement and sustain this improvement to receive a Met validation score in Activity X. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 1-7 Health Services Advisory Group, Inc. 2. FINDINGS for AMERIGROUP Community Care Aggregate Validation Findings HSAG organized, aggregated, and analyzed AMERIGROUP's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of interventions. Based on its review, HSAG determined the overall methodological validity of the PIPs, as well as the overall success in achieving improved study indicator outcomes. The results are presented in Table 2-1. Table 2-1--Performance Improvement Project Validation Scores for AMERIGROUP Community Care PIP Adolescent Well-Care Visits Annual Dental Visits Appropriate Use of ADHD Medication Avoidable Emergency Room Visits-- Collaborative Childhood Immunization--Combo 10 Childhood Obesity Comprehensive Diabetes Care Member Satisfaction Provider Satisfaction Percentage of Evaluation Elements Scored Met 88% 97% 78% 62% 96% 82% 88% 91% 91% Percentage of Critical Elements Scored Met 86% 92% 82% 50% 100% 87% 93% 100% 100% Validation Status Not Met Partially Met Not Met Not Met Met Partially Met Not Met Met Met Three of the nine PIPs validated received an overall Met validation status. The Annual Dental Visits and Childhood Obesity PIPs received a Partially Met validation status due to the reporting of inaccurate data, implementing interventions that did not directly impact the study indicator(s) rates, inaccurate statistical testing, achieving statistically significant improvement for some but not all study indicators, and/or sustaining the statistically significant improvement for some but not all study indicators. The Childhood Obesity PIP also received a Partially Met validation score because the rate for Study Indicator 1 for this PIP did not match the performance measure rate reported in the CMO's performance measure results that were validated by the CMO's independent auditor. Additionally, many of the interventions implemented for this PIP did not directly impact the study indicators' rates. Furthermore, the CMO was unable to sustain the statistically significant improvement that was achieved at Remeasurement 2 for all of the study indicators. For the Annual Dental Visits PIP, the CMO reported a numerator for Study Indicator 1 that did not match the numerator reported in the performance measure results that were validated by the CMO's independent auditor. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-1 Health Services Advisory Group, Inc. FINDINGS The Adolescent Well-Care Visits, Appropriate Use of ADHD Medications, Avoidable Emergency Room Visits, and Comprehensive Diabetes Care PIPs received an overall Not Met validation status. None of the study indicators for these four PIPs achieved statistically significant improvement over baseline. For the Adolescent Well-Care Visits PIP, the study indicator numerator did not match the results that were validated by the CMO's independent auditor. For its Appropriate Use of ADHD Medications PIP, AMERIGROUP reported study indicator rates for both study indicators that did not match the rates that were validated by the CMO's independent auditor. The collaborative Avoidable Emergency Room Visits PIP received a Not Met validation status for several reasons. The CMO did not completely define the study population or the study indicators, or explain how the data were collected for all study indicators. Additionally, not all study indicators achieved statistically significant improvement over the baseline rates. For the Comprehensive Diabetes Care PIP, the study indicators did not achieve statistically significant improvement resulting in the Not Met validation status. Table 2-2 displays the combined validation results for all nine AMERIGROUP PIPs validated during SFY 2014. This table illustrates the CMO's application of the PIP process and its success in implementing all nine projects. Each activity was composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score satisfied the necessary technical requirements for a specific element. The validation results presented in Table 2-2 show the percentage of applicable evaluation elements that received a Met score by activity. Additionally, HSAG calculated an overall percentage of Met scores across all activities for all nine PIPs. Appendix A provides the detailed scores from the validation tool for each of the nine PIPs. Table 2-2--Performance Improvement Project Validation Results for AMERIGROUP Community Care (N=9 PIPs) Study Stage Design Activity Appropriate Study Topic Clearly Defined, Answerable Study Question(s) Clearly Defined Study Indicator(s) Correctly Identified Study Population Valid Sampling Techniques (if sampling was used) Accurate/Complete Data Collection Design Total* Percentage of Applicable Elements Met Partially Met Not Met 96% 0% 4% (49/51) (0/51) (2/51) 100% 0% 0% (18/18) (0/18) (0/18) 96% 4% 0% (53/55) (2/55) (0/55) 92% 8% 0% (23/25) (2/25) (0/25) 100% 0% 0% (36/36) (0/36) (0/36) 83% 4% 13% (64/77) (3/77) (10/77) 93% 3% 5% (243/262) (7/262) (12/262) AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-2 Health Services Advisory Group, Inc. FINDINGS Table 2-2--Performance Improvement Project Validation Results for AMERIGROUP Community Care (N=9 PIPs) Study Stage Activity Sufficient Data Analysis and Interpretation of Results Implementation Appropriate Improvement Strategies Implementation Total* Outcomes Real Improvement Achieved Sustained Improvement Achieved Outcomes Total Percentage of Applicable Evaluation Elements Scored Met * Percentages do not total 100 percent due to rounding. Percentage of Applicable Elements Met Partially Met Not Met 77% (54/70) 20% (14/70) 3% (2/70) 78% (18/23) 22% (5/23) 0% (0/23) 77% (72/93) 20% (19/93) 2% (2/93) 54% (15/28) 14% (4/28) 32% (9/28) 50% 50% 0% (1/2) (1/2) (0/2) 53% (16/30) 17% (5/30) 30% (9/30) 86% (331/385) Overall, 86 percent of the evaluation elements across all nine PIPs received a Met score. AMERIGROUP demonstrated strong performance in the Design stage, with the exception of its Avoidable Emergency Room Visits PIP; however, the CMO was less successful in the Implementation and Outcomes stages. The following subsections highlight HSAG's validation findings associated with each of the three PIP stages. Design AMERIGROUP met 93 percent of the requirements across all nine PIPs for the six activities within the Design stage. With the exception of the Avoidable Emergency Room Visits PIP, the technical design of each PIP was sufficient to measure and monitor PIP outcomes. The solid foundation of the PIPs allowed for the CMO to progress to the next stage of the PIP process. Implementation AMERIGROUP met 77 percent of the requirements for the two activities within the Implementation stage. The CMO did not report accurate data components in some of its PIPs, and not all of the statistical testing performed was completely accurate. Additionally, the CMO did not address all of the documentation requirements in the Sufficient Data Analysis and Interpretation of Results activity. For the Appropriate Improvement Strategies activity, AMERIGROUP documented that it had conducted causal/barrier analyses; however, AMERIGROUP had interventions in place that did not directly impact the study indicator outcomes. In addition, there was no process in place to evaluate the effectiveness of the interventions. Overall, approximately 54 percent of the PIPs were successful in achieving statistically significant improvement while 46 percent were not successful in achieving AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-3 Health Services Advisory Group, Inc. FINDINGS statistically significant improvement. Of the two PIPs assessed for sustained improvement, one PIP sustained improvement while one did not. Outcomes This year, five PIPs (Adolescent Well-Care Visits, Appropriate Use of ADHD Medication, Childhood Immunizations--Combo 10, Comprehensive Diabetes Care, and Avoidable Emergency Room Visits) were evaluated for achieving statistically significant improvement. Only one PIP, Childhood Immunizations--Combo 10, achieved statistically significant improvement over baseline at Remeasurement 1. Two PIPs, Annual Dental Visits and Childhood Obesity, progressed to the point of being assessed for sustained improvement. Sustained improvement is defined as statistically significant improvement in performance over baseline that is maintained or increased for at least one subsequent measurement period. Additionally, the results of the most current measurement period must reflect improvement when compared to baseline results. Both study indicators in the Annual Dental Visits PIP achieved sustained improvement, while only two of the three indicators achieved sustained improvement in the Childhood Obesity PIP. PIP-Specific Outcomes Analysis of Results Each table below displays the study indicator rates for each measurement period of the PIP, including the baseline period and each subsequent remeasurement period. Statistically significant changes between remeasurement periods are noted with an upward or downward arrow followed by an asterisk. If the PIP achieved statistically significant improvement over the baseline rate, it was then reviewed for sustained improvement. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. PIPs that did not achieve statistically significant improvement (i.e., did not meet the criteria to be assessed for sustained improvement) were not assessed (NA). Comparisons of PIP study indicator results that utilized HEDIS measures were made using the Medicaid HEDIS 2011 Audit, Means, Percentiles, and Ratios (reflecting the 2010 calendar year [CY]). AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-4 Health Services Advisory Group, Inc. FINDINGS Adolescent Well-Care Table 2-3--Performance Improvement Project Outcomes for Adolescent Well-Care Visits PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of members 1221 years of age who had at least one comprehensive well-care visit with a PCP or an 43.9% 46.6% NA OB/GYN practitioner during the measurement year. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. AMERIGROUP did not achieve statistically significant improvement from baseline to Remeasurement 1 in the Adolescent Well-Care Visits PIP. The study indicator's rate increased by 2.7 percentage points, but the change was not statistically significant. The Remeasurement 1 rate remained below the DCH target rate of 46.8 percent. A critical analysis of the CMO's improvement strategies for this PIP demonstrated the following: AMERIGROUP identified barriers to improving the Adolescent Well-Care Visits indicator rate through brainstorming and summarized the causal/barrier analysis process with a fishbone diagram. The plan discussed its general process for identifying barriers and developing interventions; however, the specific results of the CMO's causal/barrier analysis were not documented. The plan did not provide the rationale for how it prioritized barriers or why it continued interventions that were not associated with outcome improvements. Interventions addressing the barriers were implemented during the baseline and remeasurement periods; however, the CMO did not have a sound evaluation plan for any of the interventions. For example, AMERIGROUP implemented an intervention to make outreach calls to members as a strategy to increase the percentage of members receiving a well-care visit. The CMO documented that it tracked the adolescent well-care visit claims and reported that 81 percent were linked to the member outreach calls; however, there was no method documented that would support this conclusion. The CMO did not track or monitor its other intervention efforts which included expanding the Provider Quality Incentive Program (PQIP), educating providers on conducting well-care assessments during a sick visit, and educating providers on billing after-hour codes for completing services after normal business hours. AMERIGROUP also piloted its new My Health Direct program with three providers. This program allowed internal associates to schedule adolescent well-visit appointments while they were engaged with members on the telephone and allowed providers to block a portion of their day for AMERIGROUP members' appointments. The CMO did not document how the providers were selected for My Health Direct or evaluate the visit rates for these specific providers after the intervention was initiated. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-5 Health Services Advisory Group, Inc. FINDINGS CMO-specific interventions, focused on member and provider education, were delivered primarily through member and provider newsletters. This non-targeted education did not lend itself to evaluation and was not associated with any improvement in performance. Interventions that are data-driven and targeted may be an overall more effective strategy, especially with a growing Georgia Families population and finite resources. Annual Dental Visits Table 2-4--Performance Improvement Project Outcomes for Annual Dental Visits PIP Study Indicator Baseline Period (1/1/0912/31/09) Remeasurement 1 Remeasurement 2 Remeasurement 3 (1/1/1012/31/10) (1/1/1112/31/11) (1/1/1212/31/12) Sustained Improvement^ The percentage of members 221 years of age who had at least one dental visit 66.7% 69.1%* 69.7%* 69.9% Yes during the measurement year. The percentage of members 23 years of age who had at least one dental visit during the measurement year. 42.7% 47.3%* 47.7% 48.5% Yes * Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. For the Annual Dental Visits PIP, AMERIGROUP sustained statistically significant improvement over baseline rates for both indicators at Remeasurement 3. The rate for members 221 years of age (Study Indicator 1) exceeded the CY 2012 DCH target rate of 64.1 percent. Additionally, the Remeasurement 3 rates for both study indicators exceeded the national Medicaid HEDIS 2011 90th percentiles of 64.5 percent (221 years of age) and 46.9 percent (2 3 years of age). A critical analysis of the CMO's improvement strategies for this PIP demonstrated the following: AMERIGROUP's internal interdisciplinary team reviewed its fishbone analysis and determined that all barriers were still relevant; however, new interventions were implemented for barriers not previously identified. Due to the PIP's success, the CMO continued its previous year's interventions and implemented monthly meetings with SCION, its dental vendor, to discuss GeoAccess, appointment availability call center statistics, survey results, and performance tracking. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-6 Health Services Advisory Group, Inc. FINDINGS The CMO's interventions for this PIP included reminder calls, health fairs, health promotion events, additional dental coverage benefits, missed opportunity reports through the provider portal for primary care practices, and annual dental rate report cards. The CMO stated that results from statistical testing performed on the administrative rates were used to determine the effectiveness of its interventions, as well as to plan future interventions. The CMO also documented that it reviewed quarterly reports to assess for noncompliant populations. Although the CMO improved and sustained its outcomes, it should ensure that each intervention includes an evaluation plan. Without a method to evaluate the effectiveness of the intervention, the plan cannot determine which intervention to modify or discontinue, or when to implement new interventions, thereby reducing the likelihood of achieving project objectives and improving performance. Appropriate Use of ADHD Medications Table 2-5--Performance Improvement Project Outcomes for Appropriate Use of ADHD Medications PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ 1. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who had one follow-up visit with a 44.3% 42.3% NA practitioner with prescribing authority during the 30- day Initiation Phase. 2. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who remained on the medication for at least 210 days and who, in addition to the visit in the 61.2% 58.2% NA Initiation Phase, had at least two follow-up visits with a practitioner from 31300 days following the IPSD. One of the two visits (during days 31300) may be a telephone visit with a practitioner. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. Neither study indicator in the Appropriate Use of ADHD Medications PIP achieved statistically significant improvement at Remeasurement 1. Conversely, the rates of follow-up care visits for children newly prescribed ADHD medication declined for both the initiation phase (Study Indicator 1) and for the continuation and maintenance phases (Study Indicator 2), though neither decline was statistically significant. The rate for Study Indicator 1 (Initiation) remained below the DCH target rate of 48.1 percent and fell just below the national Medicaid HEDIS 2011 75th AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-7 Health Services Advisory Group, Inc. FINDINGS percentile of 43.6 percent. The rate for Study Indicator 2 (continuation) exceeded the DCH target rate of 57.6 percent and fell between the 75th (52.6 percent) and 90th (62.5 percent) percentiles of the national Medicaid HEDIS 2011 rates. An analysis of the plan's improvement strategy for this PIP identified some weaknesses which may have led to the lack of improvement in the outcomes. AMERIGROUP completed a fishbone analysis to identify its barriers. The CMO implemented most of its interventions in CY 2012. The majority of interventions for this PIP were provider-focused, addressing the barriers of "lack of provider education" and "lack of provider knowledge." These interventions included distributing ADHD Clinical Practice Guidelines (CPGs) to providers, distributing HEDIS report cards to providers that showed the providers' performance on ADHD follow-up, educating providers on how to retrieve missed opportunity reports via the provider portal so providers could take action to ensure compliance, and hand-delivered and faxed "First Fill" letters to providers who prescribed ADHD medications. The "First Fill" letters reminded providers to ensure that a follow-up visit was scheduled with their patients. The CMO did not track or monitor any of these efforts. The CMO documented that from January 2012 to June 2013, it analyzed data and noted that 94 percent of members who received robotic outbound reminder calls had ADHD claims for follow-up. Based on this analysis, the CMO determined that the "robotic calls to members" intervention was successful, although there was no method to actually evaluate whether the visits were related to the reminder call. HSAG recommends that AMERIGROUP implement a process to evaluate each PIP intervention. Specifically for this PIP, HSAG recommends that AMERIGROUP evaluate whether or not expanding the Provider Quality Incentive Program (PQIP) to the ADHD measure would be an effective approach. Based on the decline in performance for both study indicators, which should have been detected during the quarterly reviews of data, HSAG anticipated that the CMO would have evaluated the effectiveness of each intervention, performed additional data mining to determine the cause of the decline in performance, and implemented targeted improvement strategies. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-8 Health Services Advisory Group, Inc. FINDINGS Childhood Immunizations--Combo 10 Table 2-6--Performance Improvement Project Outcomes for Childhood Immunizations--Combo 10 PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis (DTaP); three polio (IPV); one measles, mumps, rubella (MMR); three H influenza type B (HiB); three hepatitis B (HepB); one chicken 10.4% 31.9%* NA pox (VZV); four pneumococcal conjugate (PCV); one hepatitis A (HepA); two or three rotavirus (RV); and two influenza (flu) vaccines by their second birthday. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. AMERIGROUP achieved statistically significant improvement over the baseline rate in the first remeasurement period for the Childhood Immunizations--Combo 10 PIP, with an increase of 21.5 percentage points. The Remeasurement 1 rate also exceeded the national Medicaid HEDIS 2011 90th percentile of 23.6 percent. An analysis of the CMO's improvement strategy for this PIP showed the following: AMERIGROUP identified barriers and developed interventions based on the outcomes from a fishbone analysis. More importantly, the CMO held focus group sessions to get a better understanding of the members' perspective and potential barriers. The focus groups were held in each region, and the CMO also checked each focus group participant's status at the time of the focus group. In CY 2012, AMERIGROUP implemented several interventions: One of the interventions was the expansion of its PQIP to incorporate 13 additional highvolume providers. To be eligible to participate, providers had to demonstrate high-quality scores. Reimbursement was dependent on eligible providers' medical loss ratios. AMERIGROUP also prepaid for PeachCare for Kids members' vaccines prior to them being given by the providers. The CMO implemented this initiative because immunizations for the PeachCare for Kids members are not available under the Vaccines for Children (VFC) program and must be provided by the CMO for this population. The CMO reviewed HEDIS specifications and aligned claims codes with these specifications to ensure providers were being paid accordingly. The CMO also sent letters to providers and conducted face-to-face discussions regarding unavailable vaccines. AMERIGROUP asked its providers to work around the availability of these vaccines to ensure all vaccines were eventually administered. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-9 Health Services Advisory Group, Inc. FINDINGS Although the study indicator achieved statistically significant improvement, HSAG encourages AMERIGROUP to develop processes to evaluate the effectiveness of each intervention. This will assist the CMO in determining what worked and which interventions could be instrumental in sustaining the statistically significant improvement achieved. Childhood Obesity Table 2-7--Performance Improvement Project Outcomes for Childhood Obesity PIP Study Indicator Baseline Period Remeasurement 1 Remeasurement 2 Remeasurement 3 Sustained (1/1/0912/31/09) (1/1/1012/31/10) (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation. 13.7% 28.5%* 33.3% 40.7%* Yes The percentage of members 317 years of age who had an outpatient visit with a PCP or 40.7% 48.8%* 58.3%* 52.3% Yes OB/GYN and who had evidence of counseling for nutrition. The percentage of members 317 years of age who had an outpatient visit with a PCP or 35.6% 30.9% 44.9%* 39.8% No OB/GYN and who had evidence of counseling for physical activity. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. The Childhood Obesity PIP sustained statistically significant improvement over the baseline rate at Remeasurement 3 for Study Indicators 1 (BMI percentile documentation) and 2 (evidence of nutrition counseling). The rate for Study Indicator 3 (evidence of physical activity counseling) declined at Remeasurement 3 and was no longer significantly higher than the baseline rate. All three of the study indicators fell below their respective DCH CY 2012 goals of 45.2 percent (BMI percentile documentation), 57.7 percent (evidence of nutrition counseling), and 45.5 percent (evidence of physical activity counseling). The CMO's rates for Study Indicators 1 (BMI AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-10 Health Services Advisory Group, Inc. FINDINGS percentile documentation) and 2 (evidence of nutrition counseling) were slightly above the 50th percentile of the national Medicaid HEDIS 2011 rates of 37.5 percent and 51.1 percent, respectively. The rate for Study Indicator 3 (evidence of physical activity counseling) fell below the national Medicaid HEDIS 2011 50th percentile of 40.6 percent. AMERIGROUP continued its two-pronged approach and targeted interventions toward both members and providers based on the outcomes of the fishbone analysis despite the fact that the study indicators for this PIP are provider driven. These study indicators are only evaluating the presence of documentation of BMI, counseling for nutrition, and counseling for physical activity documentation. The CMO also held focus group sessions to get a better understanding of the member's perspective and to identify any new barriers. The focus groups were held in each region, and the CMO also checked each participant's status at the time of the focus group. The CMO's overall efforts were not linked directly to the study outcome of documented presence of BMI, counseling for nutrition, and counseling for physical activity. The outcomes for this PIP are only impacted by the providers' documentation in the members' records. The following are some interventions the CMO implemented that would not impact the outcomes for these study indicators: Addressed obesity concerns with members through interactive case management Distributed nearly 7,000 fliers on childhood obesity Sent text messages to 5,400 households via a free cellular telephone provided by Safelink Hosted three obesity events where 180 members attended HSAG recommends that AMERIGROUP implement provider-focused interventions and address why providers had decreased documentation of counseling for nutrition and counseling for physical activity for Study Indicator 3 from Remeasurement 2 to Remeasurement 3. AMERIGROUP appears to have aims that extend beyond the HEDIS-based study indicators in this PIP to address broader issues, such as educating members on obesity. If the CMO wants to include these initiatives as part of its PIP, it should restructure the PIP to include study indicators that measure the intended outcome. In addition, HSAG recommends that the CMO have processes in place to evaluate the success of each intervention. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-11 Health Services Advisory Group, Inc. FINDINGS Comprehensive Diabetes Care Table 2-8--Performance Improvement Project Outcomes for Comprehensive Diabetes Care PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had an HbA1c control < 7.0%. 32.1% 30.6% NA The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had a LDL-C control < 100mg/ml. 26.4% 27.3% NA The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had a BP control < 140/90 mmHg. 58.2% 55.1% NA NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. None of the study indicators for the Comprehensive Diabetes Care PIP achieved statistically significant improvement over baseline rates at Remeasurement 1. While there was no statistically significant change in any of the indicators, the rates of Study Indicators 1 (HbA1c Control < 7.0%) and 3 (Blood Pressure (BP) control < 140/90 mmHg) decreased, and the rate of Study Indicator 2 (LDL-C < 100 mg/ml) increased non-significantly. The Remeasurement 1 rates for all three study indicators fell below the CY 2012 DCH targets of 35.5 percent (HbA1c control < 7.0%), 33.6 percent (LDL-C control < 100 mg/ml), and 61.6 percent (BP Control < 40/90 mmHg), and all were below the corresponding national Medicaid HEDIS 2011 50th percentile rates of 35.2 percent (HbA1c Control < 7.0% and LDL-C control < 100 mg/ml) and 61.2 percent (BP Control < 140/90 mmHg). AMERIGROUP implemented both member- and provider-focused interventions based on its causal/barrier analysis outcomes. The CMO identified lack of member knowledge regarding services needed and lack of provider education and guidance on HEDIS requirements as the primary barriers. However, the CMO implemented interventions that addressed only screening rather than control of HbA1c, LDL-C, and BP, which was what the outcomes were measuring. While increasing the number of screened members could improve the rates if those additional screened members have controlled HbA1c and LDL-C, current efforts do not appear to be focused on decreasing the percentage of members who are uncontrolled. AMERIGROUP's screening rate for HbA1c was 79.37 percent and 73.21 percent for LDL-C screening; therefore, focusing solely on increased screening only has the potential to improve rates by approximately 2027 percent. Efforts aimed at both increased screening and control may yield a greater increase and more rapid rate of improvement. The following were some of the interventions AMERIGROUP implemented: AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-12 Health Services Advisory Group, Inc. FINDINGS Placed robotic calls to diabetic members to remind them of diabetic screenings. Implemented a new member incentive program that allowed for the distribution of $25 for every member who received an HbA1c, LDL-C, and BP screening. Piloted a project that distributed an appointment book to 15 adult members with diabetes to see if this improved compliance with attending office visits would promote positive health outcomes, and improve the performance of the study indicators. Based on the result of the pilot, the intervention was expanded to provide 100 calendars that were distributed by case managers to members with diabetes who had a history of missed appointments. Distributed HEDIS report cards showing providers their performance on HbA1c testing. Mailed letters to providers that listed non-compliant diabetic members needing services. Mailed letters to non-compliant members notifying them of the diabetic services needed. Due to the lack of statistically significant improvement across all study indicators, HSAG encourages AMERIGROUP to revisit its causal/barrier analysis to determine the reason for the lack of improvement; and revise current interventions and/or implement new strategies to address members' successfully controlling their HbA1c, LDL-C, and BP levels. The CMO may need to shift focus to engaging providers for strategies to increase control for members with diabetes since members with poor control are at an increased risk for eye disease, kidney disease, heart disease, nerve damage, stroke, and lower extremity amputation, among other health problems. Avoidable Emergency Room Visits In CY 2012, AMERIGROUP began participating in a collaborative performance improvement project with DCH and two other CMOs to address avoidable emergency room visits by evaluating combined data and implementing coordinated interventions. The collaborative's goal was to reduce avoidable emergency room visits by 5 percent by the end of CY 2012. The baseline and Remeasurement 1 rates for the six study indicators documented in the PIP submission for the collaborative Avoidable Emergency Room Visits PIP are summarized in Table 2-9. Table 2-9--Performance Improvement Project Outcomes for Avoidable Emergency Room Visits PIP Study Indicator 1. The percentage of practices that provide the same day appointments for routine and urgent care. 2. The percentage of practices that provide routine and urgent care appointments after hours. 3. The percentage of practices that provide appointments for routine and urgent care after hours and have the ability to document after hours clinical advice in the patient's record. Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ 100% 100% NA 50% 70% NA 100% 100% NA AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-13 Health Services Advisory Group, Inc. FINDINGS Table 2-9--Performance Improvement Project Outcomes for Avoidable Emergency Room Visits PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ 4. The percentage of practices that have access to and utilize electronic health records. 70% 90% NA 5. The percentage of practices that receive information regarding ER visits from the study hospitals. 80% 100% NA 6. The percentage of ER visits for `avoidable' diagnoses (dx382Acute Suppurative otitis:382.9Unspecified otitis:462Acute pharyngitis:465.9Acute upper respiratory infection:466 Acute bronchitis:786.2Cough) 19.38% 20.52%* NA among members under 21 years of age who had a visit to the ED in three selected Children's Healthcare of Atlanta facilities in the Atlanta region. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. * Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. The Avoidable Emergency Room Visits PIP had six study indicators and was piloted in the metro-Atlanta region of the State. Study Indicators 1 through 5 assessed the 10 metro-Atlanta provider practices associated with the highest number of avoidable emergency room visits, and Study Indicator 6 assessed visits to the emergency departments of three Children's Healthcare of Atlanta facilities. Study Indicators 1 through 5 were incorporated at the direction of the State to serve as lead measures. Lead indicators can be helpful in predicting changes that the CMO may use to make mid-course corrections to allow for timely, rapid cycles of improvement rather than waiting for the lag or outcome measure of the PIP, which relies on annual measurement. The initial data for these lead measures were collected by the CMOs during the course of the PIP, and the results showed that these measures did not allow an opportunity for improvement in Study Indicators 1 (percentage of providers who provide same-day appointments) and 3 (percentage of practices that have the ability to document after-hours clinical advice) because the baseline rate for each indicator was 100 percent. The study indicators were created before some of the baseline data were obtained from the participating practices and as such, the CMOs were unaware these baseline rates would be 100 percent. The rates of Study Indicators 2, 4, and 5 had non-statistically significant improvement from baseline to Remeasurement 1. The rate of Study Indicator 5 (percentage of practices that receive ER visit information from study hospitals) reached 100 percent at Remeasurement 1; therefore, this indicator has no room for improvement in future measurement periods for the metro-Atlanta pilot practices. Study Indicator 6, the percentage of emergency room visits for the specified subset of avoidable diagnoses, is the only indicator that did not improve, as there was a significant increase of 1.14 percentage points in the rate of avoidable emergency room visits from baseline to Remeasurement 1. HSAG recommends the CMOs modify their reporting of this PIP for the next remeasurement period and include the lead measures in Activity VIII on the PIP report template. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-14 Health Services Advisory Group, Inc. FINDINGS Through its validation review, HSAG noted structural flaws in the documentation of the study design (Activities I through VI) for this collaborative PIP. The numerator and denominator descriptions for Study Indicators 2 and 3 that were documented by the CMOs were identical. The CMOs will need to correct this prior to the next annual submission. Within the study design, the CMOs did not completely define the study population. The CMOs stated, "The method for identifying member visits in the denominator was derived from a list of ICD-9 codes determined to be `avoidable,' i.e., non-emergent conditions that could have been treated in another outpatient setting." NA is not applicable to this element. The denominator (study population) should be composed of all emergency room visits for CMO members under the age of 21. The CPT, UB Revenue, and place of service codes used to identify an emergency room visit, and the anchor date criteria, were not included. In addition, the CMOs did not identify the 10 providers that were involved in the pilot project as part of the study population definition. For the data collection methodology, the CMOs did not include the codes used to identify emergency room visits (denominator for Study Indicator 6). Furthermore, it was unclear how the survey used by the CMOs captured data for Study Indicators 2 and 3. Prior to the three CMOs coming together, AMERIGROUP documented that it implemented interventions at the plan level to decrease avoidable emergency room visits. One of these interventions was the emergency room case management program for members with high emergency room utilization. Outbound calls were made to these members to discuss the reasons for their visits to the ER. AMERIGROUP also provided weekly ER utilization reports to targeted groups via the provider portal. In Activity VIII, Implement Intervention and Improvement Strategies, the CMOs documented that a multidisciplinary team of participants from the three CMOs, representatives from DCH, and several study participants reviewed the baseline results of the provider survey, as well as the member focus study, to determine barriers and opportunities for improvement. Interventions were developed to address member, provider, and resource barriers. The CMOs documented that provider-level interventions were designed to motivate providers to offer after-hours care, as well as to encourage the use of electronic health records in the practices. Data sharing was designed to give providers the insight into their level of performance and to identify areas of potential opportunity such as proactive member outreach to establish a medical home. The following were the collaborative provider-level interventions: Increased percentage of practices using electronic health records through referral to the Georgia Health Information Technology Regional Extension Center (GA-HITREC). Shared data regarding ER rates with practices to identify members using the ER during regular office hours. Notified providers regarding the availability of additional reimbursement for care provided after-hours. Member improvement strategies were focused on educating members regarding the available resources to prevent ER use. The following are the collaborative member-level interventions: AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-15 Health Services Advisory Group, Inc. FINDINGS Continued ER case management programs for live outreach to members who frequented the ER. Educational mailings to members regarding patient-centered medical homes (PCMHs) and nurse advice hotlines. Provided materials to members regarding transportation vendors and assistance to members to arrange transportation, when needed. The PIP documentation did not reflect any processes that were in place to evaluate the effectiveness of any interventions. Although the CMOs discussed follow-up activities planned, due to the decline in performance for the avoidable ER visit rate indicator (Study Indicator 6), HSAG recommends the CMOs, collaboratively, investigate the reasons for the decrease in performance and based on the findings, implement strategies to improve performance. Member Satisfaction Table 2-10--Performance Improvement Project Outcomes for Member Satisfaction PIP Study Indicator Baseline Period Sustained (2/22/125/9/13) Improvement^ The percentage of respondents who rate the health plan an 8, 9, or 10 in response to Q36 "Using any number from 0 to 10, where 0 is the worst health plan possible and 10 is the best health plan possible, 85.8% NA what number would you use to rate your child's health plan?" NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. AMERIGROUP initiated a new Member Satisfaction PIP in 20122013 as part of its DCH contract requirements. The study indicator, based on Question 36 of AMERIGROUP's 2012 CAHPS Child Medicaid Member Survey, assessed the overall rating parents/guardians selected for the CMO, as their child's health plan, with "0" being the lowest rating and "10" being the highest possible rating. The baseline rate of respondents giving AMERIGROUP a rating of "8" or higher was 85.8 percent, slightly lower than the CMO's baseline goal of 86.6 percent, which was derived from the national 2012 Child Medicaid Quality Compass 75th percentile benchmark. AMERIGROUP's results showed that the CMO performed better than 50 percent of all CMOs reporting rates but not higher than 75 percent of all CMOs. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-16 Health Services Advisory Group, Inc. FINDINGS Provider Satisfaction Table 2-11--Performance Improvement Project Outcomes for Provider Satisfaction PIP Study Indicator Baseline Period Sustained (8/1/1211/30/12) Improvement^ The percentage of providers who respond, "Very satisfied" or, "Somewhat satisfied" to Q48 "Please rate your overall satisfaction 79.6% NA with Amerigroup." NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. AMERIGROUP collected baseline data for a new Provider Satisfaction PIP in CY 2012. The study indicator from the CMO's 2012 provider satisfaction survey assessed providers' overall satisfaction. The baseline rate of providers who reported being "Somewhat satisfied" or "Very satisfied" with AMERIGROUP was 79.6 percent. The CMO stated in the PIP Summary Form that its goal was to increase the baseline rate by 5 percent; therefore, the goal is for 83.6 percent of providers to report being "Somewhat satisfied" or "Very satisfied" with AMERIGROUP at Remeasurement 1. Although the CMO implemented interventions for both its Member and Provider Satisfaction PIPs, the PIPs were validated through Activity VII because only baseline data were reported. As these PIPs progress to reporting Remeasurement 1 data, HSAG will evaluate the CMO's causal/barrier analysis process and interventions. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 2-17 Health Services Advisory Group, Inc. 3. CONCLUSIONS AND RECOMMENDATIONS for AMERIGROUP Community Care Conclusions AMERIGROUP's performance suggests a thorough application of the PIP Design stage (Activities I through VI). The sound study design for eight of the nine PIPs created the foundation for AMERIGROUP to progress to subsequent PIP stages--implementing improvement strategies and achieving real and sustained study indicator outcomes. With the exception of the collaborative Avoidable Emergency Room Visits PIP, AMERIGROUP exhibited sound study design for eight of its PIPs, it only achieved real and sustained improvement for two of the seven PIPs that progressed to these activities. A critical analysis of the CMO's improvement strategies and processes identified that AMERIGROUP completed a causal/barrier analysis for each PIP; however, the CMO did not provide a data-driven rationale for all of the implemented interventions. There were interventions implemented that did not directly impact the study indicator outcomes, and the CMO lacked a specific evaluation plan for most of its interventions. Without an evaluation plan, the CMO cannot determine whether to modify or discontinue existing interventions, or implement new strategies, thereby reducing the likelihood of achieving the desired goals and improving performance. Recommendations HSAG recommends that AMERIGROUP: Reference the PIP Completion Instructions to ensure that all documentation requirements for each activity have been addressed. Ensure that all data components reported in each PIP are accurate and consistently documented throughout the PIP, and align with the data that have been reported in its final audit report. Ensure that all statistical testing is done correctly and the documentation of the statistical testing outcomes is accurate and consistent throughout the PIP. Conduct an annual causal/barrier analysis including drill-down analysis along with additional quarterly analyses of its outcome data. The CMO must accurately document the analyses, providing the results, identified barriers, and the rationale for how barriers are prioritized. Ensure that the interventions implemented to address a specific barrier are directly linked to that barrier and will directly impact study outcomes. Have a process in place, for any intervention implemented, to evaluate the efficacy of the intervention to determine if it is having the desired effect. The results of each intervention's evaluation for each remeasurement period should be included in the PIP. If the interventions are not having the desired effect, AMERIGROUP should discuss how it will address these deficiencies and what changes will be made to its improvement strategies. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 3-1 Health Services Advisory Group, Inc. CONCLUSIONS AND RECOMMENDATIONS HSAG will work with DCH to create a PIP Summary Form template that is specific to the collaborative Avoidable Emergency Room Visits PIP. AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page 3-2 Health Services Advisory Group, Inc. APPENDIX A. PIP-SPECIFIC VALIDATION SCORES for AMERIGROUP Community Care Table A-1--AMERIGROUP Community Care's SFY 2014 PIP Performance Percentage of Applicable Evaluation Elements Scored Met Study Stage Activity Adolescent Well-Care Design Implementation Outcomes Appropriate Study Topic Clearly Defined, Answerable Study Question(s) Correctly Identified Study Population Clearly Defined Study Indicator(s) Valid Sampling Techniques (if sampling was used) Accurate/ Complete Data Collection Design Total Sufficient Data Analysis and Interpretation Appropriate Improvement Strategies Implementation Total Real Improvement Achieved Sustained Improvement Achieved Outcomes Total 100% 100% 100% 100% 100% 90% 97% 78% 67% 75% 50% Not Assessed 50% Validation Status Not Met Annual Dental Visits 100% 100% 100% 100% Not Applicable 100% 100% 88% 100% 92% 100% 100% 100% Partially Met Appropriate Use of ADHD Medications 100% Childhood Immunizations --Combo 10 100% Childhood Obesity 100% Comprehensive Diabetes Care 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% Not Applicable 100% 100% 80% 95% 63% 90% 97% 89% 100% 100% 56% 67% 64% 25% Not Assessed 25% Not Met 100% 92% 100% 33% 50% 50% Not Assessed 0% 100% Met 40% Partially Met 100% 100% 100% 91% 97% 89% 67% 83% 25% Not Assessed 25% Not Met Avoidable Emergency Room Visits 100% 100% 0% 71% Not Applicable 30% 59% 75% 100% 82% 25% Not Assessed 25% Not Met Member Provider Satisfaction Satisfaction 80% 75% 100% 100% 100% 100% 100% 100% 100% 100% 86% 93% 80% Not Assessed 80% Not Assessed Not Assessed Not Assessed Met 88% 93% 80% Not Assessed 80% Not Assessed Not Assessed Not Assessed Met AMERIGROUP Community Care SFY 2014 PIP Validation Report State of Georgia AMERIGROUP_GAFY2014_CMO_PIP-Val_Report_F3_1113 Page A-1 Health Services Advisory Group, Inc.