State of Georgia Department of Community Health Georgia Families Program WellCare of Georgia, Inc. PERFORMANCE IMPROVEMENT PROJECTS REPORT SFY 2014 November 2013 TABLE OF CONTENTS 1.Background ....................................................................................................................... 1-1 CMO Overview ............................................................................................................................... 1-3 Study Rationale .............................................................................................................................. 1-3 Study Summary .............................................................................................................................. 1-3 Validation Overview ........................................................................................................................ 1-5 HSAG's Validation Scoring Methodology ........................................................................................ 1-7 2.Findings ............................................................................................................................. 2-1 Aggregate Validation Findings ........................................................................................................ 2-1 Design......................................................................................................................................... 2-3 Implementation ........................................................................................................................... 2-3 Outcomes ................................................................................................................................... 2-3 PIP-Specific Outcomes ................................................................................................................... 2-4 Analysis of Results...................................................................................................................... 2-4 Adolescent Well-Care ................................................................................................................. 2-4 Annual Dental Visits .................................................................................................................... 2-6 Appropriate Use of ADHD Medications ....................................................................................... 2-7 Childhood Immunizations--Combo 10 ........................................................................................ 2-9 Childhood Obesity..................................................................................................................... 2-10 Comprehensive Diabetes Care ................................................................................................. 2-12 Avoidable Emergency Room Visits ........................................................................................... 2-14 Member Satisfaction ................................................................................................................. 2-17 Provider Satisfaction ................................................................................................................. 2-17 3.Conclusions and Recommendations .............................................................................. 3-1 Conclusions .................................................................................................................................... 3-1 Recommendations .......................................................................................................................... 3-1 Appendix A. PIP-Specific Validation Scores ......................................................................A-1 WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Health Services Advisory Group, Inc. Page i ACKNOWLEDGMENTS AND COPYRIGHTS CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA). WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Health Services Advisory Group, Inc. Page ii Performance Improvement Project Validation Report WellCare of Georgia, Inc. 1. BACKGROUND The Georgia Department of Community Health (DCH) through its Division of Medical Assistance Plans is responsible for administering the Medicaid program and the Children's Health Insurance Program (CHIP) for the State of Georgia and overseeing quality improvement activities. The State refers to its Medicaid managed care program as Georgia Families and to its CHIP program as PeachCare for Kids. For the purposes of this report, "Georgia Families" refers to all Medicaid and PeachCare for Kids members enrolled in managed care. The Georgia Families program serves the majority of Georgia's Medicaid and CHIP populations. The DCH requires its contracted Care Management Organizations (CMOs) serving members under Georgia Families to conduct performance improvement projects (PIPs) as set forth in 42 CFR 438.240 to assess and improve the quality of a targeted area of clinical or nonclinical care or service provided to members, and to report the status and results of each PIP annually. WellCare of Georgia, Inc. (WellCare) is one of the Georgia Families CMOs. The validation of PIPs is one of three federally-mandated activities for state Medicaid managed care programs. The other two required activities include the evaluation of CMO compliance with State and federal regulations and the validation of CMO performance measures. These three mandatory activities work together to assess the CMOs' performance with providing appropriate access to high-quality care for their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measure rates provide a barometer of the quality and effectiveness of the care. The DCH requires the CMOs to initiate PIPs to improve the quality of health care in targeted areas of low performance, or in areas identified as State priorities or health care issues of greatest concern. The DCH required its CMOs to conduct nine PIP studies during the 2012 calendar year and submit them for validation in 2013. PIPs are key tools in helping DCH achieve the goals and objectives outlined in its quality strategy; they provide the framework for monitoring, measuring, and improving the delivery of health care. The primary objective of PIP validation is to determine each CMO's compliance with requirements set forth in 42 CFR 438.240(b)(1), including: Measurement of performance using objective quality indicators. Implementation of system interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities to increase or sustain improvement. To meet the federal requirement for the validation of PIPs, DCH contracted with Health Services Advisory Group, Inc. (HSAG), the State's external quality review organization (EQRO), to conduct the validation of WellCare's PIPs. WellCare submitted PIPs to HSAG between June 30, WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-1 Health Services Advisory Group, Inc. BACKGROUND 2013, and August 1, 2013, and HSAG validated the PIPs between July 1, 2013, and August 8, 2013. The validated data represent varying measurement time periods as described in Table 2-3 through Table 2-11. For PIPs initiated prior to January 1, 2012 (Annual Dental Visits and Childhood Obesity), HSAG reviewed the PIPs using the Centers for Medicare & Medicaid Services (CMS) validation protocols.1-1 For PIPs initiated on or after January 1, 2012 (Adolescent Well-Care Visits, Appropriate Use of ADHD Medications, Avoidable Emergency Room Visits [Collaborative], Childhood Immunizations--Combo 10, Comprehensive Diabetes Care, Member Satisfaction and Provider Satisfaction), HSAG used CMS' updated validation protocols.1-2 Compared to the 2002 CMS PIP protocols, the only changes made to the 2012 protocols were reversing the order of Activities III and IV, and Activities VII and VIII. These changes did not impact HSAG's validation process. Table 1-1--CMS Protocol Changes PIP Activity Clearly Defined Study Indicator(s) Correctly Identified Study Population Appropriate Improvement Strategies Sufficient Data Analysis and Interpretation of Results CMS 2002 Protocol Activity III Activity IV Activity VII Activity VIII CMS 2012 Protocol Activity IV Activity III Activity VIII Activity VII HSAG evaluated two key components of the quality improvement process: 1. HSAG evaluated the technical structure of the PIPs to ensure WellCare designed, conducted, and reported PIPs using sound methodology consistent with the CMS protocol for conducting PIPs. HSAG's review determined whether a PIP could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring real and sustained improvement. 2. HSAG evaluated the outcomes of the PIPs. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. Evaluation of each PIP's outcomes determined whether WellCare improved its rates through the implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results) and, through these processes, achieved statistically significant improvement over the baseline rate. Once statistically significant improvement is achieved across all study indicators, HSAG evaluates whether WellCare was successful in sustaining the improvement. A primary goal of HSAG's PIP validation is to ensure that DCH and key stakeholders can have confidence that reported improvement in study indicator outcomes is supported by statistically significant change and the CMOs improvement strategies. 1-1 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Managed Care Organization Protocol. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, Final Protocol, Version 1.0, May 2002. 1-2 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Protocol 3: Validating Performance Improvement Projects (PIPs): A Mandatory Protocol for External Quality Review (EQR), Version 2.0, September 2012. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-2 Health Services Advisory Group, Inc. BACKGROUND CMO Overview The DCH contracted with WellCare beginning in 2006 to provide services to the Georgia Families program population. Since implementation of the Georgia Families program, WellCare has served the eligible population in all geographic regions of Georgia--Atlanta, Central, East, North, Southeast, and Southwest. The HEDIS technical specifications that WellCare used for its PIP indicators require a member to be continuously enrolled with the CMO. While the new population was included in the PIPs' interventions, the measurement of the PIPs' effectiveness (i.e., the indicator results) excluded members who did not meet the indicators' continuous enrollment criteria. Study Rationale The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas. Although HSAG has validated WellCare's PIPs for six years, the number of PIPs, study topics, and study methods has evolved over time. WellCare submitted nine PIPs for validation. The PIP topics include: Adolescent Well-Care Visits Annual Dental Visits Appropriate Use of ADHD Medications Avoidable Emergency Room Visits (Collaborative) Childhood Immunizations--Combo 10 Childhood Obesity Comprehensive Diabetes Care Member Satisfaction Provider Satisfaction Study Summary As noted in its Quality Strategic Plan Update (November 2011), DCH identified the improvement and enhancement of the quality of patient care provided through ongoing, objective, and systematic measurement, analysis, and improvement of performance as one of its four performance-driven goals. The goals are designed to demonstrate success or identify challenges in achieving intended outcomes related to providing quality, accessible, and timely services. WellCare's June 30, 2013, through August 1, 2013, PIP submissions included six clinical HEDIS-based PIPs: Adolescent Well-Care Visits, Annual Dental Visits, Appropriate Use of ADHD Medications, Avoidable Emergency Room Visits, Childhood Immunizations--Combo 10, Childhood Obesity, Comprehensive Diabetes Care, two nonclinical PIPs: Member Satisfaction and Provider Satisfaction, and one collaborative Avoidable Emergency Room Visits PIP. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-3 Health Services Advisory Group, Inc. BACKGROUND Table 1-2 outlines the key study indicators incorporated for the six clinical HEDIS-based PIPs. Table 1-2--PIP Study Topics and Indicator Descriptions PIP Study Topic Adolescent WellCare Visits Annual Dental Visits Appropriate Use of ADHD Medications Childhood Immunizations-- Combo 10 Childhood Obesity Comprehensive Diabetes Care PIP Study Indicator Description The percentage of members 1221 years of age who had at least one comprehensive well-care visit with a PCP or an OB/GYN practitioner during the measurement year. The percentage of members 23 years of age and 221 years of age who had at least one dental visit during the measurement year. 1. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who had one follow-up visit with a practitioner with prescribing authority during the 30-day Initiation Phase. 2. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who remained on the medication for at least 210 days and who, in addition to the visit in the Initiation Phase, had at least two follow-up visits with a practitioner from 31300 days following the IPSD. One of the two visits (during days 31300) may be a telephone visit with a practitioner. The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis (DTaP); three polio (IPV); one measles, mumps, rubella (MMR); three H influenza type B (HiB); three hepatitis B (HepB); one chicken pox (VZV); four pneumococcal conjugate (PCV); one hepatitis A (HepA); two or three rotavirus (RV); and two influenza (flu) vaccines by their second birthday. The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation, nutrition counseling and physical activity counseling. The percentage of members 18-75 years of age with diabetes (type 1 and type 2) who had a HbA1c control < 7.0%, LDL-C control < 100mg/ml, and BP control < 140/90 mmHg. Table 1-3 outlines the key study indicators for the collaborative Avoidable Emergency Room Visits PIP. Table 1-3--Collaborative PIP Study Topic and Indicator Descriptions PIP Study Topic Avoidable Emergency Room Visits PIP Study Indicator Description 1. The percentage of practices that provide the same day appointments for routine and urgent care. 2. The percentage of practices that provide routine and urgent care appointment after hours. 3. The percentage of practices that provide appointments for routine and urgent care after hours and have the ability to document after hours clinical advice in the patient's record. 4. The percentage of practices that have access to and utilize electronic health records. 5. The percentage of practices that receive information regarding ER visits from the study hospitals 6. The percentage of ER visits for `avoidable' diagnoses (dx382Acute Suppurative otitis: 382.9Unspecified otitis: 462Acute pharyngitis:465.9Acute upper respiratory infection: 466Acute bronchitis:786.2Cough) among members under 21 years of age who had a visit to the ED in three selected Children's Healthcare of Atlanta facilities in the Atlanta region. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-4 Health Services Advisory Group, Inc. BACKGROUND Table 1-4 outlines the key study indicators incorporated for the two satisfaction-based PIPs. The effectiveness of the Member Satisfaction PIP was measured using the Consumer Assessment of Healthcare Providers and Systems (CAHPS) 5.0H, Medicaid Child Survey. This survey provided information on parents' experiences with their child's provider and CMO. The final WellCare PIP topic was Provider Satisfaction. WellCare contracted with a vendor to produce and administer a survey to document the effectiveness of this performance improvement project. Survey Type Member Provider Table 1-4--Satisfaction-Based PIP Study Indicators Question #36 #42 Survey Question The percentage of respondents who rate the health plan an 8, 9, or 10 to the question "Using any number from 010, where 0 is the worst health plan and 10 is the best, what number would you use to rate your child's health plan?" The percentage of providers who respond "very satisfied" or "somewhat satisfied" to question "Please rate your overall satisfaction with WellCare of Georgia." Validation Overview HSAG obtained the data needed to conduct the PIP validation from WellCare's PIP Summary Forms. These forms provided detailed information about WellCare's completed PIP activities. Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be scored Met. Given the importance of critical elements to the scoring methodology, any critical element that received a Not Met score resulted in an overall validation status for the PIP of Not Met. A CMO would be given a Partially Met validation status if 60 percent to 79 percent of all evaluation elements were scored Met or one or more critical elements were scored Partially Met. HSAG provided a Point of Clarification when the CMO fully met the evaluation element criteria and only minor documentation edits not critical to the validity of the PIP were recommended to the CMO. In addition to the overall validation status (e.g., Met) HSAG provided an overall percentage for all evaluation elements (including critical elements) scored Met. HSAG calculated the overall percentage by dividing the total number of elements scored Met by the total number of elements scored Met, Partially Met, and Not Met. HSAG also calculated a critical element overall percentage by dividing the total number of critical elements scored Met by the sum of the critical elements scored Met, Partially Met, and Not Met. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-5 Health Services Advisory Group, Inc. BACKGROUND Figure 1-1 illustrates the three study stages of the PIP process: Design, Implementation, and Outcomes. The Design stage establishes the methodological framework for the PIP. The activities in this stage include development and documentation of the study topic, question, indicators, population, sampling, and data collection. A sound study design is necessary for the successful implementation of improvement strategies. Once the study design is established, the PIP process moves into the Implementation stage. This stage includes data analysis and implementation of improvement strategies. During the Implementation stage, CMOs should incorporate a continuous or rapid cycle improvement model such as the Plan-Do-Study-Act (PDSA) Cycle. Figure 1-1--PIP Study Stages Incorporating the PDSA Cycle Outcomes Design The PDSA cycle includes the following actions: Plan--conduct barrier analyses; prioritize barriers; develop targeted intervention(s) to address barriers; and develop an intervention evaluation plan for each intervention Do--implement intervention; track and monitor the intervention; and record the data Study--analyze the data; compare results; and evaluate the intervention's effectiveness Act--based on the evaluation results, standardize, modify, or discontinue the intervention The PDSA cycle is repeated throughout each measurement period. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final Outcomes stage evaluates for statistically significant and sustained improvement of the project outcomes. Once statistically significant improvement in the outcomes is achieved, the improvement must be sustained in a subsequent measurement period. If the study outcomes do not improve, the CMO's responsibility is to continue the PDSA cycle until statistically significant improvement is achieved and sustained. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-6 Health Services Advisory Group, Inc. BACKGROUND HSAG's Validation Scoring Methodology The scoring methodology evaluates whether or not the CMOs met all the documentation requirements according to the CMS protocols, as well as evaluates whether or not all study indicators have achieved statistically significant improvement over the baseline rate. In Activity IX (real improvement achieved), the CMO must achieve statistically significant improvement across all study indicator(s) between the baseline and a subsequent measurement period to receive a Met score. For Activity X (sustained improvement achieved), HSAG assesses for sustained improvement once all study indicators achieve statistically significant improvement over the baseline and the CMO reports a subsequent measurement period. All study indicators must achieve statistically significant improvement and sustain this improvement to receive a Met validation score in Activity X. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-7 Health Services Advisory Group, Inc. 2. FINDINGS for WellCare of Georgia, Inc. Aggregate Validation Findings HSAG organized, aggregated, and analyzed WellCare's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of interventions. Based on its review, HSAG determined the overall methodological validity of the PIPs, as well as the overall success in achieving improved study indicator outcomes. The results are presented in Table 2-1. Table 2-1--Performance Improvement Project Validation Scores for WellCare of Georgia, Inc. PIP Adolescent Well-Care Visits Annual Dental Visits Appropriate Use of ADHD Medication Avoidable Emergency Room Visits--Collaborative Childhood Immunization-- Combo 10 Childhood Obesity Comprehensive Diabetes Care Member Satisfaction Provider Satisfaction Percentage of Evaluation Elements Scored Met 98% 100% 78% 62% 90% 88% 86% 91% 94% Percentage of Critical Elements Scored Met 100% 100% 91% 50% 93% 93% 93% 100% 100% Validation Status Met Met Not Met Not Met Partially Met Partially Met Not Met Met Met Four of the nine PIPs validated received an overall Met validation status. Although the CMO reported the correct baseline numerator and denominator for the Childhood Immunizations-- Combo 10 PIP, the PIP received a Partially Met validation status because the CMO reported the incorrect baseline rate. The Childhood Obesity PIP received a Partially Met validation status because only two of three study indicators achieved statistically significant improvement over the baseline. The Appropriate Use of ADHD Medication, Avoidable Emergency Room Visits, and Comprehensive Diabetes Care PIPs received an overall Not Met validation status. Both the Appropriate Use of ADHD Medication and Comprehensive Diabetes Care PIPs did not achieve statistically significant improvement for any of the study indicators. The collaborative Avoidable Emergency Room Visits PIP received a Not Met validation status for several reasons. The CMO did not completely define the study population or the study indicators, or explain how the data were collected for all study indicators. Additionally, not all study indicators achieved statistically significant improvement over the baseline rates. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-1 Health Services Advisory Group, Inc. FINDINGS Table 2-2 displays the combined validation results for all nine WellCare PIPs validated during SFY 2014. This table illustrates the CMO's application of the PIP process and its success in implementing all nine projects. Each activity was composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score satisfied the necessary technical requirements for a specific element. The validation results presented in Table 2-2 show the percentage of applicable evaluation elements that received a Met score by activity. Additionally, HSAG calculated an overall percentage of Met scores across all activities for all nine PIPs. Appendix A provides the detailed scores from the validation tool for each of the nine PIPs. Table 2-2--Performance Improvement Project Validation Results for WellCare of Georgia, Inc. (N=9 PIPs) Study Stage Activity Appropriate Study Topic Clearly Defined, Answerable Study Question(s) Design Clearly Defined Study Indicator(s) Correctly Identified Study Population Valid Sampling Techniques (if sampling was used) Accurate/Complete Data Collection Design Total* Implementation Sufficient Data Analysis and Interpretation of Results Appropriate Improvement Strategies Implementation Total* Outcomes Real Improvement Achieved Sustained Improvement Achieved Outcomes Total Percentage of Applicable Evaluation Elements Scored Met * Percentages do not total 100 percent due to rounding. Percentage of Applicable Elements Met Partially Met Not Met 92% (47/51) 0% (0/51) 8% (4/51) 100% (18/18) 0% (0/18) 0% (0/18) 96% 4% 0% 53/55 2/55 0/55 92% 8% 0% 24/26 2/26 0/26 97% 0% 3% 35/36 0/36 1/36 83% 8% 9% 64/77 6/77 7/77 92% 241/263 4% 10/263 5% 12/263 89% 11% 0% 62/70 8/70 0/70 76% 20% 4% 19/25 5/25 1/25 85% 14% 1% 81/95 13/95 1/95 57% 18% 25% 16/28 5/28 7/28 100% 0% 0% 1/1 0/1 0/1 59% 17/29 17% 5/29 24% 7/29 88% (339/387) WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-2 Health Services Advisory Group, Inc. FINDINGS Overall, 88 percent of the evaluation elements across all nine PIPs received a Met score. WellCare demonstrated a strong performance in the Design stage, with the exception of its Avoidable Emergency Room Visits PIP; however, the CMO was less successful in the Implementation and Outcomes stages. The following subsections highlight HSAG's validation findings associated with each of the three PIP stages. Design WellCare met 92 percent of the requirements across all nine PIPs for the six activities within the Design stage. With the exception of the Avoidable Emergency Room Visits PIP, the technical design of each PIP was sufficient to measure and monitor PIP outcomes. The solid foundation of the PIPs allowed for the CMO to progress to the next stage of the PIP process. Implementation WellCare met 85 percent of the requirements for the two activities within the Implementation stage. The CMO did not report accurate data components in some of its PIPs, and not all of the statistical testing performed was completely accurate. Additionally, the CMO did not successfully link interventions to the identified barriers; not all interventions implemented directly impacted indicator outcomes; and the CMO lacked sufficient processes to evaluate the efficacy of its interventions. Overall, the improvement strategies were not successful in achieving statistically significant improvement across all study indicators for all PIPs and for sustaining any improvement achieved. Outcomes This year, five PIPs (Adolescent Well-Care Visits, Appropriate Use of ADHD Medications, Childhood Immunizations--Combo 10, Comprehensive Diabetes Care, and Avoidable Emergency Room Visits) were evaluated for achieving statistically significant improvement. Two PIPs, Adolescent Well-Care Visits and Childhood Immunizations--Combo 10 achieved statistically significant improvement over baseline at Remeasurement 1. The Annual Dental Visits PIP progressed to the point of being assessed for sustained improvement. Sustained improvement is defined as statistically significant improvement in performance over baseline that is maintained or increased for at least one subsequent measurement period. Additionally, the results of the most current measurement period must reflect improvement when compared to baseline results. Both study indicators in the Annual Dental Visits PIP achieved sustained improvement. The Childhood Obesity PIP has yet to achieve statistically significant improvement across all study indicators and could not be evaluated for sustained improvement. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-3 Health Services Advisory Group, Inc. FINDINGS PIP-Specific Outcomes Analysis of Results Each table below displays the study indicator rates for each measurement period of the PIP, including the baseline period and each subsequent remeasurement period. Statistically significant changes between remeasurement periods are noted with an upward or downward arrow followed by an asterisk. If the PIP achieved statistically significant improvement over the baseline rate, it was then reviewed for sustained improvement. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. PIPs that did not achieve statistically significant improvement (i.e., did not meet the criteria to be assessed for sustained improvement) were not assessed (NA). Comparisons of PIP study indicator results that utilized HEDIS measures were made using the Medicaid HEDIS 2011 Audit, Means, Percentiles, and Ratios (reflecting the 2010 calendar year [CY]). WellCare was not successful in achieving the desired outcomes for all study indicators. The CMO either did not demonstrate improvement or it could not be determined whether the improvement was due to the implementation of the CMO's improvement strategy or due to chance. The identification of barriers through barrier analysis and the subsequent selection of appropriate interventions to address those barriers are necessary steps to improve outcomes. WellCare's choice of interventions, the combination of intervention types, and the sequence of intervention implementation are all essential to the overall success of the performance improvement projects. Deficiencies were identified during the validation process in each of these areas and will be explained in further detail below. The following section discusses the improvement strategies the CMO implemented in conjunction with the PIPs' study indicator results. Comparisons of PIP study indicator results that utilized HEDIS measures were made using the Medicaid HEDIS 2011 Audit, Means, Percentiles, and Ratios (reflecting the 2010 calendar year [CY]). Adolescent Well-Care Table 2-3--Performance Improvement Project Outcomes for Adolescent Well-Care Visits PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of members 1221 years of age who had at least one comprehensive well-care visit with a PCP or an OB/GYN practitioner during the measurement year. 41.4% 51.6%* NA NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-4 Health Services Advisory Group, Inc. FINDINGS WellCare achieved statistically significant improvement at Remeasurement 1 in the Adolescent Well-Care Visits PIP, with an increase of 10.2 percentage points over the baseline rate. The CMO's CY 2012 rate of eligible adolescent members who had at least one well-care visit during the measurement year exceeded the DCH target rate of 46.8 percent and was between the 50th and 75th percentiles of the national Medicaid HEDIS 2011 rates. WellCare's Utilization Management Medical Advisory Committee (UMAC) and Quality Improvement Committee (QIC) met quarterly to identify and address barriers. These teams used brainstorming and completed a Force Field Analysis to assist them with the development of improvement strategies to target identified barriers. WellCare documented that its UMAC evaluated the CMO's data continuously to monitor the effectiveness of improvement activities. For the Adolescent Well-Care Visits PIP, the CMO focused on member and provider interventions. The following were interventions implemented by the CMO: Member telephone outreach to educate members on the importance of adolescent well-care visits and schedule appointments. Community Outreach and Field Short Term Case Management Program--Outreach to educate members and identify any needs members had regarding their health. This outreach reinforced the need for members to make well-care appointments that addressed early and periodic screening, diagnostic, and treatment (EPSDT) services. Targeted Health Check schedule reminder letters sent at 120 days of plan enrollment and during the member's birthday month. Monthly provider membership lists that specified children eligible for health check visits who had not had an encounter within 120 days of joining the health plan or were not in compliance with the Health Check Program. Provider Pay for Performance Incentive. Although the study indicator achieved statistically significant improvement and the CMO monitored its data continuously, WellCare did not have processes in place to evaluate the effectiveness of each implemented intervention. For example, the CMO did not track those members who received outreach calls or education, or were in the short term case management program to see if they had an encounter for an adolescent well-care visit as a result of the intervention. As a consequence of not evaluating the interventions, the CMO did not have information to determine which of the interventions implemented were successful, making it difficult to eliminate any ineffective interventions. Continuing to implement ineffective interventions prevents the CMO from redirecting efforts and resources that could be used for other areas in need of improvement or expanding and sustaining effective interventions. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-5 Health Services Advisory Group, Inc. FINDINGS Annual Dental Visits Table 2-4--Performance Improvement Project Outcomes for Annual Dental Visits PIP Study Indicator The percentage of members 23 years of age who had at least one dental visit. Baseline Period Remeasurement 1 (1/1/0912/31/09) (1/1/1012/31/10) 40.4% 45.5%* Remeasurement 2 (1/1/1112/31/11) 50.0%* Remeasurement 3 Sustained (1/1/1212/31/12) Improvement^ 52.2%* Yes The percentage of members 221 years of age who had at least one dental visit. 65.2% 67.5%* 70.5%* 71.5%* Yes * Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. At Remeasurement 3 for Annual Dental Visits, WellCare sustained significant improvement over baseline for both study indicators. Both indicators also increased significantly over the previous year's rates. The CMO's CY 2012 rates for members 23 years of age and 221 years of age who had at least one dental visit during the measurement year were better than the national Medicaid HEDIS 2011 90th percentiles of 46.9 percent and 64.5 percent, respectively. The rate for members 221 years of age also surpassed the DCH CY 2012 target of 64.1 percent by 7.4 percentage points. WellCare's UMAC and QIC used brainstorming, completed a Force Field Analysis, and worked collaboratively with the plan's dental vendor, DentaQuest, to address barriers and implement targeted interventions. The CMO documented that interventions were designed to change behavior at the practitioner level by focusing the attention on the non-compliant members and at the member level by increasing member knowledge regarding the importance of dental care, scheduling appointments, and reminding members of when services were due. The following were interventions implemented by the CMO: In January 2012, DentaQuest conducted a targeted provider mailing which included the GA WellCare Preventistry Sealant Program Letter and member listing. Targeted 120-Day Provider Reminder letters with a list of non-compliant members. Targeted dental missed appointment letters were sent to members who had not had a dental service in the prior six months. Targeted 120-Day Member Reminder letters. Targeted Periodicity letters sent to members annually. Inbound Care Gap Program: At the time the member called in, the customer service representative identified whether the member had a dental or other HEDIS measure service needed. If so, the representative advised the member of the needed service(s) and assisted them with scheduling the appointments. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-6 Health Services Advisory Group, Inc. FINDINGS Although both study indicators achieved statistically significant and sustained improvement, it was unclear to HSAG how the Preventistry Sealant Program and mailed letters for this program would increase the percentage of members receiving an annual dental exam. HSAG recommends that WellCare implements processes to evaluate the effectiveness of each implemented intervention. For example, for the targeted missed dental appointment letters, how many of those members who were mailed a letter, had an encounter for a dental visit following the mailing of the letter? Without an evaluation process in place, the CMO cannot determine if an intervention was successful. Appropriate Use of ADHD Medications Table 2-5--Performance Improvement Project Outcomes for Appropriate Use of ADHD Medications PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ 1. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who had 40.0% 39.4% NA one follow-up visit with a practitioner with prescribing authority during the 30-day Initiation Phase. 2. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who remained on the medication for at least 210 days and who, in addition to the visit in the Initiation Phase, had at 54.6% 53.1% NA least two follow-up visits with a practitioner from 31 300 days following the IPSD. One of the two visits (during days 31300) may be a telephone visit with a practitioner. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. At Remeasurement 1 in the Appropriate Use of ADHD Medications PIP, WellCare did not achieve statistically significant improvement over baseline. The CY 2012 rates of ADHD followup visits for the initiation phase (Study Indicator 1) and the continuation phase (Study Indicator 2) were lower than the respective baseline rates, though neither declined significantly. The CMO's CY 2012 rates of follow-up visits did not meet the corresponding DCH target rates of 48.1 percent (initiation) and 57.6 percent (continuation and maintenance). Compared to the national Medicaid HEDIS 2011 rates, the rate for the initiation phase was slightly above the 50th percentile of 38.3 percent, and the rate for the continuation phase was slightly better than the 75th percentile of 52.6 percent. For the Appropriate Use of ADHD Medication PIP, WellCare documented the following barriers: member lack of education over giving medications on holidays/weekends, member unaware of needing follow-up appointment, providers writing prescriptions without seeing the WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-7 Health Services Advisory Group, Inc. FINDINGS member, providers unaware that members were not returning for follow-up care, providers unaware of best practices, and providers unaware of the requirements for the HEDIS measure. To address these identified barriers, the CMO implemented the interventions listed below: Providers received a "Best Practice" flyer identifying the need to educate members on the importance of continuation of medication. Best Practice flyer sent to PCPs and psychiatrists stressing the importance of the visit, as well as education on the HEDIS measure. Provider visits from pharmacy, public relations, and quality department staff to provide education on HEDIS measures and the importance of the visit. Provider newsletter stressing the importance of the visit. Targeted provider faxing to ensure members with newly prescribed medication were scheduled for a visit. Targeted member mailing reminding members to schedule a follow-up visit. WellCare hired a Licensed Master Social Worker to focus on behavioral health initiatives with an emphasis on ADHD. Targeted member mailings reminding members of follow-up appointments. Not all listed interventions addressed the barriers documented by the CMO. It was unclear from the CMO's documentation which intervention(s) addressed the barrier of member lack of education over giving medications on holidays/weekends or providers writing prescriptions without seeing the member at a follow-up visit. The CMO also documented that it would be continuing all interventions. HSAG encourages WellCare to provide a more detailed description of how the barriers listed in the PIP and in the attachment were identified, the process of how the CMO prioritized its barriers, and how the barriers were linked to the interventions. WellCare must ensure that the interventions implemented logically link to the barrier and can directly impact the study indicator outcomes. WellCare also needs to have processes in place to evaluate the effectiveness of each implemented intervention, and must investigate the reasons for the decline in performance and based on the findings, develop new improvement strategies. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-8 Health Services Advisory Group, Inc. FINDINGS Childhood Immunizations--Combo 10 Table 2-6--Performance Improvement Project Outcomes for Childhood Immunizations--Combo 10 PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis (DTap); three polio (IPV); one measles, mumps, rubella (MMR); three H influenza type B (HiB); three hepatitis B (HepB); one chicken 20.2% 38.4%* NA pox (VZV); four pneumococcal conjugate (PCV); one hepatitis A (HepA); two or three rotavirus (RV); and two influenza (flu) vaccines by their second birthday. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. WellCare demonstrated significant improvement in the Childhood Immunizations--Combo 10 PIP, with an increase of 18.2 percentage points from baseline to Remeasurement 1 in the rate of eligible child members who received the recommended vaccinations by their second birthday. The CMO's rate also exceeded the national Medicaid HEDIS 2011 90th percentile of 23.6 percent. WellCare's UMAC and QIC identified barriers and developed member, provider, and plan-level interventions through data analysis and process review. The CMO documented barriers such as members refusing assistance with appointments, member lack of education regarding preventive screenings, provider unaware of non-compliant eligible members, and provider unaware of HEDIS requirements. To address these barriers, WellCare implemented the following interventions: Customer Service team incentivized $5 per appointment process. Outbound member reminder calls. Centralized telephonic outreach program. Targeted 120-Day Member Reminder letters. Targeted Periodicity letters sent to members annually. Monthly member non-compliant list to providers. Targeted 120-Day Provider Reminder letters with a list of non-compliant members. HEDIS Toolkits distributed during Pay-for-Performance visits. Although the study indicator achieved statistically significant improvement, HSAG identified that not all listed interventions addressed the barriers documented by the CMO. It was unclear from the CMO's documentation which intervention(s) addressed the barrier of members refusing WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-9 Health Services Advisory Group, Inc. FINDINGS assistance with appointments. The CMO did not link all of its interventions to identified barriers. As stated previously, WellCare did not have processes in place to evaluate the effectiveness of each intervention. HSAG encourages the CMO to track the members who were mailed the Targeted 120-Day Member Reminder letter to see if any of these members had an encounter for the necessary services after receiving the letter. Childhood Obesity Table 2-7--Performance Improvement Project Outcomes For Childhood Obesity PIP Study Indicator Baseline Period Remeasurement 1 Remeasurement 2 Remeasurement 3 Sustained (1/1/0912/31/09) (1/1/1012/31/10) (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation. 36.5% 30.4% 56.9%* 38.7%* NA The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of counseling for nutrition. 42.3% 48.9% 50.4%* 55.5% NA The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of counseling for physical activity. 38.7% 30.9%* 37.0% 42.1% NA NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). * Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. One of the three study indicators for the Childhood Obesity PIP, evidence of counseling for nutrition, continued its year-over-year improvement at Remeasurement 3 and maintained a significant increase from the baseline rate. The Remeasurement 3 rates for the remaining study indicators, BMI percentile documentation and evidence of counseling for physical activity, are not significantly better than their respective baseline rates, with the rate of BMI documentation 18.2 percentage points lower than the previous year, a significant decline. The CMO's rates fell WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-10 Health Services Advisory Group, Inc. FINDINGS below the respective CY 2012 DCH targets of 45.2 percent for BMI percentile documentation, 57.7 percent for evidence of counseling for nutrition, and 45.5 percent for evidence of counseling for physical activity. In comparison with the national Medicaid HEDIS 2011 benchmarks, WellCare's CY 2012 rates were slightly better than the 50th percentile for BMI percentile documentation (37.5 percent) and evidence of counseling for physical activity (40.6 percent) and between the 50th (51.1 percent) and 75th percentile (61.6 percent) for evidence of counseling for nutrition. In this PIP, WellCare documented that it was working on concrete data analysis strategies to evaluate the effectiveness of interventions such as the Weight Watchers program. WellCare also documented that it would continue to implement strategies to improve BMI percentile documentation due to the dramatic decline in performance. Barriers were identified after medical record reviews, provider and member interviews, as well as feedback received from provider relations. These barriers included the following: lack of member knowledge and restricted access to health activities, members' lack of education around a healthy lifestyle, members unaware of plan benefits related to obesity, cost for the Weight Watchers program, providers unaware of timelines, and providers unaware of HEDIS requirements. To address these barriers, WellCare implemented the following interventions: Partnered with the Boys & Girls Club to establish memberships for youth across the State in an effort to engage them in healthy lifestyle activities. WellCare paid for 604 memberships in 2012. Published an article in the provider newsletter stating the ages for which BMI percentile is required. Held 250 WellCare days at the Women, Infants, and Children (WIC) offices, provider offices, health departments, and Division of Family and Children Services (DFCS) offices across the state. Over 2,367 members were reached during this outreach. Launched a Weight Watchers program for youth to teach them how to eat healthy balanced meals. One hundred twenty-six youth between the ages of 1317 were enrolled in the program in 2012. Distributed HEDIS tool kits by mail and hand delivered others to targeted providers. Targeted pediatricians received postcards outlining the Weight Assessment and Counseling for Nutrition and Physical Activity for Children/Adolescents (WCC) measure. Postcards outlining the WCC measure were handed out to providers at a pediatric conference. Provided a DCH-approved BMI percentile documentation form for providers via their provider Web site. E-mailed independent practice associations (IPAs) and included BMI percentile forms and WCC postcards. Despite HSAG's feedback in last year's PIP validation, the CMO continued to address many barriers and implement interventions that did not have and will continue not to have an effect on the study indicator outcomes. WellCare documented that "they will work closely with Provider Relations in an effort to drill down the issue with BMI percentile documentation. WellCare will work on targeted interventions with providers in order to improve rates for all measures. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-11 Health Services Advisory Group, Inc. FINDINGS Interventions such as Weight Watchers Program and provider education are likely to induce permanent change and hopefully increase compliance." The three study indicators for this PIP are all process measures that only evaluate the presence of documentation of BMI, counseling for nutrition, and counseling for physical activity. Given the measures, all member-based interventions will not impact the rates for any of the study indicators. Only a few of the interventions implemented have the potential to affect the indicator rates. HSAG encourages WellCare to focus its efforts and resources on improvement strategies that will directly impact the rate, and pay special attention as to why there was such a decline in performance for Study Indicator 1, documentation of BMI percentile. For the providers that received a HEDIS tool kit or postcard outlining the WCC measure, the CMO should assess to see if these providers demonstrated better compliance with the WCC documentation requirements. This is an example of the type of intervention tracking WellCare must do to determine if interventions are successful. Comprehensive Diabetes Care Table 2-8--Performance Improvement Project Outcomes for Comprehensive Diabetes Care PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had an 32.4% 32.4% NA HbA1c control < 7.0%. The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had a 25.2% 28.1% NA LDL-C control < 100mg/ml. The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had a BP 51.6% 51.6% NA control < 140/90 mmHg. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. There was essentially no change from baseline to Remeasurement 1 in the study indicator rates for the Comprehensive Diabetes Care PIP. The rates of Study Indicator 1 (HbA1c control < 7.0%) and Study Indicator 3 (BP control < 140/90 mmHg) remained constant, and the rate of Study Indicator 2 increased non-significantly by 2.9 percentage points. The CMO's rates fell below the CY 2012 DCH goals of 35.5 percent (HbA1c control < 7.0%), 33.6 percent (LDL-C control < 100 mg/ml), and 61.6 percent (BP control < 140/90 mmHG), respectively. The rates for Study Indicators 1 and 2 were between the 25th and 50th percentiles of the national Medicaid HEDIS 2011 rates, and the rate for Study Indicator 3 fell below the 25th percentile. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-12 Health Services Advisory Group, Inc. FINDINGS WellCare implemented both member- and provider-focused interventions based on its causal/barrier analysis findings. The CMO identified lack of member education as the primary barrier. For this barrier, WellCare implemented the following interventions: Periodicity Letters--Letters were mailed to new members within 45 days of joining the plan and during the birth month of current members to remind them of upcoming health screenings and immunizations. Community Education Events--The Member Outreach team invited diabetic members to attend community education events. Ten-to-fifteen members met with a member of the Member Outreach team and were educated on diabetes. Diabetes Education Program--Member Outreach staff identified diabetic members who needed to be educated on their chronic disease and the management of it to avoid complications. Members received one-on-one education in their homes, provider offices, or telephonically, depending on their preference. The diabetes presentation included information on the background of diabetes, complications, care (HbA1c, blood pressure, cholesterol, diet and dental, exercise and eye exam, foot care, and glucose monitoring), and a diabetes care schedule. HEDIS Education Screening Program--WellCare identified members with a care gap during the calendar year based on claims data. RNs across the company contacted those diabetic members with care gaps. During the call, the nurse provided education and assisted with making an appointment to visit the provider's office. The CMO identified the barriers of lack of provider incentive, providers unaware of HEDIS requirements, and providers unaware of members who are non-compliant. To address these barriers, WellCare implemented the following interventions: The Pay-for-Performance Quality Incentive Program (P4P Program) was designed to promote the timely completion of health care and preventive services and improve the quality of care for eligible members in its Georgia Medicaid managed care products by paying their P4P Program Providers a bonus for ensuring eligible members received the applicable HEDIS Program Measure services, and the CMO received claims encounters or medical records documenting these services. Provider face-to-face visits were conducted to discuss P4P and to furnish the provider with a list of non-compliant members. HEDIS toolkits were distributed to providers. HSAG encourages WellCare to provide a more detailed description of how the barriers listed were identified, prioritized, and linked to the interventions. The PIP documentation must include a description of the CMO's process for revising its interventions. Based on the lack of statistically significant improvement achieved and stagnant rates for Study Indicators 1 and 2, WellCare must revisit its causal/barrier analysis process and drill down to determine the cause for the lack of improvement. Additionally, some of the interventions were focused on member screening when the focus of the study indicators was on good control of the HbA1c, LDL, and blood pressure. While increasing the number of screened members could improve the study indicator rates if those additional screened members had HbA1c and LDL levels controlled, current efforts do not appear to be targeting the increase in the percentage of members whose WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-13 Health Services Advisory Group, Inc. FINDINGS diabetes is controlled. WellCare's indicator rate for screening for HbA1c was 78.41 percent, and for LDL-C screening, the rate was 69.71 percent; therefore, focusing solely on increased screening only has the potential to improve rates by approximately 2231 percent. Efforts aimed at both increased screening and control may yield a greater increase and more rapid rate of improvement. Avoidable Emergency Room Visits In CY 2012, WellCare began participating in a collaborative performance improvement project with DCH and two other CMOs to address avoidable emergency room (ER) visits by evaluating combined data and implementing coordinated interventions. The collaborative's goal was to reduce avoidable emergency room visits by 5 percent by the end of CY 2012. The baseline and Remeasurement 1 rates for the six study indicators documented in the PIP submission for the collaborative Avoidable Emergency Room Visits PIP are summarized in Table 2-9. Table 2-9--Performance Improvement Project Outcomes for Avoidable Emergency Room Visits PIP Study Indicator Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^ 1. The percentage of practices that provide the same day appointments for routine and urgent care. 2. The percentage of practices that provide routine and urgent care appointments after hours. 3. The percentage of practices that provide appointments for routine and urgent care after hours and have the ability to document after hours clinical advice in the patient's record. 4. The percentage of practices that have access to and utilize electronic health records. 100% 50% 100% 70% 100% NA 70% NA 100% NA 90% NA 5. The percentage of practices that receive information regarding ER visits from the study hospitals. 80% 100% NA 6. The percentage of ER visits for `avoidable' diagnoses (dx382Acute Suppurative otitis:382.9Unspecified otitis:462Acute pharyngitis:465.9Acute upper respiratory infection:466 Acute bronchitis:786.2Cough) 19.38% 20.52%* NA among members under 21 years of age who had a visit to the ED in three selected Children's Healthcare of Atlanta facilities in the Atlanta region. NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. * Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-14 Health Services Advisory Group, Inc. FINDINGS The Avoidable Emergency Room Visits PIP had six study indicators and was piloted in the metro-Atlanta region of the State. Study Indicators 1 through 5 assessed the 10 metro-Atlanta provider practices associated with the highest number of avoidable emergency room visits, and Study Indicator 6 assessed visits to the emergency departments of three Children's Healthcare of Atlanta facilities. Study Indicators 1 through 5 were incorporated at the direction of the State to serve as lead measures. Lead indicators can be helpful in predicting changes that the CMO may use to make mid-course corrections to allow for timely, rapid cycles of improvement rather than waiting for the lag or outcome measure of the PIP, which relies on annual measurement. The initial data for these lead measures were collected by the CMOs during the course of the PIP, and the results showed that these measures did not allow an opportunity for improvement in Study Indicators 1 (percentage of providers who provide same-day appointments) and 3 (percentage of practices that have the ability to document after-hours clinical advice) because the baseline rate for each indicator was 100 percent. The study indicators were created before some of the baseline data were obtained from the participating practices and as such, the CMOs were unaware these baseline rates would be 100 percent. The rates of Study Indicators 2, 4, and 5 had non-statistically significant improvement from baseline to Remeasurement 1. The rate of Study Indicator 5 (percentage of practices that receive ER visit information from study hospitals) reached 100 percent at Remeasurement 1; therefore, this indicator has no room for improvement in future measurement periods for the metro-Atlanta pilot practices. Study Indicator 6, the percentage of emergency room visits for the specified subset of avoidable diagnoses, is the only indicator that did not improve, as there was a significant increase of 1.14 percentage points in the rate of avoidable emergency room visits from baseline to Remeasurement 1. HSAG recommends the CMOs modify their reporting of this PIP for the next remeasurement period and include the lead measures in Activity VIII on the PIP report template. Through its validation review, HSAG noted structural flaws in the documentation of the study design (Activities I through VI) for this collaborative PIP. The numerator and denominator descriptions for Study Indicators 2 and 3 that were documented by the CMOs were identical. The CMOs will need to correct this prior to the next annual submission. Within the study design, the CMOs did not completely define the study population. The CMOs stated, "The method for identifying member visits in the denominator was derived from a list of ICD-9 codes determined to be `avoidable,' i.e., non-emergent conditions that could have been treated in another outpatient setting." NA is not applicable to this element. The denominator (study population) should be composed of all emergency room visits for CMO members under the age of 21. The CPT, UB Revenue, and place of service codes used to identify an emergency room visit, and the anchor date criteria, were not included. In addition, the CMOs did not identify the 10 providers that were involved in the pilot project as part of the study population definition. For the data collection methodology, the CMOs did not include the codes used to identify emergency room visits (denominator for Study Indicator 6). Furthermore, it was unclear how the survey used by the CMOs captured data for Study Indicators 2 and 3. Prior to the three CMOs coming together, WellCare implemented the ER Outreach initiative. The CMO provided education (via telephone and mail) to specific target members to change behavior on utilization of the emergency room. WellCare evaluated ER reports to identify members with frequent ER visits, three or more narcotics, three or more physicians, and WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-15 Health Services Advisory Group, Inc. FINDINGS utilization of three or more pharmacies. These members were contacted within 2448 hours of the visit. In Activity VIII, Implement Intervention and Improvement Strategies, the CMOs documented that a multidisciplinary team of participants from the three CMOs, representatives from DCH, and several study participants reviewed the baseline results of the provider survey, as well as the member focus study, to determine barriers and opportunities for improvement. Interventions were developed to address member, provider, and resource barriers. The CMOs documented that provider-level interventions were designed to motivate providers to offer after-hours care, as well as to encourage the use of electronic health records in the practices. Data sharing was designed to give providers the insight into their level of performance and to identify areas of potential opportunity such as proactive member outreach to establish a medical home. The following were the collaborative provider-level interventions: Increased percentage of practices using electronic health records through referral to the Georgia Health Information Technology Regional Extension Center (GA-HITREC). Shared data regarding ER rates with practices to identify members using the ER during regular office hours. Notified providers regarding the availability of additional reimbursement for care provided after-hours. Member improvement strategies were focused on educating members regarding the available resources to prevent ER use. The following are the collaborative member-level interventions: Continued ER case management programs for live outreach to members who frequented the ER. Educational mailings to members regarding patient-centered medical homes (PCMHs) and nurse advice hotlines. Provided materials to members regarding transportation vendors and assistance to members to arrange transportation, when needed. The PIP documentation did not reflect any processes that were in place to evaluate the effectiveness of any interventions. Although the CMOs discussed follow-up activities planned, due to the decline in performance for the avoidable ER visit rate indicator (Study Indicator 6), HSAG recommends the CMOs, collaboratively, investigate the reasons for the decrease in performance and based on the findings, implement strategies to improve performance. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-16 Health Services Advisory Group, Inc. FINDINGS Member Satisfaction Table 2-10--Performance Improvement Project Outcomes for Member Satisfaction PIP Study Indicator Baseline Period (1/1/135/31/13) Sustained Improvement^ The percentage of respondents who rate the health plan an 8, 9, or 10 to Q36 - "Using any number from 010, where 0 is the worst health plan and 10 is the 88.3% NA best, what number would you use to rate your child's health plan?" NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. WellCare initiated a new Member Satisfaction PIP in CY 2013 as part of its DCH contract requirements. The study indicator, based on Question 36 of WellCare's 2013 CAHPS Child Medicaid Member Survey, assessed the overall rating parents/guardians selected for the CMO, as their child's health plan, with "0" being the lowest possible score and "10" being the highest possible score. The baseline rate of respondents giving WellCare a score of "8" or higher was 88.3 percent. It should be noted that the baseline rate for this PIP was already above the CMO's baseline goal of 85.0 percent. Provider Satisfaction Table 2-11--Performance Improvement Project Outcomes for Provider Satisfaction PIP Study Indicator Baseline Period (8/1/1210/31/12) Sustained Improvement^ The percentage of providers answering, "Very satisfied" or, "Somewhat satisfied" to Q42 - "Please rate your overall satisfaction with WellCare of 81.0% NA Georgia." NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed. ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. WellCare also collected baseline data for a new Provider Satisfaction PIP in 2012. The study indicator from the CMO's 2012 provider satisfaction survey assessed providers' overall satisfaction. The baseline rate of providers who reported being "Somewhat satisfied" or "Very satisfied" with WellCare was 81.0 percent, surpassing the CMO's baseline goal of 74.7 percent. The CMO stated in the PIP Summary Form that its goal was to increase the rate of overall provider satisfaction by 5 percentage points annually; therefore, the CY 2013 survey goal is for 86.0 percent of providers to report being "Somewhat satisfied" or "Very satisfied" with WellCare at Remeasurement 1. WellCare had not progressed to reporting its causal/barrier analysis processes or interventions for both satisfaction PIPs. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-17 Health Services Advisory Group, Inc. 3. CONCLUSIONS AND RECOMMENDATIONS for WellCare of Georgia, Inc. Conclusions WellCare demonstrated a thorough application of the PIP Design stage (Activities I through VI). The sound study design for eight of the nine PIPs created the foundation for the CMO to progress to subsequent PIP stages--implementing improvement strategies and achieving real and sustained study indicator outcomes. WellCare appeared to appropriately select and conduct the sampling and data collection activities. These activities ensured that the CMO properly defined and collected the necessary data to produce accurate study indicator rates. With the exception of the collaborative Avoidable Emergency Room Visits PIP, WellCare exhibited sound study design for eight of its PIPs, it only achieved real and sustained improvement for three of the seven PIPs that progressed to these activities. A critical analysis of the CMO's improvement strategies and processes identified that WellCare completed a causal/barrier analysis for each PIP; however, the CMO lacked sufficient processes to evaluate the effectiveness of the implemented interventions. Without an evaluation plan, the CMO cannot determine whether to modify or discontinue existing interventions, or implement new strategies, thereby reducing the likelihood of achieving the desired goals and improving performance. In addition, WellCare did not provide a data-driven rationale for all of the implemented interventions. Interventions need to be logically linked to study indicators or they will not impact the study indicator outcomes. Recommendations HSAG recommends that WellCare: Reference the PIP Completion Instructions to ensure that all documentation requirements for each activity have been addressed. Ensure that all data components reported in each PIP are accurate and consistently documented throughout the PIP, and align with the data that have been reported in its final audit report. Ensure that all statistical testing is done correctly and the documentation of the statistical testing outcomes is accurate and consistent throughout the PIP. Conduct an annual causal/barrier analysis including drill-down analysis as well as additional quarterly analyses of its outcome data. The CMO must accurately document the analyses, providing the results, identified barriers, and the rationale for how barriers are prioritized. Ensure that the interventions implemented to address a specific barrier are directly linked to that barrier and will directly impact study outcomes. For any intervention implemented, the CMO should have a process in place to evaluate the efficacy of the intervention. The results of the intervention evaluation conducted during each remeasurement period should be included in the PIP. If the interventions are not having the WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 3-1 Health Services Advisory Group, Inc. CONCLUSIONS AND RECOMMENDATIONS desired effect, WellCare should discuss how it will address these deficiencies by modifying or discontinuing current interventions or implementing new improvement strategies. HSAG will work with DCH to create a PIP Summary Form template that is specific to the collaborative Avoidable Emergency Room Visits PIP. WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 3-2 Health Services Advisory Group, Inc. APPENDIX A. PIP-SPECIFIC VALIDATION SCORES for WellCare of Georgia, Inc. Table A-1--WellCare of Georgia, Inc.'s SFY 2014 PIP Performance Study Stage Activity Appropriate Study Topic Clearly Defined, Answerable Study Question(s) Design Correctly Identified Study Population Clearly Defined Study Indicator(s) Valid Sampling Techniques (if sampling was used) Accurate/Complete Data Collection Design Total Implementation Sufficient Data Analysis and Interpretation Appropriate Improvement Strategies Implementation Total Outcomes Real Improvement Achieved Sustained Improvement Achieved Outcomes Total Validation Status Adolescent Well-Care 100% 100% 100% 100% 100% 90% 97% 100% 100% 100% 100% Not Assessed 100% Met Annual Dental Visits 100% 100% 100% 100% Not Applicable 100% 100% 100% 100% 100% 100% 100% 100% Met Percentage of Applicable Evaluation Elements Scored Met Appropriate Use of Childhood ADHD Immunizations Medications --Combo 10 Childhood Obesity Avoidable Comprehensive Emergency Diabetes Care Room Visits 83% 83% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 0% 100% Not Applicable 80% 91% 88% 100% 100% 90% 94% 67% 100% 100% 100% 100% 89% 100% 83% 91% 94% 100% 71% Not Applicable 30% 59% 75% 33% 73% 25% Not Assessed 25% Not Met 100% 77% 100% 50% 77% 25% 33% 83% 25% Not Assessed Not Assessed Not Assessed 100% 25% Partially Met Partially Met 25% Not Met 100% 82% 25% Not Assessed 25% Not Met Member Satisfaction 80% 100% 100% 100% 100% 86% 93% 80% Not Assessed 80% Not Assessed Not Assessed Not Assessed Met Provider Satisfaction 75% 100% 100% 100% 100% 88% 93% 100% Not Assessed 100% Not Assessed Not Assessed Not Assessed Met WellCare of Georgia, Inc. SFY 2014 PIP Validation Report State of Georgia WellCare_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page A-1 Health Services Advisory Group, Inc.