State of Georgia Department of Community Health Georgia Families Program WellCare of Georgia, Inc. PERFORMANCE IMPROVEMENT PROJECTS REPORT FY 2011 December 2010 3133 East Camelback Road, Suite 300 Phoenix, AZ 85016 Phone 602.264.6382 Fax 602.241.0757 2007 Health Services Advisory Group, Inc. TABLE OF CONTENTS 1. BACKGROUND ...................................................................................................................... 1-1 CMO Overview ................................................................................................................................. 1-2 Study Rationale ................................................................................................................................ 1-2 Study Summary ................................................................................................................................ 1-2 Validation Overview.......................................................................................................................... 1-4 2. FINDINGS ............................................................................................................................. 2-1 Aggregate Validation Findings ......................................................................................................... 2-1 Study Design .................................................................................................................................. 2-2 Study Implementation .................................................................................................................... 2-2 Study Outcomes............................................................................................................................. 2-2 PIP-Specific Outcomes .................................................................................................................... 2-3 Analysis of Results ......................................................................................................................... 2-3 Barriers/Interventions ..................................................................................................................... 2-5 3. STRENGTHS.......................................................................................................................... 3-1 Individual PIP Strengths ................................................................................................................... 3-1 Global Strengths Across all PIPs ..................................................................................................... 3-1 4. OPPORTUNITIES FOR IMPROVEMENT ....................................................................................... 4-1 Individual PIPs.................................................................................................................................. 4-1 Global Issues.................................................................................................................................... 4-1 Appendix A. PIP-SPECIFIC VALIDATION SCORES....................................................................... A-1 WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc. Page i ACKNOWLEDGMENTS AND COPYRIGHTS CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA). NCQA HEDIS Compliance AuditTM is a trademark of NCQA. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc. Page ii FY 2011 PIP Validation Report WellCare of Georgia, Inc. 1. BACKGROUND The Code of Federal Regulations (CFR), specifically 42 CFR 438.350, requires states that contract with managed care organizations to conduct an external quality review (EQR) of each entity. An EQR includes the analysis and evaluation by an external quality review organization (EQRO) of aggregated information on health care quality, timeliness, and access. In Georgia, the EQR analyzes and evaluates the health care services that a care management organization (CMO) or its contractors furnish to Georgia Families recipients. At a minimum, the State must report EQRO findings to the federal government on the following mandatory activities: Evaluation of CMO Compliance with Managed Care Regulations Validation of CMO Performance Measures Validation of CMO Performance Improvement Projects (PIPs) These three mandatory activities work together to ensure that Georgia Families' Program and the CMOs are providing quality care to their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measures provides a barometer of the quality and effectiveness of care. When performance measures highlight areas of low performance, the Department of Community Health (DCH) and the CMOs employ PIPs to improve the quality of health care in targeted areas. PIPs are a key tool in the CMOs' overall quality strategy; they provide the framework for monitoring, measuring, and improving the delivery of health care. This is the third year Health Services Advisory Group, Inc. (HSAG), as the State's EQRO, conducted a validation of the CMOs' PIPs. HSAG reviewed each submitted PIP using the Centers for Medicare & Medicaid Services (CMS) validation protocol1 and evaluated two key components of the quality improvement process, as follows: 1) HSAG evaluated the technical structure of the PIPs to ensure the CMOs designed, conducted, and reported PIPs in a methodologically sound manner that met all State and federal requirements. HSAG's review determined whether a PIP's design (e.g., the study indicators, data collection methodology, and analysis plan) was based on sound methodological principles and could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and are capable of measuring sustained improvement. 2) HSAG evaluated the implementation of the PIP. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. This component evaluates how well a CMO improved 1 The Centers for Medicare & Medicaid Services. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, final protocol Version 1.0, May 1, 2002. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-1 Health Services Advisory Group, Inc. BACKGROUND its rates through implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results). A primary goal of HSAG's PIP validation is to ensure that DCH and key stakeholders can have confidence that any reported improvement in outcomes is related to a given PIP. CMO Overview DCH contracted with WellCare of Georgia, Inc. (WellCare) beginning in 2006 to provide services to the Georgia Families program (Medicaid and PeachCare for KidsTM) population. WellCare, a CMO, currently serves the eligible population in all geographic regions of Georgia--Atlanta, Central, East, North, Southeast, and Southwest. Study Rationale The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas. Although HSAG has validated WellCare's PIPs for three years, the number of PIPs, study topics, and study methods have evolved over time. In fiscal year (FY) 2009, DCH chose three PIP topics for validation (i.e., Provider Satisfaction, Well-Child Visits, and Lead Screening in Children). While similar to national, standardized Healthcare Effectiveness Data and Information Set (HEDIS) measures, these PIPs were based on State-defined methodology. In FY 2010, DCH incorporated three additional PIP topics (i.e., Childhood Immunizations, Member Satisfaction, and Adults' Access to Care) for a total of six PIPs. DCH modified the methodology used by the CMOs to reflect the National Committee for Quality Assurance's (NCQA's) HEDIS technical specifications. The incorporation of national, standardized methodologies allowed comparisons to national benchmarks. The second-year validation results for the aforementioned performance measures included the same four HEDIS measures represented by the PIPs; therefore, improvement in the PIP study outcomes would also be seen in the performance measure results. Using the results from prior PIP and performance measure outcomes, DCH directed the CMOs to continue their PIPs on the current topics. The CMOs were required to report both baseline and first remeasurement period data using the HEDIS hybrid method, where applicable. The hybrid method required data to be collected from member medical records, as well as administrative data sources (e.g., claims and encounters). The study topics selected by DCH addressed CMS' requirements related to quality outcomes--specifically, the quality and timeliness of, and access to, care and services. Study Summary As noted in its Quality Strategic Report Plan Update (March 2009), DCH identified the improvement of performance measures in the PIP studies as a key objective. The current PIP submission included three clinical PIPs (i.e., Lead Screening in Children, Childhood WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-2 Health Services Advisory Group, Inc. BACKGROUND Immunizations, and Well-Child Visits) and three nonclinical PIPs (i.e., Adults' Access to Care, Member Satisfaction, and Provider Satisfaction). The three clinical PIP topics were based on HEDIS specifications and addressed children's preventive health (i.e., Lead Screening in Children, Childhood Immunizations, and Well-Child Visits). Children's primary health care is a vital part of the effort to prevent, recognize, and treat health conditions that can result in significant developmental and health status consequences for children and adolescents. These PIP topics represent a key area of focus for improvement. The study indicator for the Adults' Access to Care PIP was also a HEDIS measure. This PIP topic represents an essential component in developing a relationship with a health care provider and establishing a medical home. Table 11 outlines the key study indicators incorporated in these four PIPs. Table 11--HEDIS-based PIP Study Indicators HEDIS Measure/Study Indicator Lead Screening in Children Childhood Immunization Status-- Combo 2 Well-Child Visits in the First 15 Months of Life (Six or More Visits) Adults' Access to Preventive/ Ambulatory Health Services HEDIS Measure Description The percentage of children two years of age who had one or more capillary or venous lead blood tests for lead poisoning by their second birthday. The percentage of children two years of age who had four diphtheria, tetanus, and acellular pertussis (DTaP); three polio (IVP); one measles, mumps, and rubella (MMR); two H influenza type B (Hib); three hepatitis B; and one chicken pox (VZN) by their second birthday. The percentage of members who turned 15 months old during the measurement year and who had six or more well-child visits with a primary care provider (PCP) during their first 15 months of life. The percentage of members 2044 years of age who had an ambulatory or preventive care visit. The remaining two PIPs addressed member and provider satisfaction. Table 12 outlines the key study indicators incorporated in these PIP topics. The Member Satisfaction PIP corresponded to the specifications of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey 4.0H, Child Version measures. These measures provided information on parents' experiences with their child's provider and the care management organization. The plan measured the percentage of members responding favorably to select questions on the Member Satisfaction Survey. The final State-mandated PIP topic was Provider Satisfaction, an area that represented an opportunity for improvement for the CMOs. Each CMO contracted with a vendor to produce and administer this survey, and the CMOs submitted their second remeasurement period data this year. The plan measured the percentage of providers responding favorably (i.e., "Excellent" or "Very Good") to the selected Provider Satisfaction Survey questions. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-3 Health Services Advisory Group, Inc. BACKGROUND Table 12--Satisfaction-based PIP Study Indicators Survey Type Identifier Survey/Study Question Member "Using any number from 0 to 10, where 0 is the worst doctor possible Q24 and 10 is the best doctor possible, what number would you use to rate your child's personal doctor?" Member "In the last 6 months, how often did your child's personal doctor seem Q23 informed and up to date about the care your child got from other doctors/providers?" Provider Q11* "Specialist network has an adequate number of high quality specialists to whom I can refer my patients." Provider Q5* "Timeliness to answer questions and/or resolve problems." Provider Q15* "Timeliness of UM's precertification process." * Providers were requested to respond if they agreed with the statement regarding the CMO. Validation Overview The primary objective of PIP validation was to determine each CMO's compliance with the requirements of 42 CFR 438.240(b)(1), including: Measurement of performance using objective quality indicators. Implementation of systematic interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities for increasing or sustaining improvement. HSAG obtained the data needed to conduct the PIP validation from the CMO's PIP Summary Forms. These forms provided detailed information about each CMO's PIPs related to the activities they completed and HSAG evaluated for the FY 2011 validation cycle. Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be Met. Given the importance of critical elements to this scoring methodology, any critical element that received a Not Met score resulted in an overall validation rating for the PIP of Not Met. A CMO would be given a Partially Met score if 60 percent to 79 percent of all evaluation elements were Met or one or more critical elements were Partially Met. HSAG provided a Point of Clarification when enhanced documentation would have demonstrated a stronger understanding and application of the PIP activities and evaluation elements. In addition to the validation status (e.g., Met) HSAG gave each PIP an overall percentage score for all evaluation elements (including critical elements). HSAG calculated the overall percentage score by dividing the total number of elements scored as Met by the total number of elements scored as Met, Partially Met, and Not Met. HSAG also calculated a critical element percentage WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-4 Health Services Advisory Group, Inc. BACKGROUND score by dividing the total number of critical elements scored as Met by the sum of the critical elements scored as Met, Partially Met, and Not Met. Figure 11 illustrates the three stages of the PIP process--i.e., Study Design, Study Implementation, and Study Outcomes. Each sequential stage provides the foundation for the next stage. The Study Design stage establishes the methodological framework for the PIP. The activities in this section include development of the study topic, question, indicators, and population. To implement successful improvement strategies, a strong study design is necessary. Figure 11--PIP Stages III. STUDY OUTCOMES II. STUDY IMPLEMENTATION I. STUDY DESIGN Once a CMO establishes its study design, the PIP process moves into the Study Implementation stage. This stage includes data collection, sampling, and interventions. During this stage, the CMOs collect measurement data, evaluate and identify barriers to performance, and develop interventions targeted to improve outcomes. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final stage is Study Outcomes, which involves data analysis and the evaluation of real and sustained improvement based on reported results and statistical testing. Sustained improvement is achieved when outcomes exhibit statistical improvement over time and multiple measurements. This stage is the culmination of the previous two stages. If the study outcomes do not improve, the CMOs investigate the data they collected to ensure that they have correctly identified the barriers and implemented appropriate and effective interventions. If they have not, the CMOs revise their interventions and collect additional data to remeasure and evaluate outcomes for improvement. This process becomes cyclical until sustained statistical improvement is achieved. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-5 Health Services Advisory Group, Inc. 2. Findings for WellCare of Georgia, Inc. Aggregate Validation Findings HSAG organized, aggregated, and analyzed WellCare's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of interventions. Based on its technical review, HSAG determined the overall methodological validity of the PIPs. Table 21 displays the combined validation results for all six WellCare PIPs evaluated during FY 2011. This table illustrates the CMO's overall understanding of the PIP process and its success in implementation of the study. Each activity is composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score have satisfied the necessary technical requirements for a specific element. The validation results presented in Table 21 show the percentage of applicable evaluation elements that received a Met score by activity. Additionally, HSAG calculated an overall score across all activities. Appendix A provides the detailed validation scores for each of the six PIPs. Table 21FY 2011 Performance Improvement Project Validation Results for WellCare of Georgia, Inc. (N=6 PIPs) Percentage of Applicable Elements Stage Activity Met Partially Met Not Met I. Appropriate Study Topic 100% (32/32) 0% (0/32) 0% (0/32) II. Clearly Defined, Answerable Study Question(s) Study Design III. Clearly Defined Study Indicator(s) 100% (12/12) 100% (36/36) 0% (0/12) 0% (0/36) 0% (0/12) 0% (0/36) IV. Correctly Identified Study Population 100% (18/18) 0% (0/18) 0% (0/18) V. Valid Sampling Techniques (if sampling was used) 100% (30/30) 0% (0/30) 0% (0/30) Study Implementation VI. Accurate/Complete Data Collection 100% (51/51) 0% (0/51) 0% (0/51) VII. Appropriate Improvement Strategies 95% (18/19) 5% (1/19) 0% (0/19) VIII. Sufficient Data Analysis and Interpretation 98% (52/53) 2% (1/53) 0% (0/53) Study Outcomes IX. Real Improvement Achieved 63% (15/24) 8% (2/24) 29% (7/24) X. Sustained Improvement Achieved* 100% 0% 0% (1/1) (0/1) (0/1) Percentage Score of Applicable Evaluation Elements Met 96% (265/276) * Only the Provider Satisfaction PIP had progressed to this phase in the review period and was assessed for sustained improvement. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-1 Health Services Advisory Group, Inc. FINDINGS Overall, 96 percent of the evaluation elements across all six PIPs received a score of Met. While WellCare's strong performance in the Study Design and Study Implementation phases indicated that each PIP was designed and implemented appropriately to measure outcomes and improvement, it was less successful in achieving the desired outcomes. The following subsections highlight HSAG's validation findings associated each of the three PIP stages. Study Design WellCare met 100 percent of the requirements across all six PIPs for all four activities within the Study Design stage. Overall, WellCare designed scientifically sound studies that were supported by use of key research principles. The technical design of each PIP was sufficient to measure and monitor PIP outcomes associated with WellCare's improvement strategies. The solid design of the PIPs allowed the successful progression to the next stage of the PIP process. Study Implementation WellCare met 100 percent of the requirements for both the sampling and data collection activities in the Study Implementation phase; however, the CMO did not meet all of the requirements for the third activity of this phase, implementation of improvement strategies. Five individual PIPs received a Met score for 100 percent of the evaluation elements while the WellChild Visits PIP only received a Met score for 67 percent of the evaluation elements. These results produced an overall aggregate score of 95 percent of the applicable elements receiving a Met score for this activity. These findings suggested that while the CMO accurately documented and executed the implementation of the study design, WellCare's process for developing interventions in its Well-Child Visits PIP continued to be an area for improvement. With the successful implementation of appropriate improvement strategies, the CMO could achieve improved outcomes in the future. Study Outcomes WellCare met the requirements for two of the three activities in the Study Outcomes stage. The CMO correctly conducted analyses and interpreted its results as demonstrated in Activity VIII (i.e., Sufficient Data Analysis and Interpretation) with individual PIP scores ranging from 89 percent to 100 percent. However, as seen in Table 22 and Table 23, not all of the PIPs demonstrated statistically significant improvement related to Activity IX (i.e., Real Improvement Achieved). Individual PIP scores ranged from 25 percent to 100 percent. Consequently, the aggregated results for Activity IX across all six PIPs reflected this deficiency (63 percent of the evaluation elements received a Met score) even though the Adults' Access to Care PIP scored considerably higher (100 percent). To be successful, the PIPs must show real, or statistical, improvement in their study indicators. Only the Provider Satisfaction PIP had progressed to the point of reporting a second remeasurement period and demonstrated sustained improvement for two of the three study indicators. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-2 Health Services Advisory Group, Inc. FINDINGS PIP-Specific Outcomes Analysis of Results Table 22 and Table 23 display outcome data for WellCare's six PIPs. The CMO submitted Remeasurement 1 data for five of the PIPs and Remeasurement 2 data for the Provider Satisfaction PIP. Table 22HEDIS-based Performance Improvement Project Outcomes for WellCare of Georgia, Inc. PIP #1--Lead Screening in Children PIP Study Indicator Baseline Period Remeasurement 1 Remeasurement 2 Sustained (1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10) Improvement The percentage of children 2 years of age who received one blood lead test (capillary or 65.9% 67.4% venous) on or before their second birthday. PIP #2--Childhood Immunizations The percentage of children who received the recommended vaccinations based on the Childhood 75.9% 81.0% Immunization Status--Combo 2 (4:3:1:2:3:1) guidelines. PIP #3--Well-Child Visits The percentage of children who had six or more wellchild visits with a PCP during 57.4% 57.4% their first 15 months of life. PIP #4--Adults' Access to Care The percentage of members 2044 years of age who had an ambulatory or preventive 78.6% 84.7%* care visit. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-3 Health Services Advisory Group, Inc. FINDINGS Table 23Satisfaction-based Performance Improvement Project Outcomes for WellCare of Georgia, Inc. PIP #5--Member Satisfaction PIP Study Indicator Baseline Period Remeasurement 1 Remeasurement 2 Sustained (2/1/095/31/09) (2/1/105/31/10) (2/1/115/31/11) Improvement 1) The percentage of members responding with either a "9" or "10" to Q24-- "Using any number from 0 to 10, where 0 is the worst personal doctor possible and 10 is the best personal 72.2% 71.2% doctor possible, what number would you use to rate your child's personal doctor?" 2) The percentage of eligible members responding with either "Always" or "Usually" to Q23--"In the last 6 months, how often did your child's 77.1% 78.4% personal doctor seem informed and up to date about the care your child got from other doctors/providers?" PIP #6--Provider Satisfaction PIP Study Indicator^ Baseline Period Remeasurement 1 Remeasurement 2 Sustained (10/1/069/30/07) (10/1/079/30/08) (10/1/089/30/09) Improvement 1) The percentage of providers answering "Excellent" or "Very Good" to Q11-- "Specialist network has an adequate 22.2% 19.7% 24.7% number of high quality specialists to whom I can refer my patients." 2) The percentage of providers answering "Excellent" or "Very Good" to Q5-- "Timeliness to answer and/or resolve 22.2% 29.6%* 31.3% Yes problems." 3) The percentage of providers answering "Excellent" or "Very Good" to Q15-- "Timeliness of UM's pre-certification process." 22.5% 25.5% 29.3% Yes ^ Providers were requested to respond if they agreed with the statements regarding the CMO. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). The Adults' Access to Care PIP demonstrated statistically significant improvement from Baseline to Remeasurement 1. The percentage of adult members that accessed ambulatory or preventive care increased by approximately six percentage points to 84.7 percent. Statistically significant improvement is the standard for assessing real improvement and supports the conclusion that the improvement was not due to chance. Although WellCare's performance improved, it remained 0.1 percentage points below the DCH target (84.8 percent) and fell between the national 2009 HEDIS Medicaid 50th and 75th percentiles (81.44 percent and 85.58 percent). WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-4 Health Services Advisory Group, Inc. FINDINGS Additionally, the performance for three PIPs--i.e., Lead Screening in Children, Childhood Immunizations, and the second study indicator for the Member Satisfaction PIP ("How often did your child's personal doctor seem informed and up to date about the care your child got from other doctors/providers?"), increased from Baseline to Remeasurement 1. However, the increases were not statistically significant and, therefore, not considered real improvement. Both the Lead Screening in Children and Childhood Immunizations study indicator rates remained above the DCH target rates for these measures (65.9 percent and 72.0 percent, respectively). The Remeasurement 1 rate for Lead Screening in Children was below the 50th national 2009 HEDIS Medicaid percentile (70.21 percent) while the Remeasurement 1 rate for Childhood Immunizations was above the national 2009 HEDIS Medicaid 50th percentile (78.01 percent). WellCare's performance for the Well-Child Visits study indicator (57.4 percent) did not change from Baseline to Remeasurement 1 and remained 8 percentage points below the DCH target of 65.4 percent and fell between the 25th and 50th national 2009 HEDIS Medicaid percentiles (51.58 percent and 60.52 percent). The first study indicator for the Member Satisfaction PIP ("...what number would you use to rate your child's personal doctor?") was the only study indicator rate of any of the PIPs that decreased during the most recent measurement period. The rate decreased by one percentage point; however, the decrease was not statistically significant. Rates for all three of the Provider Satisfaction PIP's study indicators increased from the first to the second remeasurement. More importantly, the second and third study indicators demonstrated sustained improvement since they improved between all measurement periods. These findings highlight success in the implementation of quality strategies for improving overall satisfaction. The first study indicator, though, will require another measurement period before HSAG can assess it for sustained improvement because the rate had initially decreased from Baseline to Remeasurement 1. However, the increase observed during Remeasurement 2 suggests that WellCare's interventions and quality improvement processes will positively affect the outcome for this indicator. Barriers/Interventions The identification of barriers through barrier analysis and the subsequent selection of appropriate interventions to address those barriers are necessary steps to improve outcomes. The CMO's choice of interventions, the combination of intervention types, and the sequence of the implementation of the interventions are all essential to the CMO's overall success. WellCare identified the lack of provider and member knowledge regarding the required screenings and immunization schedules as primary barriers for three of its PIPs--i.e., Childhood Immunizations, Lead Screening in Children, and Well-Child Visits. While WellCare documented more than nine ongoing interventions for each of these PIPs, the CMO implemented very few new interventions. New or modified interventions are needed to improve rates during the PIP process since ongoing interventions are associated with current rates and not associated with rate changes during the PIP study period. The lack of significant improvement of outcomes for these PIPs was also due in part to the timing of the interventions as described below. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-5 Health Services Advisory Group, Inc. FINDINGS For the Well-Child Visits PIP, the CMO did not initiate any new interventions until July 2009. The interventions included distributing the HEDIS Provider Toolkit and noncompliant member lists to providers. The CMO also conducted telephone outreach to noncompliant members. The 2009 improvement strategies required more time to have any effect on the CY 2009 results; however, they could affect both CY 2010 HEDIS rates and PIP remeasurement rates. In March 2010, the CMO distributed the provider letter and well-child billing guide developed by the CMO Well-Child Collaborative to providers. Similarly, for the Lead Screening in Children PIP, WellCare initiated one new provider education intervention in 2008 that educated staff on the Georgia Childhood Lead Poisoning Prevention Program (GCLPPP). The CMO also initiated two interventions during the second half of 2009: (1) distribution of the HEDIS Provider Toolkit and noncompliant member lists to providers and (2) telephone outreach to noncompliant members; however, the result of these interventions was limited since they were only in effect for six months of the year. The full effect of these strategies would potentially be demonstrated in both calendar year (CY) 2010 HEDIS rates and PIP remeasurement rates. WellCare used the same improvement strategy for the Childhood Immunizations PIP as it used for the Lead Screening in Children PIP. The CMO initiated one new provider education intervention in 2008 that included sending a blast fax to all providers in reference to the 2008 childhood immunization schedule. Additionally, the CMO initiated two interventions in the second half of 2009: (1) distribution of the HEDIS Provider Toolkit and noncompliant member lists to providers and (2) telephone outreach to noncompliant members. As with the Lead Screening in Children PIP, these 2009 strategies required more time to have any effect on the CY 2009 results; however, they could affect both CY 2010 HEDIS rates and PIP remeasurement rates. Conversely, for the Adults' Access to Care PIP, the timing of the interventions affected the remeasurement period rates reported in CY 2009 and led to an increase of approximately 6 percentage points. The CMO initiated interventions in both 2008 and 2009. In 2008, the CMO identified the provider's lack of understanding regarding the need to provide services as the primary barrier. WellCare implemented sequential interventions specifically targeted to the barrier, including the following: 1) Reviewed medical records to identify providers noncompliant with adult preventive health care guidelines 2) Updated adult preventive health care guidelines 3) Distributed adult preventive health care guidelines through the provider handbook 4) Distributed adult preventive health care guidelines through the member newsletter 5) Posted the adult preventive health care guidelines on the Web site and included information in the provider newsletter In the last quarter of 2009, the CMO distributed the 2009 adult preventive health care guidelines through both the member newsletter and the member handbook. Additionally in 2009, the CMO conducted its quarterly quality improvement meeting and identified through a cause and effect diagram that members were going to the emergency room (ER) instead of a PCP; therefore, WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-6 Health Services Advisory Group, Inc. FINDINGS preventive services were not being performed. The CMO implemented a two-pronged approach to address this pattern. First, the CMO initiated system interventions ensuring that members had access to preventive services. Second, the CMO realigned staff resources so it could conduct focused member outreach to members within 48 hours of an ER visit. The outreach consisted of member education on the PCP's role and assistance with care and/or transportation needs. The CMO also created a database to track member contacts. The Member Satisfaction PIP outcomes remained unchanged from the baseline period. WellCare did not initiate any interventions in CY 2008. Additionally, of all the interventions that the CMO implemented in CY 2009, only one directly related to the study outcomes--the CMO distributed a Patient Safety Tip Sheet to providers addressing the lack of coordination between primary care providers and specialists. However, the timing of the intervention was such that it could not affect the current PIP cycle, and it will not likely affect rates until the second remeasurement period. The CMO's other interventions dealt with barriers such as the prior-authorization process, members unaware of translation services, provider directories not available on the Web portal, members not understanding how to change providers, coordination of care, etc. Even if these interventions affect identified barriers, they will not affect the outcomes for the PIP study indicators. Conversely, for the Provider Satisfaction PIP, WellCare implemented numerous targeted interventions that linked directly to the identified barriers. Examples of the CMO's interventions addressing "timeliness to answer and/or resolve problems" and "timeliness of UM's precertification process" included the following: Documenting provider concerns and feedback identified by provider relations representatives in a database, then training representatives on how to trend provider dissatisfaction Opening a Customer Service Call Center for providers Implementing a new prior-authorization checklist Employing a reconsideration process for authorization requests that included a peer-to-peer process Incorporating a new database to enhance timeliness and tracking of prior authorizations The CMO educated staff and providers on all initiatives. The study outcomes for the second and third study indicators for this PIP increased over time, demonstrating both real and sustained improvement. WellCare, as part of its quarterly barrier analysis, prioritized the identified barriers to provider satisfaction. The reevaluation of quality strategies allowed the CMO to address changes in PIP outcomes more effectively. For the first study indicator ("specialist network has an adequate number of high quality specialists to whom I can refer my patients"), the CMO responded to the decrease in the remeasurement result and implemented focused interventions that used provider feedback and referral patterns to recruit needed specialists. The result was an upward trend by the second remeasurement period. Overall, WellCare exhibited a strong understanding of the key steps necessary for ensuring improvement. However, the execution of intervention strategies across the six PIPs was inconsistent, resulting in the improvement of some outcomes, but not all. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-7 Health Services Advisory Group, Inc. FINDINGS The PIP validation process relies on an annual evaluation; however, CMOs should perform an interim evaluation of the results in addition to the formal annual evaluation. Evaluation of interim measurement results could assist the CMO in identifying and eliminating barriers that impede improvement. Furthermore, evaluation of the study outcomes would assist the CMO in determining if the interventions are having the desired effect or if modifications to current interventions or new interventions are necessary to improve results. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-8 Health Services Advisory Group, Inc. 3. Strengths for WellCare of Georgia, Inc. Individual PIP Strengths The Adult's Access to Care PIP received a Met score for 100 percent of the evaluation elements in all three PIP validation stages--Study Design, Study Implementation, and Study Outcomes. The outcome for the Adults' Access to Care PIP, which improved significantly from the baseline to the first remeasurement, reflected the effects of a strong quality strategy. Although the performance was 0.1 percentage points below the DCH target (84.8 percent) and 0.9 percentage points below the national 2009 HEDIS 75th percentile of 85.58 percent, WellCare's success on this PIP could continue to improve the CMO's general performance on the Adults' Access to Preventive/Ambulatory Health Services HEDIS measure. WellCare was successful in achieving real and sustained improvement for two of the three study indicators in the Provider Satisfaction PIP. The CMO responded to the decline in the first study indicator's Remeasurement 1 results and implemented revised, targeted interventions that positively affected the outcome. Moreover, WellCare's implementation of the revised Provider Satisfaction interventions suggested that the CMO may be successful in achieving real and sustained improvement in the future. Global Strengths Across all PIPs All six PIPs received an overall Met validation status, which represented an area of strength for WellCare and provided confidence in the technical aspects of the studies. The performance on these PIPs suggested a thorough understanding of the PIP Study Design stage.The sound study design of the PIPs created the foundation for the CMO to progress to subsequent PIP stages-- i.e., implementing improvement strategies and accurately assessing study outcomes. The CMO appeared to understand and appropriately conduct the sampling and data collection activities of the Study Implementation stage. These activities ensured that the studies properly defined and collected the necessary data to produce accurate study indicator rates. Additionally, WellCare appropriately documented improvement strategies, an activity which ensured that study outcomes could improve. Furthermore, in the Study Outcomes stage, the CMO properly analyzed and interpreted the results. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 3-1 Health Services Advisory Group, Inc. 4. Opportunities for Improvement for WellCare of Georgia, Inc. Individual PIPs The Well-Child Visits and Member Satisfaction PIPs had the lowest validation scores for the Study Outcomes stage (69 and 77 percent, respectively); therefore, to improve study outcomes in the future, WellCare should focus on implementing new and/or enhanced quality strategies for these PIPs. The past and ongoing interventions have not yielded improved results. Specifically, the study outcome for the Well-Child Visits PIP remained unchanged during remeasurement and was below the DCH target of 65.4 percent and the national 2009 HEDIS Medicaid 50th percentile of 60.52 percent. WellCare's process for developing interventions in its Well-Child Visits PIP continued to be an area for improvement. Similarly, the study outcome for the Lead Screening in Children PIP was also statistically unchanged and remained below the national 2009 HEDIS Medicaid 50th percentile of 70.21 percent. However, the outcome for this PIP was above the DCH target rate (65.9 percent). To increase the measurable effects of its quality improvement activities, WellCare should ensure that the implementation of interventions occurs early enough in the measurement period to provide sufficient time for the outcomes to be affected and demonstrate improvement. Global Issues While WellCare exhibited a strong understanding of the key steps necessary for ensuring improvement, the execution of intervention strategies across the six PIPs was inconsistent. WellCare should plan and implement its improvement strategies more efficiently, providing enough time for the interventions to affect study outcomes. Additionally, the CMO should analyze its data to determine if any subgroup within its population has a disproportionately lower rate that negatively affected the overall rates. This "drill-down" type of analysis should be conducted both before and after the implementation of any intervention. For example, WellCare should evaluate whether rates differ by geographic region, gender, race/ethnicity, age, etc. The CMO could then target its interventions to those subgroups with the lowest rates, allowing the implementation of more precise, concentrated interventions. The process of targeting interventions to the appropriate subgroups is more efficient and effective. The CMO should be mindful that the submission of PIPs for validation will be an annual activity without an opportunity to resubmit. WellCare should carefully complete all necessary documentation. The CMO should refer to the PIP Validation Tool and address all Points of Clarification and all Partially Met and Not Met scores in the FY 2012 submission. WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 4-1 Health Services Advisory Group, Inc. Appendix A. PIP-Specific Validation Scores for WellCare of Georgia, Inc. Review Step Study Design I. Review the Selected Study Topic(s) II. Review the Study Question(s) III. Review the Selected Study Indicator(s) IV. Review the Identified Study Population Study Implementation V. Review Sampling Methods VI. Review Data Collection Procedures VII. Assess Improvement Strategies Study Outcomes VIII. Review Data Analysis and Study Results IX. Assess for Real Improvement X. Assess for Sustained Improvement Percentage Score for Applicable Evaluation Elements Met Percentage Score for Applicable Critical Elements Met Validation Status Table A1--WellCare's FY 2011 PIP Performance Lead Screening in Children 17/17 (100%) 6/6 (100%) 2/2 (100%) Childhood Immunizations 17/17 (100%) 6/6 (100%) 2/2 (100%) Well-Child Visits 17/17 (100%) 6/6 (100%) 2/2 (100%) Adults' Access to Care 16/16 (100%) 5/5 (100%) 2/2 (100%) 6/6 (100%) 6/6 (100%) 6/6 (100%) 6/6 (100%) 3/3 (100%) 19/19 (100%) 6/6 (100%) 10/10 (100%) 3/3 (100%) 12/13 (92%) 9/9 (100%) 3/4 (75%) Not Assessed 98% 3/3 (100%) 19/19 (100%) 6/6 (100%) 10/10 (100%) 3/3 (100%) 12/13 (92%) 9/9 (100%) 3/4 (75%) Not Assessed 98% 3/3 (100%) 18/19 (95%) 6/6 (100%) 10/10 (100%) 2/3 (67%) 9/13 (69%) 8/9 (89%) 1/4 (25%) Not Assessed 90% 3/3 (100%) 8/8 (100%) 0/0 5/5 (100%) 3/3 (100%) 12/12 (100%) 8/8 (100%) 4/4 (100%) Not Assessed 100% 100% Met 100% Met 100% Met 100% Met Member Satisfaction 16/16 (100%) 5/5 (100%) 2/2 (100%) 6/6 (100%) 3/3 (100%) 18/18 (100%) 6/6 (100%) 8/8 (100%) 4/4 (100%) 10/13 (77%) 9/9 (100%) 1/4 (25%) Not Assessed 94% 100% Met Provider Satisfaction 15/15 (100%) 4/4 (100%) 2/2 (100%) 6/6 (100%) 3/3 (100%) 17/17 (100%) 6/6 (100%) 8/8 (100%) 3/3 (100%) 13/14 (93%) 9/9 (100%) 3/4 (75%) 1/1 (100%) 98% 100% Met WellCare of Georgia, Inc. FY 2011 PIP Validation Report State of Georgia WellCare_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page A-1 Health Services Advisory Group, Inc.