State of Georgia Department of Community Health Georgia Families Program Peach State Health Plan PERFORMANCE IMPROVEMENT PROJECTS REPORT FY 2011 December 2010 3133 East Camelback Road, Suite 300 Phoenix, AZ 85016 Phone 602.264.6382 Fax 602.241.0757 2007 Health Services Advisory Group, Inc. TABLE OF CONTENTS 1. BACKGROUND ...................................................................................................................... 1-1 CMO Overview ................................................................................................................................. 1-2 Study Rationale ................................................................................................................................ 1-2 Study Summary ................................................................................................................................ 1-3 Validation Overview.......................................................................................................................... 1-4 2. FINDINGS.............................................................................................................................. 2-1 Aggregate Validation Findings ......................................................................................................... 2-1 Study Design .................................................................................................................................. 2-2 Study Implementation .................................................................................................................... 2-2 Study Outcomes............................................................................................................................. 2-2 PIP-Specific Outcomes .................................................................................................................... 2-3 Analysis of Results ......................................................................................................................... 2-3 Barriers/Interventions ..................................................................................................................... 2-5 3. STRENGTHS.......................................................................................................................... 3-1 Individual PIP Strengths ................................................................................................................... 3-1 Global Strengths Across all PIPs ..................................................................................................... 3-1 4. OPPORTUNITIES FOR IMPROVEMENT ....................................................................................... 4-1 Individual PIPs.................................................................................................................................. 4-1 Global Issues.................................................................................................................................... 4-1 Appendix A. PIP-SPECIFIC VALIDATION SCORES....................................................................... A-1 Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc. Page i ACKNOWLEDGMENTS AND COPYRIGHTS CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA). NCQA HEDIS Compliance AuditTM is a trademark of NCQA. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc. Page ii FY 2011 PIP Validation Report Peach State Health Plan 1. BACKGROUND The Code of Federal Regulations (CFR), specifically 42 CFR 438.350, requires states that contract with managed care organizations to conduct an external quality review (EQR) of each entity. An EQR includes analysis and evaluation by an external quality review organization (EQRO) of aggregated information on health care quality, timeliness, and access. In Georgia, the EQR analyzes and evaluates the health care services that a care management organization (CMO) or its contractors furnish to Georgia Families recipients. At a minimum, the State must report EQRO findings to the federal government on the following mandatory activities: Evaluation of CMO Compliance with Managed Care Regulations Validation of CMO Performance Measures Validation of CMO Performance Improvement Projects (PIPs) These three mandatory activities work together to ensure that Georgia Families' Program and the CMOs are providing quality care to their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measures provides a barometer of the quality and effectiveness of care. When performance measures highlight areas of low performance, the Department of Community Health (DCH) and the CMOs employ PIPs to improve the quality of health care in targeted areas. PIPs are a key tool in the CMOs' overall quality strategy; they provide the framework for monitoring, measuring, and improving the delivery of health care. This is the third year, Health Services Advisory Group, Inc. (HSAG), as the State's EQRO, conducted a validation of the CMOs' PIPs. HSAG reviewed each submitted PIP using the Centers for Medicare & Medicaid Services (CMS) validation protocol1 and evaluated two key components of the quality improvement process, as follows: 1) HSAG evaluated the technical structure of the PIPs to ensure the CMOs designed, conducted, and reported PIPs in a methodologically sound manner, meeting all State and federal requirements. HSAG's review determined whether a PIP's design (e.g., study indicators, the data collection methodology, and analysis plan) was based on sound methodological principles and could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring sustained improvement. 2) HSAG evaluated the implementation of the PIP. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. This component evaluates how well a CMO improved 1 The Centers for Medicare & Medicaid Services. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, final protocol Version 1.0, May 1, 2002. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-1 Health Services Advisory Group, Inc. BACKGROUND its rates through implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results). A primary goal of HSAG's PIP validation is to ensure that DCH and key stakeholders can have confidence that any reported improvement in outcomes is related to a given PIP. CMO Overview DCH contracted with Peach State Health Plan (Peach State) beginning in 2006 to provide services to the Georgia Families Program (Medicaid and PeachCare for KidsTM) population. Peach State, a CMO, serves the eligible population in the Atlanta, Central, and Southwest geographic regions of Georgia. Study Rationale The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in both clinical and nonclinical areas. Although HSAG has validated Peach State's PIPs for three years, the number of PIPs, study topics, and study methods have evolved over time. In fiscal year (FY) 2009, DCH chose three PIP topics for validation (i.e., Provider Satisfaction, Well-Child Visits, and Lead Screening in Children). While similar to national, standardized Healthcare Effectiveness Data and Information Set (HEDIS) measures, these PIPs were based on State-defined methodology. In FY 2010, DCH incorporated three additional PIP topics (i.e., Childhood Immunizations, Member Satisfaction, and Adults' Access to Care) for a total of six PIPs. DCH modified the methodology used by the CMOs to reflect the National Committee for Quality Assurance's (NCQA's) HEDIS technical specifications. The incorporation of national, standardized methodologies allowed comparisons to national benchmarks. The second-year validation results for the aforementioned performance measures included the same four HEDIS measures represented by the PIPs; therefore, improvement in the PIP study outcomes would also be seen in the performance measure results. Using the results from prior PIP and performance measure outcomes, DCH directed the CMOs to continue their PIPs on the current topics. The CMOs were required to report both baseline and first remeasurement period data using the HEDIS hybrid method, where applicable. The hybrid method required data to be collected from member medical records, as well as administrative data sources (e.g., claims and encounters). The study topics selected by DCH addressed CMS' requirements related to quality outcomes--specifically, the quality and timeliness of, and access to, care and services. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-2 Health Services Advisory Group, Inc. BACKGROUND Study Summary As noted in its Quality Strategic Report Plan Update (March 2009), DCH identified the improvement of performance measures in the PIP studies as a key objective. The current PIP submission included three clinical PIPs (i.e., Lead Screening in Children, Childhood Immunizations, and Well-Child Visits) and three nonclinical PIPs (i.e., Adults' Access to Care, Member Satisfaction, and Provider Satisfaction). The three clinical PIP topics were based on HEDIS specifications and addressed children's preventive health (i.e., Lead Screening in Children, Childhood Immunizations, and Well-Child Visits). Children's primary health care is a vital part of the effort to prevent, recognize, and treat health conditions that can result in significant developmental and health status consequences for children and adolescents. These PIP topics represent a key area of focus for improvement. The study indicator for the Adults' Access to Care PIP was also a HEDIS measure. This PIP topic represents an essential component in developing a relationship with a health care provider and establishing a medical home. Table 11 outlines the key study indicators incorporated in these four PIPs. Table 11--HEDIS-based PIP Study Indicators HEDIS Measure/Study Indicator Lead Screening in Children Childhood Immunization Status-- Combo 2 Well-Child Visits in the First 15 Months of Life (Six or More Visits) Adults' Access to Preventive/ Ambulatory Health Services HEDIS Measure Description The percentage of children two years of age who had one or more capillary or venous lead blood tests for lead poisoning by their second birthday. The percentage of children two years of age who had four diphtheria, tetanus, and acellular pertussis (DTaP); three polio (IVP); one measles, mumps, and rubella (MMR); two H influenza type B (Hib); three hepatitis B; and one chicken pox (VZN) by their second birthday. The percentage of members who turned 15 months old during the measurement year and who had six or more well-child visits with a primary care provider (PCP) during their first 15 months of life. The percentage of members 2044 years of age who had an ambulatory or preventive care visit. The remaining two PIPs addressed member and provider satisfaction. Table 12 outlines the key study indicators incorporated in these PIP topics. The Member Satisfaction PIP corresponded to the specifications of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey 4.0H, Child Version measures. These measures provided information on parents' experiences with their child's provider and care management organization. The plan measured the percentage of members responding positively ("Usually" or "Always") to the selected Member Satisfaction Survey questions. The final State-mandated PIP topic was Provider Satisfaction, an area that represented an opportunity for improvement for the CMOs. Each CMO contracted with a vendor to produce and administer this survey, and the CMOs submitted their second remeasurement period data this year. The plan measured the percentage of providers responding favorably (i.e., "Excellent" or "Very Good") to the selected Provider Satisfaction Survey questions. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-3 Health Services Advisory Group, Inc. BACKGROUND Table 12--Satisfaction-based PIP Study Indicators Survey Type Identifier Survey/Study Question Member Q26 "Ease of getting appointment with a specialist" Member Q30 "Getting care, tests, or treatments necessary" Member Q32 "Getting information/help from customer service" Member Q33 "Treated with courtesy and respect by customer service staff" Provider Q5* "Timeliness to answer questions and/or resolve problems" Provider Q6* "Quality of provider orientation process" Provider Q18* "Health plan takes physician input and recommendations seriously" Provider Q34* "Accuracy of claims processing" * Providers were requested to respond if they agreed with the statement regarding the CMO. Validation Overview The primary objective of PIP validation was to determine each CMO's compliance with the requirements of 42 CFR 438.240(b)(1), including: Measurement of performance using objective quality indicators. Implementation of systematic interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities for increasing or sustaining improvement. HSAG obtained the data needed to conduct the PIP validation from the CMO's PIP Summary Forms. These forms provided detailed information about each CMO's PIPs related to the activities they completed and HSAG evaluated for the FY 2011 validation cycle. Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be Met. Given the importance of critical elements to this scoring methodology, any critical element that received a Not Met score resulted in an overall validation rating for the PIP of Not Met. A CMO would be given a Partially Met score if 60 percent to 79 percent of all evaluation elements were Met or one or more critical elements were Partially Met. HSAG provided a Point of Clarification when enhanced documentation would have demonstrated a stronger understanding and application of the PIP activities and evaluation elements. In addition to the validation status (e.g., Met) HSAG gave each PIP an overall percentage score for all evaluation elements (including critical elements). HSAG calculated the overall percentage score by dividing the total number of elements scored as Met by the total number of elements scored as Met, Partially Met, and Not Met. HSAG also calculated a critical element percentage Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-4 Health Services Advisory Group, Inc. BACKGROUND score by dividing the total number of critical elements scored as Met by the sum of the critical elements scored as Met, Partially Met, and Not Met. Figure 11 illustrates the three stages of the PIP process--i.e., Study Design, Study Implementation, and Study Outcomes. Each sequential stage provides the foundation for the next stage. The Study Design stage establishes the methodological framework for the PIP. The activities in this section include development of the study topic, question, indicators, and population. To implement successful improvement strategies, a strong study design is necessary. Figure 11--PIP Stages III. STUDY OUTCOMES II. STUDY IMPLEMENTATION I. STUDY DESIGN Once a CMO establishes its study design, the PIP process moves into the Study Implementation stage. This stage includes data collection, sampling, and interventions. During this stage, the CMOs collect measurement data, evaluate and identify barriers to performance, and develop interventions targeted to improve outcomes. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final stage is Study Outcomes, which involves data analysis and the evaluation of real and sustained improvement based on reported results and statistical testing. Sustained improvement is achieved when outcomes exhibit statistical improvement over time and multiple measurements. This stage is the culmination of the previous two stages. If the study outcomes do not improve, the CMOs investigate the data they collected to ensure that they have correctly identified the barriers and implemented appropriate and effective interventions. If they have not, the CMOs revise their interventions and collect additional data to remeasure and evaluate outcomes for improvement. This process becomes cyclical until sustained statistical improvement is achieved. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-5 Health Services Advisory Group, Inc. 2. Findings for Peach State Health Plan Aggregate Validation Findings HSAG organized, aggregated, and analyzed Peach State's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of improvement interventions. Based on its technical review, HSAG determined the overall methodological validity of the PIPs. Table 21 displays the combined validation results for all six Peach State PIPs evaluated during FY 2011. This table illustrates the CMO's overall understanding of the PIP process and its success in implementation of the study. Each activity is composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score have satisfied the necessary technical requirements for a specific element. The validation results presented in Table 21 show the percentage of applicable evaluation elements that received a Met score by activity. Additionally, HSAG calculated an overall score across all activities. Appendix A provides the detailed validation scores for each of the six PIPs. Table 21FY 2011 Performance Improvement Project Validation Results for Peach State Health Plan (N=6 PIPs) Percentage of Applicable Elements Stage Activity Met Partially Met Not Met I. Appropriate Study Topic 100% (32/32) 0% (0/32) 0% (0/32) Study Design II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s) 100% (12/12) 100% (36/36) 0% (0/12) 0% (0/36) 0% (0/12) 0% (0/36) IV. Correctly Identified Study Population 100% (18/18) 0% (0/18) 0% (0/18) V. Valid Sampling Techniques (if sampling was used) 100% (24/24) 0% (0/24) 0% (0/24) Study Implementation VI. Accurate/Complete Data Collection 100% (51/51) 0% (0/51) 0% (0/51) VII. Appropriate Improvement Strategies 100% (20/20) 0% (0/20) 0% (0/20) VIII. Sufficient Data Analysis and Interpretation 90% (47/52) 6% (3/52) 4% (2/52) Study Outcomes IX. Real Improvement Achieved^ 67% (16/24) 21% (5/24) 13% (3/24) X. Sustained Improvement Achieved* 100% 0% 0% (1/1) (0/1) (0/1) Percentage Score of Applicable Evaluation Elements Met 95% (257/270) ^ The percentage total for this activity does not equal 100 percent due to rounding. * Only the Provider Satisfaction PIP had progressed to this phase in the review period and was assessed for sustained improvement. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-1 Health Services Advisory Group, Inc. FINDINGS Overall, 95 percent of the evaluation elements across all six PIPs received a score of Met. While Peach State's strong performance in the Study Design and Study Implementation phases indicated that each PIP was designed and implemented appropriately to measure outcomes and improvement, it was less successful in achieving the desired outcomes. The following subsections highlight HSAG's validation findings associated with each of the three PIP stages. Study Design Peach State met 100 percent of the requirements across all six PIPs for all four activities within the Study Design stage. Overall, Peach State designed scientifically sound studies that were supported by use of key research principles. The technical design of each PIP was sufficient to measure and monitor PIP outcomes associated with Peach State's improvement strategies. The solid design of the PIPs allowed the successful progression to the next stage of the PIP process. Study Implementation The CMO met 100 percent of the requirements across all six PIPs for all three activities within the Study Implementation phase. These findings suggest that Peach State accurately documented and executed the implementation of the study design, and established a robust process for identifying barriers and developing interventions. With the successful implementation of improvement strategies, the CMO could achieve improved outcomes. Study Outcomes Peach State met the requirements for two of the three activities for the Study Outcomes stage. The CMO correctly conducted analyses and interpreted its results as demonstrated in Activity VIII (i.e., Sufficient Data Analysis and Interpretation), with individual PIP scores ranging from 88 percent to 100 percent. However, as seen in Table 22 and Table 23, not all of the PIPs demonstrated statistically significant improvement related to Activity IX (i.e., Real Improvement Achieved). Individual PIP scores ranged from 50 percent to 100 percent. Consequently, the aggregated results for Activity IX across all six PIPs reflected this deficiency (67 percent of the evaluation elements received a Met score) even though the Member Satisfaction and Adults' Access to Care PIPs scored considerably higher (92 percent). To be successful, the PIPs must show real, or statistical, improvement in their study indicators. Only the Provider Satisfaction PIP had progressed to the point of reporting a second remeasurement period and demonstrated sustained improvement for all four study indicators. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-2 Health Services Advisory Group, Inc. FINDINGS PIP-Specific Outcomes Analysis of Results Table 22 and Table 23 display outcome data for Peach State's six PIPs. The CMO submitted Remeasurement 1 data for five of the PIPs and Remeasurement 2 data for the Provider Satisfaction PIP. Table 22HEDIS-based Performance Improvement Project Outcomes for Peach State Health Plan PIP #1--Lead Screening in Children PIP Study Indicator Baseline Period Remeasurement 1 Remeasurement 2 Sustained (1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10) Improvement The percentage of children 2 years of age who received one blood lead test (capillary or venous) on or before 57.2%^ 62.3% their second birthday. PIP #2--Childhood Immunizations The percentage of children who received the recommended vaccinations based on the Childhood 62.8%^ 67.6% Immunization Status--Combo 2 (4:3:1:2:3:1) guidelines. PIP #3--Well-Child Visits The percentage of children who had six or more well-child visits with a PCP during their first 15 months of 51.6%^ 52.3% life. PIP #4--Adults' Access to Care The percentage of members 2044 years of age who had an ambulatory 78.8% 84.3%* or preventive care visit. ^ Rates did not include the PeachCare for Kids population. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement. * Designates statistically significant improvement over the prior measurement period (p value < 0.05). Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-3 Health Services Advisory Group, Inc. FINDINGS Table 23Satisfaction-based Performance Improvement Project Outcomes for Peach State Health Plan PIP #5--Member Satisfaction PIP Study Indicator Baseline Period Remeasurement 1 Remeasurement 2 Sustained (3/13/095/31/09) (3/12/105/31/10) (3/1/115/31/11) Improvement 1) "Ease of getting appointment with a specialist" (Q26) 71.7% 71.8% 2) "Getting care, tests, or treatments necessary" (Q30) 79.9% 81.1% 3) "Getting information/help from customer service" (Q32) 68.5% 80.8%* 4) "Treated with courtesy and respect by customer service 86.4% 90.4% staff" (Q33) PIP #6--Provider Satisfaction PIP Study Indicator^ Baseline Period Remeasurement 1 Remeasurement 2 Sustained (8/1/0710/30/07) (11/1/082/28/09) (9/29/0910/27/09) Improvement 1) The percentage of providers answering "Excellent" or "Very Good" to Q5--"Timeliness to 15.8% 28.0%* 32.3% Yes answer questions and/or resolve problems." 2) Percentage of providers answering "Excellent" or "Very Good" to Q6--"Quality of the 14.2% 24.1%* 31.0%* Yes provider orientation process." 3) Percentage of providers answering "Excellent" or "Very Good" to Q18--"Health plan takes physician input and recommendations seriously." 10.7% 15.2% 24.5%* Yes 4) Percentage of providers answering "Excellent" or "Very Good" to Q34--"Accuracy of claims processing." 12.1% 16.0% 28.8%* Yes ^ Providers were requested to respond if they agreed with the statements regarding the CMO. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement. * Designates statistically significant improvement over the prior measurement period. Overall, the rates for every study indicator in all six PIPs increased during the most recent measurement period. The Adults' Access to Care PIP demonstrated statistically significant improvement from Baseline to Remeasurement 1 with the percentage of adult members that accessed ambulatory or preventive care increasing by 5.5 percentage points to 84.3 percent. Statistically significant improvement is the standard for assessing real improvement and supports the conclusion that the noted improvement is not due to chance. Although Peach State's performance improved, it Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-4 Health Services Advisory Group, Inc. FINDINGS remained 0.5 percentage points below the DCH target (84.8 percent) and fell between the national 2009 HEDIS Medicaid 50th and 75th percentiles (81.44 percent and 85.58 percent). Additionally, while the study indicator rates for three PIPs (i.e., Lead Screening in Children, Childhood Immunizations, and Well-Child Visits) increased from Baseline to Remeasurement 1, the increases were not statistically significant and were not considered real improvement. Further, the results for the first remeasurement period included the PeachCare for Kids population while the baseline results did not. As such, comparisons between the two measurement periods were limited and should be interpreted with caution. A more critical issue, however, was that the calendar year (CY) 2009 outcomes for all three PIPs remained below the DCH target rates (65.9 percent, 72.0 percent, and 65.4 percent, respectively). Furthermore, the study indicator rates for Lead Screening in Children and Well-Child Visits PIPs were below the national 2009 HEDIS Medicaid 50th percentile (70.21 percent and 60.52 percent, respectively) while the study indicator rate for Childhood Immunizations was below the national 2009 HEDIS Medicaid 25th percentile (68.45 percent). All indictor rates suggested an opportunity for improvement. While the four study indicator outcomes for the Member Satisfaction PIP increased from Baseline to Remeasurement 1, only the third study indicator (getting information/help from customer service) demonstrated a statistically significant increase of 12.3 percentage points. Additionally, three of the four study indicators of the Provider Satisfaction PIP improved significantly (quality of the provider orientation process, health plan takes physician input and recommendations seriously, and accuracy of claims processing). More importantly, all four study indicators of the Provider Satisfaction PIP demonstrated sustained improvement, having shown improvement between all measurement periods. Barriers/Interventions The identification of barriers through barrier analysis and the subsequent selection of appropriate interventions to address those barriers are necessary steps to improve outcomes. The CMO's choice of interventions, the combination of intervention types, and the sequence of implementation of the interventions are all essential to the CMO's overall success. Peach State implemented the same quality improvement strategy for three of its PIPs--i.e., Childhood Immunizations, Lead Screening in Children, and Well-Child Visits. The interventions focused on the Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) program and addressed all three outcomes for these PIPs. Peach State identified lack of member and provider awareness and reduced plan/provider communication as the primary barriers to the improvement of these measures. While the CMO documented numerous ongoing interventions for each of these PIPs, the CMO only implemented a few new interventions in 2008, which consisted of a member and provider newsletter article and a newly-revised provider manual. In January 2009, Peach State hosted "Health Check Days" in pediatric offices and high-volume provider offices to promote healthy Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-5 Health Services Advisory Group, Inc. FINDINGS living activities. Based on the first remeasurement results, these activities had a minimal affect on PIP outcomes. Peach State implemented most of its interventions during the second half of 2009. These strategies required more time to have an effect on the CY 2009 results. However, it is likely that these interventions will affect the CY 2010 rates. Some of the interventions included: Medical record stickers with required immunizations, well-child visits, and lead screenings sent to providers EPSDT-focused outreach to 733 network pediatricians HEDIS measurement criteria magnets and printed specifications and billing codes distributed to providers An enhanced EPSDT telephone message for members A Healthy Rewards Program established to provide incentives to members for health check visits Additionally, in January 2010 Peach State established a Provider Incentive Program based on HEDIS metrics (with initial implementation January 1, 2010). For the Well-Child Visits PIP, Peach State distributed the CMO Well-Child Collaborative newsletter and billing guide to providers in the first quarter of 2010. Additionally, in the second quarter of 2010, Peach State launched a HEDIS "SWAT" team to support providers in meeting HEDIS requirements through face-to-face interactions. Similarly, for the Adults' Access to Care PIP, Peach State identified both member and provider barriers but concentrated specifically one barrier--providers' lack of knowledge regarding HEDIS measure requirements. Subsequently, the CMO developed an intervention that included medical sticker reminders that contained key adult screening requirements. The Provider Incentive Program based on HEDIS metrics also included the Adults' Access to Care measure. For the Provider Satisfaction PIP, Peach State identified specific barriers, then implemented targeted interventions for each study outcome. The CMO attributed the improvement in outcomes to the following interventions: 1) Timeliness in answering questions and/or resolving problems--The CMO established time frames to respond to inquiries and created internal metrics to track and communicate those time frames to the provider. 2) Quality of the provider orientation process--The CMO distributed the toolkit on disc to current PCPs. 3) Taking physician input and recommendations seriously--Peach State created a Customer Service Form, which included recommendations received from physicians. The CMO evaluated the recommendations and responded to the physicians in a timely manner. 4) Accuracy of claims processing--Peach State created a scorecard to identify provider billing errors. After the scorecards were distributed, providers were educated and retrained weekly. Peach State identified specific barriers through brainstorming meetings, then developed interventions, a process the CMO attributed to Member Satisfaction PIP outcomes. For the study Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-6 Health Services Advisory Group, Inc. FINDINGS indicator (i.e., "getting care, tests, or treatments necessary"), the CMO increased its quality monitoring from 95 percent to 97 percent, enhanced the call scripts so that one member service representative could handle a call without needing to transfer the member, and aligned staff to specific call queues to accommodate English- and Spanish-speaking members. For the other three outcomes (i.e., "ease of getting an appointment with a specialist", "getting information/help from customer service", and "treated with courtesy and respect by customer service staff"), the CMO could not attribute its success to any specific intervention since the improvement was not statistically significant and was possibly due to chance. Overall, Peach State exhibited a strong understanding of the key steps necessary for ensuring improvement. All study results across the six PIPs demonstrated improvement; however, the improvement was not statistically significant for all study outcomes. The PIP validation process relies on an annual evaluation; however, CMOs should perform an interim evaluation of the results in addition to the formal annual evaluation. Evaluation of interim measurement results could assist the CMO in identifying and eliminating barriers that impede improvement. Furthermore, evaluation of the study outcomes would assist the CMO in determining if the interventions are having the desired effect or if modifications to current interventions or new interventions are necessary to improve results. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-7 Health Services Advisory Group, Inc. 3. Strengths for Peach State Health Plan Individual PIP Strengths The Adult's Access to Care PIP received a Met score for 100 percent of the evaluation elements in two of the three PIPs' validation stages--Study Design and Study Implementation. Peach State received a Met score for 92 percent of the evaluation elements in the Study Outcomes stage for this PIP. Furthermore, the study indicator outcome for the Adults' Access to Care PIP, which improved significantly from the baseline to the first remeasurement, reflected the effects of a strong quality strategy. Although Peach State's performance was 0.5 percentage points below the DCH target (84.8 percent), the CMO's current success could continue to improve the CMO's general performance on the Adults' Access to Preventive/Ambulatory Health Services HEDIS measure. For the Member Satisfaction PIP, the third study indicator demonstrated a statistically significant increase of more than 12 percentage points and represented an increase in member satisfaction with Peach State's customer service. Peach State was also successful in achieving real and sustained improvement for all four study indicator outcomes of the Provider Satisfaction PIP. This finding was Peach State's greatest strength. The improved outcomes illustrated providers' increased satisfaction with the CMO's timeliness of answering questions and/or resolving problems, the quality of the provider orientation process, the CMO's response to providers' input and recommendations, and the accuracy of claims processing. Moreover, Peach State's implementation and evaluation of targeted Provider Satisfaction interventions suggested that the CMO could translate its success in achieving real and sustained improvement to other PIP topics. Global Strengths Across all PIPs All six PIPs received an overall Met validation status, which represented an area of strength for Peach State and provided confidence in the technical aspects of the studies. The performance on these PIPs suggests a thorough understanding of the PIP Study Design stage and the development and implementation of appropriate interventions. The sound study design of the PIPs created the foundation for the CMO to progress to subsequent PIP stages--i.e., implementing improvement strategies and accurately assessing study outcomes. The CMO appeared to understand and appropriately conduct the sampling and data collection activities of the Study Implementation stage. These activities ensured that the studies properly defined and collected the necessary data to produce accurate study indicator rates. Additionally, Peach State appropriately documented improvement strategies, an activity which ensured that study outcomes could improve. Furthermore, in the Study Outcomes stage, the CMO properly analyzed and interpreted the outcome results. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 3-1 Health Services Advisory Group, Inc. 4. Opportunities for Improvement for Peach State Health Plan Individual PIPs The Childhood Immunizations, Lead Screening in Children, and Well-Child Visits PIPs had the lowest validation scores for the Study Outcomes stage (77 percent each); therefore, to improve study outcomes in the future, Peach State should focus on implementing new and/or enhanced quality strategies for these PIPs. The past and ongoing interventions have not yielded improved results. In reviewing Peach State's PIPs, HSAG identified that the CMO did not implement the majority of the new interventions in time to have a positive effect on the study outcomes. As a result, the CY 2009 outcomes for all three PIPs remained below the DCH target rates (72.0 percent, 65.9 percent, and 65.4 percent, respectively). Furthermore, the study indicator rates for the Lead Screening in Children and Well-Child Visits PIPs were below the 50th national 2009 HEDIS Medicaid percentile (70.21 percent and 60.52, respectively) while the study indicator rate for Childhood Immunizations was below the 25th national 2009 HEDIS Medicaid percentile (68.45 percent). To increase the measurable effects of its quality improvement activities, Peach State should ensure that the implementation of interventions occurs early enough in the measurement period to provide sufficient time for the outcomes to be affected and demonstrate improvement. Global Issues While Peach State exhibited a strong understanding of the key steps necessary for ensuring improvement, the execution of intervention strategies across the six PIPs was inconsistent. Peach State should plan and implement its improvement strategies more efficiently, providing enough time for the interventions to affect the study outcomes. Additionally, the CMO should analyze its data to determine if any subgroup within its population has a disproportionately lower rate that negatively affected the overall rates. This "drill-down" type of analysis should be conducted both before and after the implementation of any intervention. For example, Peach State should evaluate whether rates differ by geographic region, gender, race/ethnicity, age, etc. The CMO could then target its interventions to those subgroups with the lowest rates, allowing the implementation of more precise, concentrated interventions. The process of targeting interventions to the appropriate subgroups is more efficient and effective. The CMO should be mindful that the submission of PIPs for validation will be an annual activity without an opportunity to resubmit. Peach State should carefully complete all necessary documentation. The CMO should refer to the PIP Validation Tool and address all Points of Clarification and all Partially Met and Not Met scores in the FY 2012 submission. Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 4-1 Health Services Advisory Group, Inc. Appendix A. PIP-Specific Validation Scores for Peach State Health Plan Table A1--Peach State's FY 2011 PIP Performance Review Step Study Design I. Review the Selected Study Topic(s) II. Review the Study Question(s) III. Review the Selected Study Indicator(s) IV. Review the Identified Study Population Study Implementation V. Review Sampling Methods VI. Review Data Collection Procedures VII. Assess Improvement Strategies Study Outcomes VIII. Review Data Analysis and Study Results IX. Assess for Real Improvement X. Assess for Sustained Improvement Percentage Score for Applicable Evaluation Elements Met Percentage Score for Applicable Critical Elements Met Validation Status Lead Screening in Children 17/17 (100%) 6/6 (100%) 2/2 (100%) 6/6 (100%) 3/3 (100%) 19/19 (100%) 6/6 (100%) 10/10 (100%) 3/3 (100%) 10/13 (77%) 8/9 (89%) 2/4 (50%) Not Assessed 94% 100% Met Childhood Immunizations 17/17 (100%) 6/6 (100%) 2/2 (100%) Well-Child Visits 17/17 (100%) 6/6 (100%) 2/2 (100%) Adults' Access to Care 16/16 (100%) 5/5 (100%) 2/2 (100%) Member Satisfaction 16/16 (100%) 5/5 (100%) 2/2 (100%) Provider Satisfaction 15/15 (100%) 4/4 (100%) 2/2 (100%) 6/6 (100%) 6/6 (100%) 6/6 (100%) 6/6 (100%) 6/6 (100%) 3/3 (100%) 19/19 (100%) 6/6 (100%) 10/10 (100%) 3/3 (100%) 10/13 (77%) 8/9 (89%) 2/4 (50%) Not Assessed 94% 3/3 (100%) 19/19(100%) 6/6 (100%) 10/10 (100%) 3/3 (100%) 10/13 (77%) 8/9 (89%) 2/4 (50%) Not Assessed 94% 3/3 (100%) 8/8 (100%) 0/0 5/5 (100%) 3/3 (100%) 11/12 (92%) 7/8 (88%) 4/4 (100%) Not Assessed 97% 3/3 (100%) 3/3 (100%) 18/18 (100%) 6/6 (100%) 8/8 (100%) 4/4 (100%) 12/13 (92%) 12/12 (100%) 0/0 8/8 (100%) 4/4 (100%) 11/13 (85%) 9/9 (100%) 7/8 (88%) 3/4 (75%) Not Assessed 3/4 (75%) 1/1 (100%) 98% 95% 100% Met 100% Met 100% Met 100% Met 100% Met Peach State Health Plan FY 2011 PIP Validation Report State of Georgia PeachState_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page A-1 Health Services Advisory Group, Inc.