Peach State Health Plan: performance improvement projects report SFY 2014 [Nov. 2013]

State of Georgia
Department of Community Health Georgia Families Program
Peach State Health Plan
PERFORMANCE IMPROVEMENT PROJECTS REPORT
SFY 2014
November 2013

TABLE OF CONTENTS
1. Background....................................................................................................................... 1-1 CMO Overview ...............................................................................................................................1-3 Study Rationale ..............................................................................................................................1-3 Study Summary ..............................................................................................................................1-3 Validation Overview........................................................................................................................1-5 HSAG's Validation Scoring Methodology ........................................................................................1-7
2. Findings ............................................................................................................................ 2-1 Aggregate Validation Findings ........................................................................................................2-1 Design..........................................................................................................................................2-3 Implementation ............................................................................................................................. 2-3 Outcomes ..................................................................................................................................... 2-3 PIP-Specific Outcomes...................................................................................................................2-4 Analysis of Results .......................................................................................................................2-4 Adolescent Well-Care...................................................................................................................2-4 Annual Dental Visits .....................................................................................................................2-6 Appropriate Use of ADHD Medications.........................................................................................2-7 Childhood Immunizations--Combo 10 .........................................................................................2-9 Childhood Obesity ......................................................................................................................2-10 Comprehensive Diabetes Care...................................................................................................2-12 Avoidable Emergency Room Visits.............................................................................................2-13 Member Satisfaction...................................................................................................................2-16 Provider Satisfaction ..................................................................................................................2-16
3. Conclusions and Recommendations..............................................................................3-1 Conclusions ....................................................................................................................................3-1 Recommendations .......................................................................................................................... 3-1
Appendix A. PIP-Specific Validation Results .................................................................... A-1

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Health Services Advisory Group, Inc.

Page ii

ACKNOWLEDGMENTS AND COPYRIGHTS
CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA).

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Health Services Advisory Group, Inc.

Page ii

Performance Improvement Project Validation Report Peach State Health Plan

1. BACKGROUND
The Georgia Department of Community Health (DCH) through its Division of Medical Assistance Plans is responsible for administering the Medicaid program and the Children's Health Insurance Program (CHIP) for the State of Georgia and overseeing quality improvement activities. The State refers to its Medicaid managed care program as Georgia Families and to its CHIP program as PeachCare for Kids. For the purposes of this report, "Georgia Families" refers to all Medicaid and PeachCare for Kids members enrolled in managed care.
The Georgia Families program serves the majority of Georgia's Medicaid and CHIP populations. The DCH requires its contracted Care Management Organizations (CMOs) serving members under Georgia Families to conduct performance improvement projects (PIPs) as set forth in 42 CFR 438.240 to assess and improve the quality of targeted areas of clinical or nonclinical care or service provided to members, and to report the status and results of each PIP annually. Peach State Health Plan (Peach State) is one of the Georgia Families CMOs.
The validation of PIPs is one of three federally-mandated activities for state Medicaid managed care programs. The other two required activities include the evaluation of CMO compliance with State and federal regulations and the validation of CMO performance measures.
These three mandatory activities work together to assess the CMOs' performance with providing appropriate access to high-quality care for their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measure rates provide a barometer of the quality and effectiveness of the care. The DCH requires the CMOs to initiate PIPs to improve the quality of health care in targeted areas of low performance, or in areas identified as State priorities or health care issues of greatest concern. The DCH required its CMOs to conduct nine PIP studies during the 2012 calendar year and submit them for validation in 2013. PIPs are key tools in helping DCH achieve goals and objectives outlined in its quality strategy; they provide the framework for monitoring, measuring, and improving the delivery of health care.
The primary objective of PIP validation is to determine each CMO's compliance with requirements set forth in 42 CFR 438.240(b)(1), including:
Measurement of performance using objective quality indicators. Implementation of system interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities to increase or sustain improvement.
To meet the federal requirement for the validation of PIPs, DCH contracted with Health Services Advisory Group, Inc. (HSAG), the State's external quality review organization (EQRO), to conduct the validation of Peach State's PIPs. Peach State submitted PIPs to HSAG between June

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-1
Health Services Advisory Group, Inc.

BACKGROUND

30, 2013, and August 1, 2013, and HSAG validated the PIPs between July 1, 2013, and August 8, 2013. The validated data represent varying measurement time periods as described in Table 2-3 through Table 2-11.
For PIPs initiated prior to January 1, 2012 (Annual Dental Visits and Childhood Obesity), HSAG reviewed the PIPs using the Centers for Medicare & Medicaid Services (CMS) validation protocols.11 For PIPs initiated on or after January 1, 2012 (Adolescent Well-Care Visits, Appropriate Use of ADHD Medications, Avoidable Emergency Room Visits [Collaborative], Childhood Immunizations-- Combo 10, Comprehensive Diabetes Care, Member Satisfaction and Provider Satisfaction), HSAG used CMS' updated validation protocols.1-2 Compared to the 2002 CMS PIP protocols, the changes made to the 2012 protocols consisted of reversing the order of Activities III and IV, and Activities VII and VIII. These changes did not impact HSAG's validation process.

Table 1-1--CMS Protocol Changes

PIP Activity Clearly Defined Study Indicator(s) Correctly Identified Study Population Appropriate Improvement Strategies Sufficient Data Analysis and Interpretation of Results

CMS 2002 Protocol Activity III Activity IV Activity VII Activity VIII

CMS 2012 Protocol Activity IV Activity III Activity VIII Activity VII

HSAG evaluated two key components of the quality improvement process, as follows:
1. HSAG evaluated the technical structure of the PIPs to ensure Peach State designed, conducted, and reported PIPs using sound methodology consistent with the CMS protocol for conducting PIPs. HSAG's review determined whether a PIP could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring real and sustained improvement.
2. HSAG evaluated the outcomes of the PIPs. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. Evaluation of each PIP's outcomes determined whether Peach State improved its rates through the implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results) and, through these processes, achieved statistically significant improvement over the baseline rate. Once statistically significant improvement is achieved across all study indicators, HSAG evaluates whether Peach State was successful in sustaining the improvement. A primary goal of HSAG's PIP validation is to ensure that DCH and key stakeholders can have confidence that reported improvement in study indicator outcomes is supported by statistically significant change and the CMO's improvement strategies.

1-1 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Managed Care Organization Protocol. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, Final Protocol, Version 1.0, May 2002.
1-2 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Protocol 3: Validating Performance Improvement Projects (PIPs): A Mandatory Protocol for External Quality Review (EQR), Version 2.0, September 2012.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-2
Health Services Advisory Group, Inc.

BACKGROUND

CMO Overview
The DCH contracted with Peach State beginning in 2006 to provide services to the Georgia Families program population. Prior to 2012, Peach State served the eligible populations in the Atlanta, Central, and Southwest geographic regions of Georgia. In early 2012, the CMO expanded coverage statewide and added the North, East, and Southeast regions. The HEDIS technical specifications that Peach State used for its PIP indicators require a member to be continuously enrolled with the CMO. While the new population was included in the PIPs' interventions, the measurement of the PIPs' effectiveness (the PIPs' indicator results) excluded members who did not meet the indicators' continuous enrollment criteria.

Study Rationale
The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas. Although HSAG has validated Peach State's PIPs for six years, the number of PIPs, study topics, and study methods has evolved over time.
Peach State submitted nine PIPs for validation. The PIP topics included:
Adolescent Well-Care Visits Annual Dental Visits Appropriate Use of ADHD Medications Avoidable Emergency Room Visits (Collaborative) Childhood Immunizations--Combo 10 Childhood Obesity Comprehensive Diabetes Care Member Satisfaction Provider Satisfaction

Study Summary
As noted in its Quality Strategic Plan Update (November 2011), DCH identified the improvement and enhancement of the quality of patient care provided through ongoing, objective, and systematic measurement, analysis, and improvement of performance as one of its four performance-driven goals. The goals are designed to demonstrate success or identify challenges in achieving intended outcomes related to providing quality, accessible, and timely services. Peach State's June 30, 2013, through August 1, 2013, PIP submissions included six clinical HEDIS-based PIPs: Adolescent Well-Care Visits, Annual Dental Visits, Appropriate Use of ADHD Medications, Avoidable Emergency Room Visits, Childhood Immunizations--Combo 10, Childhood Obesity, Comprehensive Diabetes Care, two nonclinical PIPs: Member Satisfaction and Provider Satisfaction, and one collaborative Avoidable Emergency Room Visits PIP.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-3
Health Services Advisory Group, Inc.

BACKGROUND

Table 1-2 outlines the key study indicators incorporated for the six clinical HEDIS-based PIPs.

PIP Study Topic
Adolescent WellCare Visits Annual Dental Visits
Appropriate Use of ADHD Medications
Childhood Immunizations-- Combo 10
Childhood Obesity Comprehensive Diabetes Care

Table 1-2--PIP Study Topics and Indicator Descriptions
PIP Study Indicator Description
The percentage of members 1221 years of age who had at least one comprehensive well-care visit with a PCP or an OB/GYN practitioner during the measurement year.
The percentage of members 23 years of age and 221 years of age who had at least one dental visit during the measurement year.
1. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who had one follow-up visit with a practitioner with prescribing authority during the 30-day Initiation Phase.
2. The percentage of members 612 years of age as of the Index Prescription Start Date (IPSD) with an ambulatory prescription dispensed for ADHD medication, who remained on the medication for at least 210 days and who, in addition to the visit in the Initiation Phase, had at least two follow-up visits with a practitioner from 31300 days following the IPSD. One of the two visits (during days 31300) may be a telephone visit with a practitioner.
The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis (DTaP); three polio (IPV); one measles, mumps, rubella (MMR); three H influenza type B (HiB); three hepatitis B (HepB); one chicken pox (VZV); four pneumococcal conjugate (PCV); one hepatitis A (HepA); two or three rotavirus (RV); and two influenza (flu) vaccines by their second birthday.
The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation, nutrition counseling and physical activity counseling.
The percentage of members 1875 years of age with diabetes (type 1 and type 2) who had a HbA1c control < 7.0%, LDL-C control < 100mg/ml, and BP control < 140/90 mmHg.

Table 1-3 outlines the key study indicators for the collaborative Avoidable Emergency Room Visits PIP.

PIP Study Topic
Avoidable Emergency Room Visits

Table 1-3--Collaborative PIP Study Topic and Indicator Descriptions
PIP Study Indicator Description
1. The percentage of practices that provide the same day appointments for routine and urgent care.
2. The percentage of practices that provide routine and urgent care appointments after hours. 3. The percentage of practices that provide appointments for routine and urgent care after hours
and have the ability to document after hours clinical advice in the patient's record. 4. The percentage of practices that have access to and utilize electronic health records. 5. The percentage of practices that receive information regarding ER visits from the study
hospitals. 6. The percentage of ER visits for `avoidable' diagnoses (dx382Acute Suppurative otitis:382.9
Unspecified otitis:462Acute pharyngitis:465.9Acute upper respiratory infection:466Acute bronchitis:786.2Cough) among members under 21 years of age who had a visit to the ED in three selected Children's Healthcare of Atlanta facilities in the Atlanta region.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-4
Health Services Advisory Group, Inc.

BACKGROUND

Table 1-4 outlines the key study indicators incorporated for the two satisfaction-based PIPs.

The effectiveness of the Member Satisfaction PIP was measured using the Consumer Assessment of Healthcare Providers and Systems (CAHPS) 5.0H, Medicaid Child Survey. This survey provided information on parents' experiences with their child's provider and CMO.

The final Peach State PIP topic was Provider Satisfaction. Peach State contracted with a vendor to produce and administer a survey to document the effectiveness of this performance improvement project.

Survey Type Member Provider

Table 1-4--Satisfaction-Based PIP Study Indicators

Question #36 #48

Survey Question
The percentage of respondents who rate the health plan an 8, 9, or 10 in response to the question "Using any number from 010, where 0 is the worst health plan and 10 is the best, what number would you use to rate your child's health plan?"
The percentage of providers who respond "very satisfied" or "somewhat satisfied" to the question "Please rate your overall satisfaction with Peach State Health Plan."

Validation Overview
HSAG obtained the data needed to conduct the PIP validations from Peach State's PIP Summary Forms. These forms provided detailed information about Peach State's completed PIP activities.
Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be scored Met. Given the importance of critical elements to the scoring methodology, any critical element that received a Not Met score resulted in an overall validation status for the PIP of Not Met. A CMO would be given a Partially Met validation status if 60 percent to 79 percent of all evaluation elements were scored Met or one or more critical elements were scored Partially Met. HSAG provided a Point of Clarification when the CMO fully met the evaluation element criteria and only minor documentation edits not critical to the validity of the PIP were recommended to the CMO.
In addition to the overall validation status (e.g., Met), HSAG provided an overall percentage for all evaluation elements (including critical elements) scored Met. HSAG calculated the overall percentage by dividing the total number of elements scored Met by the total number of elements scored Met, Partially Met, and Not Met. HSAG also calculated a critical element overall percentage score by dividing the total number of critical elements scored Met by the sum of the critical elements scored Met, Partially Met, and Not Met.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-5
Health Services Advisory Group, Inc.

BACKGROUND
Figure 1-1 illustrates the three study stages of the PIP process: Design, Implementation, and Outcomes. The Design stage establishes the methodological framework for the PIP. The activities in this stage include development and documentation of the study topic, question, indicators, population, sampling, and data collection. A sound study design is necessary for the successful implementation of improvement strategies.
Once the study design is established, the PIP process moves into the Implementation stage. This stage includes data analysis and implementation of improvement strategies. During the Implementation stage, CMOs should incorporate a continuous or rapid cycle improvement model such as the Plan-Do-Study-Act (PDSA) Cycle.
Figure 1-1--PIP Study Stages Incorporating the PDSA Cycle
Outcomes

Design
The PDSA cycle includes the following actions:
Plan--conduct barrier analyses; prioritize barriers; develop targeted intervention(s) to address barriers; and develop an intervention evaluation plan for each intervention
Do--implement intervention; track and monitor the intervention; and record the data Study--analyze the data; compare results; and evaluate the intervention's effectiveness Act--based on the evaluation results, standardize, modify, or discontinue the intervention
The PDSA cycle is repeated throughout each measurement period. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final Outcomes stage evaluates for statistically significant and sustained improvement of the project outcomes. Once statistically significant improvement in the outcomes is achieved, the improvement must be sustained in a subsequent measurement period. If the study outcomes do not improve, the CMO's responsibility is to continue the PDSA cycle until statistically significant improvement is achieved and sustained.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-6
Health Services Advisory Group, Inc.

BACKGROUND
HSAG's Validation Scoring Methodology
The scoring methodology evaluates whether or not the CMO met all the documentation requirements according to the CMS protocols, as well as evaluates whether or not all study indicators have achieved statistically significant improvement over the baseline rate. In Activity IX (real improvement achieved), the CMO must achieve statistically significant improvement across all study indicator(s) between the baseline and a subsequent measurement period to receive a Met score. For Activity X (sustained improvement achieved), HSAG assesses for sustained improvement once all study indicators achieve statistically significant improvement over the baseline and the CMO reports a subsequent measurement period. All study indicators must achieve statistically significant improvement and sustain this improvement to receive a Met validation score in Activity X.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 1-7
Health Services Advisory Group, Inc.

2. FINDINGS
for Peach State Health Plan

Aggregate Validation Findings

HSAG organized, aggregated, and analyzed Peach State's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of interventions. Based on its review, HSAG determined the overall methodological validity of the PIPs, as well as the overall success in achieving improved study indicator outcomes. The results are presented in Table 2-1.

Table 2-1--Performance Improvement Project Validation Scores for Peach State Health Plan

PIP
Adolescent Well-Care Visits Annual Dental Visits Appropriate Use of ADHD Medication Avoidable Emergency Room Visits--Collaborative Childhood Immunization-- Combo 10 Childhood Obesity Comprehensive Diabetes Care Member Satisfaction Provider Satisfaction

Percentage of Evaluation Elements Scored Met 95% 92%
84%
62%
98%
90% 84% 100% 100%

Percentage of Critical Elements Scored Met
82% 92%
82%
50%
93%
86% 79% 100% 100%

Validation Status Partially Met Partially Met Not Met
Not Met
Partially Met Partially Met
Not Met Met Met

Two of the nine PIPs validated received an overall Met validation status. The Adolescent WellCare Visits, Annual Dental Visits, Childhood Immunization--Combo 10, and Childhood Obesity PIPs received a Partially Met validation status due to lack of information about the causal/barrier analysis process or tools used to identify barriers and interventions. While all four PIPs described the committee involved in the causal/barrier analysis, and brainstorming was mentioned as a strategy in the Childhood Obesity and Annual Dental Visits PIPs, none of the PIPs documented a process for using data to drive barrier identification. In addition to deficiencies in the Appropriate Improvement Strategies activity, the Adolescent Well-Care Visits and Childhood Obesity PIPs also incorrectly reported some of the indicator rates in Activity IX.
The Appropriate Use of ADHD Medication, Avoidable Emergency Room Visits, and Comprehensive Diabetes Care PIPs received an overall Not Met validation status. None of these study indicators achieved statistically significant improvement over baseline. The collaborative Avoidable Emergency Room Visits PIP received a Not Met validation status for several reasons. The CMOs did not completely define the study population or the study indicators, or explain

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-1
Health Services Advisory Group, Inc.

FINDINGS

how the data were collected for all study indicators. Additionally, not all study indicators achieved statistically significant improvement over the baseline rates.

Table 2-2 displays the combined validation results for all nine Peach State PIPs validated during SFY 2014. This table illustrates the CMO's application of the PIP process and its success in implementing all nine projects. Each activity was composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score satisfied the necessary technical requirements for a specific element. The validation results presented in Table 2-2 show the percentage of applicable evaluation elements that received a Met score by activity. Additionally, HSAG calculated an overall percentage of Met scores across all activities for all nine PIPs. Appendix A provides the detailed scores from the validation tool for each of the nine PIPs.

Table 2-2--Performance Improvement Project Validation Results for Peach State Health Plan (N=9 PIPs)

Study Stage

Activity

Appropriate Study Topic

Clearly Defined, Answerable Study Question(s)

Design

Clearly Defined Study Indicator(s) Correctly Identified Study Population

Valid Sampling Techniques (if sampling was used)

Accurate/Complete Data Collection

Design Total*

Sufficient Data Analysis and Interpretation of Results Implementation
Appropriate Improvement Strategies

Implementation Total*

Outcomes

Real Improvement Achieved Sustained Improvement Achieved

Outcomes Total

Percentage of Applicable Evaluation Elements Scored Met * Percentages do not total 100 percent due to rounding.

Percentage of Applicable Elements

Met

Partially Met Not Met

100% (51/51)

0% (0/51)

0% (0/51)

100% (18/18)

0% (0/18)

0% (0/18)

96% (53/55)

4% (2/55)

0% (0/55)

92% (23/25

8% (2/25)

0% (0/25)

100% (30/30)

0% (0/30)

0% (0/30)

90% (66/73)

3% (2/73)

7% (5/73)

96% (241/252)

2% (6/252)

2% (5/252)

88% (61/69)

12% (8/69)

0% (0/69)

50% (12/24)

50% (12/24)

0% (0/24)

79 (73/93)

22 (20/93)

0% (0/93)

64% (18/28)

11% (3/28)

25% (7/28)

100%

0%

0%

(1/1)

(0/1)

(0/1)

66% (19/29)

10% (3/29)

24% (7/29)

89% (333/374)

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-2
Health Services Advisory Group, Inc.

FINDINGS
Overall, 89 percent of the evaluation elements across all nine PIPs received a Met score. The 89 percent score demonstrates a sound application of the PIP process. While Peach State's strong performance in the Design stage, with the exception of its Avoidable Emergency Room Visits PIP, indicated that each PIP was designed appropriately to measure outcomes and improvement, Peach State was less successful in the Implementation and Outcomes stages. The following subsections highlight HSAG's validation findings associated with each of the three PIP stages.
Design
Peach State met 96 percent of the requirements across all nine PIPs for the six activities within the Design stage. With the exception of the Avoidable Emergency Room Visits PIP, the technical design of each PIP was sufficient to measure and monitor PIP outcomes. The solid foundation of the PIPs allowed for the CMO to progress to the next stage of the PIP process.
Implementation
Peach State met 79 percent of the requirements for the two activities within the Implementation stage. The CMO did not report accurate data components in some of its PIPs. Another deficiency in the Sufficient Data Analysis and Interpretation of Results activity was the CMO's failure to address all documentation requirements. For the Appropriate Improvement Strategies activity, some of the PIPs lacked sufficient information about the causal/barrier analysis process used to identify barriers and interventions. While the PIPs generally described the committee involved in the process and some of the PIPs reported using brainstorming as a process, most of the PIPs did not provide additional detail about how data were used to identify barriers or link interventions to barriers and outcomes.
Outcomes
This year, six PIPs (Adolescent Well-Care Visits, Appropriate Use of ADHD Medication, Childhood Immunizations--Combo 10, Childhood Obesity, Comprehensive Diabetes Care, and Avoidable Emergency Room Visits) were evaluated for achieving statistically significant improvement. Two PIPs, Adolescent Well-Care Visits and Childhood Immunizations--Combo 10, achieved statistically significant improvement over baseline for all indicators at Remeasurement 1. Only one PIP, Annual Dental Visits, progressed to the point of being assessed for sustained improvement. Sustained improvement is defined as statistically significant improvement in performance over baseline that is maintained or increased for at least one subsequent measurement period. Additionally, the results of the most current measurement period must reflect improvement when compared to baseline results. Both study indicators in the Annual Dental Visits PIP achieved sustained improvement.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-3
Health Services Advisory Group, Inc.

FINDINGS

PIP-Specific Outcomes

Analysis of Results
Each table below displays the study indicator rates for each measurement period of the PIP, including the baseline period and each subsequent remeasurement period. Statistically significant changes between remeasurement periods are noted with an upward or downward arrow followed by an asterisk. If the PIP achieved statistically significant improvement over the baseline rate, it was then reviewed for sustained improvement. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators. PIPs that did not achieve statistically significant improvement (i.e., did not meet the criteria to be assessed for sustained improvement) were not assessed (NA). Comparisons of PIP study indicator results that utilized HEDIS measures were made using the Medicaid HEDIS 2011 Audit, Means, Percentiles, and Ratios (reflecting the 2010 calendar year [CY]).

Adolescent Well-Care

Table 2-3--Performance Improvement Project Outcomes for Adolescent Well-Care Visits

PIP Study Indicator

Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^

The percentage of members 1221 years of age who had at

least one comprehensive well-care visit with a PCP or an

38.5%

39.1%*

NA

OB/GYN practitioner during the measurement year.

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.

* Designates statistically significant improvement over the prior measurement period (p value < 0.05).
^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

In the first remeasurement period of the Adolescent Well-Care Visits PIP, Peach State achieved statistically significant improvement in the rate of members 1221 years of age who had at least one well-care visit during the measurement year. The Remeasurement 1 rate of 39.1 percent is still, however, below the CY 2012 DCH target of 46.8 percent and below the 25th percentile (39.6 percent) of national Medicaid HEDIS 2011 rates. The rates reported for this PIP were based on administrative data.

A critical analysis of the CMO's improvement strategies for this PIP demonstrated the following:

Peach State identified barriers to improving the Adolescent Well-Care Visits indicator rate through monthly and quarterly analysis of data by an interdisciplinary HEDIS steering committee. The committee determined that there were no significant geographic or member age variations in the study indicator rate; therefore, interventions were implemented in a

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-4
Health Services Advisory Group, Inc.

FINDINGS
standardized fashion across the State. The CMO did not document the tools used to determine the barriers/interventions in the PIP (e.g., brainstorming, fishbone diagram). Additionally, the CMO did not continue its analysis to identify barriers that were affecting the study outcome.
Three types of barriers were identified: member, provider, and system. However, the CMO did not prioritize the barriers. Additionally, the CMO did not provide any specific results of the barrier analysis or any data-driven rationale for the selection of the interventions.
Interventions addressing members and providers were implemented during the baseline period while all three types of interventions (member, provider, and system) were implemented during Remeasurement 1. Peach State documented that it placed outreach calls to non-compliant members in need of an adolescent well-child visit. Some of the other interventions that were implemented are listed below. Implemented a provider bonus program based on the provider successfully contacting non-compliant members and providing them with well-child visits. The CMO did not, however, track or monitor the intervention. Educated providers using a "tip sheet" on conducting well-child assessments during sick visits/sports physicals. The CMO did not, however, follow up with the providers that received this education to determine if it affected the study indicator rates. Implemented CareGaps, an internal system alert to let Peach State employees and members (secure portal) know about members who are due or past due for preventive services. Again, there was no indication of how many members actually accessed the portal or how employees acted on the information to improve the rate of well-care visits. To address member barriers, Peach State called members, scheduled appointments, placed reminder calls, and facilitated non-emergency transportation to appointments. The CMO did not provide documentation related to the evaluation of these interventions. For example, the CMO did not track how many members were called, how many made appointments, and how many of those members kept the appointment. Peach State documented that provider turnover was a barrier necessitating continued training and education of the provider network. Peach State partnered with its medical record review vendor to extend provider education through "tip sheets" and face-to face meetings, communicating that preventive care could be performed during a sick visit or sports physical. The CMO did not evaluate the effectiveness of this intervention even though it required considerable resources.
Peach State documented that the implemented interventions have caused the statistically significant improvement reported. However, the CMO did not provide any data to support this documentation. HSAG anticipated that the CMO would have documented a data-driven process that monitored the interventions and measured the study outcomes for the targeted population. HSAG encourages Peach State to have processes in place to evaluate the effectiveness for each of its interventions.
With the implementation of any intervention (and especially for multiple interventions), the CMO must ensure that each intervention includes an evaluation plan. Without a method to evaluate the effectiveness of each intervention, the CMO cannot determine which intervention to

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-5
Health Services Advisory Group, Inc.

FINDINGS

modify or discontinue, or when to implement new interventions, thereby reducing the likelihood of achieving project objectives and improving performance.

Annual Dental Visits

Table 2-4--Performance Improvement Project Outcomes for Annual Dental Visits

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Remeasurement 3 Sustained (1/1/0912/31/09) (1/1/1012/31/10) (1/1/1112/31/11) (1/1/1212/31/12) Improvement^

The percentage

of members 23

years of age who

33.8%

38.8%*

43.9%*

44.0%

Yes

had at least one

dental visit.

The percentage

of members 221

years of age who

60.2%

63.6%*

67.5%*

67.9%*

Yes

had at least one

dental visit.

* Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators
that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

Peach State sustained statistically significant improvement at Remeasurement 3 in the Annual Dental Visits PIP. The CY2012 rates for both study indicators continued to demonstrate significant and real improvement over baseline rates. Furthermore, the rate for Study Indicator 2 (members 221 years of age) exceeded the CY 2012 DCH target rate of 64.1 percent and the national Medicaid HEDIS 2011 90th percentile of 64.5 percent.

Peach State implemented the following interventions:

Peach State's interdisciplinary HEDIS steering committee reviewed data monthly and used brainstorming to identify barriers and interventions for the baseline period. While barriers and interventions were documented for Remeasurement 3, the CMO did not document the tools used to determine the barriers/interventions (e.g., brainstorming, fishbone diagram). Additionally, the CMO did not provide any specific results of the barrier analysis or any data-driven rationale for the selection of the interventions.
Although no new barriers were identified for the most recent measurement period, the CMO identified three interventions that were implemented during Remeasurement 3 in addition to continuing several interventions dating back to 2006 without providing any rationale for its decisions.
Peach State implemented a provider-based intervention, "Preventistry Provider Sealant Program," to help prevent damage to tooth enamel. It was unclear to HSAG how this intervention would increase the percentage of members receiving an annual dental exam. The

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-6
Health Services Advisory Group, Inc.

FINDINGS

barrier for this intervention was documented as "Provider education to enhance knowledge of cost-effective preventive therapies," which did not directly link to the study outcome.
In addition to the above-mentioned intervention, Peach State implemented a revised "Mobile Van" program by adding the "Safety Net" program. This program includes sending a mobile van to area schools so that dental exams can be performed, scheduling appointments for dental exams, educating members on the importance of recommended dental visits, and assisting with transportation, if needed. The CMO did not document the effects of this intervention.
Peach State documented that it discontinued the 2011 provider incentive program, stating that analysis showed the program did not directly increase the study indicator rates. The CMO provided no data analysis to support this conclusion.
Peach State documented that it believed the above interventions had caused the reported improvement in the study indicator rates. However, the CMO did not provide any data to support this assertion. HSAG anticipated that the CMO would have documented a data-driven process that linked the interventions to the study indicator outcomes. For example, Peach State could indicate the number of members who received a dental exam as a result of its Mobile Van program. HSAG encourages Peach State to have processes in place that evaluate the effectiveness for all of its implemented interventions.

Appropriate Use of ADHD Medications

Table 2-5--Performance Improvement Project Outcomes for Appropriate Use of ADHD Medications

PIP Study Indicator

Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^

1. The percentage of members 612 years of age as of the

Index Prescription Start Date (IPSD) with an ambulatory

prescription dispensed for ADHD medication, who had

43.7%

43.7%

NA

one follow-up visit with a practitioner with prescribing

authority during the 30-day Initiation Phase.

2. The percentage of members 612 years of age as of the

Index Prescription Start Date (IPSD) with an ambulatory

prescription dispensed for ADHD medication, who

remained on the medication for at least 210 days and who, in addition to the visit in the Initiation Phase, had at least

57.4%

58.6%

NA

two follow-up visits with a practitioner from 31300 days

following the IPSD. One of the two visits (during days 31

300) may be a telephone visit with a practitioner.

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.

^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-7
Health Services Advisory Group, Inc.

FINDINGS
Neither study indicator in the Appropriate Use of ADHD Medications PIP achieved statistically significant improvement from baseline to Remeasurement 1. Though the rate of follow-up care visits for children newly prescribed ADHD medication did not decline for either study indicator, the rate of follow-up visits during the initiation phase (Study Indicator 1) remained constant at 43.7 percent, and there was only a non-significant increase of 1.2 percentage points in the rate of follow-up visits during the continuation and maintenance phases (Study Indicator 2). The Remeasurement 1 rates for both indicators fell below the CY 2012 DCH targets of 48.1 percent (initiation) and 57.6 percent (continuation), respectively. In comparison with the national Medicaid HEDIS 2011 rates, Peach State's CY 2012 rates were better than the corresponding 75th percentile rates of 43.6 percent (initiation phase) and 52.6 percent (continuation phase). The following paragraphs describe interventions implemented by Peach State.
Peach State's HEDIS Steering Committee, a multidisciplinary quality improvement council, identified barriers to improving the Appropriate Use of ADHD Medications indicator rates through monthly and quarterly analysis of data. While the CMO noted that a causal/barrier analysis was completed and reported barriers and interventions, the documentation did not include the tool(s) used to link barriers and interventions. The documentation also lacked specific data to support the prioritization of identified barriers.
Interventions implemented at the institutional, provider, and member levels included:
Implementation of a Clinical Practice Guideline (CPG) compliance program. Initiation of a Quality Improvement and Public Relations collaboration to educate behavioral
health providers on HEDIS measures and the ADHD CPG. Peach State Days--targeting non-compliant members with appointment scheduling,
transportation assistance, and nominal incentives. Pharmacy Liaison education visits to non-psychiatric practitioners with high-volume ADHD
prescriptions.
Despite a lack of significant improvement in the study indicators, Peach State's HEDIS Steering Committee identified the CPG compliance program and the Quality Improvement--Public Relations collaboration as the most effective interventions. It was unclear, however, what data or process was used to identify these as effective interventions. The committee also recommended additional member outreach interventions to further improve the rate of appropriate ADHD medication follow-up visits in future measurement periods. Again, it was not clear what data supported the recommendation to prioritize additional member outreach interventions for implementation.
The CMO reported that it would be pursuing a more in-depth causal/barrier analysis in CY 2013 to identify increasingly effective interventions. The causal/barrier analysis process should include clear documentation of the data-driven tools and processes used to identify and link barriers and interventions. Additionally, HSAG recommends that Peach State implement a process to evaluate the effectiveness of each intervention's impact on the study indicator rates.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-8
Health Services Advisory Group, Inc.

FINDINGS

Childhood Immunizations--Combo 10

Table 2-6--Performance Improvement Project Outcomes for Childhood Immunizations--Combo 10

PIP Study Indicator

Baseline Period Remeasurement 1

Sustained

(1/1/1112/31/11) (1/1/1212/31/12) Improvement^

The percentage of children 2 years of age who had

four diphtheria, tetanus and acellular pertussis

(DTaP); three polio (IPV); one measles, mumps,

rubella (MMR); three H influenza type B (HiB); three hepatitis B (HepB); one chicken pox (VZV);

17.6%

27.9%*

NA

four pneumococcal conjugate (PCV); one hepatitis A

(HepA); two or three rotavirus (RV); and two

influenza (flu) vaccines by their second birthday.

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.
* Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators
that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

For the Childhood Immunizations--Combo10 PIP, Peach State achieved statistically significant improvement over baseline at Remeasurement 1, with an increase of 10.3 percentage points in the rate of eligible child members who had received all necessary immunizations by their second birthday. The Remeasurement 1 rate also surpassed the national Medicaid HEDIS 2011 90th percentile of 23.6 percent.

Peach State identified barriers to improving the Childhood Immunizations--Combo 10 indicator rate through monthly and quarterly analysis of data by a multidisciplinary quality improvement committee. The committee determined that there were no significant geographic or member age variations in the study indicator rate; therefore, interventions were implemented in a standardized fashion across the State.
While barriers and interventions were documented, the CMO did not document the tools used to determine the barriers/interventions in the PIP (e.g., brainstorming, fishbone diagram) or data to support the barrier prioritization or intervention selection. Three types of barriers were identified: member, provider, and system. Interventions addressing members and providers were implemented during the baseline period while all interventions to address all three types of barriers (member, provider and system) were implemented during Remeasurement 1. Peach State documented that it placed outreach calls to non-compliant members in need of immunizations. Some of the other interventions that were implemented are listed below.
The CMO implemented CareGaps, an internal system alert to let Peach State employees and members (secure portal) know about members who are due or past due for preventive services. Again, there was no indication of how many members actually accessed the portal or how employees acted on the information to improve the rate of childhood immunizations.
To address member barriers, Peach State called members, scheduled appointments, performed reminder calls, and facilitated non-emergency transportation to appointments.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-9
Health Services Advisory Group, Inc.

FINDINGS

Providers were sent a list of non-compliant members to enable provider outreach to members. The CMO implemented a member incentive program targeting non-compliant members to
receive immunizations by 2 years of age. Peach State participated in the "Centene Childhood Immunization Mailing" pilot program.
Quarterly, members were mailed postcards encouraging them to contact their PCP to find out which immunizations had not been administered to date. The PCP's name and address were included in the postcard.
Peach State documented that it believed the interventions implemented had caused the statistically significant improvement reported. However, the CMO did not provide any data to support this claim. HSAG anticipated that the CMO would have documented a data-driven evaluation of the intervention's effectiveness. HSAG recommends that Peach State implement processes to evaluate the effectiveness of all of its interventions, ensuring a linkage between the members who received the interventions and the study indicator outcome.
Childhood Obesity

Table 2-7--Performance Improvement Project Outcomes for Childhood Obesity

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Remeasurement 3 Sustained (1/1/0912/31/09) (1/1/1012/31/10) (1/1/1112/31/11) (1/1/1212/31/12) Improvement^

The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation.

32.1%

29.0%

22.7%*

47.7%*

NA

The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of counseling for nutrition.

36.7%

45.5%*

40.7%

56.0%*

NA

The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of counseling for physical activity.

28.2%

32.0%

29.4%

47.7%*

NA

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.
* Designates statistically significant improvement over the prior measurement period (p value < 0.05).
* Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained
or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-10
Health Services Advisory Group, Inc.

FINDINGS
The outcomes for the Childhood Obesity PIP at Remeasurement 3 were significantly better than the previous year for all three study indicators. Additionally, the CY 2012 rates of BMI percentile documentation (Study Indicator 1) and Physical Activity Counseling (Study Indicator 3) achieved statistically significant improvement over baseline for the first time. The PIP will be evaluated for sustained improvement at Remeasurement 4, when all three study indicators will have at least one subsequent measurement after achieving significant improvement. The CMO's rates of BMI Documentation and Physical Activity Counseling also surpassed the respective CY 2012 DCH target rates of 45.2 percent and 45.5 percent, while the rate of Nutrition Counseling (Study Indicator 2) fell short of the DCH target rate of 57.7 percent. When compared to the national Medicaid HEDIS 2011 rates, the Remeasurement 3 rates of all three study indicators fell between the 50th and 75th percentiles.
A critical analysis of the CMO's improvement strategies demonstrated the following:
While the CMO noted brainstorming as a causal/barrier analysis process during the baseline measurement period, it did not document any specific data-driven processes or tools used for causal/barrier analysis during the Remeasurement 3 period to identify barriers and interventions. The PIP documentation stated, "The Committee determined that the two most significant barriers were (1) member benefit education regarding the importance of preventive visits (including height/weight/BMI percentile, and anticipatory guidance) and (2) provider education to enhance knowledge of performing and documenting services." The first barrier is not directly linked to the study outcome.
The interventions Peach State implemented to address these barriers included: Quarterly meetings with the medical record review vendor to reinforce content and materials for practitioner training on BMI percentile documentation, counseling for nutrition, and counseling for physical activity. One-on-one provider education on the importance of obtaining and documenting BMI percentile, counseling for nutrition, and counseling for physical activity. The "Start Strong" education and goal-setting pilot program targeting overweight members 417 years of age. HSAG has concluded that this intervention would not have any effect on the study outcome.
Despite HSAG's feedback last year, Peach State continued to implement interventions that could not be clearly linked to the Childhood Obesity PIP study indicators. For example, the CMO documented that the member education intervention, "Start Strong," targeting overweight members 417 years of age, had the primary goals of reducing BMI percentile and attaining lifestyle goals of participants. Improvement in the outcomes for these study indicators are dependent on providers performing and documenting the necessary services during an office visit, not on member education per se.
While Peach State acknowledged the importance of evaluating the effectiveness of interventions, the CMO did not have an evaluation plan in place for any of the interventions.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-11
Health Services Advisory Group, Inc.

FINDINGS

Comprehensive Diabetes Care

Table 2-8--Performance Improvement Project Outcomes for Comprehensive Diabetes Care

PIP Study Indicator

Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^

The percentage of members 1875 years of age with

diabetes (type 1 and type 2) who had an HbA1c control <

28.8%

27.6%

NA

7.0%.

The percentage of members 1875 years of age with

diabetes (type 1 and type 2) who had a LDL-C control <

27.5%

20.4%*

NA

100mg/ml.

The percentage of members 1875 years of age with

diabetes (type 1 and type 2) who had a BP control <

58.0%

53.7%

NA

140/90 mmHg.

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.

* Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators
that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

None of the study indicators in the Comprehensive Diabetes Care PIP achieved statistically significant improvement over baseline at Remeasurement 1. All three study indicators declined during the remeasurement period with the decline in Study Indicator 2 (LDL-C control < 100 mg/ml) being statistically significant. The CY 2012 rates for all three indicators fell below the DCH target rates of 35.5 percent (HbA1c control < 7.0%), 33.6 percent (LDL-C control < 100 mg/ml), and 61.6 percent (BP control < 140/90 mmHg), respectively. The Remeasurement 1 rates for all three study indicators also fell below the 25th percentile of the respective national Medicaid HEDIS 2011 rates.

An analysis of the plan's improvement strategy identified some weaknesses which may have led to the lack of improvement in this PIP's study indicator rates.
Peach State's multidisciplinary HEDIS Steering Committee reviewed administrative rates to identify barriers and interventions for the Comprehensive Diabetes Care PIP. The CMO did not document any causal/barrier analysis tools, nor did it describe a process linking data to identified barriers and interventions. However, the CMO implemented interventions that addressed screening rather than control of HbA1c, LDL-C, and BP, which was what the outcomes were measuring. The CMO even stated in the PIP documentation that "The Plan's interventions were geared toward getting the recommended screenings done for our members." The following were some of the interventions that likely would not impact the study outcomes:
Provider outreach to obtain screening results identified as missing in the HEDIS reporting system.
"Push" Initiative--live member outreach to schedule appointments, assist with transportation, and offer an incentive for obtaining due and past due preventive services.
CareGaps, an internal system alert to let Peach State employees, providers, and members (secure portal) know about due or past due preventive services.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-12
Health Services Advisory Group, Inc.

FINDINGS

Peach State reported that the HEDIS Steering Committee identified the Diabetes Management Program and the CareGaps system as the most effective interventions; however, the CMO did not explain how the committee arrived as this conclusion. While Peach State stated that it monitors monthly administrative rates, it did not describe a process by which individual interventions were evaluated for effectiveness. For example, the CMO did not track the members who were reached through the Diabetes Disease Management program to determine their performance on the study indicators. Evaluating the effectiveness of each intervention is an integral step in achieving significant improvement in the study indicators.

Avoidable Emergency Room Visits
In CY 2012, Peach State began participating in a collaborative performance improvement project with DCH and two other CMOs to address avoidable emergency room visits by evaluating combined data and implementing coordinated interventions. The collaborative's goal was to reduce avoidable emergency room visits by 5 percent by the end of CY 2012. The baseline and Remeasurement 1 rates for the six study indicators documented in the PIP submission for the collaborative Avoidable Emergency Room Visits PIP are summarized in Table 2-9.

Table 2-9--Performance Improvement Project Outcomes for Avoidable Emergency Room Visits

PIP Study Indicator

Baseline Period Remeasurement 1 Sustained (1/1/1112/31/11) (1/1/1212/31/12) Improvement^

1. The percentage of practices that provide the same day appointments for routine and urgent care.
2. The percentage of practices that provide routine and urgent care appointments after hours.
3. The percentage of practices that provide appointments for routine and urgent care after hours and have the ability to document after hours clinical advice in the patient's record.
4. The percentage of practices that have access to and utilize electronic health records.

100% 50% 100% 70%

100%

NA

70%

NA

100%

NA

90%

NA

5. The percentage of practices that receive information regarding ER visits from the study hospitals.

80%

100%

NA

6. The percentage of ER visits for `avoidable' diagnoses

(dx382Acute Suppurative otitis:382.9Unspecified

otitis:462Acute pharyngitis:465.9Acute upper

respiratory infection:466 Acute bronchitis:786.2Cough)

19.38%

20.52%*

NA

among members under 21 years of age who had a visit to

the ED in three selected Children's Healthcare of Atlanta

facilities in the Atlanta region.

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.

* Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that
is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-13
Health Services Advisory Group, Inc.

FINDINGS
The Avoidable Emergency Room Visits PIP had six study indicators and was piloted in the metro-Atlanta region of the State. Study Indicators 1 through 5 assessed the 10 metro-Atlanta provider practices associated with the highest number of avoidable emergency room visits, and Study Indicator 6 assessed visits to the emergency departments of three Children's Healthcare of Atlanta facilities. Study Indicators 1 through 5 were incorporated at the direction of the State to serve as lead measures. Lead indicators can be helpful in predicting changes that the CMO may use to make mid-course corrections to allow for timely, rapid cycles of improvement rather than waiting for the lag or outcome measure of the PIP, which relies on annual measurement. The initial data for these lead measures were collected by the CMOs during the course of the PIP, and the results showed that these measures did not allow an opportunity for improvement in Study Indicators 1 (percentage of providers who provide same-day appointments) and 3 (percentage of practices that have the ability to document after-hours clinical advice) because the baseline rate for each indicator was 100 percent. The study indicators were created before some of the baseline data were obtained from the participating practices and as such, the CMOs were unaware these baseline rates would be 100 percent. The rates of Study Indicators 2, 4, and 5 had non-statistically significant improvement from baseline to Remeasurement 1. The rate of Study Indicator 5 (percentage of practices that receive ER visit information from study hospitals) reached 100 percent at Remeasurement 1; therefore, this indicator has no room for improvement in future measurement periods for the metro-Atlanta pilot practices. Study Indicator 6, the percentage of emergency room visits for the specified subset of avoidable diagnoses, is the only indicator that did not improve, as there was a significant increase of 1.14 percentage points in the rate of avoidable emergency room visits from baseline to Remeasurement 1. HSAG recommends the CMOs modify their reporting of this PIP for the next remeasurement period and include the lead measures in Activity VIII on the PIP report template.
Through its validation review, HSAG noted structural flaws in the documentation of the study design (Activities I through VI) for this collaborative PIP. The numerator and denominator descriptions for Study Indicators 2 and 3 that were documented by the CMOs were identical. The CMOs will need to correct this prior to the next annual submission.
Within the study design, the CMOs did not completely define the study population. The CMOs stated, "The method for identifying member visits in the denominator was derived from a list of ICD-9 codes determined to be `avoidable,' i.e., non-emergent conditions that could have been treated in another outpatient setting." NA is not applicable to this element. The denominator (study population) should be composed of all emergency room visits for CMO members under the age of 21. The CPT, UB Revenue, and place of service codes used to identify an emergency room visit, and the anchor date criteria, were not included. In addition, the CMOs did not identify the 10 providers that were involved in the pilot project as part of the study population definition. For the data collection methodology, the CMOs did not include the codes used to identify emergency room visits (denominator for Study Indicator 6). Furthermore, it was unclear how the survey used by the CMOs captured data for Study Indicators 2 and 3.
Prior to the three CMOs coming together, Peach State implemented an ER case management program where high-volume hospitals notified the CMO of members considered "frequent flyers." These members received a mailing and telephone call to discuss the appropriate use of an ER and their medical home.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-14
Health Services Advisory Group, Inc.

FINDINGS
In Activity VIII, Implement Intervention and Improvement Strategies, the CMOs documented that a multidisciplinary team of participants from the three CMOs, representatives from DCH, and several study participants reviewed the baseline results of the provider survey, as well as the member focus study, to determine barriers and opportunities for improvement. Interventions were developed to address member, provider, and resource barriers.
The CMOs documented that provider-level interventions were designed to motivate providers to offer after-hours care, as well as to encourage the use of electronic health records in the practices. Data sharing was designed to give providers the insight into their level of performance and to identify areas of potential opportunity such as proactive member outreach to establish a medical home. The following were the collaborative provider-level interventions:
Increased percentage of practices using electronic health records through referral to the Georgia Health Information Technology Regional Extension Center (GA-HITREC).
Shared data regarding ER rates with practices to identify members using the ER during regular office hours.
Notified providers regarding the availability of additional reimbursement for care provided after-hours.
Member improvement strategies were focused on educating members regarding the available resources to prevent ER use. The following are the collaborative member-level interventions:
Continued ER case management programs for live outreach to members who frequented the ER.
Educational mailings to members regarding patient-centered medical homes (PCMHs) and nurse advice hotlines.
Provided materials to members regarding transportation vendors and assistance to members to arrange transportation, when needed.
The PIP documentation did not reflect any processes that were in place to evaluate the effectiveness of any interventions. Although the CMOs discussed follow-up activities planned, due to the decline in performance for the avoidable ER visit rate indicator (Study Indicator 6), HSAG recommends the CMOs, collaboratively, investigate the reasons for the decrease in performance and based on the findings, implement strategies to improve performance.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-15
Health Services Advisory Group, Inc.

FINDINGS

Member Satisfaction

Table 2-10--Performance Improvement Project Outcomes for Member Satisfaction

PIP Study Indicator

Baseline Period

Sustained

(3/13/135/22/13) Improvement^

The percentage of respondents who rate the health plan an 8, 9, or 10 to

Q36 "Using any number from 0 to 10, where 0 is the worst health plan possible and 10 is the best health plan possible, what number would you

87.0%

NA

use to rate your child's health plan?"

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before

sustained improvement can be assessed.

^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study

indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current

measurement period's results must reflect statistically significant improvement when compared to the baseline results for all

study indicators.

Peach State initiated a new Member Satisfaction PIP in 20122013 as part of its DCH contract requirements. The study indicator, based on Question 36 of Peach State's 2013 CAHPS Child Medicaid Member Survey, assessed the overall rating parents/guardians selected for the CMO, as their child's health plan, with "0" being the lowest possible rating and "10" being the highest possible rating. The baseline rate of respondents giving Peach State a score of "8" or higher was 87.0 percent, slightly lower than the CMO's baseline goal (The Myers Group 90th percentile) of 88.7 percent.

Provider Satisfaction

Table 2-11--Performance Improvement Project Outcomes for Provider Satisfaction

PIP Study Indicator

Baseline Period

Sustained

(11/14/121/16/13) Improvement^

The percentage of providers answering, "very satisfied" or, "somewhat satisfied" to Q48 "Overall satisfaction with Peach State Health Plan?"

76.3%

NA

NA Statistically significant improvement over baseline and a subsequent measurement must occur for all study indicators before sustained improvement can be assessed.
^ Sustained improvement is defined as statistically significant improvement in performance over baseline for all study indicators that is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results for all study indicators.

Peach State also collected baseline data for a new Provider Satisfaction PIP in 20122013. The study indicator from the CMO's 2012 provider satisfaction survey assessed providers' overall satisfaction. The baseline rate of providers who reported being "Somewhat satisfied" or "Very satisfied" with Peach State was 76.3 percent. The CMO stated in the PIP Summary Form that its goal was to increase the baseline rate by 2 percentage points; therefore, the goal is for 78.3 percent of providers to report being "Somewhat satisfied" or "Very satisfied" with Peach State at Remeasurement 1.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-16
Health Services Advisory Group, Inc.

FINDINGS
The Member and Provider Satisfaction PIPs were validated through Activity VII because the CMO reported only baseline data and did not report interventions. As these PIPs progress to reporting Remeasurement 1 data, HSAG will evaluate the CMO's causal/barrier analysis process and interventions. HSAG recommends that Peach State incorporate the feedback provided for its other PIPs as it pertains to having targeted and relevant interventions that will directly impact study indicator outcomes and implement processes to evaluate the effectiveness of each intervention.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 2-17
Health Services Advisory Group, Inc.

3. CONCLUSIONS AND RECOMMENDATIONS
for Peach State Health Plan

Conclusions
With the exception of the collaborative Avoidable Emergency Room Visits PIP, Peach State appeared to have sound methodologies in place for the PIP Design stage (Activities I through VI). The sound study design for eight of nine PIPs created the foundation for the CMO to progress to subsequent PIP stages--implementing improvement strategies and achieving real and sustained study indicator outcomes. A critical analysis, however, revealed that Peach State has room for considerable improvement in the Implementation and Outcomes stages.
In the Appropriate Improvement Strategies Activity of the Implementation stage, the CMO frequently failed to document specific tools and processes used for causal/barrier analysis. The documentation generally lacked data to support the selection of barriers and interventions. Additionally, the wording of barriers in many of the PIP reports suggested that Peach State either did not understand the definition of a barrier or was not accurately documenting barriers. Likely a result of deficiencies in the causal/barrier analysis process, Peach State's interventions in several PIPs could not be logically linked to the study indicators and therefore were unlikely to improve the study outcomes.
Although Peach State exhibited sound study design for eight of its PIPs, it achieved real and sustained improvement in only one PIP, Annual Dental Visits. In addition to lacking the documentation of causal/barrier analysis processes, the CMO also did not demonstrate that it had processes in place to evaluate intervention effectiveness. While Peach State identified certain interventions as being effective, the CMO did not provide a data-driven rationale or criteria for determining effectiveness.

Recommendations
HSAG recommends that Peach State:
Ensure that all data components reported in each PIP are accurate and consistently documented throughout the PIP, and align with the data that have been reported in its final report audit.
Ensure that all statistical testing is done correctly and the documentation of the statistical testing outcomes is accurate and consistent throughout the PIP.
Reference and carefully follow the PIP Summary Form completion instructions in the Appropriate Improvement Strategies Activity related to defining barriers and interventions and documenting the causal/barrier analysis process.
Ensure that it has an accurate understanding of the barrier analysis process, requesting technical assistance if necessary, so that it is effectively identifying and documenting barriers resulting from the causal/barrier analysis process.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 3-1
Health Services Advisory Group, Inc.

CONCLUSIONS AND RECOMMENDATIONS
Conduct an annual causal/barrier analysis including drill-down analysis along with additional quarterly analyses of its outcome data. The CMO must accurately document the analyses, providing the results, identified barriers, and the rationale for how barriers are prioritized.
Have a process in place, for any intervention implemented, to evaluate the efficacy of the intervention. The results of each intervention's evaluation conducted during each remeasurement period should be included in the PIP. If the interventions are not having the desired effect, Peach State should discuss how it will address these deficiencies by modifying or discontinuing current interventions or implementing new improvement strategies.
HSAG will work with DCH to create a PIP Summary Form template that is specific to the collaborative Avoidable Emergency Room Visits PIP.

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page 3-2
Health Services Advisory Group, Inc.

APPENDIX A. PIP-SPECIFIC VALIDATION RESULTS
for Peach State Health Plan

Table A-1--Peach State Health Plan's SFY 2014 PIP Performance

Study Stage

Activity

Adolescent Well-Care

Design
Implementation Outcomes

Appropriate Study Topic
Clearly Defined, Answerable Study Question(s)
Correctly Identified Study Population
Clearly Defined Study Indicator(s)
Valid Sampling Techniques (if sampling was used)
Accurate/Complete Data Collection
Design Total
Sufficient Data Analysis and Interpretation
Appropriate Improvement Strategies
Implementation Total
Real Improvement Achieved
Sustained Improvement Achieved

100%
100%
100%
100%
Not Applicable
100% 100% 88%
75% 83% 100% Not Assessed

Outcomes Total 100%

Validation Status

Partially Met

Annual Dental Visits 100%
100%
100%
100%
Not Applicable
100% 100% 75%
75% 75% 100%
100%
100% Partially
Met

Percentage of Applicable Evaluation Elements Scored Met

Appropriate Use of ADHD Medications
100%

Childhood Immunizations
--Combo 10
100%

Childhood Obesity
100%

Comprehensive Diabetes Care
100%

Avoidable Emergency Room Visits
100%

Member Satisfaction
100%

Provider Satisfaction
100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

100%

Not Applicable

100%

100% 100% 100%

100% 100% 100%

0% 73% 25%

75% 92% 100%

Not Assessed Not Assessed

25%

100%

Not Met

Partially Met

100%
100%
100%
100% 100% 89%
0% 67% 75% Not Assessed 75% Partially Met

100% 100%
100%
100% 100% 78%
0% 58% 25% Not Assessed 25% Not Met

0%
71%
Not Applicable
30% 59% 75%
100% 82% 25% Not Assessed 25%
Not Met

100%
100%
100%
100% 100% 100%
Not Assessed
100% Not
Assessed Not
Assessed Not
Assessed Met

100%
100%
100%
100% 100% 100%
Not Assessed
100% Not
Assessed Not
Assessed Not
Assessed Met

Peach State Health Plan SFY 2014 PIP Validation Report State of Georgia

PeachState_GASFY2014_CMO_PIP-Val_Report_F1_1113 Page A-1
Health Services Advisory Group, Inc.