AMERIGROUP Community Care: performance improvement projects report FY 2011 [Dec. 2010]

State of Georgia
Department of Community Health Georgia Families Program
AMERIGROUP Community Care
PERFORMANCE IMPROVEMENT PROJECTS REPORT
FY 2011
December 2010
3133 East Camelback Road, Suite 300 Phoenix, AZ 85016 Phone 602.264.6382 Fax 602.241.0757 2007 Health Services Advisory Group, Inc.

TABLE OF CONTENTS
1. BACKGROUND ...................................................................................................................... 1-1
CMO Overview ................................................................................................................................. 1-2 Study Rationale ................................................................................................................................ 1-2 Study Summary ................................................................................................................................ 1-3 Validation Overview.......................................................................................................................... 1-4
2. FINDINGS ............................................................................................................................. 2-1
Aggregate Validation Findings ......................................................................................................... 2-1 Study Design .................................................................................................................................. 2-2 Study Implementation .................................................................................................................... 2-2 Study Outcomes............................................................................................................................. 2-3
PIP-Specific Outcomes .................................................................................................................... 2-3 Analysis of Results ......................................................................................................................... 2-3 Barriers/Interventions ..................................................................................................................... 2-6
3. STRENGTHS.......................................................................................................................... 3-1
Individual PIP Strengths ................................................................................................................... 3-1 Global Strengths Across All PIPs ..................................................................................................... 3-1
4. OPPORTUNITIES FOR IMPROVEMENT ....................................................................................... 4-1
Individual PIPs.................................................................................................................................. 4-1 Global Issues.................................................................................................................................... 4-2
Appendix A. PIP-SPECIFIC VALIDATION SCORES....................................................................... A-1

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc.

Page i

ACKNOWLEDGMENTS AND COPYRIGHTS
CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA). NCQA HEDIS Compliance AuditTM is a trademark of NCQA.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc.

Page ii

FY 2011 PIP Validation Report AMERIGROUP Community Care

1. BACKGROUND
The Code of Federal Regulations (CFR), specifically 42 CFR 438.350, requires states that contract with managed care organizations to conduct an external quality review (EQR) of each entity. An EQR includes analysis and evaluation by an external quality review organization (EQRO) of aggregated information on health care quality, timeliness, and access. In Georgia, the EQR analyzes and evaluates the health care services that a care management organization (CMO) or its contractors furnish to Georgia Families recipients. At a minimum, the State must report EQRO findings to the federal government on the following mandatory activities:
Evaluation of CMO Compliance with Managed Care Regulations Validation of CMO Performance Measures Validation of CMO Performance Improvement Projects (PIPs)
These three mandatory activities work together to ensure that Georgia Families' Program and the CMOs are providing quality care to their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measures provides a barometer of the quality and effectiveness of care. When performance measures highlight areas of low performance, the Department of Community Health (DCH) and the CMOs employ PIPs to improve the quality of health care in targeted areas. PIPs are a key tool in the CMOs' overall quality strategy; they provide the framework for monitoring, measuring, and improving the delivery of health care.
This is the third year, Health Services Advisory Group, Inc. (HSAG), as the State's EQRO, conducted a validation of the CMOs' PIPs. HSAG reviewed each submitted PIP using the Centers for Medicare & Medicaid Services (CMS) validation protocol1 and evaluated two key components of the quality improvement process, as follows:
1) HSAG evaluated the technical structure of the PIPs to ensure the CMOs designed, conducted, and reported PIPs in a methodologically sound manner, meeting all State and federal requirements. HSAG's review determined whether a PIP's design (e.g., study indicators, the data collection methodology, and data analysis plan) was based on sound methodological principles and could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring sustained improvement.
2) HSAG evaluated the implementation of the PIP. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. This component evaluates how well a CMO improved
1 The Centers for Medicare & Medicaid Services. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, final protocol Version 1.0, May 1, 2002.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-1 Health Services Advisory Group, Inc.

BACKGROUND
its rates through implementation of effective processes (i.e., barrier analyses, intervention design, and evaluation of results). A primary goal of HSAG's PIP validation is to ensure that DCH and key stakeholders can have confidence that any reported improvement in outcomes is related to a given PIP.
CMO Overview
DCH contracted with AMERIGROUP Community Care (AMERIGROUP) beginning in 2006 to provide services to the Georgia Families Program (Medicaid and PeachCare for KidsTM) population. AMERIGROUP, a CMO, serves the eligible population in the Atlanta, North, East, and Southeast geographic regions of Georgia.
Study Rationale
The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas. Although HSAG has validated AMERIGROUP's PIPs for three years, the number of PIPs, study topics, and study methods have evolved over time.
In fiscal year (FY) 2009, DCH chose three PIP topics for validation (i.e., Provider Satisfaction, Well-Child Visits, and Lead Screening in Children). While similar to national, standardized Healthcare Effectiveness Data and Information Set (HEDIS) measures, these PIPs were based on State-defined methodology. In FY 2010, DCH incorporated three additional PIP topics (i.e., Childhood Immunizations, Member Satisfaction, and Adults' Access to Care) for a total of six PIPs. DCH modified the methodology used by the CMOs to reflect the National Committee for Quality Assurance's (NCQA's) HEDIS technical specifications. The incorporation of national, standardized methodologies allowed comparisons to national benchmarks. The second-year validation results for the aforementioned performance measures included the same four HEDIS measures represented by the PIPs; therefore, improvement in the PIP study outcomes would also be seen in the performance measure results.
Using the results from prior PIP and performance measure outcomes, DCH directed the CMOs to continue their PIPs on the current topics. The CMOs were required to report baseline and first remeasurement period data using the HEDIS hybrid method, where applicable. The hybrid method required data to be collected from member medical records, as well as administrative data sources (e.g., claims and encounters). The study topics selected by DCH addressed CMS' requirements related to quality outcomes--specifically, the quality and timeliness of, and access to, care and services.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-2 Health Services Advisory Group, Inc.

BACKGROUND

Study Summary

As noted in its Quality Strategic Report Plan Update (March 2009), DCH identified the improvement of performance measures in the PIP studies as a key objective. The current PIP submission included three clinical PIPs (i.e., Lead Screening in Children, Childhood Immunizations, and Well-Child Visits) and three nonclinical PIPs (i.e., Adults' Access to Care, Member Satisfaction, and Provider Satisfaction).

The three clinical PIP topics were based on HEDIS specifications and addressed children's preventive health (i.e., Lead Screening in Children, Childhood Immunizations, and Well-Child Visits). Children's primary health care is a vital part of the effort to prevent, recognize, and treat health conditions that can result in significant developmental and health status consequences for children and adolescents. These PIP topics represent a key area of focus for improvement.

The study indicator for the Adults' Access to Care PIP was also a HEDIS measure. This PIP topic represents an essential component in developing a relationship with a health care provider and establishing a medical home. Table 11 outlines the key study indicators incorporated in these four PIPs.

Table 11--HEDIS-based PIP Study Indicators

HEDIS Measure/Study Indicator Lead Screening in Children
Childhood Immunization Status-- Combo 2
Well-Child Visits in the First 15 Months of Life (Six or More Visits)
Adults' Access to Preventive/ Ambulatory Health Services

HEDIS Measure Description
The percentage of children two years of age who had one or more capillary or venous lead blood tests for lead poisoning by their second birthday.
The percentage of children two years of age who had four diphtheria, tetanus, and acellular pertussis (DTaP); three polio (IVP); one measles, mumps, and rubella (MMR); two H influenza type B (Hib); three hepatitis B; and one chicken pox (VZN) by their second birthday.
The percentage of members who turned 15 months old during the measurement year and who had six or more well-child visits with a primary care provider (PCP) during their first 15 months of life.
The percentage of members 2044 years of age who had an ambulatory or preventive care visit.

The remaining two PIPs addressed member and provider satisfaction. Table 12 outlines the key study indicators incorporated in these PIP topics.
The Member Satisfaction PIP corresponded to the specifications of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey 4.0H, Child Version measures. These measures provided information on parents' experiences with their child's provider and care management organization. The plan measured the percentage of members responding "Yes" to the Member Satisfaction Survey questions.
The final State-mandated PIP topic was Provider Satisfaction, an area that represented an opportunity for improvement for the CMOs. Each CMO contracted with a vendor to produce and administer this survey, and the CMOs submitted their second remeasurement period data this year. The plan measured the percentage of providers responding favorably (i.e., "Excellent" or "Very Good") to the selected Provider Satisfaction Survey question.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-3 Health Services Advisory Group, Inc.

BACKGROUND

Table 12--Satisfaction-based PIP Study Indicators

Survey Type

Identifier

Survey/Study Question

Member

"In the last 6 months, did your child's doctor or other health provider talk

Q10

with you about the pros and cons of each choice for your child's treatment

or health care?"

Member

"In the last 6 months, when there was more than one choice for your child's

Q11

treatment or health care, did your child's doctor or other health provider ask

you which choice you thought was best for your child?"

Provider

Q34C*

"Contacting the AMERIGROUP pharmacy call center to find out about formulary medications and alternatives to nonformulary medications."

* Providers were requested to respond if they agreed with the statement regarding the CMO.

Validation Overview
The primary objective of PIP validation was to determine each CMO's compliance with the requirements of 42 CFR 438.240(b)(1), including:
Measurement of performance using objective quality indicators. Implementation of systematic interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities for increasing or sustaining improvement.
HSAG obtained the data needed to conduct the PIP validation from the CMO's PIP Summary Forms. These forms provided detailed information about each CMO's PIPs related to the activities they completed and HSAG evaluated for the FY 2011 validation cycle.
Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be Met. Given the importance of critical elements to the scoring methodology, any critical element that received a Not Met score resulted in an overall validation rating for the PIP of Not Met. A CMO would be given a Partially Met score if 60 percent to 79 percent of all evaluation elements were Met or one or more critical elements were Partially Met. HSAG provided a Point of Clarification when enhanced documentation would have demonstrated a stronger understanding and application of the PIP activities and evaluation elements.
In addition to the validation status (e.g., Met) HSAG gave each PIP an overall percentage score for all evaluation elements (including critical elements). HSAG calculated the overall percentage score by dividing the total number of elements scored as Met by the total number of elements scored as Met, Partially Met, and Not Met. HSAG also calculated a critical element percentage

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-4 Health Services Advisory Group, Inc.

BACKGROUND
score by dividing the total number of critical elements scored as Met by the sum of the critical elements scored as Met, Partially Met, and Not Met. Figure 11 illustrates the three stages of the PIP process--i.e., Study Design, Study Implementation, and Study Outcomes. Each sequential stage provides the foundation for the next stage. The Study Design stage establishes the methodological framework for the PIP. The activities in this section include development of the study topic, question, indicators, and population. To implement successful improvement strategies, a strong study design is necessary.
Figure 11--PIP Stages
III. STUDY OUTCOMES
II. STUDY IMPLEMENTATION

I. STUDY DESIGN

Once a CMO establishes its study design, the PIP process moves into the Study Implementation stage. This stage includes data collection, sampling, and interventions. During this stage, the CMOs collect measurement data, evaluate and identify barriers to performance, and develop interventions targeted to improve outcomes. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final stage is Study Outcomes, which involves data analysis and the evaluation of real and sustained improvement based on reported results and statistical testing. Sustained improvement is achieved when outcomes exhibit statistical improvement over time and multiple measurements. This stage is the culmination of the previous two stages. If the study outcomes do not improve, the CMOs investigate the data they collected to ensure that they have correctly identified the barriers and implemented appropriate and effective interventions. If they have not, the CMOs revise their interventions and collect additional data to remeasure and evaluate outcomes for improvement. This process becomes cyclical until sustained statistical improvement is achieved.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 1-5 Health Services Advisory Group, Inc.

2. Findings
for AMERIGROUP Community Care

Aggregate Validation Findings

HSAG organized, aggregated, and analyzed AMERIGROUP's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of interventions. Based on its technical review, HSAG determined the overall methodological validity of the PIPs.
Table 21 displays the combined validation results for all six AMERIGROUP PIPs evaluated during FY 2011. This table illustrates the CMO's overall understanding of the PIP process and its success in implementation of the study. Each activity is composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score have satisfied the necessary technical requirements for a specific element. The validation results presented in Table 21 show the percentage of applicable evaluation elements that received a Met score by activity. Additionally, HSAG calculated an overall score across all activities. Appendix A provides the detailed validation scores for each of the six PIPs.

Table 21FY 2011 Performance Improvement Project Validation Results for AMERIGROUP Community Care (N=6 PIPs)

Percentage of Applicable Elements

Stage

Activity

Met

Partially Met

Not Met

I. Appropriate Study Topic

100% (32/32)

0% (0/32)

0% (0/32)

II. Clearly Defined, Answerable Study Question(s) Study Design
III. Clearly Defined Study Indicator(s)

100% (12/12) 100% (36/36)

0% (0/12)
0% (0/36)

0% (0/12)
0% (0/36)

IV. Correctly Identified Study Population

100% (17/17)

0% (0/17)

0% (0/17)

V.

Valid Sampling Techniques (if sampling was used)

100% (30/30)

0% (0/30)

0% (0/30)

Study Implementation

VI.

Accurate/Complete Data Collection

98% (50/51)

2% (1/51)

0% (0/51)

VII. Appropriate Improvement Strategies

14% (3/21)

0% (0/21)

86% (18/21)

Study Outcomes

VIII. Sufficient Data Analysis and Interpretation IX. Real Improvement Achieved

27% (13/49)
25% (5/20)

4% (2/49)
0% (0/20)

69% (34/49)
75% (15/20)

X. Sustained Improvement Achieved







Percentage Score of Applicable Evaluation Elements Met

74% (198/268)

The PIPs did not progress to this phase during the review period and could not be assessed for sustained improvement.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-1 Health Services Advisory Group, Inc.

FINDINGS
Overall, only 74 percent of the evaluation elements across all six PIPs received a score of Met. This suggests considerable room for improvement. While AMERIGROUP's strong performance in the Study Design phase indicated that each PIP was designed appropriately to measure outcomes and improvement, it was less successful in the implementation and outcomes. The following subsections highlight HSAG's validation findings associated with each of the three PIP stages.
Study Design
AMERIGROUP met 100 percent of the requirements across all six PIPs for all four activities within the Study Design stage. Overall, AMERIGROUP designed scientifically sound studies that were supported by the use of key research principles. The technical design of each PIP was sufficient to measure and monitor PIP outcomes associated with AMERIGROUP's improvement strategies. The solid design of the PIPs allowed the successful progression to the next stage of the PIP process.
Study Implementation
AMERIGROUP met 100 percent of the requirements for the sampling activity and 98 percent for the data collection activity in the Study Implementation phase; however, the CMO did not meet the requirements for the third activity of this phase, implementation of improvement strategies. Overall, only 14 percent of the applicable elements received a Met score for this activity. This finding suggests that while the CMO accurately documented and executed the implementation of the study design, AMERIGROUP's process for developing interventions was an area for improvement. Interestingly, the scores associated with this activity (i.e., implementation of improvement strategies) varied by PIP. Only the Provider Satisfaction PIP received a Met score for 100 percent of the evaluation elements in this activity. For the Member Satisfaction PIP, the CMO conducted a barrier analysis; however, the interventions implemented were not likely to induce permanent change. As a result, only one-third (33 percent) of the evaluation elements received a Met score. AMERIGROUP did not document any barrier analyses, nor did it propose/implement interventions for the remaining four PIPs--Adults' Access to Care, Childhood Immunizations, Lead Screening in Children, and Well-Child Visits. All of these PIP topics received a Not Met score for the majority of the evaluation elements. Without the successful implementation of appropriate improvement strategies, the CMO cannot achieve improved outcomes in the future.
A compliance issue was noted in the FY 2011 submission of AMERIGROUP's PIPs. For the FY 2011 validation, DCH required all CMOs to complete Activities IIX and submit baseline and the first remeasurement period results for all PIPs except the Provider Satisfaction PIP. AMERIGROUP redesigned the Provider Satisfaction PIP. With DCH's approval, the CMO restarted this PIP, submitting only baseline data. The CMO satisfied the submission requirements for the Member Satisfaction PIP, submitting both baseline and remeasurement data. However, AMERIGROUP did not meet this requirement for four of its PIPs--Adults' Access to Care, Childhood Immunizations, Lead Screening in Children, and Well-Child Visits. AMERIGROUP's

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-2 Health Services Advisory Group, Inc.

FINDINGS
lack of compliance with DCH's submission requirements affected the scores for the Study Implementation stage (Activity VII) and the Study Outcomes stage (Activities VIII and IX).
Study Outcomes
AMERIGROUP reported only baseline data for the four HEDIS-based PIPs--i.e., Adults' Access to Care, Childhood Immunizations, Lead Screening in Children, and Well-Child Visits. Therefore, these PIPs did not meet DCH's submission requirements, which affected HSAG's ability to assess the PIP outcomes. In accordance with DCH requirements, HSAG assessed all required activities. However, without the necessary documentation and data for these activities, AMERIGROUP's PIPs did not meet the requirements necessary to receive a Met score. As a result, these PIP validation scores were very low for this stage. Individual PIP scores for this stage ranged from 8 percent to 62 percent of the applicable activities receiving a Met score. Consequently, the aggregated results across all six PIPs reflected this deficiency (i.e., Activity VIII--27 percent and Activity IX--25 percent of the evaluation elements receiving a Met score) even though the Member Satisfaction and Provider Satisfaction PIPs scored considerably higher (62 percent and 60 percent, respectively).
Since none of the PIPs had progressed to the point of a second remeasurement period, or were required to do so, HSAG did not assess the PIPs for sustained improvement.
PIP-Specific Outcomes
Analysis of Results
Table 22 and Table 23 display outcome data for AMERIGROUP's six PIPs. Although the CMO did not include Remeasurement 1 results with its PIP documentation for four of its PIPs, HSAG used the results from the 2008 HEDIS Performance Measurement Report to review the outcome results. The inclusion of these results was necessary to evaluate the true progression and outcomes of these PIPs; however, since the CMO failed to meet the submission requirements, the results are informational only. The CMO submitted baseline data for the Provider Satisfaction PIP and Remeasurement 1 data for the Member Satisfaction PIP.
In its FY 2012 PIP submission, the CMO will be requested to include the corrected results and to submit Remeasurement 1 data for the Provider Satisfaction PIP. For the other five PIPs, the CMO will be requested to submit results for baseline through the second remeasurement.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-3 Health Services Advisory Group, Inc.

FINDINGS

Table 22--HEDIS-based Performance Improvement Project Outcomes for AMERIGROUP Community Care

PIP #1--Lead Screening in Children

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10) Improvement

The percentage of children 2 years

of age who received one blood lead test (capillary or venous) on or

68.2%

67.8%





before their second birthday. ^

PIP #2--Childhood Immunizations

The percentage of children who

received the recommended

vaccinations based on the

29.8%

72.0%





Childhood Immunization Status--

Combo 2 (4:3:1:2:3:1) guidelines. ^

PIP #3--Well-Child Visits

The percentage of children who had

six or more well-child visits with a PCP during their first 15 months of

62.3%

55.0%





life. ^

PIP #4--Adults' Access to Care

The percentage of members 2044

years of age who had an ambulatory

81.2%

85.5%*





or preventive care visit. ^

^ Reported rates were derived from the 2008 HEDIS Performance Measure Report. They are reported for informational purposes only.

The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement.

Caution should be used when comparing the results for Baseline and Remeasurement 1 due to changes in the study methodology.

* Designates statistically significant improvement over the prior measurement period (p value < 0.05). Designates a statistically significant decrease in performance over the prior measurement period (p value < 0.05).

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-4 Health Services Advisory Group, Inc.

FINDINGS

Table 23--Satisfaction-based Performance Improvement Project Outcomes for AMERIGROUP Community Care

PIP #5--Member Satisfaction

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (2/13/095/10/09) (2/17/105/2/10) (2/13/115/10/11) Improvement

1) The percentage of members

responding "Yes" to Q10--"In the

last six months, did your child's

doctor or other health provider talk

68.9%

60.3%





with you about the pros and cons of

each choice for your child's

treatment or health care?"

2) The percentage of members

responding "Yes" to Q11--"In the

last six months, when there was

more than one choice for your child's treatment or health care, did

61.1%

55.1%





your child's doctor or other health

provider ask you which choice you

thought was best for your child?"

PIP #6--Provider Satisfaction

PIP Study Indicator^

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (9/1/0912/31/09) (9/1/1012/31/10) (9/1/1112/31/11) Improvement

Percentage of providers answering

"Excellent" or "Very Good" to

Q34C--"Contacting the

AMERIGROUP pharmacy call center

18.3%



to find out about formulary medications

and alternatives to nonformulary

medications."





^ Providers were requested to respond if they agreed with the statements regarding the CMO. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement.

Only the Adults' Access to Care PIP demonstrated statistically significant improvement from Baseline to Remeasurement 1. The percentage of adult members that accessed ambulatory or preventive care increased by 4.3 percentage points and was 0.7 percentage points above DCH's target (84.8 percent) and 0.08 percentage points below the 2009 national HEDIS 75th percentile (85.58 percent). Statistically significant improvement is the standard for assessing real improvement and supports the conclusion that the noted improvement is not due to chance.
Conversely, the performance for three PIPs-- Lead Screening in Children, Well-Child Visits, and Member Satisfaction decreased from Baseline to Remeasurement 1. Moreover, the decrease of approximately seven percentage points (i.e., 62.3 percent versus 55 percent) was statistically significant for the Well-Child Visits PIP. The result was 10.4 percentage points below the DCH target (65.4 percent) and below the 2009 national HEDIS 50th percentile of 60.52 percent.
HSAG could not compare the Childhood Immunizations PIP rates between Baseline and Remeasurement 1 due to changes in the measurement methodology. The remeasurement results

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-5 Health Services Advisory Group, Inc.

FINDINGS

used a hybrid methodology and included data from the Georgia Registry of Immunization Transactions and Services (GRITS), whereas baseline results were calculated from administrative data alone. The methodology change is expected to improve results.

Barriers/Interventions
The identification of barriers through barrier analysis and the subsequent selection of appropriate interventions to address them are necessary steps to improve outcomes. The CMO's choice of interventions, the combination of intervention types, and the sequence of implementation of the interventions are essential to the CMO's overall success. AMERIGROUP did not provide documentation in the PIP submission of conducting any barrier analyses or implementing interventions for its Adults' Access to Care, Lead Screening in Children, Childhood Immunizations, or Well-Child Visits PIPs. Therefore, HSAG could not evaluate improvement strategies.
For its Member Satisfaction PIP, the plan identified several barriers based on a brainstorming session conducted by the CMO. Barriers identified included members feeling rushed or ignored, lack of member awareness and knowledge, time constraints of providers, and potential language barriers. AMERIGROUP's interventions were limited to publishing an article in both the member and provider newsletters. The member newsletter addressed how members can make a visit with their provider more helpful, while the provider newsletter addressed communication skills, the availability of interpreter services, and how to make a visit more helpful for the member.
However, the interventions the plan implemented did not adequately address the identified barriers. In general, newsletters have only short-term effects on study outcomes, with very limited effects on member-based outcomes. A single article distributed only once (similar to AMERIGROUP's intervention) has even less impact. As soon as the remeasurement rates were available, the plan should have conducted another causal/barrier analysis to identify specific, actionable barriers and selected interventions that were more appropriate. Additionally, the plan should have implemented more targeted interventions, including system-based interventions to ensure that any improvement was sustainable over time. System interventions include organization-wide initiatives including, but not limited to, changes in policy, targeting of additional resources, etc.
For the Provider Satisfaction PIP, the plan performed a barrier analysis; initially identifying noncompliance-related telephone statistics and poor functionality of the pharmacy Web site as the primary barriers. The plan implemented two interventions related to the call center, but neither intervention directly addressed the PIP measure outcome related to provider satisfaction with pharmacy formularies. Instead, one intervention addressed information technology quality checks for members incorrectly flagged as having other health insurance, and the second intervention revised staff schedules to match call arrival patterns.
Approximately one year later, after conducting another barrier analysis, the plan identified providers' lack of knowledge regarding use of the pharmacy Web site as the primary barrier. The plan increased the functionality of the Web site and conducted educational training of providers

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-6 Health Services Advisory Group, Inc.

FINDINGS
on use of the Web site. However, implementation of these interventions did not occur until the third quarter of 2010, so their impact on the second remeasurement period outcomes may be limited. In addition, the plan did not document a direct link between the call center and the pharmacy Web site; however, the improvement strategy implemented may result in improved provider satisfaction related to obtaining information on medication formularies.
The PIP validation process relies on an annual evaluation; however, CMOs should perform an interim evaluation of the results in addition to the formal annual evaluation. Evaluation of interim measurement results could assist the CMO in identifying and eliminating barriers that impede improvement. Furthermore, evaluation of the study outcomes would assist the CMO in determining if the interventions are having the desired effect or if modifications to current interventions or new interventions are necessary to improve results.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 2-7 Health Services Advisory Group, Inc.

3. Strengths
for AMERIGROUP Community Care
Individual PIP Strengths
The Provider Satisfaction and Member Satisfaction PIPs both received an overall Met validation status and represented areas of strength for AMERIGROUP. Performance on these PIPs suggested a thorough understanding of PIP study design. AMERIGROUP's revisions of the Provider Satisfaction interventions suggested that the CMO may succeed in achieving real and sustained improvement in future PIP submissions.
The outcome for the Adults' Access to Care PIP also improved significantly from Baseline to Remeasurement 1 and was 0.7 percentage points above the DCH target (84.8 percent). Further, the Remeasurement 1 result was only 0.08 percentage points below the national 2009 HEDIS 75th percentile of 85.58 percent. AMERIGROUP's success on this PIP could affect the CMO's general performance on the Adults' Access to Preventive/Ambulatory Health Services HEDIS measure.
Global Strengths Across All PIPs
AMERIGROUP demonstrated an excellent understanding of the activities related to the Study Design stage of the PIP process across all PIPs. This represented a strength for AMERIGROUP. The sound study design of the PIPs created the foundation necessary for the CMO to progress to subsequent PIP stages--i.e., implementing improvement strategies and accurately assessing study outcomes. Additionally, the CMO appeared to understand and appropriately conduct the sampling and data collection activities of the Study Implementation stage. These activities are critical to collecting the necessary data to produce accurate study indicator rates.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 3-1 Health Services Advisory Group, Inc.

4. Opportunities for Improvement
for AMERIGROUP Community Care

Individual PIPs
AMERIGROUP has an opportunity to improve documentation related to study implementation and study outcomes for the Adults' Access to Care, Lead Screening in Children, Childhood Immunizations, and Well-Child Visits PIPs. These PIPs all received an overall validation status of Not Met. Most importantly, the CMO should adhere to DCH's PIP submission requirements, including the completion of necessary activities and outcomes. The CMO's noncompliance with these requirements in FY 2011 restricted HSAG's assessment of the PIPs and its ability to provide feedback in preparation for the FY 2012 submission.
Although the study indicator rates included in this report for these PIPs were informational due to the CMO's failure to submit the required data, the rates declined for the Lead Screening in Children, Childhood Immunizations, and Well-Child Visits PIPs. Moreover, the decrease in the study indicator rate for the Well-Child Visits PIP was statistically significant and represented an area of concern since the rate was more than 10 percentage points below the DCH target (65.4 percent). AMERIGROUP should focus on its study outcome barriers--identification through analyses, prioritization based on resources, and eventually, reduction of their effects with the implementation of targeted interventions. The CMO should ensure that there is a direct link between each intervention and the identified barrier.
Additionally, AMERIGROUP should incorporate a method to evaluate the success of its interventions. The CMO should analyze its data to determine if any subgroup within its population had a disproportionately lower rate that negatively affected the overall rates. This "drill-down" type of analysis should be conducted both before and after the implementation of any intervention. For example, AMERIGROUP should evaluate whether rates differ by geographic region, gender, race/ethnicity, age, etc. The CMO could then target its interventions to the subgroups with the lowest rates, thereby facilitating the implementation of more precise, concentrated interventions. The process of targeting interventions to the appropriate subgroup is more efficient and effective. Global interventions directed at the entire eligible population may not achieve the desired results while requiring the same resources. After implementation of the targeted intervention, the CMO should again evaluate the applicable subgroups to determine the intervention's success. The documentation of this entire process should be included in the PIP submission.
Despite the Member Satisfaction PIP receiving an overall Met validation status, providing confidence in the study results, the interventions implemented were not likely to induce permanent change. In fact, the reported change in rates was not statistically significant and could be due to chance rather than any effort of the CMO. Any interventions should directly affect the identified barrier and should not rely on one-time, member-based actions. Additionally, the

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 4-1 Health Services Advisory Group, Inc.

OPPORTUNITIES FOR IMPROVEMENT
CMO should include an evaluation of its interventions. For the CMO's newsletter intervention, it would be difficult for the CMO to know how many members read the newsletter article and applied the information during their next visit with a provider. The decline in member satisfaction between the two measurement periods emphasizes the CMO's need to revise its improvement strategies.
Global Issues
Generally, AMERIGROUP did not demonstrate improvement in outcomes. AMERIGROUP's PIPs were well designed; however, the implementation of improvement strategies has been ineffective in producing long-term, permanent change in outcomes. AMERIGROUP's focus should shift to the development of appropriate improvement strategies. Without effective strategies, the CMO will not be able to improve PIP outcomes.
The CMO should be mindful that the submission of PIPs for validation will be an annual activity without an opportunity to resubmit. AMERIGROUP should carefully complete and timely submit all necessary documentation. The CMO should refer to the PIP Validation Tool and address all Points of Clarification and all Partially Met and Not Met scores in the FY 2012 submission.

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Page 4-2 Health Services Advisory Group, Inc.

Appendix A. PIP-Specific Validation Scores
for AMERIGROUP Community Care

Table A1--AMERIGROUP's FY 2011 PIP Performance

Review Activity

Lead Screening Childhood In Children Immunizations

Study Design I. Review the Selected Study Topic(s) II. Review the Study Question(s) III. Review the Selected Study
Indicator(s) IV. Review the Identified Study
Population Study Implementation V. Review Sampling Methods VI. Review Data Collection Procedures VII. Assess Improvement Strategies Study Outcomes VIII. Review Data Analysis and Study
Results IX. Assess for Real Improvement X. Assess for Sustained Improvement Percentage Score for Applicable Evaluation Elements Met Percentage Score for Applicable Critical Elements Met Validation Status

17/17 (100%) 6/6 (100%) 2/2 (100%) 6/6 (100%)
3/3 (100%) 16/20 (80%) 6/6 (100%) 10/10 (100%)
0/4 (0%) 2/13 (15%) 1/9 (11%) 1/4 (25%) Not Assessed
70%
85% Not Met

17/17 (100%) 6/6 (100%) 2/2 (100%) 6/6 (100%)
3/3 (100%) 16/20 (80%) 6/6 (100%) 10/10 (100%)
0/4 (0%) 2/13 (15%) 1/9 (11%) 1/4 (25%) Not Assessed
70%
85% Not Met

Well-Child Visits
17/17 (100%) 6/6 (100%) 2/2 (100%) 6/6 (100%)
3/3 (100%) 16/20 (80%) 6/6 (100%) 10/10 (100%)
0/4 (0%) 2/13 (15%) 1/9 (11%) 1/4 (25%) Not Assessed
70%
85%
Not Met

Adults' Access to
Care 16/16 (100%)
5/5 (100%) 2/2 (100%)
6/6 (100%)
3/3 (100%)
5/9 (56%) 0/0
5/5 (100%) 0/4 (0%) 1/12 (8%)
0/8 (0%)
1/4 (25%) Not Assessed
59%
80%
Not Met

Member Satisfaction 16/16 (100%) 5/5 (100%) 2/2 (100%)
6/6(100%)
3/3 (100%) 15/17(88%) 6/6 (100%) 8/8 (100%) 1/3 (33%) 8/13 (62%) 7/9 (78%) 1/4 (25%) Not Assessed
85%
100%
Met

Provider Satisfaction 14/14 (100%) 4/4 (100%) 2/2 (100%) 6/6 (100%)
2/2 (100%) 15/16 (94%) 6/6 (100%)
7/8 (88%) 2/2 (100%) 3/5 (60%) 3/5 (60%) Not Assessed Not Assessed
91%
100%
Met

AMERIGROUP Community Care FY 2011 PIP Validation Report State of Georgia

AMERIGROUP_GA2010-11_CMO_PIP-Val_Report_F2_1210 Health Services Advisory Group, Inc.

Page A-1