WellCare of Georgia, Inc.: performance improvement projects report, SFY 2012

State of Georgia
Department of Community Health Georgia Families Program
WellCare of Georgia, Inc.
PERFORMANCE IMPROVEMENT PROJECTS REPORT
SFY 2012
October 2011

TABLE OF CONTENTS
1. BACKGROUND ...................................................................................................................... 1-1
CMO Overview..................................................................................................................................1-2 Study Rationale ................................................................................................................................1-2 Study Summary ................................................................................................................................1-3 Validation Overview ..........................................................................................................................1-4 HSAG's New Validation Scoring Methodology .................................................................................1-6
2. FINDINGS .......................................................................................................................... 2-1
Aggregate Validation Findings ..........................................................................................................2-1 Design ............................................................................................................................................2-3 Implementation ...............................................................................................................................2-3 Outcomes .......................................................................................................................................2-3
PIP-Specific Outcomes .....................................................................................................................2-4 Analysis of Results .........................................................................................................................2-4 Adults' Access to Care....................................................................................................................2-6 Emergency Room Utilization ..........................................................................................................2-6 Children's Preventive Services .......................................................................................................2-7 Member and Provider Satisfaction..................................................................................................2-9 Member Satisfaction .....................................................................................................................2-10 Provider Satisfaction.....................................................................................................................2-10
3. STRENGTHS ......................................................................................................................... 3-1
Individual PIP Strengths ...................................................................................................................3-1 Global PIP Strengths ........................................................................................................................3-1
4. OPPORTUNITIES FOR IMPROVEMENT....................................................................................... 4-1
Individual PIP Issues ........................................................................................................................4-1 Global PIP Issues .............................................................................................................................4-1
Appendix A. PIP-SPECIFIC VALIDATION SCORES ....................................................................... A-1

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Health Services Advisory Group, Inc.

Page i

BACKGROUND
ACKNOWLEDGMENTS AND COPYRIGHTS
CAHPS refers to the Consumer Assessment of Healthcare Providers and Systems and is a registered trademark of the Agency for Healthcare Research and Quality (AHRQ). HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National Committee for Quality Assurance (NCQA).

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Health Services Advisory Group, Inc.

Page ii

Performance Improvement Project Validation Report WellCare of Georgia, Inc.
1. BACKGROUND
The Georgia Department of Community Health (DCH) is responsible for administering the Medicaid managed care program for the State of Georgia and overseeing quality improvement activities. The DCH requires its contracted Care Management Organizations (CMOs) to conduct performance improvement projects (PIPs) as set forth in 42 CFR 438.240 to assess and improve the quality of a targeted area of clinical or nonclinical care or service provided to members and to report the status and results of each PIP annually.
The validation of PIPs is one of three federally-mandated activities for state Medicaid managed care programs. The other two required activities include the evaluation of CMO compliance with State and federal regulations and the validation of CMO performance measures.
These three mandatory activities work together to ensure that the CMOs are providing quality care to their members. While a CMO's compliance with managed care regulations provides the organizational foundation for the delivery of quality health care, the calculation and reporting of performance measures provides a barometer of the quality and effectiveness of the care. When performance measures highlight areas of low performance, the DCH requires the CMOs to initiate PIPs to improve the quality of health care in targeted areas. PIPs are key tools in helping the DCH achieve goals and objectives outlined in its quality strategy; they provide the framework for monitoring, measuring and improving the delivery of health care.
The primary objective of PIP validation is to determine each CMO's compliance with requirements set forth in 42 CFR 438.240(b)(1), including:
Measurement of performance using objective quality indicators Implementation of system interventions to achieve improvement in quality Evaluation of the effectiveness of the interventions Planning and initiation of activities to increase or sustain improvement
To meet the federal requirement for the validation of PIPs, the DCH contracted with Health Services Advisory Group, Inc. (HSAG), the State's EQRO, to conduct the validation of WellCare of Georgia, Inc.'s (WellCare) PIPs. WellCare submitted PIPs to HSAG between June 30, 2011, and August 1, 2011, and HSAG validated the PIPs between July 1, 2011, and August 3, 2011. The validated data represents varying measurement time periods as described in Table 2-3 and Table 2-4.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 1-1 Health Services Advisory Group, Inc.

BACKGROUND

HSAG reviewed each PIP using the Centers for Medicare & Medicaid Services (CMS) validation protocol1-1 and evaluated two key components of the quality improvement process, as follows:
1. HSAG evaluated the technical structure of the PIPs to ensure WellCare designed, conducted and reported PIPs using sound methodology consistent with the CMS protocol for conducting PIPs. HSAG's review determined whether a PIP could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring sustained improvement.
2. HSAG evaluated the outcome of the PIP. Once designed, a PIP's effectiveness in improving outcomes depends on the systematic identification of barriers and the subsequent development of relevant interventions. Outcome evaluation determined whether WellCare improved its rates through implementation of effective processes (i.e., barrier analyses, intervention design and evaluation of results). A primary goal of HSAG's PIP validation is to ensure that the DCH and key stakeholders can have confidence that any reported improvement in outcomes is related to a given PIP.

CMO Overview
The DCH contracted with WellCare beginning in 2006 to provide services to the Georgia Families program (Medicaid and PeachCare for KidsTM) population. WellCare, a CMO, currently serves the eligible population in all geographic regions of Georgia--Atlanta, Central, East, North, Southeast and Southwest.

Study Rationale
The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas. Although HSAG has validated WellCare's PIPs for four years, the number of PIPs, study topics and study methods has evolved over time.
WellCare submitted nine (9) PIPs for validation. Six of the nine PIPs were ongoing PIPs and three were new additions. The PIP topics include:
Adults' Access to Care Annual Dental Visits Childhood Immunizations Childhood Obesity Emergency Room Utilization
1-1 U.S. Department of Health and Human Services, Centers for Medicare & Medicaid Services. EQR Managed Care Organization Protocol. Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, Final Protocol, Version 1.0, May 2002.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 1-2 Health Services Advisory Group, Inc.

BACKGROUND

Lead Screening in Children Member Satisfaction Provider Satisfaction Well-Child Visits
The effectiveness of WellCare's performance improvement efforts was measured using study indicators that aligned with HEDIS performance measures.

Study Summary

As noted in its Quality Strategic Plan Update (January 2010), the DCH identified the improvement of performance measures in the PIP studies as a key objective. The June 30, 2011, through August 1, 2011 PIP submission included seven clinical PIPs: Adults' Access to Care , Annual Dental Visits, Childhood Immunizations, Childhood Obesity, Emergency Room Utilization, Lead Screening in Children, and Well-Child Visits and two nonclinical PIPs: Member Satisfaction and Provider Satisfaction.

Five of the clinical PIP topics directly relate to performance measure outcomes that link to preventive health services delivery. They include: Annual Dental Visits, Childhood Immunizations, Childhood Obesity, Lead Screening in Children, and Well-Child Visits. Children's primary health care is a vital part of the effort to prevent, recognize and treat health conditions that can result in significant developmental and health status consequences for children and adolescents. Timely screening and interventions can reduce future complications such as those related to obesity.

The other two clinical PIPs, Adults' Access to Care and Emergency Room Utilization represent an essential component in developing a relationship with a health care provider and establishing a medical home, as well as ensuring that members have access to and receive care from the most appropriate care setting. These PIP topics represent a key area of focus for improvement.

Table 1-1 outlines the key study indicators incorporated for the seven HEDIS-based PIPs.

Table 1-1--PIP Study Topics and Indicator Descriptions

PIP Study Topic Adults' Access to Care
Annual Dental Visits
Childhood Immunizations

PIP Study Indicator Description
The percentage of members 2044 years of age who had an ambulatory or preventive care visit.
The percentage of members who had at least one dental visit: 23 years of age, and 221 years of age.
The percentage of children 2 years of age who had four diphtheria, tetanus and acellular pertussis (DTaP); three polio (IVP); one measles, mumps and rubella (MMR); two H influenza type B (Hib); three hepatitis B; and one chicken pox (VZN) by their second birthday.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 1-3 Health Services Advisory Group, Inc.

BACKGROUND

PIP Study Topic Childhood Obesity
Emergency Room Utilization Lead Screening in Children Well-Child Visits

PIP Study Indicator Description
The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation, nutrition counseling and physical activity counseling.
The number of emergency department visits that did not result in an inpatient stay, per 1,000 member months.
The percentage of children 2 years of age who had one or more capillary or venous lead blood tests for lead poisoning by their second birthday.
The percentage of members who turned 15 months old during the measurement year and who had six or more well-child visits with a primary care provider (PCP) during their first 15 months of life.

Table 1-2 outlines the key study indicators incorporated for the two satisfaction-based PIPs.

The effectiveness of the Member Satisfaction PIP was measured using the Consumer Assessment of Healthcare Providers and Systems (CAHPS) Health Plan Survey 4.0H, Child Version measures. This survey provided information on parents' experiences with their child's provider and the care management organization.

The final WellCare PIP topic was Provider Satisfaction. WellCare contracted with a vendor to produce and administer a survey to document the effectiveness of this performance improvement project.

Table 1-2--Satisfaction-based PIP Study Indicators

Survey Type

Question

Survey Question

Member

"Using any number from 0 to 10, where 0 is the worst doctor possible

#24

and 10 is the best doctor possible, what number would you use to rate

your child's personal doctor?"

Member

"In the last 6 months, how often did your child's personal doctor seem

#23

informed and up to date about the care your child got from other

doctors/providers?"

Provider

#11*

"Specialist network has an adequate number of high quality specialists to whom I can refer my patients."

Provider

#5*

"Timeliness to answer questions and/or resolve problems."

Provider

#15*

"Timeliness of UM's precertification process."

* Providers were requested to respond if they agreed with the statement regarding the CMO.

Validation Overview
HSAG obtained the data needed to conduct the PIP validation from WellCare's PIP Summary Forms. These forms provided detailed information about WellCare's PIPs related to the activities they completed.
Each required activity was evaluated on one or more elements that form a valid PIP. The HSAG PIP Review Team scored each evaluation element within a given activity as Met, Partially Met,

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 1-4 Health Services Advisory Group, Inc.

BACKGROUND
Not Met, Not Applicable, or Not Assessed. HSAG designated some of the evaluation elements deemed pivotal to the PIP process as critical elements. For a PIP to produce valid and reliable results, all of the critical elements had to be Met. Given the importance of critical elements to the scoring methodology, any critical element that received a Not Met score resulted in an overall validation rating for the PIP of Not Met. A CMO would be given a Partially Met score if 60 percent to 79 percent of all evaluation elements were Met or one or more critical elements were Partially Met. HSAG provided a Point of Clarification when enhanced documentation would have demonstrated a stronger understanding and application of the PIP activities and evaluation elements. In addition to the validation status (e.g., Met) HSAG gave each PIP an overall percentage score for all evaluation elements (including critical elements). HSAG calculated the overall percentage score by dividing the total number of elements scored as Met by the total number of elements scored as Met, Partially Met and Not Met. HSAG also calculated a critical element percentage score by dividing the total number of critical elements scored as Met by the sum of the critical elements scored as Met, Partially Met and Not Met. Figure 1-1 illustrates the three study stages of the PIP process: Design, Implementation and Outcomes. Each sequential stage provides the foundation for the next stage. The Design stage establishes the methodological framework for the PIP. The activities in this section include development of the study topic, question, indicators and population. To implement successful improvement strategies, a strong study design is necessary.
Figure 1-1--PIP Study Stages
III. OUTCOMES
II. IMPLEMENTATION
I. DESIGN

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 1-5 Health Services Advisory Group, Inc.

BACKGROUND
Once the study design was established, the PIP process moved into the Implementation stage. This stage included data collection, sampling and interventions. During this stage, WellCare collected measurement data, evaluated and identified barriers to performance, and developed interventions targeted to improve outcomes. The implementation of effective improvement strategies is necessary to improve PIP outcomes. The final stage was Outcomes, which involved data analysis and the evaluation of real and sustained improvement based on reported results and statistical testing. Sustained improvement is achieved when outcomes exhibit statistical improvement over time and multiple measurements. This stage is the culmination of the previous two stages. If the study outcomes did not improve, WellCare's responsibility was to investigate the data it collected to ensure it had correctly identified the barriers and implemented targeted interventions to address the identified barriers. If it had not, WellCare would revise its interventions and collect additional data to remeasure and evaluate outcomes for improvement. This process becomes cyclical until sustained statistical improvement is achieved.
HSAG's New Validation Scoring Methodology
To ensure that WellCare achieves improvement in the study outcomes for all PIPs submitted for validation in the future, HSAG worked with the DCH to modify the existing PIP validation scoring methodology. These modifications will add emphasis to achieving improved study indicator outcomes while keeping the number of evaluation elements the same. The new PIP Validation Tool (new tool) is identical to the current PIP Validation Tool (current tool) for Activities I through VII. In Activity VIII (sufficient data analysis and interpretation), WellCare must present study results that are accurate, clear and easily understood. Sufficient data analysis and interpretation is now a critical element; therefore, if the study indicator results are not accurate, the PIP cannot receive an overall Met validation status. In Activity IX (real improvement achieved), the CMO must achieve statistically significant improvement for the study indicator outcomes between the baseline and remeasurement period. Real improvement achieved will now be a critical element for all PIPs that progress to this stage; therefore, any PIP that does not achieve statistically significant improvement will not receive an overall Met validation status. For Activity X (sustained improvement achieved), HSAG assesses each study indicator for sustained improvement after the PIP indicator achieves statistically significant improvement. For PIPs with multiple indicators, all indicators that can be assessed must achieve sustained improvement to receive a Met score for Activity X.
The new validation scoring methodology will be applied to the PIPs that WellCare will submit for validation from June 2012, through August 2012. In preparation for this change, HSAG first scored the PIPs using the current tool then with the new tool. The scores included in this report were calculated using the current tool and the scores using the new tool were provided for informational purposes only and reflect the validation scores WellCare would receive if HSAG validated the PIP using the modified validation scoring methodology described above.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 1-6 Health Services Advisory Group, Inc.

2. FINDINGS
for WellCare of Georgia, Inc.

Aggregate Validation Findings

HSAG organized, aggregated, and analyzed WellCare's PIP data to draw conclusions about the CMO's quality improvement efforts. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the outcomes associated with the implementation of interventions. Based on its technical review, HSAG determined the overall methodological validity of the PIPs using the current tool. Using the new tool, HSAG determined the overall methodological validity as well as the overall success in achieving improved study indicator outcomes. The scores provided in the new tool this year are for informational purposes only. The results using both tools are presented in Table 2-1.

Table 2-1--Performance Improvement Project Validation Scores for WellCare of Georgia, Inc.

PIP
Adults' Access to Care Annual Dental Visits Childhood Immunizations Childhood Obesity Emergency Room Utilization Lead Screening in Children Member Satisfaction Provider Satisfaction Well-Child Visits

Percentage Score of Evaluation Elements Met

Current Tool New Tool

100%

100%

89%

89%

92%

94%

84%

84%

95%

95%

98%

100%

90%

91%

89%

89%

98%

96%

Percentage Score of Critical Elements Met

Current Tool New Tool

100%

100%

80%

82%

100%

93%

85%

79%

100%

100%

100% 100% 100% 100%

100% 93% 93% 93%

Validation Status

Current Tool Met
Partially Met Met
Partially Met

New Tool Met
Partially Met Not Met Not Met

Met

Met

Met

Met

Met

Not Met

Met

Partially Met

Met

Not Met

Using the current tool, seven PIPs received an overall Met validation status while two PIPs-- Annual Dental Visits and Childhood Obesity received a Partially Met validation status due to inaccurate documentation of the denominator for the study indicators. When the scoring methodology of the new tool was applied, two PIPs--Annual Dental Visits and Provider Satisfaction--received a Partially Met validation status. The new tool also scored down the Annual Dental Visits PIP because of inaccurate study indicators. For the Provider Satisfaction PIP, not all of the study indicators demonstrated statistically significant improvement. Four PIPs--Childhood Immunizations, Childhood Obesity, Member Satisfaction and Well-Child Visits received a Not Met validation status, since the study indicator(s) did not achieve statistically significant improvement.

Table 2-2 displays the combined validation results for all nine WellCare PIPs validated during FY 2012. This table illustrates the CMO's application of the PIP process and its success in implementing the study. Each activity is composed of individual evaluation elements scored as

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-1 Health Services Advisory Group, Inc.

FINDINGS

Met, Partially Met, or Not Met. Elements receiving a Met score have satisfied the necessary technical requirements for a specific element. The validation results presented in Table 2-2 show the percentage of applicable evaluation elements that received a Met score by activity for both the current and new tool. Additionally, HSAG calculated an overall score across all activities. Appendix A provides the detailed validation scores from the current tool for each of the nine PIPs.

Table 2-2--Performance Improvement Project Validation Results for WellCare of Georgia, Inc. (N=9 PIPs)

Study Stage

Activity

Percentage of Applicable Elements

Scored Met

Current Tool1

New Tool2

I. Appropriate Study Topic

98% (49/50)

98% (49/50)

Design

II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s)

100% (18/18)
89% (48/54)

100% (18/18)
89% (48/54)

IV. Correctly Identified Study Population

100% (27/27)

100% (27/27)

Design Total

95% (142/149)

95% (142/149)

V.

Valid Sampling Techniques (if sampling was used)

100% (36/36)

100% (36/36)

Implementation VI. Accurate/Complete Data Collection

99% (71/72)

99% (71/72)

VII. Appropriate Improvement Strategies

97% (32/33)

97% (32/33)

Implementation Total

99% (139/141)

99% (139/141)

VIII. Sufficient Data Analysis and Interpretation

94% (73/78)

94% (73/78)

Outcomes IX. Real Improvement Achieved

61% (22/36)

61% (22/36)

X. Sustained Improvement Achieved

60%

100%

(3/5)

(2/2)

Outcomes Total

82% (98/119)

84% (97/116)

Percentage Score of Applicable Evaluation Elements Met

93% (379/409)

93% (378/406)

1 The current tool was used to score the CMO for the current validation year, FY 2012. 2 The new tool incorporated the revised scoring methodology for Activities VIII through X which will be used for next year's

validation, FY 2013, and is provided for informational purposes only.

Of the nine PIPs evaluated for real improvement, only five PIPs were evaluated for sustained improvement using the current tool.

Only two of those five PIPs could be evaluated for sustained improvement using the new tool, For the new tool, the CMO must

first achieve statistically significant improvement in order to be evaluated for sustained improvement in a subsequent

remeasurement period.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-2 Health Services Advisory Group, Inc.

FINDINGS
Overall, 93 percent of the evaluation elements across all nine PIPs received a score of Met. This was true for both the current tool and the new tool. The 93 percent score demonstrates a sound application of the PIP process. While WellCare's strong performance in the Design and Implementation stages indicated that each PIP was designed appropriately to measure outcomes and improvement, WellCare was less successful in the Outcomes stage. The following subsections highlight HSAG's validation findings associated with each of the three PIP stages.
Design
WellCare met 100 percent of the requirements across all nine PIPs for two of the four activities within the Design stage. As part of the validation process, HSAG assessed whether WellCare followed HEDIS specifications when defining its study indicators and found that for the Annual Dental Visits and Childhood Obesity PIPs, WellCare incorrectly defined the study indicators' denominator in Activity III, resulting in a lower score for that activity (50 percent). Overall, WellCare designed scientifically sound studies that were supported by the use of key research principles. The technical design of each PIP was sufficient to measure and monitor PIP outcomes associated with WellCare's improvement strategies. The solid design of the PIPs allowed the successful progression to the next stage of the PIP process.
Implementation
WellCare met 100 percent of the requirements for the sampling activity, 99 percent for the data collection activity, and 97 percent for the implementation of improvement strategies in the Implementation phase. With the successful implementation of appropriate improvement strategies, the CMO should be able to achieve improved outcomes in the future.
Outcomes
WellCare was successful in analyzing and interpreting its results; however, not all of the study indicator outcomes achieved statistically significant improvement. Without statistically significant improvement, the CMO either did not demonstrate improvement or it could not be determined whether the improvement was due to the implementation of the CMO's improvement strategy or due to chance.
Using the current tool, five PIPs (Adults' Access to Care, Childhood Immunizations, Lead Screening in Children, Member Satisfaction and Provider Satisfaction) were evaluated for sustained improvement. Three of the five PIPs (Adults' Access to Care, Lead Screening in Children and Provider Satisfaction) that were assessed for sustained improvement achieved sustained improvement. Sustained improvement is defined as improvement in performance over baseline, which is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect improvement when compared to the baseline results.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-3 Health Services Advisory Group, Inc.

FINDINGS
When the new tool's scoring methodology was applied, HSAG could only assess the two PIPs (Adults' Access to Care and Provider Satisfaction) that achieved statistically significant improvement with a subsequent measurement period so that Activity X could be assessed. Both PIPs sustained the statistically significant improvement over that subsequent measurement period.
PIP-Specific Outcomes

Analysis of Results

Table 2-3 and Table 2-4 display the outcome data for WellCare's nine PIPs.

Table 2-3--HEDIS-based Performance Improvement Project Outcomes for WellCare of Georgia, Inc.

PIP Study Indicator
The percentage of members 2044 years of age who had an ambulatory or preventive care visit.
The percentage of children who received the recommended vaccinations based on the Childhood Immunization Status--Combo 2 (4:3:1:2:3:1) guidelines.
The percentage of children 2 years of age who received one blood lead test (capillary or venous) on or before their second birthday.
The percentage of children who had six or more well-child visits with a PCP during their first 15 months of life.
PIP Study Indicator
Percentage of members 23 years of age who had at least one dental visit. Percentage of members 221 years of age who had at least one dental visit.

Sustained Baseline Period Remeasurement 1 Remeasurement 2 Improvement
(1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10) Current New Tool^ Tool
Adults' Access to Care

78.6%

84.7%*

85.4%*

Yes Yes

Childhood Immunizations

75.9%

81.0%

75.9%

No



Lead Screening in Children

65.9%

67.4%

73.0%

Yes



Well-Child Visits

57.4%

57.4%

59.1%





Sustained Baseline Period Remeasurement 1 Remeasurement 2 Improvement
(1/1/0912/31/09) (1/1/1012/31/10) (1/1/1112/31/11) Current New Tool^ Tool
Annual Dental Visits

65.2%

67.5%*







40.4%

45.5%*







WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-4 Health Services Advisory Group, Inc.

FINDINGS

PIP Study Indicator

Sustained Baseline Period Remeasurement 1 Remeasurement 2 Improvement
(1/1/0912/31/09) (1/1/1012/31/10) (1/1/1112/31/11) Current New Tool^ Tool

Childhood Obesity

The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of BMI percentile documentation.

36.5%

30.4%







The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of counseling for nutrition.

42.3%

48.9%







The percentage of members 317 years of age who had an outpatient visit with a PCP or OB/GYN and who had evidence of counseling for physical activity.

38.7%

30.9%*







Emergency Room Utilization

The number of emergency room visits

that did not result in an inpatient stay

65.9

per 1,000 member months

61.7*







The PIP did not report Remeasurement 1 results and could not be assessed for real or sustained improvement, or the PIP did not report Remeasurement 2 results and could not be assessed for sustained improvement.
Improvement over baseline must occur before sustained improvement can be assessed using the current tool. Using the new tool, statistically significant improvement over baseline must occur before sustained improvement can be assessed.
A subsequent measurement period is required before sustained improvement can be assessed.
* Designates statistically significant improvement over the prior measurement period (p value < 0.05).
* Designates statistically significant decline in performance over the prior measurement period (p value < 0.05). ^ Sustained improvement in the current tool is defined as improvement in performance over baseline, which is maintained or increased
for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect improvement when compared to the baseline results.
Sustained improvement in the new tool is defined as statistically significant improvement in performance over baseline, which is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results

The following section discusses the improvement strategies the CMO implemented in conjunction with the PIP study indicator results. The identification of barriers through barrier analysis and the subsequent selection of appropriate interventions to address those barriers are necessary steps to improve outcomes. WellCare's choice of interventions, the combination of intervention types and the sequence of intervention implementation are all essential to its overall success.
Comparisons to HEDIS benchmarks were made using the Medicaid HEDIS 2010 Audit, Means, Percentiles and Ratios.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-5 Health Services Advisory Group, Inc.

FINDINGS

Adults' Access to Care
The Adults' Access to Care PIP demonstrated statistically significant improvement from Remeasurement 1 to Remeasurement 2. The percentage of adult members who accessed ambulatory or preventive care during the measurement year increased by 0.7 percentage points to 85.4 percent. Statistically significant improvement is the standard for assessing real improvement and supports the conclusion that the improvement was not due to chance. Although WellCare's performance improved, it remained 3.4 percentage points below the FY 2010 DCH target (88.8 percent) and fell between the national HEDIS 2010 Medicaid 50th and 75th percentiles (82.9 percent and 86.7 percent, respectively). However, the Remeasurement 2 results demonstrated that the CMO was able to sustain the statistically significant improvement that was first achieved from baseline to Remeasurement 1.
For the Adults' Access to Care PIP, WellCare identified the provider's lack of understanding regarding the need to provide preventive health services as the primary barrier. WellCare implemented sequential interventions specifically targeted to the barrier, including the following:
Reviewed medical records to identify providers noncompliant with adult preventive health care guidelines.
Updated adult preventive health care guidelines. Distributed adult preventive health care guidelines through the provider handbook. Distributed adult preventive health care guidelines through the member newsletter. Posted the adult preventive health care guidelines on the Web site and included information
in the provider newsletter.
In the last quarter of 2009, WellCare distributed the 2009 adult preventive health care guidelines through both the member newsletter and the member handbook, which the CMO will continue to do throughout the study. Additionally in 2009, the CMO conducted its quarterly quality improvement meeting and identified through a cause and effect diagram that members were going to the emergency room (ER) instead of a PCP; therefore, preventive services were not being performed. The CMO implemented a two-pronged approach to address this pattern. First, the CMO initiated system interventions ensuring that members had access to preventive services. Second, the CMO realigned staff resources so it could conduct focused member outreach to members within 48 hours of an ER visit. The outreach consisted of member education on the PCP's role and assistance with care and/or transportation needs. The CMO also created a database to track member contacts. In July 2010, the CMO launched a provider incentive program to reward PCPs for following the preventive health guidelines.

Emergency Room Utilization
The Emergency Room Utilization PIP study indicator outcome demonstrated a statistically significant decrease in emergency room visits from 65.9 visits per 1000 member months to 61.7 visits per 1000 member months, which represented an improvement. While the emergency room utilization measure included both emergent and nonemergent visits, the premise was that by reducing the nonemergent visits the overall utilization rate would decrease. WellCare's

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-6 Health Services Advisory Group, Inc.

FINDINGS
emergency room utilization was above the FY 2010 DCH target (48.4 percent) and between the national HEDIS 2010 Medicaid 25th percentile and the 50th percentile (58.5 per 1000 member months and 67.7 per 1000 member months, respectively). For this measure, the HEDIS 2010 Medicaid 10th percentile is the top level of performance.
WellCare used subgroup analyses to focus on the avoidable ER visits by diagnostic category, by age groups and a separate analysis of the outreach calls to identify barriers specific to the members; however, WellCare did not provide detail describing its methods used to evaluate the effectiveness of its interventions targeted to each subgroup. Intervention descriptions should provide enough detail that the intervention can be thoroughly evaluated during validation.
Work groups were formed to implement targeted interventions to address these barriers. Based on the analyses results, WellCare's outreach promoted the PCP medical home and targeted members 5-to-12 years of age and 18-to-34 years of age with avoidable emergency room visits based on diagnoses. Additionally, the CMO launched an educational campaign that targeted members 5-to-8 years of age. The campaign materials included:
Cover letter. Educational book titled What To Do When Your Child Gets Sick. Digital Thermometer. Educational flyer.
Children's Preventive Services
The performance for two PIPs--Lead Screening in Children and Well-Child Visits and the second study indicator for the Childhood Obesity PIP improved from the prior measurement period; however, the increases were not statistically significant. Only the two study indicators for the Annual Dental Visits PIP demonstrated a statistically significant increase from baseline to Remeasurement 1. The Lead Screening in Children, Well-Child Visits and Childhood Obesity PIP study indicator rates remained below the FY 2010 DCH target rates for these measures. Only the Childhood Immunizations PIP and the Annual Dental Visits PIP for the 2-to-3-year-old age group were above the FY 2009 DCH target and the FY 2010 DCH target, respectively.
For the Annual Dental Visits PIP, the CMO did not develop any targeted interventions. The global interventions consisted of handbooks, letters and reminder cards.
WellCare identified the lack of provider and member knowledge regarding the required screenings and immunization schedules as primary barriers for three of its PIPs--Childhood Immunizations, Lead Screening in Children and Well-Child Visits. WellCare documented more than nine ongoing interventions for each of these PIPs and then implemented both provider and member incentive programs in CY 2011. The provider incentive program consisted of a twenty dollar bonus given to providers for each required immunization, each lead screening test and each well-child visit. Members received a ten dollar gift card for each of the requirements they completed: all well-child visits; all immunizations; and a lead screening test. The CMO should

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-7 Health Services Advisory Group, Inc.

FINDINGS
evaluate the effectiveness of the new incentive interventions quarterly to facilitate timely revisions to the interventions if the effect on the outcomes is less than expected.
For the Childhood Obesity PIP, WellCare used reminder cards and handbooks to educate both members and providers. To sustain improvement over subsequent measurement periods, the plan should implement more targeted interventions, including system-based interventions to ensure that any improvement is sustainable over time. System interventions include organization-wide initiatives such as, but not limited to, changes in policy, changes in staffing resources, implementing information system modifications/enhancements.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-8 Health Services Advisory Group, Inc.

FINDINGS

Member and Provider Satisfaction

Table 2-4--Satisfaction-based Performance Improvement Project Outcomes for WellCare of Georgia, Inc.

PIP Study Indicator

Baseline Period (2/1/095/31/09)

Remeasurement 1 (2/1/105/31/10)

Remeasurement 2 (2/1/115/31/11)

Sustained Improvement Current New Tool^ Tool

Member Satisfaction

1. The percentage of members responding with either a "9" or "10" to Q24--"Using any number from 0 to 10, where 0 is the worst personal doctor possible and 10 is the best personal doctor possible, what number would you use to rate your child's personal doctor?"
2. The percentage of eligible members responding with either "Always" or "Usually" to Q23--"In the last 6 months, how often did your child's personal doctor seem informed and up to date about the care your child got from other doctors/providers?"

72.2% 77.1%

71.2% 78.4%

72.6%





74.6%

No



PIP Study Indicator

Baseline Period (10/1/069/30/07)

Remeasurement 1 (10/1/079/30/08)

Remeasurement 2 (10/1/089/30/09)

Remeasurement 3 (10/1/09-9/30/10)

Sustained Improvement
Current New Tool^ Tool

Provider Satisfaction

1. The percentage of providers answering "Excellent" or "Very Good" to Q11-- "Specialist network has an adequate number of high quality specialists to whom I can refer my patients."
2. The percentage of providers answering "Excellent" or "Very Good" to Q5-- "Timeliness to answer and/or resolve problems."
3. The percentage of providers answering "Excellent" or "Very Good" to Q15-- "Timeliness of UM's precertification process."

22.2% 22.2% 22.5%

19.7% 29.6%* 25.5%

24.7% 31.3% 29.3%

24.1%

Yes



33.6%*

Yes Yes

30.3%

Yes Yes

Improvement over baseline must occur before sustained improvement can be assessed using the current tool. Using the new tool, statistically significant improvement over baseline must occur before sustained improvement can be assessed.

A subsequent measurement period is required before sustained improvement can be assessed.
* Designates statistically significant improvement over the prior measurement period (p value < 0.05). ^ Sustained improvement in the current tool is defined as improvement in performance over baseline, which is maintained or increased for at least one
subsequent measurement period. Additionally, the most current measurement period's results must reflect improvement when compared to the baseline results. Sustained improvement in the new tool is defined as statistically significant improvement in performance over baseline, which is maintained or increased for at least one subsequent measurement period. Additionally, the most current measurement period's results must reflect statistically significant improvement when compared to the baseline results

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-9 Health Services Advisory Group, Inc.

FINDINGS

Member Satisfaction
The first study indicator outcome for the Member Satisfaction PIP ("...what number would you use to rate your child's personal doctor?") increased during the most recent measurement period, although the increase was not statistically significant. The second study indicator outcome ("...how often did your child's personal doctor seem informed...?") decreased by 3.8 percentage points and the improvement noted from baseline to Remeasurement 1 was not sustained. When the scoring methodology for the new tool was applied, none of the study indicators had achieved statistically significant improvement; therefore, none of the indicators could be assessed for sustained improvement.
The Member Satisfaction PIP outcomes remained basically unchanged from the baseline period. WellCare did not initiate any interventions in CY 2008. Additionally, of all the interventions that the CMO implemented in CY 2009, only one directly related to the study outcomes--the CMO distributed a Patient Safety Tip Sheet to providers addressing the lack of coordination between primary care providers and specialists. The CMO's other interventions dealt with barriers such as the prior-authorization process, members unaware of translation services, provider directories not available on the Web portal, members not understanding how to change providers, coordination of care, etc. Even if these interventions affect identified barriers, they will not affect the outcomes for the PIP study indicators. WellCare should identify and address the barriers that prevented members from rating their child's doctor as being "the best personal doctor" and from perceiving their child's provider as being informed and up to date regarding the care their child received from other providers.
Improvement strategies implemented in 2010 and early 2011 consisted primarily of articles in newsletters. In general, newsletters have only short-term effects on study outcomes, with very limited effects on member-based outcomes. A single article distributed only once (similar to WellCare's intervention) has even less impact. As soon as the remeasurement rates were available, the plan should have conducted another causal/barrier analysis to identify specific, actionable barriers and selected interventions that were more appropriate. Additionally, the plan should have implemented more targeted interventions, including system-based interventions, to ensure that any improvement was sustainable over time.

Provider Satisfaction
Rates for two of the three Provider Satisfaction PIP's study indicators increased from the second to the third remeasurement. For the second study indicator, the increase was statistically significant. For the current tool, all three study indicators demonstrated sustained improvement since they improved between measurement periods and remained above the baseline rate. Applying the scoring methodology for the new tool, only the second and third study indicators had achieved statistically significant improvement which was sustained over a subsequent measurement period. These findings highlight success in the implementation of quality strategies for improving overall satisfaction.
For the Provider Satisfaction PIP, WellCare implemented numerous targeted interventions that linked directly to the identified barriers. Examples of the CMO's interventions addressing

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-10 Health Services Advisory Group, Inc.

FINDINGS
"timeliness to answer and/or resolve problems" and "timeliness of UM's pre-certification process" included the following:
Documenting provider concerns and feedback identified by provider relations representatives in a database, then training representatives on how to trend provider dissatisfaction.
Opening a customer service call center for providers. Implementing a new prior-authorization checklist. Employing a reconsideration process for authorization requests that included a peer-to-peer
process. Incorporating a new database to enhance timeliness and tracking of prior authorizations.
The CMO educated staff and providers on all initiatives. The study outcomes for the second and third study indicators for this PIP increased over time, demonstrating both real and sustained improvement. WellCare, as part of its quarterly barrier analysis, prioritized the identified barriers to provider satisfaction. The reevaluation of quality strategies allowed the CMO to address changes in PIP outcomes more effectively. For the first study indicator ("specialist network has an adequate number of high quality specialists to whom I can refer my patients"), the CMO responded to the decrease in the remeasurement result and implemented focused interventions that used provider feedback and referral patterns to recruit needed specialists. The result was an upward trend by the second remeasurement period and a non-statistically significant decline from Remeasurement 2 to Remeasurement 3.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 2-11 Health Services Advisory Group, Inc.

3. STRENGTHS
for WellCare of Georgia, Inc.
Individual PIP Strengths
The Adults' Access to Care PIP demonstrated statistically significant improvement from Remeasurement 1 to Remeasurement 2. The Remeasurement 2 results demonstrated that the CMO was able to sustain the statistically significant improvement that was first achieved from baseline to Remeasurement 1. WellCare's success on this PIP could affect the CMO's general performance on the Adults' Access to Preventive/Ambulatory Health Services HEDIS measure.
The two study indicators for the Annual Dental Visits PIP demonstrated a statistically significant increase from baseline to Remeasurement 1. Additionally, the dental visit rate for 2-to-3-yearolds was above the FY 2010 DCH target.
Performance on the Provider Satisfaction PIPs suggested a thorough application of the PIP study design and the development and implementation of appropriate interventions. Moreover, WellCare's implementation of the revised Provider Satisfaction interventions may have contributed to the CMO achieving real and sustained improvement.
Global PIP Strengths
Seven of the nine PIPs received an overall Met validation status using the current tool, which represented an area of strength for WellCare in documentation of its PIPs and provided confidence in the technical aspects of the studies. The performance on these PIPs suggests a thorough application of the PIP Design stage. The sound study design of the PIPs created the foundation for the CMO to progress to subsequent PIP stages--implementing improvement strategies and accurately assessing study outcomes. The CMO appeared to appropriately select and conduct the sampling and data collection activities of the Implementation stage. These activities ensured that the CMO properly defined and collected the necessary data to produce accurate study indicator rates. Additionally, WellCare appropriately documented improvement strategies, an activity which ensured that study outcomes could improve. Furthermore, in the Outcomes stage, the CMO properly analyzed and interpreted the outcome results.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 3-1 Health Services Advisory Group, Inc.

4. OPPORTUNITIES FOR IMPROVEMENT
for WellCare of Georgia, Inc.

Individual PIP Issues
To maintain high validation scores when the new scoring methodology is applied for PIPs submitted in 2012, WellCare will need to concentrate its efforts on the six PIPs--Annual Dental Visits, Childhood Immunizations, Childhood Obesity, Member Satisfaction, Provider Satisfaction and Well-Child Visits that would not receive Met validation status due to either a lack of statistically significant improvement or the lack of sustained improvement.
For the six PIPs that have not demonstrated statistically significant improvement for all of the study indicators or sustained the statistically significant improvement, WellCare should incorporate a method to evaluate the success of its interventions. The CMO should analyze its data to determine if any subgroup within its population had a disproportionately lower rate that negatively affected the overall rates. This "drill-down" type of analysis should be conducted both before and after the implementation of any intervention. For example, WellCare should evaluate whether rates differ by geographic region, gender, race/ethnicity, age, etc. The CMO could then target its interventions to the subgroups with the lowest rates, thereby facilitating the implementation of more precise, concentrated interventions. The process of targeting interventions to the appropriate subgroup is more efficient and effective. Global interventions directed at the entire eligible population may not achieve the desired results while requiring the same resources. After implementation of the targeted intervention, the CMO should again evaluate the applicable subgroups to determine the intervention's success. The documentation of this entire process should be included in the PIP submission.
Despite the Member Satisfaction PIP receiving an overall Met validation status, providing confidence in the study results, the interventions implemented were not likely to induce permanent change. In fact, the reported change in rates was not statistically significant and could be due to chance rather than any of the CMO's efforts. Any interventions should directly affect the identified barrier and should not rely on one-time, member-based actions. Additionally, the CMO should include an evaluation of its interventions. For the CMO's newsletter intervention, it would be difficult for the CMO to know how many members read the newsletter article and applied the information during their next provider visit. The decline in member satisfaction between the two measurement periods emphasizes the CMO's need to revise its improvement strategies.
Global PIP Issues
The CMO should be mindful that the submission of PIPs for validation will be an annual activity without an opportunity to resubmit. WellCare should carefully complete all necessary documentation. The CMO must ensure that the information it reports in the demographic page is accurate, complete and consistent with DCH's expectations of the study. The CMO should refer

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 4-1 Health Services Advisory Group, Inc.

OPPORTUNITIES FOR IMPROVEMENT
to the PIP Validation Tool and address all Points of Clarification and all Partially Met and Not Met scores before the next submission in 2012.
Generally, WellCare did not demonstrate improvement in outcomes. WellCare's PIPs were well designed and documented; however, the implementation of improvement strategies has been ineffective in producing long-term, sustained change in outcomes. WellCare's focus should shift to developing appropriate improvement strategies. Without effective strategies, the CMO will not be able to improve PIP outcomes.
WellCare should include the methods used to evaluate the effectiveness of its interventions. Intervention descriptions should provide enough detail that the intervention can be thoroughly evaluated during validation.

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page 4-2 Health Services Advisory Group, Inc.

APPENDIX A. PIP-SPECIFIC VALIDATION SCORES
for WellCare of Georgia, Inc.
Table A-1--WellCare of Georgia, Inc.'s FY 2012 PIP Performance1

Percentage of Applicable Evaluation Elements Scored Met

Study Stage

Activity

Adults' Access to
Care

Annual

Childhood Childhood

Dental Visits Immunizations Obesity

ER Utilization

I. Appropriate Study Topic

100%

100%

100%

83%

100%

Design

II. Clearly Defined, Answerable Study Question(s)
III. Clearly Defined Study Indicator(s)

100% 100%

100% 50%

100% 100%

100% 50%

100% 100%

IV. Correctly Identified Study Population

100%

100%

100%

100%

100%

Design Total 100%

82%

100%

76%

100%

V. Valid Sampling Techniques

Not

Not

(if sampling was used)

Applicable Applicable

100%

100%

Not Applicable

Implementation

VI. Accurate/Complete Data Collection

100%

100%

100%

100%

80%

VII. Appropriate Improvement Strategies

100%

100%

100%

100%

67%

Implementation Total 100%

100%

100%

100%

75%

VIII. Sufficient Data Analysis and Interpretation

100%

88%

100%

89%

100%

Outcomes

IX. Real Improvement Achieved

100%

100%

25%

25%

100%

X. Sustained Improvement Achieved

100%

Not Assessed

0%

Not Assessed

Not Assessed

Outcomes Total 100%

92%

71%

69%

100%

Validation Status

Met

Partially Met

Met

Partially Met

Met

1 Scores and validation status for the PIPs are based on the current tool, and therefore, the current scoring methodology.

Lead Screening in
Children 100%
100%
100%
100% 100% 100%
100%
100% 100% 100%
75%
100% 93% Met

Member Satisfaction
100% 100%
100% 100% 100% 100% 100% 100% 100% 89% 25%
0% 64% Met

Provider Satisfaction
100% 100%
100% 100% 100% 100% 100% 100% 100% 78% 25% 100% 64% Met

Well-Child Visits 100%
100%
100%
100% 100% 100%
100%
100% 100% 100%
75% Not Assessed 92% Met

WellCare of Georgia, Inc. SFY 2012 PIP Validation Report State of Georgia

WellCare_GASFY2012_CMO_PIP-Val_Report_F2_1011 Page A-1 Health Services Advisory Group, Inc.