SFY 2011 external quality review annual report...for Georgia families care management organizations

State of Georgia
Department of Community Health
SFY 2011 EXTERNAL QUALITY REVIEW ANNUAL REPORT
INCLUDING CY 2009 PERFORMANCE MEASURES SFY 2010 REPORTED PERFORMANCE IMPROVEMENT PROJECTS
SFY 2011 COMPLIANCE REVIEWS
for
Georgia Families Care Management Organizations
August 2011
3133 East Camelback Road, Suite 300 Phoenix, AZ 85016 Phone 602.264.6382 Fax 602.241.0757

CONTENTS
1. Executive Summary ........................................................................................................................1-1 Purpose of Report .............................................................................................................................1-1 Overview of the External Quality Review ..........................................................................................1-2 Overall Findings, Conclusions, and Recommendations....................................................................1-2 Quality ............................................................................................................................................ 1-3 Access ...........................................................................................................................................1-4 Timeliness ...................................................................................................................................... 1-4 Conclusions ...................................................................................................................................1-5 Recommendations .........................................................................................................................1-5
2. Background and Overview .............................................................................................................2-1 Georgia Medicaid Managed Care Service Delivery System Overview .............................................2-1 Georgia Families Care Management Organizations .........................................................................2-2 AMERIGROUP Community Care ..................................................................................................2-2 Peach State Health Plan................................................................................................................2-2 WellCare of Georgia, Inc. ..............................................................................................................2-2 Georgia Families Quality Strategy.....................................................................................................2-2 Georgia Families Quality Initiatives Driving Improvement.................................................................2-3 Auto-Assignment Program.............................................................................................................2-3 Encounter Data Validation .............................................................................................................2-4 Quality Improvement and Performance Measure Transparency ...................................................2-4 Collaborative Partnerships.............................................................................................................2-4
3. Review of Compliance With Standards.........................................................................................3-1 Review of Compliance With Standards .............................................................................................3-1 Findings ............................................................................................................................................. 3-1 CMO Comparison Key Findings ....................................................................................................3-3 AMERIGROUP .................................................................................................................................. 3-3 Peach State.......................................................................................................................................3-4 WellCare of Georgia, Inc. ..................................................................................................................3-5 Recommendations .........................................................................................................................3-5 Conclusions ...................................................................................................................................3-5
4. Performance Measures...................................................................................................................4-1 Performance Measure Requirements and Targets ...........................................................................4-1 Findings ............................................................................................................................................. 4-2 Performance Measure Validation Key Findings.............................................................................4-2 Performance Measure Result Findings .........................................................................................4-2 CMO Comparison Key Findings ....................................................................................................4-9 Conclusions .................................................................................................................................4-16 Recommendations .......................................................................................................................4-17
5. Performance Improvement Projects..............................................................................................5-1 Validation of Performance Improvement Projects .............................................................................5-1 Findings ............................................................................................................................................. 5-2 Performance Improvement Project Validation Key Findings..........................................................5-2 CMO Comparison Key Findings ....................................................................................................5-3 Outcome Results ...........................................................................................................................5-4 Conclusions ...................................................................................................................................5-6 Recommendations .........................................................................................................................5-7

SFY 2011 External Quality Review Annual Report State of Georgia

Page i GA2010-11_EQR_AnnRpt_F2_0811

CONTENTS
6. CMO-specific Follow-up on Prior-Year Recommendations ........................................................6-1 Introduction .......................................................................................................................................6-1 AMERIGROUP Community Care......................................................................................................6-1 Review of Compliance With Operational Standards ......................................................................6-1 Validation of Performance Improvement Projects..........................................................................6-3 Validation of Performance Measures.............................................................................................6-5 Peach State Health Plan ...................................................................................................................6-5 Review of Compliance With Operational Standards ......................................................................6-5 Validation of Performance Improvement Projects..........................................................................6-7 Validation of Performance Measures.............................................................................................6-7 WellCare of Georgia, Inc. ..................................................................................................................6-8 Review of Compliance With Operational Standards ......................................................................6-8 Validation of Performance Improvement Projects..........................................................................6-9 Validation of Performance Measures...........................................................................................6-10
Appendix A. Methodology for Reviewing Compliance With Standards ........................................ A-1 Appendix B. Methodology for Conducting Validation of Performance Measures ....................... B-1 Appendix C. Methodology for Conducting Encounter Data Validation ......................................... C-1 Appendix D. Methodology for Conducting Validation of Performance Improvement Projects .. D-1

SFY 2011 External Quality Review Annual Report State of Georgia

Page ii GA2010-11_EQR_AnnRpt_F2_0811

1. Executive Summary
Purpose of Report
The Georgia Department of Community Health (DCH) is responsible for administering the Medicaid managed care program in the State of Georgia to approximately 1.1 million beneficiaries.1-1 DCH contracts with three privately owned managed care organizations, referred to by the State as care management organizations (CMOs), to deliver services to members who are enrolled in the State's Medicaid program and Children's Health Insurance Program (CHIP). The State refers to its Medicaid managed care program as Georgia Families and to its CHIP program as PeachCare for KidsTM. For the purposes of this report, "Georgia Families" refers to all members enrolled in managed care.
The Code of Federal Regulations (CFR) at 42 CFR 438.3581-2 requires that states use an external quality review organization (EQRO) to prepare an annual, independent technical report that analyzes and evaluates aggregated information on the quality and timeliness of, and access to, the health care services that managed care organizations provide.
The technical report must describe how conclusions were drawn as to the quality and timeliness of, and access to, care furnished by a state's managed care organizations. The report of results must also contain an assessment of the strengths and weaknesses of the managed care organizations regarding health care quality, timeliness and access and must make recommendations for improvement. Finally, the report must assess the degree to which the managed care organizations addressed recommendations made within the previous external quality review (EQR).
To comply with this requirement, DCH contracted with Health Services Advisory Group, Inc. (HSAG), an EQRO, to aggregate and analyze the Georgia Families CMOs' data and prepare an annual technical report.
This report provides:
An overview of the Georgia Families program. A description of the scope of EQR activities included in this report. An aggregate assessment of health care timeliness, access and quality across CMS-required
mandatory activities for compliance with standards, performance measures and quality improvement projects. CMO-specific findings and an assessment of CMO strengths and weaknesses.

1-1 Georgia Department of Community Health. Georgia Families Quality Strategic Plan Update, January 2010. 1-2 Department of Health and Human Services, Centers for Medicare & Medicaid Services. Federal Register/Vol. 68, No.
16/Friday, January 23, 2003/Rules and Regulations, p. 3597. 42 CFR Parts 433 and 438 Medicaid Program; External
Quality Review of Medicaid Managed Care Organizations, Final Rule.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 1-1 GA2010-11_EQR_AnnRpt_F2_0811

EXECUTIVE SUMMARY
Recommendations to DCH to improve the CMOs' compliance with State and federal requirements and subsequently to improve the quality, timeliness of and access to services provided to Georgia Families members.
Recommendations for the CMOs to improve member access to care, quality of care and timeliness of care.
Overview of the External Quality Review
To produce this report, HSAG analyzed and aggregated data submitted and/or gathered by the CMOs. The data addressed the following three federally mandated EQR activities:
Review of compliance with federal and State-specified operational standards. HSAG evaluated the CMOs' compliance with State and federal requirements for organizational and structural performance in three DCH-selected performance categories. HSAG conducted on-site compliance reviews in October 2010. The CMOs submitted documentation that covered the review period of October 1, 2009, through September 30, 2010. HSAG provided detailed, final audit reports to the CMOs and DCH in February 2011.
Validation of performance measures. HSAG validated performance measures required by DCH to evaluate the accuracy of the performance measure results reported by the CMOs. The validation also determined the extent to which DCH-specific performance measures calculated by the CMOs followed specifications established by DCH. HSAG assessed performance measure results and their impact on improving the health outcomes of members. HSAG began performance measure validation in February 2010 and completed validation in June 2010. The CMOs submitted performance measure data that generally reflected the period of January 1, 2009, through December 31, 2009. HSAG provided final performance measure validation reports to the CMOs and DCH in July 2010.
Validation of performance improvement projects (PIPs). HSAG reviewed PIPs for each CMO to ensure the CMOs designed, conducted and reported projects in a methodologically sound manner. HSAG assessed the PIPs for real improvements in care and services to give confidence to the reported improvements. HSAG assessed the CMOs' PIP outcomes and impacts on improving care and services provided to members. HSAG began PIP validation in July 2010 and completed validation in September 2010. The CMOs submitted PIP data that reflected varying time periods, depending on the PIP topic. HSAG provided final, CMO-specific PIP reports to the CMOs and DCH in December 2010.
Overall Findings, Conclusions, and Recommendations
The Centers for Medicare & Medicaid Services (CMS) chose the domains of quality, access and timeliness as keys to evaluating the performance of Medicaid managed care plans. In this report, HSAG provides overall findings, conclusions and recommendations regarding the CMOs' aggregate performance during the review period for each domain of care.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 1-2 GA2010-11_EQR_AnnRpt_F2_0811

EXECUTIVE SUMMARY
Quality
The quality domain of care relates to the CMOs' structural and operational characteristics and their ability to increase desired health outcomes for Georgia Families' members (through the provision of health care services).
Performance measures and PIP results are used to assess care delivered to members by the CMOs in areas such as preventive screenings and well-care visits, management of chronic disease and appropriate treatment for acute conditions, all of which are likely to improve health outcomes. In addition, DCH monitors aspects of each CMO's operational structure that supports the delivery of quality care including: the adoption of practice guidelines; the quality assessment and performance improvement program; and the CMOs' health information systems.
HSAG used the CMOs' performance measure rates (which reflect calendar year 2009 measurement data), PIP validation results and outcomes, and rates of compliance with review standards related to measurement and improvement to assess the quality domain of care.
The DCH set six CY 2009 performance measure targets for the CMOs. The CMO performance measure results showed that the overall CY 2009 CMO weighted average rates met the CY 2009 performance targets for Childhood Immunization Status--Combination 2 and Lead Screening in Children measures. Well-child visits and diabetes care measures showed the greatest opportunities for improvement for the CMOs as a whole. Individual CY 2009 CMO rates showed that AMERIGROUP met three of the six performance measure targets for CY 2009, followed by WellCare, which met two of six targets. Peach State did not achieve any of the CY 2009 performance targets and was the CMO identified with the most opportunity for improvement. When comparing the CMOs' performance to each other, although Peach State and WellCare had areas of strength, AMERIGROUP showed better performance overall.
The review of compliance with standards showed that all of the CMOs scored 100 percent on the standards related to clinical practice guidelines, quality assessment and performance improvement, and health information systems. This demonstrated that the CMOs had the organizational structure and systems to support the delivery of quality care, thus meeting the intent of State and federal requirements. Additionally, the CMOs reported valid performance measure rates and received high compliance scores for having required documentation for a quality management program; however, they demonstrated mixed results when impacting health care outcomes. CY 2009 performance measure rates showed the CMOs did not achieve all CY 2009 performance targets.
Performance improvement project results related to quality of care showed mixed results, with two of the three CMOs demonstrating sustained or improved performance measure rates when compared with the prior measurement period. Factors that appeared to contribute to their improved rates included: interventions that linked to identified barriers; timely interventions that allowed for improvement to take place during the measurement period; and interventions resulting in system changes. PIPs that did not achieve desired outcomes lacked many of these elements.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 1-3 GA2010-11_EQR_AnnRpt_F2_0811

EXECUTIVE SUMMARY

Access
The access domain of care relates to a CMO's standards, established by the State, to ensure the availability of and access to all covered services for Georgia Families members.
The DCH contracts require the CMOs to ensure access to and the availability of services to members. DCH uses HSAG to conduct monitoring processes, including audits, to assess CMO compliance with access standards. These standards include assessment of network adequacy and availability of services, coordination and continuity of care, and coverage of services.
Additionally, many performance measures fall under more than one domain of care. Measures such as well-care visits for children and adolescents, childhood immunizations, timeliness of prenatal care and postpartum care, cancer screening and diabetes care fall under both quality and access because members rely on access to and the availability of these services to receive care according to generally accepted clinical guidelines. Member satisfaction results also provide useful information to evaluate access to care.
The CMO weighted average showed a statistically significant rate increase between the CY 2008 rate and the CY 2009 rate for Adults' Access to Preventive/Ambulatory Health Services for members 2044 years of age. This measure was selected by DCH for the 2010 auto-assignment program and as a required PIP topic. These factors may have contributed to the improved performance.
While the access to care performance measure rates showed that the CMOs had adequate provider networks in place for members to access care, the performance measure rates for well-care visits for both children and adolescents remained low. This suggests that providers missed opportunities to provide well-care services when members sought care or that providers failed to properly code and document a delivered well-care service.
The CMOs' greatest opportunities for improvement in the area of access were related to women's health for cancer screening services and prenatal care. The CMOs should determine if there were structural barriers such as the distance from screening locations, limited hours of operation, lack of day care for children, and language and cultural factors that prevented members from accessing these services.
Timeliness
The timeliness domain of care relates to the CMOs' ability to: make timely utilization decisions based on the clinical urgency of the situation; minimize any disruptions to care; and provide a health care service quickly after a need is identified.
The DCH CMO contracts require that CMOs ensure timeliness of care. HSAG conducts review activities to assess the CMOs' compliance with these standards in areas such as: enrollee rights and protections; the grievance system; continuity and coordination of care; and utilization management. Performance measures such as childhood immunizations, well-care visits, prenatal and postpartum care fall under the timeliness domain of care because they relate to the provision of a health care

SFY 2011 External Quality Review Annual Report State of Georgia

Page 1-4 GA2010-11_EQR_AnnRpt_F2_0811

EXECUTIVE SUMMARY

service within a recommended period of time after a need is identified. Members' satisfaction with receiving timely care also falls under the timeliness domain of care.
The CMO weighted average rate for the Childhood Immunization Status--Combination 2 achieved the DCH-established CY 2009 performance measure target. DCH added childhood immunizations as one of the auto-assignment measures in April 2010, established a performance target in 2009, and selected this topic as a formal PIP. DCH's emphasis on this measure may have further impact on the CMOs' rates in the future. Other performance measures related to timeliness, such as well-child visits and prenatal and postpartum care, also present opportunities for improvement.
HSAG validated six PIPs per CMO during the review period. Only two of the 18 total PIPs evaluated during the review period progressed to at least two periods of remeasurement, which allowed HSAG to assess for sustained improvement. Both of these PIPs targeted provider satisfaction and both PIPs demonstrated sustained improvement. Several of the selected study indicators showed improvement in providers' satisfaction with the timeliness of CMO decision making for claims payment and prior authorization. These areas can impact the timeliness of services provided to members after a need is identified by minimizing delays.

Conclusions
Based on a review of performance measure results, PIP outcomes, and compliance with State and federal standards, HSAG found that the CMOs had organizational structures and resources to support the quality, timeliness of and access to care delivered to its Georgia Families members. Overall, the CMOs demonstrated strength in the area of childhood immunizations and lead screening, with the CMO weighted average rates achieving the CY 2009 performance targets. The CMOs performed well related to compliance with standards reviewed, which set a foundation for the quality program.
Based on the review of the submitted PIP documents, the CMOs had the structure and resources necessary to support quality improvement and HSAG found that the CMOs generally performed well on the documentation requirements. However, the CMOs did not demonstrate the application/implementation, measurement, monitoring and evaluation of improved care and service delivery that should have resulted from implementation of the PIPs. This was evidenced by high compliance audit scores received by the CMOs but generally low performance measure rates that did not meet the DCH targets. Overall, the CMOs were able to report valid and reliable performance measure rates but the rates generally remained below national Medicaid averages. These issues demonstrate a disconnect between documentation compliance and actual improvement in member health outcomes. This disconnect represents the greatest obstacle to improving health outcomes for Georgia Families members.

Recommendations
Based on the review of the CMOs' performance on the performance measure results, PIP outcomes, and compliance with State and federal standards, HSAG provides the following global recommendations. Specific recommendations based on each activity's review findings are included at the end of each section.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 1-5 GA2010-11_EQR_AnnRpt_F2_0811

EXECUTIVE SUMMARY
DCH should continue to work with the CMOs on prioritizing areas for improvement on an annual basis given the extensive number of performance measures to help align and focus resources toward a common goal.
DCH and the CMOs need to identify greater opportunities to collaborate as a strategy for increasing performance. HSAG noted that each CMO had strengths and successes that could be spread to other CMOs, a practice that was not evident.
DCH, the CMOs and the EQRO need to explore approaches and develop a plan for moving the CMOs from documentation compliance to improvement of health outcomes.
DCH has already begun taking action related to HSAG's recommendations. DCH's focus for its 2011 CMO conference was on highlighting and prioritizing areas for improvement, initiating dialogue among the CMOs to foster collaboration, and exploring options to place a greater emphasis on CMO accountability for improved health outcomes across the federally required activities.
HSAG will evaluate DCH's and the CMOs' progress in the next annual report.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 1-6 GA2010-11_EQR_AnnRpt_F2_0811

2. Background and Overview
Georgia Medicaid Managed Care Service Delivery System Overview
DCH was created in 1999 to serve as the lead agency for health care planning and purchasing issues in Georgia. The General Assembly created DCH by consolidating four agencies involved in purchasing, planning and regulating health care. As the largest division in the Department of Community Health, the Medicaid Division administers the Medicaid and CHIP programs, which provide health care for children, pregnant women, and people who are aged, blind, and disabled. The Department is designated as the single State agency for Medicaid.
The State of Georgia implemented its Georgia Families program in 2006. Georgia Families delivers health care services to members of Medicaid and PeachCare for KidsTM within a managed care model. Through its three CMOs that DCH selected in a competitive bid process, DCH provides services to individuals enrolled in its Georgia Families program.
By providing a choice of health plans, Georgia Families allows members to select a CMO that fits their needs. DCH contracted with each CMO to deliver services within three or more of the six designated geographic regions. To ensure a smooth and successful transition from fee for service to the Georgia Families managed care program, DCH implemented the program in two phases, beginning with two of the six regions (Atlanta and Central) on June 1, 2006, followed by the remaining four regions (North, East, Southeast, and Southwest) on September 1, 2006. DCH awarded contracts to at least two CMOs within each of the six geographic regions. The Georgia Families program includes more than half of the State's Medicaid population and a majority of the State's PeachCare for KidsTM population. Enrollment is mandatory for the following Medicaid eligibility categories:
Low-Income Medicaid (LIM) program Transitional Medicaid Pregnant women and children in the Right from the Start Medicaid (RSM) program Newborns of Medicaid-covered women Refugees Women with breast and cervical cancer

SFY 2011 External Quality Review Annual Report State of Georgia

Page 2-1 GA2010-11_EQR_AnnRpt_F2_0811

BACKGROUND AND OVERVIEW
Georgia Families Care Management Organizations
DCH held contracts with three CMOs during SFY 2011. All three CMOs provide services to the State's Georgia Families members. In addition to providing medical and mental health, Medicaidcovered services to members, the CMOs also provide a range of enhanced services, including dental and vision services, disease management and education, and wellness/preventive programs.
AMERIGROUP Community Care
AMERIGROUP Community Care (AMERIGROUP) is a wholly-owned subsidiary of AMERIGROUP Corp., a multistate managed health care company serving people who receive health care benefits through publicly sponsored programs, including Medicaid and CHIP. AMERIGROUP serves members in the Atlanta, East, North, and Southeast regions.
Peach State Health Plan
Peach State Health Plan (Peach State) is part of the multistate national parent company, Centene Corp. Peach State serves members in the Atlanta, Central, and Southwest regions.
WellCare of Georgia, Inc.
WellCare of Georgia, Inc., (WellCare) is part of the national corporation, WellCare Health Plans, Inc., a multistate provider of only government-sponsored health products. WellCare serves members in all of the regions (i.e., Atlanta, Central, East, North, Southwest, and Southeast).
Georgia Families Quality Strategy
Federal regulations require that state Medicaid agencies develop and implement a written quality strategy for assessing and improving the quality of health care services offered to their members. The written strategy must describe the standards the state and its contracted plans must meet. The state must conduct periodic reviews to examine the scope and content of its quality strategy, evaluate its effectiveness, and update it as needed.
To comply with federal regulations, DCH submitted to CMS its initial Georgia Families Quality Strategic Plan in June 2007 for ensuring that the Department provided timely, accessible and quality services to members of Georgia Families. A quality strategic plan update was completed in January 2010.2-1 DCH published the updated plan on its Web site.

2-1 Georgia Department of Community Health. Georgia Families Quality Strategic Plan Update, January 2010.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 2-2 GA2010-11_EQR_AnnRpt_F2_0811

BACKGROUND AND OVERVIEW

The 2010 Update:
Described DCH's progressive and innovative approaches and activities focused on redesigning its processes, policies, procedures, operations and organization to perform more effectively as it moves into a phase of higher growth and development.
Affirmed DCH's continued commitment to be in compliance with the federal and State laws and regulations binding on State Medicaid managed care programs.
Described for each of DCH's four goals its performance-driven objectives designed to demonstrate success or to identify challenges in meeting intended outcomes related to providing quality, accessible and timely services. The four goals were described as: Promotion of an organization-wide commitment to quality of care and services. Improvement and enhancement of the quality of patient care provided through ongoing, objective, and systematic measurement, analysis and improvement of performance. Promotion of a system of health care delivery that provides coordinated and improved access to comprehensive health care and enhanced provider and client satisfaction. Promotion of acceptable standards of health care within the managed care program by monitoring internal/external processes for improvement opportunities.
DCH used recommendations in the EQR technical report as part of its process to assess the effectiveness of its strategic goals and objectives and to provide a road map for potential changes and new goals and strategies.
Georgia Families Quality Initiatives Driving Improvement
HSAG noted several DCH initiatives that supported the improvement of quality of care and services for Georgia Families members, as well as activities that supported CMO improvement efforts.
Auto-Assignment Program
DCH implemented an auto-assignment program beginning in 2010, which awarded CMOs with increased default enrollment based on a cost/quality indicator methodology. For CY 2010, DCH selected six clinical performance measures to determine the quality scores. This program encouraged the CMOs to achieve better quality outcomes for members. The performance measures selected were:
Well-Child Visits in the First 15 Months of Life--Six or More Visits Adults' Access to Preventive/Ambulatory Health Services--2044 years of age and
4564 years of age Use of Appropriate Medications for People With Asthma Childhood Immunization Status--Combination 2 Lead Screening in Children Comprehensive Diabetes Care--HbA1c Testing

SFY 2011 External Quality Review Annual Report State of Georgia

Page 2-3 GA2010-11_EQR_AnnRpt_F2_0811

BACKGROUND AND OVERVIEW
Encounter Data Validation
DCH contracted with HSAG to conduct an encounter data validation study during SFY 2009 to analyze the quality and timeliness of encounters submitted by the CMOs during the 2008 calendar year to DCH. Additionally, the study assessed the completeness and accuracy of Early and Periodic Screening, Diagnostic and Treatment (EPSDT) encounters. Final reports were provided to DCH in June 2010.
Findings from the analyses suggested that the encounter data submitted by the CMOs were relatively complete and accurate, however, results from the medical record review pointed to opportunities for improvement since not all services documented in the members' medical records were found in the electronic encounter data, and few medical records contained documentation of all required EPSDT services.
This project demonstrated DCH's oversight and monitoring of CMO-reported data, which revealed strengths as well as potential areas for improvement.
Quality Improvement and Performance Measure Transparency
DCH has increased the degree of transparency to the public with the release of its quality reporting on the DCH Web site. DCH has made efforts to improve the readability of technical reports to increase comprehension for members, plans, legislators, advocacy groups, and other stakeholders. This effort promotes informed decision making and opportunities for dialogue.
Collaborative Partnerships
Improving Birth Outcomes Work Group
DCH and the CMOs continue to partner with key stakeholder groups with a focus on reducing Georgia's low birth weight rate. DCH demonstrated a strong commitment in this area by adding the Rate of Infants With Low Birth Weight performance measure to the CMO required set in 2009, with the first set of rates reported in 2010.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 2-4 GA2010-11_EQR_AnnRpt_F2_0811

3. Review of Compliance With Standards

Review of Compliance With Standards
DCH contracted with HSAG to perform a review of the CMOs' compliance with standards, one of the three federally mandated activities. The purpose of the activity was to assess the CMOs' compliance with State and federal requirements related to enrollee rights and protections, access to services, structure and operations, measurement and improvement, and grievance system standards.
HSAG reviews one-third of the full set of standards each year over a three-year cycle. HSAG conducted on-site compliance reviews in October 2010. The CMOs submitted documentation that covered the review period of October 1, 2009, through September 30, 2010. HSAG provided detailed, final audit reports to the CMOs and DCH in February 2011. HSAG reviewed the CMOs' performance in the following areas:
Practice guidelines Quality assessment and performance improvement program and activities Information system's ability to collect and report data on performance across multiple indicators
Appendix A contains a detailed description of HSAG's methodology for conducting the review.

Findings

HSAG organized, aggregated and analyzed results from the compliance reviews to draw conclusions about the CMOs' performance in providing quality, accessible and timely health care services to Georgia Families members.
Table 3-1 displays the standards and compliance scores, which were the same for all three CMOs.

Standard #

Table 3-1Standards and Compliance Score

Standard Name

# of Elements*

# of Applicable Elements**

# Met

#

#

#

Partially Not

Not

Met Met Applicable

Total Compliance
Score

I Practice Guidelines

10

10

10 0

0

0

100%

II

Quality Assessment and Performance Improvement

29

29

29 0

0

0

100%

III Health Information Systems

8

8

8

0

0

0

100%

Totals

47

***Total Compliance Score Across the Three Standards

47

47 0

0

0

100%

* Total # of Elements: The total number of elements in each standard.

** Total # of Applicable Elements: The total number of elements within each standard minus any elements that received a designation of NA.

*** Total Compliance Score: The overall percentages were calculated by adding the number of elements that received a score of Met to the weighted (multiplied by 0.50) number that received a score of Partially Met, then dividing this total by the total number of applicable elements.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 3-1 GA2010-11_EQR_AnnRpt_F2_0811

REVIEW OF COMPLIANCE WITH STANDARDS

For standards assessed during the review period, HSAG found that performance for all three CMOs on each of the 47 applicable requirements across the three standards was sufficient to result in a Met score.
The CMOs had ample documentation describing their processes, practices, action plans and performance results/outcomes related to each review requirement. During the on-site interviews, CMO staff members' responses to HSAG's questions, including their descriptions and examples of their processes and practices for ensuring compliance with the requirements, were consistent with the documentation.
The statewide percentage-of-compliance score for each of the three standards and statewide performance across the standards was 100 percent, reflecting commendable CMO performance.
The following overall strengths were noted by HSAG across the three CMOs for each of the standards:
Standard I: Practice Guidelines
Developing and/or adopting existing clinical and preventive guidelines, including those focused on the DCH-required conditions (i.e., asthma, diabetes, and chronic kidney disease), that were selected based on the health needs of members and the opportunities identified through the quality program for improving performance in meeting those needs.
Involving medical and clinical leadership at both the corporate and local CMO level, as well as contracted providers, in the decisions to adopt and, as applicable, revise the guidelines.
Making the guidelines easily accessible to providers and working collaboratively with individual providers or groups of providers to improve their adherence to the guidelines in their practices.
Using the CMOs' disease management/health education programs and newsletters to provide members with easy-to-understand information about prevention and treatment related to the conditions addressed by the guidelines and the importance of members' participation in their care.
Establishing thresholds for provider performance consistent with the guidelines and providing incentives and/or rewards for strong performance as demonstrated by claims payment data and as documented in the medical record.
Standard II: Quality Assessment and Performance Improvement
Having multidisciplinary medical and quality committees at the corporate and local Georgia CMO level to oversee and direct the CMOs' quality programs and performance results.
Ensuring that the quality programs and activities are conducted by qualified and experienced senior leadership and are designed and carried out consistent with best practices and the most current research in the area of quality assurance and improvement.
Designing the quality programs to be broad in scope and focused on identifying opportunities for improvement, selecting/designing interventions, and measuring improvement and outcomes.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 3-2 GA2010-11_EQR_AnnRpt_F2_0811

REVIEW OF COMPLIANCE WITH STANDARDS

The programs included conducting ad hoc and focused studies; provider profiling; patient safety policies, procedures, and plans; and mechanisms for determining the severity of adverse incidents and the level of review required for each. Including input from multiple sources in developing the CMO quality programs and performance improvement processes (e.g., the CMO executive and management staff, providers, and members and their families/guardians). Identifying and developing targeted resources and interventions to appropriately manage the care of at-risk members. Including key stakeholders (i.e., providers, members, and representatives from community resources) in reviewing performance data, provider practice patterns, and the quality and appropriateness of member care and member outcomes, as well as selecting targeted performance improvement interventions. Regularly measuring, reporting and evaluating performance across a broad range of quality measures (including process and outcome measures), including DCH-required measures.
Standard III: Health Information Systems
Having sophisticated electronic health information systems supported by corporate and CMO resources.
Having data systems that support the capture of key information, such as: Member and provider demographics. Member health care needs/conditions and the services furnished to them. CMO performance results across multiple DCH-required and CMO-selected indicators including utilization patterns, member grievances and appeals, provider profiling, and member and provider satisfaction.
Having the capability to produce regular reports and alerts to staff members related to compliance with DCH contract requirements and upcoming dates for such things as revisions to written policies/procedures and review of practice guidelines.

CMO Comparison Key Findings
All the CMOs were 100 percent compliant with the standards evaluated during the review period. Based on the individual CMO reviews, HSAG highlights the following strengths and recommendations for each of the CMOs.

AMERIGROUP
Strengths Informing members about several diseases/conditions and how they could manage the conditions through well-written and informative materials included in the CMO's AMERITIPS newsletters. The newsletters included the telephone numbers for the CMO's Member Services Department and the 24-hour nurse help line if members had questions. When conducting

SFY 2011 External Quality Review Annual Report State of Georgia

Page 3-3 GA2010-11_EQR_AnnRpt_F2_0811

REVIEW OF COMPLIANCE WITH STANDARDS
provider on-site visits, network representatives and other staff ensured that providers had copies of the AMERITIPS newsletters for members in their waiting rooms. The CMO had a documented process to guide its customer service representatives' responses and the steps representatives should take if a member requests a copy of the guidelines (both utilization review and practice guidelines). Having in its Strategic Outcomes Analysis and Reporting (SOAR) program two modules that facilitated the CMO's ability to collect, analyze, and report provider performance data. The methodologies AMERIGROUP used to analyze primary care provider (PCP) claims facilitated its ability to examine all claims by members and their network utilization patterns, analyze patterns of underutilization, and evaluate care compared with its nationally accepted guidelines. Using the data it had collected and analyzed, AMERIGROUP was able to identify both its highand low-performing providers and provide feedback to them about their individual performance. Having initiatives to establish the PCP as the member's medical home and to divert members back to the PCP to prevent inappropriate emergency room (ER) visits or hospitalizations. The CMO instituted a provider report based on members' use of the ER or hospital for the following ambulatory care-sensitive conditions: upper respiratory infections, acute otitis media, nausea and vomiting, cellulitis, and gastroenteritis.
Recommendations Make practice guidelines available to providers and members and inform members how they can request a copy of the guidelines. Include in the written practice guidelines the methodology the CMO uses to evaluate its providers' performance related to the guidelines. Strengthen processes and documentation to demonstrate that CMO methods and activities for ensuring that all member informational materials, utilization review criteria/guidelines, and covered services are consistent with the information in its clinical practice guidelines.
Peach State
Strengths Using its Compliance 360 software program to ensure that its practice guidelines and associated policies and procedures were reviewed at least annually. Documenting its evidence-based best practices and achieving proven results in the Connections Plus home visit program, which URAC (formerly the Utilization Review Accreditation Commission) recognized with its best practice award. Having data available for quality and utilization management reporting through its executive dashboard, and a data warehouse (EDW) for integrating and reporting service data. The dashboard, supported by Centene at the corporate level, allowed drill-down at the CMO level and supported reporting at the organization's functional and departmental levels. As an example, the CMO used and trended data over time to identify the need for a discharge planning intervention, which resulted in positive outcomes in its readmission rates following placement of a Peach State case manager in the hospital setting.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 3-4 GA2010-11_EQR_AnnRpt_F2_0811

REVIEW OF COMPLIANCE WITH STANDARDS
Recommendations:
Strengthen documentation of its consistency review(s) related to practice guidelines. This step would add greater detail than the information contained in the signed attestation of having reviewed member materials, utilization review criteria, and other documents compared with the applicable guidelines.
WellCare of Georgia, Inc.
Strengths Offering a pay-for-performance program to providers for strong performance in working with members to improve their knowledge about the importance of and ability to access needed services as some of the ways WellCare was working to improve practices and service delivery consistent with the guidelines. Using an automated quality improvement work plan to facilitate collaboration across all departments, WellCare was able to track performance results for each activity or project undertaken throughout the year. The CMO's work plan provided a centralized document in which activities were aligned with contractual, accreditation, and/or regulatory requirements, and the work plan identified the measurements to assess progress toward the associated goals. Having the ability for utilization management staff members, through the CMO's information system, to flag cases for scheduled review, perform case management duties, refer cases for medical director review, and assign authorization numbers and lengths of stay. The system also generated reports of UM activities, including adverse determination tracking, authorizations by type, length of stay vs. average length of stay, bed day utilization, and pended cases.
Recommendations
Monitor the impact of a policy change on the timely dissemination of current information on practice guidelines to providers. The policy changed from requiring practice guideline revisions "as needed, but no less than annually," to "as needed, but not less than every two years."
Conclusions
All the CMOs achieved compliance with the review standards for practice guidelines, quality assessment and performance improvement, and health information systems. These standards provided a foundation from which the CMOs delivered care and services. While CMO documentation met the intent of the standards, actual health outcomes and performance measure rates for the Georgia Families program as a whole performed below many of the national Medicaid percentiles. This suggests that the CMOs have an opportunity to more closely evaluate providers' practicing patterns against adopted clinical guidelines. The CMOs may consider, for example, whether their medical record review audit includes the elements contained in the performance measure areas such as blood sugar and cholesterol screening for members with diabetes.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 3-5 GA2010-11_EQR_AnnRpt_F2_0811

4. Performance Measures
DCH selected performance measures to evaluate the quality of care delivered to Georgia Families members by the CMOs on an annual basis. The selected performance measures reflect the State's priorities and areas of concern for Georgia Families members and include a DCH-developed measure and HEDIS 4-1 and Agency for Healthcare Research and Quality (AHRQ) measures. The CMOs calculate and report data consistent with the most current reporting-year specifications.
In addition to collecting performance measure data, validating performance measures is one of the three mandatory EQR activities described at 42 CFR 438.358(b)(2). The requirement allows states, agents that are not a managed care organization, or an EQRO to conduct the performance measure validation. DCH contracted with HSAG to conduct the functions associated with validating performance measures.
Performance results can be calculated and reported to the state by the managed care organization or the state can calculate the managed care organization's performance on the measures for the preceding 12 months. DCH requires its plans to calculate their own performance measures rates for validation.
All Georgia Families CMOs underwent an independent HEDIS Compliance AuditTM 4-2 by a licensed organization to ensure that the CMOs followed specifications to produce valid and reliable HEDIS measure results. HSAG received the final, audited CMO rates and ensured that the HEDIS compliance protocol met CMS' requirements for validating performance measures. Additionally, HSAG validated performance measures that were not covered under the scope of the HEDIS Compliance Audit, which consisted of measures developed by AHRQ and one DCH-developed measure. Appendix C contains a more detailed description of the method for conducting the review.

Performance Measure Requirements and Targets
DCH requires that CMOs collect and report performance measure rates, allowing for a standardized method to objectively evaluate plans' delivery of services. DCH's requirement for the CMOs to report performance measure data annually supports the overall Georgia Families strategic plan objective: improvement and enhancement of the quality of patient care provided through ongoing, objective, and systematic measurement, analysis and improvement of performance.
DCH adopted standardized and nationally accepted performance measures beginning in 2008 to better allow for comparability among the CMOs as well as against other state and national benchmarks.
DCH required plans to report rates for 32 measures consisting of clinical quality measures, utilization measures and health plan descriptive information.
DCH established performance targets for six of the required performance measures. The six measures represent the original standardized performance measures selected by DCH in 2008. DCH's minimum performance targets were typically the National Committee for Quality Assurance's (NCQA's) 50th national Medicaid percentile. In some cases, if CMOs had already

4-1 HEDIS refers to the Healthcare Effectiveness Data and Information Set and is a registered trademark of the National
Committee for Quality Assurance (NCQA). 4-2 HEDIS Compliance AuditTM is a trademark of NCQA.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-1 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES
achieved or were close to achieving the 50th percentile, higher targets were selected. CMO contracts were amended and approved by DCH in July 2010 to allow the CMOs to develop performance incentives for their contracted providers to drive achievement of the targets. Additionally, the DCH contracts with the CMOs provided DCH the ability to impose financial penalties for the CMOs that failed to achieve the established performance targets.
The CMOs submitted performance measure data that generally reflected the period of January 1, 2009, through December 31, 2009. HSAG provided final performance measure validation reports to the CMOs and DCH in July 2010.
Findings
Performance Measure Validation Key Findings
All three DCH-contracted CMOs underwent performance measure validation for rates calculated using CY 2009 measurement period data.
Strengths
DCH opted to use the CMO-reported HEDIS rates that were audited by an independent licensed organization and reviewed by HSAG to ensure the validation process was consistent with CMS requirements for performance measure validation. DCH's decision to use the audited CMO-reported rates, instead of unaudited rates calculated by the State's vendor using administrative data only, provided a more accurate reflection of the CMOs' performance since the CMOs used the hybrid method to collect additional data from the medical record for some performance measures in conjunction with administrative, claims/encounter data. Using CMO reported rates allows DCH and the CMOs to better prioritize areas for improvement.
Challenges
DCH's 2010 reporting requirements included AHRQ measures for the first time. The CMOs experienced challenges with these measures as the technical specifications did not address enrollment criteria and anchor dates, which can impact how the CMOs interpret whether to include or exclude members. HSAG identified this as an issue during source code review, finding that the CMOs were not interpreting criteria consistently. To resolve this issue, HSAG provided DCH and the CMOs with criteria on enrollment and anchor dates for each of the AHRQ measures to ensure that the CMOs were reporting rates consistently to allow for comparison.
Performance Measure Result Findings
Using the validated performance measure rates, HSAG organized, aggregated, and analyzed the data to draw conclusions about CMO performance in providing accessible, timely and quality care and services to Georgia Families members.
Table 4-1 through Table 4-6 present the following data:

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-2 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES
CY 2008 and 2009 statewide CMO weighted averages for clinical measures from the CMOs' reported and audited data
CY 2009 Georgia Families rates calculated using DCH MMIS administrative data (validated by HSAG)
CY 2009 State of Georgia fee-for-service (FFS) Medicaid data (validated by HSAG) CY 2009 performance targets for the six DCH-selected performance measures
Similar to groupings used in the Georgia Families Quality Strategy, HSAG grouped clinical performance measures into the areas of access, children's health, women's health, diabetes care, asthma and behavioral health to assess the overall care provided by the CMOs. HSAG used the CY 2009 CMO weighted average rates when making the comparisons to the prior-year data, the FFS data and the CMOs' performance targets. The CMO-reported data may reflect a more accurate assessment of care provided since the CMOs have the ability to conduct medical record review in addition to using administrative data for hybrid measures.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-3 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Table 4-1--2008/2009 Performance Measure Results--Access

CY 2008 CMO Rate1

CY 2009 CMO Rate2

CY 2009
Georgia
Families Rate3

Children's and Adolescents' Access to Primary Care Providers

CY 2009 FFS Rate4

CY 2009 Performance
Target5

Ages 1224 Months

- -

96.4%

93.7%

93.6%

Ages 25 Months6 Years

- -

91.3%

86.0%

84.5%

Ages 711 Years

- -

91.3%

86.9%

84.2%

Ages 1219 Years

- -

88.3%

Adults' Access to Preventive/Ambulatory Health Services

Ages 2044 Years

79.2%

84.7%

83.0% 85.2%

77.2% 75.1%

84.8%

Ages 4564 Years

- -

Oral Health (Annual Dental Visit Rate)

85.3%

86.0%

75.5%

Ages 23 Years

- -

38.9%

31.4%

29.9%

Ages 46 Years

- -

72.4%

62.3%

56.8%

Ages 710 Years

- -

75.2%

66.4%

58.2%

Ages 1114 Years

- -

67.5%

60.1%

54.3%

Ages 1518 Years

- -

57.2%

52.4%

48.3%

Ages 1921 Years

- -

37.3%

34.8%

33.0%

All Members (Ages 221 Years)

- -

64.1%

55.9%

49.5%

1 CY 2008 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement

year, which is January 1, 2008, through December 31, 2008.

2 CY 2009 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the

measurement year, which is January 1, 2009, through December 31, 2009. Statistically significant changes between 2008

and 2009 rates are displayed where applicable.

3 CY 2009 Georgia Families rates were calculated by Thomson Reuters using CMO-submitted administrative data only

pulled from the GA MMIS.

4 CY 2009 FFS rates reflect fee-for-service claims data submitted to DCH for the measurement year, which is January 1,

2009, through December 31, 2009.

5 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed

when no DCH CY 2009 performance target was established.

Indicates a statistically significant increase between the 2008 and 2009 weighted average rates.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-4 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Table 4-2--2008/2009 Performance Measure Results--Children's Health

CY 2008 CMO Rate1

CY 2009 CMO Rate2

CY 2009 Georgia Families
Rate3

CY 2009 FFS Rate4

CY 2009 Performance Target5

Well-Child/Well-Care Visits

First 15 Months of Life: Six or More Visits
Third, Fourth, Fifth, and Sixth Years of Life
Adolescent Well Care Immunization and Screening

60.1% 55.5%* 36.0% - - 61.4%* 54.0% - - 35.9%*

29.8% 47.7% 21.6%

65.4%

Childhood Immunization Status Combination 2

76.6%

Lead Screening in Children

71.6%

Weight Assessment and Counseling

Body Mass Index (BMI) Percentile

- -

Counseling for Nutrition

- -

Counseling for Physical Activity

- -

Upper Respiratory Infection

75.2%* 66.0%*
30.2%* 40.4%* 35.1%*

35.5% 45.4%
NR NR NR

27.9% 41.7%
NR NR NR

72.0% 65.9%

Appropriate Treatment for
Children With Upper Respiratory - - 78.4% 77.9% 77.3%
Infection
1 CY 2008 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year, which is January 1, 2008, through December 31, 2008. 2 CY 2009 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year, which is January 1, 2009, through December 31, 2009. Statistically significant changes between 2008 and 2009 rates are displayed where applicable. 3 CY 2009 Georgia Families rates were calculated by Thomson Reuters using CMO-submitted administrative data only pulled from the GA MMIS. 4 CY 2009 FFS rates reflect fee-for-service claims data submitted to DCH for the measurement year, which is January 1, 2009, through December 31, 2009. 5 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed when no DCH CY 2009 performance target was established.
*Rates are derived from the hybrid methodology in which both administrative data and medical record review data are used.
NR Not Reported. The measure should not be reported because the results were not accurate using the data available.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-5 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Table 4-3--2008/2009 Performance Measure Results--Women's Health

Prevention and Screening

CY 2008 CMO Rate1

CY 2009 CMO Rate2

CY 2009 Georgia Families
Rate3

CY 2009 FFS Rate4

CY 2009 Performance Target5

Cervical Cancer Screening

- -

66.7%* 66.7% 30.0%

Breast Cancer Screening Prenatal Care and Birth Outcomes

- -

51.1% 49.8% 40.3%

Timeliness of Prenatal Care

- -

82.2%* 53.5% 44.2%

Postpartum Care

- -

66.6%* 40.1% 25.7%

Cesarean Delivery Rates (AHRQ measure)

- -

31.9% 30.1% 35.1%

Rate of Infants With Low Birth Weight (AHRQ measure)

- -

Frequency of Ongoing Prenatal Care

7.5%

6.5% 14.1%

< 21 Percent

- -

15.5%* 66.0% 70.1%

21-40 Percent

- -

5.7%* 23.8% 21.1%

41-60 Percent

- -

6.7%* 5.5%

5.0%

61-80 Percent

- -

12.3%* 2.7%

2.3%

81+ Percent

- -

59.8%* 2.0%

1.5%

1 CY 2008 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2008, through December 31, 2008. 2 CY 2009 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2009, through December 31, 2009. Statistically significant changes between 2008 and 2009 rates are displayed

where applicable. 3 CY 2009 Georgia Families rates were calculated by Thomson Reuters using CMO-submitted administrative data only pulled from

the GA MMIS. 4 CY 2009 FFS rates reflect fee-for-service claims data submitted to DCH for the measurement year, which is January 1, 2009, through

December 31, 2009. 5 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed when no

DCH CY 2009 performance target was established.

*Rates are derived from the hybrid methodology in which both administrative data and medical record review data are used.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-6 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Table 4-4--2008/2009 Performance Measure Results--Diabetes

Comprehensive Diabetes Care

CY 2008 CMO Rate1

CY 2009 CMO Rate2

CY 2009 Georgia Families
Rate3

CY 2009 FFS Rate4

CY 2009 Performance Target5

Hemoglobin A1c (HbA1c) Testing
HbA1c Poor Control (>9.0) A lower rate indicates better performance

70.4%* 76.6%*

- -

59.3%*

70.5% NR

43.0% NR

79.0%

HbA1c Control (<8.0)

- -

34.1%*

NR

NR

HbA1c Control (<7.0)

- -

29.5%*^

NR

NR

Eye Exam (retinal) Performed

- -

40.9%* 35.8% 32.5%

LDL-C Screening

- -

66.7%* 61.0% 36.5%

LCL-C Control (<100 mg/dL)

- -

21.8%*

NR

NR

Medical Attention for Nephropathy

- -

68.7%* 62.0% 48.4%

Blood Pressure Control (<130/80 mm/Hg)

- -

24.2%*

NR

NR

Blood Pressure Control (<140/90 mm/Hg)

- -

49.9%*

NR

NR

Diabetes Admission Rate

Diabetes Short-Term Complications Admission Rate (per 100,000)

- -

26.4

32.0

90.4

1 CY 2008 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,
which is January 1, 2008, through December 31, 2008. 2 CY 2009 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2009, through December 31, 2009. Statistically significant changes between 2008 and 2009 rates are
displayed where applicable. 3 CY 2009 Georgia Families rates were calculated by Thomson Reuters using CMO-submitted administrative data only pulled from

the GA MMIS. 4 CY 2009 FFS rates reflect fee-for-service claims data submitted to DCH for the measurement year, which is January 1, 2009,

through December 31, 2009. 5 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed when
no DCH CY 2009 performance target was established.

* Rates are derived from the hybrid methodology in which both administrative data and medical record review data are used.

^ The CY 2009 CMO rate for this measure was calculated from two CMOs' reported and audited data since one CMO did not report

a rate for this measure.
Indicates a statistically significant increase between the 2008 and 2009 weighted average rates.

NR Not Reported. The measure should not be reported because the results were not accurate using the data available.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-7 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Asthma

Table 4-5--2008/2009 Performance Measure Results--Asthma

CY 2008 CMO Rate1

CY 2009 CMO Rate2

CY 2009 Georgia Families
Rate3

CY 2009 FFS Rate4

CY 2009 Performance Target5

Use of Appropriate Medications for People With Asthma

91.1% 90.5% 90.0% 90.5%

93.0%

Members With ER/Urgent Care Office Visits for Asthma in the Past Six Months

- -

1.5%

1.4%

1.1%

Asthma Admission Rate (per 100,000)

- -

104.4

151.5

385.3

1 CY 2008 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2008, through December 31, 2008. 2 CY 2009 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2009, through December 31, 2009. Statistically significant changes between 2008 and 2009 rates are displayed

where applicable. 3 CY 2009 Georgia Families rates were calculated by Thomson Reuters using CMO-submitted administrative data only pulled from

the GA MMIS. 4 CY 2009 FFS rates reflect fee-for-service claims data submitted to DCH for the measurement year, which is January 1, 2009, through

December 31, 2009. 5 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed when no

DCH CY 2009 performance target was established.

Table 4-6--2008/2009 Performance Measure Results--Behavioral Health

CY 2008 CMO Rate1

CY 2009 CMO Rate2

Follow-up Care for Children Prescribed ADHD Medication

CY 2009 Georgia Families
Rate3

CY 2009 FFS Rate4

CY 2009 Performance Target5

Initiation Phase

- -

Continuation and Maintenance Phase

- -

Follow-up After Hospitalization for Mental Illness

43.4% 53.1%

45.8% 56.3%

48.7% 61.7%

Follow-up Within 7 Days

- -

72.7% 17.5% 9.2%

Follow-up Within 30 Days

- -

84.2% 32.2% 20.5%

1 CY 2008 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2008, through December 31, 2008. 2 CY 2009 CMO rates reflect the weighted averages from the three (3) CMOs' reported and audited data for the measurement year,

which is January 1, 2009, through December 31, 2009. Statistically significant changes between 2008 and 2009 rates are displayed

where applicable. 3 CY 2009 Georgia Families rates were calculated by Thomson Reuters using CMO-submitted administrative data only pulled from

the GA MMIS. 4 CY 2009 FFS rates reflect fee-for-service claims data submitted to DCH for the measurement year, which is January 1, 2009, through

December 31, 2009. 5 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed when no

DCH CY 2009 performance target was established.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-8 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES
CMO Weighted Average Performance Measure Result Findings
HSAG generated the CMO weighted average performance measure rates for the six measures that DCH selected for performance targets for CY 2009. HSAG compared the CMO results against the CY 2009 performance targets, which were set by DCH using national Medicaid benchmarks. The remaining CMO CY 2009 performance measure results served as baseline results for DCH and as such national Medicaid benchmarks were not set for them by DCH. Therefore, HSAG did not assess CMO performance against performance targets for these measures.
Two of the six overall CY 2009 CMO weighted average performance measure rates, Childhood Immunization Status--Combination 2 and Lead Screening in Children, met the CY 2009 performance targets. Four of the six CY 2009 CMO weighted average performance measures rates for Adults' Access to Preventive/Ambulatory Health Services for members 2044 years of age; Well-Child Visits in the First 15 Months of Life--Six or More Visits; Comprehensive Diabetes Care--Hemoglobin A1c (HbA1c) Testing; and Use of Appropriate Medications for People With Asthma
did not meet the CY 2009 performance targets.
Two statistically significant rate increases were shown between CY 2008 and CY 2009 rates for the CMO weighted average for Adults' Access to Preventive/Ambulatory Health Services for members 2044 years of age, and for its Comprehensive Diabetes Care--Hemoglobin A1c (HbA1c) Testing measures. CMO weighted average rates had no statistically significant declines in performance between years.
CMO Comparison Key Findings
HSAG assessed CMO-specific rates for all CY 2009 required performance measures in the areas of access to care, children's health, women's health, diabetes care, asthma care, and behavioral health.
Access to Care
Table 4-7 displays CMO plan-specific results for access measures.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-9 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Table 4-7--Access Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

CY 2009 Rate1

CY 2009 Rate

CY 2009 Rate

CY 2009 Performance
Target2

Children's and Adolescents' Access to Primary Care Providers

Ages 1224 Months

96.3%

95.8%

96.7%

Ages 25 Months6 Years

91.7%

90.6%

91.4%

Ages 711 Years

92.9%

90.5%

Ages 1219 Years

89.7%

87.1%

Adults' Access to Preventive/Ambulatory Health Services

91.2% 88.3%

Ages 2044 Years

85.5%

84.3%

84.7%

84.8%

Ages 4564 Years Annual Dental Visit

87.4%

82.5%

86.1%

Ages 23 Years Ages 46 Years Ages 710 Years Ages 1114 Years Ages 1518 Years Ages 1921 Years

42.7% 74.9% 77.3% 69.6% 59.4% 40.3%

33.8% 69.2% 72.1% 63.9% 53.1% 35.1%

40.4% 73.2% 76.1% 68.7% 58.6% 37.6%

Total

66.7%

60.2%

65.2%

1 CY 2009 rates reflect CMO-reported and audited data for the measurement year, which is January 1, 2009, through

December 31, 2009. 2 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are

displayed when no DCH CY 2009 performance target was established.

AMERIGROUP performed best on measures in the area of access, followed by WellCare, then Peach State. AMERIGROUP was the only CMO to achieve the 2009 performance target of 84.8 percent for the Adults' Access to Preventive/Ambulatory Health Services 2044 years measure, with a rate of 85.5 percent. WellCare was just under the goal, with a rate of 84.7 percent, followed by Peach State, with a rate of 84.3. Findings in the area of access suggested the CMOs have adequate provider networks for Georgia Families members to access preventive care and dental visits.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-10 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Children's Health

Table 4-8--Children's Health Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

CY 2009 Rate1 CY 2009 Rate CY 2009 Rate

2009 Performance
Target2

Well-Child/Well-Care Visits in the First 15 Months of Life

First 15 Months of Life: Six or More Visits

55.0%

52.3%

57.4%

65.4%

Third, Fourth, Fifth, and Sixth Years of Life

64.1%

63.8%

58.9%

Adolescent Well Care Immunization and Screening

40.5%

37.2%

32.9%

Childhood Immunization Status-- Combo 2

72.0%

67.6%

81%

72.0%

Lead Screening in Children

67.8%

62.3%

67.4%

65.9%

Weight Assessment and Counseling for Nutrition and Physical Activity for Children/Adolescents

BMI Percentile (Total)

13.7%

32.1%

36.5%

Counseling for Nutrition (Total)

40.7%

36.7%

42.3%

Counseling for Physical Activity (Total) Upper Respiratory Infection (URI)

35.6%

28.2%

38.7%

Appropriate Treatment for Children With URI

78.7%

79.1%

77.8%

1 CY 2009 rates reflect CMO-reported and audited data for the measurement year, which is January 1, 2009, through

December 31, 2009.

2 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed

when no DCH CY 2009 performance target was established.

All the CMOs had challenges with the Well-Child Visits in the First 15 Months of Life--Six or More Visits measure, with no CMO achieving the CY 2009 performance target. WellCare performed best when compared with AMERIGROUP and Peach State for this measure. Findings from the independent encounter data validation study conducted during CY 2009-2010 indicated that while the encounter data submitted by the CMOs were relatively complete and accurate, few medical records contained documentation of all required EPSDT services. Since access to care measure rates showed that members had appropriate access to care, there were missed opportunities for providers to address these aspects of care. HSAG's methodology for conducting the encounter data validation study is included in Appendix C.

In contrast to the CMOs' performance on the well-child visits measure, two of the three CMOs, AMERIGROUP and WellCare, achieved the CY 2009 performance target of 72.0 percent for childhood immunizations. Additionally, AMERIGROUP and WellCare achieved the CY 2009 performance target for Lead Screening in Children.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-11 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

WellCare performed the best on the Weight Assessment and Counseling for Nutrition and Physical Activity for Children and Adolescents measure compared with AMERIGROUP and Peach State.
Women's Health

Table 4-9--Women's Health Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

CY 2009 Rate1

CY 2009 Rate

CY 2009 Rate

2009 Performance
Target2

Prevention and Screening

Cervical Cancer Screening

70.6%

65.5%

65.9%

Breast Cancer Screening Prenatal Care and Birth Outcomes

54.2%

48.7%

51.3%

Timeliness of Prenatal Care

75.1%

86.9%

82.2%

Postpartum Care

57.4%

67.6%

69.6%

Cesarean Delivery Rate

33.3%

33.2%

30.5%

Rate of Infants With Low Birth Weight

7.7%

8.2%

6.9%

Frequency of Ongoing Prenatal Care

< 21 Percent

27.0%

10.7%

13.9%

21-40 Percent

9.8%

7.3%

3.2%

41-60 Percent

5.3%

10.2%

5.1%

61-80 Percent

10.1%

18.5%

9.5%

81+ Percent

47.9%

53.3%

68.4%

1 CY 2009 rates reflect CMO-reported and audited data for the measurement year, which is January 1, 2009, through

December 31, 2009. 2 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed

when no DCH CY 2009 performance target was established.

AMERIGROUP outperformed Peach State and WellCare on the breast and cervical cancer screening measures. Peach State had the highest CY 2009 CMO rate for timeliness of prenatal care while WellCare had the highest CY 2009 CMO postpartum care rate.
All of the CMOs had Cesarean delivery rates above the most current Medicaid benchmark available (2004) of 27.4 percent. WellCare was the only CMO with a result for Rate of Infants With Low Birth Weight that was equivalent to the Medicaid 2004 benchmark of 6.9, while AMERIGROUP and Peach State had higher rates of low-birth-weight infants. WellCare outperformed AMERIGROUP and Peach State for frequency of prenatal care--81 percent or more of the expected prenatal visits by women.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-12 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Diabetes Care

Table 4-10--Physical Health Conditions: Diabetes Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

CY 2009 Rate1

CY 2009 Rate

CY 2009 Rate

CY 2009 Performance
Target2

Comprehensive Diabetes Care

HbA1c Testing
HbA1c Poor Control A lower rate indicates better performance
HbA1c Good Control <8.0

73.7% 60.8% 31.5%

74.7% 67.2% 27.7%

78.7% 54.4% 38.7%

79.0%

HbA1c Good Control <7.0

22.9%

NR

32.0%

Eye Exam

43.4%

46.0%

37.2%

LDL-C Screening

62.8%

65%

69.2%

LDL-C Level

20.8%

19.7%

23.4%

Medical Attention for Nephropathy

67.8%

65.5%

70.8%

Blood Pressure Control < 130/80

25.3%

21.4%

25.4%

Blood Pressure Control < 140/90 Diabetes Admission Rate

47.9%

44.8%

53.5%

Diabetes Short-term

Complications Admission

14.0

34.6

28.6

Rate (per 100,000)

1 CY 2009 rates reflect CMO-reported and audited data for the measurement year, which is January 1, 2009, through December 31,

2009. 2 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed when

no DCH CY 2009 performance target was established.

NR Not Reported. The measure should not be reported because the results were not accurate using the data available.

All three CMOs have an opportunity to improve on the diabetes care measures. All the CMO CY 2009 rates were below the CY 2009 performance target of 79.0 percent for HbA1c testing. Overall, WellCare performed best compared with AMERIGROUP and Peach State.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-13 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Asthma Care

Table 4-11--Physical Health Conditions: Asthma Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

CY 2009 Rate1

CY 2009 Rate

CY 2009 Rate

CY 2009 Performance
Target2

Use of Appropriate Medications for People With Asthma

91.3%

90.8%

89.9%

93.0%

Percent of Members Who Have Had a Visit to an Emergency Department/ Urgent Care Office for Asthma in the Past Six Months

1.6%

1.4%

1.4%

Asthma Admission Rate (per 100,000)

68.4

136.9

104.7

1 CY 2009 rates reflect CMO-reported and audited data for the measurement year, which is January 1, 2009, through

December 31, 2009. 2 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are displayed

when no DCH CY 2009 performance target was established.

AMERIGROUP performed best on the Use of Appropriate Medications for People With Asthma measure when compared with Peach State and WellCare; however, no CMO met the CY 2009 performance target of 93.0 percent.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-14 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

Behavioral Health

Table 4-12--Behavioral Health Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

CY 2009 Rate1 CY 2009 Rate CY 2009 Rate

CY 2009 Performance
Target2

Follow-Up Care for Children Prescribed ADHD Medication

Initiation Phase

37.6%

47.0%

43.3%

Continuation and Maintenance Phase

50.7%

Follow-Up After Hospitalization for Mental Illness

57.3%

51.4%

30-Day Follow-Up

71.6%

74.9%

88.2%

7-Day Follow-Up

48.6%

59.6%

79.6%

1 CY 2009 rates reflect CMO-reported and audited data for the measurement year, which is January 1, 2009, through

December 31, 2009. 2 CY 2009 performance targets reflect the DCH-established CMO performance targets for 2009. Shaded boxes are

displayed when no DCH CY 2009 performance target was established.

WellCare outperformed Peach State and AMERIGROUP on the Follow-up After Hospitalization for Mental Illness measures while Peach State had the highest rates for the Follow-up Care for Children Prescribed ADHD Medication measures.
Fee-For-Service Comparisons

In addition to comparing CMO weighted average performance to national benchmarks and targets, HSAG compared the CMOs' performance to the Medicaid FFS population. While HSAG assessed the rates of CMO weighted averages and FFS, comparisons should be made with caution. CMOreported data may reflect a more accurate assessment of care provided since the CMOs had the ability to conduct medical record review in addition to using administrative data for hybrid measures. FFS rates were calculated using only claims data, which may not be as accurate as the CMO-reported data that includes the medical record reviews.
Performance measure results showed that the CMOs had better performance than Medicaid FFS when comparing the overall CMO weighted averages to FFS data on nearly all measures except Use of Appropriate Medications for People With Asthma, for which both programs had the same rate, and on Follow-Up Care for Children Prescribed ADHD Medication. This suggests that CMO members received better care than FFS members.
Utilization Measures
In addition to clinical performance measures, DCH required the CMOs to report utilization rates for inpatient utilization, mental health utilization, antibiotic utilization and outpatient drug utilization. Utilization information can be helpful to the CMOs in reviewing patterns of suspected under- and overutilization of services, however, this data should be used with caution as high and low rates do not necessarily indicate better or worse performance. Appendix B contains a table of utilization

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-15 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES

measures by CMO and an overall CMO weighted average rate for each measure. The CMOs should use these comparisons to further analyze utilization patterns for potential problem areas related to provider practice patterns, geographic accessibility, etc. Some utilization rates, such as maternity and inpatient discharges, do not indicate a need to evaluate performance; rather they simply provide the CMOs and DCH with information on the plans' rates and allow them to be compared to national rates.
Health Plan Demographics
The CMOs reported health plan demographic information for Race/Ethnicity of Membership, Language Diversity of Membership and Weeks of Pregnancy at Time of Enrollment. Appendix B contains the CMO rates for these measures.
The data showed that 47.5 percent of Georgia Medicaid managed care members were Black, 43.2 percent were White, 1.9 percent were Asian, 1.9 percent were Hispanic or Latino and 5.5 percent were categorized as unknown. Ethnicity data were not captured as 93.4 percent showed an unknown ethnicity. Eighty-nine percent of Georgia Families members spoke English and 6.8 percent spoke Spanish or Spanish Creole. While the CMOs captured this information, it is only meaningful if these data are compared to Georgia Census demographics.
The data also showed that 56.9 percent of Georgia Medicaid managed care members who were pregnant were enrolled in the program between 13 and 27 weeks of pregnancy. A contributing factor to this rate is the fact that Georgia Medicaid-eligible managed care members are first enrolled into FFS Medicaid and then must select a CMO. This selection process may take up to sixty (60) days, thus giving the appearance in this measure that some pregnant members are without health care coverage until their second trimester.
Health plan demographic information may be useful to DCH and the CMOs when considering targeted interventions to ensure that strategies are appropriate for the targeted populations and to ensure that culturally and linguistically appropriate services are available to members.

Conclusions
HSAG found that all the CMOs were compliant with the required information system standards to report valid performance measure rates. Overall, the CMOs demonstrated the ability to process, receive, and enter medical and service data efficiently, accurately, timely and completely. The CMOs experienced some challenges with producing consistent rates for the AHRQ measures due to the lack of specific criteria for enrollment and anchor dates.
Overall, of the six measures with corresponding CY 2009 performance targets, the CMOs performed best in the areas of childhood immunizations and lead screening in children, achieving the CY 2009 performance targets for the CMO weighted averages.
Furthermore, regarding the six performance measures with rates for both CY 2008 and CY 2009, the CMO weighted average rates showed statistically significant improvement on two measures: Adults' Access to Preventive/Ambulatory Health Services for members 2044 years of age and Comprehensive Diabetes Care--Hemoglobin A1c (HbA1c) Testing. Both of these measures were

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-16 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES
among the six measures selected as part of the DCH auto-assignment program for CY 2010 which provided the CMOs an incentive to achieve high performance levels by assigning the CMOs additional members. It is possible that the auto-assignment program helped to drive improvement in these areas. HSAG will be able to better assess the impact of this in subsequent measurement periods.
The areas of well-child visits, diabetes care and asthma have the greatest opportunities for improvement for the CMOs as a whole. None of the CMOs achieved the CY 2009 performance target for Well-Child Visits in the First 15 Months of Life, Comprehensive Diabetes Care--HbA1c Testing, and Use of Appropriate Medications for People With Asthma.
Based on CY 2009 CMO performance, AMERIGROUP was the highest overall performer compared with Peach State and WellCare. While AMERIGROUP performed best overall, WellCare demonstrated high performance in specific areas and outperformed the other CMOs in the areas of diabetes care, children's immunizations and some measures of prenatal and postpartum care and birth outcomes. AMERIGROUP met three of the six performance measure targets for CY 2009, followed by WellCare, which met two of six targets. Peach State did not achieve any of the CY 2009 performance targets and was the CMO with the greatest opportunity for improvement with diabetes care and well-care visits. AMERIGROUP's greatest opportunities for improvement are in diabetes care and prenatal and postpartum care. WellCare's priority areas for improvement are in diabetes care and well-care visits.
Recommendations
Based on the CY 2009 performance measure rates and validation of those rates, HSAG provides the following recommendations for improving the quality, timeliness of and access to care and services for members:
DCH should continue to require the CMOs to report on the same set of performance measures for CY 2010 to allow for year-to-year comparisons and trending over time to determine if the CMOs are improving the delivery of quality care to Georgia Families members.
DCH and the CMOs should determine what strategies contributed to high performance measure rates and evaluate whether these strategies can be applied to areas of low performance.
DCH may want to consider measures with low performance for the auto-assignment program as a mechanism to drive improvement.
CMOs should consider collaborating with other CMOs that have a common area of low performance as part of a formal quality improvement process. This has been an effective strategy for many managed care organizations in improving performance measure rates, especially when there is significant overlap of provider networks within a geographic area.
DCH should evaluate PIP topics and prioritize areas of low performance for future statewide collaborative efforts.
AMERIGROUP needs to focus quality improvement efforts in the areas of diabetes care and prenatal and postpartum care by conducting a causal/barrier analysis, evaluating existing strategies and developing new, targeted strategies that address the identified barriers.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-17 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE MEASURES
Peach State and WellCare need to focus quality improvement efforts in the areas of diabetes care and well-care visits by conducting a causal/barrier analysis, evaluating existing strategies and developing new, targeted strategies that address the identified barriers.
Using demographic data, the CMOs need to ensure that interventions are culturally appropriate for their Medicaid managed care population. The CMOs should use evidenced-based strategies when implementing interventions to reach the targeted populations.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 4-18 GA2010-11_EQR_AnnRpt_F2_0811

5. Performance Improvement Projects
The purpose of a performance improvement project (PIP) is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in both clinical and nonclinical areas.
HSAG reviewed each PIP using CMS' validation protocol to ensure that the plans designed, conducted and reported the PIPs in a methodologically sound manner and met all State and federal requirements. The validation was to ensure that DCH and interested parties could have confidence in the reported improvements that resulted from the PIPs.
The CMOs each had six DCH-selected PIP topic areas in progress during the review period. Four topic areas were clinical areas of focus and included the following HEDIS measures:
Lead Screening in Children Childhood Immunization Status--Combination 2 Well-Child Visits in the First 15 Months of Life--Six or More Visits Adults' Access to Preventive/Ambulatory Health Services--2044 Years of Age
In addition, two non-clinical PIP topics were selected by DCH for the CMOs in the areas of member satisfaction and provider satisfaction.
Validating PIPs is one of three federally mandated external quality review activities. The requirement allows states, agents that are not a managed care organization, or an EQRO to conduct the PIP validation. DCH contracted with HSAG to conduct the functions associated with validation of PIPs.
Validation of Performance Improvement Projects
HSAG organized, aggregated and analyzed the three CMOs' PIP data to draw conclusions about the CMOs' quality improvement efforts in the areas of quality, access and timeliness. The PIP validation process evaluated both the technical methods of the PIP (i.e., the study design) and the performance measure outcomes associated with the implementation of interventions. Based on its technical review, HSAG determined the overall methodological validity of the PIPs. Appendix D provides additional detail on the methodology HSAG used for validating the PIPs.
Table 5-1 displays aggregate CMO validation results for all PIPs evaluated from July 2010 to September 2010. The CMOs submitted PIP data that reflected varying time periods, depending on the PIP topic. HSAG provided final, CMO-specific PIP reports to the CMOs and DCH in November 2010. This table illustrates the CMOs' overall understanding of the PIP process for the study's Design, Implementation and Outcomes phases. Each activity is composed of individual evaluation elements scored as Met, Partially Met, or Not Met. Elements receiving a Met score have satisfied the necessary technical requirements for a specific element. Appendix D, Tables D2, D5, and D 8 provide the CMO-specific validation scores.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-1 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE IMPROVEMENT PROJECTS

Table 5-1--SFY 2011 Performance Improvement Project Validation Results

for Georgia Families (N=18 PIPs)

Study Stage

Activity

Percentage of Applicable Elements

Met

Partially Met

Not Met

I. Appropriate Study Topic

100% (96/96)

0% (0/96)

0% (0/96)

Design

II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s)

100% (36/36) 100% (108/108)

0% (0/36)
0% (0/108)

0% (0/36)
0% (0/108)

IV. Correctly Identified Study Population Design Total

100%
(53/53) 100% (293/293)

0%
(0/53) 0% (0/293)

0%
(0/53) 0% (0/293)

Implementation

V. Valid Sampling Techniques (if sampling was used)
VI. Accurate/Complete Data Collection

100% (84/84)
99% (152/153)

0% (0/84)
1% (1/153)

0% (0/84)
0% (0/153)

VII. Appropriate Improvement Strategies Implementation Total

68%
(41/60) 93% (277/297)

2%
(1/60) 1% (2/297)

30%
(18/60) 6%
(18/297)

VIII. Sufficient Data Analysis and Interpretation

Outcomes

IX. Real Improvement Achieved

X. Sustained Improvement Achieved

Outcomes Total

Overall Percentage of Applicable Evaluation Elements Scored Met

73% (112/154)
53% (36/68)
100% (2/2) 67% (150/224)

4%
(6/154)
10%
(7/68)
0%
(0/2) 6% (13/224) 88% (720/814)

23% (36/154)
37% (25/68)
0% (0/2) 27% (61/224)

Findings

Performance Improvement Project Validation Key Findings
The overall aggregated validation results for the Design total during the review period demonstrated the CMOs' proficiency and thorough application of the Design stage. The sound design of the PIPs created a foundation for the CMOs to progress to subsequent PIP stages--i.e., implementing improvement strategies and accurately assessing and achieving study outcomes.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-2 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE IMPROVEMENT PROJECTS

The results demonstrated that the CMOs appropriately conducted the sampling and data collection activities of the Implementation stage. These activities ensured the studies properly defined and collected the necessary data to produce accurate study indicator results.
Lower validation scores for implementing appropriate improvement strategies can result in lower scores for the Outcomes stage, since appropriately implementing and documenting improvement strategies increases the likelihood of success. The overall validation score decrease for the Outcomes total was attributed to the individual score by AMERIGROUP, which was significantly lower than the scores for the other two CMOs. In the Outcomes stage, HSAG assessed for statistically significant improvement between remeasurement years for PIPs that had at least one remeasurement period. Approximately half of these PIPs demonstrated statistically significant improvement during the review period. Thirty-seven percent of PIPs assessed for statistically significant improvement did not demonstrate this improvement. HSAG attributed the lack of improvement primarily to ineffective interventions.
Only two PIPs progressed to a second remeasurement period which HSAG assessed for sustained improvement. Both PIPs achieved sustained improvement.

CMO Comparison Key Findings

Table 5-2 displays the CMOs' validation results by study stage for all six PIPs conducted by each of the three CMOs and evaluated during the review period.

Table 5-2--SFY 2011 Performance Improvement Project Validation Results

Comparison by CMO (N=18 PIPs)

Study Stage

Activities

Percentage of Applicable Elements Scored Met AMERIGROUP Peach State WellCare

Design Implementation

Activities IIV Activities VVII

100% (97/97)
81% (83/102)

100% (98/98) 100% (95/95)

100% (98/98)
99% (99/100)

Outcomes

Activities VIIIX

Overall Percentage of Applicable Evaluation Elements Scored Met

26%
(18/69) 74% (198/268)

83%
(64/77) 95% (257/270)

87%
(68/78) 96% (265/276)

All three CMOs met 100 percent of the requirements across all six PIPs for all four activities within the Design stage. Overall, the CMOs designed scientifically sound studies that were supported by the use of key research principles. The technical design of each PIP was sufficient to measure and monitor PIP outcomes associated with the CMOs' improvement strategies. The solid design of the PIPs allowed the successful progression to the next stage of the PIP process.
AMERIGROUP had the lowest score for the Implementation stage, while the other two CMOs demonstrated a better application of intervention strategies. AMERIGROUP did not document any

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-3 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE IMPROVEMENT PROJECTS

barrier analyses, nor did it propose/implement interventions for its four HEDIS-based PIPs.5-1 Without the successful implementation of appropriate improvement strategies, the CMO cannot achieve and sustain improved outcomes in the future.
All three CMOs scored the lowest for the Outcomes stage. AMERIGROUP's score was significantly lower than either Peach State or WellCare. The execution of the intervention strategies across the 18 PIPs was inconsistent and resulted in mixed outcomes for the study indicators.

Outcome Results

Table 5-3 and Table 5-4 display the outcome data for the CMOs' clinical PIPs. For these HEDISbased PIPs, each CMO used the same study indicator, which allowed HSAG to compare results across the CMOs. Detailed study indicator descriptions as well as rates for each measurement period are provided in Appendix D, Tables D3, D4, D6, D7, D9, and D10. In Table 5-3, HSAG displays the CY 2009 rate and how it compared to the CY 2008 rate. The change between the CY 2008 and CY 2009 rates is noted by directional arrows.

Table 5-3--HEDIS-based Performance Improvement Project Outcomes (validated during SFY 2011)

Comparison by CMO

Remeasurement 1 Period 1/1/200912/31/2009

PIP Topic

AMERIGROUP^

Peach State

WellCare

Lead Screening in Children

67.8%



62.3%



67.4%



Childhood Immunization Status--Combination 2

72.0%



67.6%



81.0%



Well-Child Visits in the First 15

Months of Life--Six or More

55.0%

*

52.3%

Visits



57.4%



Adults' Access to

Preventive/Ambulatory Health

85.5%

*

84.3%

*

84.7%

*

Services--2044 years of age

^ The CMO did not report 2008 measurement period rates as part of the SFY 2011 PIP submission; however, the rates were documented in the 2008 HEDIS Performance Measure Report. They are used here for informational purposes only to allow for comparison between the CY2009 remeasurement period and the prior year, CY 2008, period.
Caution should be used when comparing the results for baseline and Remeasurement 1 due to changes in the study methodology.
* Designates statistically significant change from the prior measurement period (p value < 0.05).

All three CMOs demonstrated statistically significant improvement for the Adults' Access to Care PIP. Additionally, all three showed improvement for the Childhood Immunizations PIP. Peach State and WellCare demonstrated improvement for the Lead Screening in Children PIP, while only Peach

5-1 A compliance issue was noted for AMERIGROUP in the SFY 2011 submission. AMERIGROUP did not complete all the DCH-required activities, resulting in lower scores for both the Implementation and Outcomes stages for its four HEDISbased PIPs.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-4 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE IMPROVEMENT PROJECTS
State reported improvement for its Well-Child Visits PIP. Conversely, across all 12 clinical PIPs, the only statistically significant decline was reported by AMERIGROUP for its Well-Child Visits PIP.
For the satisfaction-based PIPs, each CMO selected different study indicators; therefore, comparisons across the CMOs could not be made. The results are presented only as the number of study indicators instead of specific study indicator rates.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-5 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE IMPROVEMENT PROJECTS

Table 5-4--SFY 2011 Satisfaction-based Performance Improvement Project Outcomes

Comparison by CMO

Comparison to Study Indicator Results

PIP Topic1

from Prior Measurement Period

Declined

Improved

Statistically Significant

Sustained Improvement

Improvement

AMERIGROUP (N=2)

Member Satisfaction

2

0

0



Provider Satisfaction*









Peach State (N=8)

Member Satisfaction

0

4

1



Provider Satisfaction

0

4

3

4

WellCare (N=5)

Member Satisfaction

1

1

0



Provider Satisfaction

1

2

1

2

1The number of study indicators varied per PIP topic conducted by each of the three CMOs for a total of eight study

indicators per PIP topic (N=8).

*AMERIGROUP modified the study methodology and established a new baseline; therefore, only baseline data was

submitted for validation and improvement could not be assessed.

The PIP did not progress to the phase where improvement and/or sustained improvement could be assessed.

AMERIGROUP showed a decline for both of its Member Satisfaction PIP study indicators. Conversely, Peach State demonstrated improvement for all eight of its satisfaction-based PIP study indicators. The improvement was statistically significant for four of the eight study indicators. WellCare's results were mixed, demonstrating improvement for three of five study indicators.

Peach State reported sustained improvement for all of its Provider Satisfaction study indicators while WellCare demonstrated sustained improvement for two of its Provider Satisfaction study indicators.

Conclusions
PIP performance measure outcomes showed mixed results, with some achieving improvement and others demonstrating a decline. An analysis of the interventions related to PIPs demonstrating improvement (Peach State and WellCare) suggested their successful PIP performance measure outcomes may be the result of the CMOs' strong link between identified barriers and interventions, the timing of the interventions and the selection of interventions for system change. AMERIGROUP had the greatest challenge with achieving improved outcomes, which could be due to the CMO's lack of documented barrier analysis and interventions. Other PIPs that did not have performance measure improvement had key factors that may have prevented the desired outcomes. HSAG noted that for these PIPs without improvement, the CMOs did not always implement new or revised strategies; did not implement interventions in time to have an impact on the measurement period; or did not implement interventions for system change.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-6 GA2010-11_EQR_AnnRpt_F2_0811

PERFORMANCE IMPROVEMENT PROJECTS
Recommendations
The CMOs need to ensure their selected interventions are linked to an identified barrier. The CMOs need to plan and implement intervention strategies more efficiently, providing
enough time for the interventions to affect the study outcomes. The CMOs should select interventions for system change, instead of one-time interventions, that
increase the likelihood of achieving and sustaining improvement. The CMOs should consider conducting a drill-down type of analysis before and after the
implementation of any intervention to determine if any subgroup within the population has a disproportionately lower rate that negatively affected the overall rate. The CMOs should use their interventions to target the identified subgroups with the lowest study indicator rates, allowing the implementation of more precise, concentrated interventions. The CMOs should perform interim evaluations of the results in addition to the formal annual evaluation. Evaluation of interim performance measurement results could assist the CMOs in identifying and eliminating barriers that impede improvement. The CMOs should determine if the interventions are having the desired effect or if it is necessary to modify current interventions or implement new interventions to improve results based on the interim evaluation results. DCH may consider selecting performance measures with low rates, such as the diabetes measures, as a PIP strategy for supporting CMO improvement in this area. DCH and the CMOs should explore opportunities to collaborate on the DCH-required PIPs. While the CMOs are required to conduct PIPs in the DCH-selected areas, they have not been held accountable for collaborating. HSAG has identified collaboration through either a statewide collaborative or small-group collaborative effort as an effective strategy to improve rates.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 5-7 GA2010-11_EQR_AnnRpt_F2_0811

6. CMO-specific Follow-up on Prior-Year Recommendations
Introduction
This section presents the CMOs' improvement actions taken in response to HSAG's recommendations included in its prior-year (SFY 2010) External Quality Review Annual Report for Georgia Families Care Management Organizations (CMOs). The actions taken by the CMOs were self-reported and had not yet been validated by DCH or HSAG at the time this report was published. DCH is exploring options to have the EQRO follow up on areas of noncompliance in future years. The recommendations were the result of HSAG's prior-year EQRO activities and findings from its:
Review of the CMOs' compliance with the federal Medicaid managed care structure and operations standards described at 42 CFR 438.214210 (i.e., provider selection, enrollee information, confidentiality, enrollment and disenrollment, grievance systems, subcontractual relationships and delegation) and with the associated DCH contract requirements.*
Validation of the CMOs' PIPs. Validation of the CMOs' performance measures.
* Specific to the compliance review, for each of the requirements for which HSAG found the CMOs'
performance as not fully compliant, the CMOs were required to prepare and submit to DCH and, when approved, implement corrective action plans (CAPs) addressing each HSAG recommendation. The CMOs were also required to provide to DCH documentation related to implementing its CAPs.
Note: The following information describing the CMOs' follow-up actions is a high-level summary of the more detailed information the CMOs reported in documentation they submitted to HSAG.
AMERIGROUP Community Care
Review of Compliance With Operational Standards
Standard I--Provider Selection, Credentialing, and Recredentialing
To improve AMERIGROUP's compliance, HSAG recommended that the CMO ensure all providers' credentialing records include documentation of verification through the Office of Inspector General (OIG) Web site and documentation of primary source verification.
The CMO reported that it:
Re-educated credentialing staff on the established process of documenting OIG verification and including the documentation in the provider credentialing file as well as the contracted delegated vendor files.
Trained its staff on the process for primary source verification including the documentation in the provider's credentialing file.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-1 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS

Standard II--Subcontractual Relationships and Delegation
HSAG recommended that AMERIGROUP:
Define in each of its written delegation agreements the specific functions, activities and reporting responsibilities for each delegated activity.
Revise its delegation agreement with National Imaging Associates (NIA) to reflect the actual (current) activities the CMO delegated to the contractor.
The CMO reported that it:
Initially revised its NIA agreement to reflect actual delegated activities prior to addressing other management service agreements.
Reviewed and amended, where necessary, all management service agreements with delegated vendors to specify delegated activities, remove activities no longer delegated, and specify the reporting requirements as well as the CMO's monitoring and oversight activities.
Standard IV--Member Information
HSAG recommended that AMERIGROUP provide additional information to members about their rights related to: (1) not being liable for the CMO's debts or payment for covered services; (2) the name of the appropriate State agency for filing complaints concerning provider noncompliance with advance directive requirements (3) obtaining assistance when filing an appeal; and (4) the rules that govern representation at an administrative law hearing.
AMERIGROUP reported that in response to HSAG's specific findings, it revised and produced an updated member handbook to include the appropriate language and/or information. AMERIGROUP provided detailed information as to the sections and the exact revised language it included in the updated handbook.
Standard V--Grievance System
HSAG recommended that the CMO:
Update all applicable documents to include complete definitions of an action and the accurate timelines associated with an action.
Develop a process for ensuring that a notice of action is sent to the member when the CMO fails to meet grievance and appeal/administrative review timelines.
Ensure that the revised member handbook includes accurate information about filing grievances and the CMO's review process, including the definition of an action, the telephone number for the teletype/telecommunications device for the deaf (TTY/TDD), the right to present evidence and review files during an administrative law hearing and the time frames for requesting continuation of benefits and how to begin the process.
Update its provider manual to include information about each element of the member grievance system.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-2 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS
Revise its template documents to be consistent with the grievance filing requirements so that members who file an oral grievance are not required to follow up with a written grievance submission in order to have their grievances investigated and resolved.
Go beyond mailing an "unable to contact" letter to members after multiple attempts to follow up on the initial grievance and investigate all matters to the extent possible. The CMO should send a resolution letter that includes any information the CMO was able to obtain, as well as the resolution.
The CMO reported that it:
Revised the member handbook and provider manual to include accurate and complete information in response to each of HSAG's findings related to its notice of action letters, administrative reviews (member appeals), administrative law hearings and member grievances.
Implemented, through its internal and external (DCH) approval process, the revised member handbook and provider manual.
Identified and updated/revised all applicable policies, procedures, and related documents to include accurate and complete information related to its actions to deny or limit requested services or to reduce, suspend or terminate previously approved services. The updates included the associated timelines and required notice to members.
Developed a process for sending notices of action in response to the CMO's failure to act within the time frames for resolution of grievances and appeals; revised all applicable written materials, including information in the member handbook, provider manual, and policies and procedures; and provided staff training on the requirements and processes.
Validation of Performance Improvement Projects
Based on HSAG's validation results, AMERIGROUP had three evaluation elements that did not receive a Met score and a total of seven unique Points of Clarification for its PIPs. HSAG recommended that:
AMERIGROUP focus on the elements that received either a Point of Clarification or a score of Partially Met or Not Met, including those in Activities VIII and IX, and make appropriate changes associated with those evaluation elements. More specifically, HSAG recommended that AMERIGROUP ensure that the study results are presented in a way that provides accurate, clear and easily understood information and that the CMO provide accurate statistical testing results.
AMERIGROUP carefully review each PIP across all activities before submission to ensure consistency throughout each PIP and that results and processes are included correctly in the PIP Summary Form when working with vendors.
AMERIGROUP reported that, as applicable to the individual PIP, it:
Consistently conducted a z test with a 95 percent confidence level for all PIPs internally and with vendors.
Ensured that the correct population was used when preparing a PIP.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-3 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS

Ensured that the study used a representative sampling technique and used accepted research designs and statistical analysis.
Ensured that the survey respondent group was representative of the entire targeted study population.
Clarified when there were exclusions based on length of time in the health plan. Continued to evaluate action steps to determine their ongoing impact on improving PIPs. Identified technical and quality improvement leadership and other resources at the CMO and
corporate level to conduct a review of the PIPs before submission to ensure consistency throughout each PIP. The effort was also to ensure that results and processes are included correctly in the PIP Summary Form when working with vendors.
Additional CMO improvement actions included those specific to designing the studies and selecting and strengthening interventions.
Specific to the Study Design:
Used multidisciplinary staff with input from its Medical Advisory Committee to evaluate interim HEDIS results quarterly and to assess the efficacy of the CMO's interventions for continuation or discontinuation.
Analyzed the demographics of its population to include gender, age, race, ethnicity and geographic location and developed additional interventions based on subgroup analysis to target subpopulations.
Completed the necessary documentation in the required format and submitted the documentation to DCH in a timely manner.
Specific to Interventions:
Implemented a Strategic Outcomes and Analysis provider report for 91 providers with more than 250 assigned members, representing 65 percent of AMERIGROUP's population. Included in this report was the missed opportunity report for 10 HEDIS measures that included lead screening in children, well-child visits and childhood immunizations.
Implemented a member incentive program for well-child visits and childhood immunizations. Met with high-volume providers (with more than 500 members) and gave them lists of members
with ambulatory-sensitive conditions to drive physician office follow-up and reduce ER visits. Hired two full-time member outreach associates to contact members not accessing appropriate
services specific to lead screening in children, well-child visits, childhood immunizations, and adults' access to care. The CMO implemented Televox (robocalls) to contact these members. Continued data analysis to improve administrative data. Continued quarterly EPSDT medical record reviews to ensure provider compliance with the measures of well-child visits, childhood immunizations, and lead screening in children. Tracked and trended member grievances related to treatment dissatisfaction to identify opportunities to improve physician/member relationships and communication. Convened a work group to address differences in responses based on race and ethnicity for more targeted intervention.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-4 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS

Validation of Performance Measures
Although AMERIGROUP did not have any data collection and reporting issues related to the measures, the CMO's performance on these measures suggested opportunities for improvement. Only one measure, Use of Appropriate Medications for People With Asthma, was close to the national 2008 HEDIS Medicaid 90th percentile, and four of the seven measures ranked between the national Medicaid 50th and 75th percentiles. HSAG recommended that AMERIGROUP evaluate which measures require targeted interventions to meet DCH's performance targets.
AMERIGROUP reported that in response to HSAG's recommendation, the CMO took the following improvement actions, in addition to those described above for improving results for the associated PIPs:
Hired two full-time member outreach associates to contact members who had not received timely HbA1c testing.
Added two questions to the diabetes follow-up screening tool for case managers to capture HbA1c and LDL-C testing.
Conducted mailings to members with diabetes that provided education about the disease management program.
Conducted focused member outreach and follow-up for members not responsive to case management intervention for medication compliance.
Conducted mailings to asthmatics related to disease management and medication compliance. Used ER front-end reviewers to identify asthmatic members with high ER utilization for
referrals to case management. Continued using hybrid data collection (which includes medical record review) as a way to
identify opportunities to improve member outcomes. Developed missed opportunity reports related to HbA1c, LDL-C, and the use of appropriate
medications for people with asthma.

Peach State Health Plan

Review of Compliance With Operational Standards

Standard II--Subcontractual Relationships and Delegation

To improve its compliance, HSAG recommended that Peach State:
Review each delegation agreement and ensure that the functions/activities listed as delegated reflect those currently performed by the delegate.
Revise each agreement as needed.

Peach State reported that it:
Updated its written delegation agreements and its written policies related to oversight of delegates.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-5 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS

Corrected a typo identified in its audit grid document that outlined the dates for annual review for delegates.
Standard IV--Member Information
HSAG recommended that Peach State:
Use easy-to-understand terms and language when informing members about their right to get services in agreement with QAPI access standards and define terms such as "administrative law hearing" and "administrative review."
Clarify its written information about providers' appeal rights.
Peach State reported that it:
Made the required changes to its member handbook and included the page numbers with the revisions.
Standard V--Grievance System
HSAG recommended that Peach State:
Revise its member handbook to include the time frame for filing requests for administrative reviews, requirements related to continuation of benefits, a clear definition of appeals and administrative reviews, procedures for obtaining assistance for requesting administrative law hearings, and the fact that the time frame for authorization decisions may be extended.
Review and revise all applicable documents and other materials related to multiple aspects of the administrative review processes and the CMO's notices of action and resolution letters.
Train its staff on the changes to processes, notices and resolution letters. Include all required information about the member grievance system in all appropriate provider
materials.
Peach State reported that it:
Made the required changes to its member handbook and provider manual. Peach State's documentation included the page numbers where it made each of the required changes.
Revised applicable policies and procedures to include the required changes, noting the policy number for each policy it updated.
Trained its staff on the changes to processes, notices and resolution letters.
Standard VI--Disenrollment Requirements and Limitations
HSAG recommended that Peach State:
Revise the member handbook to include all the allowable reasons to request disenrollment. Include in its disenrollment policy the fact that one of the CMO's allowable reasons for
requesting member disenrollment was a member's noncompliance with the treating physician's plan of care.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-6 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS

Peach State reported that it:
Made the required change to its member handbook, including the additional allowable reasons to request disenrollment and the fact that one of its allowable reasons was a member's noncompliance with the treating physician's plan of care.
Revised its disenrollment policy to include lack of member compliance with the treating physician's plan of care as one of the reasons the CMO could request member disenrollment. Included in the documentation the page numbers where the CMO added the information.

Validation of Performance Improvement Projects
While all PIPs received an overall Met status, HSAG identified opportunities for improvement for two of the PIPs. Based on HSAG's PIP validation results, HSAG recommended that:
Peach State focus on and make appropriate changes to the evaluation elements that received either a Point of Clarification or a score of Partially Met, including, as applicable, those in Activity IX.
Peach State carefully review each PIP across all activities before submission to ensure the consistency of statements made in more than one activity of the PIP and to ensure that results and processes are included correctly in the PIP Summary Form when working with vendors.
Peach State reported that, based on HSAG's Point of Clarification related to several of the PIPs the CMO submitted in SFY 2009, the CMO initiated the following improvements for its PIPs validated in SFY 2010 and/or for future PIP submissions:
Childhood Immunizations PIP--All references to the pneumococcal vaccine were removed, and updates reflected 2010 HEDIS specifications for Combo 2.
Well-Child Visits PIP--Rates prior to baseline were included in Activity I. Adults' Access to Care PIP--Timelines with complete date ranges for all measurement periods
were added for Remeasurement 2. Provider Satisfaction PIP--The PIP validated in SFY 2010 with remeasurement results showed
that all four PIP study indicators demonstrated sustained improvement, having shown improvement between all measurement periods. Member Satisfaction PIP--As recommended, the goal for Question 33 was increased (to 94.4 percent). Also, as recommended, the margin of error was included in Activity V. The Peach State Quality Management Department reviewed each PIP activity prior to submission on July 30, 2010, to ensure the consistency of statements made in all activities of the PIP and that results and processes were included correctly in the PIP Summary Form when working with its vendors.

Validation of Performance Measures
Although Peach State did not have any data collection and reporting issues related to the measures, the CMO's performance on these measures suggested opportunities for improvement. For

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-7 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS
Childhood Immunization Status--Combination 2, Peach State performed between the national 2008 HEDIS Medicaid 10th and 25th percentiles. For Comprehensive Diabetes Care--HbA1c Testing, Peach State performed below the national 2008 HEDIS Medicaid 10th percentile. HSAG recommended that Peach State:
Include all appropriate populations in the calculations of the performance measures. Evaluate which measures required targeted interventions to meet DCH's performance targets.
Peach State reported that:
It included all appropriate populations in the calculations of the performance measures and followed the 2010 HEDIS specifications. The CMO stated that these performance measures were audited by Attest (PSHP's auditor).
The CMO evaluated its Childhood Immunization Status and Comprehensive Diabetes Care measure results to determine targeted interventions. The CMO's interventions included data integrity review, member and provider outreach and working with the CMO's lab vendors to obtain data.
WellCare of Georgia, Inc.
Review of Compliance With Operational Standards
Standard IV--Member Information
HSAG recommended that WellCare:
Include in the list of member rights it communicated to members and providers the right to be furnished services in accordance with federal requirements and the right to be responsible for cost sharing only as specified in the DCH contract.
Clarify the member's right to request, receive or amend his or her medical records and the right not to be held liable for the CMO's debts.
Provide information to members about: (1) the State agency to which they should direct complaints concerning provider noncompliance with advance directive requirements and (2) rules governing representation at an administrative law hearing.
Remove a statement in the member handbook that required members to tell the plan before seeking emergent/urgent care and poststabilization services.
WellCare reported that the CMO:
Revised its member handbook and, when applicable, its provider manual to ensure that the information about member rights was complete and accurate.
Revised the member handbook to include:

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-8 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS

A statement informing members that complaints concerning noncompliance with the advance directive requirements could be filed with the appropriate State agency (Georgia Department of Human Services, Office of Regulatory Services).
Information about the various rules that govern representation at an administrative law hearing (ALH), per the DCH contractual requirements.
Removed a statement from the member handbook that told members that they must tell the CMO before getting emergent/urgent care and poststabilization services.
Standard V--Grievance System
HSAG recommended that WellCare:
Clarify in its policies and procedures the definition of a proposed action and the time frames associated with all grievance-related processes.
Revise its policies and corresponding training documents to ensure they address and are consistent with all applicable requirements.
Revise its member handbook and applicable provider materials to include all required information about the requirements and procedures related to the member grievance system.
Develop a method to ensure it uses easy-to-understand language in the customized sections of the notices of proposed action letters.
WellCare reported with considerable detail that it:
Updated the Georgia Medicaid Grievance policy (C6GR GA-010) under the Definitions section with the term "action," as it is stated in the Georgia DCH contract and the Grievance Filing section, to state: "Prior to accessing the ALH process, the member must exhaust the internal grievance process."
Revised the Adverse Determinations/Proposed Action policy and the Notice of Proposed Action letter to include the time frame of 10 days to file a request for continuation of benefits during an administrative review.
Developed and conducted training that addressed notice requirements according to federal regulations and the DCH contract. Following the training, the CMO developed and implemented a monitoring plan to ensure compliance. The plan included conducting monthly reviews of a random sample of member notice of proposed action (NPA) letters to ensure: the language in the letters meets the required format and grade level requirement; the information/explanation is in language that is easy to understand and the letters do not include acronyms used without definition or explanation.
Revised its member and provider handbooks to include all required information about the requirements and procedures related to the member grievance system.

Validation of Performance Improvement Projects
WellCare had three evaluation elements that did not receive a Met score and HSAG documented three unique Points of Clarification for each PIP. Based on the validation results for these PIPs, HSAG recommended that WellCare:

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-9 GA2010-11_EQR_AnnRpt_F2_0811

CMO-SPECIFIC FOLLOW-UP ON PRIOR-YEAR RECOMMENDATIONS
Focus on and make appropriate changes to the evaluation elements that received a Point of Clarification or a score of Partially Met, including those in Activity IX.
Carefully review each PIP across all activities before submission to ensure the consistency of statements made in more than one activity of the PIP and to ensure that results and processes are included correctly in the PIP Summary Form when working with vendors.
WellCare reported that in response to HSAG's scores and points of clarification, the CMO took the following improvement actions or developed the following improvement plans:
For its Childhood Immunization PIP, the CMO revised and more accurately described the study indicator.
For the Well-Child Visits in the First 15 Months PIP, the CMO clarified that rates prior to the baseline period were not available.
For the Provider Satisfaction PIP, the CMO updated benchmarks on an annual basis and provided the z values as well as the p values.
While not in response to an HSAG Point of Clarification, WellCare reported that it also continued to develop and implement interventions to ensure sustained improvement.
Validation of Performance Measures
While HSAG determined that WellCare's processes related to data integration, data control and performance indicator documentation were all acceptable, HSAG did recommend that WellCare:
Continue to enhance its mechanism for tracking and monitoring rejected claims/encounters from the data clearinghouses.
Implement a formal reconciliation process for its provider data between CACTUS, the initial database into which data are entered, and Paradigm, the database where data are eventually loaded.
WellCare reported that it:
Continued to enhance its mechanism for tracking and monitoring rejected claims/encounters from the data clearinghouses, including implementing new processes to track and monitor rejections of claims/encounters.
Implemented mechanisms to enhance the CMO's ability to have confidence in the accuracy and completeness of the data it used in calculating and reporting its performance data. For example, the CMO reported that it implemented: More edits to comply with federal and State mandates for cleaner claim submissions. A formal reconciliation process between CACTUS and Paradigm for its provider data.

SFY 2011 External Quality Review Annual Report State of Georgia

Page 6-10 GA2010-11_EQR_AnnRpt_F2_0811

Appendix A. Methodology for Reviewing Compliance With Standards
The following is a description of how HSAG conducted the external quality review of compliance with standards for the CMOs. It includes:
The objective for conducting the review. The technical methods used to collect and analyze the data. A description of the data obtained.
HSAG followed standardized processes in conducting the review of each CMO's performance.

Objective
The primary objective of the compliance review was to provide meaningful information to DCH and the CMOs about the CMOs' compliance with federal measurement and improvement standards and the related DCH contract requirements. DCH and the CMOs can use the information and findings from the review to:
Evaluate the quality and timeliness of, and access to, care and services furnished to members. Identify, implement, and monitor interventions to improve these aspects of care and services.

Technical Methods of Collecting and Analyzing the Data
HSAG developed and used a data collection tool to assess and document the CMOs' compliance with the selected federal Medicaid managed care regulations, State rules, and the associated DCH contractual requirements. The review tool addressed the following three performance areas:
Standard I--Practice Guidelines Standard II--Quality Assessment and Performance Improvement Standard III--Health Information Systems
HSAG conducted on-site compliance reviews in October 2010. The CMOs submitted documentation that covered the review period of October 1, 2009, through September 30, 2010. HSAG provided detailed, final audit reports to the CMOs and DCH in February 2011. The on-site review in October 2010 was the third year of a three-year cycle of compliance reviews that HSAG conducted for the CMOs under its contract with DCH.
HSAG requested and obtained from the CMOs documentation related to the standards and used this written information for its pre-on-site desk review. HSAG obtained additional information through interactions, discussions, system demonstrations, and interviews with the CMOs' key staff members during the on-site portion of the review.
To draw conclusions about the CMOs' performance, HSAG aggregated and analyzed the data resulting from its desk and on-site review activities. HSAG used scores of Met, Partially Met, and

SFY 2011 External Quality Review Annual Report State of Georgia

Page A-1 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR REVIEWING COMPLIANCE WITH STANDARDS

Not Met to indicate the degree to which the CMOs' performance complied with the requirements. A designation of NA was used when a requirement was not applicable to a CMO during the period covered by the review. This scoring methodology was consistent with CMS' final protocol, Monitoring Medicaid Managed Care Organizations (MCOs) and Prepaid Inpatient Health Plans (PIHPs): A Protocol for Determining Compliance With Medicaid Managed Care Proposed Regulations at 42 CFR Parts 400, 430, et al, February 11, 2003.
From the scores it assigned for each of the requirements, HSAG calculated a total percentage-ofcompliance score for each of the three standards and an overall percentage-of-compliance score across the three standards. HSAG calculated the total score for each of the standards by adding the weighted score for each requirement in the standard receiving a score of Met (value: 1 point), Partially Met (value: 0.50 points), Not Met (0 points), and Not Applicable (0 points) and dividing the summed, weighted scores by the total number of applicable requirements for that standard.

Description of Data Obtained

To assess the CMOs' compliance, HSAG reviewed a wide range of written documents produced by the CMOs, including the following:

Committee meeting agendas, minutes, and handouts Written policies and procedures Clinical practice guidelines The provider manual and other communication to providers/subcontractors The member handbook and other written member informational materials Technical system specification manuals and on-site system demonstrations Narrative and/or data reports across a broad range of performance and content areas

The following table lists the major data sources HSAG used in determining the CMOs' performance in complying with requirements and the time period to which the data applied.

Table A-1--Description of Data Sources

Data Obtained

Time Period to Which the Data Applied

Documentation submitted for HSAG's desk review and additional documentation available to HSAG during the on-site review (from the CMOs)

October 1, 2009, through the last day of the on-site review

Information obtained through interviews with CMO October 1, 2009, through the last day of the on-site

staff members

review

HSAG provided CMO-specific reports to DCH and the CMOs containing detailed information about the process and findings from the review of compliance with standards.

SFY 2011 External Quality Review Annual Report State of Georgia

Page A-2 GA2010-11_EQR_AnnRpt_F2_0811

Appendix B. Methodology for Conducting Validation of Performance Measures
The following is a description of how HSAG conducted the validation of performance measures activity for the DCH Georgia Families CMOs. It includes:
The objectives for conducting the activity. The technical methods used to collect and analyze the data. A description of the data obtained.

Objectives
The primary objectives of HSAG's performance measure validation process were to:
Evaluate the accuracy of the performance measure data collected by the CMOs and DCH. Determine the extent to which the specific performance measures calculated by the CMOs or
the State (or on behalf of the CMOs or the State) followed the specifications established for each performance measure.
HSAG began performance measure validation in February 2010 and completed validation in June 2010. The CMOs submitted performance measure data that reflected the period of January 1, 2009, through December 31, 2009. HSAG provided final performance measure validation reports to the CMOs and DCH in July 2010.

Technical Methods of Data Collection and Analysis
HSAG conducted the validation activities as outlined in the CMS publication, Validating Performance Measures: A Protocol for Use in Conducting External Quality Review Activities, final protocol, Version 1.0, May 1, 2002. Pre-on-site activities and document review were conducted, followed by an on-site visit to each CMO and DCH that included interviews with key staff and system demonstrations. Finally, post-review follow-up was conducted with each CMO and DCH on any issues identified during the site visit. Information and documentation from these processes were used to assess the validity of the performance measures.
The CMS protocol identified key types of data that should be collected and reviewed as part of the validation process. The list below describes how HSAG collected and analyzed these data:
An Information Systems Capabilities Assessment Tool (ISCAT) was requested from each CMO as well as DCH and their subcontracted vendor. HSAG conducted a high-level review of each ISCAT to ensure that all sections were completed and all attachments were present. The validation team reviewed all ISCAT documents, noting issues or items that needed further follow-up, and began completing the review tools, as applicable.
Source code (programming language) for performance indicators was requested. Each CMO and DCH submitted source code for measures that were not calculated using NCQA-certified software. HSAG completed line-by-line code review and observation of program logic flow to ensure compliance with performance measure definitions. Areas of deviation were identified

SFY 2011 External Quality Review Annual Report State of Georgia

Page B-1 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE MEASURES

and shared with the lead auditor to evaluate the impact of the deviation on the indicator and assess the degree of bias (if any). Supporting documentation included any documentation that provided reviewers with additional information to complete the validation process, including policies and procedures, file layouts, system flow diagrams, system log files, and data collection process descriptions. The validation team reviewed all supporting documentation, with issues or clarifications flagged for further follow-up.
The following table displays the data sources used in the validation of performance measures and the time period to which the data applied.

Table B-1--Description of Data Sources
Data Obtained
Roadmap (From the CMOs) Source Code (Programming Language) for Performance Measures (From the CMOs and DCH) Supporting Documentation (From the CMOs and DCH) Current Performance Measure Results (From the CMOs and DCH) On-site Interviews and Demonstrations (From the CMOs and DCH)

Time Period to Which the Data Applied CY 2009
CY 2009
CY 2009 CY 2009 CY 2009

SFY 2011 External Quality Review Annual Report State of Georgia

Page B-2 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE MEASURES

Table B-2--Utilization Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

Rate

CY 2009 Percentile
Rank1

Symbol

Rate

CY 2009 Percentile
Rank

Symbol

Rate

CY 2009 Percentile
Rank

Symbol

Inpatient Utilization--General Hospital/Acute Care

Total Inpatient Discharges Per 1,000 Member Months

6.6

Total Inpatient Days Per 1,000 Member Months

22.2

Total Inpatient Average Length of Stay

3.4

Medicine Discharges Per 1,000 Member Months

1.1

Medicine Days Per 1,000 Member Months

4.1

Medicine Average Length of Stay

3.8

Surgery Discharges Per 1,000 Member Months

0.6

Surgery Days Per 1,000 Member Months

4.7

Surgery Average Length of Stay

8.4

Maternity Discharges Per 1,000 Member Months

10.8

Maternity Days Per 1,000 Member Months

29.5

Maternity Average Length of Stay

2.7

Mental Health Utilization

P10-P24 P10-P24 P25-P49
<P10 <P10 P50-P74 <P10 P10-P24 P75-P89 P75-P89 P75-P89 P50-P74

7.2

P25-P49

23

P10-P24

3.2 P10-P24

1.4

<P10

4.6

<P10

3.3 P25-P49

0.6

<P10

4.7 P10-P24

7.7 P75-P89

12.1

>=P90

31.6

>=P90

2.6 P25-P49

7.6

P25-P49

24.3 P10-P24

3.2 P25-P49

1.6

<P10

5.4 P10-P24

3.4 P25-P49

0.8 P10-P24

4.8 P10-P24

6.4 P50-P74

11.8

>=P90

32

>=P90

2.7 P50-P74

Any Services 7.4

P25-P49

6.6

P25-P49

7.4

P25-P49

Inpatient 0.4

P10-P24

0.2

<P10

0.9

P50-P74

Georgia Families

Rate

CY 2009 Percentile
Rank

Symbol

7.2

P25-P49

23.5 P10-P24

3.2

P25-P49

1.4

<P10

4.9

P10-P24

3.4

P25-P49

0.7

<P10

4.8

P10-P24

7.1

P50-P74

11.7

>=P90

31.3

>=P90

2.7

P25-P49

7.1

P25-P49

0.6

P25-P49

SFY 2011 External Quality Review Annual Report State of Georgia

Page B-3 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE MEASURES

Table B-2--Utilization Domain Measures, CMO Comparison

AMERIGROUP

Peach State Health Plan

WellCare

Measure

Rate

CY 2009 Percentile
Rank1

Symbol

Rate

CY 2009 Percentile
Rank

Symbol

Rate

CY 2009 Percentile
Rank

Symbol

Intensive Outpatient/Partial Hospitalization

0.1

P25-P49

1.4

P75-P89

Outpatient/ED 7.3 P25-P49

6.4

P25-P49

Antibiotic Utilization

Average Scrips PMPY for Antibiotics

1.4

Average Days Supplied per Antibiotic Scrip

9.3

Average Scrips PMPY for Antibiotics of Concern

0.6

P75-P89 P50-P74 P75-P89

1.4

P75-P89

9

P25-P49

0.6 P50-P74

Percentage of Antibiotics of Concern of all Antibiotic 44.3 Scrips
Outpatient Drug Utilization

P50-P74

42.9 P50-P74

Average Cost of Prescriptions Per Member Per Month

24.8

P10-P24

24.5 P10-P24

Average Number of

Prescriptions Per Member Per 7.6 P10-P24

7.2 P10-P24

Month
1 CY 2009 percentile rank was based on NCQA's 2009 Audit, Means, Percentiles and Ratios.

1.4

P75-P89

7.1

P25-P49

1.5

P75-P89

9

P25-P49

0.7 P75-P89

44.4 P50-P74

24.5 P10-P24 7.8 P10-P24

Below 25th Percentile 25th74th Percentile 75th Percentile or Above

Georgia Families

Rate

CY 2009 Percentile
Rank

Symbol

1.1

P75-P89

6.9

P25-P49

1.4

P75-P89

9.1

P25-P49

0.6

P75-P89

44

P50-P74

24.6 P10-P24

7.6

P10-P24

SFY 2011 External Quality Review Annual Report State of Georgia

Page B-4 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE MEASURES

Table B-3--Health Plan Membership Information

Health Plan and Membership Measure

2010 CMO Rate1

CY 2009 Percentile
Rank

Race/Ethnicity Diversity of Membership

Symbol

White

43.2

P50-P74

Black

47.5

P75-P89

Asian

1.9

P50-P74

Unknown

5.5

P10-P24

Hispanic or Latino

1.9

P25-P49

Not Hispanic or Latino

4.8

P25-P49

Unknown Ethnicity

93.4

Language Diversity of Membership

P75-P89

English

89.1

P50-P74

Spanish or Spanish Creole

6.8

P75-P89

Unknown

4

P25-P49

Weeks of Pregnancy at Time of Enrollment

<0 Weeks

8.8

<112 Weeks

6.3

<1327 Weeks

56.9

<28 or More Weeks

19.7

Unknown

8.4

Total

100

1 CY 2009 percentile rank was based on NCQA's 2009 Audit, Means, Percentiles and Ratios.

Below 25th Percentile 25th74th Percentile 75th Percentile or Above

SFY 2011 External Quality Review Annual Report State of Georgia

Page B-5 GA2010-11_EQR_AnnRpt_F2_0811

Appendix C. Methodology for Conducting Encounter Data Validation
The following is a description of how HSAG conducted the encounter data validation (EDV) study for the DCH Georgia Families CMOs. It includes:
The objective for conducting the study. The technical methods used to collect and analyze the data. A description of the data obtained.
HSAG followed standardized processes in conducting the review of each CMO's encounter data.

Objective
The primary objective of the encounter data validation was to provide meaningful information to DCH and the CMOs about the accuracy and completeness of the electronic encounter data submitted by the CMOs to DCH. DCH relies on encounter data submissions to monitor and improve the quality of care, calculate performance measures, generate accurate reports, and set valid capitation rates. The completeness and accuracy of these data are essential to the overall management and oversight of the Georgia Families managed care program.

Technical Methods of Collecting and Analyzing the Data
The EDV study was composed of two analytic components: analysis of electronic encounter data and medical record review.
The first component examined the quality of encounters submitted to DCH to assess the timeliness, completeness, appropriateness, and reasonableness of the data in required fields on the encounters. Analysts also evaluated the data by CMO and across time to determine consistency of volume (averages per member and by encounter type) and to look for monthly variations that might indicate gaps in data submission.
The second component of the EDV study assessed the completeness and accuracy of Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) professional encounters through medical record review. This assessment determined if electronic encounter records contained complete and accurate documentation for the specific service based on members' medical records, and whether the required components of EPSDT visits were documented as having been completed during the visit.
HSAG used random sampling to select encounters from each CMO for the medical record review. The CMOs were responsible for procuring the medical records for selected members from providers. HSAG's coders used an electronic abstraction tool to evaluate the sample visits and collect the medical record review data for further analysis.
Once data collection was completed, HSAG used five study indicators to analyze and report the medical record review results:

SFY 2011 External Quality Review Annual Report State of Georgia

Page C-1 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING ENCOUNTER DATA VALIDATION

1. Medical record agreement rate--The percentage of sampled dates of service identified in the electronic encounter data that were also found in members' medical records. This rate was also calculated for diagnosis and procedure codes.
2. Medical record omission rate--The percentage of sampled dates of service identified in the electronic encounter data that were not found in members' medical records. This rate was also calculated for diagnosis and procedure codes.
3. Encounter data omission rate--The percentage of dates of service from members' medical records that were not found in the electronic encounter data. This rate was also calculated for diagnosis and procedure codes.
4. Accuracy rate of coding--The percentage of diagnosis codes associated with validated dates of service from the electronic encounter data that were correctly coded based on members' medical records. This rate was also calculated for procedure codes.
5. Required EPSDT component completion rate--The percentage of sample EPSDT visits, based on dates of service identified in the electronic encounter data, in which all of the required EPSDT components were documented in members' medical records.

Description of Data Obtained

The following table lists the major data sources HSAG used in conducting the encounter data validation study and the time period to which the data applied.

Table C-1--Description of Data Sources

Data Obtained

Time Period to Which the Data Applied

Professional, institutional, and pharmacy encounter files submitted to DCH by the CMOs (from DCH's data vendor)

Dates of service were on or between January 1, 2008, and December 31, 2008

Medical records selected through Dates of service were on or between January 1, 2008, and December sampling (from CMOs/providers) 31, 2008

Information and reports from DCH CMO encounter data submission policy (revised March 2, 2009)

Encounter 837 Companion Guide V2.21 (July 24, 2009)

NCPDP Encounters Companion Guide V1.14 (August 3, 2007)

Encounter Exceptions (September 22, 2009)

Health Check manual (version 6, July 2009)

Policies and procedures for Health Check (October 2008)

Information and reports from Myers and Stauffer

Georgia CMO encounter reports covered February 2008 through March 2010 and were published on June 11, 2010

HSAG provided DCH with a comprehensive report containing detailed information on the process and findings from the encounter data validation study.

SFY 2011 External Quality Review Annual Report State of Georgia

Page C-2 GA2010-11_EQR_AnnRpt_F2_0811

Appendix D. Methodology for Conducting Validation of Performance Improvement Projects
The following is a description of how HSAG conducted the validation of performance improvement projects (PIPs) for the Georgia Families CMOs. It includes:
Objectives for conducting the activity. Technical methods used to collect and analyze the data. Description of data obtained.
HSAG followed standardized processes in conducting the validation of each CMO's PIPs.
Objective
The primary objective of PIP validation was to determine each CMO's compliance with requirements set forth in 42 CFR 438.240(b)(1), including:
Measurement of performance using objective quality indicators. Implementation of systematic interventions to achieve improvement in quality. Evaluation of the effectiveness of the interventions. Planning and initiation of activities for increasing or sustaining improvement.
Technical Methods of Data Collection and Analysis
In this third year of validating CMO PIPs, HSAG conducted PIP validation on six DCH-selected PIPs for each CMO. The topics were:
Access/Service Capacity Childhood Immunization- Combination 2 Improving Childhood Lead Screening Rates Member Satisfaction Provider Satisfaction Well-Child Visits during the First 15 Months of Life With Six or More Visits
The HSAG PIP Review Team consisted of, at a minimum, an analyst with expertise in statistics and study design and a clinician with expertise in performance improvement processes. The methodology used to validate PIPs was based on CMS guidelines as outlined in the CMS publication, Validating Performance Improvement Projects: A Protocol for Use in Conducting Medicaid External Quality Review Activities, final protocol, Version 1.0, May 1, 2002. Using this protocol, HSAG, in collaboration with DCH, developed a PIP Summary Form to ensure uniform validation of PIPs. The PIP Summary Form standardized the process for submitting information regarding the PIPs and ensured that all CMS PIP protocol requirements were addressed.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-1 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Using the CMS PIP validation protocol as its guide, HSAG developed a PIP Validation Tool, which was approved by DCH. This tool ensured the uniform assessment of PIPs across all CMOs and contained the following validation activities:

Activity I. Activity II. Activity III. Activity IV. Activity V. Activity VI. Activity VII. Activity VIII. Activity IX. Activity X.

Appropriate Study Topic(s) Clearly Defined, Answerable Study Question(s) Clearly Defined Study Indicator(s) Correctly Identified Study Population Valid Sampling Techniques (if sampling was used) Accurate/Complete Data Collection Appropriate Improvement Strategies Sufficient Data Analysis and Interpretation Real Improvement Achieved Sustained Improvement Achieved

Each required protocol activity consisted of evaluation elements necessary to complete a valid PIP. The HSAG PIP Review Team scored evaluation elements within each activity as Met, Partially Met, Not Met, Not Applicable, or Not Assessed. To ensure a valid and reliable review, HSAG designated some of the elements as critical elements. All of the critical elements had to be Met for the PIP to produce valid and reliable results. Given the importance of critical elements to this scoring methodology, any critical element that received a Not Met score resulted in an overall validation rating for the PIP of Not Met. A CMO would be given a Partially Met score if 60 percent to 79 percent of all evaluation elements were Met or one or more critical elements were Partially Met.

HSAG included a Point of Clarification in its reports when documentation for an evaluation element included the basic components to meet requirements for the evaluation element, but enhanced documentation would demonstrate a stronger understanding of the CMS protocol.

In addition to the validation status (e.g., Met) each PIP was given an overall percentage score for all evaluation elements (including critical elements). HSAG calculated the overall percentage score by dividing the total number of elements scored as Met by the total number of elements scored as Met, Partially Met, and Not Met. HSAG also calculated a critical element percentage score by dividing the total number of critical elements scored as Met by the sum of the critical elements scored as Met, Partially Met, and Not Met.

HSAG assessed the implications of the study's findings on the validity and reliability of the results with one of the following three determinations of validation status:

Met: High confidence/confidence in the reported PIP results. Partially Met: Low confidence in the reported PIP results. Not Met: Reported PIP results that were not credible.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-2 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Description of Data Obtained
To validate the PIPs, HSAG obtained and reviewed information from each CMO's PIP Summary Form. The CMOs were required to submit a PIP Summary Form for each of the DCH-selected topics for validation. The PIP Summary Forms contained detailed information about each PIP and the activities completed for the validation cycle. HSAG began PIP validation in July 2010 and completed validation in September 2010. The CMOs submitted PIP data that reflected varying time periods, depending on the PIP topic. HSAG provided final, CMO-specific PIP reports to the CMOs and DCH in November 2010.
The following table displays the data source used in the validation of each performance improvement project and the time period to which the data applied.

CMO AMERIGROUP
Peach State WellCare

Table D-1--Description of Data Sources

Data Obtained

Time Period to Which the Data Applied

Lead Screening in Children PIP

Childhood Immunizations PIP Well-Child Visits PIP

January 1, 2009December 31, 2009

Adults' Access to Care PIP

Member Satisfaction PIP

February 17, 2010May 2, 2010

Provider Satisfaction PIP

September 1, 2009December 31, 2009

Lead Screening in Children PIP

Childhood Immunizations PIP Well-Child Visits PIP

January 1, 2009December 31, 2009

Adults' Access to Care PIP

Member Satisfaction PIP

March 12, 2010May 31, 2010

Provider Satisfaction PIP

September 29, 2009October 27, 2009

Lead Screening in Children PIP

Childhood Immunizations PIP Well-Child Visits PIP

January 1, 2009December 31, 2009

Adults' Access to Care PIP

Member Satisfaction PIP

February 1, 2010May 31, 2010

Provider Satisfaction PIP

October 1, 2008September 30, 2009

HSAG provided CMO-specific reports to DCH and the CMOs that contained detailed information about the process and findings from the validation of PIPs.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-3 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

AMERIGROUP

Table D-2--SFY 2011 Performance Improvement Project Validation Results for AMERIGROUP Community Care (N=6 PIPs)

Study Stage

Activity

I. Appropriate Study Topic

Design

II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s)

IV. Correctly Identified Study Population

Design Total Implementation

V. Valid Sampling Techniques (if sampling was used)
VI. Accurate/Complete Data Collection

VII. Appropriate Improvement Strategies Implementation Total

Outcomes

VIII. Sufficient Data Analysis and Interpretation IX. Real Improvement Achieved

Percentage of Applicable Elements

Met

Partially Met

Not Met

100% (32/32) 100% (12/12)

0% (0/32)
0% (0/12)

0% (0/32)
0% (0/12)

100%
(36/36)
100%
(17/17) 100% (97/97)

0%
(0/36)
0%
(0/17) 0% (0/97)

0%
(0/36)
0%
(0/17) 0% (0/97)

100% (30/30)

0% (0/30)

0% (0/30)

98%
(50/51)
14%
(3/21) 81% (83/102)

2%
(1/51)
0%
(0/21) 1% (1/102)

0%
(0/51)
86%
(18/21) 18% (18/102)

27% (13/49)

4% (2/49)

69% (34/49)

25% (5/20)

0% (0/20)

75% (15/20)

X. Sustained Improvement Achieved







Outcomes Total Overall Percentage of Applicable Evaluation Elements Scored Met

26% (18/69)

3% (2/69)
74% (198/268)

71% (49/69)

The PIPs did not progress to this phase during the review period and could not be assessed for sustained improvement.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-4 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Table D-3--HEDIS-based Performance Improvement Project Outcomes

for AMERIGROUP Community Care

PIP #1--Lead Screening in Children

PIP Study Indicator^

Baseline Period Remeasurement 1 Remeasurement 2 (1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10)

Sustained Improvement

The percentage of children 2 years

of age who received one blood lead test (capillary or venous) on or

68.2%

67.8%





before their second birthday.

PIP #2--Childhood Immunizations

The percentage of children who

received the recommended

vaccinations based on the Childhood

29.8%

72.0%





Immunization Status--Combo 2

(4:3:1:2:3:1) guidelines.

PIP #3--Well-Child Visits

The percentage of children who had

six or more well-child visits with a PCP during their first 15 months of

62.3%

55.0%





life.

PIP #4--Adults' Access to Care

The percentage of members 2044

years of age who had an ambulatory

81.2%

85.5%*





or preventive care visit.

^ The CMO did not report 2008 measurement period rates as part of the SFY 2011 PIP submission; however, the rates were

documented in the 2008 HEDIS Performance Measure Report. They are reported here for informational purposes only to

allow for comparison between the CY2009 remeasurement period and the prior year, CY 2008, period.

The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement.

Caution should be used when comparing the results for baseline and Remeasurement 1 due to changes in the study

methodology. * Designates statistically significant improvement over the prior measurement period (p value < 0.05).

Designates a statistically significant decline in performance over the prior measurement period (p value < 0.05).

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-5 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Table D-4--Satisfaction-based Performance Improvement Project Outcomes for AMERIGROUP Community Care

PIP #5--Member Satisfaction

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (2/13/095/10/09) (2/17/105/2/10) (2/13/115/10/11) Improvement

1) The percentage of members

responding "Yes" to Q10--"In the

last six months, did your child's

doctor or other health provider talk

68.9%

60.3%





with you about the pros and cons

of each choice for your child's

treatment or health care?"

2) The percentage of members

responding "Yes" to Q11--"In the

last six months, when there was

more than one choice for your

child's treatment or health care,

61.1%

55.1%





did your child's doctor or other

health provider ask you which

choice you thought was best for

your child?"

PIP #6--Provider Satisfaction

PIP Study Indicator^

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (9/1/0912/31/09) (9/1/1012/31/10) (9/1/1112/31/11) Improvement

Percentage of providers answering

"Excellent" or "Very Good" to

Q34C--"Contacting the

AMERIGROUP pharmacy call

18.3%



center to find out about formulary

medications and alternatives to

nonformulary medications."





^ Providers were requested to respond if they agreed with the statements regarding the CMO.
The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-6 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Peach State

Table D-5--SFY 2011 Performance Improvement Project Validation Results for Peach State Health Plan (N=6 PIPs)

Study Stage

Activity

I. Appropriate Study Topic

Design

II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s)

IV. Correctly Identified Study Population
Design Total V. Valid Sampling Techniques (if sampling was used)
Implementation VI. Accurate/Complete Data Collection

VII. Appropriate Improvement Strategies Implementation Total

Outcomes

VIII. Sufficient Data Analysis and Interpretation IX. Real Improvement Achieved^

X. Sustained Improvement Achieved Outcomes Total^ Overall Percentage of Applicable Evaluation Elements Scored Met

Percentage of Applicable Elements

Met

Partially Met

Not Met

100% (32/32)
100% (12/12)
100% (36/36)
100% (18/18) 100% (98/98)

0% (0/32)
0% (0/12)
0% (0/36)
0% (0/18) 0% (0/98)

0% (0/32)
0% (0/12)
0% (0/36)
0% (0/18) 0% (0/98)

100% (24/24)

0% (0/24)

0% (0/24)

100%
(51/51)
100%
(20/20) 100% (95/95)

0%
(0/51)
0%
(0/20) 0% (0/95)

0%
(0/51)
0%
(0/20) 0% (0/95)

90% (47/52)
67% (16/24)
100% (1/1) 83% (64/77)

6%
(3/52)
21%
(5/24)
0%
(0/1) 10% (8/77) 95% (257/270)

4% (2/52)
13% (3/24)
0% (0/1) 6% (5/77)

^ The percentage total for this activity or study stage does not equal 100 percent due to rounding.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-7 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Table D-6--HEDIS-based Performance Improvement Project Outcomes for Peach State Health Plan

PIP #1--Lead Screening in Children

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10) Improvement

The percentage of children 2

years of age who received one

blood lead test (capillary or

57.2%^

62.3%





venous) on or before their second

birthday.

PIP #2--Childhood Immunizations

The percentage of children who

received the recommended

vaccinations based on the

62.8%^

67.6%





Childhood Immunization Status--

Combo 2 (4:3:1:2:3:1) guidelines.

PIP #3--Well-Child Visits

The percentage of children who

had six or more well-child visits with a PCP during their first 15

51.6%^

52.3%





months of life.

PIP #4--Adults' Access to Care

The percentage of members 20

44 years of age who had an ambulatory or preventive care

78.8%

84.3%*





visit.

^ Rates did not include the PeachCare for KidsTM population. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained
improvement. * Designates statistically significant improvement over the prior measurement period (p value < 0.05).

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-8 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Table D-7--Satisfaction-based Performance Improvement Project Outcomes for Peach State Health Plan

PIP #5--Member Satisfaction

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (3/13/095/31/09) (3/12/105/31/10) (3/1/115/31/11) Improvement

1) "Ease of getting appointment with a specialist" (Q26)

71.7%

71.8%





2) "Getting care, tests, or treatments necessary" (Q30)

79.9%

81.1%





3) "Getting information/help from customer service" (Q32)

68.5%

80.8%*





4) "Treated with courtesy and

respect by customer service

86.4%

90.4%





staff" (Q33)

PIP #6--Provider Satisfaction

PIP Study Indicator^

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (8/1/0710/30/07) (11/1/082/28/09) (9/29/0910/27/09) Improvement

1) The percentage of providers

answering "Excellent" or

"Very Good" to Q5-- "Timeliness to answer

15.8%

28.0%*

32.3%

Yes

questions and/or resolve

problems."

2) Percentage of providers

answering "Excellent" or

"Very Good" to Q6--

14.2%

24.1%*

31.0%*

Yes

"Quality of the provider

orientation process."

3) Percentage of providers

answering "Excellent" or

"Very Good" to Q18-- "Health plan takes physician

10.7%

15.2%

24.5%*

Yes

input and recommendations

seriously."

4) Percentage of providers

answering "Excellent" or

"Very Good" to Q34--

12.1%

16.0%

28.8%*

Yes

"Accuracy of claims

processing."

^ Providers were requested to respond if they agreed with the statements regarding the CMO. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained
improvement. * Designates statistically significant improvement over the prior measurement period.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-9 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

WellCare

Table D-8--SFY 2011 Performance Improvement Project Validation Results for WellCare of Georgia, Inc. (N=6 PIPs)

Study Stage

Activity

I. Appropriate Study Topic

Design

II. Clearly Defined, Answerable Study Question(s) III. Clearly Defined Study Indicator(s)

IV. Correctly Identified Study Population

Design Total Implementation

V. Valid Sampling Techniques (if sampling was used)
VI. Accurate/Complete Data Collection

VII. Appropriate Improvement Strategies Implementation Total

VIII. Sufficient Data Analysis and Interpretation

Outcomes

IX. Real Improvement Achieved

X. Sustained Improvement Achieved* Outcomes Total Overall Percentage of Applicable Evaluation Elements Scored Met

Percentage of Applicable Elements

Met

Partially Met

Not Met

100% (32/32)

0% (0/32)

0% (0/32)

100% (12/12)
100% (36/36)
100% (18/18) 100% (98/98)

0% (0/12)
0% (0/36)
0% (0/18) 0% (0/98)

0% (0/12)
0% (0/36)
0% (0/18) 0% (0/98)

100% (30/30)
100% (51/51)
95% (18/19) 99% (99/100)

0% (0/30)
0% (0/51)
5% (1/19) 1% (1/100)

0% (0/30)
0% (0/51)
0% (0/19) 0% (0/100)

98% (52/53)
63% (15/24)
100% (1/1) 87% (68/78)

2%
(1/53)
8%
(2/24)
0%
(0/1) 4% (3/78) 96% (265/276)

0% (0/53)
29% (7/24)
0% (0/1) 9% (7/78)

* Only the Provider Satisfaction PIP had progressed to this phase in the review period and was assessed for sustained improvement.

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-10 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Table D-9--HEDIS-based Performance Improvement Project Outcomes for WellCare of Georgia, Inc.

PIP #1--Lead Screening in Children

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (1/1/0812/31/08) (1/1/0912/31/09) (1/1/1012/31/10) Improvement

The percentage of children

2 years of age who

received one blood lead test (capillary or venous)

65.9%

67.4%





on or before their second

birthday.

PIP #2--Childhood Immunizations

The percentage of children

who received the

recommended vaccinations

based on the Childhood

75.9%

81.0%





Immunization Status--

Combo 2 (4:3:1:2:3:1)

guidelines.

PIP #3--Well-Child Visits

The percentage of children

who had six or more well-

child visits with a PCP

57.4%

57.4%





during their first 15 months

of life.

PIP #4--Adults' Access to Care

The percentage of

members 2044 years of age who had an ambulatory

78.6%

84.7%*





or preventive care visit.

The PIP did not progress to this phase during the review period and could not be assessed for real or sustained improvement. * Designates statistically significant improvement over the prior measurement period (p value < 0.05).

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-11 GA2010-11_EQR_AnnRpt_F2_0811

METHODOLOGY FOR CONDUCTING VALIDATION OF PERFORMANCE IMPROVEMENT PROJECTS

Table D-10--Satisfaction-based Performance Improvement Project Outcomes for WellCare of Georgia, Inc.

PIP #5--Member Satisfaction

PIP Study Indicator

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (2/1/095/31/09) (2/1/105/31/10) (2/1/115/31/11) Improvement

1) The percentage of members

responding with either a "9" or

"10" to Q24--"Using any

number from 0 to 10, where 0 is

the worst personal doctor

72.2%

71.2%





possible and 10 is the best

personal doctor possible, what

number would you use to rate

your child's personal doctor?"

2) The percentage of eligible

members responding with either

"Always" or "Usually" to Q23--

"In the last 6 months, how often did your child's personal doctor

77.1%

78.4%





seem informed and up to date

about the care your child got

from other doctors/providers?"

PIP #6--Provider Satisfaction

PIP Study Indicator^

Baseline Period Remeasurement 1 Remeasurement 2 Sustained (10/1/069/30/07) (10/1/079/30/08) (10/1/089/30/09) Improvement

1) The percentage of providers

answering "Excellent" or "Very

Good" to Q11--"Specialist network has an adequate number

22.2%

19.7%

24.7%



of high quality specialists to

whom I can refer my patients."

2) The percentage of providers

answering "Excellent" or "Very Good" to Q5--"Timeliness to

22.2%

29.6%*

31.3%

Yes

answer and/or resolve problems."

3) The percentage of providers answering "Excellent" or "Very Good" to Q15--"Timeliness of UM's pre-certification process."

22.5%

25.5%

29.3%

Yes

^ Providers were requested to respond if they agreed with the statements regarding the CMO. The PIP did not progress to this phase during the review period and could not be assessed for real or sustained
improvement. * Designates statistically significant improvement over the prior measurement period (p value < 0.05).

SFY 2011 External Quality Review Annual Report State of Georgia

Page D-12 GA2010-11_EQR_AnnRpt_F2_0811