Georgia Breast and Cervical Cancer Program evaluation and performance measurement plan, 2019-2022

Georgia Breast and Cervical Cancer Program Evaluation and Performance Measurement Plan
Project Period: 6/30/2019 6/29/2022 Revised: October 2019
Prepared by: Janet Y Shin, Breast and Cervical Cancer Program Evaluator, Office of Health Science, Chronic Disease Prevention Section, Georgia Department of Public Health

Table of Contents
1. Introduction................................................................................................................................. 1 1.1 Background ........................................................................................................................... 1 1.2 Plan overview ........................................................................................................................ 1 1.3 Evaluation purpose ................................................................................................................ 1
2. Stakeholders of evaluation results .............................................................................................. 1 3. Program description .................................................................................................................... 3
3.1 Program purpose and priority populations ............................................................................ 3 3.2 Program activities.................................................................................................................. 3 3.3 Program impact ..................................................................................................................... 3 3.4 Logic model .......................................................................................................................... 4 4. Evaluation focus.......................................................................................................................... 5 5. Data collection ............................................................................................................................ 6 6. Analysis and interpretation ......................................................................................................... 8 6.1 Data analysis ......................................................................................................................... 8 6.2 Data interpretation................................................................................................................. 8 6.3 Contribution to collaborating with health systems and communities ................................... 8 7. Dissemination and use of evaluation findings ............................................................................ 9 7.1 Use of findings ...................................................................................................................... 9 7.2 Dissemination of findings ..................................................................................................... 9 7.3 Documenting and monitoring audience feedback and action steps .................................... 10 8. Evaluation Timeline.................................................................................................................. 10 Appendix....................................................................................................................................... 11

1. Introduction
1.1 Background: Breast cancer is the leading cause of cancer among women in Georgia, with almost 8,000 women expected to be diagnosed in 2019. It is also the second leading cause of cancer deaths among Georgia women, with approximately 1,350 expected deaths in 2019 (Cancer Facts & Figures, 2019; ACS, 2019). Black women in Georgia have traditionally had the paradox of having lower breast cancer incidence rates but higher breast cancer mortality rates than White women. However, in recent years, Black women's breast cancer incidence rate has also progressively increased compared to that among White women (Georgia Comprehensive Cancer Registry (GCCR), 2012-2016). Despite widespread use of Pap testing in recent years, 420 women are still being diagnosed, and 135 women are dying from cervical cancer in Georgia each year (GCCR, 2012-2016). Racial disparities in cervical cancer are still apparent, with Black women in Georgia having a significantly higher cervical cancer mortality rate than White women. Early detection of breast and cervical cancer are the keys to survival; however, mammography for breast cancer and Pap testing for cervical cancer are only useful tools when these services are available and accessible among women. Barriers to breast and cervical cancer care and prevention include financial burdens, issues related to health insurance, lack of transportation, cultural/linguistic issues, as well as lack of knowledge and awareness about cancer prevention methods and the benefits of screening. As of 2017, there were approximately 640,000 women under age 65 in Georgia without health insurance, and almost 400,000 of these women had incomes below 200% of the federal poverty level (Small Area Health Insurance Estimates, U.S. Census, 2019). As not all women have access to cancer screening services, health disparities persist, and cancer morbidity and mortality continue to affect the wellbeing of Georgia's population.
1.2 Plan overview: This comprehensive evaluation plan will follow the procedures and standards recommended by the CDC's Framework for Program Evaluation in Public Health. Ms. Janet Shin, an evaluator within the Office of Health Science, Chronic Disease Prevention Section, Georgia Department of Public Health (DPH) staff, will serve as the lead evaluator for the Georgia Breast and Cervical Cancer Program (GBCCP). The evaluator will use a mixed methods approach that involves quantitative and qualitative methodologies. Process and outcome evaluations will be performed. Key evaluation questions are summarized as follows: What are the facilitators and barriers to implementing program strategies and activities as planned? To what extent does the program implement screening, diagnostic services, patient navigation, and evidence-based interventions? Do breast and cervical cancer screening rates change after implementing evidence-based interventions and supportive activities?
1.3 Evaluation purpose: The purposes of the program evaluation and performance measurement are to monitor the program activities; to determine the program effectiveness; to identify areas to improve program implementation; and to promote accountability among program stakeholders.
2. Stakeholders of evaluation results
Primary stakeholders for the evaluation include the GBCCP staff, DPH Health Districts and local health department staff, other participating providers (e.g., federally qualified health centers, or FQHCs), and the Centers for Disease Control and Prevention (CDC) (Table 1). The GBCCP evaluator will collaborate with these stakeholders throughout the project duration to ensure that the program takes a participatory approach in planning and implementing the evaluation activities.
1

Table 1. Stakeholder assessment and engagement plan

Stakeholder Name

Role of Stakeholder

Priority Areas for Evaluation

Centers for Disease Control and Prevention (CDC)
Georgia Department of Public Health (DPH) Georgia Breast and Cervical Cancer Program (GBCCP)

Monitor program deliverables, requirements and performance measures
Ensure program success through monitoring of program goals, objectives, funding, reports and data

Provide technical assistance and support for evaluation plan implementation; assess program monitoring and evaluation performance objectives; summarize, document, and disseminate evaluation results Guide evaluation design and implementation; use evaluation results to inform program planning and quality improvement

DPH Health Districts and county health departments

Perform screening, follow-up diagnostic evaluation, case management and evidencebased interventions (EBIs)

Collect and provide data; use evaluation results to inform program planning and quality improvement

DPH, Chronic Disease Prevention Section, Office of Health Science

Collect, analyze, report and evaluate program data

Develop and implement evaluation plan; provide data-driven recommendations; summarize, document, and disseminate evaluation results

DPH, Related Chronic Collaborate with the GBCCP

Disease Programs

to streamline chronic disease

prevention efforts

Use evaluation results to implement and enhance performance of respective program

DPH Patient Navigation Program (PNP)

Implement navigation,

Collect the PNP data; use evaluation results

community outreach, EBIs and to inform PNP planning and quality

supportive activities

improvement

Participating providers Perform screening, follow-up

e.g., FQHCs

diagnostic evaluation, case

management and EBIs

Collect and provide data; use evaluation results to inform program planning and quality improvement

Women's Health Medicaid Program

Provide treatment fees for the GBCCP eligible women diagnosed with cancer

Collect data

Georgia Cancer Control Consortium

Implement statewide cancer plan

Disseminate evaluation results

Women receiving the Receive the GBCCP services Provide data GBCCP services

2

3. Program description
3.1 Program purpose and priority populations: The purpose of the GBCCP is to offer timely and appropriate breast and cervical cancer screening and diagnostic services to uninsured or under-insured women in Georgia at or below 200% of the federal poverty level. The program has special focus on priority population groups, including women of all races and ethnicities without other sources for cancer screening services; women in priority age groups (i.e., age 40-64 for breast cancer screening and 21-64 for cervical cancer screening); women who have been rarely or never screened; and, those who are likely to access healthcare services through local health departments, FQHCs, or other participating health systems. By concentrating efforts to reach our priority populations, the GBCCP aims to reduce health disparities and reduce cancer morbidity and mortality rates in Georgia.
3.2 Program activities: Focus of this program is to provide high quality breast and cervical cancer screening and diagnostic services to eligible women. The GBCCP implements activities related to three primary strategies (environmental approaches, community-clinical linkages and health systems changes) and four cross-cutting strategies (program collaboration, external partnerships, cancer data/surveillance and program monitoring and evaluation). The GBCCP works closely with Albany Area Primary Health Care (AAPHC) to implement evidence-based interventions (EBIs) and health systems change activities to increase clinic-level breast and cervical screening rates at East Albany Medical Center (EAMC). The program implements community-clinical linkage strategies that link women to clinical services and environmental approaches that promote wellness policy at worksites. Through collaboration with the Georgia Center for Oncology Research and Education (GA CORE), women at high risk for the hereditary breast and ovarian cancer genes are screened in the DPH health districts. The Patient Navigators (PNs) provide population-based community education on cancer and facilitate access to receive screening and diagnostic services offered through the GBCCP. Navigators implement EBIs, including client reminders, group education, one-on-one education, reduction of structural barriers, and small media. Women with positive cancer diagnoses are enrolled into the Women's Health Medicaid Program and referred to treatment services and other programs for additional support. Assessing patients for tobacco use and referring those who smoke to the Georgia Tobacco Quit Line is an ongoing activity of the program. The GBCCP provides breast and cervical cancer education and training to statewide public health providers.
3.3 Program impact: In fiscal year (FY) 2018, thirty-six percent (36%) of Pap tests were provided to women rarely or never screened for cervical cancer. Ninety-six percent (96%) of clients with abnormal breast cancer screening results completed follow-up services, and ninety-eight percent (98%) of women diagnosed with breast cancer initiated treatment. Eighty-eight percent (88%) of women with abnormal Pap tests completed follow-up services, and seventy-nine percent (79%) of women with diagnosis of HSIL, CIN2, CIN3, CIS, or invasive cervical carcinoma started treatment.
3.4 Logic model: The GBCCP logic model shows what the GBCCP plans to accomplish, and how program inputs, strategies and activities relate to anticipated outputs and outcomes (Figure 1).
3

INPUTS
CDC
DPH
Public Health Districts
& Counties
GBCCP Providers
e.g.
FQHCs
GA CORE
Women receiving GBCCP services

Stable, experienced, and effective program management and leadership

Figure 1. Georgia Breast and Cervical Cancer Program Logic Model

STRATEGIES AND ACTIVITIES
Strategy 1: Program Collaboration
Strategy 2: External Partnerships
Strategy 3: Cancer Data and Surveillance
Strategy 4: Environmental Approach Partner with employers to develop/enhance wellness policy
Strategy 5: Community-Clinical Linkage Implement patient navigation and evidencebased interventions (EBIs) to facilitate access to B & C cancer screening and diagnostic services

OUTPUTS
No. of cancer programs leadership team meetings held
No. of partners developed; no. of partners maintained
High quality cancer data measured and used

Short-term
Established health system and community partnerships to increase B & C cancer screening

No. of written wellness policies developed/enhanced
No. of staff and providers recruited and retained

Retention of staff and providers

Provide genomic screening by using the Breast Cancer Genetics Referral Screening Tool (B-RSTTM)
Strategy 6: Health Systems Change Provide timely and appropriate B & C cancer screening, diagnostic follow-up, and treatment referral services

No. of clients navigated
No. of clients screened by using B-RSTTM
No. of B & C cancer screening & diagnostic services provided

Increased access to B & C cancer screening and diagnostic services among priority populations

Implement health system changes and EBIs e.g. client reminders, provider assessment and feedback, reducing structural barriers
Conduct assessment of partner health systems, including clinic-level B & C cancer screening rates and implementation of EBIs
Implement provider training and quality improvement

No. and types of EBIs implemented
High quality clinic-level data collected and used
No. of training sessions, no. of providers trained

Improved tumor detection, improved provider confidence in performing B & C cancer examinations

4 OUTCOMES
Intermediate
Increased appropriate B & C cancer screening, rescreening and surveillance among priority populations
Increased timely and appropriate diagnostic followup and cancer treatment initiation among priority populations

Long-term
Reduced B & C cancer morbidity and mortality Reduced disparities in B & C cancer morbidity and mortality

Strategy 7: Program Monitoring and Evaluation

4. Evaluation focus
Both process and outcome evaluations will be conducted. Key process evaluation questions include:
1. What are the facilitators and barriers to implementing program strategies and activities as planned?
2. What are grantees' training and technical assistance needs? 3. To what extent do providers perform cancer screening and diagnostic services?
a. How many women receive cancer screening and diagnostic services? 4. To what extent do navigators and/or program staff perform patient navigation, EBIs, and
supportive activities to increase cancer screening? a. Which EBIs and supportive activities does each provider site (i.e., Public Health District, partner health system) implement? b. How many women are served through patient navigation? c. How many women receive client reminders and recalls? d. Which structural barriers to cancer care are identified and reduced? For each type of barrier, how many cases are identified and reduced? e. How many providers are given a provider assessment and feedback by their health system?
Key outcome evaluation questions are as follows: 5. Is the GBCCP meeting target values of clinical quality indicators? a. Does the GBCCP reach the priority population for cancer screening? b. What percentage of clients with abnormal screening results complete diagnostic follow-up? c. What percentage of clients with abnormal screening results receive timely diagnostic follow-up? d. What percentage of clients diagnosed with cancer initiate cancer treatment? e. What percentage of clients diagnosed with cancer initiate timely cancer treatment? 6. What percentage of patients receiving navigation for diagnostic follow-up complete diagnostic testing? 7. Do clinic-level screening rates change after implementing EBIs and supportive activities?
These evaluation questions were selected and prioritized based on programmatic needs, selected evaluation purpose, stakeholder interests and feasibility. The GBCCP evaluator will collaborate with program stakeholders to assess whether priorities and feasibility issues hold for these focused evaluation activities and refine these evaluation questions during the project duration.
5

5. Data collection

A mixed-methods approach, including quantitative and qualitative methodologies, will be used. Data collection plan is summarized in Tables 2 and 3. More detailed data collection plan and data management plan are included in Appendix.

Table 2. Summary of data collection plan for process evaluation

Evaluation Question

Indicator

Performance Measure

Method

Data Source

Responsibility

What are the facilitators and barriers to implementing program strategies and activities as planned?

Facilitators and barriers in program implementation

Qualitative

Meeting notes, grantee reports, site visit forms

Program Director (PD), Data Management/Quality Assurance (DMQA) team, Providers*

What are grantees' training and Training and technical assistance needs technical assistance needs?

Qualitative Meeting notes, site PD, DMQA team visit forms, survey

To what extent do providers perform cancer screening and diagnostic services?

Implementation of screening and diagnostic services

No. and % of breast and cervical screening and diagnostic services provided

Quantitative Patient-level clinical DMQA team, data (Minimum Data Providers* Elements, or MDEs)

To what extent do navigators and/or program staff perform patient navigation, EBIs, and supportive activities?

Implementation of patient navigation, EBIs and supportive activities

No. and types of EBIs implemented; no. Quantitative Clinic-level data,

and types of barriers identified and

Qualitative Patient Navigation

reduced; no. of women served through

(PN) program data,

patient navigation, no. of women

survey

receiving client reminders/recalls; no. of

provider assessment and feedback

performed; successes and challenges in

implementing priority EBIs within

health system clinics

PN team, Providers*, Program evaluator

*Providers include the DPH health districts, local health departments and other providers funded by the GBCCP (e.g., FQHCs, other health systems).

6

Table 3. Summary of data collection plan for outcome evaluation

Evaluation Question

Indicator

Performance Measure

Method

Data Source

Responsibility

Is the GBCCP meeting target values of clinical quality indicators?

Appropriate B & % of initial Pap tests provided to women rarely or never screened Quantitative Patient-

C cancer screening for cervical cancer (Goal: 20%); % of screening mammograms

level

among priority provided to women 50 years (Goal: 75%); % of abnormal

clinical

populations;

breast cancer screening results with complete follow-up (Goal:

data

timely and

90%); % of abnormal cervical cancer screening results with

(MDEs)

appropriate

complete follow-up (Goal: 90%); % of abnormal breast

diagnostic follow- screening results with time from screening test result to final

up and cancer

diagnosis >60 days (Goal: 25%); % of abnormal cervical

treatment referral screening results with time from screening to final diagnosis >90

among priority days (Goal: 25%); % of final diagnosis of breast cancer where

populations

treatment has been started (Goal: 90%); % of final diagnosis of

HSIL, CIN2, CIN3/CIS, or invasive cervical cancer where

treatment has been started (Goal: 90%); % of women diagnosed

with breast cancer with time from date of diagnosis to treatment

started >60 days (Goal: 20%); % of women diagnosed with

premalignant high-grade cervical lesions with time from date of

diagnosis to treatment started >90 days (Goal: 20%); % of

women diagnosed with invasive cervical cancer with time from

date of diagnosis to treatment started >60 days (Goal: 20%)

Providers*, DMQA team

What percentage of patients receiving navigation for diagnostic follow-up complete diagnostic testing?

Appropriate diagnostic followup among priority populations

% of patients receiving navigation for diagnostic follow-up complete diagnostic testing

Quantitative PN data PN team

Do clinic-level screening rates change after implementing EBIs and supportive activities?

Appropriate B & Clinic-level breast cancer screening rate, clinic-level cervical C cancer screening cancer screening rate

Quantitative Clinic- Health system level data staff, Program evaluator

*Providers include the DPH health districts, local health departments and other providers funded by the GBCCP (e.g., FQHCs, other health systems).

7

6. Analysis and interpretation
6.1 Data analysis: The GBCCP evaluator will compile, clean, code, and analyze data from multiple data sources as described in the 5. Data collection section. Both quantitative and qualitative data analysis will be performed.
MDE data will be exported from Microsoft Access into SAS (Version 9.4). Survey data will be exported into SAS to conduct the descriptive data analysis, including frequencies and percentages, and chi-square tests. Key outcome variables will be stratified by demographics, such as age, race/ethnicity and region. Pre- and post-test survey data will be analyzed by performing descriptive data analysis, t-tests and McNemar's tests. Rates related to breast and cervical cancer screening, incidence and mortality will be calculated by following the CDC standards.
Qualitative data, including responses to open-ended questions in survey data and interview data, will be analyzed by performing thematic analysis. The evaluator will create a codebook, identify codes based on the qualitative responses, and assess common themes.
6.2 Data interpretation: Upon completion of preliminary data analysis, the GBCCP evaluator will present and discuss the initial evaluation findings with the GBCCP staff to interpret the results and apply context to analysis of evidence gathered. Involving relevant stakeholders in data interpretation process will facilitate the program staff to draw appropriate, meaningful and databased conclusions and ensure credibility and acceptability of evaluation findings. Evaluation findings will be interpreted by considering the programmatic goals, evaluation goals, social and political context of the program and needs of program stakeholders.
6.3 Contribution to collaborating with health systems and communities: Through triangulation of multiple data sources, the evaluator will summarize activities completed by the program staff, and highlight the program progress, successes, challenges, outcomes, and lessons learned. Evaluation findings on facilitators and challenges of implementing strategies and activities related to health systems changes (i.e., screening and patient navigation) and communityclinical linkages in Georgia will enhance our understanding of the advantages and challenges of working collaboratively with health systems and communities to promote breast and cervical cancer screening.
8

7. Dissemination and use of evaluation findings

7.1 Use of findings: The GBCCP evaluator will collaborate with the GBCCP staff, DPH Chronic Disease Prevention Section leadership team, and other stakeholders, including Health Districts and local health departments, to ensure the use of evaluation findings for continuous quality improvement. The evaluator will work collaboratively with the program staff to identify targeted recommendations and action steps and make data-based decisions, so that responsible staff can implement programmatic changes to enhance program quality, effectiveness and efficiency. The CDC Project Consultant and Evaluation Technical Advisor will have access to evaluation findings and participate in consensus building exercises and planning discussion if major programmatic changes are recommended.

7.2 Dissemination of findings: Evaluation findings will be disseminated to program stakeholders through various channels, such as staff meetings, statewide and national conferences, emails, the DPH website, conference calls, and webinars. Evaluation reports that include evaluation results, success stories about program strategies, challenges, and lessons learned will be disseminated to program staff and stakeholders, including the CDC. Program progresses and challenges will be communicated with the CDC Project Consultant during quarterly technical assistance calls. The GBCCP team will present the evaluation findings to other state NBCCEDPs and local, state, and national level stakeholders through webinars and conference calls. This comprehensive program evaluation and performance measurement will contribute to developing an evidence base in cancer care and prevention. Throughout the project duration, the GBCCP evaluator will submit abstracts to academic and professional conferences about evaluation approach and findings. The audience, format and channel of dissemination, and responsible staff involved in dissemination are described in Table 4.

Table 4. Dissemination plan

Audience GBCCP staff

Format and Channel Monthly in person updates on data collection and preliminary findings

CDC Program Consultant and evaluation staff DPH Health Districts and other participating providers

In person PowerPoint presentation of evaluation findings Email evaluation report upon completion Email evaluation report upon completion
In person PowerPoint presentation of evaluation findings PowerPoint presentation of evaluation

Responsibility Program data manager, Program evaluator Program evaluator
Program evaluator Program evaluator, Program director
Program evaluator
Program evaluator

9

DPH Chronic Disease Prevention Section leadership and relevant program staff Public health professionals
Program stakeholders and general public

findings via webinar and teleconference Email evaluation report upon completion Program evaluator Email evaluation report upon completion Program evaluator

Oral and/or poster presentation at public health conference(s) upon acceptance of abstract Upload evaluation report on DPH Website

Program evaluator Program evaluator

7.3 Documenting and monitoring audience feedback and action steps: Feedback from grantees, requests for technical assistance, and action steps will be documented by using site visit forms. By working closely with program staff, the GBCCP evaluator will compile and monitor audience feedback and action steps for continuous quality improvement.

8. Evaluation timeline

Timeline of evaluation activities that will be performed during this project period is outlined in Table 5.

Table 5. Timeline for evaluation activities

Time frame Monthly Tasks: July 2019 June 2022
Quarterly Tasks: July 2019 June 2022 Annual Tasks: July 2019 June 2022 1st Quarter: July September 2nd Quarter: October December
3rd Quarter: January March 4th Quarter: April June

Evaluation Activities Collect MDE data, PN data and success stories; document meeting notes; perform monthly data review; each program staff report on their progress and barriers/facilitators to implementation at monthly team meeting Collect data and quarterly reports submitted from providers; analyze/synthesize data and quarterly reports Collect feedback/action steps data by monitoring site visit forms
Review workplans submitted from providers; develop evaluation report; collect and submit annual clinic records to CDC Finalize and disseminate evaluation report to CDC and other stakeholders; Submit MDE data to CDC; collect and analyze annual update and training meeting evaluation survey data; disseminate meeting evaluation report to program staff and use findings to improve future meeting/training Collect and submit annual clinic-level screening data to CDC; develop/submit annual progress report/continuing application to CDC Submit MDE data to CDC

10

Appendix

Table 6. Data collection plan in detail

Indicator/Performance Measure

Data Source

Assessment Frequency

Responsibility

Facilitators and barriers in program implementation

Meeting notes, grantee Quarterly reports, site visit forms annually

Program Director (PD), Data Management/Quality Assurance (DMQA) team, Providers*

Training needs for staff/providers*; technical assistance needs for staff/providers*

Meeting notes, grantee Quarterly PD, DMQA team, reports, site visit forms annually Providers*

Number of clients receiving client reminders and recalls for mammography and Pap test; % of Patient Navigation

patients receiving navigation for diagnostic follow-up complete diagnostic testing; number and % Program (PNP) data,

of breast/cervical screening completed among navigated women; type and number of reduced

CCL survey, MDE,

barriers to cancer care; number of participants of group/one-on-one education about breast/cervical CCL tracking data

cancer; number of clients referred to BCCP through community clinical linkage (CCL) activities,

no. of clients completing breast/cervical screening through community-based referrals

Monthly annually

Patient Navigators (PNs), Providers*, Program Evaluator

Clinic-level breast/cervical cancer screening rates; health system (HS) and clinic characteristics; demographics; implementation of EBIs, patient navigation, and supportive community clinical linkages activities

Clinic data

Baseline HS staff, Program annually Evaluator

Current HS environment; intervention needs; intervention selected; resources and barriers in program implementation

HS EBI implementation plan, HS assessment interview, HS data review

Baseline

HS staff, Program Evaluator

% of navigated women with improved knowledge/attitude/satisfaction about breast/cervical cancer PNP satisfaction survey TBD screening

TBD

11

Number of staff/providers recruited and retained

Performance evaluation Annually PD

% of initial Pap tests provided to women rarely or never screened for cervical cancer (Goal:

Patient-level clinical

20%); % of screening mammograms provided to women 50 years (Goal: 75%); Number and % data (Minimum Data

of breast and cervical screening and re-screening provided

Elements, or MDEs)

Monthly Providers* bi-annually

% of abnormal breast screening results with complete follow-up (Goal: 90%); % of abnormal cervical screening results with complete follow-up (Goal: 90%)

% of abnormal breast screening results with time from screening test result to final diagnosis >60 days (Goal: 25%); % of abnormal cervical screening results with time from screening to final diagnosis >90 days (Goal: 25%)

% of final diagnosis of breast cancer where treatment has been started (Goal: 90%); % of final diagnosis of HSIL, CIN2, CIN3/CIS, or invasive cervical cancer where treatment has been started (Goal: 90%)

% of women diagnosed with breast cancer with time from date of diagnosis to treatment started >60 days (Goal: 20%); % of women diagnosed with premalignant high-grade cervical lesions with time from date of diagnosis to treatment started >90 days (Goal: 20%); % of women diagnosed with invasive cervical cancer with time from date of diagnosis to treatment started >60 days (Goal: 20%)

Number of breast lesions detected; number of false detections; % sensitivity; % specificity (i.e., Positive Predictive Value, or PPV); % of exam thoroughness (i.e., area coverage); % of training participants with improved confidence in performing Clinical Breast Exams (CBEs)

CBE simulator training Baseline, MammaCare

data

during & Foundation staff

post-training

Successes and lessons learned from implementing community clinical linkage strategies

CCL survey

Baseline Providers*, Program annually Evaluator

Stories about navigated patients who completed plan of cancer care

PNP success stories TBD

PNs

GBCCP data accuracy rate (Goal: 98%), timely GBCCP data submission rate (Goal: 75%)

Patient-level clinical Monthly Program Data

data (MDEs)

bi-annually Manager

12

% of employees self-reporting receipt of breast/cervical cancer screening based on current

Employee survey

guidelines; % of employees aware of workplace's written breast/cervical cancer screening policy;

types of breast and/or cervical cancer screening policies at worksites

Baseline Program Evaluator annually

Number of written breast/cervical cancer screening policy developed/enhanced, successes and challenges in developing/enhancing breast/cervical cancer screening policy at worksites

Grantee reports

Annually Providers*, Program Evaluator

% of women self-reporting receipt of breast/cervical cancer screening based on current guidelines Behavioral Risk Factor Annually Surveillance System data

Program Epidemiologist

Breast and cervical cancer incidence and mortality among targeted populations; breast and cervical Cancer registry, cancer Annually Program

cancer incidence and mortality by race/ethnicity and region

death clearance data

Epidemiologist

*Providers include the DPH health districts and contractor providers funded by the GBCCP (e.g., FQHCs, other health systems).

Data sources, data standards and plans for storage, access, archival and preservation are summarized in Table 7. All released data will have accompanying data dictionary and appropriate documentation that describes the data collection method and potential limitations for usage of the data. For public use, de-identified datasets, data dictionary and relevant documentation will be saved in the DPH file server to provide access to the data.

13

Table 7. Data Management Plan

Data Sources

Standards

Patient-level clinical data (MDEs)
Patient Navigation Program (PNP) data, CCL tracking data, pre- and post-test survey for Women's Health Exam (WHE) refresher trainings, health system assessment interview

MDE definition and data quality indicators Data dictionary

Clinic data

Clinic data dictionary

Storage

Access

Archival and Preservation Plan

Stored in the DPH file server in compliance with HIPAA regulations.
Stored in the DPH file server in compliance with HIPAA regulations All survey and interview data will have participant names and contact information removed, with a unique identifier allowing linkage if the need arises while maintaining confidentiality. Stored in the DPH file server and the Breast and Cervical Baseline and Annual Reporting System (B&CBARS) in compliance with HIPAA regulations.

The GBCCP database can be accessed by the GBCCP state staff only. Aggregated data reports are shared with the program staff at all levels.
Clinic data can be accessed by the GBCCP state staff and responsible health system staff only.

Electronic records: stored in the DPH file server indefinitely
Paper records: stored for 5 years (3 years in the state office and 2 years in the state retention center) Stored in the DPH file server indefinitely

Data access The GBCCP identifiable patient-level clinical data are not designated for public use and can be accessed by the GBCCP state staff with appropriate access rights only. De-identified datasets are provided when data requests are received through and approved by the DPH data request system. Aggregated screening and diagnosis data reports for program management, performance monitory funding tracking are shared with the program staff at all levels and public upon request.

14

The data are stored in the GBCCP database in the DPH file server in compliance with HIPAA regulations and the DPH information security policies. The GBCCP state staff are required to attend refresher information security training and follow all protocols for receiving, storing, editing and sharing data.
Data archiving and long-term preservation All identifiable patient-level data collected from the providers electronically or in paper forms are entered in the GBCCP database. The database, with its documentations is stored in the DPH file server indefinitely. The GBCCP data management team supports the data through changing technologies, new media, and data formats.
15

Figure 2.
16