Georgia Breast and Cervical Cancer Program Evaluation and Performance Measurement Plan Project Period: 6/30/2019 6/29/2022 Revised: October 2019 Prepared by: Janet Y Shin, Breast and Cervical Cancer Program Evaluator, Office of Health Science, Chronic Disease Prevention Section, Georgia Department of Public Health Table of Contents 1. Introduction................................................................................................................................. 1 1.1 Background ........................................................................................................................... 1 1.2 Plan overview ........................................................................................................................ 1 1.3 Evaluation purpose ................................................................................................................ 1 2. Stakeholders of evaluation results .............................................................................................. 1 3. Program description .................................................................................................................... 3 3.1 Program purpose and priority populations ............................................................................ 3 3.2 Program activities.................................................................................................................. 3 3.3 Program impact ..................................................................................................................... 3 3.4 Logic model .......................................................................................................................... 4 4. Evaluation focus.......................................................................................................................... 5 5. Data collection ............................................................................................................................ 6 6. Analysis and interpretation ......................................................................................................... 8 6.1 Data analysis ......................................................................................................................... 8 6.2 Data interpretation................................................................................................................. 8 6.3 Contribution to collaborating with health systems and communities ................................... 8 7. Dissemination and use of evaluation findings ............................................................................ 9 7.1 Use of findings ...................................................................................................................... 9 7.2 Dissemination of findings ..................................................................................................... 9 7.3 Documenting and monitoring audience feedback and action steps .................................... 10 8. Evaluation Timeline.................................................................................................................. 10 Appendix....................................................................................................................................... 11 1. Introduction 1.1 Background: Breast cancer is the leading cause of cancer among women in Georgia, with almost 8,000 women expected to be diagnosed in 2019. It is also the second leading cause of cancer deaths among Georgia women, with approximately 1,350 expected deaths in 2019 (Cancer Facts & Figures, 2019; ACS, 2019). Black women in Georgia have traditionally had the paradox of having lower breast cancer incidence rates but higher breast cancer mortality rates than White women. However, in recent years, Black women's breast cancer incidence rate has also progressively increased compared to that among White women (Georgia Comprehensive Cancer Registry (GCCR), 2012-2016). Despite widespread use of Pap testing in recent years, 420 women are still being diagnosed, and 135 women are dying from cervical cancer in Georgia each year (GCCR, 2012-2016). Racial disparities in cervical cancer are still apparent, with Black women in Georgia having a significantly higher cervical cancer mortality rate than White women. Early detection of breast and cervical cancer are the keys to survival; however, mammography for breast cancer and Pap testing for cervical cancer are only useful tools when these services are available and accessible among women. Barriers to breast and cervical cancer care and prevention include financial burdens, issues related to health insurance, lack of transportation, cultural/linguistic issues, as well as lack of knowledge and awareness about cancer prevention methods and the benefits of screening. As of 2017, there were approximately 640,000 women under age 65 in Georgia without health insurance, and almost 400,000 of these women had incomes below 200% of the federal poverty level (Small Area Health Insurance Estimates, U.S. Census, 2019). As not all women have access to cancer screening services, health disparities persist, and cancer morbidity and mortality continue to affect the wellbeing of Georgia's population. 1.2 Plan overview: This comprehensive evaluation plan will follow the procedures and standards recommended by the CDC's Framework for Program Evaluation in Public Health. Ms. Janet Shin, an evaluator within the Office of Health Science, Chronic Disease Prevention Section, Georgia Department of Public Health (DPH) staff, will serve as the lead evaluator for the Georgia Breast and Cervical Cancer Program (GBCCP). The evaluator will use a mixed methods approach that involves quantitative and qualitative methodologies. Process and outcome evaluations will be performed. Key evaluation questions are summarized as follows: What are the facilitators and barriers to implementing program strategies and activities as planned? To what extent does the program implement screening, diagnostic services, patient navigation, and evidence-based interventions? Do breast and cervical cancer screening rates change after implementing evidence-based interventions and supportive activities? 1.3 Evaluation purpose: The purposes of the program evaluation and performance measurement are to monitor the program activities; to determine the program effectiveness; to identify areas to improve program implementation; and to promote accountability among program stakeholders. 2. Stakeholders of evaluation results Primary stakeholders for the evaluation include the GBCCP staff, DPH Health Districts and local health department staff, other participating providers (e.g., federally qualified health centers, or FQHCs), and the Centers for Disease Control and Prevention (CDC) (Table 1). The GBCCP evaluator will collaborate with these stakeholders throughout the project duration to ensure that the program takes a participatory approach in planning and implementing the evaluation activities. 1 Table 1. Stakeholder assessment and engagement plan Stakeholder Name Role of Stakeholder Priority Areas for Evaluation Centers for Disease Control and Prevention (CDC) Georgia Department of Public Health (DPH) Georgia Breast and Cervical Cancer Program (GBCCP) Monitor program deliverables, requirements and performance measures Ensure program success through monitoring of program goals, objectives, funding, reports and data Provide technical assistance and support for evaluation plan implementation; assess program monitoring and evaluation performance objectives; summarize, document, and disseminate evaluation results Guide evaluation design and implementation; use evaluation results to inform program planning and quality improvement DPH Health Districts and county health departments Perform screening, follow-up diagnostic evaluation, case management and evidencebased interventions (EBIs) Collect and provide data; use evaluation results to inform program planning and quality improvement DPH, Chronic Disease Prevention Section, Office of Health Science Collect, analyze, report and evaluate program data Develop and implement evaluation plan; provide data-driven recommendations; summarize, document, and disseminate evaluation results DPH, Related Chronic Collaborate with the GBCCP Disease Programs to streamline chronic disease prevention efforts Use evaluation results to implement and enhance performance of respective program DPH Patient Navigation Program (PNP) Implement navigation, Collect the PNP data; use evaluation results community outreach, EBIs and to inform PNP planning and quality supportive activities improvement Participating providers Perform screening, follow-up e.g., FQHCs diagnostic evaluation, case management and EBIs Collect and provide data; use evaluation results to inform program planning and quality improvement Women's Health Medicaid Program Provide treatment fees for the GBCCP eligible women diagnosed with cancer Collect data Georgia Cancer Control Consortium Implement statewide cancer plan Disseminate evaluation results Women receiving the Receive the GBCCP services Provide data GBCCP services 2 3. Program description 3.1 Program purpose and priority populations: The purpose of the GBCCP is to offer timely and appropriate breast and cervical cancer screening and diagnostic services to uninsured or under-insured women in Georgia at or below 200% of the federal poverty level. The program has special focus on priority population groups, including women of all races and ethnicities without other sources for cancer screening services; women in priority age groups (i.e., age 40-64 for breast cancer screening and 21-64 for cervical cancer screening); women who have been rarely or never screened; and, those who are likely to access healthcare services through local health departments, FQHCs, or other participating health systems. By concentrating efforts to reach our priority populations, the GBCCP aims to reduce health disparities and reduce cancer morbidity and mortality rates in Georgia. 3.2 Program activities: Focus of this program is to provide high quality breast and cervical cancer screening and diagnostic services to eligible women. The GBCCP implements activities related to three primary strategies (environmental approaches, community-clinical linkages and health systems changes) and four cross-cutting strategies (program collaboration, external partnerships, cancer data/surveillance and program monitoring and evaluation). The GBCCP works closely with Albany Area Primary Health Care (AAPHC) to implement evidence-based interventions (EBIs) and health systems change activities to increase clinic-level breast and cervical screening rates at East Albany Medical Center (EAMC). The program implements community-clinical linkage strategies that link women to clinical services and environmental approaches that promote wellness policy at worksites. Through collaboration with the Georgia Center for Oncology Research and Education (GA CORE), women at high risk for the hereditary breast and ovarian cancer genes are screened in the DPH health districts. The Patient Navigators (PNs) provide population-based community education on cancer and facilitate access to receive screening and diagnostic services offered through the GBCCP. Navigators implement EBIs, including client reminders, group education, one-on-one education, reduction of structural barriers, and small media. Women with positive cancer diagnoses are enrolled into the Women's Health Medicaid Program and referred to treatment services and other programs for additional support. Assessing patients for tobacco use and referring those who smoke to the Georgia Tobacco Quit Line is an ongoing activity of the program. The GBCCP provides breast and cervical cancer education and training to statewide public health providers. 3.3 Program impact: In fiscal year (FY) 2018, thirty-six percent (36%) of Pap tests were provided to women rarely or never screened for cervical cancer. Ninety-six percent (96%) of clients with abnormal breast cancer screening results completed follow-up services, and ninety-eight percent (98%) of women diagnosed with breast cancer initiated treatment. Eighty-eight percent (88%) of women with abnormal Pap tests completed follow-up services, and seventy-nine percent (79%) of women with diagnosis of HSIL, CIN2, CIN3, CIS, or invasive cervical carcinoma started treatment. 3.4 Logic model: The GBCCP logic model shows what the GBCCP plans to accomplish, and how program inputs, strategies and activities relate to anticipated outputs and outcomes (Figure 1). 3 INPUTS CDC DPH Public Health Districts & Counties GBCCP Providers e.g. FQHCs GA CORE Women receiving GBCCP services Stable, experienced, and effective program management and leadership Figure 1. Georgia Breast and Cervical Cancer Program Logic Model STRATEGIES AND ACTIVITIES Strategy 1: Program Collaboration Strategy 2: External Partnerships Strategy 3: Cancer Data and Surveillance Strategy 4: Environmental Approach Partner with employers to develop/enhance wellness policy Strategy 5: Community-Clinical Linkage Implement patient navigation and evidencebased interventions (EBIs) to facilitate access to B & C cancer screening and diagnostic services OUTPUTS No. of cancer programs leadership team meetings held No. of partners developed; no. of partners maintained High quality cancer data measured and used Short-term Established health system and community partnerships to increase B & C cancer screening No. of written wellness policies developed/enhanced No. of staff and providers recruited and retained Retention of staff and providers Provide genomic screening by using the Breast Cancer Genetics Referral Screening Tool (B-RSTTM) Strategy 6: Health Systems Change Provide timely and appropriate B & C cancer screening, diagnostic follow-up, and treatment referral services No. of clients navigated No. of clients screened by using B-RSTTM No. of B & C cancer screening & diagnostic services provided Increased access to B & C cancer screening and diagnostic services among priority populations Implement health system changes and EBIs e.g. client reminders, provider assessment and feedback, reducing structural barriers Conduct assessment of partner health systems, including clinic-level B & C cancer screening rates and implementation of EBIs Implement provider training and quality improvement No. and types of EBIs implemented High quality clinic-level data collected and used No. of training sessions, no. of providers trained Improved tumor detection, improved provider confidence in performing B & C cancer examinations 4 OUTCOMES Intermediate Increased appropriate B & C cancer screening, rescreening and surveillance among priority populations Increased timely and appropriate diagnostic followup and cancer treatment initiation among priority populations Long-term Reduced B & C cancer morbidity and mortality Reduced disparities in B & C cancer morbidity and mortality Strategy 7: Program Monitoring and Evaluation 4. Evaluation focus Both process and outcome evaluations will be conducted. Key process evaluation questions include: 1. What are the facilitators and barriers to implementing program strategies and activities as planned? 2. What are grantees' training and technical assistance needs? 3. To what extent do providers perform cancer screening and diagnostic services? a. How many women receive cancer screening and diagnostic services? 4. To what extent do navigators and/or program staff perform patient navigation, EBIs, and supportive activities to increase cancer screening? a. Which EBIs and supportive activities does each provider site (i.e., Public Health District, partner health system) implement? b. How many women are served through patient navigation? c. How many women receive client reminders and recalls? d. Which structural barriers to cancer care are identified and reduced? For each type of barrier, how many cases are identified and reduced? e. How many providers are given a provider assessment and feedback by their health system? Key outcome evaluation questions are as follows: 5. Is the GBCCP meeting target values of clinical quality indicators? a. Does the GBCCP reach the priority population for cancer screening? b. What percentage of clients with abnormal screening results complete diagnostic follow-up? c. What percentage of clients with abnormal screening results receive timely diagnostic follow-up? d. What percentage of clients diagnosed with cancer initiate cancer treatment? e. What percentage of clients diagnosed with cancer initiate timely cancer treatment? 6. What percentage of patients receiving navigation for diagnostic follow-up complete diagnostic testing? 7. Do clinic-level screening rates change after implementing EBIs and supportive activities? These evaluation questions were selected and prioritized based on programmatic needs, selected evaluation purpose, stakeholder interests and feasibility. The GBCCP evaluator will collaborate with program stakeholders to assess whether priorities and feasibility issues hold for these focused evaluation activities and refine these evaluation questions during the project duration. 5 5. Data collection A mixed-methods approach, including quantitative and qualitative methodologies, will be used. Data collection plan is summarized in Tables 2 and 3. More detailed data collection plan and data management plan are included in Appendix. Table 2. Summary of data collection plan for process evaluation Evaluation Question Indicator Performance Measure Method Data Source Responsibility What are the facilitators and barriers to implementing program strategies and activities as planned? Facilitators and barriers in program implementation Qualitative Meeting notes, grantee reports, site visit forms Program Director (PD), Data Management/Quality Assurance (DMQA) team, Providers* What are grantees' training and Training and technical assistance needs technical assistance needs? Qualitative Meeting notes, site PD, DMQA team visit forms, survey To what extent do providers perform cancer screening and diagnostic services? Implementation of screening and diagnostic services No. and % of breast and cervical screening and diagnostic services provided Quantitative Patient-level clinical DMQA team, data (Minimum Data Providers* Elements, or MDEs) To what extent do navigators and/or program staff perform patient navigation, EBIs, and supportive activities? Implementation of patient navigation, EBIs and supportive activities No. and types of EBIs implemented; no. Quantitative Clinic-level data, and types of barriers identified and Qualitative Patient Navigation reduced; no. of women served through (PN) program data, patient navigation, no. of women survey receiving client reminders/recalls; no. of provider assessment and feedback performed; successes and challenges in implementing priority EBIs within health system clinics PN team, Providers*, Program evaluator *Providers include the DPH health districts, local health departments and other providers funded by the GBCCP (e.g., FQHCs, other health systems). 6 Table 3. Summary of data collection plan for outcome evaluation Evaluation Question Indicator Performance Measure Method Data Source Responsibility Is the GBCCP meeting target values of clinical quality indicators? Appropriate B & % of initial Pap tests provided to women rarely or never screened Quantitative Patient- C cancer screening for cervical cancer (Goal: 20%); % of screening mammograms level among priority provided to women 50 years (Goal: 75%); % of abnormal clinical populations; breast cancer screening results with complete follow-up (Goal: data timely and 90%); % of abnormal cervical cancer screening results with (MDEs) appropriate complete follow-up (Goal: 90%); % of abnormal breast diagnostic follow- screening results with time from screening test result to final up and cancer diagnosis >60 days (Goal: 25%); % of abnormal cervical treatment referral screening results with time from screening to final diagnosis >90 among priority days (Goal: 25%); % of final diagnosis of breast cancer where populations treatment has been started (Goal: 90%); % of final diagnosis of HSIL, CIN2, CIN3/CIS, or invasive cervical cancer where treatment has been started (Goal: 90%); % of women diagnosed with breast cancer with time from date of diagnosis to treatment started >60 days (Goal: 20%); % of women diagnosed with premalignant high-grade cervical lesions with time from date of diagnosis to treatment started >90 days (Goal: 20%); % of women diagnosed with invasive cervical cancer with time from date of diagnosis to treatment started >60 days (Goal: 20%) Providers*, DMQA team What percentage of patients receiving navigation for diagnostic follow-up complete diagnostic testing? Appropriate diagnostic followup among priority populations % of patients receiving navigation for diagnostic follow-up complete diagnostic testing Quantitative PN data PN team Do clinic-level screening rates change after implementing EBIs and supportive activities? Appropriate B & Clinic-level breast cancer screening rate, clinic-level cervical C cancer screening cancer screening rate Quantitative Clinic- Health system level data staff, Program evaluator *Providers include the DPH health districts, local health departments and other providers funded by the GBCCP (e.g., FQHCs, other health systems). 7 6. Analysis and interpretation 6.1 Data analysis: The GBCCP evaluator will compile, clean, code, and analyze data from multiple data sources as described in the 5. Data collection section. Both quantitative and qualitative data analysis will be performed. MDE data will be exported from Microsoft Access into SAS (Version 9.4). Survey data will be exported into SAS to conduct the descriptive data analysis, including frequencies and percentages, and chi-square tests. Key outcome variables will be stratified by demographics, such as age, race/ethnicity and region. Pre- and post-test survey data will be analyzed by performing descriptive data analysis, t-tests and McNemar's tests. Rates related to breast and cervical cancer screening, incidence and mortality will be calculated by following the CDC standards. Qualitative data, including responses to open-ended questions in survey data and interview data, will be analyzed by performing thematic analysis. The evaluator will create a codebook, identify codes based on the qualitative responses, and assess common themes. 6.2 Data interpretation: Upon completion of preliminary data analysis, the GBCCP evaluator will present and discuss the initial evaluation findings with the GBCCP staff to interpret the results and apply context to analysis of evidence gathered. Involving relevant stakeholders in data interpretation process will facilitate the program staff to draw appropriate, meaningful and databased conclusions and ensure credibility and acceptability of evaluation findings. Evaluation findings will be interpreted by considering the programmatic goals, evaluation goals, social and political context of the program and needs of program stakeholders. 6.3 Contribution to collaborating with health systems and communities: Through triangulation of multiple data sources, the evaluator will summarize activities completed by the program staff, and highlight the program progress, successes, challenges, outcomes, and lessons learned. Evaluation findings on facilitators and challenges of implementing strategies and activities related to health systems changes (i.e., screening and patient navigation) and communityclinical linkages in Georgia will enhance our understanding of the advantages and challenges of working collaboratively with health systems and communities to promote breast and cervical cancer screening. 8 7. Dissemination and use of evaluation findings 7.1 Use of findings: The GBCCP evaluator will collaborate with the GBCCP staff, DPH Chronic Disease Prevention Section leadership team, and other stakeholders, including Health Districts and local health departments, to ensure the use of evaluation findings for continuous quality improvement. The evaluator will work collaboratively with the program staff to identify targeted recommendations and action steps and make data-based decisions, so that responsible staff can implement programmatic changes to enhance program quality, effectiveness and efficiency. The CDC Project Consultant and Evaluation Technical Advisor will have access to evaluation findings and participate in consensus building exercises and planning discussion if major programmatic changes are recommended. 7.2 Dissemination of findings: Evaluation findings will be disseminated to program stakeholders through various channels, such as staff meetings, statewide and national conferences, emails, the DPH website, conference calls, and webinars. Evaluation reports that include evaluation results, success stories about program strategies, challenges, and lessons learned will be disseminated to program staff and stakeholders, including the CDC. Program progresses and challenges will be communicated with the CDC Project Consultant during quarterly technical assistance calls. The GBCCP team will present the evaluation findings to other state NBCCEDPs and local, state, and national level stakeholders through webinars and conference calls. This comprehensive program evaluation and performance measurement will contribute to developing an evidence base in cancer care and prevention. Throughout the project duration, the GBCCP evaluator will submit abstracts to academic and professional conferences about evaluation approach and findings. The audience, format and channel of dissemination, and responsible staff involved in dissemination are described in Table 4. Table 4. Dissemination plan Audience GBCCP staff Format and Channel Monthly in person updates on data collection and preliminary findings CDC Program Consultant and evaluation staff DPH Health Districts and other participating providers In person PowerPoint presentation of evaluation findings Email evaluation report upon completion Email evaluation report upon completion In person PowerPoint presentation of evaluation findings PowerPoint presentation of evaluation Responsibility Program data manager, Program evaluator Program evaluator Program evaluator Program evaluator, Program director Program evaluator Program evaluator 9 DPH Chronic Disease Prevention Section leadership and relevant program staff Public health professionals Program stakeholders and general public findings via webinar and teleconference Email evaluation report upon completion Program evaluator Email evaluation report upon completion Program evaluator Oral and/or poster presentation at public health conference(s) upon acceptance of abstract Upload evaluation report on DPH Website Program evaluator Program evaluator 7.3 Documenting and monitoring audience feedback and action steps: Feedback from grantees, requests for technical assistance, and action steps will be documented by using site visit forms. By working closely with program staff, the GBCCP evaluator will compile and monitor audience feedback and action steps for continuous quality improvement. 8. Evaluation timeline Timeline of evaluation activities that will be performed during this project period is outlined in Table 5. Table 5. Timeline for evaluation activities Time frame Monthly Tasks: July 2019 June 2022 Quarterly Tasks: July 2019 June 2022 Annual Tasks: July 2019 June 2022 1st Quarter: July September 2nd Quarter: October December 3rd Quarter: January March 4th Quarter: April June Evaluation Activities Collect MDE data, PN data and success stories; document meeting notes; perform monthly data review; each program staff report on their progress and barriers/facilitators to implementation at monthly team meeting Collect data and quarterly reports submitted from providers; analyze/synthesize data and quarterly reports Collect feedback/action steps data by monitoring site visit forms Review workplans submitted from providers; develop evaluation report; collect and submit annual clinic records to CDC Finalize and disseminate evaluation report to CDC and other stakeholders; Submit MDE data to CDC; collect and analyze annual update and training meeting evaluation survey data; disseminate meeting evaluation report to program staff and use findings to improve future meeting/training Collect and submit annual clinic-level screening data to CDC; develop/submit annual progress report/continuing application to CDC Submit MDE data to CDC 10 Appendix Table 6. Data collection plan in detail Indicator/Performance Measure Data Source Assessment Frequency Responsibility Facilitators and barriers in program implementation Meeting notes, grantee Quarterly reports, site visit forms annually Program Director (PD), Data Management/Quality Assurance (DMQA) team, Providers* Training needs for staff/providers*; technical assistance needs for staff/providers* Meeting notes, grantee Quarterly PD, DMQA team, reports, site visit forms annually Providers* Number of clients receiving client reminders and recalls for mammography and Pap test; % of Patient Navigation patients receiving navigation for diagnostic follow-up complete diagnostic testing; number and % Program (PNP) data, of breast/cervical screening completed among navigated women; type and number of reduced CCL survey, MDE, barriers to cancer care; number of participants of group/one-on-one education about breast/cervical CCL tracking data cancer; number of clients referred to BCCP through community clinical linkage (CCL) activities, no. of clients completing breast/cervical screening through community-based referrals Monthly annually Patient Navigators (PNs), Providers*, Program Evaluator Clinic-level breast/cervical cancer screening rates; health system (HS) and clinic characteristics; demographics; implementation of EBIs, patient navigation, and supportive community clinical linkages activities Clinic data Baseline HS staff, Program annually Evaluator Current HS environment; intervention needs; intervention selected; resources and barriers in program implementation HS EBI implementation plan, HS assessment interview, HS data review Baseline HS staff, Program Evaluator % of navigated women with improved knowledge/attitude/satisfaction about breast/cervical cancer PNP satisfaction survey TBD screening TBD 11 Number of staff/providers recruited and retained Performance evaluation Annually PD % of initial Pap tests provided to women rarely or never screened for cervical cancer (Goal: Patient-level clinical 20%); % of screening mammograms provided to women 50 years (Goal: 75%); Number and % data (Minimum Data of breast and cervical screening and re-screening provided Elements, or MDEs) Monthly Providers* bi-annually % of abnormal breast screening results with complete follow-up (Goal: 90%); % of abnormal cervical screening results with complete follow-up (Goal: 90%) % of abnormal breast screening results with time from screening test result to final diagnosis >60 days (Goal: 25%); % of abnormal cervical screening results with time from screening to final diagnosis >90 days (Goal: 25%) % of final diagnosis of breast cancer where treatment has been started (Goal: 90%); % of final diagnosis of HSIL, CIN2, CIN3/CIS, or invasive cervical cancer where treatment has been started (Goal: 90%) % of women diagnosed with breast cancer with time from date of diagnosis to treatment started >60 days (Goal: 20%); % of women diagnosed with premalignant high-grade cervical lesions with time from date of diagnosis to treatment started >90 days (Goal: 20%); % of women diagnosed with invasive cervical cancer with time from date of diagnosis to treatment started >60 days (Goal: 20%) Number of breast lesions detected; number of false detections; % sensitivity; % specificity (i.e., Positive Predictive Value, or PPV); % of exam thoroughness (i.e., area coverage); % of training participants with improved confidence in performing Clinical Breast Exams (CBEs) CBE simulator training Baseline, MammaCare data during & Foundation staff post-training Successes and lessons learned from implementing community clinical linkage strategies CCL survey Baseline Providers*, Program annually Evaluator Stories about navigated patients who completed plan of cancer care PNP success stories TBD PNs GBCCP data accuracy rate (Goal: 98%), timely GBCCP data submission rate (Goal: 75%) Patient-level clinical Monthly Program Data data (MDEs) bi-annually Manager 12 % of employees self-reporting receipt of breast/cervical cancer screening based on current Employee survey guidelines; % of employees aware of workplace's written breast/cervical cancer screening policy; types of breast and/or cervical cancer screening policies at worksites Baseline Program Evaluator annually Number of written breast/cervical cancer screening policy developed/enhanced, successes and challenges in developing/enhancing breast/cervical cancer screening policy at worksites Grantee reports Annually Providers*, Program Evaluator % of women self-reporting receipt of breast/cervical cancer screening based on current guidelines Behavioral Risk Factor Annually Surveillance System data Program Epidemiologist Breast and cervical cancer incidence and mortality among targeted populations; breast and cervical Cancer registry, cancer Annually Program cancer incidence and mortality by race/ethnicity and region death clearance data Epidemiologist *Providers include the DPH health districts and contractor providers funded by the GBCCP (e.g., FQHCs, other health systems). Data sources, data standards and plans for storage, access, archival and preservation are summarized in Table 7. All released data will have accompanying data dictionary and appropriate documentation that describes the data collection method and potential limitations for usage of the data. For public use, de-identified datasets, data dictionary and relevant documentation will be saved in the DPH file server to provide access to the data. 13 Table 7. Data Management Plan Data Sources Standards Patient-level clinical data (MDEs) Patient Navigation Program (PNP) data, CCL tracking data, pre- and post-test survey for Women's Health Exam (WHE) refresher trainings, health system assessment interview MDE definition and data quality indicators Data dictionary Clinic data Clinic data dictionary Storage Access Archival and Preservation Plan Stored in the DPH file server in compliance with HIPAA regulations. Stored in the DPH file server in compliance with HIPAA regulations All survey and interview data will have participant names and contact information removed, with a unique identifier allowing linkage if the need arises while maintaining confidentiality. Stored in the DPH file server and the Breast and Cervical Baseline and Annual Reporting System (B&CBARS) in compliance with HIPAA regulations. The GBCCP database can be accessed by the GBCCP state staff only. Aggregated data reports are shared with the program staff at all levels. Clinic data can be accessed by the GBCCP state staff and responsible health system staff only. Electronic records: stored in the DPH file server indefinitely Paper records: stored for 5 years (3 years in the state office and 2 years in the state retention center) Stored in the DPH file server indefinitely Data access The GBCCP identifiable patient-level clinical data are not designated for public use and can be accessed by the GBCCP state staff with appropriate access rights only. De-identified datasets are provided when data requests are received through and approved by the DPH data request system. Aggregated screening and diagnosis data reports for program management, performance monitory funding tracking are shared with the program staff at all levels and public upon request. 14 The data are stored in the GBCCP database in the DPH file server in compliance with HIPAA regulations and the DPH information security policies. The GBCCP state staff are required to attend refresher information security training and follow all protocols for receiving, storing, editing and sharing data. Data archiving and long-term preservation All identifiable patient-level data collected from the providers electronically or in paper forms are entered in the GBCCP database. The database, with its documentations is stored in the DPH file server indefinitely. The GBCCP data management team supports the data through changing technologies, new media, and data formats. 15 Figure 2. 16