Collect and Analyze Quantitative and Qualitative Data

To conduct a successful evaluation, community health programs will need to identify strategies for gathering appropriate data and evidence. The data collected should align with the evaluation objectives and should seek to answer the evaluation research questions.

Data gathered for program evaluation can be qualitative or quantitative:

• Qualitative data is descriptive data that is often used to capture the context around the outcomes of the program. Qualitative data is important in evaluations of programs that have contextual and external factors that may impact success—such as geography, population characteristics, and relationships among federal, state, local, tribal, and territorial stakeholders. Qualitative data collection strategies that may be used include interviews, focus groups, and qualitative program reviews.
• Quantitative data is numeric data that can be counted to show how much change has occurred as a result of the program. Quantitative data can be collected from surveys, questionnaires, or program tracking materials.

Knowing that there are different kinds of data that can be collected, it is important to consider the data sources used for evaluation. The availability, reliability, and relevance of data can help determine which data sources to use for program evaluation. It is also important to decide how much data you want to collect for the evaluation. Rural program evaluations often use multiple data sources, including one or more of the following:

• Surveys and questionnaires: Surveys and questionnaires gather information from respondents through open- and close-ended questions. Patients, providers or other organizations may participate in surveys about their experience or satisfaction with the program. They can be administered in-person, by telephone, by mail, or electronically through email or web-based programs.
• Pre- and post-program knowledge/attitude test: Documents knowledge gain or changes in attitude as a result of participation in the program.
• Focus groups and interviews: Focus groups collect insight and observational information from a group of people selected for their relevance to an evaluation. Interviews can be structured around a specific theme or issue and allow for more in-depth exploration of the topic. Both methods collect information through speaking with grantees or stakeholder groups to elicit feedback regarding experiences in program implementation and consideration of intended and unintended policy impacts.
• Technical expert panels: Experts can be involved to review policy options, provide feedback, assess pros and cons, and consider intended and unintended impacts based on contextual features of the program.
• Observations and progress tracking: Observation uses standardized procedures to record behaviors, situations, and events that occur over the course of the program. Observation is useful for gathering direct information or to document on-going events.
• Program documentation: Different data sources can report dissemination of program materials, the number of program participants, or improvements in participant health, among other things. These data sources can include outreach logs, electronic health record data, administrative data, or referral forms. Qualitative program data reviews can investigate programmatic data provided by grantees or other stakeholders on outputs and outcomes, among other topics.

Resources to Learn More

Data Collection & Analysis
Website
Informational briefs on different data collection methods and data analysis.
Organization(s): Centers for Disease Control and Prevention

Data Collection for Program Evaluation
Website
Online course that explains how to collect data to successfully evaluate a community health program. It includes interactive exercises, audio clips, and more. Users must create a free account with PH LearnLink to access.
Organization(s): Northwest Center for Public Health Practice, University of Washington

Improving Data Collection across the Health Care System
Document
Identifies steps toward improving data collection, and provides information on understanding opportunities and challenges within the context of current practice.
Organization(s): Agency for Healthcare Research and Quality
Date: 10/2014