Evaluation Strategies and Considerations for Rural SDOH Programs

To effectively evaluate social determinants of health (SDOH) programs in rural communities, it is important to select evaluation strategies that capture a wide range of effects. The Rural Community Health Toolkit provides an overview of evaluation planning that can guide rural communities in designing and implementing an evaluation. Rural communities may also benefit from evaluation strategies outlined in the following topic-specific toolkits that focus on issues related to SDOH:

Collecting Process and Outcome Measures

When evaluating rural SDOH programs, it is important to assess both how the program is implemented (process) and what results are achieved (outcomes). Process measures can provide information on how a program performs. Rural communities can also use process measures to assess program planning, partner involvement, and other aspects of program implementation. Outcome measures can provide information on the program's influence and effectiveness, including progress toward its intended goals and impact on population(s) of focus and the community. Combining process and outcome measures can provide a more comprehensive picture of the program's effectiveness, identify opportunities for improvements, and demonstrate the program's impact.

Demonstrating Program Value

Clearly demonstrating a program's impact is essential for ensuring continued support and investment in SDOH initiatives. Programs can demonstrate impact and value by highlighting the financial benefits of the program. Measuring return on investment (ROI), for example, can show funders and partners the advantages of rural SDOH programs. A first step in planning to demonstrate value is to map out how the program will achieve its intended results and outcomes. Rural programs can use tools like logic models or theories of change to describe how the program's resources, activities, and outputs connect to intended outcomes. Using these tools can help program planners identify what to measure and how positive outcomes may contribute to broader benefits, including cost savings. For example, rural programs using cross-sectoral approaches can calculate how partnerships between community-based organizations, social services, and medical systems may contribute to decreased utilization of costly healthcare services, demonstrating tangible financial benefits.

Incorporating Varied Data Types and Sources

Rural SDOH programs are complex, so it is important to use a combination of qualitative and quantitative data during evaluation. Qualitative data can help evaluators investigate the experiences and perspectives of participants and community members and provide depth and context for quantitative data. Qualitative data can also be used in techniques like digital storytelling, which help lift and amplify the voices of community members. Evaluation methods might include surveys, focus groups, interviews, and secondary analysis of other data sources, such as the Census.

Measuring Short-Term, Intermediate, and Long-Term Outcomes

SDOH progress and outcomes can take time, so evaluation should be designed to capture impacts at different stages of program implementation. Communities should plan to measure short-term and intermediate outcomes in addition to long-term outcomes. Using a range of measures can help demonstrate progress toward goals over time, provide data needed to improve or adapt program strategies and activities, and support longer-term strategic planning efforts. When possible, communities should establish a baseline for key measures before the program begins to assess changes over time.

Promoting Community Participation

Rural programs addressing SDOH often emphasize the importance of engaging community members in evaluation. Involving the community helps ground evaluation findings in the lived experiences of rural residents, builds trust, and increases the likelihood of utilizing evaluation results.

Community-based participatory research (CBPR) is a framework in which community members are equal partners in the research and evaluation process. CBPR uses research methods that help researchers build trust within communities and enables community participation in all stages of the research process. This helps ensure the rural community values, heritage, geography, and location are considered in evaluation efforts. CBPR builds upon community assets and helps residents identify strengths and opportunities for action, empowering community members to address local challenges, including SDOH.

Rural SDOH programs may involve community members in data collection and analysis through various community-based participatory research strategies, such as:

Photovoice is a method that asks community members to capture and share images about their communities, environments, and experiences. Photovoice can help amplify the voices of rural residents who are traditionally not included in the research process. For example, as part of the MI-PHOTOS project, researchers from the University of North Carolina at Chapel Hill collaborated with women in rural North Carolina to highlight the resiliency of mothers in rural communities. Using Photovoice, the partnership promoted health among rural mothers by generating fuller understandings of mothers' experiences having and raising children.
Most Significant Change (MSC) is a strategy to solicit stories from community members that demonstrate significant change. Once gathered, a panel collectively identifies which stories demonstrate the most significant change. This method requires the willingness of investigators to accept unexpected results; part of the value of MSC resides in identifying where different groups and individuals find meaning. MSC is about the process of change (how change happened, what change looked like, and on what timeline that change occurred) as much as it is about collecting and reporting stories.
Storytelling is an effective qualitative method of collecting and sharing nuanced information with the purpose of effecting change. It can be used to share success stories and identify needs. For example, in southwestern Wisconsin, storytelling has been used to support community healing and advocate for recovery policy after floods. Elected officials and other decision-makers appreciate the humanity of storytelling, which is not necessarily evident in statistics. Storytelling may be used on its own or in conjunction with quantitative data methods.

When collaborating with community members on evaluation efforts, it is important to consider several key questions. These include determining who has ownership of and access to the data, and who should be involved in selecting measures, conducting data analysis, and deciding how data will be used. Programs should also explore opportunities for community members to build capacity in evaluation, such as by serving as paid members of the research team. In addition, careful thought should be given to how the evaluation team will share results and findings back with the community, how the community will be involved in interpreting and framing the findings, and how the evaluation will directly benefit community members.

Tailoring Approaches to Populations

When promoting community involvement in evaluation, rural programs must be sensitive to the unique needs and experiences of specific community subgroups, particularly tribal communities. Programs addressing SDOH among tribal communities can engage in tribally-driven participatory research to guide issues of ownership and consent, enhance trust, and ensure evaluation findings accurately reflect tribal contexts.

Resources to Learn More

EvaluATE Resource Library
Website
Provides a variety of documents, toolkits, guides, templates, checklists, and webinars offering guidance when developing an evaluation plan. Covers data collection, report writing, and finding and selecting an evaluator. Includes resources on community engagement during the evaluation process.
Organization(s): EvaluATE, The Evaluation Center Western Michigan University