There are different designs that can be used to evaluate programs. Given that each program is unique, it is
important to choose an evaluation that aligns with:
Evaluation research questions
Purpose of the evaluation
Your evaluation should be designed to answer the identified evaluation research questions.
To evaluate the effect that a program has on participants’ health outcomes, behaviors, and knowledge,
there are three different potential designs:
Experimental design: Used to determine if a program or intervention is more effective than
the current process. Involves randomly assigning participants to a treatment or control group. This type of
design is often considered to be the gold standard against which other research designs are judged, as it
offers a powerful technique for evaluating cause and effect. Fully experimental designs are unusual in
evaluation research for rural community health programs, but may be feasible.
Quasi-experimental design: Does not have a random assignment component, but may involve
comparing a treatment group to a similar group that is not participating in the program. Quasi-experimental
methods are used to estimate the effect of a treatment, policy, or intervention when controlled experiments
are not feasible.
Non-experimental design: Does not involve a comparison group. Non-experimental designs may
include pre- and post-intervention studies with no control or comparison group, case study approaches, and
post-intervention-only approaches, among others. The key feature of a non-experimental design is the lack of
a control group. While non-experimental evaluation studies are likely to produce actionable findings
regarding program outcomes, best practices, and performance improvement, they cannot control for extraneous
factors that could influence outcomes, such as community contextual factors or selection bias.
Other frameworks that have been used to evaluate rural initiatives or programs include:
Process Evaluation: Process evaluation is a systematic, focused plan for collecting data to
determine whether the program model is implemented as originally intended and, if not, how operations differ
from those initially planned. It seeks to answer the question, “What services are actually being
delivered and to whom?” This framework also gathers information on stakeholders' perceptions of the
Outcome Evaluation: Outcome evaluation examines how well a project achieved the outcomes it
set at the beginning. It is generally a summative evaluation of the program which can be used to make
recommendations for future program improvements.
Impact Evaluation: Impact evaluation reviews the effect that a program had on participants
and stakeholders of the project. It measures the outcomes, but also the changes that resulted from those
Performance Monitoring: Performance monitoring is ongoing evaluation of the program to
have data at the baseline and at key milestones in the work plan. This provides continuous, real-time
feedback on program progress so that changes to the program can be made to better align with the program
objectives and goals.
Cost-benefit Evaluation: Cost-benefit evaluations study the cost-effectiveness of the
program by reviewing the relationship between the project costs and the outcomes (or benefits) from the
program. Data collected is used to determine whether the program outcomes were worth the investment in
program development and operation.
Resources to Learn More
Framework for Program Evaluation in Public
Guided steps and standards for program evaluation and information on how findings from research will lead to
program plans that are clearer and more logical; stronger partnership; integrated information systems will
support more systematic measurement; and lessons learned from previous programs.
Organization(s): Centers for Disease Control and Prevention
Community Tool Box: Chapter 36 - Section 1. A Framework for Program Evaluation: A Gateway to Tools
Describes why evaluations are important, frameworks for program evaluation, and standards for developing a
strong program evaluation.
Organization(s): University of Kansas Work Group for Community Health and
to Evidence-Based Program Evaluation
This guide is an overview of program evaluation and each of the main steps, including evaluation goals, types of
evaluation, and evaluation designs. It clearly walks through each of these steps and provides examples
Organization(s): Substance Abuse and Mental Health Services Administration(SAMHSA)
The Step-By-Step Guide to Evaluation: How to Become Savvy Evaluation
This evaluation guide provides a framework, detailed information, and tips for employing evaluation to inform
and track progress of strategies.
Organization(s): W.K. Kellogg Foundation
an Appropriate Evaluation Design
This webpage provides background information on designing project evaluations, with descriptions of various
design-types and factors to consider when choosing an evaluation design.
Organization(s): Prevention Solutions@EDC