Rural organizations partnering with philanthropies should consider the types of questions they each would like to answer in the evaluation. It is important to start with the end in mind. Namely, what does success look like? Then, work backwards to explore the types of questions it is possible to answer and what is measurable in the program.
This section provides examples of common evaluation questions that rural organizations use to measure the impact of programs funded by philanthropies. These questions are relevant to organizations conducting process and outcome evaluations.
The exact questions of interest will depend on the program's goals, which may include service delivery, research, practice change, community development, policy change, or communications. Rural organizations should be prepared to answer these or similar questions as part of the philanthropy's evaluation or annual reporting process.
For additional information about evaluation, please see Evaluation Design in the Rural Community Health Toolkit.
Questions for a Process Evaluation
- What did the organization accomplish during the reporting period? For example:
- How many people were served?
- How and where were the services provided?
- What was produced during the reporting period?
- How did community members or clients perceive the program?
- How did staff members perceive the program?
- What other organizations is the program collaborating with, including subcontractors?
- What were the barriers and challenges that affected program implementation?
- Who facilitated the implementation of the program?
- How did the philanthropy support the program? (Some philanthropies are interested in understanding to what extent their program staff supported the program.)
- What program activities were not completed and why? If activities changed, why did they change?
- What are key lessons learned?
Questions for an Outcome Evaluation
- What were the outcomes of the program? For example:
- Did the program increase knowledge or awareness?
- Did the program change attitudes or beliefs?
- Did the program contribute to improvements in health outcomes?
- Did the program result in changes at the community level?
- Which outcomes are important to the community?
- Did the grant help the program leverage other funding?
- What was the program's impact? Rural organizations may wish to collect testimonials or capture stories of people or communities that the program helped and how the program helped them.
- Is the program contributing to outcomes? Can the outcomes be attributed to the program? Understanding whether the evaluation is showing contribution or attribution is important. Rigorous evaluation methods using experimental designs are often not practical, feasible, appropriate, or cost-effective when evaluating systems change initiatives. To learn more about alternative study designs, including case study approaches, see the W.K. Kellogg Foundation's The Step-by-Step Guide to Evaluation.
- What is the program's social and economic impact? Community Development Financial Institutions, in particular, may be interested in collecting information from rural organizations (borrowers) about social and economic impact (for example, the number of jobs created).