Evaluation Strategies and Considerations for Health Literacy Programs

To capture the effects of a program that aims to improve health literacy, rural communities may need to consider a range of evaluation strategies. Depending on the focus of the intervention, these strategies can measure health literacy at the personal or individual patient level as well as aspects of health literacy at the organizational or system level.

When designing any evaluation of health literacy program, certain considerations may help tailor the approach to best fit the community, such as:

Including members of the community and population of focus in the evaluation design process
Engaging other organizations and champions in the community with similar goals
Remembering that context matters, including experiences of the population of focus and aspects of rural geography

The Rural Community Health Toolkit provides an overview of evaluation considerations for rural communities designing an evaluation plan. In addition, the Conducting Rural Health Research, Needs Assessment, and Program Evaluation Topic Guide provides information and resources about the importance of program evaluation and considerations for evaluating rural programs.

Evaluating Personal Health Literacy

An evaluation of a program focused on improving personal health literacy can be designed to allow for a better understanding of whether a program is helping individuals find, comprehend, and use health information to make informed decisions for themselves and others. Some evaluations may also measure whether an individual's health outcomes are improving due to changes in health literacy.

Evaluations to assess the impact of health literacy programs on personal health literacy can include both process and outcome measures. Frameworks or theoretical approaches can be applied to measure behavioral change through concepts such as self-efficacy, autonomy, readiness, and empowerment. In addition, a logic model can help describe how the program works and expected results, which can help with creating an overall evaluation plan.

Programs focused on improving personal health literacy often evaluate outcomes such as changes in health literacy levels. Changes in health literacy levels can be measured using a personal health literacy scale included within a survey or questionnaire that assesses personal health literacy.

In addition, programs focused on improving personal health literacy may choose to also evaluate changes in personal numeracy skills as a component of health literacy. Measuring numeracy skills assesses an individual's ability to understand numbers and concepts such as probabilities and percentages related to diagnoses and health conditions, treatment regimens, and medication adherence.

Evaluating Organizational Health Literacy

Programs designed to improve organizational health literacy can use evaluation to measure program activities and services and assess changes in organizational characteristics. This information may help assess successes of the organizational health literacy program.

Health literate organizations are organizations that understand the nuances of communication and that find ways to effectively improve individuals' health by addressing systems-level factors to ensure people have all the information they need in the most accessible and understandable format. According to the Ten Attributes of Health Literate Health Care Organizations, programs can embed evaluation aspects while incorporating health literacy characteristics at the organizational level such as:

Integrate health literacy expertise in the workforce to ensure the organization's mission and operations align with goals of improving health literacy
Provide trainings to staff and evaluate the impact of trainings on staff health literacy levels
Bring in outside health literacy professionals to augment in-house training resources
Create and disseminate appropriate health literacy curricula
Work in partnership with patients and staff who can provide anecdotal stories and evidence-based information

Resources to Learn More

Evaluating Community-Based Health Literacy Interventions
Document
Summary of a session during a National Academies workshop about interventions that specifically focus on strategies and considerations for evaluating community-based health literacy programs.
Organization(s): National Academies of Sciences, Engineering, and Medicine
Date: 2018

Evaluating Your Programs
Website
Provides guidelines and a framework for program evaluation and offers a summary of the steps that individuals and organizations can take to evaluate their program.
Organization(s): Centers for Disease Control and Prevention (CDC)

Health Literacy: A Prescription to End Confusion
Document
Describes, defines, and highlights the importance of health literacy measures and nuances of measuring health literacy in different contexts.
Organization(s): The National Academies Press, Institute of Medicine (IOM)
Date: 2004

How to Develop the Right Research Questions for Program Evaluation
Presentation Slides
Addresses the importance of identifying appropriate research questions for program evaluation. Includes four basic steps for developing research questions and offers an example of a logic model for a health literacy program.
Organization(s): Corporation for National and Community Service, AmeriCorps
Date: 2015