Evaluation Tools

Evaluation assessments to determine the efficacy of mental health programs can be conducted internally by members of the organization or can be conducted by an outside source. If conducted internally, organizations should be aware of potential bias that might impact evaluation results. One way to mitigate this bias are by having multiple members of a team participate. Alternately, using an external evaluation source will also help avoid bias.

Rural communities may face specific challenges when using evaluation tools depending on resources such as funds and staffing. Rural communities can help to overcome these challenges by engaging with community partners. By working with stakeholders, rural programs also may be able to find more affordable ways to assess their organization.

Having a clear evaluation guide to follow will ensure an accurate evaluation process regardless of staff shortages or turnover. There are numerous methods for identifying evaluation tools, but two distinct measures that can be used for rural mental health programs are fidelity assessments and outcome evaluations. These approaches are especially relevant to rural mental health because they provide a framework for effective evaluation.

Fidelity Assessments

• Gauge whether a program is meeting standard practice requirements
• Use objective rating systems to determine if programs align with evidence-based practices

The Fidelity Assessment Common Ingredients Tool (FACIT) can by used to measure program effectiveness, specifically for certain mental health interventions. This test is crucial for rural mental health programs because organizations may have limited resources and maximizing resources is essential for effectiveness.

Outcome Evaluations

• Look at participant perspectives to determine the success of a program based off of patient satisfaction, which is especially important in mental health programing, as satisfaction can predict patient engagement and adherence
• Results include both objective and subjective outcomes

The Peer Outcomes Protocol (POP) provides a means to assess consumer outcomes.

Resources to Learn More

Evaluation Approaches for Mental Health Prevention and Early Intervention Programs
Document
Handbook for providers and organizations providing mental health services to evaluating prevention and early intervention programs.
Author(s): Eberhart, N. K., Cerully, J. L., Shearer, A. L., et al.
Organization(s): RAND Corporation
Date: 2017

Program Evaluations
Website
Provides overview of evaluation policies and procedures for substance abuse and mental health programming.
Organization(s): Substance Abuse and Mental Health Services Administration