Identifying Evaluation Measures for MOUD Programs

Rural programs providing medication for opioid use disorder (MOUD) are using a variety of measures for evaluation. Evaluation measures should connect to the program goals defined at the start of the program. When selecting evaluation measures, programs should consider the type of evaluation being conducted, the patient's progress in treatment, MOUD program setting, support services offered by the program, and partnerships.

Evaluation Type

Evaluation measures should connect to the type of evaluation being conducted, such as a process or outcome evaluation. Process evaluations may assess MOUD program quality, reach, and patient satisfaction, providing information needed to recognize opportunities for improvement. Researchers have identified seven quality measures for evaluating the treatment of OUD, all of which are process measures for assessing service delivery patterns.

Examples of process evaluation measures include:

• Percent of patients who initiated MOUD within 14 days of their first intake visit
• Percent of patients retained on MOUD for 180+ days
• Percent of patients drug tested monthly
• Number of patients treated in the program
• Number of MOUD-waivered providers
• Number of clinical and social service providers

Examples of outcome evaluation measures include:

• Percent of patients with opioid negative toxicology in the first 14 days of MOUD
• Percent of patients who have a reduction in drug use
• Percent of patients who no longer meet OUD DSM-5 diagnostic criteria
• Number of patients discharged to other care settings
• Number of patients who cease treatment (and reasons why)

Stages of Treatment

When evaluating an MOUD program, it can be helpful to consider evaluation measures that fall within four different stages of treatment:

1. Engagement in care
2. Starting medication
3. Retention
4. Remission

For example, a process evaluation measure for engagement in care and starting MOUD might be the percent of patients who have initiated medication within 14 days of their first intake visit.

Program Setting

Evaluation measures may differ depending on the setting and approach to delivering MOUD. This is because programs may define and measure patient progress and recovery differently, depending on the setting. A program providing MOUD in an inpatient residential setting, for example, may measure the number of patients moving out of the residential setting and into outpatient care management (such as primary care) for MOUD, but this measure may be less appropriate for a clinic or office-based program. Programs using a hub and spoke model, as described in this evaluation of the California Hub-and-Spoke MAT Expansion Program, may consider measures related to patients becoming more stable in their medication needs and consequently moving to different treatment levels (from the “hub” level at an opioid treatment program, to the “spoke” level through a qualified primary care provider).

Support Services

When evaluating the effectiveness of an MOUD program, it is important to consider all aspects of the program, including support and social services offered, such as counseling, housing, and transportation. For example, for a program that pairs MOUD with counseling, changes in frequency of counseling sessions (for example, moving from weekly to bi-weekly sessions) may be one way to measure patient progress and program effectiveness. Other examples of measures include those that address referrals to treatment for needed support services, and whether patients who are homeless are able to secure or maintain stable housing.

Partnerships

A rural MOUD program may consider how partnerships can help support the identification of evaluation measures. For example, some rural MOUD programs are working with other organizations in the community as part of an accountable care organization (ACO). ACOs gather and report data on specific quality measures to demonstrate progress towards established program goals. As organizations work together as part of an ACO to improve the health of the communities they serve, they can review and discuss progress on quality measures as a group. For example, the Southern Oregon Model, which provides MOUD through a rural primary care network, holds regular meetings with a variety of collaborators and contributors, including regional ACOs. These meetings focus on development of guidance and practice standards, as well as education and training on medication for OUD. The In-House Chronic Pain and Opioid Use Reporting Guide includes specific process and outcome measures the model uses that may be helpful to other organizations evaluating MOUD in primary care settings.