Evaluation Measures for Telehealth Programs

A study conducted by the National Quality Forum proposes four domains and subsequent subdomains to be used as a measurement framework for telehealth programs.

Access to Care

Access to care refers to telehealth services' ability to effectively provide clinical services to remote patients and feasibility in providing specialized services such as emergency care. This domain has five components: affordability, availability, accessibility, accommodation, and acceptability. Examples of measures under this domain include:

How many encounters and claims are submitted and reimbursed (by specialty/service, by care setting, and by type of telehealth method used)?
What services are available in rural areas?
How many patients are retained in treatment at 30, 60, and 90 days and at one year?
How many people are using new services?

Financial Impact/Cost

When measuring financial impact and program cost, consider the program's financial impact on patients and their families/caregivers; the care team and the larger health system; insurance providers and other payers; and the larger community. Financial impact/cost measures include:

Patient travel miles saved
Cost per visit
Return on investment

Experience

This domain determines how user-friendly and effective the program is for patients, their providers and other care team members, and the larger community. In addition, this domain explores the program's ability to meet patients' and providers' needs. Experience measures include:

Providers were satisfied with the telemedicine experience
Instructions for care were clear to the patient
Patients demonstrated compliance with their care plan

The design of a question focusing on collecting satisfaction information could include a 10-item Likert-type survey administered after care and on an ongoing basis.

Effectiveness

This domain explores a program's clinical, operational, and technical aspects. Factors influencing a program's effectiveness and acceptance include socioeconomic and environmental setting, the type of technology used, and the type of care being delivered. Effectiveness measures include:

Time between provider request for a consult and receipt of consultation
User-friendly technologies that facilitate providers' ability to work
The extent to which telehealth is clinically integrated within a care setting

For telehealth programs, non-standard evaluation measures play a crucial role in improving clinical and financial outcomes. Patient experience and staff satisfaction surveys are examples of methods to collect non-traditional feedback from patients. Staff satisfaction surveys may vary in target audience. The Southwest Telehealth Resource Center houses many sample satisfaction surveys about store-forward and real-time consultation for the referring healthcare provider, the consulting healthcare provider, and the patient.

Additional information on identifying strategies and measures for gathering appropriate data and evidence can be found in the Rural Community Health Toolkit.

Resources to Learn More

Creating a Framework to Support Measure Development for Telehealth
Document
Summarizes a quality measurement framework to inform evaluations of telehealth programs. Includes examples and case studies illustrating proposed measure concepts and future considerations for further development of the quality framework.
Organization(s): National Quality Forum
Date: 8/2017

Health Information Technology Evaluation Toolkit: 2009 Update
Document
Outlines a step-by-step process for developing a health information technology evaluation plan. Includes information on setting evaluations goals, determining and collecting measures, and evaluating costs and feasibility. Describes project examples and evaluation measures.
Author(s): Cusack, C.M., Byrne, C.M., Hook, J.M, et al.
Organization(s): Agency for Healthcare Research and Quality
Date: 6/2009

The CTEC Telehealth Program Developer Kit: A Roadmap For Successful Telehealth Program Development
Document
Provides information on and considerations for successfully implementing a telehealth program. Includes telehealth performance indicators for inclusion in data collection to aid in program evaluation.
Organization(s): California Telehealth Resource Center
Date: 2020