Evaluation Measures for Telehealth Programs
When designing telehealth evaluation plans, programs should consider the types of measures that are important to
key stakeholders. These stakeholders may include providers, payers, health plans, and hospital boards, among
others. Research suggests that programs should incorporate flexible multi-method and
multi-phase evaluation approaches to allow for easy adaptation and to consider the characteristics and
challenges that are unique to individual telehealth programs. Additionally, evaluating process measures can help
inform implementation and allow programs to make adjustments as needed to improve processes and outcomes. For
innovative programs, tracking quality improvement and quality assurance efforts may be important for ensuring
that patients receive the same standard of care that they would through in-person visits.
Access to Care
Access to care refers to telehealth services' ability to effectively provide clinical services to remote
patients and feasibility in providing specialized services such as emergency care. This domain has five
components: affordability, availability, accessibility, accommodation, and acceptability. Examples of measures
under this domain include:
How many encounters and claims are submitted and reimbursed (by specialty/service, by care setting,
and by type of telehealth method used)?
What services are available in rural areas?
How many patients are retained in treatment at 30, 60, and 90 days and at one year?
How many people are using new services?
When measuring financial impact and program cost, consider the program's financial impact on patients and their
families/caregivers; the care team and the health system; insurance providers and other payers; and the larger
community. Financial impact/cost measures include:
- Patient travel miles saved
- Cost per visit
- Return on investment
This domain determines how user-friendly and effective the program is for patients, their providers and other
care team members, and the larger community. In addition, this domain explores the program's ability to meet
patients' and providers' needs. Experience measures include:
- Providers were satisfied with the telemedicine experience
- Instructions for care were clear to the patient
- Patients demonstrated compliance with their care plan
The design of a question focusing on collecting satisfaction information could include a 10-item
Likert-type survey administered after care and on an ongoing basis.
This domain explores a program's clinical, operational, and technical aspects. Factors influencing a program's
effectiveness and acceptance include socioeconomic and environmental setting, the type of technology used, and
the type of care being delivered. Effectiveness measures include:
- Time between provider request for a consult and receipt of consultation
- User-friendly technologies that facilitate providers' ability to work
- The extent to which telehealth is clinically integrated within a care setting
Rural communities can also consider a number of other measures to demonstrate the impacts of their telehealth
services. Other measures may include:
Health outcomes, which will vary by the telehealth service provided
Health equity, or the extent to which provision of telehealth services affect disparities among the patient
Program implementation and key performance measures, such as number of telehealth visits delivered
For telehealth programs, non-standard
evaluation measures play a crucial role in improving clinical and financial outcomes. Patient experience
and staff satisfaction surveys are examples of methods to collect non-traditional feedback from patients. Staff
satisfaction surveys may vary in target audience. The Southwest Telehealth Resource Center houses many sample
satisfaction surveys about store-forward and real-time
consultation for the referring healthcare provider, the consulting healthcare provider, and the patient.
Additional information on identifying strategies and measures for gathering appropriate data and evidence can be
found in the Rural Community Health Toolkit.
Resources to Learn More
a Framework to Support Measure Development for Telehealth
Summarizes the development of a quality measurement framework for telemedicine to be used as a guide by
researchers, analysts and others in healthcare for future evaluation efforts. Includes examples and case studies
illustrating proposed measure concepts and future considerations for additional development of the
Organization(s): National Quality Forum
CTEC Telehealth Program Developer Kit: A Roadmap For Successful Telehealth Program Development
A step by step methodology for healthcare organizations and individuals planning, developing, implementing, and
monitoring new telehealth programs. Includes a performance indicator and data matrix, and additional guides on
staffing, job descriptions, workflow, room design, job descriptions, and developing a marketing plan.
Organization(s): California Telehealth Resource Center
for Evaluating Telehealth
Offers a table listing examples of measures useful when evaluating telehealth services.
Author(s): Chuo, J., Macy, M.L., Lorch, S.
Location: Pediatrics, 146 (5)