Evaluation Strategies and Considerations for Telehealth Programs
Rural telehealth programs are complex and varied addressing different types of health issues, healthcare
challenges, technologies, populations, outcomes, and stakeholders. Therefore, no single evaluation
strategy can be applied to all rural telehealth programs. A number of research and evaluation
studies have investigated successful telehealth
evaluation criteria. Due to its wide scope and varying reimbursement structures, there are many
aspects of telehealth programs that can be evaluated. This range is accompanied with a large set of
variables, including clinical applications, characteristics of the information being transmitted,
technology functionality, and organizational context.
The California Telehealth Resource Center Telehealth Program Developer Kit defines three key
of telehealth program development and implementation. These phases can be used to support evaluation
- Assess and define – Programs should determine service needs and consider their
environment, define their program's model, and develop a business case for the telehealth program.
Identifying these components helps telehealth programs design and frame their evaluation plans.
- Develop and plan – Programs should focus on creating a detailed project plan while
also identifying the evaluation process and metrics. This includes developing a quality improvement
process, communication plan, and technology implementation plan. Ensuring evaluation measures and
efforts are tailored for each component of the plan will allow programs to obtain an accurate,
holistic picture of their program strengths and opportunities for improvement. Programs should
consider the data needed, methods for collecting the data, ways to present the data, and the
frequency at which these data analytics reports will be generated.
- Implement and monitor – As telehealth programs implement their activities,
continuous program monitoring is important to help identify potential points of improvement. Regular
evaluation of the program allows the implementation team to identify areas of high and low
performance and therefore make changes as needed. This information may be provided in data reports
and patient satisfaction surveys.
Evaluation is important for demonstrating the value of telehealth programs, which can be fundamental
to justifying continued or expanded investment in telehealth infrastructure and program operations.
Strategic planning is important
from the start and helps telehealth programs develop plans for evaluations that capture metrics to
help show value. Incorporating a robust evaluation plan and tracking the appropriate measures can
help programs to show success and build a business case. Thoughtful evaluation plans can also help
programs sustain or expand over time.
When designing telehealth evaluation plans, programs should consider the types of measures that are important to
key stakeholders. These stakeholders may include providers, payers, health plans, and hospital boards, among
others. Research suggests
that programs should incorporate flexible multi-method and
multi-phase evaluation approaches to allow for
easy adaptation and to consider the characteristics and challenges that are unique to individual
telehealth programs. Additionally, evaluating process measures can help inform implementation and
allow programs to make adjustments as needed to improve processes and outcomes. For innovative
programs, tracking quality improvement and quality assurance efforts may be important for ensuring
that patients receive the same standard of care that they would through in-person visits.
Rural programs interested in conducting evaluation activities can contact regional
Telehealth Resource Centers to receive free technical
Resources to Learn More
A Standard Telemental Health Evaluation
Model: The Time is Now
Discusses the process and feasibility of developing a new standard evaluation model for telemental
health programs that considers multiple factors, such as healthcare costs and savings, patient and
provider interactions, telecommunications, and more.
Author(s): Kramer, G.M., Shore, J.H., Mishkind, M.C., et al.
Citation: Telemedicine Journal and e-Health, 18(4), 309-313
Approaches to Evaluating Telemedicine
Builds upon the Institute of Medicine's (IOM) broad framework for telehealth evaluation. Identifies
issues that evaluations of telemedicine can focus on and the resulting information they can
Organization(s): Office of the Assistant Secretary for Planning and Evaluation
Considerations for Planning and Evaluating
Economic Analysis of Telemental Health
Summarizes issues and limitations of existing economic analyses of telemental health. Reviews
research and methodology considerations in support of a telemental health economic evaluation.
Author(s): Luxton, D.D.
Citation: Psychological Services, 10(3), 276-282
Evaluation Framework for Telemedicine
Using the Logical Framework Approach and a Fishbone Diagram
Summarizes themes from multiple evaluation strategies to describe a comprehensive evaluation
framework for telemedicine implementation.
Author(s): Chang, H.
Citation: Healthcare Informatics Research, 21(4), 230-238
How Do I Know If It Works?
Compiles resources on telehealth program evaluation and the evidence base for telehealth efficacy and
Organization(s): Mid-Atlantic Telehealth Resource Center
“Real-World” Practical Evaluation
Strategies: A Review of Telehealth Evaluation
Describes current telehealth evaluation strategies, best practices, and challenges to outline
approaches and evaluations considerations for telehealth programs.
Author(s): Agboola, S., Hale, T., Masters, C., Kvedar, J., & Jethwani, K.
Citation: JMIR Research Protocols, 3(4), e75
Return on Investment (ROI)
Lists articles and resources for telehealth programs to consider when working to develop a business
plan for telehealth.
Organization(s): Mid-Atlantic Telehealth Resource Center
The Business of Telemedicine: Strategy
Describes a structural framework for strategic planning and telehealth program implementation.
Provides recommendations to support telehealth programs moving from a grant-funded to profit-centered
status for program sustainment.
Author(s): LeRouge, C., Tulu, B., & Forducey, P.
Citation: Telemedicine Journal and e-Health, 16(8), 898-909
CTRC Telehealth Program Developer Kit: A Roadmap for Successful Telehealth Program Development
Describes considerations and resources for healthcare organizations and individuals starting new
telehealth programs. Includes measures looking at sustainability and operational considerations and
lessons learned from existing telehealth programs.
Organization(s): California Telehealth Resource Center