Data Collection Strategies
Strategies for collecting evaluation information should map to the evaluation questions as well as the assumed relationship(s) between the questions and the program's process and outcomes.
Well-defined evaluation questions consider the purpose of the evaluation, intended use of the results, the needs of stakeholders, and the real world circumstances that influence program success. Depending on the evaluation objectives, process and outcome assessments should address the following:
Process: How is the program being implemented? Under what conditions does the program work?
Is the target population participating at expected levels? Can the program be replicated?
- Engagement and involvement of local health and related providers
- Dates when benchmarks for diabetes program success were reached
- Expected versus actual costs of operating program
- Proportion of target population engaged in program activities
- What settings were most effective in engaging target population
- Availability of translation and language support services for non-English-speaking participants
- Documenting type and number of educational materials developed for program
Outcome: What is the impact of the program? To what extent is the program meeting its
intended goals and objectives?
- Proportion of target population screened for diabetes/prediabetes
- Successful referrals for those identified at-risk via screenings
- Inclusion of patient self-management goals in medical records
- Documentation of participation in community-based services
- Results demonstrating successful self-management/self-efficacy behaviors
- Documentation of increased diabetes knowledge
- Improvements in blood sugars determined by glucometer readings
- Records of weight loss/reduced smoking rates for target population
- Reductions in the target population that do not have needed supplies for blood glucose monitoring such as test strips and glucometers
- Increases in the target population that are now on medications to control their high blood pressures
- Lowered cholesterol levels
Successful evaluations rely on the credibility of data collected and analysis methods. Data collection plans should illustrate how, where, and from whom data will be collected. Evaluation methods to address evaluation questions include:
Quantitative data: Data that are counted and expressed in numerical terms. Answers the
question: How much change has occurred?
- Clinical records and lab results for average glucose levels recorded by the patient's glucometer
- A1C measures
- Weight loss
- Increase of physical activity
- Reductions in diabetes related emergencies and healthcare utilization
- Reduction in days lost from work or school related to poor blood sugar control
Qualitative data: Information that provides the context for understanding outcomes.
Although difficult to measure, these data can provide additional insight into program results. Answers the
question: Why or how has change occurred?
- Exit interviews
- Focus groups that identify the existence of improved physician/patient partnerships
- Satisfaction with program participation and the results achieved
- Improvements in the level of knowledge of diabetes and the need to take action to limit or avoid complications
Important contextual (qualitative) information for rural areas includes local beliefs and behaviors, system capabilities, and limitations regarding capacity, technology, and needed resources. Quantitative and qualitative data should be collected within a framework consistent with stakeholder expectations, project timelines, program objectives, and an evaluation framework that is compatible with program structure. Types of quantitative and qualitative data collection methods include:
- Surveys and questionnaires: Surveys and questionnaires gather information from respondents through a series of questions and prompts. Surveys can be administered in person, by telephone, by mail, or electronically through email or web-based programs.
- Focus groups or interviews: Focus groups collect information from several individuals at the same time and are conducted in a question and response format. Interviews can be structured around a specific theme or issue and allow for more in-depth exploration of the topic. Interviews can be unstructured, allowing for flexibility in participants' response.
- Observations and progress tracking: Observation uses standardized procedures to record behaviors, situations, and events. Observation is useful for gathering information where other methods are inadequate.
- Electronic health record/electronic medical records where available can provide important records of activities and interactions that can influence the ability to achieve intermediate and longer term program goals. The use of EMR/EHR in rural communities is increasing rural programs' ability to draw data directly from the patient health record.
Resources to Learn More
Resources: Assessment Materials, Forms & Instruments
Documentation forms and assessment tools for clinical and community-based diabetes programs.
Organization(s): Robert Wood Johnson Foundation