funds, paying bills, and referring patients) will ordinarily be identified as part of program and evaluation planning.

In addition to describing technical, clinical, and administrative processes as they are expected to operate and establishing steps to implement these processes, evaluators need to track processes as they actually occur to identify shortfalls and unanticipated problems or complications. If, for example, a homebound patient is to demonstrate range of motion in front of a camera, an evaluation should document whether patients follow the instructions well enough for the distant clinician to make an assessment. To cite another case, if military clinicians try to use telemedicine services but find that the clinical protocols are irritating, the equipment does not work, or the consultants are not scheduled appropriately, an evaluation needs to document this and, if possible, suggest how the problem could be resolved. Event or problem logs kept by project personnel may be used to record (for later analysis) departures from planned processes as well as unexpected events and problems.

Without efforts to implement interventions as planned and to monitor the extent to which this happens, evaluators will find it difficult to distinguish between a failure of the telemedicine application and a failure to implement the application as intended. Such distinctions are critically important to those making decisions about whether to adopt, substantially redesign, or discontinue telemedicine programs.

Measurable Outcomes

Measurable outcomes identify the variables and the data to be collected to determine whether the project is meeting its clinical and strategic objectives. This committee was asked to focus on issues in evaluating quality, access, and costs for clinical applications of telemedicine. It also concluded that the acceptability of telemedicine to patients and clinicians warranted separate attention, although patient satisfaction frequently figures in assessments of quality of care, access, and cost-effectiveness. Depending on its objectives, an evaluation may consider a range of other outcomes related to an organization's competitive position, its relationships with other institutions, the demand for different kinds of health care personnel, the economic health of a community, or other effects.

In addition to outcomes desired from the project, decisionmakers

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement