research projects discussed in Chapter 5 attempt experimental and quasi-experimental research strategies. Even with less demanding designs, tension will exist between the principles of design and the pressures of real-world evaluation.
Another stream of work on alternatives or supplements to the RCT has emphasized nonexperimental research based on the retrospective analysis of large databases that have often been compiled for other purposes (Roos et al., 1982; Moses, 1990; Hannan et al., 1992; NAHDO, 1993). Until telemedicine applications become much more common and routine and are assigned codes to identify them, large databases are unlikely to be useful sources of data on telemedicine applications.
Nonetheless, those looking ahead to more widespread use of telemedicine should consider how routine collection of data about telemedicine may be useful and what would be required to incorporate such data in large data systems. The appeal of these data sources lies in their relative convenience, large numbers of cases, and ease of statistical analysis. Questions or criticisms related to use of large databases for health services research, performance monitoring, and other purposes involve their completeness, accuracy, relevance, and security from authorized access (IOM, 1994b; Maklan et al., 1994; Kuller, 1995). A variety of initiatives have focused on means to reduce the amount of missing data, validate and improve coding of clinical and other information, add information (e.g., death records), and develop methods to adjust comparisons for differences in severity of patient conditions (IOM, 1994b; Roos et al., 1995). Even with improvements, data collected for one purpose (e.g., claims administration) may remain questionable for other purposes (e.g., outcomes research) if they lack reliable information about patient medical status, processes of care, and other variables. The OTA, for example, warned that "focusing on this research method as a relatively simple, inexpensive first-line tool for answering comparative questions [about the effectiveness of treatment alternatives] is unwarranted" (OTA, 1994, p. 74).