Technology for Adaptive Aging (2004) / Chapter Skim
Currently Skimming:
4 Methodological Issues in the Assessment of Technology Use for Older Adults
4 Methodological Issues in the Assessment of Technology Use for Older Adults
Pages 93-128
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 93... ...
. Moreover, the normative and non-normative changes that accompany aging create a situation in which individuals with changing physical, social, and cognitive resources (the "gains and losses" associated with adult development)
|
From page 94... ...
. Arguably, selection or development of sensitive and valid measurement techniques is the most important aspect of successful evaluation research.
|
From page 95... ...
BASIC ISSUES IN EVALUATING TECHNOLOGICAL INTERVENTIONS What Are the Research Questions? It is important for social scientists to appreciate that the nature of the research question is often different in evaluation research than in research focused on identifying basic mechanisms.
|
From page 96... ...
A practical outcome for comprehension researchers might be guidelines for stimulus characteristics in experiments that eliminate or at least minimize the influence of perceptual limitations on comprehension task performance. A researcher interested in the functional impact of perceptual limitations in typical environments would, in contrast, not want to remove the influences of perceptual limitations, except as the target of an intervention design.
|
From page 97... ...
Research design needs to carefully consider rival explanations for observed outcomes-often referred to as internal validity threats (Berk and Rossi, 1999; Campbell and Stanley, 1966; Schaie, 1988) -- so as to ensure a valid assessment of intervention effects, controlling for potentially confounded variables that could mimic the pattern of expected treatment benefits.
|
From page 98... ...
However, research in several domains, including human abilities, cognitive aging, and educational psychology, points to the fact that treatments or interventions may be differentially effective for indi
|
From page 99... ...
Variables of Interest? Choice of dependent variables will necessarily depend on the research question at hand.
|
From page 100... ...
The use of ATMs may produce benefits for both users and banks in terms of time saved for transactions, but there may be potential costs in terms of frustration in learning to use the ATM and in loss of personal contact with bank employees. Telemedicine may bring diagnostic and health maintenance services to those without transportation or who live far from specialized medical facilities, but there may be costs if the accuracy of the instrumentation is not high or if there are differences in the nature of information provided by patients to healthcare providers via telecommunication devices and in face-to-face interactions.
|
From page 101... ...
Introduction of time or event sampling in the design can improve the ability of the design to address critical research questions, but it is almost always accompanied by an increase in the complexity of the design and the resulting data analyses.
|
From page 102... ...
Generically, the most important issue facing any research is ensuring the construct validity of target measures, that is, whether a measure actually taps the construct of interest as conceptualized in a research question (Cronbach and Meehl, 1955; Guion, 2002)
|
From page 103... ...
In practice, addressing issues of construct validity often involves the use of complex multivariate statistics, such as confirmatory factor analysis and structural equation models (Hertzog, 1996; McArdle and Prescott, 1994; Schaie and Hertzog, 1985)
|
From page 104... ...
, all highly relevant for assessing the impact of assistive technologies on everyday life (Gladis, Gosch, Dishuk, and Crits-Christoph, 1999; Namba and Kuwano, 1999)
|
From page 105... ...
. To take another example, it is often found that young and older adults give very similar subjective health ratings, but the number of medical conditions reported increases with age; this strongly suggests that young and older adults have different bases for assigning scale values on indices of subjective health.
|
From page 106... ...
. Finding similar subjective health ratings in young and older groups may well be an example of such adaptation-level phenomena.
|
From page 107... ...
. Use of self-report measures requires attention to a number of methodological issues that we discuss in some detail because of their centrality to valid measurement.
|
From page 108... ...
That is, particular items may be endorsed at higher rates depending on their position in a list, perhaps because list position is associated with cognitive resources necessary to evaluate the items. Such effects may be exacerbated by oral rather than visual presentation (e.g., Schwarz, Hippler, and Noelle-Neumann, 1992)
|
From page 109... ...
. Failure to answer filter questions may also lead to distraction and reduction of effort in answering subsequent questions, leading to different patterns of correlations between responses to questions in related and unrelated domains.
|
From page 110... ...
To the extent that the interviewer, in deviating from the standard script for an interview, manages to convey negative impressions about aging, the self-efficacy of older adults about encounters with technology could affect their responses. In carrying out laboratory experiments on cognitive aging, it is our experience that young and older adults construe the interaction between experimenter and participant in different ways.
|
From page 111... ...
Certain types of task may also not lend themselves well to this approach. Imagine thinking aloud while trying to evaluate information from heads-up devices that show obstacles ahead on the highway and simultaneously steering a vehicle.
|
From page 112... ...
For example, a researcher interested in evaluating a new technique for training data management might assign people at random to participate in either the new training protocol or a no-training or the current training control condition. All other things being equal, randomized experiments afford the greatest control of unwanted research confounds and are in general to be preferred to other approximate techniques
|
From page 113... ...
are not well suited for true experiments, unless screening on variables such as prior exposure to the technology is considered adequate, given the research questions. Furthermore, randomized experiments may not be warranted in some circumstances.
|
From page 114... ...
. The design predicts equivalent pretest scores on all dependent variables (due to random assignment)
|
From page 115... ...
Nevertheless, testing the hypothesis is more straightforward if estimates of experimental benefits are available for all individuals. Often it is the case that evaluation requires sampling larger units and then assigning treatment conditions to persons within those units.
|
From page 116... ...
Statistical analysis of differences in pretest scores and other individual attributes becomes an important part of assessing whether the treatment and control groups are maximally similar on relevant variables prior to introduction of the treatment. In such quasi-experimental contexts, experiments that can effectively employ within-subjects administration of treatment conditions are even more beneficial than when truly randomized experiments are feasible.
|
From page 117... ...
Given this goal, and the need to control for static individual differences variables that are correlated with the outcomes measures of interest, repeated assessment of persons as they use the technology or become introduced to the technology becomes a valuable design feature. Longitudinal designs, in effect, become a more powerful means of evaluating changes in outcomes as a function of exposure to technology.
|
From page 118... ...
. CONSIDERING THE IMPLICATIONS OF RESEARCH RESULTS Within the community of physical and mental healthcare providers, issues of clinical significance are increasingly under evaluation (see e.g.,
|
From page 119... ...
or the extent to which the intervention makes a real difference to the individual or society. For the individual, clinical significance is obviously important.
|
From page 120... ...
One can ask whether the use of particular walkers leads to a significant decrease in number of falls at home, fewer visits to healthcare providers, greater independence, reductions in difficulty performing activities of daily living and instrumental activities of daily living -- and of course all of the interpretive issues described above would apply. In the case of hearing aids, one can ask what it means to be able to report back correctly one more word or sentence in noise.
|
From page 121... ...
To the contrary, we believe that inoculating researchers with information about challenges to good research in this domain is the best method of fostering quality research outcomes and sound scientific and policy inferences based on them. Indeed, the other chapters in this volume are a testament to the fact that high-quality scientific research about aging is abundant and is having a major impact on how we understand the aging process, older adults, and the impact of technology on them.
|
From page 122... ...
. Randomized experiments for planning and evaluation.
|
From page 123... ...
. Quality of life: Ex panding the scope of clinical significance.
|
From page 124... ...
. Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives.
|
From page 125... ...
Rawlings (Eds.) , Methodological issues in aging research (pp.
|
From page 126... ...
. Implications of perceptual deterioration for cognitive aging research.
|
From page 127... ...
. De mentia caregiver intervention research: In search of clinical significance.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.