|
Items in cart [4] |
Questions? Call 888-624-8373 |
| ||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 149
APPENDIX B
DESIGNING AND BUILDING THE BRIDGE
The body of this report and Appendix A cover the substantive aspects of
the CASH project. This appendix describes the process: the organization
and structure of the June 1983 seminar at St. Michaels, the January 1984
follow-up meeting in Baltimore, and related project activities. It can
be thought of as a case study of an effort to foster interdisciplinary
collaboration. Readers who have found the products of the seminar to be
Of some value may want to consider some of the same approaches for
simi lar undertakings.
PREPARING FOR THE SEMINAR
Plans for the CASH seminar were developed by the chair, Judith Tanur,
with assistance from staff of the Committee on National Statistics
(CNSTAT), based on a proposal developed earlier by Stephen Fienberg and
Hiron Straf. Three specific objectives were set for the St. Michaels
meeting: (~) to review recent work in the cognitive sciences and its
potential applications to survey research; (2) to propose specific
research and experimentation that might lead to improvements in the
questionnaire and interviewing procedures used in the National Health
Interview Survey (NHIS) and other surveys; and (3) to generate ideas for
basic research, including research in the cognitive sciences using
surveys as vehicles for experimentation and the collection of relevant
data.
In planning the seminar, the organizers made three basic decisions.
First, because the number of participants would be relatively
Mall--about 20--and they would meet for approximately one week, it was
important to find an informal and isolated setting where the
participants' full attention could be given to the work of the seminar.
A suitable location was round at St. Michaels, Maryland, on the Eastern
Shore of the Chesapeake Bay, and the seminar was scheduled to convene
there for six days, from June 15 to June 21 (Wednesday through Mbaday),
1983.
Second, while the scope of the seminar was to be broader than that of
the 1980 Bureau of Social Science Resarch Workshop, experience from the
149
OCR for page 150
150
earlier meeting showed the value of structuring the discussions around
specific problems encountered in one or more complex, large-scale
surveys. The NHIS, which is conducted for the National Center for Health
Statistics (NCHS) by the Census Bureau, was selected for primary focus at
St. Michaels for several reasons. First, the subject matter of the NHIS
is complex, requiring performance of difficult cognitive tasks by
interviewers and respondents and Shus providing a rich source of examples
for analysis. Second, the error structure of the survey results is
reasonably well understood as the result of extensive and continuing
methodological research that began Just prior to the inception of the
NHIS in 1957. Last--and very important--the two agencies responsible for
the NHIS, the NCHS and the Census Bureau, expressed a strong interest in
the objectives of the seminar and a willingness to assist in
familiarizing participants with the NHIS. In addition, the organizers
selected two other surveys for less intensive concentration: the General
Social Survey, conducted by the National Opinion Research Center, and the
National Election Survey, conducted by the Institute for Social Research
of the University of Michigan.
third. it was derided that the oartictinants would be asked not only
the pro'] ect
_ _, ~ ~ _ _ ~ ~ ~
to attend the seminar, but also to give some of their time to
both before and after the seminar. Prior to the seminar, all the
participants would be asked to read relevant background materials and to
be interviewed in the NHIS. Some participants would also be asked to
provide background materials for distribution in advance of briefings at
the seminar. Following the seminar, all participants would be urged to
submit specific research proposals and would be asked to attend a shorter
follow-up meeting if the group decided that such a meeting would be
valuable.
Preparation for the St. Michaels meeting included--in addition to the
obvious (though crucial) aspects such as participant selection, agenda
development, and logistics--four special undertakings detailed below.
( 1~ I nrQnaration Id distribution to the DarticiD~ta of a set
b~r~nrn~ Laterals. The complete set of background materials, listed
_
in Appendix C, was mailed to all participants early in May 1983. Of
special significance were two papers prepared expressly for the CASH
project: Cognitive Science and Survey Methodic by Roger Tourangeau and
"Potential Contributions of Cognitive Sciences to Survey Questionnaire
Desigan by Norman Bradburn and Catalina Danis (presented in Appendix A).
80th papers are about existing and potential links between the cognitive
sciences arid Burner research, but the authors approach" the subject in
different ways.
The 16 other background materials fell into several different
categories: the relationships between cognitive science and survey
research (items 1, 2, and 11~; the NHIS content and procedures and
related methodological research (items 8-10~; other surveys to be
discussed at the seminar (items 12-14~; important concepts and research
in the cognitive sciences (items 3-5~; two pertinent tools used in survey
research--computer-assisted telephone interviewing (CATI) and validity
checks (items 6, 7, and ~5) ; and a list of selected additional readings
in cognitive science and survey methods (item 167.
OCR for page 151
151
(2) ~
NHIS. Thanks to excels ent cooperation by the Census Bureau, nearly all
of the participants were interviewed by regular Census Bureau
interviewers for the NHIS. The experience of being respondents in NHIS
personal interviews gave the seminar participants a first-hand
understanding of the nature and difficulty of the cognitive tasks
required of respondents to that survey. Although these interviews were
not part of the regular NHIS sample and were not included in the survey
estimates, standard interview procedures were used except for a
requirement that the seminar participant be one of the respondents to the
core questions and that he or she be selected to respond to the
supplemental questions on use of alcohol and tobacco. After being
reviewed for interviewer errors at the Census Bureau, the completed
questionnaires were returned to the participants for their use.
(3)
the! aemi~ar. Interviews with two volunteer respondents were conducted by
experienced Census Bureau interviewers in the respondents' homes and were
videotaped by a specialist in documentary videography. The interviewers
were paid by the Committee on National Statistics to conduct the
interviews; they were not acting as Census Bureau employees. To protect
the rights of the interviewers and respondents, informed consent
procedures were developed and were reviewed and approved by the National
Academy of Sciences' Committee to Review Human Studies, and the
videotapes were copyrighted.
(4) ~ ~
.
The adaptation of the questionnaire was done by Albert Madansky, one of
the semi nar participants, using a CATI system that he had developed . The
objective of demonstrating a CATI system was to familiarize participants
with a new interviewing technique that in likely to be increasingly used
in surveys and that may involve significant changes in the cognitive
tasks to be performed by interviewers and respondents.
THE ST. MICHAELS MEETING
The agenda for the St. Michaels meeting was divided into four principal
phases, intended to proceed more or less in sequence with some overlap.
The first phase was intended to facilitate getting acquainted. (line
first step had been taken before the meeting by circulating curriculum
vitae of all participants. ~ At the initial group cession, each
participant was asked Deco descri be his or her research interests and put
on the table rough ideas for research relevant to the CASH project
ob] ectives .
The second phase consisted of background presentations and
discussions. These sessions were intended to give all participants a
common information base from which they could work together to develop
OCR for page 152
152
research proposals. The topics of these discussions were essentially the
same an those covered by the background materials; cognitive sciences for
survey researchers; survey methods (including CATI) for cognitive
scientists; specific surveys, with mayor focus on the NHIS; and reactions
to participant and videotaped interviews. The background presentations
and discussion occupied most of the second and third days of the seminar.
The third phase was devoted to working group meetings. In order to
allow substantial time for informal, intense ve discussions in smaller
groups, two sets of. three working groups were established. The first set
of the three groups covered the major cognitive aspects of survey
interviews ~ comprehension, retrieval , and Judgment and response ), and the
second set covered the principal topics included in the core portion of
the NHIS questionnaire (utiliza~cion of health services, health
conditions, and restricted activity).
Prior to the seminar, participants had been asked to rank their
choices for the topics in each set of working groups. Assignments were
then made on the basis of several criteria: participant choices
(virtually all designations were either first or second choices within
each set); formation of heterogeneous groups, with more or leas
proportional representation of cognitive scientists, survey researchers,
agency representatives, and pro ject staff.; and minimization of overlap
between working groups in the two sets, so that each participant would
work with the largest possible number of the other participants in the
working group setting. For each working group, the chair appointed a
convener and a member of the project staff to serve as rapporteur.
In response to suggestions by several participants during the
seminar, a third whet of working groups--which came to be known as the
brainstorming groups--was organized. These brainstorming groups had two
objectives: (~) to allow and encourage all participants to present and
get reactions to additional proposals for research and (2) to allow
participants to work in a small group setting with come of the seminar
participants with whom they had not been associated in either of. the
other two sets of. working groups. The designation of these groups took
into account requests from some participants to be in the same group with
specific individuals.
The fourth and S.inal phase was feedback arid integration. General
feedback sessions were scheduled for the evenings of the third and fifth
days in order to permit discussion of possible agenda changes and
improvements in procedures. Plenary sessions were scheduled toward the
close of the seminar for working group reports. In addition to the
plenary sessions, a final session was scheduled at which the participants
were asked to summarize their thoughts about relevant research that they
would like to undertake or participate in.
With minor exceptions, al' seminar participants attended the entire
seminar. Several other people were invited to be guests at the seminar:
they included representatives from the National Science Foundation, the
National Center for Health Statistics, the Bureau of the Census, the
Committee on National Statistics, and a university (see pp. v-vi). Most
of the visitors came to the seminar site on Saturday or Sunday (the
fourth and fifth days or the seminar) and stayed through noon on Monday.
This allowed them to attend the plenary sessions at which the reports of
OCR for page 153
153
the first two sets of working groups were presented and to view one of
the videotaped NHIS interviews. In addition, each guest was invited, at
a plenary session, to react to the conclusions and research ideas
presented by the working groups.
All plenary sessions were tape recorded, and most of them were
transcribed after the seminar. Members of the project staff took
detailed notes at each session. These materials, plus the written
working group reports, form the basis of the proceedings of the CASH
project presented in Chapter ~ of this report.
At the final cession, participants agreed that a report of their
observations, conclusions, and research ideas should be prepared and that
the CASH participants should reconvene for about two days in late 1983 or
early ~ 984 to review a draft report and discuss ideas and plans for
f urther research.
FOLLOW-UP: THE BALTIMORE MEETING
After the St. Michaels seminar, the CASM chair and staff developed the
agenda for a two-day meeting in Baltimore in January 1984 and prepared a
partial draft of the project report, which was mailed to participants for
review about five weeks before that meeting.
More important, initial results from the St. Michaels seminar began
to take shape. Several of the participants, working individually or in
small groups, began new research activities or developed proposals for
research along the lines that had been discussed at the seminar. It soon
became clear that part of the Baltimore meeting could be usefully devoted
to presentation and discussion of these activities and proposals. Brief
write-ups of several of them were included in the draft report Rent out
before the meeting.
In order to involve other researchers, some steps were talcen to begin
publicizing the interim results of the CASM project. One of the
videotaped NHIS interviews was shown to and discussed with a small group
of Census Bureau employees in November 1983. Plans were made for a
cession on the CASM project at the annual meeting of the American
Association for Public Opinion Research in May ~ 984. The CASM chair,
Judith Tanur, in response to a request, organized an invited paper
session on cognitive aspects of survey methodology for the annual meeting
of the American Statistical Association in August 1984.
All of the regular participants in the St. Michaels seminar returned
for the January 1984 meeting in Baltimore. This fact may indicate their
perceptions of the value of the CASM project and their commitment to it.
Three guests were invited for part of the meeting: two (who had also
been guests at St. Michael from the National Science Foundation and one
from the Social Science Research Council.
The agenda for the Baltimore meeting retained some of the features of
the St. Michaels seminar, adapted to the shorter time aval table. At the
start of the meeting, to stimulate further thinking on the interview
process, parts of two videotaped interviews from the pretest for the 1984
round of the National Opinion Research Center 's General Social Survey
were shown and discussed. Unlike the NHIS, the General Social Survey has
OCR for page 154
154
many questions on respondent attitudes and perceptions and thus involves
cognitive processes different from those required to respond to the
basically factual questions of the NHIS.
Part of the time was spent in small group discussions. The
participants were again divided into three working groups, using
essentially the came criterion of heterogeneity within groups used to
form the groups earlier at St. Hichaels. In this case, all groups were
to discuss the same two topics: ~ ~ ~ further ideas for relevant research
and (2) proposals for ways of continuing and expanding collaboration
between cognitive scientists and survey researchers. On the final day of
the meeting, the working groups reported back to the full group.
A key feature of the Baltimore meeting, to which considerable time
wan allocated, was the presentation and discussion of the several
research activities and ideas that had been developed by individuals and
small groups of participants following the St. Michaels seminar. Prior
written reports on those activities and ideas were updated, and several
of them evoked strong interest and constructive comments from other
participants.
The final agenda item for the Baltimore meeting was to "achieve
closures on the CASH project. The participants agreed on the form of
this report and discussed means of publicizing the activities and output
of CASH so that other researchers might become involved in cross-
disciplinary collaboration. Several other kinds of outreach were
suggested in these discussions, including:
--publishing relevant articles in Journals read by cognitive
scientists and predating papers at meetings of associations to which
they belong;
--extending the dissemination process to international Journals and
conferences, such as the 1985 meeting of the International Association of
Survey Statisticians;
--arranging for appropriate Journals to publish special issues
devoted to relevant themes, e.g., cognitive studies and survey research
methods;
--when sufficient results are available from collaborative research
studies, holding symposia in university nettings to present and discuss
them;
--organizing a short course on cognitive aspects of surveys in
conjunction with an annual meeting of the American Statistical
Association;
--encouraging cognitive scientists interested in surveys as vehicles
for research to attend courses in survey methods, e.g., the seamer course
presented annually at the Survey Research Center at the University of
Michigan;
--preparing short annotated bibliographies (~) for survey researchers
interested in learning about relevant aspects of the cognitive sciences
and (2) for cognitive scientists who want to become familiar with survey
research methods;
--asking cognitive scientists to participate in proposing and
planning specific investigations in the Census Bureau's planned three-
year program of research on telephone survey methodology.
OCR for page 155
155
The question of how and by whom such outreach activities might be
undertaken led to the broader question of the future of the CASH project
in a more formal sense. The consensus of the participants in Baltimore
was that CASH had met and perhaps exceeded its original goals: it had
generated several promising interdisciplinary research activities and
plans, and it had established an informal network of scientists who
appreciate the benefits of collaboration between cognitive scientists and
survey researchers. The participants agreed that the necessary momentum
had been established for the development of collaborative research and
that the important thing now was to proceed with the research. The
network would continue to {unction and expand without requiring formal
identification or support at this time. Support for specific research
plans or, when more research findings are available, for symposia, would,
of course, be necessary.
Thus' the formal aspects of the CASH project have been completed.
Evaluation of the results, however, will continue an the organizers and
participants monitor the progress of relevant research, the application
of cognitive research findings in surveys, and the use of surveys as
vehicles for cognitive research.
OCR for page 156
Representative terms from entire chapter:
census bureau
