|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 32
Format and Conduct of Consensus
Development Conferences:
A Multination Comparison*
Elizabeth A. McGlynn, Jacqueline Kosecoff, and
Robert H. Brook
INTRODUCTION
The purpose of this paper is to compare the methods used in nine
countries to organize and conduct consensus development confer-
ences on scientific issues related to the delivery of medical care.
Our comments focus primarily on the format and conduct of these
conferences. However, we also briefly discuss the context, selec-
tion, and scope of topics; the role of evidence; and the dissemination
of results. We confine our remarks on these latter topics to descr~p-
tive statements that illuminate issues relevant to a discussion of
process.
We address the process by which consensus development confer-
ences are conducted because the approach taken may influence the
results, or the acceptability of the results, either intentionally or
unintentionally. Thus, at each step of the process it is important to
consider whether the mechanics of conducting the conference are
enhancing or detracting from achieving its goals and objectives. To
the extent that the mechanism itself interferes with the intended
outcomes, changes in the approach are warranted.
*This paper is to be published in a forthcoming issue of the International Jour-
nal of Technology Assessment in Health Care, Cambridge University Press, New
York.
32
OCR for page 33
FORMAT AND CONDUCT
METHODS
33
We set out to examine the process by which consensus develop-
ment conferences are conducted in Canada, Denmark, Finland, The
Netherlands, Norway, Sweden, Switzerland, the United Kingdom,
and the United States. Although most of these countries began
with the mode} used in the United States by the National Institutes
of Health (NIH), many important variations have been introduced
(Casparie and van Everdingen, 1985; Bennett, 1985; Perry, 1987,
19XX; Stocking, 1985; yang, 1986~.
The 10 programs considered are:
I. The Canadian Task Force on the Periodic Health Examina-
tion (Canada I)
2. The Canadian Research Group (Canada II)
3. The Danish Medical Research Council and the Danish Hospi-
tal Institute
4. The Medical Research Council of the Academy of Finland
5. National Organization for Quality Assurance in Hospitals in
The Netherlands
6. The Norwegian Institute for Hospital Research and the Nor-
wegian National Research Council
7. The Swedish Planning and Rationalization Institute for the
Health and Social Services
8. The Swiss Institute of Public Health
9. The King's Fund Forum in the United Kingdom
10. The U.S. National Institutes of Health Office of Medical Ap-
plications of Research (OMAR).
The consensus development conference is a complex entity, and
researchers have undertaken scientific evaluations of the conferences
and their impact in their respective countries (Calitorp, 1988;
Johnsson, 1988; Kanouse et al., 1989; Lomas et al., 1988; Worunan
et al., 1988~. An international comparison is challenging because
there may be as much or more variation within a particular country
regarding how individual conferences are conducted as there is among
countries (Andreasen, 1988~. For example, more than 75 such con-
ferences have been held by NIH in the United States, and although
the mode} was basically the same, implementation certainly varied
across the conferences. We have attempted to use the most recent
data available to represent each country in order to capture any
OCR for page 34
34
CONSENSUS DEVELOPMENT
changes that have taken place over time in the conduct of the confer-
ences.
We used three sources of information to prepare this paper:
I. Profiles prepared by representatives from each country for the
June 1989 workshop on international consensus development for
medical technology assessment organized by the Institute of Medi-
cine
2. Published articles and consensus statements related to such
conferences
3. Comments received after the conference from representatives
of each country.
We divided the consensus development process into four stages for
our investigation:
I. Context of the consensus development process
2. Prepane} process
3. Pane} composition
4. Consensus pane! meeting.
The context within which consensus conferences are conducted
encompasses the nature of the audience, the topics considered, and
how the topics are selected. The prepare} process includes all ac-
tivities that are required to stage the consensus conference, such as
selecting the chair, panel members, and presenters as well as prepa-
ration of background information. Pane} composition includes the
type of panelists, their qualifications, and the process by which they
are selected. The consensus panel meeting stage describes the ac-
tivities at the actual conference, such as the use of public forums
and private sessions, the type of information considered in arriving
at consensus, and the group process by which consensus is actually
achieved.
RESULTS
In this section, we describe how each of the countries described in
this paper conducts consensus conferences. We have organized the
section around the four major areas investigated: context, prepanel
process, pane} composition, and the consensus panel meeting. Within
each area we addressed several questions. Tables ~ through 4 sum-
marize our findings.
i
OCR for page 35
FORMAT AND CONDUCT
Context
35
We begin by examining the context in which consensus develop-
ment conferences are conducted because the format of conferences
may differ depending on the intended audience for the conference
and what issues are addressed. The results are displayed in Table 1.
Audience
The potential audience for any of these conferences is fairly broad.
In most cases, countries indicate that a variety of different groups
are included in the target audience. We, however, tried to determine
which audience was the primary target for each country in order to
detect differences in the focus of planning and conference efforts.
Six of the programs consider health professionals to be the audience
for their consensus conferences (Canada ~ and II, Finland, The Neth
eriands, Switzerland, and the United States). Within those programs,
most seem to be addressing physicians who are involved in direct
patient care; however, academic-based physicians and research scien-
tists are also targeted. Two countries (Norway and Sweden) consider
national health authorities, such as health planners and political
decision makers, as well as health professionals to be the primary
audience for the conference. Denmark and the United Kingdom
indicate that the public and health professionals are their primary
audiences; most countries suggest that the public may be interested
in the findings. Switzerland is the only country that included third-
party payers in its audience list.
Issues
The scope of issues addressed in any one conference varies con-
siderably, and most countries appear to have a fairly broad perspec-
tive. Countries typically indicate that they intend to address a vari-
ety of issues, including safety, effectiveness, efficacy, appropriate-
ness, service requirements, economics, political and social concerns,
and needs for future research. The evidence that is available in any
particular area limits the issues that can be addressed. Most consen-
sus development conferences address issues of effectiveness and
efficacy (the strength of the clinical literature), while few address
economic or cost issues (rarely addressed in clinical or epidemio-
OCR for page 36
36
CONSENSUS DEVELOPMENT
TABLE 1 Context in Which Consensus Development Conferences Are
Conducted
Context United States HUH) Canada I
Who ~ the audience
for consensus
product?
What iggucs does
panel address?
Who selects topics?
No. of topics
Health professionals;
public
Benefits;
appropriate use of
technology;
future research
Instituteg within
NIH and OMAR
75
Practicing primary
. . .
care physicians;
medical educators;
policymakers;
general public
Appropriateness of
procedures to
prevent adverse
outcomes;
focus is on the
condition rather
than the procedure;
effectiveness
(benefit > harm?) or,
if no information,
efficacy
Task Force on
Periodic Health
Examination
1
OCR for page 37
FORMAT AND CONDUCT
Canada ~ Denmark Finland
Practicing physicians
Appropriateness
benefit ~ risk
Investigators from
participating universities
(caster, Univ. of Toronto,
Univ. of Briush Columbia)
2 6 3
37
Public;
health planners;
health professionals
Effecuiveness;
safety;
adoption level;
economic,
organi7~honal,
and ethical
implications
Subcommittee of
Danish Research
Council
Health care
providers;
researchers;
public,
providers of
agricultural products
(cholesterol)
Safety;
efficacy;
effectiveness;
. .
service requirements;
adoption level;
economic and other
(e.g., ethical)
. .. .
unpllcanals
Medical Research
Combs
Table I continues
OCR for page 38
38
TABLE 1 Continued
CONSENSUS DE~ELOPMEl1T
Context The Netherlands Norway Sweden
Who is the audience Clinicians; National health Politicians;
for consensus hospital authorities; administrators;
product? administrators; hospital owners; educators;
policymakers health professionals planners;
public
Whatissues does Primary: safety, Effectiveness; Safety;
panel address? efficacy, efficacy; efficacy;
effectiveness; risks; effectiveness;
also: cost- costs; cost/economic
effectiveness, consequences: implications;
service requirements organizational, minor role for
psychologicaVsocial, ethical, legal,
ethical social
implications
Who selects Scientific Council Technology Representatives from
topics? of COO Assessment Medical Research
Committee Council and
Swedish Planning
end Rationalization
Institute of Health
Services form a
steering committee
No. of topics 22 2 9
OCR for page 39
FORMAT AND CONDUCT
Switzerland
Physicians;
health care
administrators and
decision makers
third-part~r payers
Safety;
effectiveness;
substitution versus
replacement;
appropriate uses;
cost
Planning committee
39
United Kingdom
NationaUlocal
policymakers;
. . .
c 1nlclans;
health professionals;
public
Scientific merits of
technology for
different patients;
cost;
safety;
effectiveness;
legal, social,
service, ethical
. . .
imp. Cations
King's Fund steering
group including
. . .
G mans,
professional bodies,
representatives of
Dept. of Health
7
Degree of Variation
among Countries
Six focus on health
professionals;
one on public
two on national
politics or planners;
United Kingdom
considers all
potential audience
groups to be equally
important
Scope varies:
most have global
perspectives;
few seem likely to
truly address
efficacy;
four examine
appropriateness;
one examines whether
technology is a
replacement or
~ .
substitute
Most combine
government and
specialty society
input (except United
States and Canada II)
140+
OCR for page 40
40
CONSENSUS DEVELOPMENT
logic studies). Four programs (Canada I and II, Switzerland, and the
United States) address appropriateness of technology use.
Topic Selection
Most countries rely upon a combination of government and spe-
cialty societies to select topics for consensus development confer-
ences. The exceptions are the United States and Canada II. The
United States relies upon staff within the government-funded NIH to
select topics, but does not formally involve relevant U.S. medical
specialty societies (Office of Medical Applications of Research,
National Institutes of Health, 1988~. Canada lI selects topics based
on interests of the principal investigators and the availability of fund-
ing to explore various technologies; subsequent to topic selection,
the researchers generally seek government funding and specialty
society support. In most countries, topic selection comes from a
standing committee responsible for technology assessment activities
(e.g., Scientific Council of CBO in The Netherlands, Research Coun-
cils in Denmark and Norway).
Topics
The variation in topic selection within any country depends in
part on how many consensus conferences have been conducted. In
the United States and The Netherlands, which have conducted more
than 75 and 27 conferences, respectively, there is considerable vari-
atic~n in the tulles of topics selected
try r In general, across ad countries,
four types of topics have been addressed: treatment of conditions
(e.g., otitis media, schizophrenia), diagnostic procedures (e.g., de-
tection of breast cancer, diagnosis of venous thrombosis), therapeutic
procedures (e.g., adjuvant chemotherapy for breast cancer, coronary
artery bypass surgery), and planning issues (e.g., need for asylum
for the mentally ill, impact of routine HTEV-TII [human T-lym-
photropic virus type ITI] antibody testing on the safety of the blood
supply).
Conferences have also considered technologies at varying stages
of development, from new or emerging to established and outmoded.
The stage of development is important both because it has implica-
tions for the availability of evidence and because the degree of con-
OCR for page 41
FORMAT AND CONDUCT
41
troversy surrounding the technology may be related to its stage of
development.
Prepane' Process
Although most countries spend between six months and one year
planning for the consensus development conference, different types
of activities are undertaken during that time, as shown in Table 2.
Responsibility for Planning
Three different types of groups have responsibility for planning
the consensus conferences. In seven countries (Denmark, Finland,
The Netherlands, Norway, Sweden, Switzerland, and the United
Kingdom), planning is done by a special working group generally
appointed by the committee responsible for selecting the topic. In
two countries (Canada and the United States), staff are responsible
for preparing for the conference. In the United States, employees of
the relevant bureaus, divisions, or institutes within NTH conduct
planning activities. In Canada II, planning is done by the academic-
based research team. In one country (Canada D, planning is done by
the consensus pane! itself.
Review of Literature
Most of the conferences rely upon oral presentations of scientific
evidence as the basis for making judgments about the technology
under consideration. In most countries, information is sent ahead of
time to panel members to prepare them for the conference. For
example, in the United Kingdom, pane] members receive introduc-
tory textbook-type information (necessary for the lay members of
the panel) as well as a comprehensive set of readings, drawn from a
computer search of all relevant literature, and abstracts prepared by
the expert speakers who will present information at the actual meet-
ing.
Canada ~ and Il and Switzerland prepare a formal synthesis of the
literature in advance of the pane} meeting. For Canada I, which
arrives at a consensus through several iterations of papers, the syn-
thesis of the literature provides a starting point for developing the
OCR for page 42
42
CONSENSUS DEKELOPMENT
TABLE 2 Types of Activities Undertaken during He Prepanel Process
P'epanel Process United States (OH) Canada I
Time spent in 12-15 months Ongoing process
preparation for
panel meeting?
Who is responsible N1H staff: Task force members
for planning end OMAR coordinator,
preparation? bureau, institute, or division
(BE)) coordinator,
representatives from sponsor
agencies
Extent and nature of Bibliography and Original reports of
literature review? paper abstracts sent clinical trials or
in advance; epidemiologic studies;
sometimes background quality of evidence
reports prepared or is graded;
expert summaries of expert opinion used
state of science; if no evidence is
BE coordinator detennines available
scope of literature search
Is there a review of For sane topics, Some efforts made to
current clinical cum nt uses are measure patient
practice (e.g., summarized compliance;
efficacy, current working on rules for
uses of die using data on
technology)? efficacy (e.g.,
decision analysis
techniques)
Are patient outcome Both intermediate
data used in and final outcomes
planning? are considered;
disease prevention
is main outcome;
examining broader
range of outcome
measures (e.g.,
quality of life)
Are commendations
or Specific
questions prepared
for panel in
advance?
Conference questions
developed by
planning committee
(four to six questions with
one pertaining to
future research);
structure of questions allows
answers to be drawn strictly
from scientific literature
Part of background
paper which is
prepared by panel
member and then
revised over time
OCR for page 58
58
TABLE 4 Continued
CONSENSUS DEVELOPMENT
Consensus The Netherlands Norway
Panel Meeting
Sweden
What is the final Consensus statement
product?
How long does the One to two days
panel meet? (including nights)
Extent of public forums; Practitioners and
number of audience public;
members? audience: 150-1,000
Consensus statement Consensus
press conference statement
Two and a half days Two and a half days
(through night on
last day if necessary)
Physicians, experts
and public
audience: ~200
One and a half days
for expert
Private panel Experts meet several Private sessions to Panel starts writing
sessions? times ahead of the write the consensus on afternoon of
actual conference statement second day
How is consensus Chair formulates Panel sets the rules
defined? guidelines
Who makes the final Audience is asked if Panel Panel presents at
decision? they agree; final press conference
decision is with panel
Are formal votes No No No
taken?
What criteria are None None None
used for making
decisions?
How is disagreement Points of Not clear Press conference
handled? disagreement are
mentioned in
consensus
statement text
Sensitivity to group Not clear Not clear Not clear
process issues?
OCR for page 59
FORMAT AND CONDUCT
59
Switzerland
United Kingdom Degree of Variation
among Countries
Consensus statement;
summary presentation
in of forum;
press conference
Two and a half days (no all
night session)
Consensus statement
Three days,
21 hours;
no overnight session
Nine consensus statements;
one practice guideline;
one indication rated (plus
consensus statement);
many hold press
conference to
release statement
Most are two and a half days
and write through night
Canada I is four to eight
iterations over one to two years
One day of evidence; Same as NIH; All except Canada I
half day of discussion; audience: 200 rely on public foam
audience: 150
Two half-day Executive session All write statements
executive sessions; for writing in executive session;
meet night before statement; some Canada II has
plenary session revisions made after indications rated before
public presentation and during meeting
No formal voting; No clear definitions No clear definitions
comments on draft
Chairman Panel Panel makes decision in all but one
case; public input in two;
chairman decides in Switzerland
No No No, except Canada II
Value of scientific None
evidence; confonnity
with state of practice
Detailed discussion; Not clear
if no consensus,
disagreement
mentioned in final
document
Little experience handling Not clear
problems in a
nonhomogeneous group;
panel did not question
experts thoroughly
Generally no
. . . . .
exp. lClt cntena used
U.S., some minority
reports; Denmark, public
debate; most have press
conferences; Canada II,
ratings; Switzerland and
lye Netherlands, mentioned in
consensus statement
OCR for page 60
60
CONSENSUS DEVELOPMENT
four to eight iterations of a paper over one to two years), Canada II
(which relies upon rated indications from an iterative process using
the RAND/UCLA modified Delphi method to form the basis for
pane! recommendations), and The Netherlands (which meets 5 to 10
times before the final meeting at which the consensus statement is
completed).
Public Forums
All programs except Canada I rely upon public forums during
which evidence is presented. The audience size ranges from 150 to
1,000, with most audiences being about 200. In most cases, the
audience may ask questions or make comments, although the degree
to which this is done varies. It is not clear whether or not the
audience contributes substantially to the process; rather, it appears
that the audience is in attendance primarily to observe the proceed-
ings.
Private Pane! Sessions
The bulk of work required to answer questions and to draft, re-
vise, and complete the consensus statement is done in private ses-
sions. Most panels meet the night before the conference begins to
receive instructions about the process of the meeting itself and to
hear We `'charge" to the panel (i.e., questions to be answered). In
many cases, the questions to be addressed have already been made
public (Norway). In Finland and The Netherlands, the panel con-
venes prior to the actual conference. In Finland, one or two meet-
ings are held; in The Netherlands, several meetings are held during
the conference planning process. The familiarity of panelists with
one another presumably affects the group process dynamics. This
may be accomplished in some countries without preconference meet-
ings because Me pane} members already know each other (Wortman
et al., 1988).
Definition of Consensus
Little information exists on how each country defines consensus.
The implied definition is unanimous agreement with the consensus
OCR for page 61
FORMAT AND CONDUCT
61
statement. Because unanimity is expected, there are few formal
mechanisms available for dealing with disagreement (with the ex-
ception of Canada II). The Netherlands and Sweden indicate that
the chair and pane} (respectively) are responsible for formulating
guidelines to define consensus, but whether that requirement intro-
duces any more systematic consideration into the process is unclear.
Most countries release consensus statements at press conferences
and/or in journals, which presumably allows a forum for dissenters
to present their opinions. In the United States, a minority report
may be used if serious disagreement remains, although this option is
rarely employed (Iacoby, INS). Switzerland and The Netherlands
mention any remaining sources of disagreement in the consensus
statement.
Decision-Making Process
In all but one country, the pane} makes the final consensus deci-
sion; in Switzerland, the chair is responsible for final decisions.
Public input is solicited in two countries (The Netherlands and the
United States). No formal votes are taken in the process of writing
the consensus statement. There are also no formal criteria for mak-
ing decisions. In Canada I, there is a formal rating of the evidence
using criteria that are predetermined, and the consensus statements
are tied to this rating of the quality of evidence.
Sensitivity to Group Process Issues
There is little available evidence on the extent to which confer-
ences are sensitive to group process issues. The size of many of the
consensus panels suggests that, in the absence of concerted efforts,
many panelists may not voice their opinions during discussion. The
lack of formal decision-making criteria and formal voting suggests
that undercurrents of dissent may go unrecognized. The practice of
writing consensus statements in an aH-night session suggests that
individuals may agree because they are simply too tired to continue
disagreeing. The presentation of evidence only in oral form would
seem to particularly disadvantage lay members or scientists in unre-
lated fields who are unfamiliar with the literature and/or the science
of the technology. This is likely to result in less participation in the
OCR for page 62
62
CONSENSUS DEVELOPMENT
discussion by the generalists, potentially resulting in decision mak-
ing that is biased by particularly knowledgeable or vocal pane}
members.
DISCUSSION
Consensus development conferences are used in many countries
to assess the uses of medical technologies. Most of the conferences
are conducted using a variation of the mode] that was developed in
the United States by NIH. In this paper we have described the pro-
cesses by which such conferences are conducted and the extent of
variation among several countries along a series of specific dimen-
sions. In this section we discuss the major concerns that arose from
our review.
Link between Goals and Inputs
Most countries indicate that they use consensus development con-
ferences to address a variety of issues, including the safety, effec-
tiveness, efficacy, appropriateness, and consequences (political, social,
ethical) of medical care and technology. In most cases, however,
there appears to be a gap between these goals and the information
available upon which to base such judgments.
Few countries undertake a systematic examination of the clinical
or health services research literature in order to construct a synthesis
of what is known from published sources about the use of a technol-
ogy. If a synthesis is done, it rarely relies upon a formal meta-
analysis or any quantitative process of combining evidence.
There are several advantages to beginning the process of planning
a consensus development conference by conducting a thorough lit-
erature synthesis. First, the synthesis provides all panelists with a
common starting point for discussions. Few individuals, even those
who work with a technology in clinical practice or research settings,
are completely informed or have synthesized and can remember all
findings from the literature. Panelists may tend to remember the
studies that support their respective positions and forget or disregard
information that runs counter to their beliefs. The common ground
provided by the synthesis may be particularly important for lay pan-
elists or scientists from different disciplines; in most countries, about
half the panelists are in such categories.
OCR for page 63
FORMAT AND CONDUCT
63
Second, the synthesis and the information about the state of clini-
cal practice as it exists in a community setting provide "the facts"
from which to build a set of questions to be addressed by a confer-
ence. In one evaluation of the consensus development conference
process in the United States, the authors found that at the beginning
of the conference, panelists believed that the right questions were
being asked about a technology, but that their opinions about the
critical issues changed as evidence was presented during the course
of the conference (Wortman et al., 1988~. Other research found that
conferences focused on areas of clinical medicine believed to be in
need of change when actual clinical practice had already been cor-
rected (Kosecoff et al., 1987~.
A synthesis of the literature provides a means of identifying not
only what is known but also what is not known from published
sources (Iacoby, 1988~. The missing pieces may be answered through
presentations by experts based on research that has not yet been
published or through a systematic evaluation of sources such as pa-
tient medical records, use of decision analysis techniques, or expert
opinion. In fact, we found that many countries conduct ad hoc
surveys prior to a conference to collect data on current uses of the
technology being evaluated. In addition, a synthesis, if done care-
fully, can help combine conflicting information and focus the panel
on the most critical concerns. For instance, the literature synthesis
may reveal that studies supporting the use of a technology are based
on data obtained from poorly conducted studies, while data that do
not support technological use come from randomized controlled clini-
cal trials.
Third, the synthesis provides a systematic way of identifying the
experts in the field. Previous evaluations have suggested that the
selection of experts is not necessarily systematic (Wortman et al.,
1988~. This is not meant to suggest that the informal network ap-
proach results in the selection of worse (or even different) experts
than a systematic approach, but that the informal approach is less
scientifically defensible and has the potential for introducing unin-
tended biases into the consensus development process.
Finally, linking of each consensus statement to its supporting evi-
dence would allow users of the consensus findings to understand the
scientific bases for the recommendations and would make clear where
judgments are based on clinical trials, epidemiological studies, ob-
servational studies, or expert opinion. This might also enable panels
OCR for page 64
64
CONSENSUS DEVELOPMENT
to make statements with varying degrees of certainty based on the
science available to back the statement, which could, in turn, en-
courage the discussion of controversial topics.
Formalizing the Group Process
Consensus development conferences appear to be run fairly infor-
mally with respect to the criteria for making decisions, definition of
consensus, formal voting or polling of pane} members, and handling
disagreements. A more formal process might not change the out-
comes substantially, but it would make the process more scientifi-
cally defensible and replicable. An increase in the formality of the
process might also allow for the devotion of more discussion time to
controversial issues, if provisions are made for a range of consensus
definitions.
The group process might be enhanced if the discussion is divided
into disagreements about underlying assumptions. For example,
panelists could discuss the probabilities of positive and negative
outcomes associated with the use of a technology versus the patient
utilities associated with those outcomes. If a conference recom-
mended doing procedure x in clinical situation y, the recommenda-
tion should be based on achieving certain specified outcomes. If
both good and bad outcomes result, it would be useful to know what
weights patients place on those outcomes (e.g., how much risk of
death a person is willing to take to gain a specified improvement in
functioning). Finally, for those conferences that deal with economic
or management issues, it should be made clear how these considera-
tions modified the conference findings regarding clinical policy (e.g.,
the use of the technology is clinically acceptable but the financial
implications are unacceptable).
Although not stated explicitly, it appears that consensus is de-
fined in most conferences as unanimity. It has been suggested that
requiring unanimity may result in statements that represent the "lowest
common denominator" of opinion. Allowing consensus to take on a
meaning that is less restrictive than unanimity expands the range
and type of issues that can be addressed. The most straightforward
way of introducing this type of definition of consensus into the
process is by taking formal votes throughout the development of the
consensus statement. Voting also provides a mechanism that allows
for disagreement without necessarily endangering the overall pro-
cess.
OCR for page 65
FORMAT AND CONDUCT
65
Alternatively, polling could be used (as it is in juries where una-
nimity is required) to ascertain how close the group is getting to full
agreement. This type of polling throughout the process provides a
means for ensuring that everyone can be heard. (The chair can ask
those who disagree with a statement to state the source of their
differences).
Voting or polling can be done in two ways. The first is anony-
mous (i.e., using secret ballots), where individuals are not identified
with particular positions. The RAND/UCLA and Canada II confer-
ences used this approach, allowing those who disagree some protec-
tion from undue pressure to change a position. The second approach
is public voting, which may be more intimidating from the stand-
point of a group process, but which may allow the group to focus on
unresolved problems and force dissenters to defend their positions.
Further' voting can be done simply (i.e., yes or no) or on some
scale that reflects the level of agreement or disagreement. Canada
II, for example, uses the method developed at RAND/UCLA that
asks panelists to rate indications for the use of a procedure on a nine
point scale ranging from "extremely appropriate" to "extremely in-
appropriate." Another approach is to have panelists rate their agree-
ment with a particular statement or recommendation on, for example,
a f~ve-point scale ranging from "strongly agree" to "strongly dis-
agree." Obviously, yes/no votes provide a method that is simpler
and perhaps easier to implement, to increase the certainty of adopt-
ing or rejecting recommendations. The advantage of using scales is
that they allow panelists somewhat more latitude in agreeing or dis-
agreeing with recommendations. Further, this information can be
used to suggest a hierarchy or degree of certitude of those recom-
mendations that should be implemented immediately versus those
recommendations that require further study or that do not have strong
support from the panel. The RAND Coloration has developed soft-
ware for quickly entering the votes (through secret ballot) and pro-
cessing the ratings; thus, technology is available to allow for reason-
~ ~ · ~ .
-
~ ~ ~ , ~ ~
, %,.,
anle implementation of the more complex approach.
Writing the Consensus Statement
Most countries appear to use the U.S. model when the panel writes
the statement in an overnight session and presents the results the
next morning. Although this has the benefit of producing a state-
ment very promptly, it would seem to raise questions about the qual-
OCR for page 66
66
CONSENSUS DEVELOPMENT
ity of the product. As sleep deprivation and clear thinking are rarely
compatible, the danger arises that the consensus reflects the opin-
ions of those with stamina rather than representing a thoughtful
consideration and deliberation based on the evidence. Although the
introduction of an additional day or two into the process raises im-
plications concerning the availability of experts and increased costs,
the gains in the quality of the consensus findings may be well worth
the additional time.
There may be alternative approaches for conserving the amount
of available meeting time. Along with the literature review prepared
in advance, some countries begin drafting the consensus statement
before the meeting, giving panelists the opportunity to consider the
issues and focus on those areas of particular disagreement or contro-
versy. The Canada IT method developed at RAND/UCLA, for ex-
ample, requires that panelists rate indications for the appropriate use
of a procedure in advance using a modified Delphi method. Analy-
sis of the first set of ratings indicates areas of potential disagree-
ment, which are then given more time during the pane} meeting. By
shifting some of the responsibility onto panelists to prepare for the
conference, the use of the standard meeting time may be more effec-
tively targeted to those activities for which the group process and
interactions are important. One alternative is seen in The Nether-
lands, where CBO staff help to complete the statements at a meeting
that occurs one to three months after the consensus meeting devel-
opment conference. This provides time for writing and reflection
before concluding the process.
CONCLUSIONS
The processes by which consensus development conferences are
conducted can affect the value and validity of the final product.
Although the NTH mode} has been used extensively in other coun-
tries, it has never been shown to produce replicable results or to be
preferable to other models. Further, for all approaches, there is
always the potential for improvement in the process. The continued
use of consensus development conferences to achieve agreement on
the state of the science for a particular technology suggests a need to
enhance the inputs to the process and the methods by which issues
and experts are identified. An improvement in the inputs to the
process has several advantages and seems likely to enhance the final
OCR for page 67
FORMAT AND CONDUCT
67
product. Formalizing the process by which consensus conferences
are conducted would enhance the credibility of the statements by
making allowances for varying degrees of consensus and providing
an outlet for disagreement that does not detract from the overall
endeavor. Such formal mechanisms as voting or polling, whether
done anonymously or not, may allow for more controversial issues
to be considered. Finally,~ me current proclivity toward writing the
consensus statement in an all-night session seems to be unnecessar-
ily grueling and may have a negative effect on the final product. By
improving inputs at the planning stages, it should be possible to
develop schedules that allow for consideration of serious issues on a
more humane time frame.
REFERENCES
Andreasen, P.B. 1988. Consensus conferences in different countries, aims and per-
spectives. International Journal of Technology Assessment in Health Care 4:305-
308.
Battista, R.N., and S.W. Fletcher. 1988. Making recommendations on preventive
practices: Methodological issues. American Journal of Preventive Medicine
4~4~(Suppl.~:53-67.
Calltorp, J. 1988. Consensus development conferences in Sweden: Effects on health
policy and administration. International Journal of Technology Assessment in
Health Care 4:75-88.
Casparie, A.F., and J.J.E. van Everdingen. 1985. Consensus development confer-
ences in The Netherlands. International Journal of Technology Assessment in
Health Care 1 :905-912.
Chassin, M.R., J. Kosecoff, R.E. Park, C.M. Winslow, K.L. Kahn, N.J. Merrick, J.
Keesey, A. Fink, D.H. Solomon, and R.H. Brook. 1987. Does inappropriate use
explain geographic variation in the use of health care services? A study of three
procedures. Journal of the American Medical Association 258~18~:2533-2537.
Jacoby, I. 1985. The consensus development program of the National Institutes of
Health: Current practices and historical perspectives. International Journal of
Technology Assessment in Health Care 1~2~:420432.
Jacoby, I. 1988. Evidence and consensus. Journal of the American Medical Asso-
ciation 259~20~:309.
Jennett, B. 1985. First consensus development conference in United Kingdom: On
coronary artery bypass grafting, commentary by the chairman of the conference.
British Medical Journal 291:716-718.
Johnsson, M. 1988. Evaluation of the consensus conference program in Sweden: Its
impact on physicians. International Journal of Technology Assessment in Health
Care 4:89-94.
Kanouse, D.E., R H. Brook, J.D. Winkler, J. Kosecoff, S.H. Berry, G.M. Carter, J.P.
Kahan, L. McCloskey, W.H. Rogers, C.M. Winslow, G.M. Anderson, L. Brodsley,
A. Fink, and L. Meredith. 1989. Changing medical practice Trough technology
OCR for page 68
68
CONSENSUS DEVELOPMENT
assessment: An evaluation of the NIH consensus development program. Santa
Monica, Calif.:RAND Corporation.
Klazinga, N.S., A.F. Casparie, and J.J.E. van Everdingen. 1987. Contribution of
medical decision-making to consensus development conferences. Health Policy
8:339-346.
Kosecoff, J., D.E. Kanouse, W.H. Rogers, L. McCloskey, C.M. Winslow, and R.H.
Brook. 1987. Effects of the National Institutes of Health consensus development
program and physician practice. Journal of the American Medical Association
258:2708-2713.
Lomas, J., G.A. Anderson, M. Enkin, E. Vayda, R. Roberts, arid B. MacKinnon.
1988. The role of evidence in the consensus process, results from a Canadian
consensus exercise. Journal of the American Medical Association 259~20~:3001-
3005.
Mullan, F., and I. Jacoby, 1985. The town meeting for technology: The maturation
of consensus conferences. Journal of the American Medical Association
254(B): 1068-1072.
Office of Medical Applications of Research, National Institutes of Health. 1988.
Guidelines for the selection and management of consensus development confer-
ences. Bethesda, Md. Pamphlet.
Park, R.E., C.M. Winslow, K.L. Kahn, N.J. Merrick, J. Keesey, A. Fink, D.H.
Solomon, and R.H. Brook. 1986. Physician ratings of appropriate indications for
six medical and surgical procedures. American Journal of Public Health 76~7~:766-
772.
Perry, S. 1987. The NIH consensus development program a decade later. New
England Journal of Medicine 317:485488.
Perry, S. 1988. Consensus development: An historical note. International Journal
of Technology Assessment in Health Care 4:481-484.
Stocking, B. 1985. First consensus development conference in United Kingdom: On
coronary artery bypass grafting, views of audience, panel, and speakers. British
Medical Journal 291:713-716.
yang, J. 1986. The consensus development conference and the European experi-
ence. International Journal of Technology Assessment in Health Care 2:65-76.
Wortman, P.M., A. Vinokur, and L. Sechrest. 1988. Do consensus conferences
work.? A process evaluation of die NIH consensus development program. Jour-
nal of Health Politics, Policy, and Law 13~3~(Fall):469498.
Representative terms from entire chapter:
consensus statement
