Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 102
Evaluating HIV
Testing and Counseling Projects
In this chapter the panel outlines strategies for evaluating CDC's widely
disseminated program of funding HIV testing and counseling services.
Goals for this program are similar to those discussed for CBOs: behavior
change, HIV surveillance, and public education (CDC, 1988~. Strategies
for evaluating the counseling and testing program are somewhat different,
however, because the most credible research design for answenng, "Does
the program make a difference?" is not appropriate here: that design is
the randomized experiment in which some people (controls) receive no
services. In the case of counseling and testing for HIV, the panel be-
lieves strongly that having a no-treatment control group—that is, denying
access to information.that could have important consequences for peo-
ple's personal planning and medical management of infection would be
unethical.
BACKGROUND AND OBJECTIVES
In terms of expenditures $100 million in fiscal year 1989—CDC's
support of counseling and testing services is its largest AIDS intervention
program. At present, the Center for Prevention Services (CPS) channels
funds for such programs through 62 cooperative agreements with states,
territories, and a handful of major cities to support this widescale program.
Grantees provide HIV testing and counseling services free of charge in a
variety of health care settings.
Through the program, individuals are offered a dual AIDS inter-
vention: confidential (frequently anonymous) HIV testing and pre-and
102
OCR for page 103
HIV TESTING AND COUNSELING
~ 103
pastiest counseling. In addition, in cases of seropositivity, a third service
may be available: partner notification and referral. CDC encourages
infected individuals to notify their partners of their exposure risk and to
refer them to testing and counseling; if clients refuse, CDC recommends
that a well-tra~ned site practitioner do the notification. However, partner
notification is not mandatory.)
From discussions with program staff, the pane] learned that the
purpose of the counseling and testing program has evolved and may
still be evolving. One of the original motives for the program was to
divert individuals from using blood basics to learn Heir antibody status.
Accordingly, shortly after the licensure of He ELISA test in March 1985,
a program was initiated to deploy a series of what was called alternative
test sites around the country. Since then, the demand for counseling
and testing services has increased dramatically. In March 1986, CDC
recommended that infected but asymptomatic individuals be encouraged
to come In for counseling (CDC, 1986), and services were expanded
beyond alternative test sites to other health facilities.2 In the next year,
demand grew threefold when the Surgeon General recommended testing
for heterosexually active individuals and recipients of blood products.
There has been widespread support in the public health community
for expanded programs of voluntary testing for all those who may have
been exposed to HIV (see, for example, IOM/NAS, 1988:74~. ~ estab-
lishing new HIV testing and counseling sites, priority has been given to
projects that serve those segments of the population Hat are most likely
to be infected or that engage In behaviors that risk HIV transmission
(CDC, 1987:510~. Halfway through 1989, there were more than 1,600
counseling and testing sites nationwide, and that number is expected to
grow to 2,000 by the end of the year.
HOW WELL ARE SERVICES DELIVERED?
As noted above, the overall objectives for the counseling and testing
program are surveillance, promotion of behavior change to reduce the
risk of infection, and public education. In addition, one of CDC's internal
documents on process performance indicates that "quality" counseling is
1 For example, the 1987 CDC (1987: 513) guidelines for counseling and testing note that persons
who are antibody positive should be instructed in how to notify their partners and to refer them for
counseling and testing. If they are unwilling to notify their partners or if it cannot be assured that
their partners will seek counseling, physicians or health deparanent personnel should use confidential
procedures to assure Mat their partners are notified.
2The shift in emphasis from testing to counseling was accompanied by a change in nomenclature to
refer to the sites as counseling and testing sites.
OCR for page 104
104 ~ EVALUATING AIDS PREVENTION PROGRAMS
a goal, which introduces a new concept to be evaluated.3 Finally, from
He history of CDC's repeated recommendations to encourage various
populations to take advantage of counseling and testing, along with
the ever-widen~ng deployment of testing sites, the pane} inferred Hat
accessibility was also an implicit goal of the program.
To abbess these goals, the pane! believes that five aspects of service
delivery need to be evaluated. To do so, a number of information
gathering me~ods can be used.
The pane' recommends that data be gathered from multiple
sources including testing sites, clients, groups at increased
risk of HIV infection, and independent observer~to eval-
uate five aspects of service delivery: the adequacy of the
counseling and testing protocol, the adequacy of the coun
seling that is actually provided, the proportion of clients
that complete the full protocol, the accessibility of services,
and the nature of the barriers, if any, to clients seeking and
completing counseling and testing.
The rate of completion of the program and the identification of banders
to participation in the program are subsumed under "adequacy" and
"accessibility. "
We use "adequacy" to mean correspondence with client needs. In
terms of the testing protocol, client needs include confidentiality; rea-
sonable waiting periods; secure linkage between counseling and testing;
provision of test results; and, possibly, partner notification. In terms of
counseling, client needs include support; risk assessment; and accurate
and appropriate information about the transmission of the virus, risk
factors, risk reduction behaviors and techniques, coping skills, and the
meaning of test results. In the case of seropositivity, client needs also
include information and counseling about the medical and psychological
management of infection and partner notification. We realize that as-
sessing adequacy and accessibility involves judgment on He part of an
evaluator; nonetheless, we believe a system of cataloguing the fulfillment
of needs can be implemented. CDC has in fact developed a prototype
"HIV Disease Intervention Skills Inventory" for managers that could be
useful in the evaluation of adequacy (CDC, n.d.).
There are four sources of information on the venous aspects of ser-
vice delivery: the administrators and staff of testing and counseling sites;
the clients who use the service; specific population groups who should
sin addition to any "calculable results" from He program, counseling efforts are to "be judged by
the quality of the process performance" (CDC, n.d.).
OCR for page 105
HIV TESTING AND COUNSELING ~ 105
TABLE 5-1 Four Sources of Formation about Delivery of Counseling and Testing
SOURCES OF INFORMATION
SERVICE ASPECT Sites
Population
Clients Groups Observers
Protocol adequacy X X
Counseling adequacy X X
Client completion X X
Accessibility X X X
Client barriers X X
use services (but may not); and independent observers who visit the sites.
Table 5-1 identifies the sources of information that are most likely to be
useful for evaluating each feature of HIV testing and counseling service
delivery. Each of these sources of information can be translated more
or less directly into a method of data collection or study design. In the
following sections, the panel suggests several designs for collecting data
from each of these information sources. Note that these designs are not
mutually exclusive; rather, they are complementary, as each provides a
different perspective on the adequacy and accessibility of services.
A Site Services Inventory
A typical setting for counseling and testing is a local health deparunent,
but services are also offered In institutional facilities, health clinics for the
treatment of sexually transmitted diseases or drug use, family planning
clinics, and other settings. Because clients of these over sites often have
non-HIV related reasons for their visits, they may not be motivated to
return for test results and pastiest counseling. Thus, the sexing in which
testing and counseling services are delivered may be a significant factor
to be taken into account in analyzing data collected from project sites.
As a first step, the panel believes CDC should prepare an inventory
of the venous services that are delivered by HIV testing and counseling
sites. Although recipients of CDC funding for testing and counseling
are required to provide quarterly summary data about their services, we
believe these data are insufficient to descnbe the range of testing and
counseling activities now being undertaken at the 1,600 CDC-supported
sites across the county. The 62 grantees funded by CDC have no uniform
method for reporting data from their counseling and testing sites, and the
level of detail provided on any particular service may be inconsistent from
OCR for page 106
106 ~ EVALUATING AIDS PREVENTION PROGRAMS
one grantee to the next. These inconsistencies preclude cross-tabulation
or any detailed analysis of how and to whom services are delivered.
A promising solution to these problems would be the widespread
adoption of an enhanced version of the data management system devel-
oped by CPS last year. The central feature of this system is the REV
Counseling and Testing Report Form, an electronically scannable record
of each visit to a testing and counseling site (see Figure 5-~. The form of-
fers the advantages of uniform reporting and relatively easy data analysis.
It assigns an identification number to and elicits data on each counseling
intervention, recording the type and date of the site visit as well as visitor
demographics and risk factors. These data can then be used in four ways:
to link test data with type of service site; to identify the demographic
distribution of clients and determine trends ~ client utilization of the
site; to link test data with risk behavior and demographic data to assess
trends in seropositivity; and to link test results with pastiest information
to determine trends in return rates for results, counseling, and partner
notification and referral. The data gathered with this form would also
allow analysis of variations among project sites and geographic regions.
The pane! believes that He value of this inventory can be enhanced
and that it would be valuable to augment the current form to collect
information on other relevant variables (such as counselor characteristics,
length of counseling session, whether the session is an initial or repeat
visit).4 Furthermore, the pane] believes that the required use of this
form by all HIV testing and counseling sites funded by CDC and other
government agencies would permit the development of uniform and
manipulable data bases that could be used for the evaluation of testing
and counseling projects. Some states have developed and are using
alternative forms and are already building data bases. To avoid requiring
these states to modify their efforts, it may be feasible for CDC to furnish
technical assistance to make state data sets compatible with the federal
form.
Client Surveys
The mere presence of clients indicates Hat He testing and counseling
intervention has been successful in attracting people to receive services.
The gross number of individuals served, however, does not tell us whether
4 The form could also provide an indicator of the socioeconomic status of clients. It has been postulated
but not proven that, increasingly, the AIDS epidemic is becoming lodged in the most disadvantaged
segments of the American population. Until now, such arguments have been based on trends in the
race and ethnicity of new AIDS case~which does not provide a wholly appropriate analysis. To
track trends in the socioeconomic status of persons served by counseling and testing projects, the
panel suggests that a question on education level be added lo the form.
OCR for page 107
HIV TESTING AND COUNSELING | 107
~ U5 GOVERNMEN7 PRIFITING OFFICE:1990—734~539 ~
HIV COUNSELING AND TESTING REPORT FORM A
IDENTIFICATION NQ
_ _ _
PROJ SmE DATE OF INmAL VISIT LOCAL USE ONLY
AREA SITE TYPE NUMBER MONTH DAY YR
m m rl I I c~ mm rl I I I I I
G) (D ~) O (y) ~) C HiV CTS ~) (3D~ C JAN O~) ~C (3D Q) 0(31)~)(0~)
G) ~) G) ~) O) O CE) STD O 2 3 O FEB (D ~) ~D CD ~ O CO O (D CD
(~:) ~ Q OD ~) ˘) GE) DRUG TRMT O) O) ~) O MAR ˘) ~) 48D (D ~) CD (D (O ~) ~)
3 ~ ~ ~ Cl) GO ~) FAMILY PL ~ GO Gl) O APR (~ (D CED O O (:O ~ CD Gl:
O) CI) ~ ~ O ~) G) PRENAT/OB Q (:iD ~) O MAY ~) 4~ O Q 09 0 ~ `33 ~ ~
CD CD ~) ~) O (~) OD TB O CD ~ O JUN 0 ~ C CD ~ (~) CD CD
~ ) 5D ~ G[) O) OTHER HD G) (O ~ O JUL GD (O (D G) CID (30 (O 60
CD CO ~ (O CD (D ~) PRISON l:D O) tI) O AUG CI) O (D (D ~ C ~)
3 ~ O O O G) ~:) COLLEGE (O G) ~ O SEP OD G) (D G) CID CD G) (~
C ~) (E) (D CD~) 6i9PRIV MD O)(ID~) OOCT O) C~ O ~)6D~
G~ OTHER O NOV
~) UNKNOWN O DEC
RESIDENCE
STATE COUNTY ZIP CODE AGE
m rl I I I I I I I I m
=~) CO(D~ ~QG)Ci:)
=m =~= ~CDC~= ~CD
~ GD ~ CD CD Gi) GO ~ GD ~ C~ GO
CD~ ~o mmo~ ~
mm ~CD~ =~CD=~ ~CD
~ ~ ~ 6DCD
CD(D =~(D om~a)m mm
m CD~ C~
~<~ ~C30~ ~C~
SEX
(3 MALE
~ FEMALE
RACE/ETHNICITY O
Q WHITE
CE) BLACK
Q) HISPANIC. F8R~ I tST
O ASIAN/PACIFIC ISL COUNSELED
C AM INDIAN/AK NATIVE
~) OTHER
(D UNDETERMINED
· IF HISPANIC, SKCIFY O
~MEXICAN/MEX AMER
~) PUERTO RICAN
CO CUBAN
G) OTHER
CD UNDE I tRMINED
REASON FOR VISm
(mark all that apply)
O REQUESTING HIV TEST
O FOLLOW-UP TO STD VISIT
O STD EXAM,fTREATMENT
O REf BY HIV+ SEX PARTNER
O REF BY STD SEX PARTNER
O REF 8Y HEALTH DEPT/HIV
O REF 8Y HEALTH DEPT/STD
O REF BY PMD/BBtHOSP
O REQ IMMIGRATION
O PRENATAL
O TB INFECTION/DISEASE
O SYMPT HlV/AtDS DISEASE
O ASYMPT, WORRIED A80UT AIDS
O OTHER
O NOT STATED
O UNKNOWN
MOhml
O JAN
O FEB
O MAR
O APR
O MAY
O JUN
O JUL
O AUG
O SEP
O OCT
(3~) NO O NOV
O O) YES O DEC
O ~)NO
CD YES
P05 I I tST
COUNSELED
~) NO
IJ G) YES
REFERRED
FOR ~ TEST
RISK EXPOSURE GROUP
(mark all that apply)
O MAN WHO HAD SEX W/A MAN
O IV DRUG USER
O PERSON WITH HEMOPHIL]A
O BLOOD RECIPIENT. 1978~85
C HETEROSEXUAL
O SEX PARTNER OF
HOMOSEXUALfBISEXUAL
G SEX PARTNER OF IV DRUG USER
O SEX PARTNER Of PWA/+HN
O SEX PARTNER OF PERSON WITH
HEMOPHILIA
C EXCHANGED DRUGStMONEY
FOR SEX
O NO KNOWN RISK EXPOSURE
DAY YR
m m ~ DAn- POS-l I EST COUNSELED
~) (3D ~}c
oim ~
CD GD ~D PARTNER NOT~CAnON (po~ Only) O
PATIENT ALREADY REfERRED PARTNERS
CD ~ PATIENT WLL REFER ALL PARTNERS
G) GO HD W~L REFER ALL PARTNERS
2) GD PATtENT/HD EACH TO MAKE REFERRALS
ALL PARTNERS UNKNOWN/UNLOCATA3LE
PATIENT DECLINES PARTICtPAT10N
~) SUBJECT NOT RAISED WITH PATIENT
RESERVED
1 2 3 4 5 e 7 8 9 10
REFUSED HIV TEST O O YES O O O O O O O O O O
~) ~ (31) (O ~ C3D O) ~) GD (3D
CD ~ 6) Q ~) CD O) G) CD 6)
LAB TIEST RESUL~ 2 ~ ~ ~) (D ~) CO CD ˘) CD
WESTERN OTHER C :) GO CD CD CD ~ GO CD Cl) t3
ELtiA BLOT CONFIRMATORY Q <~~ Q (D (~ (ED
O O O ~ ~ ~ ~ ~ ~ ~ ~ ~ im
NEGAnVE (D NEGATIVE (3D NEGATNE G) (~ ~ ~) GD (D (O C3~) '~ ~
CD REPEATEDLY ~ POSITIVE ~ POSITIVE ~ C2) O) 0) ~ CD iO) ~ ~ CD
REACTNE ~ INDETEF.=NATE ~) G) 6D ~ C~ OD CO ~) G)
(O (O ~) (O C~ CD i~ ~ D ( (O
FIGURE 5-1 HIV Testing and Counseling Report Form
OCR for page 108
OCR for page 110
OCR for page 111
OCR for page 112
OCR for page 113
OCR for page 114
OCR for page 115
OCR for page 116
OCR for page 117
OCR for page 118
OCR for page 119
OCR for page 120
OCR for page 121
OCR for page 122
OCR for page 123
Representative terms from entire chapter:
counseling services
108 ~ EVALUATING AIDS PREVENTION PROGRAMS
IDENTIFICATION NO.
-
CO
CD
CD
c3 ~
CD CD
CD ~
DATE PAST I EST
COUNSE' EN
MONThI DAY YE
~ mm
O JAN CD A) @;
O FEB 6) Ci) ~
5 MAR If) AD
—APR CD~ GED
O MAY A)
O JUN CD
O JUL ED
O AUG CD
O SEP (A
O OCT (A
O NOV
O DEC
LOCAL USE ONLY
(9 ~
G) 2
1 1 ~ 1 1 1 1 1 1
(iD ~ (D ~ ID
~=CDCOG)~
COW
CD~COGOCDCOGOCD
ID
~CDCED~CE)
~6DGD~
=~a)(D
HIV TESTING AND COUNSELING | 109
the intervention is providing satisfactory services to the groups that need
them. A survey of the clients of a testing and counseling site can:
· provide specific information on the accessibility of He site
and its services to specific populations;
· reveal problems with the intervention as it is currently de-
livered;
· elicit information about users' experiences in obtaining the
service, such as how long it takes to get an appointment or
to get test results, etc.;
· gather data on the scope of the counseling clients receive:
the extent of pertinent information provided about HIV
infection and AIDS, the time spent with and the emotional
support lent by the counselor, the nature of referrals for
medical and over support; and
gather information about whether the respondent completed
the testing and counseling protocol, and if not, why.
While the reliability and validity from such surveys must always be con-
sidered, information from such surveys might also be helpful in design-
ing more inviting and accessible settings for testing and counseling (e.g.,
"attractive" physical surroundings, "convenient" locations for services),
understanding what aspects of the pretest counseling session encourage
clients to return to learn tests results, assessing the optimal content and
timing of a pastiest counseling session and the provision of referral ser-
vices, and specifying He profiles of the more effective counselors for
different circumstances.
.
Population Surveys
Another way to evaluate how well services are delivered is to conduct
surveys of populations that include potential and actual clients to deter-
m~ne whether counseling and testing services are accessible to all who
need or want them. This strategy can be used to evaluate barriers to
access. The surveys can be directed toward the general population, to-
ward particular neighborhoods or communities, or toward high-risk or
hard-to-reach groups. They can be used to measure the proportion of the
specific population that has had experience with counseling and testing
and the proportion of He population that wants services but cannot get
Hem (or has chosen not to seek them). Data from such surveys could
be analyzed according to demographic and risk factors to identify groups
that are not adequately served and the barriers that need to be overcome
to make the services more accessible to those groups.
110 ~ EVALUATING AIDS PREVENTION PROGRAMS
Such surveys present other opportunities as well. They could be
used to identify the testing and counseling services that are desired or
expected by a given population and could help project administrators
better understand the service needs of these individuals. They might also
afford insight into the group's awareness of the availability of services
and the perceived cultural relevance of the project to users and nonusers
alike. Based on this information, efforts might be made to publicize
the availability of services and to modify as necessary such project
characteristics as location, hours, site design, esthetics, informational
materials, and counselor sensitivity. This kind of survey could also gather
users! information on the group's concerns about the confidentiality (or
anonymity) of HIV testing and counseling and over concerns or fears
about Me test procedure.
Case Studies Using Direct Observation
The direct observation of interactions at testing and counseling cen-
ters is the panel's final suggested study design. This design could be
implemented by both nonparticipant observers as well as "professional
customer" participants.5 This method mainly provides qualitative data,
although quantitative data can be gathered as well.
The pane] is aware that CDC already employs regional monitors and
public health advisers to conduct oversight and quality assurance activities
for the counseling and testing centers.6 Nevertheless, the pane} believes
that case studies using direct observation might identify those factors
that produce testing and counseling environments that are particularly
supportive of clients and that provide effective contexts for educational
messages. In selecting the project sites for case studies, choices should be
spread across different types of facilities (e.g., public health deponents,
clinics for sexually transmitted diseases, drug treatment centers, etc.)
and regions of the country, both geographically and by level of HIV
prevalence.
Methodological Issues
As noted above, the four separate options for assessing service delivery
are not mutually exclusive. Indeed, the approaches suggested are com-
plementary to one another, and all might be undertaken to yield He most
5As noted in chapter 1, the panel believes that project administrators should be given advance notifi-
cation that professional customers will be visiting their sites for counseling and testing services, and
prior consent should be solicited before this method of data collection is used.
fits document on process perfollllance standards covers a series of steps its counselors are to talce in
providing counseling and partner notification, and it provides a skills inventory to be used by managers
in evaluating counselors performance (see CDC, n.d.).
HIV TESTING AND COUNSELING ~ 1il
comprehensive information. Gaps in coverage by an individual study
design can be filled in by data from research using another design.
Although a services inventory could show who is using testing and
counseling services and could monitor trends in this use over time, some
caution is warranted in interpreting inventory data. The main method-
ological problem will be the accuracy of the project site's reporting.
Inaccurate reporting may occur through errors and it is also possible
that inaccurate reports may be made purposely to convey a false picture
of site activities. But the most plausible threat to accurate reporting
will probably be the burden the forms place on the counselors. In the
course of a busy day win much "real" and pressing work to do, it is
virtually inevitable that filling in forms will not receive high priority. Fur-
thermore, where testing is anonymous, repeat testers could be counted
multiple times. This problem can be avoided by adding a question to the
scannable form that asks whether a client has been previously tested.
A major concern about client surveys is Me reliability and validity (or
mearungfuiness) of the measurements that are obtained with this memos.
Clients who are surveyed, for example, may have little or no experience
with counseling. Consequently, they may have unrealistic expectations of
what and how services should be delivered, expectations that may color
Heir responses.7
Surveys can be conducted to gather data from two levels of society:
specific, high-nsk groups and the general population. To conduct surveys
of high-risk populations, probability samples should be used whenever
possible. When such a sampling frame is not feasible, replicable con-
venience samples could be used (see the discussion in Turner, Miller,
and Moses, 1989:150-1531. However, this latter method will not provide
estimates that will be generalizable to the population of persons in the
specific high-nsk groups. As discussed In the section on client surveys,
the reliability and validity of the responses obtained in group surveys
will always be a matter of concern. The panel notes, for example, that
respondents in such surveys may have unrealistic expectations of what
and how services should be delivered.
The general population is of lesser interest than specific high-risk
groups, but a large enough sample can provide important information
about certain subpopulations. To this end, He panel suggests that CDC
take advantage of the National Health Interview Survey (NEDS) sponsored
by the National Center for Heals Statistics. As described in Chapter 3,
He NHIS is a weekly household interview survey of a probability sample
7 For discussions of these complexities, see Bradburn and Sudman (1979); Smith (1984); and lower
and Martin (1984).
112 ~ EVALUATING AIDS PREVENTION PROGRAMS
of the civilian non~nstitutional~zed adult population of He United States.
Since August 1987 the NITS has included questions about respondents'
knowledge of HIV transmission and their experience with HIV testing;
it has also collected limited information on behavioral risk factors.
The pane! recommends that the NHIS be periodically aug-
mented with several questions about accessibility and barri-
ers to HIV testing and counseling services.
Given its large size (approximately 50,000 households a year), the NEDS
can provide samples of reasonable size, even of relatively rare pop-
ulations, as long as they are found in households. For example, the
September 1988 NITS estimated that approximately 3.5 percent of He
total U.S. population (exclusive of those tested during blood donation)
expected to have an HIV test in the next 12 months.9 Given the an-
nual sample size, this means that 1,750 respondents intended to seek a
voluntary HIV test.
To conduct case studies, a site visit team might conduct open-ended
interviews with key project staff and 5 to 10 clients. The interviews
should focus on counseling and testing activities and materials, opin-
ions about activities and materials, and the identification of key content,
process, and organizational elements. Alternatively, with prior informed
consent from site administrators, "professional customers" can pose as
clients of the projects to gather information unobtrusively. A mix of
seronegative and seropositive "customers" could be recruited to gather
information about the adherence of sites to counseling protocols much In
the same way that public health monitors do. For the purposes of case
studies, the pane! notes that projects are evolving entities; changes may
occur in personnel, organizational structure, project instrumentation and
goals, and so on. Because of such developments, case studies cannot
be conducted once and considered done. Instead, frequent studies are
necessary to ensure good results.
The pane} members did not agree about whether such site visits would
be well received. Some members believe that site staff would welcome
the opportunity to demonstrate their projects; others believe that staff
would feel overly scrutinized. If there are strong negative reactions on
the part of project staff, it may be necessary to spend considerable time
8 The panel understands from program staff that planning is under way at the National Center for Heals
Statistics to add questions of this nature.
9Fith (1989:8) reports that 7 percent of the sample responded "yes" to the question, "Do you expect
to have a blood test for the AIDS vines in the next 12 months?" Moreover, 51 percent of those who
responded "yes" said the test would be "voluntarily sought" when they were asked, 'GENII it be part of
a blood donation, voluntarily sought or part of some other activity that requires a blood sample?"
,1
HIV TESTING AND COUNSELING ~ 113
convincing local staff of the need for case study research. If local staff
cannot be satisfied that the research is needed and beneficial, it may
be quite difficult or impossible to conduct. If the alternative of using
professional customers to gather information is considered, it may be
desirable to ask for a site's informed consent before that site is funded
for HIV testing and counseling services.
Resources and Aspirations
An inventory system requires record-keeping by personnel at the coun-
seling and testing sites. It also requires a centralized professional staff to
ensure the completeness of reporting, conduct data analyses, and dissem-
~nate results. Other costs of the inventory option should be relatively low
because He data management system is based on personal computers,
and the use of a scannable form minimizes labor costs for data entry. The
pane! believes an inventory system could be implemented program-wide
within 6 monks.
The aspirations for this type of evaluation research are somewhat
limited. Although a services inventory would provide data on individuals
who avail themselves of services, it cannot identify people who need or
want those services but who do not receive them. Another limitation of
the system is that it does not provide data on the counseling and testing
services provided by private physicians or clinics, blood banks, insurers,
and other non-CDC funded sources. This lack is regrettable because such
inflation could bear on an evaluation of CDC-funded services, such
as whether a client completes a protocol at CDC-funded sites, or seeks
services elsewhere.
The major advantage of client surveys is that they provide informa-
tion on most of the program aspects that must be assessed to determine
how well testing and counseling services are being provided. An addi-
tional advantage, when compared with other methods of data collection,
is that information can be gathered on over intervention activities to
which clients have been exposed. Furthermore, the client survey option
is one of the least expensive methods of obtaining information about test-
ing and counseling services. Nevertheless, it will require the involvement
of personnel who are trained in survey research design and faTniliar with
its methodological problems.
To survey the general population, the NHIS could be expanded
at periodic intervals to measure people's needs for H[V testing and
counseling services on a national basis. (The NHIS cannot be used for
local information.) This option would require some staff time for data
tabulation and analysis on He part of the personnel responsible for the
114 ~ EVALUATING AIDS PREVENTION PROGRAMS
household survey, but costs should be relatively low. Population surveys
are somewhat Innited, however: for example, the THIS does not pennit
access to homeless or institutionalized populations, which may include
higher proportions of individuals at risk for HIV Han are included in the
household population.
For a number of reasons, surveys of samples of high-risk populations
are more expensive and more difficult Han adding questions to the THIS.
These surveys are labor intensive; In addition, they must be repeated at
regular intervals In order to monitor changes in the need for counseling
and testing services. Furthermore, this kind of research requires highly
trained personnel to design and administer the surveys.
Conducting case studies requires special skills and knowledge. Ex-
pertise in AIDS prevention and in evaluation design is desirable, as is
knowledge of the particular risk factors that are accessed In the sewing
being studied (e.g., drug treatment centers). A team of two or three
site observers is preferable to a single observer because of the range of
knowledge desired ---evaluation methodology, counseling expertise, and
any other site-specific expertise. Using more than one observer also
makes it possible to assess He reliability of several reports. Site visits
of 3 or 4 days would be required for each study site. Major costs ~n-
clude observer salaries and their travel expenses. Each case study would
require about 40 person-days per site. Case studies call provide a rich
and ~n-depth look at some aspects of service delivery for a subset of test-
ing and counseling sites, which will in turn be important for developing
studies Hat evaluate comparative effectiveness.
OPTIONS FOR EVALUATING WHETHER HIV TESTING AND
COUNSELING SERVICES MAKE A D1FIERENCE
The panel weighed several options for addressing the question, "Does
He policy of providing free HIV testing and counseling services make a
difference?" As noted above, He panel seriously considered the feasibility
of randomized tests with a no-~eatment control group. One approach
would be to use a randomized experiment at the site level in which
individuals who sought services were randomly assigned either to an
intervention condition or to a control condition in which they received
services from alternative sources or no services at all (e.g., they were put
on a waiting list). The individuals in both groups would be measured
and compared on the relevant outcome variable to test the effects of
treatment. Although conducting such an experiment is the usual desired
strategy to evaluate effectiveness, the panel rejected it as unethical alla
infeasible. In the context of a deadly epidemic, it is indefensible to
HIV TESTING AND COUNSELING ~ 115
withhold this treatment in the interests of conducting an experiment
from any individual who desires it. In addition, follow-up for such a
group would be extremely difficult as they would have little incentive to
cooperate, and locating a diversely situated group would be difficult.
The panel also considered a variation of a randomized experiment
that would capitalize on delays in the implementation of projects, but
concluded that such an experiment would not usually be feasible. This
design variation assumes that lags occur in the deployment of counseling
and testing projects because of scarce resources; as funding becomes
available for some projects, sites can be randomly assigned to receive the
intervention or to continue waiting. However, such a design is not very
practical. F~rst, there are already a large number of sites throughout the
county, and it is unlikely that a waiting list of homogeneous centers Is
available. Second, the recruitment of control sites would be problematic
because sites on the waiting list would have to be offered strong incentives
to participate in data collection.
Altemative strategies for assessing effectiveness (e.g., simple before-
and-after designs that establish that a change has occurred or not oc-
curred) do not suffice because they do not condom for rival explanations
of changes In behavior, knowledge, or serostatus. Over competing ex-
planations for such changes may include natural history (the adoption
of change regardless of exposure to counseling and testing), the self-
selection ot program participants, and the effects of conducting research.
The suitability of such research designs for answering the question,
"Does it make a difference?" is so low as to invite the investment of
evaluation resources In more tractable areas, especially as the value of
counseling and testing has been so widely accented. Indeed, the panel
~ .
noted some presumptive evidence mat mv testing and counseling do
have a positive effect. There is, for example, increasing evidence that
testing can result in individual medical benefit among persons infected
`',i~h HTV he e.n~hlin~s them to monitor their immune function and Initiate
early prophylaxis tor Eneumocyst~s carinii pneumonia. Furthermore,
an individual's knowledge about serostatus can be an important factor
malting decisions about sexual behavior, needle sharing, and childbearing.
For example, one study found that gay and bisexual men who were tested
for HIV and received pretest and pastiest counseling were more likely
than those who did not to reduce their incidence of unprotected anal
intercourse (Coates, Morin, and McKusick, 19871.
J ~ —~ ~
~ — .
When HIV testing is performed, the panel believes that testing should
be accompanied by counseling, both before the test is administered and
after the test result is given to the individual. The need for such counsel-
116 ~ EVALUATING AIDS PREVENTION PROGRAMS
ing and the ethical and practical motivations for providing it are discussed
elsewhere (see, e.g., TOM/NAS, 1986, 1988; Presidential Commission on
the Human Immunodeficiency Virus Epidemic, 1988:73-75~. At a m~-
mum, pretest counseling is ethically mandated to ensure that individuads
give informed consent. Posttest counseling is appropriate to ensure that
individuals who are distressed by the results of their tests are comforted
and that all individuals are warned about the n~.k of tr~n~mi~c~inn Huh
their future behaviors.
~ ~~~ an, ~ ~^v4444~1 ~~ V~~11
Although the pane] found that the question of effectiveness should
not be experimentally tested, it did not find the question uninteresting.
On the contrary: the pane! discussed current research efforts that study
the sequelae of HIV testing.~° One impetus for such studies has been the
emerging indications that testing may have negative as well as positive
effects (see Chapter 21. Although the evidence is sparse, these studies
point to the need to monitor potentially negative as well as positive effects
of HIV testing and counseling.
WHAT WORKS BETTER?
The heart of the question "What works better?" is how to max~n~ze
the beneficial] effects of testing arid counseling. The way to learn what
these effects are is through well-controlled studies that test two or more
approaches to delivering the intervention. As recommended in Chapter
I, the panel's preferred strategy for comparative tests is randomized ex-
periments. For the "What works better?" question, the control group
is assigned not to nontreatment but to an alternative treatment. So, for
example, each individual who agrees to participate might be randomly
assigned to one of two (or more) programs of counseling and testing
that are thought to be effective but whose relative effectiveness is un-
known. Because the groups are composed randomly, the comparison of
outcomes such as client return rates for clients receiving regimen A
or regimen B is Hen a fair one.
In some circumstances, REV seroconversion may be a helpful out-
come measure for evaluating He effectiveness of different counseling
and testing projects. Yet the use of more proximate outcomes is desir-
able because seroconversion will not be informative regarding behavioral
.
10The particular approach taken in these research efforts is the natural history study. Using longitudinal
cohorts, researchers have attempted to estimate the effect of Besting and counseling, compared with
ocher factors in a person's life, on behavioral change. Such studies of gay men and IV drug users are
currently under way. For example, among the cohorts of gay men, seroconversion rates and behavioral
changes in men who have not been tested for HIV are compared with: (1) those who have been tested
but do not know their antibody status and (2) those who have been tested and do know their status.
Unfortunately, natural history studies do not lead to fully adequate, testable models of behavior, but
when there appear to be consistent effects, those effects should be noted.
HIV TESTING AND COUNSELING ~ 117
change among persons whose initial test result is positive. Similarly,
In populations In which HIV is not heavily seeded, seroconversion may
be a rare event even though the population frequently engages in risky
behaviors. Thus, experiments can assess how well different versions of
an intervention work to increase a person's willingness to return to a site
for test results, to increase his or her knowledge of risks, or to reduce
risky behavior. Furthermore, experiments can assess the effectiveness of
venous regimens in reducing identified side effects, such as psychological
distress.
With clear outcomes such as these in mind, alternative approaches to
testing and counseling, based on theory and the perceived effectiveness of
past approaches, can be evaluated through randomized experiments. After
a brief discussion of the unit of assignment and experimental regimens,
the next section presents appropriate study designs for answering the
question, "What works better?"
Randomized Experiments of Alternative Treatments
Unit of Assignment
As noted in Chapter I, the unit of assignment in a field experiment may be
a large organizational unit such as a community, a smaller organizational
unit such as a project (i.e., all of the clients of a project), or individual
participants. There are several factors involved in He choice of treatment
units (see Chapter I). The pane! suggests consideration of three types of
assignment: it
· random assignment of individual testing and counseling
sites to alternative regimens;
· random assignment of project staff members at a given site
to the use of alternative regimens, and
· random assignment of individual clients to alternative regi-
mens at a site.
In cases of random assignment of sites, we recognize that some facil-
ities, such as those whose primary mission is not HIV-related, may not be
amenable to being randomly assigned to provide different interventions.
However, senice providers In a large city with several counseling and
testing sites may be more flexible and should be encouraged to partici-
pate In controlled expenments. When the preferred design is He random
~-
11 See Tu~ner7 Miller9 and Moses (1989:Chapter 5) for a discussion of precedents in other areas regard-
ing the random assignment of entities (e.g.7 sites) tO altemative regimens.
118 ~ EVALUATING AIDS PREVENTION PROGRAMS
assignment of individuals within a project to alternative regimens, the
panel believes Mat at least two or three projects in different communities
should be encouraged to cooperate in uniform tests. Because detectable
relative differences may be small but important, the number of clients
and sites involved in the tests must be large enough to estimate relative
differences with confidence.
Experimental Regimens
There are a variety of experimental regimens that might be tested, but In
designing randomized experiments of alternative counseling and testing
treatments, it is best to test alternatives in which one regimen is not
obviously better than the others. In the three examples noted In this
section (below), the unit of analysis varies. These regimens test the effects
on behavior of alternative modes of providing counseling and testing.
Some of the alternatives fall into He category of structural variables.
For example, the setting in which testing and counseling is delivered
may have an effect on whether an individual returns for test results
and foDow-up counseling. As noted above, a client's initial purpose in
visiting a site may not be HIV testing; similarly, some sites and their
staffs may be geared primarily to providing services other than counseling
and testing (a drug ~eatrnent facility is one such example). Particular
service delivery aspects of sites may also produce different effects. For
example, Rugg and colleagues (1988) found that higher return rates for
REV test results were associated win such site characteristics as a shorter
wait for testing and the comfort and nature of He setting. The number
of sessions as well as the content of counseling may have an effect on
cognition and behavior. Other process variables Hat may influence return
rates involve the adequacy of services In teas of emotional support and
medical service referral.
The pane! recommends that evaluations of "What works
better?" focus on the comparative effectiveness of testing
and counseling services that (~) are delivered in different
settings, (2) have different content, tIuration, and intensity,
and (3) are accompanied by different types of supportive
services.
Service Delivery Setting. As discussed above, the accessibility and
suitability of testing and counseling projects are critical issues. Projects
now exist to serve gay men, IV drug users in treatment, and users of
general public health agencies such as STD or family planning clinics.
These projects are widely distributed, but they are not necessarily acces-
sible in all communities or equally accessible for all types of individuals.
HIV TESTING AND COUNSELING ~ 119
In addition to assessing whether particular projects serve more of the
individuals who desire testing than other projects, it is important to un-
derstand whether particular service settings are more effective contexts
for providing testing and counseling to different types of clients.
Content, Duration, and Intensity of Counseling. The experimental
regimen could consider a number of potentially more effective ~nterven-
tion approaches, including more frequent, longer, or intensive counseling.
The content of counseling messages could be vaned, too: for example,
the elimination of risk behaviors or the adoption of protective behaviors
could be stressed. Other enhancements may include the use of support
groups, the involvement of partners, and outreach services for individuals
who continue to engage In high-risk behavior. As the standard protocol
for counseling and testing evolves and is improved, experiments can ex-
am~ne Be increased effectiveness of enhanced programs compared with
the costs of their implementation.
Additional Services. The third major area for exploration is the
effect of providing services beyond the basic counseling and testing
intervention now being offered. Increased relapse prevention services
for IV drug users In treatment who become seropositive are a good
example. A common reaction to stress by IV drug users is to seek
and use drugs for stress reduction. Relapse prevention projects address
this issue In general and could include components that are specially
geared toward seropositive clients. Another example of potentially risk-
reducing services is psychological counseling (beyond the counseling
provided with HIV test results), which might help diminish adverse
stress reactions.
A creative evaluator is likely to identify many more Interventions
whose effectiveness can be assessed using alternative regimens (e.g.,
videotapes, group sessions, cognitive interventions, etc.~. Yet not all sites
will be suitable for testing augmented regimens. Consistent with the
overall recommendations, the panel believes that sites should be selected
for randomized teals on the basis of their willingness to cooperate and
He potential effectiveness and replicability of the augmented intervention
programs they would offer.
Methodological Issues
Conducting randomized experiments can present various problems, in-
cluding cost, impediments to random selection of treatment units, and
difficulties In collecting complete data from participants and to retaining
participants in He study.
120 ~ EVALUATING AIDS PREVENTION PROGRAMS
As noted in Chapter 1, offering alternative services to individuals
within a project deserves a special note of caution. Problems can arise
in that both clients and project staff may be uncomfortable with random
assignment, and this discomfort could preclude randomized assignment.
A practical solution In such cases is making the site He unit of assignment:
that is, all clients at site ~ receive service regimen A, all clients at site 2
receive regimen B. etc.
Data collection poses another challenge to study designers. Although
eliciting sensitive information may seem feasible given the confidentiality
assurances available under Section 303 of the Public Health Service Act,
data gathering at HIV testing and counseling sites may be difficult.
At sites that offer anonymous testing, it will not be possible to do
follow-up interviews unless either He clients agree to confidential (rather
than anonymous) data collection or the study uses a follow-up method
that preserves client anonymity. In the latter regard, O 'Reilly (CDC,
1989) has reported recontact rates of between 50 and 80 percent using
one scheme that preserves client anonymity. These recontact rates are
certainly impressive, but they leave considerable room for uncertainty
about the effects of attrition on the outcome measures. Although these
uncertainties are troubling, the pane} points out that some important
crucial outcome measures e.g., the proportion of clients that return for
HIV test results and subsequent stages of the protocol—can be known
with certainty even in an anonymous testing program.
Although the pane} recommends that a skilled evaluation team care-
fully design and conduct randomized studies, the evaluation team and the
sponsoring agencies should be prepared for a certain number of failures
in carrying out experiments. In the panel's option, a failure rate of 20
percent or more should not be surprising. ~ the event of a failure (e.g.,
the contamination of individuals at the point of their selection, substantial
attrition from the study) it will be useful to have a faIl-back position.
For example, if the randomization of individuals fails, a randomized ex-
penment might still be conducted at the clinic level. Such a redesigned
experiment may not be as "clean" as one that uses individual participants,
but it might still provide useful information.
The panel's experience with counseling and testing sites suggests
that sites will be willing to cooperate In experimental studies if they
have substantial involvement in the implementation of new approaches
and of the evaluation strategy. Most administrators are sophisticated
enough to understand the need for evaluation in the interest of improving
interventions, even if they do not necessarily understand the statistical
aspects of such studies.
HIV TESTING AND COUNSELING ~ 121
Resources and Aspirations
Three types of personnel resources will be beneficial to conduct ran-
domized field experiments of testing and counseling interventions: (~)
a qualified team of behavioral, social, and statistical scientists to design
and conduct the studies; (2) an independent scientific oversight group
for quality control (as recommended in Chapter 1~; and (3) appropriately
trained CDC staff to design and conduct the studies and to monitor stud-
ies undertaken by outside groups (also as recommended above. The
first type of personnel, the individual investigator or investigative team,
would be responsible for developing and implementing interventions.
A un~versity-based or non-public health system contractor might be a
good choice; however, because some elements of the evaluation of HIV
testing and counseling will require the direct involvement of the public
health system and community-based organizations, personnel from these
settings should not be excluded from consideration.
The oversight group should be an academic or other scientific re-
search agency team that is independent of the project investigator or
investigators. (Appendix B descnbes one such oversight approach, the
"Project Review Team.") This oversight body can be used at the outset
to facilitate consensus on evaluation protocols and to approve or develop
outcome measures. Further along in the evaluation process, it can pro-
vice strong, centralized oversight and quality control of the work. Past
experience with large-scale, decentralized social research and evaluation
programs indicates that without vigorous oversight the research may be
of poor quality. This problem seems to occur for a variety of reasons,
including a lack of coordination together with the inherent difficulties of
conducting methodologically rigorous research In the context of a social
action program (see Betsey, HoDister, and Papageorgiou, 1985~.
Finally, evaluation studies that are cattier out as randomized trials
by outside experts will require appropriately trained CDC staff to interact
win the investigators and to interpret study results. In addition, CDC's
personnel expertise and workload should also permit staff to conduct
evaluation studies themselves. The types of staff needed for such tasks are
behavioral, social, and statistical scientists trained In evaluation research
(see Chapter I).
\2 Still a fourth resource for a sponsoring agency would be an interagency coordinating body to draw
upon He expertise of the federal agencies Hat are knowledgeable in relevant areas (e.g., CDC, the
National Institute of Mental Health (NIMH), the National Institute on Drug Abuse (NINA), etc.). A
body of this kind could draw on one agency's expertise for instance, that of NIDA in the field of HIV
prevention with IV drug users—and on CDC's expertise in providing services. Together, such a body
could facilitate the development of creative interventions that are theoretically based and that could
then be empirically tested.
122 ~ EVALUATING AIDS PREVENTION PROGRAMS
In addition to personnel and funds, conducting experiments requires
time. Results will not be available quickly; indeed, when positive effects
are found, they will need to be measured again at intervals to guard against
erosion. Consequently, investigators and policy makers alike must find
the patience needed to calTy out such research and a commitment to using
evaluation research as a too] for the Tong-term improvement of testing
and counseling programs.
The pane} also notes that advances In the treatment of asymptomatic
HIV-infected individuals may increase the demand for testing and coun-
sel~ng services. Recent research suggests that some early treatments of
persons infected with HIV may postpone the onset of AIDS and decrease
morbidity and mortality from opportunistic infections. Thus, there may
be a substantial increase in the demand for HIV testing—as well as for
medical monitoring of seropositive persons. ~ a rapidly changing envi-
ronment, an ongoing program of evaluations will be essential to assess
progress toward both the goal of adequate service delivery and He goal
of reduced HIV transmission. ~ keeping with its general recommenda-
tions, the pane} urges that evaluation of HIV testing and counseling be
an ongoing activity and that selected projects be reevaluated periodically
to monitor their continued effectiveness.
REFERENCES
Betsey, C. L., Hollister, R. G., and Papageorgiou, M. R., eds. (1985) Youth Employment
and Training Programs: The YEDPA Years. Report of the ARC Committee on
Youth Employment Programs. Washington, D.C.: National Academy Press.
Bradbum, N. M., and Sudman, S. (1979) Improving Interview Method and Questionnaire
Design. San Francisco: Jossey Bass.
Centers for Disease Control (CDC) (1986) Additional recommendations to reduce
sexual and dog abuse-related transmissional human T-lymphotropic virus type
m/lymphadenopathy-associated virus. Morbidity and Mortality Weekly Report
35:152-155.
Centers for Disease Control (CDC) (1987) Public Health Service guidelines for testing
and counseling to prevent HIV infection and AIDS. Morbidity awl Mortality
Weekly Report 36:509-514.
Centers for Disease Control (CDC) (1988) Announcement No. 901. Federal Register
53~182~:36492-36493. September 20.
Centers for Disease Control (CDC) (1989) AIDS Community Demonstration Projects
Progress Report 1989. Centers for Disease Control, Atlanta, Ga.
Centers for Disease Control (CDC) (n.d.) Prototype: Process Performance Standards
for Personnel Performing HIV Disease Intervention (Counseling and Partner
Notification). Centers for Disease Control, Atlanta, Ga.
Coates, T. J., Morin, S. F., and McKusick, L. (1987) Behaviom1 consequences of AIDS
antibody testing among gay men. Journal of the American Medical Association
258:1889.
HIV TESTING AND COUNSELING | 123
Fitti, J. E. (1989) AIDS knowledge and attitudes for September, 1988. Advance Data.
No. 164. DHHS Pub. No. (PHS) 89-1250. January 3.
Institute of Medicine/National Academy of Sciences (IOM/NAS) (1988) Confronting
AIDS: Update 1988. Committee for the Oversight of AIDS Activities. Washington,
D.C.: National Academy Press.
Presidential Commission on the Human Immunodeficiency Virus Epidemic (1988)
Final Report of the Presidential Commission on the Human Immunodeficiency
Virus Epidemic. Washington, D.C.: U.S. Government Printing Office.
Rugg, D., Sweet, D., Hovell, M., and Pagan, R. (1988) Factors Affecting the Decision
to Learn HIV Test Results. Paper presented at He Fourth International AIDS
Conference, Stockholm, June 12-16.
Smith, T. (1984) Nonattitudes: A review and evaluation. In Turner, C. F., and E.
Martin, eds. Surveying Subjective Phenomena. Vol. 2. New York: Russell Sage
Foundation.
Turner, C. F., and Martin, E., eds. (1984) Surveying Subjective Phenomena. 2 vole.
New York: Russell Sage Foundation.
Turner, C. F., Miller, H. G., and Moses, L. E., eds. (1989) AIDS, Sexual Behavior
and Intravenous Drug Use. Report of the NRC Committee on AIDS Research
and the Behavioral, Social, and Statistical Sciences. Washington, D.C.: National
Academy Press.
