Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs (1994) / Chapter Skim
Currently Skimming:
EVALUATION METHODS
EVALUATION METHODS
Pages 153-252
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 155... ...
Needle exchange programs have as their goal reducing the transmission of HIV (and other pathogens) among injection drug users.
|
From page 156... ...
Hypotheses not falsified by the data at hand are "confirmed" in that they remain reasonably good explanations until they are falsified by new data, and are replaced by other hypotheses that better explain the observations (Popper 1965; Rothman 1986~. OUTCOMES OF INTEREST Outcomes Related to Reducing Risk of HIV Transmission In order to determine if needle exchange programs succeed in their aim of reducing the parenteral and sexual risk of HIV transmission among injection drug users, one could measure the prevalence or incidence of HIV, the behavioral predictors of HIV transmission, surrogate measures of those behaviors, or the number of different persons sharing individual needles.
|
From page 157... ...
1981~. Outcomes Related to Injection Drug Use It is possible that needle exchange programs promote drug use by "condoning" it, by making drug injection easier, or by fostering the initiation of new drug injectors.
|
From page 158... ...
Duration Can Be Modelled in a Number of Different Ways The One-Hit Mode} Patrons of the exchange might achieve benefit after a single exposure. This mode} reflects what might happen if the very existence of the exchange alerted clients to the risk of AIDS, and convinced them to stop sharing needles or to stop using drugs.
|
From page 159... ...
STUDY DESIGN There have been numerous attempts to glean information on the effect of needle exchange programs from surveys that do not include comparable, concurrently sampled, control groups (Donoghoe et al. 1989; Kaplan et al.
|
From page 160... ...
because the two communities differ with respect to the timing of the epidemic and many demographic, sociologic, and cultural factors which may be predictors of HIV occurrence. However, comparisons of injection drug users among comparable New York City and New Jersey communities with and without exchanges, could possibly provide more valid evidence for a possible relationship of needle exchange programs to HIV transmission.
|
From page 161... ...
. For example, in prevalence surveys among injection drug users entering drug treatment programs, we found that HIV prevalence was higher among those who patronized the needle exchange than among those who did not (Harris et al., In...
|
From page 162... ...
Aside from the expense and the length of time it takes to complete them, a major consideration in conducting cohort studies, particularly of cohort studies of injection drug users, is the importance of maximizing follow-up. A cohort study was conducted by Hartgers et al.
|
From page 163... ...
(1991) , in which the investigators considered in the analysis the potential confounding effects of receptive anal intercourse in their study of the relationship of diabetes (who have access to sterile neediest to HIV prevalence among injection drug users.
|
From page 164... ...
We assessed HIV prevalence among STD clinic patients sampled in three ways: those who self-selected by requesting HIV counseling and testing; those who agreed to participate in a cross-sectional HIV prevalence survey offered to a systematic sample of patients; and the total S ID clinic patient roster (through a "blinded" survey of leftover blood samples from all clients)
|
From page 165... ...
By its end, only 50% of the high risk group remained in the study, compared to 70% of the low- risk group (see Figure 2~. The investigators observed a declining HIV incidence among study participants, but note that the greater loss to follow-up of participants at greater risk may have biased the findings, possibly masking a true increase in HIV incidence.
|
From page 166... ...
In Seattle, for example, we have found that participation in the needle exchange and the risk of HIV infection both vary by drug of choice. If needle exchange had a different Impact among heroin Injectors than it did among amphetamine injectors, this modification of effect might be missed if the numbers of subjects in each group were too small, and missing it could result in under-estimation over-estimation of the efficacy of the exchange.
|
From page 167... ...
FUTURE NEEDS We need to know just what effects needle exchange programs have, how they have their effects, and among which sub-groups they have their greatest effects. The best epidemiological evidence pertaining to these questions will be afforded by large, statistically powerful, well-designed, observational case-control or cohort studies.
|
From page 168... ...
Kaplan EH, 1991, Evaluating Needle-Exchange Programs via Syringe Tracking and Testing (Shy) , AIDS & Public Policy I, 6~3~:109-115.
|
From page 169... ...
95. Nelson KE, 1991, Human Immunodeficiency Virus Infection in Diabetic Intravenous Drug Users, loumal of the American Medical Association, 266~16~:2259-2261.
|
From page 170... ...
We conducted a study to determine whether syringe exchange is harmful or beneficial as risk reduction for injection drug users. We evaluated an all-volunteer syringe exchange program in San Francisco, CA known as "Prevention Point." Estimates of the number of injection drug users in San Francisco, range from 13,000 to 16,000 in a city of approximately 740,000.
|
From page 171... ...
Prevention Point syringe exchange program records. The Urban Health Study is a semi-annual cross-sectional study of IDUs recruited in natural settings in three inner-city communities in San Francisco.
|
From page 172... ...
the number of client contacts and syringes exchanged as reported in program records of Prevention Point syringe exchange program; (2) frequency of visits to syringe exchange as reported by participants in the Urban Health Study from 1989 to 1992; and (3)
|
From page 173... ...
One-way analysis of variance with Scheffe's test for multiple comparisons was used to identify differences in the mean number of syringes exchanged and the reported frequency of injection over successive cross-sections. Differences in the proportion of IDUs utilizing the syringe exchange > 25 times in the past year, and the proportion of new injectors over-time were assessed using the Mantel-Haensze} %2 test for trend.
|
From page 174... ...
The proportion of respondents who reported using the syringe exchange at least 25 times in the past year doubled between 1989 and 1992, from 14% to 28% (Mantel Haenszel x2 test for trend = 40.26; df = lip < 0.00001~. We also found major shifts in the principal sources of syringes reported by Urban Health Study respondents between 1987 and 1992.
|
From page 175... ...
When all observations from 1989-1992 were considered, IDUs who reported syringe exchange use > 25 times in the past year were less likely to report needle-sharing in the past 30 days than those who used the exchange less frequently or not at all (Mantel-Haenszel summary odds ratio=0.71; 95% confidence interval= 0.59, 0.87~. The unduplicated Fall 1991/Spring 1992 sample (n=752)
|
From page 176... ...
(24) noted similar declines in needle-sharing and initiation into injection drug use among persons admitted to drug-abuse treatment programs in San Francisco before and after implementation of the syringe exchange program.
|
From page 177... ...
These differences in study outcomes may stem from different drug use patterns, exchange program structures, and/or social circumstances fol~nr1 among Fllranenn Ins Jo anti thnce studied in San Francisco. D r We found syringe exchange use to be a strong, independent predictor of not sharing needles and syringes in the recent past when adjusted for age, ethnicity, previous HIV testing and counseling, frequency of injection of cocaine, and consistent use of condoms.
|
From page 178... ...
Our results suggest that syringe exchange programs and voluntary HIV testing and counseling help reduce neecIle-sharing. Such programs should be continued, expanded to meet existing needs, and implemented in areas where not currently available.
|
From page 179... ...
HIV prevention among injecting drug users: three years experience from a syringe exchange program in Sweden.
|
From page 180... ...
Needle sharing and participation in the Amsterdam syringe exchange program among HIV-seronegative injecting drug users. Public Health Rep 1992;107~6~:675-681.
|
From page 181... ...
et al. Human immunodeficiency virus infection in diabetic intravenous drug users.
|
From page 182... ...
Table 1: Selected Demographic Characteristics: IDUs in San Francisco, 1987-1992 (N=5,644) AGE' GENDER RACEI n TO <30 995 17.6 31-40 2753 48.8 41-50 1408 25.0 >51 487 8.6 Male 3893 69.0 1750 31.0 Female AiDrican-American 2546 45.1 Caucasian 1941 34.4 795 14.1 Lat~no 6.3 CURRENTLY IN DRUG TREATMENT' Yes 1529 27.1 No 4078 72.6 Other Answer 9 .2 EVER IN DRUG TREATMENT (past 5 years)
|
From page 183... ...
CURRENTLY IN DRUG TREATMENT Yes 34/104 (32.7)
|
From page 184... ...
0.58 0.42 - 0.81 .0012 ALWAYS USE CONDOMS2 0.17 0.08 - 0.35 .0001 PREVIOUS HIV TEST RESULT 0.48 0.34 - 0.69 .0001 INJECTION COCAINE USE2 (10 use increment) 1.05 1.01 - 1.08 .0072 INTERACTION4 (age by frequency of syringe -I .0210 exchange use)
|
From page 187... ...
However, it has also been amply demonstrated that an individual's knowledge of high risk behaviors alone is insufficient to ensure discontinuance of risky activities (Davis-Berman and Brown, 1990; Friedman et al., 1992; Inciardi, 1990; Ottomanelli, et al., 1990; Page et al., 1991~. Thus, despite widespread implementation of legal prevention strategies, the epidemic of HIV infection among injection drug users (IDUs)
|
From page 188... ...
As of September 1993, we have completed field observations at two sites and depth interviews with 14 providers, 11 primary exchangers, 12 secondary exchangers and 14 non-exchangers. We are conducting this research in order to elicit the following kinds of information: first and foremost, the affect of NSE on the continuation/reduction of needle sharing, as well as sex-related risk behaviors.
|
From page 189... ...
We plan to study not only formal activities and anticipated outcomes, but also informal activities and unanticipated outcomes. Finally, we will provide a detailed description of the impact of this intervention on HTV drug and sex-related risk behaviors.
|
From page 190... ...
What services do they need? What is their knowledge of and participation in drug/sex related HIV risk behaviors?
|
From page 191... ...
The combination of field observations and depth interviews is methodological triangulation, or the multiple use of methods to study a single problem. The process of interviewing staff, participants and non-participants is data triangulation, since we are analyzing program perceptions from diverse perspectives.
|
From page 192... ...
Field observations will inform provider interviewee selection procedures with the intention of capturing the core provider perceptions that cut across variation among providers. Primary Exchangers We will recruit fifty adult primary exchangers, ten from each of five sites.
|
From page 193... ...
Non-exchangers are located via chain referral and field work. Field observations in neighborhoods surrounding Prevention Point sites is being conducted in order to locate non-exchangers for our study.
|
From page 194... ...
to provide follow-up tracking information so that s/he can be re-contacted for follow-up. Immediately following field observations, field observers will tape record key descriptive and narrative material.
|
From page 195... ...
The process evaluator then goes back and forth between the data and the emerging paradigm to verify the meaningfulness of the categories and modify the paradigmatic model as necessary. Along with the tools of the grounded theory method -- coding, memoing, and paradigm building (Glaser and Strauss, 1967; Strauss and Corbin, 1990)
|
From page 196... ...
Coding The first code list wall be derived directly from the interviews and will consist of subject areas which, by virtue of the time the respondent spent discussing them and/or their recurrent nature, seem important. We will also begin with an initial set of codes taken roughly from the interview guide, since we generated the topic areas from our research questions about issues relevant to NSE.
|
From page 197... ...
It has been our experience with over seven hundred interviews that, given the assurance of confidentiality, illicit drug users can be relied on to give valid information. Numerous other studies have shown that illicit drug users are usually truthful when interviewed (Ball, 1967; Bonito et al., 1976; Maddux and Desmond, 1975; Nurco, 1975; Stephens, 1972~.
|
From page 198... ...
(1986) "Sharing of Needles among Users of Intravenous Drugs," The New England loumal of Medicine, 314:446-447.
|
From page 199... ...
(1987b) "Human Immunodeficiency Virus Infection in Heterosexual Intravenous Drug Users in San Francisco,"Amencan loumal of Public Health, 77~2~:169-172, February.
|
From page 200... ...
(1989) "Determinants of Needle Sharing among Intravenous Drug Users," American Journal of Public Health, 79:459-462.
|
From page 201... ...
(1989) "Risk Factors for Human Immunodeficiency Virus Infection in Intravenous Drug Users," The New England loumal of Medicine, 321:874-879.
|
From page 202... ...
It does mean, however, that it is hard to know the degree to which self-reported changes in behavior reflect the truth. This paper offers a rather different set of ideas for evaluating needle exchange programs.
|
From page 203... ...
The first of these attempts to estimate the incidence of HIV infection among participating drug injectors using nothing other than data describing client visits and the measured level of infection in needles (Kaplan and Heimer, 19949. The second approach develops an operational theory of needle exchange that not only provides an estimate for the relative impact of the program on HIV incidence among participating IDUs, but also provides a compelling explanation for how the physics of needle exchange might conspire to reduce HIV transmission in the absence of major changes in behavior (Kaplan, 1994; Kaplan and Heimer, 19939.
|
From page 204... ...
2.1 An Idealized Research Environment To begin, imagine an idealized research environment where one can repeatedly test a cohort of m IDUs for HIV infection over time. If the prevalence of HIV infection in the IDUs is denoted by ~ and the incidence rate of new infections is given by p infections per uninf ected IDU per unit time, then the probability that a randomly selected IDU would become infected within a duration of length ~ is given by -Pa PrlInfection)
|
From page 205... ...
. 2.2 A Needle Exchange Environment Where Only Needles Are Tested With the above set of computations in mind for future reference, consider now the more difficult research environment of a needle exchange program.
|
From page 206... ...
Indeed, these probabilities allow for differences in needle sharing and needle cleaning patterns across IDUs, in addition to pure laboratory testing errors. Suppose that the IDU in question became infected between the sth and s+lSt needles tested.
|
From page 207... ...
. Under the null hypothesis, the maximum likelihood estimator for is given by Eni=1 xi/e.
|
From page 208... ...
enables the calculation of the exact probability of rejecting the null hypothesis conditional on the true values of the underlying parameters in the model. For example, suppose that in truth no new infection has occurred, and the probability of any needle testing positive is given by a.
|
From page 209... ...
These quantities enable the construction of a procedure for estimating HIV incidence among needle exchange participants based solely on the testing of returned needles.
|
From page 210... ...
Now, recall from equation (1) that the probability that an IDU becomes infected over some duration ~ approximately equals 8b, where ~ is the unconditional incidence rate, that is, the number of new infections per IDU per unit time.
|
From page 211... ...
It is also possible to estimate ~ via maximum likelihood. Given bi' the duration of exposure of the ith IDU to the needle exchange as discerned from the dates of first and last program visits (or perhaps more appropriately, from the dates of first and last visits from which returned needles were tested for HIV)
|
From page 212... ...
Consequently, a complete record of syringe transactions and client participation is compiled. In addition to syringe tracking, samples of returned needles are tested for the presence of HIV-1 proviral DNA using polymerase chain reaction (PCR)
|
From page 213... ...
Of the 132 change point tests conducted, six rejected the null hypothesis of no infection using the cutoff c = 5.991. If one was to simply divide this number by the total exposure time, one would estimate an unconditional HIV incidence rate of 6.3 new infections per 100 needle exchange clients per year.
|
From page 214... ...
Furthermore, since more than 42% of the needles tested HIV positive, it is clear that this population had a positive HIV incidence rate in the past. While the change point model provides an approach to estimating HIV incidence among needle exchange participants, it does not explain how needle exchange operations might be expected to effect HIV transmission rates.
|
From page 215... ...
This is the key operational link between needle exchange programs and HIV transmission, for reducing needle circulation times acts via the stochastic infection process to lower the average level of infection in the population of needles. The intuition is simple: if needles are available for shorter periods of time per needle, then the likelihood that different IDUs will use the same needle declines.
|
From page 216... ...
The theory summarized above does not imply that needle exchange is an instant success. To be effective in reducing HIV incidence, the theory suggests that exchange programs must exchange needles with sufficient rapidity to reduce circulation times by a large amount.
|
From page 217... ...
t J e (13) where n0 is the probability that the needle is already HIV-contaminated when introduced to the population (and should equal zero if the needle in question was provided via a needle exchange program)
|
From page 218... ...
Note that conditional upon selecting a needle with a complete circulation interval equal to, say, u, the elapsed circulation time will simply be distributed uniformly between O and u. Therefore, the probability density function for the elapsed circulation time of a needle selected at random (elapsed circulation time is denoted by random variable TE)
|
From page 219... ...
is a general result, in that it shows how to obtain the average level of infection in circulating needles for any needle infection process (not just the Markov process considered earlier) and for any needle circulation process.
|
From page 220... ...
3.4 Needle Exchange Reduces Circulation Times Needle exchange physically interrupts the needle circulation process by replacing circulating syringes with new ones every time an exchange transaction occurs. Thus, the physical result of needle exchange is to shorten the circulation times of needles.
|
From page 221... ...
. As HIV transmission via needle sharing requires injection with a contaminated needle, it is reasonable to consider the relative reduction in the mean level of HIV infection in circulating needles as a measure of the relative reduction in HIV incidence.
|
From page 222... ...
endemic among drug injectors. Following our notation of Section 2, an estimate of the unconditional incidence rate ~ is given simply by ~ = ~ ~ (21)
|
From page 223... ...
, in(t) denote the HIV incidence rate at time t (measured in the number of infections per unit time)
|
From page 224... ...
Backcalculation can be applied to AIDS incidence data specific to drug injectors to obtain a rough idea of HIV incidence rates in that population. It is well known that backcalculation offers little information about recent HIV incidence (for the nature of the incubation time distribution precludes the development of AIDS cases shortly after HIV infection occurs)
|
From page 225... ...
Mean needle circulation times were also indexed by month of distribution. As needles are typically exchanged in batches of at most five per visit, mean circulation times are defined as the average elapsed time between needle distribution and return for returned needles normalized by the average number of needles distributed per batch, to obtain an average circulation time per needle.
|
From page 226... ...
A major claim of the theory advanced is that more frequent exchanging should lead to a reduction in mean needle circulation times. Focusing now only on the 20 months from November 1990 through June 1992 inclusive, Figure 3 shows that mean circulation times have been reduced.
|
From page 227... ...
proposes a specific expression for the mean level of infection in circulating needles as a function of mean circulation times. Using the circulation time and PCR test results for the first 20 months of the program, it is possible to estimate the unknown parameters ~0, ~ and ~ via maximum likelihood.
|
From page 228... ...
. Now, while the true pre-needle exchange circulation times are in truth unknown, equation (23)
|
From page 229... ...
Negative results are possible, and occasionally they are obtained as in the example of the underground program. Returning to the legal needle exchange program, however, it appears that the impact of needle exchange on interrupting the needle circulation process, and hence the HIV transmission process, has been considerable.
|
From page 230... ...
Using seven years of AIDS incidence data for New Haven drug injectors yielded a mean HIV incidence rate of approximately 160 infections per year. Again assuming that 30% of all infections are sexually acquired, this leaves a baseline annual incidence rate of 112 needle borne infections among New Haven's drug injectors.
|
From page 231... ...
The somewhat surprising result is that were one to take laboratory testing errors into account in the analysis, one would discover that the modeled impact of needle exchange would increase. To see why this is the case, let ~0 and ~1 denote the true average levels of infection in circulating needles before and after a needle exchange has been implemented (and assume JO > ~1)
|
From page 232... ...
Equation (27) shows that the actual relative reduction will be at least as large as the measured relative reduction in the mean level of infection in circulating needles.
|
From page 233... ...
; the mean needle circulation time (r) , and the mean level of infection in circulating needles ((or)
|
From page 234... ...
4. A Roll Call of Results from the New Haven Studies: What Was Learned versus What Might Have Been Sections 2 and 3 of this paper have presented the details of two distinct modeling approaches for use in evaluating needle exchange programs.
|
From page 235... ...
also suggests that in the absence of needle exchange, needle circulation times were in the neighborhood of three weeks! Again, it could have been the case that the estimated pre-needle exchange circulation times were no different from the circulation times observed in the study, a result that would have constituted a strike against the efficacy of needle exchange.
|
From page 236... ...
Of the two modeling approaches advanced, the methods of Section 2 yield a maximum likelihood incidence estimate of zero, while the methods of Section 3 produce a range of post-needle exchange incidence estimates from one through four per hundred IDUs per year. These two results are not inconsistent, for while the best incidence point estimate from Section 2 is zero, rates as high as four per hundred per year are well within the noise level associated with the estimation technique.
|
From page 237... ...
Even if one discovered via a controlled experiment that the HIV transmission rate among needle exchange participants was less than among an otherwise equivalent group of IDUs without access to needle exchange, one would not know why needle exchange had the effect it had. One would not be able to formulate suggestions for what to expect from needle exchanges operating under a different set of circumstances from those in the experiment.
|
From page 238... ...
It is, however, important to stress the unique features of this approach to program evaluation. The operational modeling of needle exchange leads to a number of falsifiable conjectures regarding program function and HIV transmission risks that can be tested empirically.
|
From page 239... ...
To recap, this paper has presented the details of a new set of techniques developed explicitly for the purpose of evaluating needle exchange programs. These methods have been applied to the evaluation of the legal needle exchange program operated by the New Haven Health Department.
|
From page 240... ...
William Quinn, Director of Health Services; and the entire outreach staff of the needle exchange program care deeply for the clients they serve. Their concern and dedication cannot be overstated.
|
From page 241... ...
(1993~. Estimating the population prevalence of injection drug use and infection with human immunodeficiency virus among injection drug users in Glasgow, Scotland.
|
From page 242... ...
(1991~. HIV prevention among injecting drug users: Three years of experience from a syringe exchange program in Sweden.
|
From page 243... ...
(1989~. Risk factors for human immunodeficiency virus infection in intravenous drug users.
|
From page 244... ...
(19879. Antibodies to human immunodeficiency virus in needles and syringes used by intravenous drug abusers.
|
From page 245... ...
Figure 1 Client Participation and Visitation non 900~ 800700600500400300200100 ~ ~ ~ 1 ~ , Decriminalization ~ ~ ~ ' · :a ~ ~ ~ 1 HE _ of , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , 1 Dec 90 dun 91 Dec 91 dun 92 Dec 92 dun 93 | ~ Number ofVisits ~ Number ofClients | 245
|
From page 246... ...
4500400035003000250020001 5001 000500O- l l l l Dec 90 ,~ Figure 2 Volume of Needle Exchange 55~ // l Jun 91 Dec 91 jpcrimi~alization Jun 92 Dec 92 Jun 93 1 __ Outbound NeedIes ° Inbouncl NeedIes 246
|
From page 247... ...
Figure 3 Mean Needle Circulation Time 7 6_` In In ~ 5a) ._ ~ 4o a_ `d 3 J i_ ~ 2Al 1~e 01, , , , , , , , , , , 1 , , , , , ~ ~ Dec90 Mar91 Jun91 Sep91 Dec91 Mar92 Jun92 Month of Program Operations 247
|
From page 248... ...
Figure 4 Needle Exchange and Removal Rates 0.60.5au a)
|
From page 249... ...
o 0.3 .° 0.2 cut ~ 0.1 to an. O- l l l l l l l l l l Dec 90 Mar 91 l Jun 91 Sep 91 Date Dec 91 Mar 92 Jun 92 0 Street Needles x Underground Program .
|
From page 250... ...
He said that evidence also suggests that risk has not been totally eliminated; addicts sometimes find themselves in situations in which it is difficult to follow risk reduction protocols. These findings are supported in evaluations of drug treatment programs, community-based outreach interventions, educational office-based interventions, pre- and post-HTV testing and counseling, and needle exchanges.
|
From page 251... ...
Consequently, Booth recommended the use of multiple methods to evaluate needle exchange programs, including quantitative and qualitative studies, mathematical modeling, and use of what have been referred to as focal-local indicators, such as those used in the Tacoma and Portland studies reported on earlier, in which the researchers looked at hepatitis-B prevalence and incidence over time. In addition to hepatitis data, other indicators would include HIV and AIDS cases, drug-related crimes and arrests, and sexually transmitted diseases.
|
From page 252... ...
Booth ended by mentioning the important issue of retention, which has been stressed repeatedly in evaluations of drug treatment programs. In the context of needle exchange programs, the issue becomes: What are the criteria for success regarding retention in a program?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.