Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 339
Appendix C
International Experiences Limiting
Resident Duty Hours
Altering the resident experience through reduced duty hours is an ongoing
process in many countries. Some have implemented regulations to adjust
resident schedules, while others have adopted more guideline-based meth-
ods. These diverse approaches accommodate very different healthcare sys-
tems, medical education programs, and cultures than in the United States.
For most of the countries examined by the committee, the primary reason
presented for modification of total resident hours is worker safety. Each
country has experienced difficulties implementing its intended reductions
due to workforce shortages and some have faced strains on educational
training, including reduced clinical contact and procedural experience. The
committee reviewed the strategies used by these countries to overcome the
challenges encountered from reducing resident duty hours and identified
educational redesign, scheduling flexibility, and a period of phase-in to
adjust to changes as relevant lessons for the U.S. graduate medical educa-
tion system.
The desire to identify optimal resident duty hours is not unique to the
United States. Duty hour regulation has been in place for more than 20
years in some countries (NZRDA, 2007). The statement of task for this
Institute of Medicine (IOM) report specified an examination of the experi-
ences of Europe (primarily the United Kingdom), New Zealand, Australia,
and Canada to gain insights on additional strategies, practices, interven-
tions, and tools employed by these countries in their efforts to adapt to
changes in resident schedules. International efforts to modify the duty hours
and work environment of medical residents provide useful perspectives
on alternative systems. It is difficult to draw direct solutions from these
OCR for page 340
40 RESIDENT DUTY HOURS
experiences given that medical education, medical training, and the overall
infrastructure of health systems vary substantially between other countries
and the United States. However, lessons learned from abroad might help
residency programs in the United States anticipate potential challenges of
implementing additional changes to resident duty hours and new scheduling
practices, if they were to occur.
Therefore, this review first provides a snapshot of current duty hour
regulations and available patient safety data in the aforementioned coun-
tries. Duty hour regulations are then discussed in more detail, along with
comments on compliance with and enforcement of those regulations. Con-
sequences of reduced duty hours on the resident workforce labor supply,
and the impacts of hour reductions on resident education and training are
also examined, with a final look at efforts to overcome these challenges
through program redesign.
GENERAL OVERVIEW OF COUNTRY ExPERIENCES
The rationale for regulations to reduce duty hours in New Zealand
and Europe and for duty hour reform in Australia has been primarily from
the perspective of worker safety, more so than patient safety, as it has been
in the United States or Canada. Yet a combination of these concerns has
caused each of the countries examined to make various changes in its medi-
cal resident training, and each has used different approaches to implement
them. It is important to note that many of those changes are taking place
within an extended training system compared to U.S. training programs
(Foundation Programme, 2007; Jarvis, 2002). The average duration of
medical school in most European countries is 6 years, similar to New
Zealand and Australia, with some schools including a year or two of intern-
ship training (World Health Organization, 2007). The duration of medical
school in Canada and the United States is normally 4 years. Upon being
awarded a medical degree, graduates in the United States and Canada spend
at least 3 years of residency training for general practice and additional
years of training thereafter (usually 3 or more) depending on the specialty
pursued (Medical Council of Canada, 2008; National Recruitment Office
for General Practice Training, 2008; NHS, 2008a; Royal Australasian
College of Physicians, 2007). One example that highlights how different
training in other countries can be is the New Zealand experience. After 6
years of medical school, residents there typically spend another 4 years in
basic training for general practice before qualifying to undertake specialty
training (e.g., surgery, pediatrics, pathology). New Zealand residents who
then train in those specialties (known there as “registrars”) “can continue
providing services to patients for more than 10 years while they complete
their training” (NZRDA, 2008a, p. 1). In contrast, most specialty residency
OCR for page 341
4
APPENDIX C
programs in the United States typically last 3-5 years, similar to programs
in Canada or the United Kingdom.
The maximum weekly duty hours permitted by various recommenda-
tions, regulations, and legislation also vary considerably across the coun-
tries examined: from 37 hours per week to an unlimited number of hours
per week that residents may work (Table C-1). Much of the available infor-
mation on duty hours comes from national websites of medical professional
organizations, resident organizations, or government agencies.
TABLE C-1 Resident Duty Hour Regulations in Various Countries, 2008
Maximum Maximum Minimum
How Hours Hours Consecutive Hours of
Are Averaged Hours per Rest Between
Country Regulated per Week Shift Shifts
Europea European 48 13 11
Comission and
collective
agreements
Franceb Government 52.5 10 —
United Kingdomc Ministerial 56-64 14-24 8-12
agreements
Denmarkd Legislation and 37 13-16 11
collective
agreements
New Zealande Collective 72 16 8
agreements (not averaged)
Australiaf Collective Unrestricted NA NA
agreements
Canadag Provincial Unrestricted Varies by Varies by
collective nationally province province
agreements
Manitoba — 89 24 + 2 NA
British Columbia — NA 24 8
Ontario — 60 (ICU, ED) 24 + 1 (Anes, NA
OB/GYN, ICU,
CCU)
NOTE: Anes = anesthesiology; CCU = cardiac care unit; ED = emergency department; ICU =
intensive care unit; NA = Not applicable; OB/GYN = obstetrics-gynecology.
aEuropean Trade Union Confederation, 2006.
bCode du Travail, 2006; Woodrow et al., 2006.
cNHS Employers, 2008b.
dDanish Medical Association, 2008; Ministry of Science Technology and Innovation, 2008.
eNZRDA, 2007.
fAustralian Medical Association, 2005.
gPAIRO-CAHO, 2005; PAR-BC, 2008; PARIM, 2008.
OCR for page 342
4 RESIDENT DUTY HOURS
The committee performed a qualitative analysis of the evidence found
for the countries investigated in this report that related to residents’ duty
hours and effects on resident training, education outcomes, and patient
outcomes.1 The committee found limited documentation of the impact of
reduced resident duty hours on patient outcomes, but a substantial litera-
ture on resource management and medical training outcomes linked to duty
hour reductions. For example, despite differences in the duty hour regula-
tions of each country, a common implementation problem that all have
faced is labor supply shortages. Providing residents with sufficient exposure
to learning opportunities within the guidelines has also been difficult, but
some countries are working on ways to address the issues. Discussion of
these challenges follows the country descriptions of current regulations and
rates of compliance.
Patient Safety Events in the United States and Abroad
Regarding the international evidence on patient safety, there have been a
few large epidemiologic studies describing adverse events (AEs) and prevent-
able adverse events (PAEs) in several countries. Each study has sought to
characterize the nature and causes of these events, although none measured
the contribution of resident fatigue or duration of work hours. Table C-2
represents the results of major studies conducted in six different countries.
Among these studies, only three countries identified admissions from
teaching hospitals in their sample and stratified their data by hospital
type. AE rates of teaching hospitals compared to non-teaching hospitals
were available for Canada, Australia, and the United States (those of the
United States are only for the states of Utah and Colorado), and only the
Canadian study provided PAE rates as well (Baker et al., 2004; Thomas
et al., 2000b). The AE rates of major teaching hospitals in Australia did
not differ from those in non-teaching or private hospitals (10.8 percent vs.
10.7 percent) (Thomas et al., 2000b). However, the difference in AE rates
of major teaching hospitals in the United States compared to non-teaching
hospitals was greater (4.0 percent vs. 2.5 percent) (Thomas et al., 2000b),
as it was for Canadian teaching hospitals (10.3 percent vs. 5.2-6.0 percent)
(Baker et al., 2004). Baker and colleagues were the only authors that com-
mented on the difference in rates between hospital types, highlighting the
following explanations: (1) differences in acuity of patient populations
1 IOM staff searched Medline and Embase databases using a combination of the follow-
ing terms: junior doctors, doctors in training, residency, resident work, workload, patient
outcomes, adverse medical events, mortality, morbidity, medical education, medical training,
Europe, United Kingdom, France, Denmark, Germany, Australia, New Zealand, and Canada.
Websites of national medical and resident organizations were also searched.
OCR for page 343
TABLE C-2 International Comparison of Adverse Events (AEs) and Preventable Adverse Events (PAEs)
Country Number Hospital AE Rate from Portion of AE
(year data of Medical Medical Rate That Was
collected) Hospitals Records AE Definition Records (%) Preventable (%)
United Kingdoma 2 1,014 Unintended injury caused by medical management 10.8 47.0
(1999-2000) rather than by disease process
Denmarkb 17 1,097 Unintended injury caused by medical management 9.0 40.4
(1998) that resulted in disability, death, or prolonged
hospital stay
New Zealandc 13 6,579 Unintended injury or complication that resulted in 12.9 37.1
(1998) disability, death, or prolonged hospital stay and
was caused by healthcare management rather than
by the underlying disease
Australiad,g 28 14,179 Same as New Zealand Adverse Events Study 10.6 NA
(1992)
Canadae (2000) 20 3,745 Same as New Zealand Adverse Events Study 7.5 36.9
United States
New Yorkf (1984) 51 30,195 Unintended injury caused by medical management 3.7
that resulted in disability 27.6
Utah and 28 14,700 Injury caused by medical management that resulted 3.2 NA
Coloradod,g in prolonged hospital stay or disability at discharge
(1992)
aVincent et al., 2001.
bSchiøler et al., 2001.
cDavis et al., 2003.
dThomas et al., 2000b.
eBaker et al., 2004.
fBrennan et al., 1991.
gThe data in these two rows come from the Thomas et al. (2000b) study referenced above, which attempted to harmonize the variance in study
methods between two earlier and separate 1992 studies: one on AEs in Australia (Wilson et al., 1995) and the other on AEs in Utah and Colorado
4
in the United States (Thomas et al., 2000a). The AE and PAE rates reported in those earlier studies are AE 16.6%, PAE 51% (Wilson et al., 1995);
AE 2.9% (Utah, Colorado), PAE 32.6% (Utah), 27.4% (Colorado) (Thomas et al., 2000a).
OCR for page 344
44 RESIDENT DUTY HOURS
between settings are difficult to capture with precise accuracy, (2) teaching
hospitals may receive patients at different points in their care that place
them at higher risk for AEs than at other hospitals, and (3) care delivered
by multiple health professionals in teaching hospitals may increase risk of
AEs due to miscommunication or lack of coordination of care (Baker et al.,
2004). These same conclusions could be drawn to explain the differences
in AE rates among U.S. hospital types, although insufficient data exist to
do so. Insufficient data also prevented the committee from being able to
conclude whether a correlation exists between duty hours and AEs.
The rates of AEs and PAEs found in each of the jurisdictions studied
represent a distillation of local practices, documentation, and culture. They
also demonstrate methodological differences between studies, temporal
changes in care, and the changing nature of defining AEs. In trying to deter-
mine a relationship between duty hours and AE rates, inspection of Tables
C-1 and C-2 reveals that patients in countries with lower duty hours were
not necessarily at less risk of AEs or PAEs than patients in countries with
longer duty hours. The AE rates for the United States, represented by Utah
and Colorado, are lower than those of all the other countries by more than
50 percent. More recent data on amenable mortality rates for international
populations under the age of 75 (Nolte and Mckee, 2008) also show no ap-
parent association between preventable events and duty hours across coun-
tries. Although these studies do not establish whether a correlation exists
between AEs and duty hours, the results appear inconsistent with general
expectations that reduced duty hours improve patient safety. Furthermore,
they underscore the variability of health systems and patient safety across
countries and the complexity of achieving improved patient outcomes.
CURRENT RESIDENT DUTY HOUR REGULATIONS
Europe
The main objective of the European Working Time Directive (EWTD),
issued by the European Council, “is to promote health and safety at work,
given the clear evidence that people who work long hours run higher risks
of illness and accidents” (European Trade Union Confederation, 2006). The
EWTD was first established in 1993 to place limits on all workers’ hours
throughout Europe. That directive included physicians but excluded “doc-
tors in training.” In 2000, a new directive passed to include the “junior
doctor” constituency after European resident groups lobbied for the change
(Woodrow et al., 2006). The amended article established that medical
residents are subject to all laws of the EWTD, accompanied by a require-
ment that by 2009 all health systems in the European Union limit resident
work to a maximum of 48 hours averaged per week (European Trade
Union Confederation, 2006). Regulated transition periods allow countries
OCR for page 345
4
APPENDIX C
to reduce duty hour limits gradually until they reach the 2009 goal (NHS,
2007). For example, the United Kingdom is currently in a transitional phase
consisting of 56-hour workweek schedules (NHS, 2007), while Denmark
already meets the 48-hour workweek goal (Ministry of Science Technology
and Innovation, 2008). Some countries, whose limits were much higher in
the past and are working toward achieving the directive targets for 2009,
have found it more difficult to restructure resident schedules to fit the new
requirements while maintaining service to patients and educational op-
portunities for residents. Some reasons for disparities in transitions include
workforce supply issues (e.g., physician-to-patient ratios and resident-to-
physician ratios of each country), the general health status of a country’s
population and subsequent effects on physician workload, and the different
organizational structures of entire health systems.
The EWTD that will apply to European residents by 2009 also includes
the following (Council Directive No. 93/104/EC, 1993):
• A minimum daily rest period of 11 consecutive hours,
• A minimum rest period of 1 day (24 hours) per week,
• A maximum of 8 hours of night work on average per 24 hours,
and
• A right to 4 weeks of paid annual leave.
Prior to the establishment of the EWTD, each European country had
different duty hour regulations for its medical residents, ranging from a 65-
hour-per-week maximum in Ireland, to 56 hours averaged over 24 weeks in
Germany, to no hour restrictions at all in Denmark and France (Australian
Medical Association, 1998).
New Zealand
New Zealand has been enforcing duty hour regulations for many years
in attempting to address the issue of overworked medical residents. Since
1985, the maximum permissible duty for New Zealand residents has been
72 hours a week, with a set limit of 16 consecutive hours a day. They are
also not to work more than 12 consecutive days without a 48-hour break
(NZRDA, 2007). However, the regulations allow residents to be scheduled
to work seven consecutive night shifts of 10 hours or more roughly once a
month, which has been identified as being counter productive to resident
well-being and performance (Dula et al., 2001; Powell, 2004).
Australia
Australia reacted very differently to the issue of duty hour restrictions
for its medical residents. In 1996, the Australian Medical Association ad-
OCR for page 346
4 RESIDENT DUTY HOURS
opted federal policy for safer working environments for medical residents
and, in 1999, further adopted a National Code of Practice for them to
follow (Scallan, 2003). Its purpose is to provide “practical guidance on
how to eliminate or minimise risks arising from the hazards associated
with shiftwork and extended working hours,” which include guidelines on
performance-based scheduling, incident reporting, and education on fatigue
mitigation techniques (Australian Medical Association, 2005, p. 4). This
code is not a legal regulation nor is it mandated by any organization within
or outside the medical community. The code is simply a set of guidelines
that hospitals and doctors are strongly encouraged to follow according to
their best judgment and acknowledge the responsibilities of employers and
employees under Australia’s Occupational Health and Safety legislation
(Australian Medical Association, 2005). The committee that composed the
code based it on available empirical evidence on sleep and fatigue and their
impact on work performance. This evidence showed that extended hours
of work had negative effects on medical training for three main reasons:
“lack of time for formal and independent study, lack of motivation due to
fatigue, and work patterns that failed to provide necessary supervision and
feedback for effective learning” (Scallan, 2003, p. 910).
Canada
In Canada, individual provinces establish duty hour regulations through
collective agreements negotiated by resident associations with hospitals, re-
sulting in variations in these regulations across the country. Currently, the
only province that has an explicit hour limit is Manitoba, which enforces a
maximum of 89 hours per week averaged over 4 weeks (Fok et al., 2007;
PARIM, 2008). In the provinces of Alberta, Quebec, and the Maritimes,
collective agreements limit resident duty to 12 hours per routine day (not
averaged over a week). Ontario and Saskatchewan have no overall hour
limit, but the Professional Association of Internes of Ontario does have
limitations of 60 hours per week for residents’ performing in-hospital shift
work, such as in intensive care units (ICUs) and emergency departments
(EDs) (Fok et al., 2007). Shifts in EDs cannot exceed 12 hours, and in
ICUs they cannot exceed 24 hours (plus handover); in other hospital areas,
after 24 hours on call residents cannot admit new patients and have to be
at home by noon (PAIRO-CAHO, 2005).
ENFORCEMENT OF DUTY HOUR REGULATIONS
To ensure compliance with the regulations outlined in the previous sec-
tion, each country has developed its own method for enforcing them; none
seem to constitute an ACGME equivalent.
OCR for page 347
4
APPENDIX C
Europe
In the United Kingdom, National Health Service (NHS) Employers en-
force compliance with hour regulations. They are under contractual obliga-
tion to monitor compliance by collecting and reporting duty hour patterns
of their residents on a semiannual basis, which the NHS Executive and
the British Medical Association review jointly. Recent compliance reports
show that most health authorities perform at near-perfect compliance (NHS
Employers, 2008a). However, a 2004 independent study on resident com-
pliance in a large urban U.K. hospital showed residents working beyond the
duty hour regulations (whose weekly maximum was 56 hours at the time):
the average number of hours worked by physicians was 63.6 hours per
week and by surgeons 70.2 hours per week (Jagsi and Surender, 2004). Such
mixed results imply that monitoring efforts may not be completely rigorous
or accurate, perhaps a common problem shared with the United States.
Data from a 2002 survey gathered by the Royal College of Physicians
indicated that residents and physicians in most European countries worked
total hours or continuous duty periods in excess of those established in
the EWTD (many worked 30 or more continuous hours) despite claims
that these countries were compliant with the directive (Royal College of
Physicians, 2002). Such excesses are consistent with the previously stated
findings in the United Kingdom. The committee did not find more current
compliance rates for other European countries.
The committee also had difficulty uncovering a clear picture of enforc-
ing bodies for EWTD regulations in the remaining European countries
(Mayor et al., 2004). However, an article reporting, in part, on France,
whose national government is the regulatory force for duty hour limits,
mentioned that it has not assigned an organization the responsibility for
enforcing the established limits, nor does it currently monitor compliance
rates itself (Woodrow et al., 2006). In Germany, since residents are fully
licensed physicians and regular employees of the hospitals, their duty hours
are enforced by the same state and local institutions that are responsible for
enforcing the duty hours of all employees (trade supervisory board). These
few examples indicate that monitoring and enforcement methods likely vary
significantly across European countries.
New Zealand
The New Zealand Resident Doctors’ Association (NZRDA) negotiates
resident hours for its members through collective employer agreements and
regulates the provisions therein (NZRDA, 2008b). In 2006, the average
duty hours of New Zealand doctors was 45.8, and the average worked
by doctors aged 24 years or younger was 57.7 hours per week (Medical
OCR for page 348
4 RESIDENT DUTY HOURS
Council of New Zealand, 2006). Both numbers fall well below the national
maximum of 72 hours. A nationwide survey of residents (63 percent re-
sponse rate: 1,366 responses) conducted by Gander and colleagues in 2003
showed that 57 percent of residents worked between 50 and 70 hours a
week and that 13 percent worked more than 70 hours, substantially more
than the average hours reported in 2006 to the Medical Council (Gander
et al., 2007).
Australia
Australia’s approach to resident work scheduling is unique and may
reflect a culture with expectations that resident hours should be similar to
those of other workers in the population. Resident duty hour guidelines
are advisory, as opposed to being binding rules, and there is no designated
enforcement body. The guidelines of the code imply that working more
than 50 hours per week puts a resident at “significant risk” of fatigue
and associated negative consequences, while working 70 hours or more is
considered to put residents at “higher risk” (Australian Medical Associa-
tion, 2005; Scallan, 2003). The Australian Medical Association takes it
upon itself to conduct national surveys of physician duty hours as a way
to gauge current practice. Results of the 2001 national survey showed that
70 percent of Australian medical residents worked an average of more than
50 hours a week (Scallan, 2003). The independent study by Gander et al.
(2007) had similar results. In May 2006, the Australian Medical Associa-
tion conducted another national survey representing all doctors (not just
junior doctors), and results indicated that 62 percent of hospital doctors are
in the “significant” and “higher risk” categories for doctor safety based on
hours of work, with 85 percent of surgical doctors falling in those catego-
ries. Although the later survey noted some relative improvements (e.g., the
longest continuous period of work was 39 hours, down from 63 hours in
2001), it is inaccurate to compare the results of the two surveys since the
earlier one surveyed only residents, whereas the latter surveyed all physi-
cians. Also, there was no mention in either survey of any fatigue-mitigating
methods used by doctors during extended shifts on duty, such as periods
for sleep. The authors of the later report concluded that 39 hours is still
too many to work consecutively and that hospitals need to continue tak-
ing steps to mitigate any safety risks to residents and patients (Australian
Medical Association, 2006).
Canada
The Royal College of Physicians and Surgeons of Canada (RCPSC)
and the College of Family Physicians of Canada (CFPC) are the regula-
OCR for page 349
4
APPENDIX C
tory bodies for duty hours in Canada. “However, the RCPSC has neither
developed nor endorsed specific policies regarding duty hours. As a result,
there has been no national drive to implement duty hour restrictions, but
most provinces have nonetheless adopted them independently of each other
within a relatively short time frame” (Woodrow et al., 2006, p. 1047). Just
as duty hour regulations vary according to province, so does enforcement
of those regulations.
Data on actual hours worked in Canada are very limited, but a 2007
survey of British Columbia first-year residents is helpful in describing some
general patterns that are occurring at least in that part of Canada. The
survey reported that these residents work an average of 65.4 hours a week
(including on-call work) and sleep an average of 41.9 hours a week (Fok et
al., 2007). However, when looked at separately, surgical residents reported
working many more hours than their non-surgical colleagues. On aver-
age, surgical residents worked 80.4 hours per week compared to 57 hours
per week for non-surgical residents. Of the surgical residents, orthopedic
surgeons and general surgeons work the most hours, 102 hours and 88.9
hours a week, respectively (Fok et al., 2007).
Overall
Although governing bodies try to enforce compliance with duty hour
limits, there is some indication that the number of hours reported officially
by residents through their institutions are not necessarily accurate. Gener-
ally, the residents in separate surveys seem to report working more than
the guidelines suggest. Therefore, the committee cannot conclude that these
countries uniformly enforce their stated hour limits or that the reported
compliance rates are accurate. The following section discusses some reasons
why countries may encounter difficulties achieving compliance with their
own regulations along with other challenges that medical training has faced
because of reducing duty hours.
CONSEQUENCES AND BARRIERS
Reducing resident duty hours abroad has led to several changes in
health systems that have pushed countries to create new ways to provide
continuous services to patients and maintain educational opportunities for
residents. The most significant organizational change among hospitals to
achieve this has been the trend to replace traditional 24-hour (or longer)
shifts with shorter shifts of 12 or 16 hours in accordance with local require-
ments or preferences (BBC, 2004). Concerns about shift-based schedules re-
garding continuity of care and resident training have been articulated (Carr,
2003; Jagsi and Surender, 2004; Powell, 2004; Royal College of Surgeons
OCR for page 352
RESIDENT DUTY HOURS
hands-on training and an increase in shift-based duty has led to reports
of residents’ dissatisfaction with their degree of learning or the teaching
style provided (Rawnsley et al., 2004; West et al., 2007). The committee
listened to testimony by Dr. Bernard Ribeiro, president of the Royal Col-
lege of Surgeons of England, that reiterated many of these same sentiments
and presented additional data on reduction in the number of operations
performed across surgical specialties since the 2004 EWTD, showing some
residents performing as much as 25 percent fewer procedures (Royal Col-
lege of Surgeons of England, 2008). These findings imply that decreasing
residents’ time to perform surgical procedures may decrease their general
level of competence in these tasks due to the demands and intricacies of
the work (Chikwe et al., 2004). The committee found no other objective
measures of such outcomes, and as noted below, evaluation of these impacts
on training has yet to be published.
To compensate for reductions in training time for U.K. surgical resi-
dents, redesign efforts have focused on providing more time concentrated
on procedural activities than other activities. For example, the study by
Lim and colleagues (2006) suggests that reorganizing institutional structure
can maintain the competency levels of cardiac surgical trainees in these
activities despite the reduction in work hours. Their institution adopted a
team-based model to achieve the desired results, ensuring adequate time
with trainers to maximize learning (Lim and Tsui, 2006). However, others
in the field have noted the difficulty that less robust programs may have in
implementing such changes and the likelihood that many programs would
not be able to do so, given their lack of resources (Mestres et al., 2006;
West, 2007). In response to these issues, the Royal College of Surgeons has
developed its own set of rotation guidelines in compliance with the EWTD
for surgeons in training to follow, hoping to maximize patient safety and
resident surgeon learning (Royal College of Surgeons of England, 2007a).
This issue has yet to be resolved in the United Kingdom, and no surgeons
have yet emerged from the shortened training system that would allow the
NHS to evaluate its overall impact and costs.
Observations of resident training in Australia suggest that much resi-
dent time is spent on administrative tasks and providing service demands
for the health system (Gleason et al., 2007). A consequence of this has been
variable quality in resident education because of insufficient time to attend
some of their didactic sessions (Gleason et al., 2007). Australian medical
education leaders have observed that “the existing systems for delivery of
education and training are inefficient, under-resourced and under pressure,
and they will not be sustainable into the future” (McGrath et al., 2006,
p. 348). This may not necessarily be a result of reduced hours, but reducing
hours could further affect such limitations.
OCR for page 353
APPENDIX C
The committee concludes that reports from other countries indicate
that reducing total duty hours places strain on the healthcare workforce
and creates challenges to maintaining the quality of resident learning en-
vironments. Several countries have experienced financial and manpower
strains trying to staff hospitals sufficiently to provide adequate 24-hour
care, and many have reported insufficient time to teach residents or to have
them practice procedures. Noneducational resident workload has also been
found to be an impediment to the resident learning experience.
APPLICABLE LESSONS FOR RESIDENCY
PROGRAMS IN THE UNITED STATES
Retaining Flexibility in Work Schedules
A key lesson from these countries is the need for flexibility in how
programs arrange schedules within the confines of established regulations.
In the United Kingdom, since the introduction of the Working Time Regu-
lations, the NHS has moved toward achieving the goal of safer work con-
ditions through reduced hours for medical residents (NHS, 2007). While
considering how best to implement the newest directive with further re-
duced hours, the NHS has taken into account an array of rotation sched-
ules with various combinations of night shifts, weekend shifts, day shifts,
and rest hours that its hospitals could adopt (Royal College of Physicians,
2006). In each instance, individual hospitals need to determine the most
effective combination of workforce (e.g., numbers of residents and other
staff on duty in a given shift) and required hours to achieve safe patient care
and quality resident education. Similar to the medical education system in
the United Kingdom, different types of residency programs in the United
States have national requirements to fulfill regarding educational content
within the duty hour restrictions. Therefore, the different organizational
methods across programs require flexibility in the way work schedules are
arranged.
The committee concludes that, given the differences among healthcare
institutions, different specialties, and the needs they must satisfy, maintain-
ing flexibility in the way work hours are scheduled is necessary. This echoes
the findings in Chapter 3 of the experiences of programs in the United States
as they adjusted to the 2003 duty hour limits. It appears that other coun-
tries have experienced problems with adherence to established duty hour
limits, as have programs in the United States. (See Chapter 2 on adherence
to duty hours and its enforcement in the United States.)
OCR for page 354
4 RESIDENT DUTY HOURS
Time for Phase-in
The international experience indicates that a well-planned period of
transition would be necessary as part of any major changes in total work
hour limits. In Europe, 4 years after residents became subject to the EWTD
laws, as programs were still transitioning to current requirements, multiple
countries in the European Union (EU) were protesting the mandated sched-
ule claiming that they lacked the resources, in either workforce, finances,
or both, to successfully comply with the regulations by the given deadline
(Sheldon, 2004).
Based on the international experience, a phase-in of committee-
recommended changes would be beneficial. Changes specifically related to
resident duty hours and schedules might require some time for planning
to accommodate constraints in workforce and other resources. Chapter 7
contains the committee’s recommendations for adjusting duty hours, and
Chapter 9 includes macro-level estimates of the potential costs, as well as
the type and number of clinical personnel that would be needed to replace
existing residents under further adjusted duty hours.
Redesign of Resident Education and Training Systems
The introduction of duty hour restrictions for residents in other coun-
tries has created an impetus for changing the fundamentals of medical
education and training programs. Complying with new regulations caused
hospital providers to alter their work practices, staffing, and delivery meth-
ods in order to ensure a degree of quality care in light of these significant
changes.
For example, in Australia the National Code clearly states that it was
created as “one part of a broader education and awareness program to
change the current individual and organisational beliefs and culture that
support working hours and patterns that would be considered unaccept-
able in most other industry sectors” (Australian Medical Association, 2005,
p. 4). Culture change is at its core. To achieve this goal, the Australian
Medical Association developed a risk assessment strategy to evaluate the
extent of hazard caused by residents’ working extended hours (working
more than 50 hours per week is considered “significant risk,” working 70
hours or more is “higher risk”) and recommends principles that should be
at the foundation of work schedule design to minimize risk to patient and
resident safety. Aside from these assessments, high service demands and
inadequate funding of education and skills training have also been identi-
fied as risks to resident training (Gleason et al., 2007). As institutions try
to increase the number of residents to satisfy service demands, the authors
of the study note that merely increasing the numbers of residents will not
OCR for page 355
APPENDIX C
alleviate their work intensity, nor will it solve training deficiencies unless
adequate resources are provided. They conclude that a strong emphasis on
training, supported by sufficient resources, is necessary to deal with these
changes and reference the U.K. efforts to redesign its educational program
(examined below). Similarly, although restrictions on hours vary by prov-
ince in Canada, many medical training programs throughout the coun-
try have been focused on improving patient outcomes through improved
resident training strategies for several years. Strategies have consisted of
increasing the educational value and success of residency programs rather
than using them to produce residents as a cheaper form of healthcare work-
force (Landau, 2007).
Competency-Based Training
Perhaps the foremost issue posed by reduced duty hours for residents
is how to ensure competence during and at the end of training. To address
this issue and create long-term improvements, changes need to occur at
the educational level. This has much to do with the fact that the duty hour
restrictions provide less time to train residents. As a result, both the United
Kingdom and New Zealand have been significantly reshaping their resi-
dency programs toward competency-based or performance-based training,
which determines a trainee’s level of expertise by their ability to demon-
strate specified required skills at a given point in training, as opposed to a
more time-based educational model, which focuses on completing physi-
cian training in a certain number of years. Their programs now center on
providing an experience with more educational value for residents and have
been fundamentally restructured to ensure the desired results (NHS, 2008b;
Workforce Taskforce, 2007).
One such example from the United Kingdom is the new Foundation
Programme, which constitutes the first 2 years of residency training upon
graduating medical school. Intended to “bridge the gap between under-
graduate school and specialist/medical training” (Foundation Programme,
2007, p. 5), the Foundation Programme focuses on patient safety and
improving the quality of care by having residents demonstrate competence
in communication and consultation skills, patient safety, and teamwork, in
addition to more typical clinical skills (Major shake-up in medical training,
2005). This training replaces the previous introductory 2-year program
and is a departure from that program, which focused nearly exclusively on
demonstrating competence in clinical skills (Foundation Programme, 2007).
No assessments have been made to date as to whether the quality of resi-
dent work has been affected by the new program, although one study has
reported concerns on behalf of both the trainees and the educators about
the generic nature of some of the skills trainees were expected to acquire
OCR for page 356
RESIDENT DUTY HOURS
(O’Brien et al., 2006). Still in its nascent years, the Foundation Programme
has opportunities to modify its curricula.
Monitoring Quality Measures and New Models of Care
The NHS has also developed several pilot programs that not only alter
resident rotations and schedules, but also incorporate new quality improve-
ment practices such as regularly assessing patient and staff satisfaction and
reporting patient safety measures, as well as measuring the compliance of
residents with the time limits. In some instances the pilot programs show
that they achieved reductions in patient wait times for treatment or surgery,
reductions in the time between prescribing and administering certain medi-
cations, shorter lengths of stay in hospital wards, and improved discharging
processes, even with reduced resident hours (NHS, 2004).
One pilot created the Hospital at Night program, a new model of care
that reduces the risk of adverse events by improving the quality of care tak-
ing place during nighttime hours in hospitals, while also aiming to enhance
resident learning during daytime hours (Hospital at Night and The NHS
Information Centre, 2008; Institute for Healthcare Improvement, 2007;
NHS, 2008c). With the completion of more pilots, the NHS anticipates
providing additional solutions to the EWTD that are transferable across
residency programs and improve resident learning and the quality of patient
care across the entire health system.
Based on the above review of educational changes, it is evident that
some countries have begun to redesign their educational system, focusing
changes in both curriculum and competency-based aspects, and that these
changes have been in response to decreased duty hours and some negative
impacts on resident training.
CONCLUSION
When considering the experiences of the countries examined in this
chapter with duty hour regulations, it is important to remember that most
of these countries have lower duty hour limits than the United States (e.g.,
Europe, 48; New Zealand, 72). Even Australia, where no national maxi-
mum limit exists, tries to have residents work no more than 70 hours per
week. With these lower limits, key stakeholders in those countries report
considerable challenges as they adjust to the reduced availability of resi-
dents to provide 24-hour care. The concerns about workforce shortages and
financial constraints because of these changes are particularly noteworthy,
as are the frequently voiced concerns that the educational experience of
residency is deteriorating. However, some countries have produced innova-
tive designs and promoted culture change.
OCR for page 357
APPENDIX C
Yet based on the evidence examined from other countries, it is not
possible to determine an ideal number of hours that residents should be
required to work. There are limited or no data to show that decreasing
duty hours improves patient safety, and many redesign efforts have yet to be
fully developed or evaluated for effect on resident learning or patient safety.
Furthermore, the international data on the quality of patient safety indicate
that duty hour regulations may not be the greatest factor affecting quality
of care and that other facets of delivery systems or educational programs
likely deserve more attention as areas for improvement.
Despite the limited evidence from other countries, the committee was
able to draw lessons on retaining scheduling flexibility, allowing phase-in
time, and redesigning educational programs for residency training. How-
ever, the committee concluded that no single system is directly applicable
to that of the United States given the different construct and culture of our
healthcare system. Although the committee lacks systematic multinational
evidence on resident education and patient outcomes as a result of imple-
menting duty hour regulations, from the evidence gathered it appears that
any changes in duty hour limits would necessitate modifications to redesign
and enhance the medical training system, to have an available workforce to
substitute for hours that residents are not available to staff, and to create
substantial financial resources. A similar call to redesign medical education
and training programs exists in the United States (Meyers et al., 2007).
It may be important to maintain some degree of flexibility as residency
training programs determine which innovative educational approaches and
new scheduling designs to adopt for their healthcare delivery and resident
learning needs.
REFERENCES
Ardagh, M. 2003. Beyond Ashburton: Junior hospital doctor employment in New Zealand.
The New Zealand Medical Journal 116(1168):1-2.
Australian Medical Association. 1998. Overseas experience in regulating hours of work of
doctors in training. Kingston ACT, Australia: Australian Medical Association.
———. 2005. National code of practice—Hours of work, shiftwork and rostering for hospital
doctors. Kingston ACT, Australia: Australian Medical Association Ltd.
———. 2006. Safe hours = safe patients: AMA safe hours audit 00. Kingston, Australia:
Australian Medical Association.
Baker, G. R., P. G. Norton, V. Flintoft, R. Blais, A. Brown, J. Cox, E. Etchells, W. A. Ghali,
P. Hebert, S. R. Majumdar, M. O’Beirne, L. Palacios-Derflingher, R. J. Reid, S. Sheps,
and R. Tamblyn. 2004. The Canadian Adverse Events Study: The incidence of adverse
events among hospital patients in Canada. CMAJ: Canadian Medical Association Journal
170(11):1678-1686.
BBC (British Broadcasting Corporation). 2004. Q&A: Junior doctors’ hours. BBC News.
http://news.bbc.co.uk/1/hi/health/3941073.stm (accessed January 23, 2008).
OCR for page 358
RESIDENT DUTY HOURS
Brennan, T. A., L. L. Leape, N. M. Laird, L. Hebert, A. R. Localio, A. G. Lawthers, J. P.
Newhouse, P. C. Weiler, and H. H. Hiatt. 1991. Incidence of adverse events and neg-
ligence in hospitalized patients. Results of the Harvard Medical Practice Study I. New
England Journal of Medicine 324(6):370-376.
British Orthopaedic Association. 2002. Education and training for SHOs: A snap-shot
of the moment and recommendations for the future. London: British Orthopaedic
Association.
Carr, S. 2003. Education of senior house officers: Current challenges. Postgraduate Medical
Journal 79(937):622-626.
Chikwe, J., A. C. de Souza, and J. R. Pepper. 2004. No time to train the surgeons. BMJ
328(7437):418-419.
Child, S., and A. Old. 2004. Resident medical officer working conditions in New Zealand:
Results of a recent survey. New Zealand Medical Journal 117(1204).
Chisholm, D. 2008. Junior doctors opting out. http://www.stuff.co.nz/sundaystartimes/
4464669a20455.html (accessed May 27, 2008).
Code du Travail (Labor law in France). 2006. Paris, France: Ministère du travail, des relations
sociale, et de la famille et de la solidarité.
Council Directive No. 93/104/EC. 1993. European working time directive. Council of the
European Union, Brussels.
Danish Medical Association. 2008. Duties—in brief. http://www.laeger.dk/portal/page/portal/
LAEGERDK/LAEGER_DK/ENGLISH/DUTIES_IN_BRIEF (accessed June 11, 2008).
Davis, P., R. Lay-Yee, R. Briant, W. Ali, A. Scott, and S. Schug. 2003. Adverse events in New
Zealand public hospitals II: Occurrence and impact. New Zealand Medical Journal
116(1183).
Dula, D. J., N. L. Dula, C. Hamrick, and G. C. Wood. 2001. The effect of working serial
night shifts on the cognitive functioning of emergency physicians. Annals of Emergency
Medicine 38(2):152-155.
Dwyer, T., L. Jamieson, L. Moxham, D. Austen, and K. Smith. 2007. Evaluation of the
12-hour shift trial in a regional intensive care unit. Journal of Nursing Management
15(7):711-720.
European Trade Union Confederation. 2006. Working time directive. http://www.etuc.org/
a/504 (accessed November 2, 2007).
Fok, M. C., A. Townson, B. Hughes, and W. C. Miller. 2007. Work hours, sleep depriva-
tion, and fatigue: A British Columbia snapshot. British Columbia Medical Journal
49(7):387-392.
Foundation Programme. 2007. Curriculum, Academy of Medical Royal Colleges, London.
Gander, P., H. Purnell, A. Garden, and A. Woodward. 2007. Work patterns and fatigue-
related risk among junior doctors. Occupational and Environmental Medicine 64(11):
733-738.
Gleason, A. J., J. O. Daly, and R. E. Blackham. 2007. Prevocational medical training and the
Australian curriculum framework for junior doctors: A junior doctor perspective. Medi-
cal Journal of Australia 186(3):114-116.
Greater Metropolitan Clinical Taskforce (GMCT) Metropolitan Hospitals Locum Issues
Group. 2005. Meeting the medical locum challenge—Options for action within the
NSW public hospital system. Sydney, Australia: GMCT.
Hospital at Night and the NHS Information Centre. 2008. The case for hospital at night—The
search for evidence. Manchester, England: NHS.
Institute for Healthcare Improvement. 2007. The hospital at night project: Reducing risks at
our most vulnerable time of day. Boston, MA: Institute for Healthcare Improvement.
OCR for page 359
APPENDIX C
Jagsi, R., and R. Surender. 2004. Regulation of junior doctors’ work hours: An analysis of
British and American doctors’ experiences and attitudes. Social Science & Medicine
58(11):2181-2191.
Jarvis, J. 2002. Response from an RMO. New Zealand Medical Journal 155(1166).
Landau, L. I. 2007. Quality junior doctor training, improved workforce outcomes and patient
safety. Australian Health Review 31(Suppl 1):S106-S108.
Lim, E., and S. Tsui, on behalf of the Registrars and Consultant Cardiac Surgeons of Papworth
Hospital. 2006. Impact of the European Working Time Directive on exposure to operative
cardiac surgical training. European Journal of Cardiothoracic Surgery 30(4):574-577.
Major shake-up in medical training, UK. 2005 Medical News Today. http://www.medicalnews
today.com/articles/29074.php (accessed November 12, 2007).
Mayor, S., J. Burgermeister, K. Kosner, T. Villanueva, A. Tuffs, B. Spurgeon, F. Turone, M.
Houston, and T. Sheldon. 2004. Over the limit? Student BMJ 12(September).
McGrath, B. P., I. S. Graham, B. J. Crotty, and B. C. Jolly. 2006. Lack of integration of
medical education in Australia: The need for change. Medical Journal of Australia
184(7):346-348.
Medical Council of Canada. 2008. Medical training/licensure system in Canada. http://www.
img-canada.ca/en/licensure_overview/licensure.html#undergrad (accessed February 19,
2008).
Medical Council of New Zealand. 2006. Annual report 00: Medical workforce survey.
http://www.mcnz.org.nz/portals/0/publications/2006.pdf (accessed May 29, 2008).
Mestres, C.-A., J.-M. Revuelta, and A. C. Yankah. 2006. The European working time di-
rective: Quo vadis? A well-planned and organized assassination of surgery. European
Journal of Cardiothoracic Surgery 30(4):571-573.
Meyers, F. J., S. E. Weinberger, J. P. Fitzgibbons, J. Glassroth, F. D. Duffy, and C. P. Clayton.
2007. Redesigning residency training in internal medicine: The consensus report of the
Alliance for Academic Internal Medicine Education Redesign Task Force. Academic
Medicine 82(12):1211-1219.
Ministry of Science Technology and Innovation. 2008. Working hours. http://www.workin
denmark.dk/Working_hours (accessed June 10, 2008).
National Recruitment Office for General Practice Training. 2008. GP careers: The training
program. http://www.gprecruitment.org.uk/gpcareers/programme.htm (accessed Sep-
tember 18, 2008).
NHS (U.K. National Health Service). 2004. Working time directive: Pilots programme re-
port. http://www.healthcareworkforce.nhs.uk/pilotprojects.html (accessed November
14, 2007).
———. 2007. European working time directive (EWTD). Coventry, UK: National Library
for Health.
———. 2008a. Modernising medical careers: Training in 00. http://www.mmc.nhs.uk/
default.aspx?page=318 (accessed September 18, 2008).
———. 2008b. The past—The principles of MMC. http://www.mmc.nhs.uk/default.aspx?
page=310 (accessed June 2, 2008).
NHS Employers. 2008a. Junior doctors’ terms and conditions of service and associated docu-
ments. http://www.nhsemployers.org/pay-conditions/pay-conditions-467.cfm (accessed
May 30, 2008).
———. 2008b. National Health Service hospital medical and dental staff and doctors in
public health medicine and the community health service (England and Wales): Terms
and conditions of service. NHS.
———. 2008c. Hospital at night. http://www.healthcareworkforce.nhs.uk/hospitalatnight.
html (accessed June 2, 2008).
OCR for page 360
0 RESIDENT DUTY HOURS
Nolte, E., and C. M. McKee. 2008. Measuring the health of nations: Updating an earlier
analysis. Health Affairs 27(1):58-71.
NZRDA (New Zealand Resident Doctors’ Association). 2007. Collective agreement. Auckland,
New Zealand: New Zealand Resident Doctors’ Association.
———. 2008a. Resident doctors’ association fact sheet. Auckland, New Zealand: New
Zealand Resident Doctors’ Association.
———. 2008b. What is NZRDA: About us. http://www.nzrda.org.nz/Site/About_Us/What_
is_NZRDA.ashx (accessed June 5, 2008).
O’Brien, M., J. Brown, I. Ryland, N. Shaw, T. Chapman, R. Gillies, and D. Graham. 2006.
Exploring the views of second-year Foundation Programme doctors and their educational
supervisors during a deanery-wide pilot Foundation Programme. Postgraduate Medical
Journal 82(974):813-816.
PAIRO-CAHO. 2005. 2005-2008 PAIRO-CAHO agreement, edited by the Professional As-
sociation of Internes and Residents of Ontario (PAIRO) and the Council of Academic
Hospitals of Ontario (CAHO). Ontario, Canada.
PAR-BC (Professional Association of Residents of British Columbia). 2008. Collective agree-
ment & constitution: Scheduling. http://www.par-bc.org/collectiveagreement.php/20 (ac-
cessed May 28, 2008).
PARIM (Professional Association of Residents and Interns of Manitoba). 2008. Collective
agreement: Article 4—duty hours. http://www.parim.org/50 (accessed February 22,
2008).
Powell, D. 2004. Key issues facing Resident Medical Officers (RMOs). New Zealand Medical
Journal 117(1204).
Productivity Commission. 2005. Australia’s health workforce: Research report. Canberra,
Australia: Commonwealth of Australia.
Rawnsley, A., K. Hurst, and M. Robinson. 2004. Clinical and education implications of shift
work. Medical Teacher 26(1):71-73.
Royal Australasian College of Physicians. 2007. Requirements for physician training: New Zea-
land 00 handbook. Wellington, NZ: The Royal Australasian College of Physicians.
Royal College of Physicians. 2002. European working time directive: European dimensions.
http://www.rcplondon.ac.uk/news/statements/doc_ewtd_european.asp (accessed June 10,
2008).
———. 2006. Designing safer rotas for junior doctors in the 4-hour week. Prepared for on
behalf of a multidisciplinary working group by N. Horrocks and R. Pounder. London,
England: Royal College of Physicians of London.
Royal College of Surgeons of England. 2007a. Rota planning: Guidance from the working
time directive working party. London, England: Professional Standards and Regulation,
Royal College of Surgeons of England.
———. 2007b. Safe shift working for surgeons in training: Revised policy statement from
the working time directive working party. London, England: Professional Standards and
Regulation, Royal College of Surgeons of England.
———. 2008. Lessons from surgery. Presentation by Bernard Ribeiro to Committee on Op-
timizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve
Patient Safety, March 4, 2008, Irvine, CA.
Scallan, S. 2003. Education and the working patterns of junior doctors in the UK: A review
of the literature. Medical Education 37(10):907-912.
Schiøler, T., H. Lipczak, B. L. Pedersen, T. S. Mogensen, K. B. Bech, A. Stockmarr, A. R.
Svenning, and A. Frølich. 2001. Incidence of adverse events in hospitalized patients. The
Danish Adverse Event Study (DAES). Ugeskrift for Laeger 163(39):5378.
Sheldon, T. 2004. Pressure mounts over European working time directive. BMJ (Clinical
Research Ed.) 328(7445):911.
OCR for page 361
APPENDIX C
Skinner, C. A. 2006. Re-inventing medical work and training: A view from generation x.
Medical Journal of Australia 185(1):35-36.
Thomas, E. J., D. M. Studdert, H. R. Burstin, E. J. Orav, T. Zeena, E. J. Williams, K. M.
Howard, P. C. Weiler, and T. A. Brennan. 2000a. Incidence and types of adverse events
and negligent care in Utah and Colorado. Medical Care 38(3):261-271.
Thomas, E. J., D. M. Studdert, W. B. Runciman, R. K. Webb, E. J. Sexton, R. M. Wilson,
R. W. Gibberd, B. T. Harrison, and T. A. Brennan. 2000b. A comparison of iatrogenic
injury studies in Australia and the USA I: Context, methods, casemix, population,
patient and hospital characteristics. International Journal for Quality in Health Care
12(5):371-378.
Thorne, L., S. Burn, S. Shaw, B. Arvin, and R. Bradford. 2006. Neurosurgical trainees opera-
tive experience before and after introduction of the new deal for junior doctors. British
Journal of Neurosurgery 20(1):31-35.
Vincent, C., G. Neale, and M. Woloshynowych. 2001. Adverse events in British hospitals:
Preliminary retrospective record review. BMJ 322(7285):517-519.
West, D. 2007. European working time directive implementation and cardiothoracic training:
Larger centers may optimise training [comment]. European Journal of Cardio-thoracic
Surgery 31(5):958.
West, D., M. Codispoti, and T. Graham. 2007. The European Working Time Directive and
training in cardiothoracic surgery in the United Kingdom: A report of the specialty ad-
visory board in cardiothoracic surgery of the Royal College of Surgeons of Edinburgh.
The Surgeon 5(2):81-85.
Wilson, R. M., W. B. Runciman, R. W. Gibberd, B. T. Harrison, L. Newby, and J. D.
Hamilton. 1995. The quality in Australian health care study. Medical Journal of Australia
163(9):458-471.
Woodrow, S. I., C. Segouin, J. Armbruster, S. J. Hamstra, and B. Hodges. 2006. Duty hours
reforms in the United States, France, and Canada: Is it time to refocus our attention on
education? Academic Medicine 81(12):1045-1051.
Workforce Taskforce. 2007. Reshaping medical education and training to meet the challenges
of the st century: A report to the Ministers of Health and for tertiary education from
the Workforce Taskforce. Wellington, New Zealand: Ministry of Health.
World Health Organization. 2007. World directory of medical schools: Final updates through
December 00. http://www.who.int/hrh/wdms/en/ (accessed September 19, 2008).
OCR for page 362
