| ||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 35
4
Post-Occupancy Evaluation Processes in
Six Federal Agencies
The federal government is the largest owner of Council who volunteered to provide information about
facilities in the United States. More than 30 individual their post-occupancy evaluation processes. In some
federal agencies own, use, and acquire facilities to sup- cases, more than one person in the agency responded
port agency missions and programs. Some federal and the responses were combined and reconciled. The
agencies conduct post-occupancy evaluation (POE) agency representatives were asked to respond to the
and lessons-learned programs as ways to improve cus- following questions:
tomer satisfaction, to increase building quality and per-
formance, and to facilitate organizational learning. This 1. Approximately what year did your agency estab-
chapter provides information about POE processes in lish a post-occupancy evaluation program? On
six federal agencies: the U.S. Air Force, Office of the average, approximately how many POEs have
Civil Engineer; the General Services Administration, been conducted by your agency in each of the last
Public Buildings Service (PBS); the Department of the five fiscal years?
Interior, National Park Service (NPS); the U.S. Navy, 2. What were the driving factors for establishing a
Naval Facilities Engineering Command (NAVFAC); the POE program?
U.S. Department of State, Office of Overseas Buildings 3. What is the focus of the POE information-gather-
Operations (OBO); and the U.S. Postal Service (USPS). ing process (e.g., user satisfaction, achievement
The six agencies are sponsors of the Federal Facili- of design objectives, building performance,
ties Council and volunteered to participate in the study. other)?
The information was gathered through a questionnaire 4. What are the expectations, goals, and objectives
and telephone interviews conducted by Krista Waitz of for the program? Have they been achieved? Have
Kwaitz Consulting. National Research Council staff there been unanticipated results?
wrote the summary of findings and descriptions of POE 5. How is the information gathered through POEs
programs. tied into the feedback loop (lessons learned) for
The study design was not a scientific one, nor was it planning, programming, and capital asset man-
based on random sampling. Thus, the information pro- agement?
vided should not be generalized. The remainder of this 6. Is the information gathered through POEs used in
chapter contains information about the survey ques- real estate decision-making and capital asset man-
tions, a summary of findings, and descriptions of the agement? If yes, please note what information is
POE programs in the six agencies. used, how, and when it is used. If no, why not?
7. Is it your agency’s policy to conduct POEs for all
buildings or for selected facilities? What are the
SURVEY QUESTIONS
criteria for determining whether a POE will be
In January 2001, a questionnaire was designed and conducted?
issued to six sponsor agencies of the Federal Facilities
35
OCR for page 36
36 LEARNING FROM OUR BUILDINGS
8. What do you consider to be barriers to conduct- for client satisfaction and for management-related
ing successful POEs? issues.
9. Who is typically involved in conducting a POE in
terms of in-house personnel and consultants?
POE Process and Technologies
(Please list positions or types of skills involved
not individuals.) What types of technologies are No two agencies use the same process or tools for
used? conducting POEs and capturing lessons, although some
10. What is the estimated cost in time and dollars for share common elements. The National Park Service is
conducting a typical POE? in the process of developing new procedures and tools
11. What data collection methods, technologies, and for conducting POEs and sharing lessons learned. Cur-
survey forms have been used over the life of the rently, at the NAVFAC, an independent agent remote
program? Please provide copies of survey forms from the designer of record conducts the POE using a
that have been used. statistically based questionnaire; a focus group discus-
12. To what extent does your agency make use of data sion is then conducted to summarize the results of the
management systems, Internet tools, or other survey. The NAVFAC criteria office administers
information technology applications to share survey results. The questionnaire is being modified so
information and disseminate results of POEs? that it can be administered from field agencies and can
be completed on-line or downloaded from the Web.
The survey documents and results can be downloaded
SUMMARY OF FINDINGS
from a database on the Web.
The Air Force, in contrast, uses a questionnaire
Establishment of POE Program, Timing of Surveys,
administered by a staff team; feedback is given prima-
and Number Conducted
rily to the construction agent, although the Air Force
Each of the six agencies studied had had a POE plans to also share the results with users, the base civil
program in place at least since the 1980s. The POE engineer, and the major command.
programs of the PBS, NPS, and NAVFAC were being The Office of Overseas Buildings Operations uses
restructured to meet new objectives, and the results of pre-trip user questionnaires, on-site interviews, and
the reorganized programs were not yet available. facilitated town meetings to gather the information,
The number of POEs conducted annually, on aver- which is then summarized in a formal report. OBO’s
age, ranged from less than 1 to 30. Post-occupancy POE is conducted by a multidisciplinary in-house team
evaluations are typically performed within 4 to 24 that is customized to address known deficiencies.
months following occupancy of a new or renovated The PBS performance measures-oriented approach
facility and are performed only once for an individual uses a set of questionnaires developed in cooperation
building. with the Center for the Built Environment at the Uni-
versity of California at Berkeley. The questionnaires
are being designed to be administered over the Web.
Focus of POE Programs
The lessons learned will provide input to the ongoing
Each of the six agencies used POEs to determine performance measures program of the Office of Busi-
client or user satisfaction at some level, but it also used ness Performance.
them to fulfill other objectives. These objectives The U.S. Postal Service uses two levels of surveys.
included determining building defects within the con- The first, a basic questionnaire that can be completed
struction warranty period, supporting design and con- in about an hour by the administrative service officer
struction criteria, supporting performance measures for and the postmaster of a new facility, is required for all
asset management, evaluating construction inspectors, new construction. For larger, more complex projects,
lowering facility life-cycle costs by identifying design POEs are conducted over two to four days using a
errors that could lead to increased maintenance and multidisciplinary team. Both types of surveys use elec-
operation costs, clarifying design objectives, improving tronic questionnaires in Microsoft Excel. The informa-
building performance, and supporting corporate sales tion gathered is sent directly to the staff maintaining
and image objectives. The restructured programs in the agency’s design standards.
NAVFAC and PBS are focused on developing metrics
OCR for page 37
37
POST-OCCUPANCY EVALUATION PROCESSES IN SIX FEDERAL AGENCIES
How POE Information Is Used tered through a headquarters’ office, there may be a
lack of field-level attentiveness to the process. Obtain-
Information obtained from POE programs has been
ing the clients’ attention to ensure adequate participa-
used by the OBO, NAVFAC, PBS, and USPS in sup-
tion in the survey or getting people not originally
port of their design criteria and guidelines. The NPS
involved with the project to participate in a survey was
and the Air Force programs anticipate using POE
also identified as a barrier. Organizational structures
results for the support of design criteria, among other
can also create barriers when responsibilities are
objectives. To date, none of the six agencies reported
assigned such that POE administration and database
that POE information was used directly in future real
development require interoffice collaboration.
estate decision-making and capital asset management,
although PBS’s POE program was being restructured
Costs
with those objectives in mind.
The costs reported to conduct POEs ranged from
Barriers $1,800 for a simple standard questionnaire that could
be completed in one hour to $90,000 for an in-depth
A number of barriers to more effective use of POEs
analysis including several days of interviews, multi-
and lessons-learned programs were identified. These
disciplinary teams, site visits, and writing up reports.
could be categorized generally as resources, feedback,
The costs did not include implementation of any
and participation and commitment.
changes resulting from a POE study. Costs per square
foot of space evaluated were not available. Other vari-
Resources ables accounting for the range of costs included
whether the facilities were located in the United States
Several agencies noted it was difficult to obtain or
or abroad, whether in-house staff or consultants were
earmark the funding needed to conduct POEs regard-
used, and how the resulting information was packaged
less of whether the POEs were to be conducted using
and distributed.
consultants or in-house staff. In some cases, in-house
staff may not be available to conduct the POE or may
DESCRIPTIONS OF POE PROGRAMS
not have the technical skills needed for quality results.
National Park Service, Department of the Interior
Feedback
In the mid-1980s the National Park Service com-
Because POEs often focus on identifying deficien-
pleted formulation and development of an extensive
cies, they risk becoming instruments to focus or deflect
POE program; however, due to changes in staff and
blame for unsatisfactory results. One agency cited the
downsizing of the NPS central design office, the
concern by senior executives that lessons learned may
program was not fully executed. In 1998, the POE pro-
be considered a weakness by Congress or the Inspector
gram was reinstituted as part of the business practices
General. Other agencies noted that conducting a focus
for the NPS central design office. At that time, it was
group to achieve consensus about the cause of failures
decided to reestablish the POE program for three
without judgmental discussion can be difficult, and as
reasons: (1) to create a feedback loop that would allow
a consequence, they may be reluctant to do so.
designers to interact with facility users and learn if
facilities were meeting the needs of the users; it also
Participation and Commitment served as an opportunity for users to assess their origi-
nal requirements and determine if they had adequately
One agency noted that because of construction
identified their needs; (2) to evaluate the effectiveness
schedule constraints, staffs may be focused on future
of consultant construction inspectors who had recently
projects or those under visible construction. Thus, once
replaced all NPS construction inspectors; and (3) to
a project has been completed and occupied, items such
improve long-term facility life-cycle ratio costs by
as financial closeout, construction as-builts, and POEs
identifying any design errors that could lead to
may not be a high-priority item and may not receive
increased maintenance or operational costs.
adequate oversight or attention. For programs adminis-
OCR for page 38
38 LEARNING FROM OUR BUILDINGS
Naval Facilities Engineering Command,
The focus of the NPS POE program is to improve
Department of the Navy
user satisfaction, building performance, and designer
efficiency. The central design offices for the NPS seek
The Naval Facilities Engineering Command has
to retain highly talented designers over a long-term
conducted POEs since the 1960s. In its original form,
career. By building relationships with facility users,
NAVFAC’s POE program had an instructional base
designers better understand user needs and anticipate
and was conducted by a project team after construction
requests, allowing for more efficient use of design
to identify lessons learned. In 1997, a new statistically
funds. The POE program seeks to build on this type of
based concept was initiated. The goal is to establish a
relationship through open communications and on-site
statistical basis from which NAVFAC can measure
review of completed construction projects. The NPS
agency improvement and work toward continuous
expects the value-added component of the POE will be
improvement by implementing process changes and
in lessons learned and improved design efficiency. The
modifying design criteria. The POE program is intended
NPS noted that maintenance and operations costs for
to apply to each completed facility within its warranty
facilities are escalating annually, and if the POE pro-
period. On average, 20 POEs have been conducted in
vides information for future designs that lead to
each of the last five fiscal years.
improved maintenance or operations, the payback will
There were two driving factors for restructuring the
be dramatic. In addition, if the POE provides feedback
POE program at NAVFAC. The first was a published
on products or techniques that improve user satisfac-
Department of Defense-level survey of occupant
tion, reduce maintenance, or improve operations, this
satisfaction with their facilities. The second was a
information can be shared with designers to reduce the
NAVFAC headquarters’ management initiative to cre-
cost of design development.
ate metrics measuring how, or if, the agency exceeded
Data collection tools and the forum for sharing infor-
client expectations. The agency focuses on improving
mation are still under development. However, the POE
client satisfaction, determining where its product lines
program is envisioned to include an evaluation form
or processes give rise to client dissatisfaction and
and follow-up meetings approximately six months to
improving the individual facility on which the survey
one year after completion of construction. The POE
is conducted. NAVFAC is issuing policy that will
meetings will include the users, project manager, and
require a survey of all facilities within 6-10 months of
design team captain. After completing the evaluation,
building occupancy. The focus of the POE informa-
the materials will be shared within the project manage-
tion-gathering process is to measure user satisfaction
ment and design divisions. The results of the POE will
from the perspectives of the building owner, the cus-
also be placed in the central technical information files
tomers (student, family occupant, day care family
and may be accessed by various levels of project man-
member, etc.), the building occupant at the working
agers, designers, and other technical staff.
level, and the staff maintaining the facility. Client
The NPS anticipates its POE teams will consist of
satisfaction at the user level is measured whether or not
(1) users, including park superintendent, chief of main-
the client was involved in the planning, design, or con-
tenance, park rangers, park interpreters, and adminis-
struction phases. The measuring process includes a
trative staff; (2) project managers, a multidisciplinary
survey of participants after which a focus group dis-
group consisting of park planners, architects, landscape
cussion is conducted to summarize the positive and
architects, and various disciplines of engineers—the
negative aspects of the facility.
technical expertise will vary by project; and (3) design
Currently, all POE surveys are conducted by an
team members. Generally a team captain for a building
“independent agent” remote from the designer of
project would be a senior architect; for a road construc-
record, typically a consultant. The NAVFAC criteria
tion project, a senior civil engineer or senior landscape
office administers the surveys. The criteria manager
architect would attend the POE; and for a utility project,
for the facility type identifies criteria issues and
a senior civil engineer would attend. Depending on the
action(s) to be initiated and implements interim guid-
complexity of the project, other members attending a
ance to the organization when required. The survey will
POE meeting could include a mechanical engineer or
be used in developing and modifying planning and
an electrical engineer.
design criteria. The data will be reviewed annually to
determine process improvement needs. NAVFAC is
OCR for page 39
39
POST-OCCUPANCY EVALUATION PROCESSES IN SIX FEDERAL AGENCIES
working to determine how the survey will interface the POE information-gathering process is on user
with its knowledge management system, combining to satisfaction, achievement of design objectives, and
yield a single lessons-learned concept for the organi- building performance, including interior flexibility and
zation. functionality.
When the POE program process began, NAVFAC The POE methodology followed was adapted from
used a checklist format focusing only on design and the U.S. Postal Service. Over the life of the POE
construction. The survey was changed to accommodate program, occupant surveys, on-site interviews, and
all of the agency’s processes (i.e., planning, design, facilitated town meetings have been used for data col-
construction, and maintenance turnover). Additional lection. The first step in the POE process is to send an
minor changes are anticipated to better assess safety occupant survey to an overseas post. The preliminary
and procurement issues. A copy of the survey is con- results from the occupant survey are used to determine
tained in Appendix D. what disciplines should be represented on the multi-
The database is being modified to Web-enable the disciplinary team that will be conducting the POE; thus,
survey content and to create a field-managed site as the team is customized to address known deficiencies.
opposed to a headquarters central database. In addi- Typically, architects; electrical, mechanical, and struc-
tion, the database is being made integral to corporate tural engineers; facility maintenance; and security
ORACLE-based management systems. It will draw specialists are involved in conducting POEs. Once at
information from the management system and alert the site, the team conducts a walk-through followed up
assigned individuals when it is time for the survey. by interviews with occupants. At the conclusion of the
site visit, members of the team reconvene to discuss
their observations and to generate recommendations for
Office of the Civil Engineer, U.S. Air Force
a report. Due to constrained resources, a POE may con-
The focus of the Air Force POE information- sist solely of the occupant survey.
gathering process is user satisfaction, achievement of One result of the POE program has been the revision
design objectives, and improved building performance. of design guidance on such topics as roofs, elevators,
The purpose of the POE is to note all defective work, and Ambassadors’ residences. POE results were also
report construction deficiencies to the construction used for developing a serviceability demand profile for
agent for correction by the contractor, and document generic embassy office buildings slated for design and
problems or mistakes made during design for use as in design guidelines for future embassies being acquired
lessons learned on similar projects. under a specialized procurement process.
The POE is conducted by a staff team using a ques-
tionnaire. The Air Force plans to share the results of
Public Buildings Service,
the POE with the construction agent, the user, the base
General Services Administration
civil engineer, and the major command for use in any
future designs and for incorporation into Air Force The Public Buildings Service (PBS) of the General
design standards. It is Air Force policy for a POE to be Services Administration (GSA) first instituted a POE
accomplished sometime during the ninth to eleventh program oriented toward design and construction crite-
month following beneficial occupancy (acceptance of ria development in 1977. In-house technical experts,
the facility by the user agency). including an environmental psychologist, architects,
and engineers, conducted the surveys. The program
was curtailed in 1982 due to a reduction in staff. PBS’
Office of Overseas Buildings Operations,
POE program was reinstituted in 1986, using contractor
Department of State
support. Between 1986 and 2000, the PBS completed
At the Office of Overseas Buildings Operations approximately 30 POEs for a variety of projects,
(formerly Foreign Buildings Operations) of the U.S. including courthouses, office buildings, U.S. border
Department of State, lessons learned and design or con- stations, major renovations, and historic restorations.
struction alerts have been issued since 1985. The driv- In 2000, the PBS restructured its POE program to focus
ing factors for establishing a POE program were the on performance measures for asset management. Both
concern for user satisfaction, comfort, and safety and a the design criteria-oriented POE program and the
general desire to capture best practices. The focus of performance measure-oriented POE program were
OCR for page 40
40 LEARNING FROM OUR BUILDINGS
intended to provide an information stream that would (CDs). A compendium of lessons was distributed every
inform program managers, criteria managers, and three years. In 1998, PBS prepared a Compendium of
project managers about design- and delivery-related Lessons Learned CD-ROM that was widely distributed,
problems and associated best practices. and a DVD (Lessons Learned, Volume 2) is being pre-
pared for mass distribution. The POEs are also in the
construction criteria database of the National Institute
Criteria–Based POEs
for Building Science.
Criteria-based POEs were developed to provide A second program that resulted from the POE les-
technology- and procedure-based feedback to those in sons is HVAC (heating, ventilation, air conditioning)
PBS’s central office responsible for national program Excellence in Federal Buildings. PBS held numerous
and design criteria direction. Equally important were workshops that included staff, architects, engineers,
the perceived benefits of offering those same lessons to and representatives from professional societies and
the delivery teams responsible for new projects. The technical organizations as part of an awareness pro-
primary focus was on building systems evaluation, gram to highlight HVAC issues.
client satisfaction surveys, and interviews with major
client agencies. The building systems evaluations
Performance Measure-Oriented POEs
included functionality issues as well as an overview of
how well the building complied with design criteria. The driving factor behind the shift to a performance
The surveys were customized for different types of measure-oriented POE program was PBS senior man-
buildings and project delivery systems, including office agement’s desire to evaluate how well PBS’s assets are
buildings, courthouses, border stations, and lease-build achieving their objectives on a project and program
or design-build projects. (An example is included in basis. Performance measure-oriented POEs are being
Appendix D.) Major lessons from the POE process pursued to help indicate whether delivery practices and
related to long-term building maintainability, building criteria are effective and to identify systemic problems,
functionality, client needs, and property manager and whereby specialty studies could be pursued.
asset management needs. Specific issues concerned The focus of this POE information-gathering process
energy efficiency; indoor air quality; heating, ventila- is financial asset assessment. A set of extensive ques-
tion, air conditioning, and electrical systems; thermal tionnaires is used in an attempt to identify customer
comfort; design of loading docks; access to equipment; satisfaction with various building components or
window washing; and accessibility for the physically features. Different questionnaires are directed to dif-
disabled. The criteria were constantly revised to incor- ferent key personnel, including operating staff and
porate lessons from POEs. design-delivery team members. The measures are
From 1986-2000, criteria-based POEs typically in- intended to help determine if GSA is meeting a number
volved a team of five to six design-related disciplines, of key management indicators including comparison
using outside architect-engineer professional services. of construction “pro forma” with final pro forma, main-
PBS central office coordination and involvement were tenance and cleaning costs benchmarked against
provided through site visits, access coordination, and national standards, utility costs, sustainability, energy
report critiques. A report and a 30-minute video were usage against FY 2010 goals, accessibility for the
created and distributed to each of GSA’s field offices physically disabled, and client satisfaction.
(11 regions) and to senior-level management at head- Support for questionnaire development and database
quarters. management is currently provided by the Center for
The tieback to lessons learned was direct: the people the Built Environment, within the University of Cali-
who conducted the POEs also developed the criteria. fornia at Berkeley. Questionnaire delivery and assess-
However, because the POE reports were voluminous ments are being coordinated by senior architects and
and oriented toward detailed evaluations of technology engineers within the PBS Office of Business Perfor-
applications, getting project managers and designers to mance. PBS is currently designing tenant, operations
read and adhere to findings was a challenge. Thus, vari- and maintenance, and design and construction survey
ous forms of information exchange media were applied tools that can be administered over the Web.
to mitigate this problem, including condensed video- The goal is to perform a POE for every Congres-
tapes and interactive “lessons-learned” compact disks sionally approved new building one year after full
OCR for page 41
41
POST-OCCUPANCY EVALUATION PROCESSES IN SIX FEDERAL AGENCIES
occupancy. Once the performance measure-oriented much more powerful support for overall corporate
POE program is fully implemented, 10-20 POEs will identity than previously realized.
likely be completed each year. The tieback from the Two levels of POEs are used currently. A basic POE
ongoing performance measure POE program will go (completing the questionnaire) is required for all new
from PBS’s Office of Business Performance to other construction and for owned facilities greater than 9,000
appropriate offices within the agency. square feet within four and six months of occupancy.
The administrative service office manager and the post-
master complete the basic POE questionnaire, which
U.S. Postal Service
typically takes 30 minutes to an hour. An example is
The U.S. Postal Service established its POE program included in Appendix D.
in 1986. The driving force behind the establishment of More extensive POEs are conducted on larger
the POE program was the desire to improve the plan- projects (more than 30,000 square feet) or other special
ning, design, and construction of future facilities. The projects. These POEs involve architectural or engi-
focus of the POE information-gathering process is user neering firms (including environmental psychologists
satisfaction (customers and employees), clarification of as consultants) and are conducted over a period of two
design objectives, achievement of design objectives, to four days. Customers and employees are inter-
building performance relative to technical systems such viewed, and extensive lighting and HVAC data are
as cooling and lighting, and supporting corporate sales gathered.
and image objectives and economics. On average, the The results of POEs go directly to the staff main-
agency has conducted approximately 30 POEs in each taining the Postal Service Building Design Standards.
of last five fiscal years. The information gathered through the POE process is
The first POE application was an effort to standard- not used in real estate decision-making or capital asset
ize the hundreds of Postal Service retail spaces pro- management; however it is used in planning, design,
duced each year. Appropriate design was found to be a and construction decisions. Real estate decisions are
much more powerful factor in customer satisfaction affected only as site-planning criteria are modified.
than had been anticipated. Also, building image was a
Representative terms from entire chapter:
user satisfaction
Items in cart [1]
