Appendix A
Workshop Agenda and Participants
AGENDA
Workshop on Field Evaluation of Behavioral and Cognitive Sciences-Based Methods and Tools for Intelligence and Counterintelligence
Tuesday, September 22, 2009
8:00 a.m. |
Workshop Check-In |
8:20 |
Welcome |
Barbara Wanchisen, director, Board on Behavioral, Cognitive, and Sensory Sciences, National Research Council |
|
8:30 |
Background and Committee Genesis |
Philip Rubin, committee chair and CEO, Haskins Laboratories |
EXAMPLES OF BEHAVIORAL TOOLS AND METHODS
Moderator: Robert Fein, forensic psychologist, Harvard Medical School and committee member
8:45 |
Preliminary Credibility Assessment Screening System (PCASS) |
Donald Krapohl, special assistant to the academy director, Defense Academy for Credibility Assessment |
9:15 |
Voice Stress Technologies |
Philip Rubin |
|
9:45 |
Break |
10:00 |
APOLLO |
Charles Twardy, research assistant professor, George Mason University |
|
10:30 |
Alternative Competing Hypotheses (ACH) |
Neil Thomason, senior lecturer (retired), University of Melbourne and committee member |
|
11:00 |
Response and Discussion Respondent Reactions |
Steven Kleinman, consultant and strategist, Intelligence and National Security Policy Paul Lehner, chief engineer, Information Technology Division, Center for Integrated Intelligence Systems, MITRE Corporation |
|
General Discussion |
|
12:30 p.m. |
Working Lunch |
MODELS OF EVALUATION IN OTHER FIELDS
Moderator: Philip Rubin
1:30 |
Challenges in Field Evaluation for Education |
Grover (Russ) Whitehurst, senior fellow, Governance Studies, and director, Brown Center on Education Policy |
|
2:00 |
Validation and Evaluation Practices in the Health Sciences |
Lisa J. Colpe, senior scientist, Division of Services and Intervention Research, National Institute of Mental Health |
|
2:30 |
Evidence-Based Practices in Criminal Justice |
Cynthia Lum, deputy director, Center for Evidence-Based Crime Policy and assistant professor, Administration of Justice Department, George Mason University |
|
3:00 |
Human Factors and Organizational Psychology: Application to Training Evaluation |
Eduardo Salas, professor, Institute for Simulation and Training, University of Central Florida and committee member |
|
3:30 |
Break |
3:45 |
Response and Discussion Respondent Reaction: Possible Application to Intelligence and Counterintelligence Community Needs |
Robert Boruch, professor, Graduate School of Education and Statistics, University of Pennsylvania and committee member |
|
General Discussion |
|
4:55 |
Closing Comments |
Philip Rubin |
|
5:00 |
Adjourn |
Wednesday, September 23, 2009
9:00 a.m. |
Workshop Check-In |
9:10 |
Welcome |
Barbara Wanchisen |
|
9:20 |
Background and Review of Day One |
Philip Rubin |
ETHICAL, REGULATORY, AND CULTURAL CONSIDERATIONS
Moderator: Philip Rubin
9:30 |
Ethical Challenges in Translating Psychophysiology and Neuroscience to Technology for Intelligence and Counterintelligence |
Adil Shamoo, editor-in-chief, Accountability in Research, University of Maryland School of Medicine |
|
Commentary |
|
Jonathan Moreno, David and Lyn Silfen University Professor, Department of Medical Ethics and Department of History and Sociology of Science, University of Pennsylvania and committee member |
|
Discussion |
|
10:30 |
Break |
INTERNATIONAL PERSPECTIVES AND APPLICATION OF LESSONS FROM POLICING
Moderator: Philip Rubin
10:45 |
A UK Perspective |
George Brander, Ministry of Defence, United Kingdom |
|
11:15 |
Canadian Defense Validation Efforts |
|
David Mandel, defence scientist and group leader, Defence Research and Development Canada, Toronto |
11:45 |
Application of Lessons from Forensics |
Christian Meissner, associate professor and chair, Institutional Review Board, Department of Psychology, University of Texas at El Paso |
|
12:15 p.m. |
Working Lunch |
1:15 |
Response and Discussion |
Committee Reaction General Discussion |
SUMMATION AND IMPLICATIONS
Moderator: Philip Rubin
2:45 |
Summation, Discussion, and Suggestions for Future Evaluations Summative Comments |
Robert Fein |
|
Committee Reactions General Discussion |
|
3:45 |
Closing Comments |
Philip Rubin |
|
4:00 |
Adjourn |
PARTICIPANTS
Planning Committee Members:
Robert F. Boruch, University of Pennsylvania
Robert A. Fein, Harvard Medical School
Jonathan D. Moreno, University of Pennsylvania
Philip E. Rubin (Chair), Haskins Laboratories
Eduardo Salas, University of Central Florida
Neil Thomason, University of Melbourne
Carol H. Weiss, Harvard University
Other Workshop Presenters:
George Brander, Defence Science and Technology Laboratory, Cyber & Influence Centre; UK Ministry of Defence Human Factors Team
Lisa Colpe, National Institute of Mental Health
Steven M. Kleinman, consultant and strategist, Intelligence and National Security Policy
Donald Krapohl, Defense Academy for Credibility Assessment
Paul Lehner, MITRE Corporation
Cynthia Lum, Center for Evidence-Based Crime Policy, George Mason University
David R. Mandel, Defence Research and Development Canada
Christian Meissner, Department of Psychology, University of Texas at El Paso
Adil E. Shamoo, School of Medicine, University of Maryland
Charles R. Twardy, George Mason University
Grover “Russ” Whitehurst, Brown Center on Education Policy
National Research Council Staff:
Laudan Aron, Division of Behavioral and Social Sciences and Education
Dan Melnick, National Research Council
Patricia Morison, Center for Education
Mary Ellen O’Connell, Board on Behavioral, Cognitive, and Sensory Sciences
Robert Pool, consultant
Miron Straf, Division of Behavioral and Social Sciences and Education
Barbara Wanchisen, Board on Behavioral, Cognitive, and Sensory Sciences
Renée L. Wilson Gaines, Board on Behavioral, Cognitive, and Sensory Sciences
Registered Attendees:
Hal Arkes, Ohio State University
Zunair Ashfaq, University of Pennsylvania
Emma Barrett, UK Ministry of Defence
Monique E. Beaudoin, National Defense University, Ottawa
Ronald Benefield, Office of the Naval Counterintelligence Executive
Sujeeta Bhatt, Defense Intelligence Agency
Anthony Boemio, Booz Allen Hamilton
Susan Brandon, Defense Counterintelligence and Human Intelligence Center, Defense Intelligence Agency
Troy Brown, Defense Academy for Credibility Assessment
James Bruce, RAND Corporation
Dennis Buede, Innovative Decisions, Inc.
Michael Cassidy, Marymount University
Ron Ceasar, Ron Ceasar Photography
Paul Chatelier, Naval Postgraduate School
Michael Cheek, ExecutiveBiz
A. Egon Cholakian, Interdisciplinary Research Development Fund Project
Brian Colder, National Geospatial-Intelligence Agency
Keith Devereaus, U.S. Department of Homeland Security-Citizenship and Immigration Services
Ivy Estabrooke, Office of Naval Research
Pamela Flattau, Institute for Defense Analyses Science and Technology Policy Institute
Angelyn Flowers, University of the District of Columbia
Melanie Goodrich, Office of Naval Research
Leslie Goodyear, National Science Foundation
Julie Gravallese, MITRE Corporation
Hal Greenwald, MITRE Corporation
Joe Heaps, National Institute of Justice
Kristin Heckman, MITRE Corporation
Cheryl Hendrickson Caster, American Institutes for Research
Georgia Holmer, Pherson Associates, LLC
Alexis Jeannotte, Avian Engineering
David Kamien, Mind-Alliance Systems, LLC
Eric Kaufman, National Counterterrorism Center
Heather Kelly, American Psychological Association
A.T. Kendall, social research consultant
Mary Lee Kingsley, M.L. Kingsley, LLC
Adam Korobow, Logistics Management Institute
Robert Krikorian, U.S. Department of State
Howard Kurtzman, American Psychological Association
Thomas LaHann, Science Applications International Corporation
Eric Lang, Defense Personnel Security Research Center
Thomas Lawrence, Pennsylvania Army National Guard
Richard Lempert, Department of Homeland Security Science and Technology
Heather Leon, Navy Marine Corps Intelligence Training Center
Tod Levitt, George Mason University
Tiffany Lightbourn, Department of Homeland Security–U.S. Citizenship and Immigration Services
Deborah Loftis, Defense Intelligence Agency
Martha Lorber, MITRE Corporation
Kel McClanahan, National Security Counselors
Gary McClelland, University of Colorado
Robert McCreight, George Washington University
Nancy Merritt, National Institute of Justice
Erica Michael, University of Maryland Center for Advanced Study of Language
Avra Michelson, MITRE Corporation
Thomas Moore, Defense Advantage Research Projects Agency
Dwayne Norris, American Institutes for Research
William Norris, Defense Academy for Credibility Assessment
J. O’Connor, Department of Homeland Security Human Factors/Behavioral Sciences Division
Jason Ogden, Customs and Border Protection
Randolph Pherson, Pherson Associates, LLC
Amy Pollick, Association for Psychological Science
Matthew Potts, Marine Corps Intelligence Association, Inc.
Jim Powlen, Logos Technologies, Inc.
Joseph Psotka, U.S. Army Research Institute
Leslie Richards, University of the District of Columbia
Steven Rieber, Office of the Director of National Intelligence
Michael Rugnetta, Center for American Progress
Adam Russell, Intelligence Advanced Research Projects Activity
Christina Saylor, U.S. Special Operations Command
Grace Scarborough, Pherson Associates, LLC
Dylan Schmorrow, Office of the Secretary of Defense
Rachael Scholz, Booz Allen Hamilton
Nathan Schwade, Sandia National Laboratories
Alan Schwartz, PolicyFutures, LLC
Dan Schwartz, Schwartz and Schwartz, LLC
Allison Smith, Department of Homeland Security Science and Technology Directorate
David L. Smith, Navy Marine Corps Intelligence Training Center
Robert A. Smith, University of Maryland
Jonathan Snider, Defense Threat Reduction Agency
Kevin Spence, Department of Homeland Security
Barry Spodak, Action Training Institute
Frank Stech, MITRE Corporation
Scott Tousley, MITRE Corporation
Edwin Urie, Henley-Putnam University
Anthony B. Veney, U.S. Department of Defense
Jeremy Wacksman, U.S. Department of the Navy, U.S. Department of Defense
Mark Weiss, National Science Foundation
Joe Wholey, University of Southern California
Deborah York, Phoenix Consulting Group
Laura Zimmerman, Applied Research Associates