Attention to the technical aspects of security and privacy concerns were highlighted as major contributors to the building of trust among system stakeholders. Finally, the need to drive continuous innovation of technical approaches through constant testing and refinement and the creation of a supporting multidisciplinary research ecosystem were suggested.

Discussion of the strategic elements needing attention for the creation of a robust knowledge generation and use engine for the learning health

system encompassed issues ranging from cultural changes to the need for innovative methods development. Workshop participants suggested that a learning health system would not be possible without patients and clinicians buying into a shared learning culture. Consideration of approaches to facilitate the use of clinical records for research, such as the identification of core research-related components, was also discussed. Leveraging the full potential of health information by including sources other than just clinical records—such as patient-generated data and nonmedical health-related data—was discussed as an important strategic element. Finally, participants stressed the need to better develop innovative analytical methods to use distributed data repositories in order to address security and privacy concerns.

Maximally leveraging the digital infrastructure to better engage patients and the population in health was another principal focus of the discussions. Conveying the value proposition for stakeholder participation and creation of a shared learning culture among patients and the population were prominent themes. Participants discussed using the digital infrastructure to strengthen patient–clinician outcome partnerships through better patient portals and increased availability of lay-oriented, user-friendly clinical and nonmedical health data. Participants highlighted the need to call out the opportunity presented by a learning health system to aid in the elimination of health disparities and the role that a digital infrastructure could play to that end. Finally, the need for constant improvement through evaluation and innovation was discussed as an important component of an approach to patient and population engagement.

Explorations of the possible approaches to governance of the digital infrastructure for the learning health system were approached through the ultra-large-scale (ULS) lens, and drew from examples outside health care. Beginning with a discussion of the need to set a vision as a reference point for progress, participants explored the need to work toward identifying a minimal set of guiding principles to meet this vision while allowing for autonomy and innovation. Participant roles and responsibilities as well as delineation of the processes and protocols to be managed in support of the core learning functionalities were also identified as important components of a strategic plan. Finally, the incorporation of continuous evaluation and improvement in the approach to governance was also highlighted.

TECHNICAL PROGRESS

A ULS system is complex, constantly growing, and evolving, much like an organic, biological ecosystem. The digital infrastructure needed to support the U.S. healthcare system can be classified as a ULS system given its enormous scale including the numbers of agents, lines of code, and ever-expanding diverse sources of data; the preponderance of legacy systems that

must be incorporated; the local nature of health care and the corresponding requirement that each institution have autonomy; the specific regulatory, legal, and social requirements that must be met; and the understanding that it is too complex to be subject to effective central control. Introduced to the digital health information conversation by colleagues from the computer science field, hallmarks of a ULS system include preservation of local autonomy through decentralization of data, development, and operational authority. This allows for local innovation, personalization, and emergent behaviors without requiring consensus from all nodes. In discussions focused on developing a set of strategic scenarios for technical progress, the ULS system approach emerged as an appropriate framework since it would allow for empowerment through knowledge and control of health and health information; support a broad diversity of data sources and processes; support evolution and change; contain minimal, extensible standards; and leverage past work toward long-term goals.

In discussing the implications and issues surrounding this approach, participants identified the relevance and appeal of the engineering approach to health care—systems analyses, design, implementation, and evaluation plans—inherent to the ULS system perspective. Specifically, they noted the potential of a collaborative effort between the computer science and health information technology (HIT) communities to develop a deliberate and systematic engineering analysis—characterized by iterative testing and development of prototypes—to set technical and sociotechnical system goals, requirements, specifications, and architecture. This could be supported by a multidisciplinary research community, armed with clarified terminology for ease of collaboration, and with participation from a wide array of both private and public stakeholders (computer science, health informatics, law, policy, ethics, etc.). Similarly, workshop participants stressed the need for technical policies that support experimentation and innovation and allow for the progressive adoption and evolution of system requirements, specifications, and architecture choices.

Participants pointed to a focus on functionalities consistent with ULS systems, and their application to the digital health system, as a potential starting point in advancing the ULS approach. Definition of the ULS principles and characteristics that support learning system functionalities, including the feedback and feedforward nature of the learning engine, such as identification strategies, privacy controls, the availability of a complete longitudinal record at the point of care, inferential capacity, and research-readiness, were highlighted as critical foundational steps in the development of this technical enterprise. Noted as similarly important to system functionality was the mechanism for developing and maintaining an approach to information structure, classification, and storage.

Promoting these targeted functionalities requires advancing parsimoni-

ous system specifications and interoperability. Discussions centered on the need to specify the minimum set of standards to allow for partial interoperability. A focus on semantic comparability, maintenance of context and provenance, architectural consistency, and transportability were discussed as potential starting points. In congruence with the priorities laid out subsequently in the PCAST report (see Appendix E), particular attention was paid to the use of metadata to facilitate interoperability and information exchange—including to maintain data context and provenance, authentication, and privacy. This, in concert with a fast-prototyping component, can allow for incremental specification and system growth with the opportunity for functional enhancement, such as refinement of semantic interoperability, to meet specific requirements depending on use.

Part and parcel with the need to address the technical specifications of the digital utility for the learning health system is consideration for how these interface with users. Considerations for workflow integration were discussed by workshop participants as important to ensure that the technology is not only innovative and useful but also useable. To date, this disjuncture between established workflow patterns and an unfamiliar, often awkward, overlay of HIT tools has proved a substantial barrier to adoption.

Security and privacy safeguards were an important consideration in all areas of discussion. Participants often pointed to a lack of trust as being one of the major impediments to health information exchange. Therefore, attendance to the technical aspects of these issues was emphasized as a crucial part of building trust among stakeholders. Discussions and presentations (see Foster, Chapter 5, and Solomonides, Chapter 8) described technical approaches such as attribute-based authorization and distributed identity management, and provided examples of how they could be deployed to address these concerns and achieve a state of secure data liquidity. Additionally, innovations around data security and privacy in alternative environments such as hosted, web-based systems were suggested in order to build capacity.

Finally, the need for continuous innovation was a recurring theme in technical discussions. Participants suggested strategies such as creating a test-bed network for assessment of innovative system functionalities, the use of challenge problems to test ULS system issues and opportunities, and the cultivation of interdisciplinary research initiatives among academic, industry, and government stakeholders.

KNOWLEDGE GENERATION AND USE

Discussions of the generation and use of knowledge fell into three areas: the availability and capture of reliable data, the tools to analyze the

data, and seamless feedback of knowledge to the system. Research, quality improvement initiatives, and public health surveillance efforts are all examples of uses and drivers for these learning-associated processes.

A necessary precondition for successful progress on any of these dimensions is a shared learning environment. Technical advances and innovative research methods make it possible to bring clinical research and clinical practice much closer together. However, it was noted that the ability to take advantage of that opportunity depends on a healthcare culture in which both patients and clinicians are compelled by the prospects of clinical data to improve understanding, care delivery, and outcomes as well as provide reliable, just-in-time information to assist decision making. For these reasons, participants highlighted the need for a learning environment that is supported, shared, and nurtured by both patients and clinicians.

Several tools and approaches currently exist to provide point-of-decision support and guidance. In the face of the number of interacting factors, competing priorities, and an ever-growing set of diagnostic and therapeutic options, “best practice” can only be a theoretical notion without the ability to bring the best available information to the decision process. On the other hand, it was noted that reminders and decision prompts not successfully engineered into natural workflow patterns will be little more than ignored distractions. Consequently, approaches are needed to better marshal reliable clinical information and guidelines in time, form, and content that is seamlessly accessed and used by clinicians and patients.

Participants identified a number of needs to be addressed in order for the digital health infrastructure to reach its full potential as a source of real-time clinical research insights. For example, clinical research activities require enlisting clinician support and involvement in research-ready clinical records on both quality and content dimensions for reuse in knowledge generation. The identification of a limited set of standardized core research-related components as basic elements across vendors and systems was one suggestion to facilitate individual and cooperative clinical research activities as well as sentinel event surveillance. Concerns over the reliability and heterogeneity of data in clinical records were underscored as an important rate-limiting factor for both quality of care and clinical research activities, again highlighting the importance of the mechanisms for information structure, classification, and storage. This is particularly important for repurposing data collected for other uses, such as Food and Drug Administration (FDA) clinical trial–associated data, in order to maximally leverage efforts and investments already in place.

Discussions on the increased utility of clinical records for research went hand in hand with those on the need to take advantage of information from patients and other sources. Patient-generated data can provide

a level of context that is impossible to capture through more traditional data collection methods. Initiatives to better develop, test, and improve the capture and use of these data so that they can be used to support research, quality improvement, public reporting, and patient care were suggested as priorities.

Similarly, efforts to promote the integration and use of data across various sources—clinical, public health, commercial—were emphasized as central to effectively leveraging the full range of information for progress in improving efforts aimed at populations as well as individuals. Included in this, and considered with a longer term vision, were growing information sources outside of “mainstream” health care, such as online forums and communities. In order for such proposals to be successful, it was noted that protocols must be developed to build interoperability as a natural and seamless element of data sources.

Storage and aggregation of data for the purpose of analysis and knowledge generation have been problematic given the security and privacy issues they entail. Discussions of current and ongoing efforts in the creation of distributed data repositories, such as those being used in FDA’s Sentinel Initiative and the HMO Research Network, suggest a promising approach. Coordination between these ongoing efforts, additional support and incentives for their use for clinical research activities, and the support of coordinated intervention-specific patient registries were discussed as potential approaches moving forward. Prospects for the use of scalable, distributed, hosted, storage solutions—such as those used by Amazon—were also noted as promising future directions. These discussions, however, were often punctuated with caution around privacy and security, components that participants felt needed further exploration and development.

Finally, considerable attention was paid to the development of methods, tools, and query capacity for the generation of knowledge needed to sustain a digital learning health system. In line with the ULS system architecture approach, and the creation and support of distributed data repositories, the development of capacity for national, distributed query-based research—including the ability to identify and track sentinel events and indicators—was identified as a strategic priority. To support this, and the continuing development and innovation around other analytical approaches, the importance of collaborative interdisciplinary networks of researchers was underscored. This was discussed not only for cooperative studies, but for cooperative engagement of issues such as strategies on consistent identifiers for patients, the use of modeling and simulation for knowledge generation, evaluation of approaches for the use of diverse data types and varying data quality, and development of methods for the use of information from mobile consumer devices and patient-generated data.

PATIENT AND POPULATION ENGAGEMENT

Discussions on the roles of patients and the public in growing the digital infrastructure for the learning health system were anchored strongly in the concept of reengineering the care culture to ensure the centrality of the individual patient in the care process—a concept underscored in the Quality Chasm report (IOM, 2001) that remains elusive. Signs of change are only beginning to appear as appreciation increases for the use of web-based information and the clinical and outcome advantages of a patient who is better informed and more involved. Often referenced in the discussions was the need for the establishment of a “new norm” around engaging patients and the population in health—both theirs and that of the population—through the use of the digital infrastructure. Basic to this “re-norming” is a deepened appreciation by patients and the general population for the personal and public benefits that are likely to occur, as well as a strong measure of confidence in the security of the system.

The value proposition must be apparent to the stakeholders. Communication of the value of a digital health infrastructure in the improvement of care coordination, quality, and, ultimately, the health of the population at large, was identified in workshop discussions as a fundamental priority. Furthermore, participants pointed out that, in order to be successful, the value proposition should be approached in the context of transparent conversations about privacy, security, and other impeding concerns. The use of case studies and quantitative assessments of the contribution of HIT to improved patient experiences and outcomes was discussed as a potential starting point.

A common theme across several workshop discussions was the value in fostering a shared learning culture among system stakeholders—in particular, a culture that recognizes the unique contributions that patients and the general population can make to the learning system as collaborators, not subjects. Activities that foster patient involvement in and support of knowledge generation, including illustrating the importance of patient preference information to improving care, were discussed as potential approaches to this issue.

Following the theme of “renorming” participation of patients and the population in health improvement, and building on the framework established by previous Institute of Medicine work in this area, discussions of the opportunity for strengthening patient–clinician outcome partnerships through the digital infrastructure were discussed. The development of templates and protocols that support the use of HIT to engage patients in decision making as well as tools for more effective provider–patient communication were proposed. An important element in this respect is providing patients with secure access to and control of their health infor-

mation. This includes further development of patient portals, building on technologies already widely accepted by consumers, and supporting efforts for increased information liquidity and control such as the Veterans Health Administration/Centers for Medicare & Medicaid Services Blue Button initiative.

In concert with these efforts, participants discussed the need to increase the availability and access to lay-oriented, user-friendly clinical and nonmedical health information. Investing in templates for form and content of information for the lay consumer, as well as gathering patient-derived data for care and delivery improvement were suggested as areas of focus. Indeed, the “new norm” was discussed as involving a focus on improving patients’ health, not just health care, by emphasizing health maintenance as a lifelong process that includes a patient’s actions and decisions outside of the clinical care setting. To this end, participants proposed providing individuals with useful information concerning their clinical encounters and the relevant state of evidence, as well as giving them more responsibility for utilizing this information in their own decision making.

HIT provides an opportunity for engaging populations not historically well served by the traditional healthcare community. For this reason, the potential of the digital health utility in the elimination of health disparities was discussed as a strategic priority for further attention and action. The impact of facilitating patient and population contribution to, and control of, their health information has the potential to address disparities in underserved populations.

The importance of a component of continuous evaluation and improvement in efforts for patient and population engagement in the digital health learning system was again emphasized. Areas of focus that were highlighted include ongoing assessment of patient preferences for use in tailoring of health plans, innovative approaches to confidentiality and privacy issues, and assessments of opportunities to use contemporary sociotechnical approaches (e.g., social networking and smart phones) for patient and population engagement.

GOVERNANCE

Discussions of governance strategies for the digital infrastructure for the learning health system focused on facilitating activities to advance some very basic components and principles of the ULS digital health information system. Participants often struggled with the question “what are we proposing to govern?” and certainly the health information system as it exists now does not easily fit into most established governance models. On the other hand, upon applying the ULS lens to this issue, and considering innovative governance approaches in cases outside of health (such as

VISA and the Smart GRID, see Appendix B for more information), certain governance-related strategic elements emerged. Participants often pointed to the example of the Internet Engineering Task Force as one example of a governance approach that, while created under different circumstances, reflects many of the same governing principles.

Of principal concern is the issue of the vision. As a means of establishing a reference point for progress, workshop participants articulated the need for work to establish a shared vision of the digital health utility for the learning health system. Prospective components noted for this vision include expectations, guiding principles, modus operandi, and an appreciation for the global perspective. Considerations of the differences between a structure that governs versus one that provides guidance were included in these discussions.

Participants noted that a governance model in line with the ULS approach would be one that identified and depended on a minimal set of guiding principles with which all stakeholders must comport, maximizing local autonomy over all other decisions. Tolerance of change and adaptability were additional characteristics that participants felt were important to incorporate. Exploring the most decentralized level at which these standards might be delegated and focusing standards on major functional requirements were proposed as starting points. Additionally, the importance of tailoring the governance approach to the local situation and needs was emphasized. A focus on the ability to use an inclusive (both/and) rather than a deterministic (either/or) approach was discussed as a foundational principle that encapsulated this thinking. A related issue discussed was the broader context of the governance enterprise. Participants discussed the need to include societal values such as trust, privacy, and fairness; fair information practices such as transparency and data collection and use limitations; goals of the health sector to improve quality of care and enhance clinical knowledge; technical concepts such as innovation; and economic aspects such as promoting efficiency and reducing costs.

Possible participant roles and responsibilities in the governance structure were identified as an important early step, and different approaches were considered. These included broad participation by all stakeholders, which was pointed out to be logistically very difficult; very narrow participation, which participants felt was unlikely to be successful; or a hybrid model, that incorporated both broad and narrow participation depending on the needs at that particular level. Some participants noted that multiple layers of governance were likely to be required to address concerns at the appropriate level whether local, regional, national, or international.

Several approaches to the establishment of a governance model were considered and discussed by workshop participants. Leveraging lessons through collaborative discussions among ongoing efforts—at both the na-

tional and local levels—and establishing a working group to begin collecting initial input were suggested as starting points. To enhance the efficiency of deliberative efforts, participants suggested coordinating these activities, potentially through the Office of the National Coordinator for Health Information Technology’s Health IT Policy Committee’s Governance Working Group; building upon and aligning existing policies, such as Health Insurance Portability and Accountability Act, agency regulations, and informed consent processes to encourage learning health system activities; and nurturing the interfaces with the international community.

A potential responsibility discussed for the governance structure was the stewardship of processes and protocols associated with learning health system functionalities. Participants noted that developing processes for proposing, reviewing, and validating protocols on key elements including data gathering, security, and use is an integral part of this approach. Ongoing stewardship responsibilities for the governing entity will involve monitoring and maintaining protocols, managing variability across participants, and devising an approach to provide incentives to stakeholders to conform to stated goals and principles. A related element discussed as a governance challenge was that of implementation phasing, or sequencing protocol development activities so that barriers to progress in an entrepreneurial environment are not presented by premature initiation of activity bounding exercises.

In the spirit of a continuously improving learning health system, a process for continuous evaluation and improvement of the governance entity and approach was emphasized as important. Areas highlighted included establishing an approach to ongoing assessment of progress and problems, systematic assessment of value realization for recognition and promotion of successful practices, and the support of research on governance and orchestration of the ULS digital health utility in the United States and globally.

COMMON THEMES AND PRINCIPLES

Several common themes recurred throughout the rich and varied discussion. These themes, included in Box 9-2 and summarized below, were reflected in discussions of each of the four focus areas (technical progress, knowledge generation and use, patient and population engagement, and governance), as well as the discussions around various strategic elements. They ranged from issues related to the culture and environment for learning to the centrality of the patient and the importance of flexibility and trust.

• Build a shared learning environment. HIT provides an opportunity to change the current environment in which health decisions are made to one of shared input and active participation from patients,

caregivers, and the population at large. Approaches discussed to developing this shared learning environment include the direct involvement and support of patient and population roles in the generation of knowledge through the incorporation of user-generated data, understanding the benefits of information use in patient care and population health improvement, and improving patient access to health information to allow for a more active role in care decisions.

• Engage health and health care, population, and patient. Many participants reiterated that in order to improve health outcomes for the nation, thinking must extend beyond clinical encounters, and even beyond the individual patient, to the population as a whole. This shift of scope brought into clearer focus several issues discussed, including the opportunity to use HIT and its associated information to build a concept of health that is about more than medical care and draws on seamless interface with information from nonmedical health-related sources to generate knowledge that allows for a more inclusive view of population health improvement.
• Leverage existing programs and policies. A foundational assumption during the discussions was the advantage provided by building on, and accelerating, the substantial recent progress, both nationally and internationally, with an emphasis on the importance of fostering coordination among these efforts to capture efficiencies and prevent unnecessary duplication and waste going forward. Participants often noted that recent policies and legislation have laid a foundation for this work, and that the resulting investments and progress can be leveraged to move toward long-term system goals.

• Embed services and research in a continuous learning loop. Meeting participants often underscored that a digital infrastructure that supports both the generation and use of knowledge cannot be effective unless it is integrated seamlessly within the processes from which it draws and is meant to support care delivery, research, quality improvement, and population health monitoring. Ease of use for health system stakeholders, attention to the effects on workflow, and the delivery of useful decision support at point of care were often mentioned in discussions.
• Anchor in an ultra-large-scale systems approach. One of the most prominent features of the discussions was the notion that the health system is a complex, sociotechnical ecosystem, and therefore necessitates a unique conceptual approach. Grounding this approach to coordination and integration of the digital infrastructure for the learning health system in the principles of a ULS systems approach was suggested by several workshop participants from the computer science community (see Box 9-3). The term “ultra-large-scale system” refers to the existence of a virtual system that has bearing on a social purpose—for example, improving health and health care—and in which a few key elements, such as interchange representation, may be standardized, but whose many participants have diverse and even conflicting goals, so adaptability is key. Institutions retain flexibility for innovation in their choices, and evolutionary functional change can be shaped by architectural precepts, incentives, and compliance assessment, but not by centralized control. ULS functionality is therefore facilitated by protocols that allow maximum practical flexibility for participants. Incorporating decentralization of data, development, and operational authority and control, this approach fosters local innovation, personalization, and emergent behaviors. Participants felt that this approach was well suited to the complex adaptive characteristics of the health system, and that it could serve as an anchoring framework for approaching both the social and technical components of the overall infrastructure.
• Emphasize decentralization and specifications parsimony. In line with the complex adaptive qualities of the health system outlined in the Quality Chasm (IOM, 2001) report and reiterated during the workshops, both the social and technical components of the digital health infrastructure require a framework that allows for tailoring to specific needs, local innovation, and evolvability. In this respect, the commonly repeated refrain was a call for the principle of parsimony and minimizing centralization that might constitute a barrier to entry: specify only the minimal set of standards or requirements

necessary for key functional utility, and push the maximum amount of control to the periphery. This approach is in line with strategies such as those suggested in the PCAST report for use of metadata for wrapping individual information packets to facilitate interoperability and health information exchange, in which a primary focus would be on development of the metadata standards.

• Keep use barriers low and complexity incremental. Similarly, incentives for broad participation in the digital infrastructure by all stakeholders was discussed as a crucial factor to its success. The proposal to keep the barriers for use of the infrastructure, such as deployment and operational complexity, low was articulated by workshop participants in order to allow for maximum participation at a baseline level, and allow for incremental complexity and sophistication where possible or necessary.
• Foster a sociotechnical perspective, focused on the population. From the outset of the discussions, participants pointed out that the major barriers to technical progress often lie in social and cultural domains. Acknowledging and engaging this fact was described as being crucial to success, with discussions centering on an approach that reorients future efforts to engage the patient more directly in the collection and use of information in a way that is most useful to them.
• Weave a strong trust fabric among stakeholders. Security and privacy concerns represent a strong threat to participation in, and therefore the success of, the sociotechnical ecosystem. Accordingly, they must be dealt with from both the social and technical perspectives. Participants emphasized the need for systems security to comply with all current requirements and regulations and retain an ability to evolve to meet future needs. In addition, continued honest communication to the public and other involved stakeholders about risks and benefits will be crucial to building a foundation of trust.
• Provide continuous evaluation and improvement. A learning system is one that assesses its own performance against a set of goals and uses the results of that evaluation to change future behaviors. Workshop participants articulated the importance that all components of a digital infrastructure must themselves function as learning systems.

REFERENCES

IOM (Institute of Medicine). 2001. Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.

Northrop, L., P. H. Feiler, B. Pollak, and D. Pipitone. 2006. Ultra-large-scale systems: The software challenge of the future. Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University.

This page intentionally left blank.