National Academies Press: OpenBook

The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs (2002)

Chapter: 6 Improving the Disability Decision Process

« Previous: 5 A Work Disability Monitoring System
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 113
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 114
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 115
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 116
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 117
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 118
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 119
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 120
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 121
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 122
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 123
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 124
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 125
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 126
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 127
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 128
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 129
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 130
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 131
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 132
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 133
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 134
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 135
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 136
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 137
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 138
Suggested Citation:"6 Improving the Disability Decision Process." Institute of Medicine and National Research Council. 2002. The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs. Washington, DC: The National Academies Press. doi: 10.17226/10411.
×
Page 139

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Improving the Disability Decision Process This chapter briefly discusses the key issues identified in the com- mittee’s preliminary assessment of the Social Security Administration’s (SSA’s) research plan to redesign the disability decision process (IOM, 1998), and the subsequent decision by SSA to terminate this redesign effort and explore ways to incrementally improve the current process. The chapter then discusses and makes recommendations on the research needed to bring about fundamental improvements in the current dis- ability decision process. BACKGROUND Determination of eligibility for disability benefits under the Social Security Disability Insurance (SSDI) and Supplemental Security Income (SSI) programs is an inherently difficult task. To qualify for benefits under these programs a person must have a medically determinable impair- ment. Although the existence of a medically determinable impairment is a necessary condition, it is not a sufficient condition for receipt of benefits. The statutory definition makes clear that this program deals with work disability. The applicant is considered to be “disabled” (as defined by the Social Security Act) not just because of the existence of a medical impair- ment, but because the impairment precludes gainful work (Hu et al., 1997) anywhere in the national economy, taking into consideration the person’s age, education, and work experience, which are commonly referred to as vocational factors. Disability determination is a complex 113

114 THE DYNAMICS OF DISABILITY process, inescapably involving some interpretive judgment about capac- ity for work (GAO, 1994; Hu et al., 1997). At a minimum, making such decisions requires clinical determination of the extent of a claimant’s physical, mental, or sensory impairments; analysis of the degree to which such impairments limit the claimant’s functional capacity relevant to work roles; and consideration of the interaction of the claimant’s physical, mental, or sensory impairments with the person’s age, education, and work experience to provide an overall picture of the claimant’s future capacity for any sort of work. Finally the disability decision process requires a means for comparing those capacities with the capacities demanded by work roles in all jobs in the national economy that provide substantial gainful activity (SGA) earnings level. While many of the factual determinations are relatively straight- forward, others range from the difficult to the nearly impossible. For instance, while measures of visual acuity are reasonably well understood and can be readily translated into sensory limitations, the measurement of pain and its effect on function is much less amenable to objective deter- mination. The real demands of jobs in the national economy are con- stantly shifting in ways that make straightforward measures of functional capacity problematic guides to a worker’s true capacity for success in the workplace. Therefore, it is impossible to know precisely the extent of imperfection in the determination of disability, as evidenced by the lack of agreement observed in an examination of rater reliability as measured by the variations within and between states in the allowance rates by examiners (Gallicchio and Bye, 1980; DHHS, 1982). SSA has been strug- gling with these issues for more than 40 years in the face of high volumes of claims for adjudication (millions of claims per year decided by more than 10,000 adjudicators at various levels of the process) and high levels of legal challenge and political oversight. THE CURRENT DECISION PROCESS FOR INITIAL CLAIMS The standards for evaluating disability claims are specified in SSA’s implementing regulations (20 Code of Federal Regulation, parts 404 and 416, subparts P and I) and in written guidelines. These regulations and guidelines describe a sequential process for determining whether or not a claimant meets the statutory definition of disability. The purpose of developing the sequential decision process is to pro- vide an operationally efficient definition of disability with a degree of objectivity that can be replicated with uniformity throughout the country. SSA’s overall objective is to adjudicate claims as consistently, expedi- tiously, and cost-effectively as possible. As described briefly in Chapter 1,

IMPROVING THE DISABILITY DECISION PROCESS 115 the disability decision process for initial claims involves five sequential decision steps (SSA, 1994a). 1. In the first step, the SSA field office reviews the application and screens out claimants who are engaged in substantial gainful activity. 2. If the claimant is not engaged in SGA, step two determines whether the claimant has a medically determinable severe physical or mental impairment. 3. During the third decision step, the documented medical evidence is assessed against the medical criteria to determine whether the impairment meets or equals the degree of severity specified in SSA’s Listings of Impairments (Listings). A claimant whose impair- ment(s) meets or equals those found in the Listings is allowed benefits at this stage. The Listings serve the purpose of allowing rapid payment of benefits to claimants whose presumed residual functional capacity (RFC), given the severity of their impairments, would preclude work at virtually any job. 4. In the fourth decision step, claimants who have impairments that are severe, but not severe enough to meet or equal those in the Listings, are evaluated to determine if the person has residual func- tional capacity to perform past relevant work. Assessment of the RFC requires consideration of both exertional and nonexertional impairments. If a claimant is determined to be capable of perform- ing past relevant work, the claim is denied. 5. The fifth and final decision step considers the claimant’s RFC in conjunction with his or her age, education, and work experience to determine whether the person can perform other work that exists in significant numbers in the national economy. The determination in the fifth step is based on the 1978 Rules and Regula- tions, Medical–Vocational Guidelines (referred to as the vocational grid). The vocational grid, like the Listings, is intended to lend objectivity to the determination process and facilitate uniform administration of the voca- tional portion of the disability determination process. The grid reflects only physical (exertional) impairments. It does not consider non- exertional (e.g., mental or cognitive) impairments. The regulations also recognize that some claimants will have multiple impairments or envi- ronmental limitations (e.g., they cannot be around fumes) that are not effectively covered by the grid regulations. These cases must be decided outside the grid.

116 THE DYNAMICS OF DISABILITY Proposed Redesigned Decision Process SSA has stated that the redesigned disability decision process should • be simple to administer; • facilitate consistent application of rules at each decision level; • provide accurate and timely decisions; and • be perceived by the public as straightforward, understandable, and fair. The goal of the new decision process was to focus decision making on the functional consequences of an individual’s medically determinable impairment(s) (SSA, 1994a). Although the presence of a medically deter- minable impairment would remain the central requirement for eligibility as required by law, the redesigned process would focus directly, rather than indirectly, on the applicant’s functional ability to work and would rely on standardized instruments for measuring functional capacity to reach decisions. Medical and technological advances and societal percep- tions about the work capacity of a person with disabilities appear to sup- port a shift in emphasis from the current focus on disease conditions and medical impairments to that of functional inability. For example, people with disabilities are able to function today with personal assistants and assistive devices. SSA assumed that under this proposed decision process, the majority of disability claims would be evaluated using a standardized approach to measuring functional ability to perform substantial gainful activity. Stan- dardizing the approach to assessing individual functional ability would facilitate consistent decisions regardless of the professional training of the decision makers in the decision process. The new disability decision pro- cess, as envisioned by SSA, would assess a person’s functional ability once, relying on objective, standardized, functional assessment instru- ments. SSA believed that focusing decisions on the functional conse- quences of a person’s medical impairments would permit physicians and others who provide medical evidence, as well as decision makers, to use a consistent frame of reference for determining disability, regardless of the diagnosis and would facilitate evidence collection by reducing the need for developing extensive medical records (SSA, 1994a). In the proposed plan, decision makers would consider whether a person has a medically determinable impairment(s), but would no longer impose a threshold “severity” requirement. Instead, they would compare the individual’s impairment(s) against an “Index of Severely Disabling Impairments.” The Index would replace the Listings of Medical Impair-

IMPROVING THE DISABILITY DECISION PROCESS 117 ments (SSA, 1994a). The Index was to have described, quickly and easily, impairments that are so severely debilitating that, when appropriately documented, they can be presumed to equal a loss of functional ability for SGA without assessing the individual’s functional ability and without consideration of the person’s age, education, and work experience. The medical findings in the Index were to have been as nontechnical as possible and to exclude such things as calibrations or standardization require- ments for specific tests and/or detailed test results. The Index was to have been easy to understand and simple enough for laypersons to under- stand. SSA, therefore, believed that it would no longer need the concept of “medical equivalence” that is in the current decision process, thus eliminating one decision step in the current sequential evaluation pro- cess. If the claimant was not considered eligible for benefits based on the Index, then the person’s functional ability would be measured using stan- dardized instruments or protocols linked to clinical and laboratory find- ings (SSA, 1994a). The effect of the statutorily prescribed factors of age, education, and work experience would be considered when deciding the claimant’s ability or inability to engage in substantial gainful activity. The sequential process as it exists today and a new process as pro- posed in SSA’s redesign proposal are illustrated in Figure 6-1. SSA’S RESEARCH PLAN FOR A REDESIGNED DISABILITY DECISION PROCESS As directed by the Commissioner, SSA officials developed a research plan in 1995 for examining the feasibility of, and developing, the various components of the redesigned disability decision process. The plan had three components:1 1. Information gathering (comprehensive review and analysis of existing information) on • functional assessment instruments; • occupational classification systems; • disability determination processes used in other disability pro- grams in the United States and other countries; and • the effects of age, education, and work experience (vocational factors) of the applicant. 1SSA’s research plan, along with a time line for actions and completion dates, was pub- lished in the Federal Register in August 1996 and an update of the plan was published in November 1997 (SSA, 1996, 1997).

118 THE DYNAMICS OF DISABILITY Current Decision Process Proposed New Decision Process Earning SGA Earning SGA no yes no yes Severe Denial Denial impairments Impairments yes no yes no Denial Denial Meets medical In Index of listings impairments yes no yes no Capacity for Allowed Capacity for past work Allowed other work no yes no yes Denial Capacity for other work Allowed Denial no yes Allowed Denial FIGURE 6-1 The Social Security Administration’s current and proposed rede- signed disability decision process. SOURCE: Adapted from Hu et al., 1997, and reprinted from IOM, 1998. 2. Integration, synthesis, and development of a prototype for a new disability decision process: • analysis and evaluation of the literature reviews undertaken in the first component; and • development of prototype(s). 3. Testing, analyzing, and refining the prototype: • laboratory research and other small-scale testing; and • the National Study of Health and Activities (NSHA): a national sample survey to estimate the size and characteristics of the population eligible for disability benefits, determine factors per- mitting them to work, assess future changes in the prevalence of disability, and serve as one of the evaluation mechanisms for the decision process prototype(s).

IMPROVING THE DISABILITY DECISION PROCESS 119 Work on the first two components of the research plan was under way when this committee began its study. The research mostly involved contracts to conduct literature review and analysis in the areas of func- tional assessment instruments, occupational classification systems, dis- ability determination processes used in other disability programs, and effects of vocational factors in the disability decision process. Some of the information-gathering activities were completed when the committee began its review. Work on the second component and parts of the third component involved the award of a task-order contract at the end of fiscal year 1997. The purpose of this contract was to synthesize and integrate the results of the literature review contracts and the NSHA; to develop, test, evaluate, and refine alternative prototypes for a redesigned disability decision process; and to undertake additional research as needed. Under SSA’s redesign research plans, additional work in the research and devel- opment of the decision process would have been undertaken in subse- quent task orders under this umbrella contract and/or under separate contracts. In summary, SSA’s research plan aimed at developing and testing the functional assessment instruments in the disability decision process, examining the effect of vocational factors on decisions, exploring what is being done in other disability programs, and developing a prototype for a revised disability decision process. Committee’s Review of the Research Plan Early in the study, the committee conducted a preliminary review of SSA’s workplan and individual research projects completed and under way. The committee also explored other relevant internal documents pro- vided by SSA in response to requests for information. It heard presenta- tions from the staff of SSA on work completed to date, plans to integrate the results of the research projects and the NSHA to develop a redesigned disability decision process, and the time line for completion of all research. After reviewing the available documents and discussions with its contractors, the committee decided that a preliminary assessment of the adequacy of the research plan was needed to guide SSA management in determining whether the research activity undertaken was adequate and what more, or different, was needed to conclude whether or not the pro- posed revisions in the decision process were feasible, practical, and could be implemented nationally. According to SSA, the goal of this research is to devise a more effi- cient and more accurate method for making timely determinations of disability for Social Security claimants (SSA, 1996). In the context of that goal, the committee outlined an initial conceptual framework of issues,

120 THE DYNAMICS OF DISABILITY research steps, and methods for a research plan or design to develop and assess the proposed new decision process as a workable solution to cur- rent problems. Such a framework should delineate at the outset the nature and extent of the problem. Complaints, whether ultimately substantiated or not, often suggest that a program should be evaluated and improved. However, in order to assess the validity of the complaints, objective evalu- ative criteria should be established a priori, so that the various complaints about the program can be evaluated and the program’s performance can be measured. The next step would be to identify alternative solutions that might address the problems. Finally, the proposed new decision process should be tested to determine whether it is workable and whether it will alleviate the problems initially identified. To determine if a redesigned disability decision process would lead to improvements, one or more studies need to be conducted to provide information on how the current program is working relative to the established criteria. Analysis of data from such studies would identify the gaps between performance and the goals of the program. This framework is reproduced in Table 6-1. The committee then reviewed the general features and directions specified by SSA in its research plan and the individual projects within the plan with reference to each of the research steps identified in the framework. It reviewed and commented on both the completed research projects listed earlier and the scope of work in the relevant requests for proposals for the conduct of the research. It identified critical elements of a research design that were missing from SSA’s plan, expressed serious concerns about these gaps, and made recommendations for redirected and new research effort. The committee commended SSA for initiating the major task of re- designing to improve the disability decision process and undertaking a range of research activities related to the functional consequences of medi- cal impairments and for recognizing the need to assess the feasibility, validity, and reliability of a proposed redesigned decision process. Never- theless, the committee concluded in its second interim report (IOM, 1998) that the research completed, under way, and planned appeared to lack the necessary overall framework and lacked the critical elements of a well-designed research plan. Some of the key conclusions reached in that report are summarized below. In 1995, SSA contracted with the Virginia Commonwealth University (VCU) to review systems, methods, and instruments that measure a person’s functional capacity to perform activities and tasks, to develop a matrix of categories to classify these instruments, and to evaluate them to determine their potential application in the disability decision process. VCU’s main conclusion in its report was that no government or private entity is currently using functional assessment instruments specifically

IMPROVING THE DISABILITY DECISION PROCESS 121 TABLE 6-1 Issues and Methods to Be Addressed in a Framework for a Research Plan for a New Disability Decision Process Question Research Steps Research Methods 1. What is the nature Needs assessment • Special surveys and analytic and extent of the research studies problems with the • Assembly of existing internal disability decision and external data process? • Satisfaction surveys • Analysis of data from studies using established evaluative criteria • Focus groups 2. What alternative Identify alternative • Review and analysis of research solutions might options literature address these Small-scale testing • Specially targeted research problems? Field evaluation • Laboratory research and pilot studies and demonstrations • Field tests • Focus groups • Process engineering assessments • National surveys 3. Will the proposed Program evaluation • Clinical trials disability decision and transition to • Simulation process be workable, implementation • Evaluation studies of the and will it alleviate proposed decision process using the problems? the established criteria • Cost-effectiveness studies • Tests of the new decision process in selected sites SOURCE: Reprinted from IOM, 1998. for determining work disability benefits, and a global measure of func- tional assessment does not exist that would be a valid indicator of disabil- ity for all populations currently served by SSA. Such an instrument will likely have to be developed and tested. After reviewing the advantages and disadvantages of both functional and clinical assessment measurements for SSA’s needs, VCU concluded that objective functional assessment can and should be a component of the redesigned process. VCU, however, stopped short of constructing the global measure of functional capacity; instead, it recommended several steps SSA should take in moving toward its development. The committee expressed concern that SSA had not made clear the conceptual or theo-

122 THE DYNAMICS OF DISABILITY retical basis for believing that such a standardized, global instrument existed or can be constructed before launching the literature review project on the subject. Skeptical of one global, standardized, universally accepted measure, the committee recommended that SSA develop alternative research plans for development and use of functional measures in the disability decision process in the event that the proposed global standard- ized functional assessment instrument is not developed and tested (IOM, 1998). In 1996, SSA contracted with the American Institutes for Research (AIR) to conduct a comprehensive review of the literature pertaining to systems and methods of classifying occupations in terms of the physical and mental capacities required, to develop a taxonomy of occupational classification systems, and to assess the applicability of systems for SSA’s redesigned disability determination process. This review relates directly to one of the key elements in the proposed redesigned disability decision process, namely, assessing baseline work. The purpose of the review is to determine if a standard exists, and if not, whether it is feasible to develop one to describe basic physical and mental demands of a baseline of work. AIR concluded that while none of the occupational classification systems exactly or ideally matched SSA’s needs, the Occupational Information Network (O*NET) under development was the closest match to SSA’s needs. O*NET is an occupational classification system being developed by the Department of Labor (DOL) under contract with AIR to replace the Dictionary of Occupational Titles. One of the reasons AIR recommended O*NET over the other systems is because it uses level scales to measure the amount of skill needed to perform certain jobs. Incumbents choose a numeric rating based on their reading of the behavioral anchors. Cogni- tive and mental descriptors are also included in O*NET, but the physical ability scales that O*NET uses may not be specific enough to help SSA. Many other issues were identified by the contractor that need to be re- solved before O*NET can be used for SSA’s purposes. These issues are described in the committee’s interim report (IOM, 1998) and are discussed further later in this chapter. The committee questioned how O*NET will be used. SSA’s research design did not appear to be oriented to address this question. How does SSA plan to supplement O*NET with respect to contextual or other factors that are not well covered. There were no indi- cations in the research plan that the gaps in O*NET would be carefully considered and specific research identified to fill those gaps. The committee also was concerned about the synchronization of timing for completion of O*NET and SSA’s target completion of the research for development and implementation of the disability decision process. The committee recom- mended that SSA develop an interim plan for an occupational classifica- tion system in the event that the O*NET database is either not completed

IMPROVING THE DISABILITY DECISION PROCESS 123 or is insufficient to meet the needs of a new disability decision process. It also suggested that SSA should explore entering into an interagency agree- ment with the Department of Labor to initiate a version of O*NET that would collect information on minimum as well as average job require- ments to better serve SSA’s needs to assess ability to engage in substantial gainful activity. The committee made recommendations for changes in priorities and improvements in the research projects that were under way and others yet to be developed. It urged SSA to adopt a rigorous research design process and to develop, early in the research, objective validation criteria and validation plans to be able to make the ultimate judgments on whether or not the proposed changes would yield the desired results. In issuing the interim report the committee hoped that the recommendations embodied in that report would be incorporated in the contract research that was under way and in new research not yet initiated at the time. The committee’s detailed discussion, findings, conclusions on the various issues, and recommendations are embodied in its second interim report to SSA (IOM, 1998). All the recommendations flowing from the committee’s preliminary assessment are included in Appendix C. “POST-REDESIGN” PERIOD The committee had planned to examine and comment further on the adequacy of the entire research plan when completed, the results of the completed research, and any subsequently initiated research for the rede- sign effort. However, after the committee issued its second interim report (IOM, 1998), SSA undertook an internal reevaluation of its disability deci- sion process redesign initiatives. SSA concurred with several of the committee’s conclusions and some of the recommendations. However, rather than undertaking the additional research and redirection of the research as recommended by the committee, SSA decided to no longer actively pursue the new decision-making process proposed in Disability Redesign, but to improve the current process, focusing at this time on updating the Listings (SSA, 1999b). According to SSA, its new strategy is to concentrate on improving the overall adjudication process to ensure that decisions are made as accurately as possible, that those applicants who should receive benefits should get them as early as possible, and that the adjudication process is consistent throughout (SSA, 1999b). To make improvements, SSA has stated that it has redirected and refocused disability policy development and related research activities in order to address both the longer-term goals of redesign, and the more immediate, pragmatic needs of the disability programs (SSA, 1999b). SSA expects these process initiatives to

124 THE DYNAMICS OF DISABILITY • enhance the quality of decisions at all levels; • streamline the disability process by applying the lessons of the disability process redesign efforts; and • update medical and vocational rules used in making disability determinations. The initiatives to achieve the first two objectives relate to components of the reengineering process that are outside the purview of this commit- tee. The third objective—updating the medical and vocational guide- lines—is a direct outgrowth of the now abandoned initiative to develop a new decision process. Updating the Listings SSA states that although it is no longer focusing on development of the new decision process described in the disability redesign plan, it is continuing to explore the potential in some of those ideas. However, it is now devoting most of its resources to needed improvements to the cur- rent evaluation process (SSA, 1999b). SSA further states that the proposed new decision process was intended in part to address concerns about the current Listings by replacing it with an Index of Impairments. However, SSA has concluded that the Listings serve a vital role in ensuring the cost-effectiveness of the adjudication process. Currently about 60 percent of the allowance decisions are based on the Listings without developing and conducting a complete, in-depth functional and vocational analysis. Rather than replacing the Listings, SSA is now engaged in a concentrated effort to update and improve these medical guidelines (SSA, 1999b). Medical advances in both diagnosis and treatment have made updating the Listings long overdue. As stated by SSA, the general approach to revising a section of the Listings is to begin with its adjudicative experience and program knowledge. Having identi- fied an area of interest, medical literature is reviewed and, as warranted, experts in the field are consulted. If more extensive research is needed, a contract(s) may be negotiated to obtain the information. Medical experts from within SSA are consulted to develop an initial proposal. Functional Consequences of Impairments SSA informed the committee that “the proposed new decision pro- cess was to have relied on simple, objective readily available functional tools to assess an individual’s ability to work. Initial research conducted under contract by SSA has not shown a basis for believing that such a standardized, universally accepted global instrument applicable to indi-

IMPROVING THE DISABILITY DECISION PROCESS 125 viduals with physical and mental impairments can be constructed or that some similar approach would be possible. Further, some of the other assumptions on which the proposal for a new process was based (e.g., the assumption that these functional tools would be routinely used in clinical practice and, therefore, readily available without cost in approximately 75 percent of the cases), no longer seemed reasonable” (SSA, 1999b, p. 7). SSA further stated that this does not mean that functional ability and functional testing cannot be important components of SSA’s disability evaluation process. They can and they should. SSA has indicated that it continues to believe in the need for good information on functional ability and testing as a key part of the decision process. It has begun to focus on an alternative plan to use functional assessments in the current decision process and is addressing the issue in three different ways. 1. Ensure that functioning is appropriately considered within the cur- rent evaluation process, in terms of functional criteria that are part of SSA’s standards, and in terms of providing practical policy guidelines for the use of functional testing in the current process. Many Listings include functional criteria. As the Listings are re- vised, SSA will seek to achieve consistency, simplicity, and admin- istrative practicality of functional criteria. 2. Issue updated policy guidelines addressing the uses of functional assessments in the current decision process. The guidelines will address issues such as the evidentiary nature of functional assess- ment results, when to consider purchasing functional assessment tests, and the kind of tests to purchase. 3. SSA has asked its expert consultants to begin developing new ideas for ways to more closely investigate the use of functional testing in SSA’s disability decision process (SSA, 1999b). Committee’s Assessment of SSA’s Post-Redesign Plans This section addresses the necessary prerequisites for a scientifically sound approach to disability determination at SSA. Several of the key issues that the committee had identified earlier in the study in the context of the problems associated with SSA’s research plan for redesigning the disability decision process still have to be addressed with respect to the activities undertaken to improve the current process. Therefore, they bear emphasizing again in this report. SSA has stated that the purposes of incrementally improving selected components of the current sequential disability decision process are to enhance quality of decisions, streamline the process, and update medical

126 THE DYNAMICS OF DISABILITY and vocational rules used in determining disability. This effort then, like the previous redesign effort, calls for comparative judgments. It presumes analysis of baseline information from the current decision process to as- sess the effectiveness of the current decision process and compare it with similar analysis of changes in the new decision process. However, based on the information provided by SSA, the committee assumes that the agency has not conducted such baseline analysis with predetermined criteria for evaluating the components of the sequential disability decision process leading to the decision to redesign. Moreover, such analysis does not appear to have a place in the current research plan. SSA’s current research approach focuses mostly on the new decision and therefore fails to build in tests that may be critical to answering the com- parative questions and, ultimately, to the decision whether or not to adopt the changes in the current process. Baseline Evaluative Criteria Regardless of whether SSA attempts to redesign and develop a new disability decision process or leaves the current process in place and makes improvements within the individual components of the sequential process, SSA needs to establish objective measurable criteria against which the current process can be assessed. Studies should be conducted on the existing process and data analyzed in the context of the established criteria in order to identify the nature of the problems in the current process. Without such a capacity, proposals for “reform” may be proposals for “change,” but it is impossible to determine whether they are proposals for “improvement.” When the committee reviewed SSA’s research plan for the redesign initiative it was unable to conclude that SSA had put a satisfactory re- search plan in place (IOM, 1998). The research proposals and other docu- ments reviewed by the committee provided no indication that SSA had conducted any baseline analysis with predetermined criteria for evaluat- ing the Listings component, or for that matter any other component, of the sequential determination process leading to the decision to redesign the system. Moreover, such analysis did not appear to have a place in the research plan. SSA’s research approach mostly focused on the new deci- sion process and thus failed to build in tests that may be critical to an- swering the comparative questions and ultimately to the decision whether or not to adopt a new decision process. The same issues appear to exist today as SSA moves toward making incremental changes in selected components of the current decision pro- cess. In the absence of evidence to the contrary, the committee assumes

IMPROVING THE DISABILITY DECISION PROCESS 127 that the agency again has not conducted such baseline analysis leading to the current activities to improve the existing process incrementally. For example, in its original redesign plan, SSA had proposed to replace the current Listings with an Index of Disabling Impairments that would serve to “screen-in” the obvious cases without addressing func- tional capacity and vocational factors. However, nowhere had SSA speci- fied the levels of specificity and sensitivity that would be satisfactory for this Index. There was no attempt to determine whether the current List- ings satisfied the goals for specificity and sensitivity overall, or the degree to which those goals were satisfied by different Listings for different body groups and different conditions. Without such baseline evaluative criteria and analysis it seemed impossible to specify either what the problems were with the current Listings or whether some redesigned Index would do a better or worse job in relation to the agency’s goals for that particular screening instrument. Unfortunately, SSA appears to be going down the same path in its current efforts. Throughout the documents reviewed by the committee relating to the redesign research, including the scope of work for the research contracts and in presentations before the committee, SSA has recognized the need to test the new disability decision process by applying standards of valid- ity, reliability, sensitivity, specificity, credibility, and flexibility. In addi- tion, the stated objectives of the redesign also include requirements such as simplicity in administration, consistency, accuracy, timeliness, equity of decisions at all levels, and fairness. However, to the committee’s knowl- edge no measurement criteria have been established to test the current and the redesigned process along any of these lines. Measurement is the process of linking abstract concepts to empirical indicators (Carmines and Zeller, 1979). Various terms are commonly used to describe measurement. For instance, to determine the extent to which a particular empirical indicator(s) represents a given concept, one can exam- ine the reliability of the concept, that is, the reproducibility of a decision for each case within and/or across decision makers. But a process or indi- cator needs to be more than reliable if it is to provide accurate results. It must also be valid. Although the terms reliability and validity are often used together, they are not synonymous. A decision may be reliable but it may not be valid. Both reliability and validity reflect matters of degree. Validity represents a set of criteria by which the credibility of research may be judged. For example, it measures the degree of agreement between the disability decision and actual fact of disability. Moreover, validity has several meanings. These include construct validity; content validity; crite- rion validity, which includes concurrent validity and predictive validity; and study validity, which includes internal and external validity (Last,

128 THE DYNAMICS OF DISABILITY 1983). The definitions among the disciplines of logic, epidemiology, social science, and statistics do not always correspond. Effectiveness measures the extent to which the decision process is achieving its goals and purposes (Berk and Rossi, 1990), and the concept of effectiveness must always address the issue of “compared to what?” regardless of whether it is marginal effectiveness, relative effectiveness, or cost-effectiveness. Finally, efficiency measures the results of a process in terms of resources expended and time. The committee notes that the fields of both science and law pay unusual attention to the definition of terms. When the two fields intersect, it is particularly important to be specific about the terms used. For example, although the word “disability” is used by SSA, the actual phenomenon focuses on an attribute more narrowly defined than the inability to per- form the usual activities of daily living. It refers to the inability to engage in substantial gainful activity because of a medically determinable physi- cal or mental impairment leading to death or expected to last for at least 12 continuous months. Such distinctions are key to SSA’s use of terms such as “validity,” and here the legal use of the term is important to guide the scientific criteria. The appropriate construct to use in judging whether the current or the new decision process achieves its goals must take into account the legal language. The committee, however, recognizes that the actual fact of disability may be unobservable. For example, the number of persons with disabili- ties under the Social Security definition will vary as a result of judicial interpretations. Each aspect of the law is subject to differing interpreta- tions and judgments. Problems arise at each step of the sequential deci- sion process, whether one is determining if a person has an impairment that will result in death or that is expected to last at least 12 months or, more crucially, whether by reason of that impairment a person cannot engage in SGA. The difficulty of making such a decision, however, is not the issue. Rather there is no single true answer to the question of whether a person with that impairment should be expected to engage in SGA. Simply put, a “gold standard” does not exist. Therefore, it is necessary to substitute some criterion or target and assess how well actual determina- tions are meeting this target. Whatever SSA chooses as a criterion or target also must be disseminated to stakeholders and decision makers as soon as possible, along with a plan for validity assessment. Only with such openness will the validity assessments be accepted when they become available. The brief discussion of measurement terms and issues clearly demon- strates the need for SSA to specify early in the redesign effort what it means by the terms validity, reliability, sensitivity, specificity, credibility, flexibility, and all the other related terms that it uses; and how it plans to

IMPROVING THE DISABILITY DECISION PROCESS 129 measure them, that is, what measurable criteria will be used to assess these standards vis à vis the disability decision process research and the level of sensitivity and specificity it is willing to tolerate. The same criteria should then be used to evaluate the quality of both the current process of disability determination and any prototype to be tested. Listings of Impairments As stated in the previous section, SSA has decided to devote its atten- tion to updating the Listings but no timetable has been set for the comple- tion of the various phases of this initiative. “The Listings was originally designed to highlight readily identifiable disabling impairments. Many of the Listings have since evolved into complex and highly detailed diag- nostic requirements, demanding specialized medical evidence that may not be readily available from treating sources. Some, but not all, of the Listings consider the functional consequences of an impairment; however functional considerations vary significantly among the Listings” (SSA, 1994a, p. 11). The committee believes that it is indefensible that most Listings have not been reviewed and updated in more than 10 years. SSA has stated that limited staff resources, the need to address new legislative mandates during the 1990s, and the lack of adequate research on disabil- ity criteria to support Listings updates have been at least part of the problem. SSA had not made a comprehensive revision of the adult mental disorders listings since 1985 (OIG, 2000). The report of the Office of the Inspector General (OIG) states that by the late 1990s, the National Acad- emy of Social Insurance (NASI) (Mashaw and Reno, 1996), the General Accounting Office (GAO, 1998), and the Social Security Advisory Board (SSAB, 1998) also were expressing concern that SSA was not updating the Listings regularly, but was simply extending the expiration dates for a number of years when the Listings expired (OIG, 2000). According to the OIG report, SSA staff has acknowledged that during the 1990s they did not always have the necessary research in place to support proposing revisions to the Listings or other disability projects. SSA informed the committee in 1999 that it is correcting the situation; it has added 15 positions in the component responsible for Listings policy and has started to address some of the most critical needs. However, the committee is not aware of any attempt at this time to evaluate the cur- rency and consistency of the Listings based on specific criteria, or at least those groups of conditions that account for a significant proportion of the disability rolls. SSA appears to have made the decision to modify certain Listings without any attempt to first evaluate them and use the findings to guide the update of the Listings. The committee in its interim report to SSA (IOM, 1998) supported the

130 THE DYNAMICS OF DISABILITY conclusion of the Disability Policy Panel of NASI to give high priority to research related to the Listings as well as to evaluate the consistency of the presumptions underlying the Listings for different body systems (Mashaw and Reno, 1996). The committee stated that SSA should conduct the necessary research, prior to making changes in the Listings, to (1) determine whether or not the current Listings satisfy the agency’s goals for specificity and sensitivity, (2) determine whether or not these goals are satisfied consistently across the Listings for the different body groups or conditions, and (3) evaluate the options to correct the problems detected by these evaluations, as it develops any new list of medical impairments. The committee has not changed its position. It appears likely that the agency’s agenda for reform in this area will be driven as much by internal and external anecdotal concerns, including general perceptions of which Listings are the most outdated, as by any long-range strategy. Nevertheless, the committee believes that a success- ful process of Listings revision must be based on a systematic approach to evaluation, design, and testing. The committee has not seen any indica- tion of a plan for determining the specificity and sensitivity parameters for any existing or proposed Listing. Developing such parameters seems critical to both the scientific and the political validation of the Listings as a decisional tool. Because the Listings screen is meant to be used to identify clear cases of disability, one would expect this screen to be devised such that it is highly specific (seldom identifies false positives) and relatively sensitive (identifies some substantial number of true positives). The question for SSA is how specific and how sensitive. In order to undertake meaningful research on the validity of any medical listing, SSA must be able to specify the acceptable level of specificity and sensitivity by which it can validate the screen against those criteria. SSA provided the committee with a list of ongoing projects designed to update the Medical Listings and improve their performance. The com- mittee, however, has no information suggesting that baseline criteria were established at the outset or that any method was developed for validating the existing and proposed new Listings against those criteria. These are serious and difficult issues. As SSA moves forward to incrementally revise and reform the current decision process, it must be able to determine whether or not changes are improving the accuracy of the process. Indeed, it has to be able to make these determinations prior to the time that changes are implemented on a national basis. Whether or not specific Listings need to be improved and the direction of that improvement must await the results of the baseline evaluation and subsequent reevaluation. The committee in its interim report to SSA (IOM, 1998) recommended that early in the redesign effort SSA should specify how it will define,

IMPROVING THE DISABILITY DECISION PROCESS 131 measure, and assess the criteria it will use to evaluate the current disabil- ity decision process as well as any alternatives being developed. Because of the critical importance of this issue and the assumption that not much has been done in this area, the committee reiterates its position. In any scientific process, standards of acceptance or rejection are declared before, not after, data are analyzed. Similarly in an evaluation research process, evaluative criteria and validation plans should be determined by the agency early in the research process and not, as currently planned, after the Listings are identified for updating and changes are developed. Recommendation 6-1: The committee recommends that prior to making the changes in the current decision process, SSA should a. establish evaluative criteria for measuring the performance of the decision process; b. conduct research studies and analyses to determine how the cur- rent processes work relative to these preestablished criteria, and c. evaluate the extent to which change would lead to improvement. Analysis of data from such studies in the context of the established criteria would identify the nature of the gaps between what the program is supposed to achieve and its actual performance. Without these research steps and analysis, there is no objective way to conclude if the changes are more effective and more efficient than the existing process. Since SSA is devoting its attention to updating the Listings, this rec- ommendation is most applicable at this time to the Listings. However, the committee notes that the Listings apply only to one step (step 3) of the five-step sequential evaluation process for determining disability. The baseline evaluation recommended for the Listings should ultimately evaluate the total process and not just one component. The committee is encouraged to note the recent cooperative agree- ment awarded by SSA in December 2000 to the Disability Research Insti- tute (DRI) to undertake research for developing a process of validation of the Listings in order to assess them and to ensure that changes made actually result in improvements in the disability decisions. This research will (1) compile and list published articles in the literature pertaining to the validation of disability insurance decisions, (2) undertake a critical review of the literature on assessing validity and, primarily and most importantly, on the development of appropriate validation criteria, and (3) review methods by which these validation criteria could be operation- alized. Although completion and implementation of this project may help validation efforts for future revisions of the Listings and other compo-

132 THE DYNAMICS OF DISABILITY nents of the disability determination process, unfortunately no such input exists for the revisions currently under way or completed. The committee recognizes that SSA faces two major challenges in providing an appropriate research base for the improvement of the dis- ability decision process. First, it operates an ongoing program that requires continuous incremental adjustment in order to make appropriate decisions. Second, the environment of disability decision making is con- stantly shifting in ways that have unanticipated consequences for the current process and that generate movements for substantial reorienta- tion of the entire disability benefit programs. Moreover, it goes without saying that SSA should recognize the cost trade-offs when it sets targets for sensitivity and specificity, and other measures. Assessing Vocational Capacity Determining the Demands of Jobs in the National Economy. As indi- cated in the previous section, the Dictionary of Occupational Terminology (DOT) is no longer being updated by the Department of Labor, leaving SSA with no replacement. The DOT has served as a primary tool for determining whether a claimant has the capacity to work. The Depart- ment of Labor is replacing DOT with O*NET. The committee, in its pre- liminary assessment of the redesign research plan, had expressed its con- cerns that O*NET as it was being developed for DOL would not meet SSA’s needs (IOM, 1998). For one thing, it focuses on average rather than minimum requirements as needed by SSA. The committee also questioned how SSA planned to supplement O*NET with respect to contextual and other factors that are not well covered. Discussions at the workshop sponsored by the committee on Measuring Functional Capacity and Work Requirements (IOM, 1999a) pointed out the problems associated with using O*NET for SSA’s purposes. The DOL expects to use O*NET, as a compre- hensive database of work requirements for use in job training, job coun- seling, and job placement for the department’s employment and training programs and for use by individual state Employment Security Agencies in the extensive work that they do with workers who need jobs or who have recently become unemployed. As discussed at the workshop, although O*NET is very useful for DOL’s purposes, SSA’s purpose in defining the functional capacity to work for purposes of the disability legislation is very different from the purposes of the DOL in creating O*NET. SSA’s purpose is much more difficult. Moreover, the labor market and occupational literature indicate that there are many difficult measurement problems related to occupa- tion and job characteristics. Information developed by job incumbents is not always consistent with the information developed by job analysts,

IMPROVING THE DISABILITY DECISION PROCESS 133 and the information developed by job analysts is not always consistent with the views of workers’ supervisors. The Bureau of Labor Statistics conducts employer surveys that try to define the characteristics of a job that affect its pay levels, but even there measurement difficulties some- times exist. In addition, from the perspective of the worker—as with a disabled individual—it is often a bundle of capabilities that the worker brings to the job that makes the work experience a success or a failure. Workers with the same educational backgrounds have different skills, work ethics, and orientations to work. These in turn bring a different bundle of capabilities to a job, and their performance is affected by those capabilities. In addition, the task of developing a set of factors that cap- ture the essence of each occupation that makes practical sense is complex and difficult. Clearly, a great deal more careful research and experimenta- tion is required to evaluate what functional capacity to work really means and exactly how it would be applied to persons with disabilities. When the committee reviewed SSA’s redesign research plan, there were no indications in the plan that the gaps in O*NET will be carefully consid- ered and no specific research to fill those gaps was identified. The commit- tee, therefore, had recommended that SSA should develop an interim plan for an occupational information classification system until a more perma- nent solution is found. The committee also suggested that SSA enter into an interagency arrangement with the DOL to initiate a version of O*NET that would collect information on minimum, in addition to average, job require- ments to better serve SSA’s needs to assess ability to engage in SGA. Subsequent to the committee’s interim report, SSA revised the work requirements of its original task order contract on integration, synthesis, and development of the redesign process to focus solely on a comprehen- sive assessment of O*NET as a replacement data source for the current decision process. SSA did not necessarily expect this work to produce a comprehensive resolution to the problem. It believed, however, that it must complete such an analysis to move forward (SSA, 1999b). The final report of the contractor (AIR, 2000) identified several posi- tive and negative aspects associated with O*NET’s incorporation into SSA’s disability determination process. Some of the positive aspects iden- tified include the ability to (1) obtain consensus from a variety of sources on the set of 54 O*NET descriptors appropriate for use in SSA’s disability decision process; (2) ascertain the number and identity of occupational units that are represented at various intervals of the descriptors’ rating scales; (3) determine if the occupational units are sufficiently representa- tive for SSA’s use; (4) identify 33 occupational units that contain at least one sedentary or unskilled job, as defined by DOT, representing approxi- mately 3 percent of the 1,122 occupational units in O*NET; and (5) pro- vide excellent descriptions of occupations in the task lists that decision

134 THE DYNAMICS OF DISABILITY makers can use to help determine the specific activities that comprise the claimant’s current or previous occupations. On the other hand, several aspects of the O*NET structure and con- tent could lead to problems if SSA incorporated O*NET into the decision process. More than half of the occupational units had at least one domain for which the majority of descriptors were unreliable. The final report (AIR, 2000) emphasizes that a major overarching problem with O*NET is the numerical ratings. These ratings do not seem to be consistent across occupational units. The contractor’s analysis found that the ratings of more than half of the descriptors are unreliable. Moreover, the DOT titles are grouped by dimensions that are unrelated to worker characteristics or requirements of the O*NET descriptors. Several of the 54 selected descrip- tors contain O*NET ratings with interrater reliabilities lower than .70. The report concludes that the numerical ratings on O*NET descriptors, and therefore on any O*NET occupational unit, underlie the problems of O*NET. Therefore, SSA must exercise extreme caution in drawing infer- ences about the relation between specific numerical values on a rating scale and specific level of required functioning. The report further states that the foregoing concerns provide sufficient evidence to warrant SSA’s careful consideration of the quality of either analyst or incumbent ratings as conducted and proposed for O*NET. The report also suggests that O*NET’s descriptor data may not be as precise as they seem, resulting in measurement errors as well as improper interpretation of the severity of claimants’ impairments (AIR, 2000). Without an appropriate characterization of job requirements that can be matched to the vocational characteristics of disability claimants, SSA might be cast back into the era in which it relied extensively on the testi- mony of “vocational experts,” or their written evaluations, as the way to integrate claimants’ functional capacities, vocational factors, and the de- mands of work into an objective determination of their capacity to engage in substantial gainful employment. Barring some resolution, SSA will be left with no objective basis upon which to justify decisions concerning an individual’s capacity to do jobs in the national economy (AIR, 2000). SSA realizes that O*NET will not work for its needs without major reconstruction of the system. SSA is taking steps toward resolving the problems. The committee is informed that SSA has reopened its dialogue with DOL to explore other ways of incorporating information about the requirements of work into the decision process and is actively pursuing with DOL the issue of an occupational database on a national level to avoid two separate databases with separate funding. It is also meeting with the various associations of rehabilitation specialists, occupational and physical therapists, and workers’ compensation analysts. Private sec- tor stakeholders have organized an interdisciplinary task force. It plans to

IMPROVING THE DISABILITY DECISION PROCESS 135 meet with SSA and DOL to decide what is needed and how best to go about getting the information.2 Age, Education, and Work Experience. SSA has not updated the research base on the effect of age, education, and work experience on work disabil- ity that had been used in developing the medical–vocational guidelines of 1978, known as the “grid rules.” Since then, much has changed with regard to the relative importance of each of these factors. As part of the initiative to redesign the decision process, SSA included in its research plan the evaluation of the effect of vocational factors—age, education, and work experience—on the ability to work or adapt to work in the presence of functional impairment. To assist in deciding an appropriate way to incorporate into the redesigned disability decision process the specific statutory requirement to consider an individual’s vocational fac- tors in determining ability to work, SSA entered into a reimbursable agree- ment with the Federal Research Division of the Library of Congress to review and evaluate published literature and any other research pertain- ing to the subject. A report of the review was submitted to SSA in 1998. The findings of the review are summarized very briefly below. Although age strongly affects decision making under the vocational grid, the literature review of the existing literature suggests that age may have little or no independent influence on ability to work (as distinguished from the likelihood of being hired or retained). Rather than chronological age being a common contributing factor to declining capacities, current studies suggest that the population actually becomes much more hetero- geneous with respect to its functional capacity as it ages. Moreover, except for the relatively vague concept of “adaptability,” age does not seem to have a strong correlation with modal ability to work. Education is clearly an important factor in employability. It affects the ability to acquire new skills, and earning power is related to education level. It is especially a problem with mental impairments. However, edu- cation above basic literacy levels has an uncertain relationship to the abil- ity to do jobs that would produce substantial gainful employment. High levels of education are not necessary for jobs paying $8,400 per year. High levels of education may, of course, suggest that only the most debilitating injuries or illnesses would prevent substantial gainful employment by persons with such levels of educational attainment. In combination, these attributes suggest that education may be important as a vocational factor only at the upper and lower range of educational attainment, but not in the middle ranges. 2Personal communication, Sylvia Karman and David Barnes, Office of Disability, SSA, October 3, 2001, and December 4, 2001.

136 THE DYNAMICS OF DISABILITY Increasingly basic skills are a priority with employers in identifying new employees. Employers value basic skills that flow from education, including the capacity to learn new skills or information, much more than they value job-specific skills. As with age, the independent influence of work experience is difficult to evaluate. Work experience is certainly im- portant in terms of capacity to return to a worker’s own occupation after an injury or illness. However, the vocational factors are used most often in evaluating the capacity of workers to do jobs other than those that they have held before. In summary, the review raised questions about the utility of multiple gradations of educational attainment in evaluating the vocational factors in disability determination and the utility of making determinations based on a worker’s transferable skills. Existing knowledge concerning voca- tional factors and their impact on the ability to perform jobs in the national economy raises challenging questions about the continuing validity of the approach taken by SSA’s existing grid rules. It suggested a critical need for a program of research designed to validate or reform the use of voca- tional factors in SSA’s disability decision process. SSA recognizes that it may have to make significant revisions to the rules it uses to determine disability, especially in light of the changes that the Department of Labor is making in its occupational data. SSA’s current rules, especially the grid rules, are based in part on both the organiza- tional structure and the data content of the DOT. Without it, those rules will probably have to be revised in a fundamental way. SSA also recog- nizes that such a revision might also necessitate a change in the way it incorporates evaluation of age, education, and work experience in its disability decision-making process.3 Variations in Disability Allowances. As shown in the previous chapters, over the past two decades the number of disability beneficiaries as a share of the civilian labor force has risen steadily. Although applications for benefits have increased only moderately, the number of new beneficiaries has nearly doubled. Disability allowance rates (awards as a percentage of applications) have varied over time from 31.4 percent in 1980 to nearly 47 percent in 2000 (see Table 2-1). Variations in allowance rates occur for several reasons. For example, SSA’s standards for judging claims differ over time. Dramatic reductions in approval rates occurred when standards were abruptly tightened in 1980 and then subsequently made liberal. Significant differences are observed in approval rates across states, between the state Disability Determination Service (DDS) decision mak- 3Personal communication, David Barnes, Office of Disability, SSA, December 14, 2001.

IMPROVING THE DISABILITY DECISION PROCESS 137 ers, and between DDSs and administrative law judges (ALJs). The approval rate is also influenced by legislative changes as well as court decisions. The adequacy of resources to process and review cases also affects the disability allowance rates. Increased research is needed to explain the variation in the rates at which applications for disability benefits are approved, why these changes take place, and whether they are predictable. To what degree are the growth and changes in disability allowances related to societal factors, and to what degree have they been influenced by changes in the program rules and operations? Such research should involve examination of the disability decision-making processes and the standards applied by SSA, the differences among states, and the differences among DDSs and the ALJs and among states. For instance, in the short run, changes in the prevalence of impairments are not as likely as changes in the way SSA evaluates the various impairments. Untangling the effects of the demand side (the growth in the number of SSDI and SSI applicants) and the supply side (the SSA disability decision processes) and prescribing remedies is difficult, but careful research in these areas will help shed light on this comparatively neglected area. Recommendation 6-2: The committee recommends that the Social Security Administration conduct research on a. improving the ability to identify and measure job requirements for the purpose of determining work disability; b. investigating the role and effects of vocational factors in the dis- ability decision process; and c. understanding reasons for variations in allowance rates among states and over time. ALTERNATIVE APPROACHES TO THE CURRENT DECISION PROCESS The objectives of the current disability decision process appear to be an attempt to make accurate decisions about the capacity to engage in substantial gainful employment consistent with the statutory definition of disability as consistently, expeditiously, and cost-effectively as possible within a system that is hierarchically accountable and makes determina- tions at a relatively low cost. Mashaw (1983) refers to it as “bureaucratic rationality.” Although interpretive judgment is clearly necessary in adju- dication, the process has become an increasingly rule-bound system that strives, with greater or less success, to decide similar cases in a consistent

138 THE DYNAMICS OF DISABILITY manner, in accordance with the statutory definition of disability, and in a timely and efficient manner. Bureaucratic rationality, however, is not the only model of an adjudi- catory process that might be applied to disability benefits determination. Some parts of the process, in particular decision making at the ALJ level, are more like an adversary adjudicatory process. Moreover, one could imagine the process as one that looks like the Internal Revenue Service system in which “claims auditors” might have the capacity to grant, deny, or even “settle” claims and would then defend those decisions at sub- sequent levels of review. Workers’ compensation and unemployment compensation systems provide other examples of adversary models of benefits adjudication. Another, radically different approach would conceive of disability benefits designed to assist claimants in receiving appropriate medical attention and vocational rehabilitation as well as appropriate income sup- ports. In this model the basic goal of the program would be to move claimants back toward productive work and to use benefits both as a means to facilitate the return to work process and as an ultimate fallback for those claimants whose impairments make continued work impossible. This is the approach used by many private disability insurers who man- age employment-based disability plans in the United States, and it is the dominant model in certain foreign systems, such as those in Sweden and Germany. Concern also is expressed that environmental factors, including envi- ronmental barriers to work are not taken into consideration in defining work disability. Recent legislation makes clear that Congress is increasingly inter- ested in the “return-to-work” model and is prepared to have SSA experi- ment with some alternative strategies that might facilitate the pursuit of work rather than benefits. The Ticket to Work and Work Incentives Im- provement Act of 1999 (P.L. 106-170) was signed into law on December 17, 1999. One major provision of the law establishes the Ticket to Work and Self Sufficiency Program, or Ticket Program. This provision provides that eligible SSDI and SSI beneficiaries with disabilities will receive a ticket (or voucher) they can use to obtain employment services, vocational rehabili- tation services, or other support services from an approved provider of their choice. The law also expands Medicaid and Medicare coverage to more people with disabilities who work. Recommendation 6-3: The committee recommends that the Social Security Administration initiate a research program for testing decision process models that emphasizes rehabilitation and return to work.

IMPROVING THE DISABILITY DECISION PROCESS 139 In conclusion, although SSA has deferred a major redesign of the disability decision process, the committee believes that it is paramount that the determination of disability not only be timely, understandable, straightforward, and feasible, but also provide accurate and consistent decisions that are fair to the claimant and to the government. To this end the committee believes that SSA should undertake a systematic, long- term program of research—intramural and extramural—that provides baseline information on all key aspects of the current disability decision process and subsequent evaluative data on all future change aimed at improving the effectiveness and efficiency of the work disability determi- nation process currently in use in the United States.

Next: 7 Enhancing SSA's Research Capacity »
The Dynamics of Disability: Measuring and Monitoring Disability for Social Security Programs Get This Book
×
Buy Hardback | $68.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Society Security disability program faces urgent challenges: more people receiving benefits than ever before, the prospect of even more claimants as baby boomers age, changing attitudes culminating in the Americans With Disabilities Act. Disability is now understood as a dynamic process, and Social Security must comprehend that process to plan adequately for the times ahead. The Dynamics of Disability provides expert analysis and recommendations in key areas:

  • Understanding the current social, economic, and physical environmental factors in determining eligibility for disability benefits.
  • Developing and implementing a monitoring system to measure and track trends in work disability.
  • Improving the process for making decisions on disability claims.
  • Building Social Security's capacity for conducting needed research.

This book provides a wealth of detail on the workings of the Social Security disability program, recent and emerging disability trends, issues and previous experience in researching disability, and more. It will be of primary interest to federal policy makers, the Congress, and researchers—and it will be useful to state disability officials, medical and rehabilitation professionals, and the disability community.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!