United States Government Accountability Office GAO Report to the Chairman, Subcommittee on Social Security, Committee on Ways and Means, House of Representatives June 2012 MODERNIZING SSA DISABILITY PROGRAMS Progress Made, but Key Efforts Warrant More Management Focus GAO-12-420 June 2012 MODERNIZING SSA DISABILITY PROGRAMS Progress Made, but Key Efforts Warrant More Management Focus Highlights of GAO-12-420, a report to the Chairman, Subcommittee on Social Security, Committee on Ways and Means, House of Representatives Why GAO Did This Study What GAO Found SSA administers two of the largest The Social Security Administration (SSA) has taken steps that hold promise for federal disability programs. GAO improving the process for updating its medical criteria, but continues to face designated federal disability programs challenges ensuring timely updates. SSA now uses a two-tiered system for as a high-risk area, in part because ongoing revisions to its medical listings. First, it completes a comprehensive eligibility criteria had not been updated review of all medical conditions listed within each of 14 body systems, making to reflect medical and technological needed revisions. For subsequent updates for a body system, the agency uses a advances and labor market changes. targeted approach, selecting for review and revision only those medical Given the size and cost of its disability conditions most in need of change. To date, SSA has completed comprehensive programs, SSA needs updated criteria revisions for 8 of the 14 body systems and now is reviewing conditions under to appropriately determine who them to determine where targeted revisions are appropriate. However, some of qualifies for benefits. GAO has been asked to assess SSA’s efforts to these targeted revisions have experienced delays. Moreover, SSA has yet to update its medical criteria and develop complete comprehensive revisions for six body systems that have been ongoing a new occupational information for 19 to 33 years. SSA officials attributed delays to a lack of staff and expertise, system, and to identify other steps along with the complexity and unpredictability of the regulatory process. taken to modernize disability SSA has embarked on an ambitious plan to design by 2016 an occupational determination criteria. To do this, GAO information system for use in its disability decision-making process, but has fallen reviewed relevant publications and short of best practices for estimating costs, maintaining a schedule, and federal laws and regulations; assessed considering risks and alternatives. SSA currently relies on occupational agency plans, cost estimates, information developed by the Department of Labor which has not had a major schedules, and other documentation against established project update since 1977. In 2008, SSA initiated a project to develop its own management criteria; and interviewed occupational information system (OIS), which SSA expects will provide up-to- SSA officials, experts, and date information on the physical and mental demands of work to support its stakeholders. decision-making process. To guide the creation of its OIS, SSA established an advisory panel, collaborated with outside experts and other agencies, and in July What GAO Recommends 2011 issued a research and development plan detailing relevant activities through 2016. SSA has made progress on some baseline activities in the plan. GAO recommends that SSA (1) explicitly identify resources needed to However, SSA’s cost estimate and schedule had key deficiencies, such as not achieve its 5-year time frame for including any estimate of the cost of producing, maintaining, and operating the updating its medical listings; (2) follow system, which can inform design options. SSA also did not adequately consider best practices in its cost estimate, inherent risks or potential alternatives, which could heighten the risk of additional schedule, and risk assessment for the costs or project failure. occupational information system; and Consistent with modern views of disability, SSA has taken some concrete steps (3) conduct limited, focused studies on toward greater consideration of an individual’s ability to function with a disability how to more fully consider assistive but faces constraints in fully modernizing. SSA has incorporated some criteria devices and workplace accommodations in its disability into its medical listings to determine whether a claimant’s impairments result in determinations. SSA agreed with the functional limitations that can prohibit the ability to work. SSA is also sponsoring first two recommendations and research through the National Institutes of Health to evaluate how functional disagreed with the third, stating that abilities can further be considered in determining disability. One project aims to such studies would be inconsistent develop a computerized tool to assist adjudicators in evaluating how various with Congress’ intentions. GAO impairments affect an individual’s function and ability to work. However, SSA continues to believe the officials maintain that other modern concepts of disability cannot be fully recommendation has merit, as incorporated into SSA’s disability decisions. Specifically, SSA faces constraints discussed more fully within the report. considering the extent to which assistive devices and workplace accommodations can mitigate work disability, because these are not universally View GAO-12-420. For more information, contact Daniel Bertoni at (202) 512-7215 or available and SSA lacks the resources to conduct individualized assessments. email@example.com. United States Government Accountability Office Contents Letter 1 Background 2 SSA Has Improved Its Process for Updating Medical Listings, but Still Faces Challenges with Timely Updates 7 SSA Has Begun an Ambitious Project to Develop Updated Occupational Information, but Many Uncertainties Remain 14 SSA Has Taken Other Steps to Modernize Criteria but Faces Limitations 25 Conclusions 30 Recommendations for Executive Action 31 Agency Comments and Our Evaluation 31 Appendix I Objectives, Scope, and Methodology 35 Appendix II Progress Updating Six Body Systems since Last Comprehensive Revision 38 Appendix III Description of Cost Estimating Best Practices 39 Appendix IV Description of Scheduling Best Practices 40 Appendix V Comments from the Social Security Administration 41 Appendix VI GAO Contact and Staff Acknowledgments 44 Tables Table 1: Key Dates for Comprehensive Revisions to Body System Listings 11 Table 2: Status of Select OIS Activities as of May 2012 17 Table 3: Assessment of OIS Cost Estimate 20 Table 4: Assessment of OIS Schedule 21 Page i GAO-12-420 Modernizing SSA Disability Criteria Table 5: Six Sound Project Management Practices Used to Evaluate SSA’s Efforts 36 Figures Figure 1: SSA’s Five-Step Sequential Evaluation Process for Determining Disability 4 Figure 2: Status of Comprehensive and Targeted Revisions for SSA’s Body System Listings 9 Abbreviations ADA Americans with Disabilities Act of 1990 DI Social Security Disability Insurance DOT Dictionary of Occupational Titles Labor Department of Labor ICF International Classification of Functioning, Disability and Health NIH National Institutes of Health O*NET Occupational Information Network OIS Occupational Information System SSA Social Security Administration SSI Supplemental Security Income This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-12-420 Modernizing SSA Disability Criteria United States Government Accountability Office Washington, DC 20548 June 19, 2012 The Honorable Sam Johnson Chairman Subcommittee on Social Security Committee on Ways and Means House of Representatives Dear Mr. Chairman: The Social Security Administration (SSA) manages two of the largest federal disability benefit programs—Social Security Disability Insurance and Supplemental Security Income—which together resulted in payments of more than $165 billion to about 13.5 million people with disabilities and their families in fiscal year 2010. 1 Given the extensive size and cost of its disability programs, SSA must have current and appropriate criteria by which to assess whether an applicant’s medical conditions affect his or her ability to perform work in the national economy. However, we designated federal disability programs as high risk, in part because the medical criteria and occupational information that SSA relies on to make benefit decisions were found to be out of date. Moreover, we and others have found SSA’s disability programs—which historically have tended to equate severity of a medical condition with an inability to work—to be out of sync with a more modern concept of disability that considers the presence or lack of assistance that an individual with medical impairments might need to function more effectively in his or her environment. You recently asked us to assess SSA’s plans and efforts for revising its disability criteria. Accordingly, we examined the status and management of SSA’s efforts to update its medical listings to reflect current medical knowledge and develop a new occupational information system to reflect labor market changes, and we identified other steps taken by SSA to incorporate a modern view of disability into its eligibility criteria. 1 These data are from the fiscal year 2012 President’s Budget. Page 1 GAO-12-420 Modernizing SSA Disability Criteria To perform this work, we reviewed our prior reports and SSA Office of Inspector General reports; relevant federal laws and regulations; program documentation, including policies, procedures, strategic goals, and supporting project plans and cost estimates; relevant literature; and position papers and testimonies from disability groups and commissions. We interviewed SSA officials, key project contractors and stakeholders, disability experts, and representatives of other agencies that administer disability programs. We also evaluated SSA’s plans to update its criteria against sound project management and cost estimating practices. We narrowed our scope to criteria used for initial adult disability determinations. See appendix I for more on our scope and methodology. We conducted this performance audit from April 2011 through June 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. SSA administers two disability programs: the Social Security Disability Background Insurance (DI) program, enacted in 1956, and the Supplemental Security Income (SSI) program, enacted in 1972. 2 In order to be eligible for DI or SSI benefits based on a disability, an individual must meet the definition of disability for these programs—that is, they must have a medically determinable physical or mental impairment that (1) prevents the individual from engaging in any substantial gainful activity and (2) has lasted or is expected to last at least 1 year or result in death. 3 To determine eligibility for both programs, SSA uses a five-step sequential process that is intended, in part, to expedite disability 2 The DI program provides monthly cash benefits to eligible applicants who have a sufficient work record. The SSI program provides monthly benefits to eligible applicants with limited income and resources who are disabled, blind, or age 65 or older. 3 42 U.S.C. §§ 423(d)(1)(A) and 1382c(a)(3)(A). Substantial gainful activity is generally work activity involving significant physical or mental activities that are done for pay or profit, whether or not a profit is realized. 20 C.F.R. §§ 404.1572 and 416.972. In 2012, the substantial gainful activity threshold was $1,690 per month for blind recipients and $1,010 per month for individuals with other disabilities. Page 2 GAO-12-420 Modernizing SSA Disability Criteria decisions when possible and limit administrative costs by conducting less intensive assessments at earlier steps (see fig. 1). 4 • At steps 1 and 2 of the process, SSA determines whether an applicant is working and meets income thresholds, as well as the medical severity of impairments. If not working (or not meeting income thresholds) and with the determination of a severe impairment, the applicant moves to step 3 of the process. • At step 3, SSA examiners assess the applicant’s medical impairments against the Listings of Impairments, also known as the medical listings, which are organized into 14 major body systems for adults and reflect medical conditions that have been determined by the agency to be severe enough to qualify an applicant for benefits. 5 When using the listings to determine eligibility, SSA generally relies on information on the applicant’s diagnoses, including laboratory findings, diagnostic tests, and symptoms, as well as some limited consideration of the applicant’s functional limitations. If the individual’s impairment meets or is equal in severity to one or more of those in the listings, the individual is determined to have a disability at step 3. 6 • If a disability determination is not made at step 3, SSA performs an assessment of the individual’s physical and mental residual functional capacity. 7 Based on this assessment, SSA determines whether the 4 See 20 C.F.R. §§ 404.1520 and 416.920. 5 According to SSA officials, there are almost 300 medical impairments organized in 119 listings under the 14 major body systems for adults. 6 SSA has also implemented Compassionate Allowances to quickly identify diseases and other medical conditions that invariably qualify under the medical listings based on minimal, objective medical information. According to SSA officials, many of the Compassionate Allowances conditions are rare, and thus unfamiliar to examiners, such as certain cancers and early-onset Alzheimer’s disease. In April 2012, SSA announced that 165 conditions would be on the list effective in August 2012, up from 113 conditions as of the time this report issued in June 2012. The Compassionate Allowances initiative is one of two parts of the agency’s fast-track system for certain disability claims. When combined with the Quick Disability Determination process, which uses a predictive model to identify cases with a high likelihood of being allowed, Social Security reported approving more than 100,000 fast-track cases in 2009, usually in less than 2 weeks. 7 SSA’s physical and mental residual functional capacity assessments establish the extent to which an individual’s medically determinable impairments, including any related symptoms, such as pain, may cause physical or mental limitations or restrictions that may affect his or her capacity to do work-related physical and mental activities. Page 3 GAO-12-420 Modernizing SSA Disability Criteria individual is able to perform past relevant work (step 4) or any work that is performed in the national economy (step 5). To inform determinations at steps 4 and 5, SSA uses a Department of Labor database—known as the Dictionary of Occupational Titles (DOT), which is an inventory of occupations performed in the national economy. At step 5, SSA also uses a set of rules and guidelines, referred to as the grid rules, to evaluate the combined effect of an individual’s physical residual functional capacity, age, education, and work experience. 8 Figure 1: SSA’s Five-Step Sequential Evaluation Process for Determining Disability 8 SSA relies on the grid rules to represent the total number of unskilled jobs in the nation at three physical strength levels as classified in the DOT, and to provide for consistent “rulemaking” or application of case facts to help ensure uniform decisions. Page 4 GAO-12-420 Modernizing SSA Disability Criteria While originally created for expediency, 9 over time the medical listings used at step 3 were relied on less to make program determinations as they became increasingly outdated. In the early years of the program, more than 90 percent of cases were decided based on medical conditions specified in the listings; in 2010 only 47 percent of allowances were made at step 3. Experts attribute the decline in allowances based on the medical listings to changes in the program, workplace, and medical treatment that the medical listings had not kept pace with. Since the 1990s, we, along with SSA’s Office of Inspector General and the Social Security Advisory Board, 10 have expressed concerns that the medical listings being used no longer provide current and relevant criteria to evaluate disability applicants’ inability to work. In 2003, we deemed SSA’s and other federal disability programs as high-risk areas, in part, because their programs continue to emphasize medical conditions in assessing work capacity, without adequate consideration of work opportunities afforded by advances in medicine, technology, and changes in the labor market. In 2008, we reported that SSA had recently established a new process— referred to by SSA as the “business process”—for revising the listings to better incorporate feedback into its continuous updates. 11 This process, which has been in effect since 2003, incorporates feedback from multiple parties, including medical experts and claims examiners, to update the medical criteria. Under this process, SSA gathers external feedback from comments associated with regulatory actions, such as the publication of advanced notices of proposed rulemaking (advanced notices) and notices of proposed rulemaking (notices) in the Federal Register. 12 In addition, 9 Historically, SSA created the listings to have a clear set of medical conditions that, if present, would preclude work. This in turn would limit the number of cases requiring a more involved and individualized assessment of function. 10 In 1994, when the Congress passed legislation establishing SSA as an independent agency, it also created the seven-member bipartisan Social Security Advisory Board in part to advise the President, Congress, and the Commissioner of Social Security on Social Security and SSI policy. Pub. L. No. 103-296, § 103, 108 Stat. 1464, 1467. 11 GAO, Federal Disability Programs: More Strategic Coordination Could Help Overcome Challenges to Needed Transformation, GAO-08-635 (Washington, D.C.: May 20, 2008). 12 Any changes to the medical listings have to proceed according to an established process for rulemaking outlined in federal law. As such, when changes are made to the listings, a notice of proposed rulemaking must generally be published in the Federal Register with a public comment period before the final rule is issued. 5 U.S.C. § 553. Also, the Office of Management and Budget must review and approve any notice that is economically significant (i.e., has costs or savings greater than $100 million). Page 5 GAO-12-420 Modernizing SSA Disability Criteria this process includes conducting postimplementation reviews one year after a revision is made to assess the impacts of a revised listing, areas to improve, and whether expectations from the revisions have been achieved. 13 With respect to information on jobs in the national economy that supports SSA’s occupational criteria, we and others have reported that the DOT is outdated, although SSA still relies on it to assess eligibility at steps 4 and 5 of the process. The DOT has not had a major update since 1977 14 and the Department of Labor (Labor) replaced it with a new database in 1998 called the Occupational Information Network (O*NET). 15 However, SSA determined that O*NET is not sufficiently detailed for evaluating DI and SSI disability claims, so SSA has begun developing its own occupational information system to better reflect the physical and mental demands of work in the national economy. Beyond dated medical criteria and vocational information, numerous disability experts have expressed concern that SSA’s disability programs—which statutorily require that an adult’s medical condition prevents that person from engaging in substantial gainful activity—have historically tended to equate the severity of medical conditions with inability to work and thus are out of sync with modern concepts of disability. Modern concepts focus on an individual’s functional abilities in the workplace environment, including consideration of the presence or lack of assistance, for example, per the requirements for reasonable accommodation by the Americans with Disabilities Act of 1990. 16 These modern views are reflected in the International Classification of Functioning, Disability and Health (ICF), which is the World Health Organization’s framework for assessing health and disability. This framework takes into account the interaction of health conditions and 13 Such reviews typically rely on reviews of decision data as well as surveys of field staff responsible for applying the revised listings. 14 The DOT was updated in 1991, but for less than 20 percent of the occupations. 15 O*NET is a general purpose occupational information database used for workforce development, economic development, career exploration, and academic and policy research. Selected occupations within the O*NET system are updated each year. According to Labor officials, by July 2012, 900 occupations will have been comprehensively updated, and 413 will have more than one update. 16 See, for example, 42 U.S.C. § 12112(b)(5). Page 6 GAO-12-420 Modernizing SSA Disability Criteria contextual factors, such as products and technology, attitudes, and services, on an individual’s functional capacity, rather than viewing disability solely as a medical or biological issue. Along these lines, experts have recommended that SSA incorporate more consideration of individual function in its medical listings when doing so can improve their use as a screening tool for determining inability to work. Further, several groups, such as the Social Security Advisory Board 17 and the Urban Institute, 18 have reported that SSA’s disability programs should focus more on whether an individual can work given appropriate environmental or other supports, and that SSA—through its demonstration authority or other means—could play a role in determining how. These suggestions are consistent with SSA’s long-term targeted outcomes, which include not only regularly updating regulations and policies to incorporate the most recent medical advances, but also making it easier for individuals with disabilities to return to work. SSA Has Improved Its Process for Updating Medical Listings, but Still Faces Challenges with Timely Updates SSA Has Improved Its Since our last review in 2008, SSA has made several changes that hold Strategy to Update Medical promise for improving its medical listings updates. First, the agency is Listings using a two-tiered system for ongoing revisions to the listings. Under this system, SSA first completes a comprehensive listings update for a body system, which entails reviewing all the diseases and disorders listed within that system and making necessary revisions. Second, following a comprehensive revision, SSA will pursue a more targeted approach—that is, SSA will conduct ongoing reviews and updates of a smaller number of medical diseases or disorders within that body system. Agency officials 17 See Social Security Advisory Board, The Social Security Definition of Disability (Washington, D.C.: October 2003). 18 David Wittenburg and Pamela Loprest, A More Work Focused Disability Program?: Challenges and Options (Washington, D.C.: Nov. 1, 2003). Page 7 GAO-12-420 Modernizing SSA Disability Criteria told us that targeted updates should be completed more quickly than comprehensive updates, allowing them to focus on the most critical changes needed. 19 As of early March 2012, SSA had begun the ongoing review process to identify opportunities for targeted revisions for 8 of the 14 adult body systems that were recently comprehensively revised. Another change, according to agency officials, is that in 2010, the SSA Commissioner set a 5-year cycle for updating listings for each body system. 20 Previously, SSA established cycles for periodically updating listings under each body system, ranging from 3 to 8 years, but frequently extended them. SSA officials believe that conducting targeted reviews will generally allow the agency to conclude any necessary revisions prior to the 5-year period. Additionally, they expect that using the “business process,” which requires early public notification of changes and obtaining necessary data and feedback from internal and external parties, should help keep continuous reviews on track. See figure 2 for the status and expiration date by which the listings should be reviewed and updated, if needed, for the 14 body systems undergoing review for either comprehensive or possible targeted revisions, as of early March 2012. 19 Officials noted that these ongoing reviews could also result in major or even no changes, as appropriate. 20 The 5-year period will be applied to listings under a body system upon completion of their current revision. SSA began applying the 5-year period in 2011 when it comprehensively updated the endocrine body system listings. For other body system listings updated prior to 2011, SSA generally assigned periods extending beyond 5 years. Page 8 GAO-12-420 Modernizing SSA Disability Criteria Figure 2: Status of Comprehensive and Targeted Revisions for SSA’s Body System Listings a For the two-tiered revision process, the beginning point is the date of the last comprehensive or targeted revision. b SSA officials told us they have grouped the disorders in the special senses and speech listings into two broad sections—vision and hearing—and two standalone medical conditions. SSA has not yet comprehensively revised the two standalone medical conditions that cover speech and disturbances of labyrinthine-vestibular function. The two sections have undergone comprehensive revisions and are undergoing reviews that began in 2007 for the vision section and in 2010 for the hearing section. Special senses and speech listings are set to expire in 2015. c According to SSA officials, SSA published limited revisions for the hematological disorders in 1988. It also published final rules for limited revisions for the mental disorders listings in 2000 and the musculoskeletal system listings in 2001. SSA has made another change by more extensively engaging the medical community to identify ways to improve the medical listings. For example, SSA contracted with the Institute of Medicine to study its medical criteria for determining disability and to make recommendations for improving the timeliness and accuracy of its disability decisions, Page 9 GAO-12-420 Modernizing SSA Disability Criteria resulting in a 2007 report with recommendations 21 and a symposium of experts in 2010. SSA has addressed some of the Institute of Medicine’s recommendations, such as making better use of its administrative data to update criteria and creating a standing committee through the Institute of Medicine to provide recommendations for listings revisions. In addition, through the Institute of Medicine, SSA created consensus committees to conduct research and provide other assistance with updating SSA’s cardiovascular listings and its Human Immunodeficiency Virus (HIV) listing, which culminated in two reports with recommendations. 22 SSA Has Experienced SSA has made progress, but continues to face delays in completing both Delays with Its Revision comprehensive and ongoing targeted updates. As of early March 2012, Process SSA officials told us they had completed comprehensive revisions of listings for eight body systems, some of which resulted in significant changes. For example, in 2011, SSA removed the endocrine body system listings for all the adult disorders, such as diabetes, because they found that they were now generally diagnosed early and treated very successfully. 23 Nevertheless, according to SSA, the agency still needs to complete the comprehensive revisions for listings of the remaining six body systems—a process that has been ongoing for the last 19 to 33 years, with numerous extensions beyond the original expiration periods (see table 1). For example, it has been at least 27 years since SSA finalized comprehensive revisions for two of the six body system listings—mental and neurological disorders, which are among those SSA uses most frequently in its eligibility determination process. 24 SSA has made progress on four of the six body systems set to expire in 2012, but will likely miss targeted time frames. SSA issued a notice of proposed rulemaking to revise its listings of mental disorders in 2010 and has told 21 Institute of Medicine, Improving the Social Security Disability Decision Process (Washington, D.C.: 2007). 22 For the reports, see Institute of Medicine, Cardiovascular Disability: Updating the Social Security Listings (Washington, D.C.: 2010), and HIV and Disability: Updating the Social Security Listings (Washington, D.C.: 2010). 23 SSA officials noted that while the most common endocrine disorder, diabetes, can be disabling, this is due to its effects on other organs such as the heart or kidneys; as such, they can be assessed under listings for these body systems or the effects on functionality can be assessed at steps 4 or 5. 24 According to SSA officials, SSA made limited, but not comprehensive, revisions to the mental disorders listings in 2000. Page 10 GAO-12-420 Modernizing SSA Disability Criteria us that it plans to finalize this comprehensive revision by the end of 2012, after its current expiration date in July 2012. 25 SSA is still developing notices for the neurological and two other body system listings, which are also set to expire in 2012. Because SSA has generally taken more than a year to move from publishing the notice to final revision and may also need additional time for any internal revision and review, as well as a review by the Office of Management and Budget, it is also unlikely that SSA will meet its planned 2012 time frames for updating these listings. See appendix II for details on progress updating these listings. Table 1: Key Dates for Comprehensive Revisions to Body System Listings Year of last Years initially set Years since last comprehensive to review and comprehensive Current Body system and section revision revise revisiona expiration dateb Special senses and speechc 1979 8 33 2015 sections: • Speech • Disturbances of labyrinthine- vestibular function Neurological disorders 1985 8 27 2012 Mental disorders 1985 3 27 2012 Hematological disorders 1985 8 27 2012 Musculoskeletal system 1985 5 27 2013 Respiratory system 1993 6 19 2012 Source: GAO analysis based on information provided by SSA officials. a According to SSA officials, SSA published limited revisions for the hematological disorders in 1988. It also published final rules for limited revisions for the mental disorders listings in 2000 and the musculoskeletal system listings in 2001. b SSA has continually extended the expiration dates of the listings. c The special senses and speech listings have two sections—vision and hearing—which have been comprehensively revised. Updates using SSA’s targeted approach seem to be moving at a faster pace than the comprehensive revisions, but some delays have occurred here as well, and more are expected. For example, SSA has already extended the expiration date for its targeted review of the cardiovascular system. According to SSA officials, it may also need to extend expiration 25 See 75 Fed. Reg. 51,336 (Aug. 19, 2010). Page 11 GAO-12-420 Modernizing SSA Disability Criteria dates for listings under two other body systems set to expire in 2012. Extensions may be needed because SSA has not yet published the notice of proposed rulemaking for those two body system listings and may not have time to publish these notifications, respond to comments, and complete the final updates by their current expiration date. At the same time that SSA has been experiencing delays completing timely revisions, agency officials reported challenges with other steps in the business process. Whereas SSA’s business process includes a postimplementation review of case data at the 1-year mark to determine whether expectations from a revision were met, several did not undergo this review, such as those for skin disorders, genitourinary impairments, and impairments affecting multiple body systems. SSA officials told us they only began conducting these reviews in 2010, and to date, SSA has completed just one in 2011 that involved a targeted sample of 175 cases. SSA officials told us they are conducting or planning to conduct two more reviews at the 1-year mark in 2012. While disability experts we interviewed spoke highly of SSA’s business and targeted review processes to obtain feedback early on and update the listings more promptly, ongoing delays raise questions about the agency’s ability to fully follow its current business process while completing continuous and timely revisions for all 14 body systems. 26 SSA officials offered two key reasons for the delays in updating the listings: (1) limitations in the number and expertise of staff and (2) the complexity and unpredictability of the regulatory process. According to SSA officials, revising the medical listings requires research, deliberation, testing, regulatory review, and consensus with many stakeholders, and consequently is difficult and time-consuming to achieve. According to an SSA official in the Office of Medical Listings Improvement, the office is short-staffed and there is a lack of expertise needed to perform this work. To address these constraints, SSA has contracted with the Institute of Medicine to review and develop recommendations for revising two of the body system listings. While not finalized as of March 2012, SSA officials reported that the agency plans to renew this contract after it expires in 2012 and extend it to 2015. However, SSA has not yet determined how it 26 According to agency officials, in addition to updating adult listings the agency must also update listings for children, which adds to its workload and resource needs. Page 12 GAO-12-420 Modernizing SSA Disability Criteria will use the Institute of Medicine to revise the listings or the extent to which the contract will address staffing shortfalls. Also contributing to delays is the time required for internal review and public comment under the regulatory process, which depends largely on the number and the substance of comments received, according to an SSA official. Obtaining public comment is one way SSA receives critical information for identifying areas for revision and obtaining stakeholder consensus. As such, SSA reviews and responds to each comment that would result in a significant change, and the time required for doing so varies depending on the number of comments and resources. For example, SSA officials told us that the advanced notice of proposed rulemaking to the listings of mental disorders in 2003 resulted in 500 comments, which took SSA 5 years to incorporate into a draft notice of proposed changes for regulatory review. SSA officials reported they have considered options for automating and thereby speeding up the process of obtaining and reviewing public comment. For example, they told us that they eventually plan to use a web-based tool to obtain early public feedback on medical listing updates to help target their limited resources and more quickly make changes. According to an SSA official, the agency plans to retain its optional use of the advanced notice of proposed rulemaking because it can also help to identify appropriate areas to focus on to ultimately make timely updates. 27 27 Under the Administrative Procedure Act, a notice of proposed rulemaking is generally required for regulatory revision, but an advanced notice is not required. See 5 U.S.C. § 553. Page 13 GAO-12-420 Modernizing SSA Disability Criteria SSA Has Begun an Ambitious Project to Develop Updated Occupational Information, but Many Uncertainties Remain SSA is Designing a New In 2008, SSA began a multiyear project to research and design a new Occupational Information source of occupational information that will replace the outdated System, but Has Fallen information currently being used to determine if claimants are able to do their past work or any other work in the national economy. Since the behind with Some 1960s, SSA has been using the DOT, which contains a list of job titles Activities found in the national economy and had a last major update in 1977. 28 The DOT provides SSA with descriptions of the physical demands of work— such as climbing, balancing, and environmental requirements—for each of the more than 12,000 occupations listed. 29 According to SSA, these descriptions have been essential to its evaluations of how much a claimant can do despite his or her impairment and whether this level of functioning enables the claimant to do his or her past work or any other work. After its last limited update, Labor decided to replace the DOT with O*NET, which has far fewer occupational titles compared with the DOT and has served Labor’s purposes more efficiently. According to an SSA report, after investigating potential alternatives, SSA decided that O*NET and other existing databases with occupational information were not sufficiently detailed and not able to withstand legal challenges for use in its decision-making process. SSA further decided to develop its own occupational information system (OIS), which would contain detailed information as in the DOT, but would also include additional information, 28 The DOT provides a wide range of occupational information that could be used for job placement, occupational research, career guidance, labor-market information, curriculum development, and long-range job planning. 29 In 1966, SSA contracted with Labor to produce a companion volume to the DOT entitled the Selected Characteristics of Occupations that provides measures for additional physical demands of work for DOT occupations, such as climbing, balancing, reaching, handling, special senses requirements (visual acuity, hearing, etc.), and environmental requirements (noise levels, exposure to cold, etc.). Page 14 GAO-12-420 Modernizing SSA Disability Criteria such as the mental demands of work. In addition, SSA has determined that the OIS will (1) meet its legal, program, and data requirements; (2) be flexible enough to incorporate changes in its policies and processes; and (3) be able to be updated to reflect the evolving workplace environment. In 2008, SSA began taking steps to guide the development of its OIS. SSA created an internal office and working group, as well as an Occupational Information Development Advisory Panel. While the number of panel members has fluctuated over time, in April 2012, 14 external experts were serving on the panel and represented various affiliations, such as medicine, disability law, rehabilitation, and industrial organizational issues. 30 The advisory panel holds quarterly public meetings and has several subcommittees that review material and make recommendations to SSA on developing various components of the OIS. For example, in a 2009 report, the advisory panel supported the need for SSA to develop a new source of occupational information, rather than adapt O*NET, and recommended the type of data SSA should collect, as well as approaches for classifying occupations. To further inform its efforts, SSA has sought input from agencies or organizations that either collect occupational information or also use the DOT. For example, SSA officials held initial meetings with Labor and U.S. Census Bureau officials to gain information on sampling methods used for O*NET, the Occupational Employment Statistics program, and U.S. Census Bureau’s household surveys. 31 Additionally, since February 2011, SSA and Labor have been in the process of approving a memorandum of understanding to formalize their collaboration efforts on the new OIS. According to an SSA official, as the OIS project progresses, SSA plans to convene ad hoc roundtables with experts and other agency officials to 30 The Occupational Information Development Advisory Panel was formed in December 2008 under the Federal Advisory Committee Act, and broadly includes representatives of organizations that have used the DOT or other occupational information systems, and subject matter experts. The panel was originally comprised of 11 members, including an SSA official. Over time, some members resigned and additional members were added. 31 The Occupational Employment Statistics program produces employment and wage estimates for approximately 800 occupations. The U.S. Census Bureau’s household surveys include (1) the American Community Survey, which is an ongoing survey that provides annual data on demographics such as age, education, and disabilities, and (2) the Current Population Survey, which is primarily a labor force survey, conducted every month by the U.S. Census Bureau for the Bureau of Labor Statistics and provides data such as the national employment rate. Page 15 GAO-12-420 Modernizing SSA Disability Criteria explore specific subject areas, such as sampling issues. Besides working with Labor and U.S. Census Bureau officials, SSA officials and panel members have sought input from other experts and current users of the DOT, such as SSA disability adjudicators and external rehabilitation professionals. SSA officials conducted a user needs analysis in 2009 and have presented on the OIS project at events and conferences. In July 2011, SSA published a detailed research and development plan outlining all activities related to researching, developing, and testing the key components of the OIS in order to implement it by 2016 at an estimated total cost of $108 million. For example, the plan includes several baseline activities to identify and study other occupational information systems and various approaches for analyzing occupations that may inform or could be leveraged in SSA’s OIS data collection. The plan also includes activities to identify the primary occupational, functional, and vocational characteristics of current beneficiaries. Other key components of the plan include developing descriptions of work requirements, such as the physical and mental demands for jobs; developing data collection and analysis strategies; and identifying the occupations, categories, and definitions that will constitute the structure of the new OIS. SSA also plans to develop a strategy for piloting how it would ultimately collect data for the OIS nationwide within this time frame. As of May 2012, SSA had made progress on many of the baseline activities outlined in its research and development plan for the OIS. 32 For example, according to an SSA official, its investigation of existing occupational information systems, now complete, resulted in useful information about design issues other organizations have confronted and mitigated when creating their own system. Additionally, SSA’s preliminary analysis of its own administrative data identified the most frequently cited occupations and functional and vocational characteristics of disability applicants. SSA officials told us the agency will target the occupations identified in this analysis for its pilot studies of the OIS. Also in 2011, SSA completed a comprehensive framework for assessing an individual’s capacity to work—key to informing the OIS content, according to SSA officials—which was based on recommendations of outside experts as well as SSA’s policy and program requirements. 32 Some of the activities that were part of the research and development plan were completed in years prior to the plan’s issuance. Page 16 GAO-12-420 Modernizing SSA Disability Criteria While SSA has made progress on several key activities, agency officials delayed 2011 completion dates for certain activities and anticipate making additional changes to its timeline as a result of not meeting its staffing goals for fiscal year 2011 (see table 2). Some activities that were delayed by several months included finalizing reports for the baseline studies and conducting a literature review that would inform how occupations might be analyzed. SSA officials told us that they did not have enough staffing to complete all of the 2012 planned activities within the estimated schedule. Further, SSA officials said they did not have the budget to hire new staff in September 2011. To address this challenge, SSA officials hired consultants to meet some of their needs. SSA officials also met with the Office of Personnel Management to explore the possibility of an interagency agreement that would allow SSA to use one or two of the office’s industrial organizational psychologists to help on a part-time basis. Table 2: Status of Select OIS Activities as of May 2012 Activity and objective Fiscal year target Status Investigate existing OISs. To identify lessons learned about features, requirements, 2010-2011 Completed in fiscal year processes, and options through the examination of the development and operation 2012 (late) of existing domestic and international OISs. Occupational and medical-vocational study. To identify and record the primary 2010-2011 In progress occupational, functional, and vocational characteristics of adult disability applicants (late) at steps 4 or 5 at the initial and hearings levels. Job analysis methodologies (performed by contractor). To identify existing job 2011 Completed on time analysis methods, their features, and their potential usefulness for OIS development. Business processes for recruiting, training, and certifying job analysts (performed 2011 Completed on time by contractor). To identify existing business processes used to recruit, train, and certify job analysts and potential usefulness of these processes for OIS development. Identify OIS usability standards. To identify standards and criteria for assessing the 2009-2012 Completed 3 of 4 operational usability of the new OIS and to incorporate those standards into OIS activities (one completed research and development activities. late) Identify OIS scientific standards. To identify relevant scientific standards, 2011 Completed guidelines, and best practices that enable SSA to meet its responsibilities under applicable federal requirements, and to incorporate those standards into OIS research and development activities. Identify OIS legal requirements. To identify standards for evaluating and ensuring 2011 Completed that the new OIS accurately reflects the language and the intent of the vocational requirements of the Social Security Act. Identify OIS elements. To establish basic design parameters and requirements to 2012 In progress ensure that OIS meets SSA’s legal, scientific, and usability requirements for occupational information. Page 17 GAO-12-420 Modernizing SSA Disability Criteria Activity and objective Fiscal year target Status OIS work taxonomy. To identify a comprehensive set of constructs that may form 2011-2012 In progress the basis of OIS. These constructs must reflect the physical and mental-cognitive requirements of work and context of work that can be objectively measured or delineated to develop an occupational classification. Source: GAO based on information provided by SSA officials. SSA’s Cost Estimate and Our Cost Estimating and Assessment Guide 33 identifies a number of best Schedule Fell Short of Best practices for effective cost estimating and scheduling that should result in Practices in Key Areas reliable and valid cost estimates that management can use for making informed decisions. Per these criteria, the success of any program depends in part on having reliable cost estimates and a reliable schedule. A reliable cost estimate provides the basis for informed investment decision making, realistic budget formulation and program resourcing, meaningful progress measurement, proactive course correction when warranted, and accountability for results. A reliable schedule defines, among other things, when work activities will occur, how long they will take, and how they are related to one another. As such, the schedule not only provides a road map for systematic execution of a program, but also provides a means by which to gauge progress, identify and address potential problems, and promote accountability. We compared SSA’s cost estimate to three best practices for assuring a reliable cost estimate 34 and found SSA only minimally or partially met each of these (see table 3). First, we found SSA’s estimated cost of the OIS research and development phase—$108 million—was not well- documented. For example, after reviewing all of SSA’s documentation supporting this cost estimate as of December 2011, we did not find step by step documentation showing how the estimate was derived so that someone unfamiliar with the project could use the documentation to recreate the estimate and get the same results. Well-documented cost estimates are considered a best practice and without good documentation, SSA is not in a position to defend the reliability of its estimate. Additionally, SSA only partially met the best practice of having 33 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 34 We did not assess SSA’s estimate on a fourth criterion—credibility—which evaluates any limitations of the analysis because of uncertainty or biases surrounding data or assumptions, as this criterion is most applicable to major capital acquisitions. Page 18 GAO-12-420 Modernizing SSA Disability Criteria an accurate cost estimate. While SSA provided documentation showing how some of their estimates compared to actual costs and the reasons for any variances, SSA did not provide adequate documentation for us to determine, for example, if the estimate took inflation into account or if there were any calculation errors. Generally speaking, in the absence of a detailed cost model, third party reviewers cannot be certain that cost estimate calculations are accurate and account for all costs. We also found the cost estimate was not comprehensive as it does not include any costs beyond the research and development phase, such as the costs of producing, maintaining, and updating the final data system, which could be significant. SSA officials told us that they are still in the process of determining what information will be included in the data system, how it will be collected, and how many occupations will be covered, each of which will influence the cost of developing and maintaining the OIS. As such, they maintain that it is too early to estimate the costs of the final system or the costs for maintaining the system. According to industry best practices, cost estimates should be comprehensive and include all costs necessary to achieve agency objectives and should be updated as the agency proceeds with the project and gains more information. At this point in the project, estimating total costs of options under consideration could enhance decision making by evaluating the potential tradeoffs of different designs. Without any estimate for the cost of producing the OIS, SSA risks designing a system that would not be a viable or affordable option to complete. Additionally, without maintenance cost estimates, SSA is at risk of designing a system that would be too costly to maintain on a regular basis resulting in outdated information. Other federal information systems provide some basis for estimating the cost of producing or maintaining an OIS. For example, Labor officials indicated that the cost of maintaining O*NET—an occupational information system that includes approximately 1,000 occupations and uses a paper or web survey to collect data—is roughly $6 million a year. 35 35 O*NET’s annual maintenance cost may constitute a lower bound estimate for the OIS, since SSA has considered more intensive data collections than a paper- or web-based survey questionnaire and a larger number of occupations. Another occupational information system maintained by the U.S. Bureau of Labor Statistics—the Occupational Employment Statistics program—costs around $30 million annually to maintain. This program also covers about 800 occupations predominantly using a mail survey, but may involve additional surveys, telephone interviews, or site visits to reach nonrespondents. Page 19 GAO-12-420 Modernizing SSA Disability Criteria Table 3: Assessment of OIS Cost Estimate Best practice Extent best practice met Well-documented Minimally met Accurate Partially met Comprehensive Partially met Source: GAO analysis of SSA’s cost estimate for the OIS. Note: Not met—SSA provided no evidence that satisfies any of the criteria; minimally met—SSA provided evidence that satisfies a small portion of the criteria; partially met—SSA provided evidence that satisfies about half of the criteria; substantially met—SSA provided evidence that satisfies a large portion of the criteria; and met—SSA provided complete evidence that satisfies the entire criteria. See appendix III for more information on these best practices. We also reviewed SSA’s master schedule for the OIS against nine best practices for a reliable schedule, and found that SSA did not meet four and minimally met five of these practices (see table 4). For example, among those minimally met, we found a significant number of activities within SSA’s schedule were not logically sequenced in the order that they were going to be carried out (best practice 2) and activities that were dependent on completion of a prior activity were not identified (best practice 5, 6, and 7). For example, SSA officials and experts acknowledged that activities associated with the “OIS work taxonomy” part of the research and development plan—activities that together determine which information should be included in the OIS—are an essential building block that will inform other OIS activities, such as developing the “OIS work analysis instrument,” which will determine how OIS information will ultimately be collected. Nevertheless, when we tested SSA’s schedule for sequencing and linkages associated with the OIS work taxonomy activity, we found a significant delay of almost 1,000 days that should have significantly delayed the project actually barely affected the final completion date in the OIS project. Such missing links between key activities in the schedule represent broken logic that reduces the reliability of the forecasted dates. The OIS schedule also did not meet the best practice of conducting a schedule risk analysis (best practice 8), which is an essential tool for project managers to understand the most important risks to the project and focus on mitigating them. Based on the schedule provided to us, we do not have any indication that SSA has considered the vulnerability within its schedule for meeting time frames for individual activities which would in turn impact the time frames for the entire project. Without explicitly identifying risks to its schedule, SSA either does not know or is Page 20 GAO-12-420 Modernizing SSA Disability Criteria not conveying the probability of completing research and development activities on time, and its relationship to overall costs for this phase. However, in order to conduct a schedule risk analysis the schedule has to be properly sequenced and networked with all logic links in place for the analysis to produce credible results. Table 4: Assessment of OIS Schedule Extent best practice Best practice met 1. Capturing all activities Minimally met 2. Sequencing all activities Minimally met 3. Assigning resources to all activities Minimally met 4. Establishing the duration of all activities Minimally met 5. Integrating schedule activities horizontally and vertically Not met 6. Establishing the critical path for all activities Not met 7. Identifying float between activities Minimally met 8. Conducting a schedule risk analysis Not met 9. Updating the schedule using logic and durations to Not met determine dates Source: GAO analysis of SSA’s schedule for the OIS. Note: Not met—SSA provided no evidence that satisfies any of the criteria; minimally met—SSA provided evidence that satisfies a small portion of the criteria; partially met—SSA provided evidence that satisfies about half of the criteria; substantially met—SSA provided evidence that satisfies a large portion of the criteria; and met—SSA provided complete evidence that satisfies the entire criteria. See appendix IV for more information on these best practices. Other Challenges with OIS Beyond gaps in their cost estimate and schedule of specific activities, Development Could SSA faces broader challenges that could impede the success of the OIS Undermine the Project’s design and implementation; however, SSA has not done a formal risk analysis of these challenges. 36 While SSA officials said they had recently Success begun discussing various risks and how they might address them, they said it was premature to provide us with this information. Examples of potential challenges for OIS that experts and stakeholders we spoke with cited include: 36 Risk assessment helps decision makers identify and evaluate potential risks to an agency’s mission, so that countermeasures can be designed and implemented to prevent or mitigate the risks. As part of a risk assessment, agencies could identify alternatives to alter the likelihood or outcome of a high risk situation. Page 21 GAO-12-420 Modernizing SSA Disability Criteria • SSA’s lack of expertise with designing an OIS. SSA does not have prior experience with designing a complex and, in some respects, unprecedented occupational information system. As such, SSA will need to depend on many outside experts and contractors to complete the system. While the Occupational Information Development Advisory Panel members can provide some expert counsel, most of its members lack the technical and scientific background essential to informing this complicated effort. To help design the OIS, SSA recently hired a research psychologist and plans to hire two more individuals in the near future, but may need additional resources. In March 2012, SSA put out a request for information and, according to an agency official, received ideas from knowledgeable experts about how to move forward with technical aspects of the project. Until the agency secures required and sufficient expertise, SSA may not be able to move ahead with key technical decisions, such as defining the number of occupations to include and the data collection methodology, which may cause further delays. • Cost of maintaining an OIS. SSA has not yet made design decisions on the OIS that will ultimately define the overall cost of producing, implementing, and maintaining the OIS system. For example, SSA officials told us that they were not sure how many occupations they would include in the OIS, although they believe the total number will be somewhere between the number of occupations in the O*NET and the DOT, or between approximately 1,000 and 12,000 occupations. A 2010 study conducted by the National Academy of Sciences reviewing Labor’s O*NET cited the three major cost drivers of an occupational information system as (1) size of sample, (2) number of occupations, and (3) frequency of updates. This study noted that as the agency increases any one of these cost drivers, it does so at the expense of the others or of the overall cost. For example, if SSA chooses to frequently update the entire OIS, the agency may need to make tradeoffs with the number of occupations or sample size it chooses if it wishes to contain costs. • Managing large, multiyear projects. In the past, SSA has experienced difficulties managing complex, multiyear efforts. In prior work, we reported that SSA has cancelled numerous demonstration projects due to limitations or weaknesses in design or implementation and lacked sufficient controls to ensure effective management of its Page 22 GAO-12-420 Modernizing SSA Disability Criteria demonstration projects. 37 We have also reported on challenges that SSA faced managing two separate redesigns of its disability decision- making process. 38 While the OIS project alone represents a complex effort, stakeholders have emphasized the importance of this project progressing in close concert with other ongoing research managed by another part of SSA, placing greater emphasis on broad and effective project planning. 39 In light of past experiences and as SSA faces a potential change in leadership in 2013, the management shortcomings we identified increase the vulnerability of this complex project. Given the range of challenges and potential risks they pose to the success of the OIS project, it is important to identify and carefully explore feasible alternatives that may mitigate these risks. Examples of potential alternatives suggested by experts and other observers in the disability field include: • Leveraging O*NET. In a report issued in 2010, SSA’s advisory panel found that it did not believe that O*NET would meet SSA’s needs, in part, because it is not detailed enough. However, the National Academy of Sciences also issued a report in 2010 and concluded that O*NET could be altered to better meet SSA’s needs for disability adjudication. SSA officials have told us that modifying O*NET would not result in savings for the agency, but they have not conducted any analysis to determine this. While most disability experts agree that O*NET in its current form would not be a suitable source of occupational information for SSA, the National Academy of Sciences study noted that there are potential linkages between O*NET and an 37 GAO, Social Security Disability: Management Controls Needed to Strengthen Demonstration Projects, GAO-08-1053 (Washington, D.C.: Sept. 26, 2008). 38 SSA’s most recent redesign effort—the Disability Service Improvement initiative—was suspended in large part in 2007 by the new Commissioner, due to budgetary and other considerations. 39 Specifically, while the OIS is identifying functional requirements in the workplace, another SSA office is working through an interagency agreement with the National Institutes of Health to identify methods to more accurately capture the functional capacity of individuals. Both SSA and its advisory panel recognize that SSA’s disability criteria must reflect both the capabilities of people and work demands, and that the OIS must be designed and developed to support SSA’s process for assessing individual function. As such, some stakeholders believe it is critical that these two efforts be fully aligned for the overall success of the disability determination process. Page 23 GAO-12-420 Modernizing SSA Disability Criteria SSA OIS that are worthy of more careful exploration on SSA’s part and that leveraging one system to serve broader purposes could be most cost-effective overall. • Adjusting the scope of the OIS. SSA could reduce costs by limiting the scope of the OIS. For example, the OIS could be designed to capture fewer occupations. SSA officials have told us that since they are early in their design, they do not know how many occupations will be included in their OIS. Since this decision is fundamental to informing the OIS data collection methods, feasibility, and cost, consideration of alternative scope should be given high priority attention. • Limiting data collection methods. Another key factor that SSA must consider is the method they will use to collect data for their OIS as some methods are more costly than others. For example, among the methods SSA is considering, conducting on-site job analyses of occupations would be more expensive than surveys, which are also more resource and time intensive than collecting information through telephone interviews. • Leveraging resources from other agencies and OIS users. Other federal agencies with experience in data collection and occupational information have resources that SSA could leverage. SSA officials told us that they have been considering how they could benefit from the infrastructure that other agencies such as Labor and the U.S. Census Bureau have for data collection, but that they have not identified specific ways to leverage those resources. Additionally, while SSA needs to assure the OIS serves its most basic needs, it may be appropriate for SSA to explore the possibility of cost sharing or applying users’ fees with organizations that currently rely on the outdated DOT and could ultimately benefit from using a more modern and comprehensive OIS database. 40 40 Current users of the DOT include organizations in the private and public sectors, such as companies providing life insurance and short and long term disability insurance, and organizations providing state and federal vocational rehabilitation services, among others. Page 24 GAO-12-420 Modernizing SSA Disability Criteria SSA Has Taken Other Steps to Modernize Criteria but Faces Limitations SSA Has Taken Steps to Although SSA’s adult disability programs were initially built upon the Incorporate Modern assumption that certain severe medical conditions equate to work Concepts of Disability into incapacity, through its medical listings updates and ongoing research, SSA has taken steps to modify its eligibility criteria. SSA now is taking a more Eligibility Criteria modern view of disability that looks beyond the claimant’s medical condition by giving greater consideration to his or her functional capacity, that reflects the International Classification of Functioning, Disability and Health (ICF) framework. 41 According to the 2007 Institute of Medicine report, a modern concept of disability should recognize that disability is not just inherent in the individual and his or her medical condition, but is the result of complex interactions between the person, the person’s medical impairments, assistive devices to which they have access, and features of his or her socioeconomic environment, such as the presence or lack of accessible transportation and workplace accommodations. Under this concept, as described in this report, two people with the same impairment might have different degrees of work disability for a variety of reasons. Updates to the medical listings have been one vehicle through which SSA can include an assessment of an individual’s functional abilities to determine whether an impairment prevents work. For example, as part of SSA’s comprehensive revision to the listings for the immune system, the agency included several functional criteria, such as performing activities of daily living, maintaining social functioning, and completing tasks in a 41 As noted earlier, the ICF focuses on ability to function despite a medical impairment, including taking into account the impact of environmental factors, such as products and technology, attitudes, and services. The World Health Organization developed the ICF as a universal classification of disability and health for use in health and health-related sectors, especially as a planning and policy tool for decision makers. All 191 World Health Organization member states endorsed the use of the ICF as the international standard to describe and measure health and disability. As a result, the international community now has classification schema that provides both a common language and underlying code for disability terminology. There are four basic ICF domains, with associated codes, in the ICF classification: Body Functions, Body Structures, Activities and Participation, and Environmental Factors. Page 25 GAO-12-420 Modernizing SSA Disability Criteria timely manner despite deficiencies in concentration or persistence. Generally, SSA officials, adjudicators, and disability experts we spoke with support incorporating appropriate functional criteria into the medical listings to facilitate a more reliable assessment of an individual’s ability to work. However, some have also noted that as SSA continues to more broadly incorporate functional criteria into listings updates, such efforts may result in a more subjective assessment by adjudicators that could increase the difficulty of making step 3 disability determinations. Since 2008, SSA has had an ongoing interagency agreement with the National Institutes of Health (NIH) to conduct short- and long-term research that has informed SSA’s efforts to incorporate functional information into the disability criteria. For example, in its 2011 annual report to SSA, NIH presented its findings on the use of functional criteria, as defined by the ICF, in the adult listings. The objective of the project was to quantify the comprehensiveness and consistency of functional criteria among the adult listings and any influence this has in determination of outcomes. NIH found that while the use of functional terms were particularly apparent in five body systems, nearly one-half of the 14 body systems did not capture the influence of health conditions and impairment on human functioning. NIH is currently working on another project, which examines the presence and consistency of functional terms in the listings criteria. It is estimated the project will be completed by 2013. Besides exploring how medical listings could further incorporate functional considerations, SSA is also sponsoring longer term research through the NIH to develop an automated method to more quickly and comprehensively evaluate a claimant’s functional abilities for its disability determinations. Specifically, project researchers are developing a computer-based tool to rapidly and reliably assess the functional abilities of individual claimants considering their medical conditions. 42 As envisioned, the claimant, a medical provider, or both would respond to a series of questions through the computer-based tool, which would likely 42 Boston University’s Health and Disability Research Institute is developing this tool under a subcontract with NIH. The tool is based on Item Response Theory using Computer Adaptive Testing. The tool is adaptive in the sense that each item that is administered is tailored to the unique level of functioning of an individual. The items are selected based on the individual’s prior responses assessing that individual’s level of function, while precision is gained regarding a person’s placement along a continuum of functioning by asking more questions at the level that is appropriate for that individual. Page 26 GAO-12-420 Modernizing SSA Disability Criteria take an hour to complete the entire battery, perhaps less if a more limited assessment is warranted. The project’s intended scope encompasses six areas of function consistent with the ICF activity domain: (1) mobility, (2) interpersonal/social interaction and relationships, (3) self-care, (4) communication, (5) general tasks and demands (such as multitasking and carrying out daily routines), and (6) learning and applying knowledge. NIH researchers said they plan to consider the use of common personal assistive devices, such as wheelchairs, in developing the tool. As of April 2012, NIH had completed preliminary testing of instruments in two of the six areas—mobility and interpersonal/social interaction and relationships—with a sample of claimants and providers and had begun analyzing the data. SSA and NIH officials anticipate several benefits from the functional assessment tool, such as providing information on the impact of impairments more consistently, comprehensively, and early in the disability determination process. This information would help adjudicators more accurately assess whether a person can perform certain kinds of work given his or her functional and occupational capabilities. While this research is promising, there are several unknown variables. SSA officials said they have not yet determined when or how the tool will be integrated into the disability determination process. NIH officials indicated they believe that the tool may be most useful early in the determination process. SSA officials said they expect to pilot the functional assessment tool after all relevant domains have been tested and validated, which will likely be by 2016. Additionally, NIH researchers told us that, ultimately, this tool, which collects information on the applicant, would need to be aligned with the OIS, which collects information on the demands of work. SSA and NIH officials reported they have held meetings to share their progress to date, but both projects are in the early stages and additional coordination will be needed going forward. SSA is in the preliminary stages of taking additional steps to broaden its use of functional criteria throughout the disability determination process in response to other findings by NIH. Specifically, NIH evaluated a number of forms that SSA uses during the disability application and determination processes to determine how well these forms captured information on claimant functional activity as described by the ICF. The researchers found that current SSA assessment processes had major gaps in covering the ICF concepts. For example, SSA captured only limited information relative to sensory experiences, such as watching or listening, learning and applying knowledge, communication, and interpersonal Page 27 GAO-12-420 Modernizing SSA Disability Criteria interactions and relationships, all of which NIH considered particularly relevant to work. NIH concluded that the gaps need to be addressed in order to characterize individual functioning more comprehensively in relationship to the demands of the workplace. Another step the agency reports that it plans to take is the issuance of a Federal Notice of Solicitation of Collaboration from federal agencies in developing a standard for coding functional capacity in federal disability programs based on the ICF. The notice will explain that SSA believes using the ICF would help standardize how agencies describe and measure different aspects of disability, improve the clarity and comparability of research findings, and strengthen the base of scientific knowledge that guides public policies and health practices. In addition, SSA has asked the Institute of Medicine to plan an international symposium focused on how best to use and assess function in the disability determination process. SSA officials also stated that the agency may ultimately revise the residual functional capacity forms based on criteria from the ICF. SSA Faces Constraints to Consistent with modern views of disability, a key consideration in Incorporating Modern assessing disability is that the environment can hinder or enhance an Concepts individual’s ability to function. While assistive devices and workplace accommodations can play a critical role in an individual’s ability to function in the work environment, SSA does not always consider them in its assessment of disability. Regarding the incorporation of assistive devices into the medical listings, SSA officials and experts we spoke with expressed concern about the extent to which specific technologies should be incorporated if they are not widely available. SSA officials told us that they currently incorporate assistive devices into the medical listings once these devices become standard in the medical community—a threshold that SSA officials described as generally involving some combination of availability, accessibility, and insurance coverage. After an assistive device, such as a prosthetic device for walking, is incorporated into a listing, adjudicators must evaluate the individual’s ability to walk with the device being used. For example, evaluations of people who have had amputations involving a lower extremity or extremities are to be done with the prosthetic device in place. When we asked if wheelchairs are considered standard in the medical community, and whether SSA considers how individuals with wheelchairs might function in today’s knowledge-based labor market, given their age and education, we heard conflicting information from SSA officials. Some explained that individuals in wheelchairs are generally allowed at step 3 on the basis of their underlying medical condition, without evaluating how their disability might be assessed at steps 4 or 5. Other officials maintained that according to Page 28 GAO-12-420 Modernizing SSA Disability Criteria SSA’s policy, an individual in a wheelchair would be evaluated for upper body strength or other medical issues. However, SSA officials we spoke with said they have not evaluated these types of allowances specifically. Regarding workplace accommodations, SSA officials said their policy is to not consider them for several reasons. First, officials cited SSA’s inability to ensure that workplace accommodations are provided by employers—a concern shared by other disability experts we interviewed. SSA officials also indicated the agency would be unable to assess the effectiveness of workplace accommodations for claimants. Further, officials noted that SSA already faces resource constraints managing its disability claims workload and expanding the scope of individualized assessments would exacerbate those constraints. Finally, they noted that data on the availability and use of workplace accommodations are lacking. SSA’s policy notwithstanding, some opportunities exist for SSA to learn more about the availability of workplace accommodations. For example, developing the OIS may provide an opportunity to obtain some, albeit limited, information on workplace accommodations for the disability determination process. Specifically, while SSA officials reported that they do not plan to collect specific information on workplace accommodations as they develop the system, they may collect more specific information on the physical requirements of different jobs, such as options for sitting or standing. Some experts we spoke with agreed that while information on workplace accommodations would be immensely useful to include in the OIS, given the current scope of that project, they agreed collecting this information via the OIS would likely be too great a task for SSA to accomplish with existing budget constraints and time frames. However, another potentially viable option for collecting more information about workplace accommodations might be the Disability Research Consortium, which SSA is creating pursuant to section 1110 of the Social Security Act, as amended. 43 SSA envisions the consortium as a 5-year cooperative agreement that will serve as a national resource for fostering high quality research, communication, and education on matters related to disability policy, such as identifying or eliminating barriers encountered by people with disabilities in returning to or maintaining work. See 42 U.S.C. § 1310. 43 Page 29 GAO-12-420 Modernizing SSA Disability Criteria Researchers and disability experts have commented on the limitations of SSA’s current disability program eligibility criteria to fully consider whether an individual can function in the workplace due to limited consideration of assistive devices, including those in the workplace—limitations that we have also noted. 44 Moreover, officials we spoke with from an organization of vocational examiners expressed frustration when they see young individuals who could work with minor accommodations being provided disability benefits likely throughout their working life, rather than receiving support to pursue work. Representatives of the organization added that minor accommodations can include a stool for sitting or devices to assist with vision impairments. Although giving broad consideration of assistive devices and workplace accommodations may be difficult to incorporate into the current disability criteria and process, considering how common and inexpensive workplace supports may affect work disability seems feasible and reasonable. SSA has taken important and concrete steps toward modernizing its Conclusions disability criteria, but varied challenges may prevent the agency from achieving its goals. Specifically, SSA has a better process in place for updating medical listings but delays in meeting its own goals will likely continue unless the agency explicitly identifies and assigns the resources needed to achieve them. SSA is also taking bold and needed steps to replace outdated occupational information with a new OIS. However, especially in an era of governmentwide fiscal constraints, success of its OIS may hinge on SSA prioritizing its analysis of risks and feasible alternatives to address potential funding shortfalls and other significant challenges. Regardless of the shape and scope of the OIS, absent a complete, reliable, and transparent cost estimate and schedule, SSA risks schedule and performance shortfalls. Finally, SSA is sponsoring promising research to increase consideration of functional ability in its medical listings and other aspects of eligibility criteria—research that experts believe must be aligned with the design of SSA’s new OIS system at the earliest opportunity to best serve the disability determination process. On the other hand, SSA is not considering the full range of assistive devices and workplace accommodations available today. While SSA and others raise valid concerns as to their universal availability and 44 GAO, SSA Disability Programs: Fully Updating Disability Criteria Has Implications for Program Design, GAO-02-919T (Washington, D.C.: July 11, 2002). Page 30 GAO-12-420 Modernizing SSA Disability Criteria other considerations, in the absence of studies on how certain assistive devices and workplace accommodations are playing a role in helping individuals with impairments to stay at work or return to work, and their costs in comparison with potentially many years of disability benefit payments, SSA may be missing an opportunity to potentially assist individuals with disabilities to re-engage with the workforce and to target finite resources efficiently and effectively. 1. To achieve the goal of updating listings for each body system within Recommendations for SSA’s 5-year time frame, we recommend that the Commissioner of Executive Action Social Security explicitly identify the resources needed to achieve this goal, such as staff, contractors, and technology aids, and its plans to overcome any resource limitations. 2. To ensure that its work to revise occupational information is feasible and cost effective, and to improve its chance for success, we recommend that the Commissioner of Social Security: • formally assess risks to the success of the OIS—addressing such challenges as related to controlling cost, acquiring expertise, managing project complexity, and coordinating with ongoing and related SSA research—and develop appropriate mitigation strategies, and • develop a comprehensive and reliable cost estimate and schedule for the life cycle of the project, in accordance with best practices. 3. To help ensure that SSA’s disability decisions are as equitable and consistent with modern views of disability as possible, we recommend the Commissioner of Social Security conduct limited and focused studies on the availability and effects of considering more fully assistive devices and workplace accommodations in its disability determinations. We provided a draft copy of this report to SSA and the Departments of Agency Comments Commerce, Health and Human Services, and Labor for review and and Our Evaluation comment. The Department of Commerce did not provide comments. SSA, Labor, and the Department of Health and Human Services provided technical comments, which we incorporated as appropriate. SSA also provided written comments, which are reproduced in appendix V. Page 31 GAO-12-420 Modernizing SSA Disability Criteria SSA agreed with the first two recommendations and disagreed with the third recommendation that the agency should conduct limited and focused studies on the availability and effects of considering more fully assistive devices and workplace accommodations in its disability determinations. SSA stated that conducting such studies is inconsistent with Congress’ intentions. Specifically, SSA noted that the Americans with Disabilities Act of 1990 (ADA), as amended, addresses requirements for workplace accommodations. SSA referred to a 1999 Supreme Court case which acknowledged the complexity of SSA involvement in determining the availability of reasonable accommodations and also noted that ADA determinations are separate from disability determinations under the Social Security Act. SSA asserts that because Congress has made no effort to change the balance between its disability programs and the ADA in the past 22 years, it would be inappropriate for SSA to spend its limited administrative resources on “an initiative that would undermine the balance Congress chose to strike.” Notwithstanding SSA’s response, we continue to believe that SSA should conduct limited and focused studies on the availability and effects of considering more fully assistive devices and workplace accommodations in its disability determination process. Although SSA asserts that workplace accommodations are addressed by the ADA, this act serves a very different purpose than SSA’s disability programs. We do not think that the fact that workplace accommodations are addressed by the ADA would necessarily preclude SSA from potentially considering them in making disability determinations. Further, although we agree with SSA that Congress has not explicitly directed the agency to consider incorporating assistive devices and workplace accommodations into its disability determinations process, we also note that Congress has not explicitly prohibited SSA from making such considerations. Because the language of the Social Security Act is silent on this issue, and in the absence of clear Congressional intent, we think it would be reasonable for SSA to conduct limited studies on the feasibility of considering such factors. SSA further cites the 1999 Supreme Court decision in Cleveland v. Policy Management Systems Corporation to point out that assessing reasonable accommodation may turn on highly disputed workplace-specific matters, and that an SSA misjudgment about that detailed matter could deprive a disabled person of the financial support the statute provides. While this is true, SSA fails to acknowledge that by not incorporating such factors, it may be providing benefits to individuals who are capable of working with accommodations, thereby potentially expending scarce government Page 32 GAO-12-420 Modernizing SSA Disability Criteria resources in a manner that may not be most appropriate. SSA’s response acknowledges that the agency’s resources are limited—a concern that we share. By conducting limited and focused studies on this issue, SSA would be in a better position to thoughtfully weigh the costs and benefits of these various policy options before deciding on an appropriate course of action. As we discussed in the report, SSA has avenues to research the availability of select devices or accommodations and the impact of their inclusion on disability determinations, such as through the SSA funded Disability Research Consortium, which is charged to conduct research to identify and eliminate barriers encountered by people with disabilities in returning to work or maintaining their ability to work. SSA could also incorporate discussion of assistive devices and workplace accommodations into its planned symposium with the Institute of Medicine that will assess the inclusion of functional measures in the disability determination process. Through these efforts, SSA may be able to identify certain common and widespread workplace accommodations that could be incorporated into the disability criteria, just as it incorporates certain medical treatments and assistive devices into the medical listings when they become a medical standard. Despite SSA’s efforts to better address an individual’s functioning in its disability criteria, without at least evaluating the costs and benefits of considering more assistive devices and workplace accommodations, SSA is likely to face ongoing barriers in its efforts to integrate a more modern concept of disability into its determination process. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Commissioner of the Social Security Administration and the Secretaries of Commerce, Health and Human Services, and Labor; relevant congressional committees; and other interested parties. In addition, this report will be available at no charge on GAO’s website at http://www.gao.gov. Page 33 GAO-12-420 Modernizing SSA Disability Criteria If you or your staff have any questions about this report, please contact me at (202) 512-7215 or firstname.lastname@example.org. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI. Sincerely yours, Daniel Bertoni, Director Education, Workforce, and Income Security Issues Page 34 GAO-12-420 Modernizing SSA Disability Criteria Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology Methodology We were asked to assess the Social Security Administration’s (SSA) plans and efforts to revise its disability criteria. Specifically, we examined the status and management of efforts to update its medical listings to reflect current medical knowledge and develop a new occupational information system (OIS) to reflect labor market changes, and we identified other steps taken by SSA to incorporate a modern view of disability into its eligibility criteria. For this review, we narrowed our scope to criteria used for initial adult disability determinations that involve the medical listings at step 3 or a medical-vocational assessment at steps 4 and 5 of SSA’s five-step sequential process. To determine the status of activities to revise the medical listings and to develop the OIS, we reviewed our prior reports on the subject and SSA Office of Inspector General reports, relevant federal laws and regulations, and program documentation. This documentation included policies, such as those listed in SSA’s Program Operations Manual System for examiners; strategic goals for fiscal years 2011 and 2012 presented in the Agency Performance Plan for 2012; and other guidance. We also interviewed SSA officials from relevant offices with direct responsibility for revising disability criteria, as well as those offices that provide support for these efforts, and key project contractors and stakeholders. To obtain contextual information on modernization, we reviewed relevant literature, including studies, position papers, and testimonies from disability groups and commissions and interviewed disability experts. We assessed the reliability of data used in this report and found it to be sufficiently reliable for our purposes. To determine the extent to which SSA’s efforts to revise its medical listings and develop a new OIS were anchored in sound project management practices, we first identified sound project management practices by reviewing A Guide to the Project Management Body of Knowledge, our guidance on internal controls, and the Government Performance and Results Act of 1993. We also identified our recent work (for fiscal years 2008, 2009, and 2010) that evaluated federal planning efforts. Through these efforts, we identified six practices for sound project management which we used as a framework for evaluating SSA’s efforts to revise the medical listings and develop the OIS. Although there is no established set of requirements for all plans, we determined that these practices help implementing parties and decision makers effectively shape policies, programs, priorities, and resource allocations so that they can achieve desired results while ensuring accountability. While these practices may be organized in a variety of ways or use different terms, for the purposes of this report, we grouped them into six categories, from Page 35 GAO-12-420 Modernizing SSA Disability Criteria Appendix I: Objectives, Scope, and Methodology plan conception to implementation. Given the differences among SSA’s efforts to update the medical listings and develop the OIS, we selected broad, higher-level criteria that may apply to a wide variety of projects or plans (see table 5). Table 5: Six Sound Project Management Practices Used to Evaluate SSA’s Efforts Project management practice Description Plan to identify problem, In establishing the problem, goals, and methodology, the agency identifies the problem to be addressed goals, and methodology and the causes of the problem, as well as the strategy, including goal and objectives, and the methodology for achieving these goals and objectives. Activities and timeline An agency should identify and document the specific activities that must be performed in order to complete the project. The agency should develop a schedule that defines, among other things, when work activities will occur, how long they will take, and how they are related to one another, as well as interim milestones and checkpoints to gauge the completion of the project. Resources The agency should identify the sources and types of resources or investments needed (e.g., budgetary, human capital, information technology, research and development, contracts) to complete project activities. If resource availability cannot be assured, the agency will need to establish alternate plans. The agency should develop a reliable estimate of the costs of these resources. Coordination The agency should identify stakeholders—individuals and organizations that are involved in or may be affected by project activities—and ensure that they are included in developing and executing the project plan, allowing them to contribute appropriately. In addition to internal communications, management should ensure there are adequate means of communicating with, and obtaining information from, external stakeholders that may have a significant impact on the agency achieving its goals. Risk Risk assessment generally includes estimating the significance of risks from both external and internal sources, assessing the likelihood of its occurrence, and deciding how to manage the risk. Performance evaluation The agency should describe how goals will be achieved and establish performance indicators to be used in measuring or assessing the relevant outputs, service levels, and outcomes of each program activity, and identify a process to monitor and report on progress. Source: GAO analysis based on Project Management Institute information, the Government Performance and Results Act of 1993, and GAO reports. Additionally, we compared the cost estimate and schedule for completing SSA’s OIS and related documents with best practices in our Cost Estimating and Assessment Guide. We compared these practices to the OIS project because it requires a significant commitment of resources and time by SSA to complete and will result in an end product. For SSA’s OIS cost estimate and schedule, we scored each best practice as follows: • Not met—SSA provided no evidence that satisfies any of the criteria. • Minimally met—SSA provided evidence that satisfies a small portion of the criteria. Page 36 GAO-12-420 Modernizing SSA Disability Criteria Appendix I: Objectives, Scope, and Methodology • Partially met—SSA provided evidence that satisfies about half of the criteria. • Substantially met—SSA provided evidence that satisfies a large portion of the criteria. • Met—SSA provided complete evidence that satisfies the entire criteria. We provided the results of our schedule and cost analyses to SSA officials and met with them to confirm the results. Based on the interviews and additional documentation provided by SSA officials, we updated the results of our analyses, as needed. Page 37 GAO-12-420 Modernizing SSA Disability Criteria Appendix II: Progress Updating Six Body Appendix II: Progress Updating Six Body Systems since Last Comprehensive Revision Systems since Last Comprehensive Revision Note: SSA’s special senses and speech listings include impairments associated with speech and balance (i.e., disorders of the labyrinthine-vestibular function). Page 38 GAO-12-420 Modernizing SSA Disability Criteria Appendix III: Description of Cost Estimating Appendix III: Description of Cost Estimating Best Practices Best Practices Characteristica Characteristic description Comprehensive The cost estimate should include all costs necessary to achieve agency objectives including government and contractor labor costs as well as any necessary material or equipment costs. Comprehensive cost estimates should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted. Specifically, the cost estimate should be based on a standardized structure that allows a program to track cost and schedule consistently over time. Finally, where information is limited and judgments must be made, the cost estimate should document all cost-influencing ground rules and assumptions. Well documented The cost estimate should be supported by detailed documentation that describes the purpose of the estimate, the program background and system description, the scope of the estimate, the ground rules and assumptions, all data sources, estimating methodology and rationale, and the results of the risk analysis. Moreover, this information should be captured in such a way that the data used to derive the estimate can be traced back to, and verified against, the sources. Accurate The cost estimate should be based on an assessment of most likely costs (adjusted for inflation), documented assumptions, historical cost estimates, and actual experiences on other comparable programs. Estimates should be cross-checked against an independent cost estimate for accuracy, double counting, and omissions. In addition, the estimate should be updated to reflect any changes. Source: GAO analysis. a We did not assess SSA’s estimate on a fourth characteristic—credibility—which evaluates any limitations of the analysis because of uncertainty or biases surrounding data or assumptions, as this criterion is most applicable to major capital acquisitions. Page 39 GAO-12-420 Modernizing SSA Disability Criteria Appendix IV: Description of Scheduling Best Appendix IV: Description of Scheduling Best Practices Practices Best practice Description Capturing all activities A schedule should reflect all activities defined in the program’s work breakdown structure and include all activities to be performed by the government and contractor. Sequencing all activities The schedule should be planned so that all activities are logically sequenced in the order they are to be carried out. Assigning resources to all The schedule should realistically reflect the resources (i.e., labor material and overhead) needed to activities do the work, whether all required resources will be available when needed, and whether any funding or time constraints exist. Establishing the duration of all The schedule should reflect how long each activity will take to execute. activities Integrating schedule activities The schedule should be horizontally and vertically integrated—that is, it should link already horizontally and vertically sequenced activities with outcomes while also delineating the relation of supporting tasks and subtasks to upper-level milestones. Such mapping among levels enables different groups to work to the same master schedule. Establishing the critical path for The schedule should identify the critical path, or those activities that, if delayed, will negatively impact all activities the overall project completion date. The critical path enables analysis of the effect delays may have on the overall schedule. Identifying float between The schedule should identify float—the amount of time an activity can slip in the schedule before it activities affects other activities—so that flexibility in the schedule can be determined. As a general rule, activities along the critical path typically have the least amount of float. Conducting a schedule risk The schedule should include a schedule risk analysis that uses statistical techniques to predict the analysis probability of meeting a completion date. A schedule risk analysis can help management identify and understand the most important risks and focus on mitigating them. Updating the schedule using The schedule should use realistic durations for activities and be monitored to determine when logic and durations to forecasted completion dates differ from the planned dates. This analysis can be used to assess determine dates whether schedule variances will affect future work. Source: GAO analysis. Page 40 GAO-12-420 Modernizing SSA Disability Criteria Appendix V: Comments from the Social Appendix V: Comments from the Social Security Administration Security Administration Page 41 GAO-12-420 Modernizing SSA Disability Criteria Appendix V: Comments from the Social Security Administration Page 42 GAO-12-420 Modernizing SSA Disability Criteria Appendix V: Comments from the Social Security Administration Page 43 GAO-12-420 Modernizing SSA Disability Criteria Appendix VI: GAO Contact and Staff Appendix VI: GAO Contact and Staff Acknowledgments Acknowledgments Daniel Bertoni, (202) 512-7215 or email@example.com GAO Contact In addition to the contact named above, Michele Grgich, Assistant Staff Director; James Bennett; Kate Blumenreich; Tisha Derricotte; Jennifer Acknowledgments Echard; Julie DeVault; Alex Galuten; Sheila McCoy; Patricia M. Owens; Carol Petersen; Karen Richey; Anjali Tekchandani; Kathleen Van Gelder; and Walter Vance made key contributions to this report. Also contributing to this report were Jaime Allentuck, Susan Bernstein, and Amy Frazier. (131069) Page 44 GAO-12-420 Modernizing SSA Disability Criteria GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no Obtaining Copies of cost is through GAO’s website (www.gao.gov). Each weekday afternoon, GAO Reports and GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, Testimony go to www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov. Contact: To Report Fraud, Waste, and Abuse in Website: www.gao.gov/fraudnet/fraudnet.htm E-mail: firstname.lastname@example.org Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Katherine Siggerud, Managing Director, email@example.com, (202) 512- Congressional 4400, U.S. Government Accountability Office, 441 G Street NW, Room Relations 7125, Washington, DC 20548 Chuck Young, Managing Director, firstname.lastname@example.org, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Please Print on Recycled Paper.
Modernizing SSA Disability Programs: Progress Made, but Key Efforts Warrant More Management Focus
Published by the Government Accountability Office on 2012-06-19.
Below is a raw (and likely hideous) rendition of the original report. (PDF)