United States General Accounting Office GAO Report to the Chairman, Subcommittee on Social Security, Committee on Ways and Means, House of Representatives March 1999 SSA DISABILITY REDESIGN Actions Needed to Enhance Future Progress GAO/HEHS-99-25 United States GAO General Accounting Office Washington, D.C. 20548 Health, Education, and Human Services Division B-277774 March 12, 1999 The Honorable E. Clay Shaw Chairman Subcommittee on Social Security Committee on Ways and Means House of Representatives Dear Mr. Chairman: This report responds to your Subcommittee’s request that we (1) assess SSA’s efforts to redesign its disability claims process and (2) identify actions that SSA could take to better ensure future progress. We are sending copies of this report to The Honorable Kenneth S. Apfel, Commissioner of Social Security; appropriate congressional committees; and other interested parties. We will also make copies available to others upon request. Please contact me on (202) 512-7215 if you or your staff have any questions concerning this report. Other GAO staff who contributed to this report are listed in appendix V. Sincerely yours, Cynthia M. Fagnoni Director, Income Security Issues Executive Summary In administering its disability programs, the Social Security Administration Purpose (SSA) is still struggling to cope with workloads that resulted, in part, from a dramatic growth in the number of applications for disability benefits in the early 1990s. In the 3-year period between fiscal years 1991 and 1993, initial claims for these benefits climbed by almost one-third—from 3 million to 3.9 million. As a result, SSA began experiencing increased difficulty processing disability claims in a timely manner. For example, claimants who were dissatisfied with their initial determination and filed an appeal often had to wait as long as 1-1/2 years for a final decision. Moreover, as many as two-thirds of claimants who filed an appeal eventually received a favorable decision at the hearing level, which indicates potential problems with either initial or appellate decisions and raises questions about the fairness and efficiency of the process. SSA concluded that the only way to effectively respond to these problems was to fundamentally overhaul the process for deciding whether or not a claimant is eligible for disability benefits. In 1994, SSA set forth an ambitious plan to redesign the process over a 6-year period. Concerned about the need to improve SSA’s disability claims process, the Chairman of the Subcommittee on Social Security, House Committee on Ways and Means, asked GAO to (1) assess SSA’s efforts to redesign its disability claims process and (2) identify actions that SSA could take to better ensure future progress. Depending on the number of times a claimant files an appeal, SSA’s process Background can include up to four decision points: (1) the initial disability determination; (2) a second independent review, called reconsideration; (3) a hearing decision by an administrative law judge; and (4) a review of the hearing decision by SSA’s Appeals Council—an independent review group composed of administrative appeal judges. The process involves a large number of diverse components, including SSA’s 1,298 field offices, where initial claims are taken; 54 state agencies, where doctors and examiners work as teams to make initial and reconsidered determinations on medical eligibility; and 140 hearing offices, where attorneys and administrative law judges consider appeals. With its September 1994 redesign plan, SSA hoped to achieve five goals that would improve the process, including making it more efficient and user-friendly. The plan originally included 83 initiatives to be accomplished over 6 years—38 of which were to be completed within the first 2 years. GAO concluded in a 1996 report that SSA’s plan was overly Page 2 GAO/HEHS-99-25 Disability Claims Process Redesign Executive Summary ambitious. At that time, SSA had made little progress toward meeting its goals, lacked demonstrable results, and faced difficulties obtaining and keeping the support of some stakeholders. In response to these and other concerns, SSA issued a scaled-back redesign plan in February 1997. The new plan focused on testing and implementing eight key initiatives—each representing a major change to the system—within 9 years instead of the original 6. Five of the eight initiatives had near-term milestones; that is, they were to be tested, implemented, or both by the close of fiscal year 1998. Two of these initiatives involved establishing new decisionmaker positions intended to help make disability decisions faster and more efficiently. Each of these new positions was to be tested in a “stand-alone” fashion—that is, not in the context of other proposed and related changes. A third initiative combined in a single test the two new decisionmaker positions and several other process changes, including a proposal to eliminate the reconsideration and Appeals Council review levels. The two remaining initiatives with near-term milestones aimed to provide essential supports for the entire disability claims process—one by improving the consistency of disability decisions between the initial and appellate levels of the process and the other by improving SSA’s quality assurance process. Even with its scaled-back plan, SSA has been unable to keep its redesign Results in Brief activities on schedule and to demonstrate that its proposed changes will significantly improve the claims process. While SSA has made some progress, overall, it has not met most of the milestones for testing or implementing its five near-term initiatives. Moreover, its stand-alone tests of the two new decisionmaker positions consumed valuable resources and provided marginal or inconclusive results. As a result, the tests did not support the wider implementation of the positions. The inability to keep on schedule was caused, in part, by SSA’s overly ambitious plan, which involved numerous large tests and required SSA to move forward on many fronts simultaneously. In addition, SSA’s strategy for testing proposed changes independently, rather than together with other related changes and key supports, contributed to the disappointing test results. Finally, other problems with the design of its tests further weakened the agency’s ability to predict how the initiatives would operate if implemented. The problems that led to SSA’s redesign effort persist, and as SSA continues its efforts to improve the disability claims process the agency has an opportunity to learn from its experience and the best practices of other Page 3 GAO/HEHS-99-25 Disability Claims Process Redesign Executive Summary organizations with reengineering experience. To its credit, SSA has already taken steps in response to the problems encountered in its redesign effort by, among other things, strengthening its executive oversight. However, delays and limited progress to date suggest that these steps, while important, are not enough to ensure success. SSA could improve its chances of making future progress by further scaling back its near-term efforts to include only initiatives that are critical to improving the disability claims process. In addition, by testing related process changes together, rather than on a stand-alone basis, and at a smaller number of sites, SSA could free up resources while still obtaining valuable data. Moreover, experiences of other organizations suggest that SSA faces perhaps its greatest challenge after it completes testing of a process change and begins to implement it in a “real world” environment. Because a process change might function differently under actual operational conditions than it did in a test environment, SSA will need to take additional action to ensure that it achieves hoped-for results. For example, SSA will need to revise its performance measures to better monitor and more fully assess the impact of changes on the entire process. Further, SSA will need to ensure that an adequate quality assurance process is in place so that any changes SSA makes to the process do not compromise the quality of decisions. Principal Findings Progress Has Been Even under its revised plan, SSA has made limited progress in redesigning Hindered by Redesign its disability claims process. As of October 1998, the agency was behind Strategy schedule on all five of its near-term initiatives. SSA postponed implementation of its two new decisionmaker positions, is behind schedule with its evaluation of its integrated test, and was behind on its efforts to improve its quality assurance process. While the agency is also behind schedule in addressing the high number of claims that are allowed on appeal, it has made some progress in this area. For example, SSA provided uniform training to over 15,000 decisionmakers from all components of the disability claims process. Agency officials believe this training and other related efforts have contributed to 90,000 people receiving deserved benefits earlier in the process. However, other factors can also influence the decision-making process, and it is difficult to isolate the effects of SSA’s efforts with certainty. Page 4 GAO/HEHS-99-25 Disability Claims Process Redesign Executive Summary In addition to missing milestones, SSA has not clearly demonstrated the efficacy of its proposed changes. Its stand-alone tests of the two proposed decisionmaker positions did not demonstrate clear improvements to the process and, in one case, produced unreliable results. Consequently, SSA decided to wait for data from its integrated test—which will show the efficacy of these two positions in combination with other process changes—before making implementation decisions. Preliminary results from this test are more promising for some, but not all, of the proposed changes. SSA’s limited progress under its revised plan is due at least in part to the plan’s overly ambitious scope and the agency’s strategy for testing its proposed process changes. SSA’s approach of moving ahead on many fronts simultaneously—including conducting several large tests—was difficult to manage within established time frames. For example, in fiscal year 1998, SSA had five tests ongoing at over 100 sites involving over 1,000 test participants. Our field visits to a variety of sites and our discussions with stakeholder groups underscored the challenges of keeping the redesign effort on schedule. Each test included time-consuming activities, such as coordinating the activities of many state and federal offices and building consensus among stakeholder groups. By diluting the redesign team’s energies among so many different tests, SSA limited its ability to keep its plan on track. In addition, SSA’s decision to conduct stand-alone tests contributed to disappointing and inconclusive results because key supports and related initiatives were not in place during the tests. Finally, other limitations in SSA’s test design and management made it difficult for SSA to assess how an initiative would operate if actually implemented. For example, in one test of a new decisionmaker position, SSA did not ensure that test cases and control cases were similarly handled; as a result, SSA’s test results were not meaningful. SSA’s Experience and Best The problems that led to SSA’s redesign effort persist, and future workloads Practices Suggest Actions are likely to increase as a result of demographic changes. As a result, the to Increase Chances of agency must continue to seek ways to improve its work processes, enhance its workforce skills, and make more effective use of technology. Future Success As it proceeds with steps to improve its disability claims process, it has an opportunity to learn from its past mistakes. To its credit, SSA has begun to apply some lessons it has learned. For example, it created a new executive-level committee to oversee progress and direct resources and energies where they are most needed. But the extent of the problems experienced up to this point suggest that this and other steps taken by SSA Page 5 GAO/HEHS-99-25 Disability Claims Process Redesign Executive Summary will probably not be enough to keep its redesign plan on track, and more significant changes in its reengineering approach will probably be needed. Relatedly, SSA will need more concrete and positive results to assuage the persistent concerns of stakeholders regarding the wisdom of specific process changes. Reengineering experts agree that successful reengineering efforts require sharp focus, and SSA’s own experience confirms that additional focus is needed. As with its original redesign plan, SSA’s 1997 redesign plan called for moving forward on so many fronts that SSA continued to miss milestones and experience difficulty in making concrete improvements to the process. Prioritizing SSA’s five key redesign objectives would provide additional focus, and concentrating the agency’s efforts on the initiatives most likely to achieve critical redesign objectives would improve its chances of making concrete progress in crucial areas. SSA’s testing approach has not been adjusted to capture lessons learned from SSA’s previous experiences and commonly accepted reegineering practices. For example, SSA’s planned test of yet a third decision-making position involves many sites and does not include a number of key supporting initiatives, such as the simplified decision methodology and redesigned computer system. Conducting this test at fewer sites would free up resources and help SSA keep its initiatives on track. SSA could also reduce the likelihood of disappointing test results by developing these key supports before testing the initiative. Also, before investing significant resources on testing this particular initiative, SSA could explore alternatives on a small scale, so that feasible options would be available if this proposed initiative proved ineffective or difficult to implement. SSA is currently considering whether to broadly implement certain proposed changes to the claims process. The experiences of other public and private organizations that have attempted to significantly change a process indicate that implementing change is more difficult than testing it. Organizations naturally resist change, and new processes may function differently when they are no longer operating in a test environment. Therefore, SSA will need to closely monitor the results of changes that are implemented to determine whether the changes are achieving the intended improvements to the process. To do so, SSA will need a more complete set of performance goals and measures than it currently has. It will also need a fully developed quality assurance process before it implements major system changes. The quality assurance process is especially important Page 6 GAO/HEHS-99-25 Disability Claims Process Redesign Executive Summary because of uncertainty that the newly tested decisionmaker positions will perform as effectively outside the test environment as in it. This report makes a number of recommendations to improve SSA’s Recommendations approach to redesigning its disability claims process by focusing its resources on those initiatives that are most likely to achieve SSA’s most critical redesign objectives, modifying its testing approach to avoid previously experienced pitfalls, and taking steps to put in place a comprehensive set of performance goals and measures and adequate quality assurance procedures. The full set of recommendations can be found in chapter 4. SSA agreed with GAO’s observations that the redesign efforts to date have Agency Comments not resulted in the magnitude of benefits projected in the original redesign plan and with GAO’s recommendation that SSA focus on those areas that will make the greatest contributions to the agency’s redesign objectives. On the other hand, SSA maintains that its large-scale, stand-alone testing strategy was both necessary and effective, and that its current approach to testing a third position is consistent with GAO’s concerns and recommendations. However, GAO continues to believe that SSA could make better use of limited resources by first exploring the efficacy of this third position, and alternative concepts, on a smaller scale. SSA also stated that it will continue to use its current approach and measures to monitor and evaluate redesign changes, whereas GAO believes that a more carefully crafted set of performance measures and goals is needed to effectively monitor and evaluate changes. SSA also provided technical comments, which were incorporated as appropriate. The full text of SSA’s comments is included as appendix IV. Page 7 GAO/HEHS-99-25 Disability Claims Process Redesign Contents Executive Summary 2 Chapter 1 12 Problems With SSA’s Disability Claims Process Came to the Fore 13 Introduction in the Early 1990s SSA’s Initial Efforts to Redesign the Process Moved Slowly 20 SSA Scaled Back Its Redesign Plan 25 Objectives, Scope, and Methodology 31 Chapter 2 33 Progress Under Revised Plan Has Been Limited 33 Progress Under Limited Progress Is Related to SSA’s Testing and Implementation 39 Revised Plan Strategy Hindered by Overly Ambitious Strategy and Problems With Test Design Chapter 3 46 Many Steps Remain Under the Revised Plan 46 Actions Needed to SSA’s Actions to Revise Its Approach Have Been Helpful but 50 Improve Chances for Insufficient to Ensure Success Sharper Focus Is Needed on Most Critical Areas 52 Future Progress A Modified Testing Approach Could More Efficiently Provide 54 Valuable Information Implementation Will Require Careful Monitoring 58 Chapter 4 63 Conclusions 63 Conclusions, Recommendations 65 Recommendations, Agency Comments and Our Evaluation 65 and Agency Comments and Our Evaluation Appendixes Appendix I: Descriptions of Near-Term Redesign Initiatives and 68 Their Test and Implementation Schedules Page 8 GAO/HEHS-99-25 Disability Claims Process Redesign Contents Appendix II: SSA’s Efforts to Meet Near-Term Milestones 78 Appendix III: Results of SSA’s Tests 82 Appendix IV: Comments From the Social Security Administration 88 Appendix V: GAO Contacts and Staff Acknowledgments 93 Bibliography 94 Related GAO Products 96 Tables Table 2.1: Milestones and Status for Near-Term Initiatives 35 Table 2.2: Results of Tested Redesign Initiatives 37 Table 2.3: Projected Redesign Savings in Staff-Years for 1998 and 38 1999 President’s Budgets Table 2.4: SSA’s Test Schedule and Number of Sites and 41 Participants Table 3.1: Steps Remaining for the Eight Initiatives in SSA’s 48 February 1997 Plan Figures Figure 1.1: SSA’s Disability Claims Process as of October 1998 16 Figure 1.2: SSA’s Proposed Disability Claims Process 22 Figure 1.3: The Eight Initiatives and Milestones Under the 28 February 1997 Redesign Plan Page 9 GAO/HEHS-99-25 Disability Claims Process Redesign Contents Abbreviations AC Appeals Council ALJ administrative law judge AO adjudication officer DCM disability claims manager DDS disability determination service DI Disability Insurance FPM full process model HHS Department of Health and Human Services HO hearing office NPRM notice of proposed rulemaking OHA Office of Hearings and Appeals OQA Office of Quality Assurance PDI predecision interview POMS Program Operations Manual System QA quality assurance RDS Redesigned Disability System RFC residual functional capacity SDM single decision maker SSA Social Security Administration SSI Supplemental Security Income SSR Social Security Ruling Page 10 GAO/HEHS-99-25 Disability Claims Process Redesign Page 11 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction The Social Security Administration (SSA) manages two major federal disability programs that provide cash benefits to people with long-term disabilities: the Disability Insurance (DI) and Supplemental Security Income (SSI) programs. The DI program was enacted in 1954 and provides monthly cash benefits to severely disabled workers. SSI was enacted in 1972 as an income assistance program for aged, blind, or disabled individuals whose income and resources fall below a certain threshold. For both programs, disability for adults is defined as an inability to engage in any substantial gainful activity because of a severe physical or mental impairment.1 Both programs also use the same procedures for determining whether the severity of an applicant’s impairment qualifies him or her for disability benefits. In 1998, almost 11 million people received a total of over $73 billion in disability benefits from these programs. SSA’s complex process for determining whether an individual qualifies for a disability benefit—the disability claims process—has been plagued by a number of long-standing problems. For example, claimants who have been dissatisfied with the initial determination and have filed an appeal frequently have had to wait more than 1-1/2 years for a final decision. Moreover, as many as two-thirds of these determinations were subsequently allowed by an administrative law judge (ALJ). In the early 1990s, SSA had difficulty keeping up with a rapidly growing workload, and backlogs of appealed cases waiting for a hearing grew. In response to these problems, SSA concluded that minor improvements to the disability claims process would be insufficient and embarked on an effort to fundamentally reengineer, or redesign, its process.2 In 1994, the agency issued an ambitious plan for redesigning the process within 6 years. However, 2 years into implementing the redesign plan, SSA had not made much progress, and we and SSA concluded that the scope of the plan was too large. The agency reevaluated its approach and, in February 1997, issued a scaled-back plan with revised milestones. 1 The legal definition of disability for adults is the inability to engage in any substantial gainful activity by reason of any medically determinable physical or mental impairment that can be expected to result in death or to last for at least 12 months. A child is considered disabled if the child has a medically determinable physical or mental impairment that results in marked and severe functional limitations that can be expected to result in death or to last for at least 12 months. 2 SSA consistently uses “redesign” to describe its efforts to improve the disability claims process, although it considers this term synonymous with “reengineer.” In this report, we also use “redesign” to refer to SSA’s reengineering effort. Page 12 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction SSA’s disability claims process has long been recognized as complex and Problems With SSA’s fragmented. The decision about whether an individual is disabled is based Disability Claims on standards set forth in the Social Security Act and extensive SSA Process Came to the regulations and rulings. Moreover, disability decisions involve a multilevel process that spans many diverse components, including SSA’s 1,298 field Fore in the Early offices, 54 state agencies, and 140 hearing offices. This organizationally 1990s complex structure has contributed to a number of problems. For example, through the years a high percentage of claimants who were dissatisfied with their initial determinations received favorable decisions on appeal. Claimants have also waited a long time for final decisions on their eligibility. In the early 1990s, these problems were aggravated by mounting workloads, as applications for disability benefits escalated at the same time that SSA was experiencing a decline in its workforce. This, in turn, caused workloads to back up and increased the time it took claimants to receive decisions on their claims. The Disability Claims SSA’s disability claims process, which has not changed fundamentally in Process Is Complicated over 40 years, is inherently complex and fragmented. The process contains and Fragmented several opportunities for appeal, and the organizational unit involved, professional background of the adjudicator, and procedures for making a decision on appeal are all different from those of the initial determination. Each organizational unit has separate lines of authority and goals without responsibility for the overall outcome of the process. The claims process starts when an individual contacts one of SSA’s 1,298 field offices across the country to apply for benefits. Field office personnel help claimants complete their applications; obtain a detailed medical and work history; and identify other, nonmedical eligibility factors.3 Field office personnel then forward the claims to one of 54 disability determination service (DDS) agencies that are administered by the 50 states and the District of Columbia, Guam, Puerto Rico, and the Virgin Islands. Under a unique federal-state arrangement, SSA pays state DDSs to determine whether claimants are disabled.4 At the DDS, a team consisting of a 3 Nonmedical eligibility factors include such considerations as whether the applicant has paid Social Security taxes for enough years and recently enough to be covered under Social Security for DI benefits and whether the applicant has sufficiently low income or resources to be eligible for SSI benefits. 4 This arrangement was instituted, in part, because the states had prior experience in administering various disability-related programs and had established working relationships with the medical community. DDSs are required to follow SSA policy guidelines but are not under the direct administrative control of SSA. Page 13 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction specially trained disability examiner and an agency physician or psychologist reviews the available medical evidence and gathers additional medical evidence, if necessary. In making the disability determination, the team follows official guidance found in SSA’s Program Operations Manual System (POMS), which is based on applicable laws and SSA’s regulations and rulings and also includes detailed instructions for processing cases. If the claimant is dissatisfied with the initial determination, the claimant may request a reconsideration review within 60 days of receiving the determination. Reconsideration is also performed by the DDSs and is based on the same guidance as the initial determination but is carried out by a new adjudicative team. If the claimant is dissatisfied with the determination, he or she has 60 days to appeal and request a hearing before an ALJ. ALJs are hearing officers located at 140 hearing offices around the country that are administered by SSA’s Office of Hearings and Appeals (OHA). ALJs review the file to determine if additional medical evidence is needed, conduct a hearing, and render a decision.5 ALJs conduct de novo hearings; that is, ALJs may consider or develop new evidence and are not bound by DDS determinations. These hearings often present the first opportunity for face-to-face contact between claimants and the individuals deciding their eligibility. In rendering a decision, ALJs do not follow the POMS but rely directly on applicable laws and SSA regulations and rulings. ALJs are subject to the Administrative Procedure Act, which affords them some independence in making a disability decision.6 Finally, if the ALJ denies the claim, the claimant has 60 days to request a review by the Appeals Council, an independent review group attached to the OHA and composed of administrative appeals judges. The Appeals Council may decide to dismiss the request for review, grant the request and issue its own decision, or remand the case back to the ALJ. The Appeals Council is the claimant’s fourth and final level of administrative review. Upon exhausting these administrative remedies, the claimant may file a complaint with a federal court. Figure 1.1 shows the four decision points in SSA’s current disability claims process. 5 Under a temporary measure directed at reducing hearing office workloads, a request for a hearing may be reviewed and the claim fully allowed by a senior staff attorney. 6 The Administrative Procedure Act protects the ALJ’s independence by restricting the extent to which SSA management can exert control over these adjudicators. For example, ALJ salaries are determined by the Office of Personnel Management, not by SSA. Page 14 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Page 15 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Figure 1.1: SSA’s Disability Claims Process as of October 1998 Page 16 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Page 17 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction SSA’s approach to reviewing the quality of the disability decision reflects the complex and fragmented nature of the process. As we have previously reported, current quality assurance reviews focus on DDS determinations and ALJ decisions in isolation from one another, and the approach for reviewing DDS determinations differs from the approach for reviewing ALJ decisions.7 Reviews of DDS determinations are conducted by staff from SSA’s Office of Quality Assurance (OQA). These reviews focus heavily on DI claims that have been allowed.8 In conducting their quality review, OQA staff use the same approach, policy, and procedures that the DDSs use in reaching a determination; that is, they rely on the POMS. In contrast, only a small number of ALJ allowance decisions are selected for review by SSA’s Appeals Council. For the most part, reviews of ALJ decisions predominantly consist of reviews of claims denied by ALJs and appealed by claimants to the Appeals Council. In reviewing ALJ decisions, the Appeals Council relies on the same laws and SSA regulations and rulings as those used by ALJs. Long-Standing Problems SSA’s disability claims process has long suffered from problems associated Are Associated With the with its complexity and fragmentation. Among these problems are the high Claims Process allowance rates by ALJs of appealed DDS determinations. In fiscal year 1993, before SSA issued its redesign plan, 68 percent of determinations that were appealed received favorable decisions at the hearing level. High ALJ allowance rates have been attributed to a number of factors. According to SSA, an ALJ might arrive at a different decision than a DDS because the claimant’s condition has worsened, or because ALJs are more likely than DDS decisionmakers to meet with claimants face-to-face, and thus have access to more or different information. However, SSA studies have also found that DDS and ALJ adjudicators often arrive at different conclusions even when presented with the same evidence.9 Disability decisions require difficult judgments, and adjudicators sometimes reach different conclusions. Further, DDS and ALJ adjudicators use medical expertise 7 Social Security Disability: SSA Must Hold Itself Accountable for Continued Improvement in Decision-Making (GAO/HEHS-97-102, Aug. 12, 1997). 8 On the basis of statutory requirements, OQA staff review 50 percent of all allowed DI claims and, if they find errors, return these claims to the DDSs for correction before the determination becomes final. These reviews—called pre-effectuation reviews—are conducted in order to avoid erroneously awarded claims and to protect the DI trust fund. Also, SSA’s OQA staff randomly select a small percentage of claims awarded or denied by the DDSs and use the results to compute the accuracy rates of DDS offices. 9 SSA, Office of Program and Integrity Reviews, Findings of the Disability Hearings Quality Review Process (Washington, D.C.: SSA, Sept. 1994) and Secretary of Health and Human Services, Implementation of Section 304 (g) of Public Law 96-265, Social Security Disability Amendments of 1980 (the Bellmon Report) (Washington, D.C.: HHS, Jan. 1982). Page 18 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction differently and rely on different documents for guidance when making decisions. Finally, training has not been delivered consistently or simultaneously to all groups of decisionmakers. This high rate of allowances at the hearing level has raised questions about the fairness, integrity, and cost of SSA’s disability program. In fiscal year 1998, the cost of making a determination at the DDS level was $547 per case, while the cost of an ALJ decision was an additional $1,385. In general, the costs of administering these disability programs reflect the demanding nature of the process: in fiscal year 1998, SSA spent about $4.3 billion, or almost 66 percent of its administrative budget, on the disability programs, even though disability beneficiaries are only 21 percent of the agency’s total number of beneficiaries. Another long-recognized problem with SSA’s claims process is that many claimants must wait a long time for their final decisions. Because of the multiple levels and decision points in the process, a great deal of time passes while a claimant’s file is passed from one employee or office to another. Delays are also caused by the need to obtain extensive medical evidence from health care providers to document the basis for disability.10 One SSA study conducted in 1993 showed that an average claimant waited up to 155 days from initial contact with SSA until receiving an initial determination notice, during which time 16 to 26 employees might have handled the claim. Only 13 hours of these 155 days were spent on “task time”—that is, time spent working directly on the case. Further, the study found that it could take up to 550 days from initial contact to receipt of a hearing decision, with only 32 hours of this time spent on task time. As a result of these multiple handoffs and the general complexity of the process, SSA believes claimants do not understand the process and have had difficulty obtaining meaningful information about the status of their claims. The Mounting Disability In the early 1990s, SSA’s problems with its disability claims process came to Workload in the Early the fore as the growing workload placed additional pressure on SSA’s 1990s Exacerbated already inefficient process. The number of initial claims had been rising steadily, but it increased dramatically between fiscal years 1991 and Problems 1993—from about 3 million to 3.9 million, or almost 32 percent.11 Moreover, future increases were expected. At the same time, SSA had to 10 According to SSA, these providers often do not understand the requirements, find the forms confusing, or feel burdened by the requests for evidence. 11 This increase does not include applications for SSI by aged claimants. Page 19 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction manage this growing workload with staffing levels that had been falling since the 1980s. As a result, SSA’s disability workload began to accumulate during this period. Most dramatically, the number of pending hearings almost doubled between 1991 and 1993—from 183,471 to 357,564. To address these long-standing problems and dramatically improve SSA’s Initial Efforts to customer service, SSA embarked on a plan in 1994 to radically redesign its Redesign the Process disability claims process by completing 83 initiatives over 6 years. We Moved Slowly concluded in a 1996 report, however, that 2 years into the plan, SSA had yet to achieve significant progress.12 SSA’s slow progress was due in part to the overly ambitious nature of the redesign plan, the complexity of the redesign initiatives, and inconsistent stakeholder support and cooperation. SSA’s Initial Redesign Concerned about the inefficiency of the disability claims process and its Efforts effect on the quality of service to the public, SSA’s leadership decided in 1993 that the agency needed a strategy for radically improving the process. SSA reviewed reengineering efforts and approaches in other organizations and concluded that process reengineering was critical to achieving its strategic objective of providing world-class service.13 SSA then created a Disability Process Redesign Team composed of 18 SSA and state DDS employees with varied experience and backgrounds and charged it with fundamentally rethinking and redesigning SSA’s claims process from start to finish.14 Consistent with commonly held reengineering principles, the team collected extensive information on the process itself and options for improving it.15 These efforts culminated in a redesign proposal that was widely distributed throughout SSA and the state DDSs and to interested public and private individuals and organizations to solicit comments, concerns, and ideas for improvement. The proposal was also published in the Federal Register, and a comment period elicited 6,000 written 12 SSA Disability Redesign: Focus Needed on Initiatives Most Crucial to Reducing Costs and Time (GAO/HEHS-97-20, Dec. 20, 1996). 13 The agency decided to follow reengineering principles championed by Michael Hammer and others at the time. Reengineering was, therefore, defined as the fundamental rethinking and radical redesigning of business processes to bring about dramatic improvements in performance. 14 Several aspects of the process were designated to be outside the scope of this reengineering project—for example, the use of an ALJ as the presiding officer at administrative hearings. 15 The team received briefings from staff in all components involved in the disability claims process, visited and interviewed experts and interested parties both within and outside SSA, held focus group sessions with claimants, studied the claims process in successful organizations to identify best practices and process innovations, researched and documented the current process, and used computer models to compare the current processes with possible alternatives. Page 20 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction responses, which were considered as SSA finalized its initial redesign proposal. In September 1994, SSA issued its vision for fundamentally redesigning the disability claims process. SSA’s vision included five objectives for the redesigned process: (1) making the process “user-friendly,” (2) allowing claims that should be allowed at the earliest possible level, (3) making the disability decision as quickly as possible, (4) making the process efficient, and (5) providing a satisfying work environment for employees. SSA’s vision was based on more consistent guidance and training for all adjudicators; an automated and simpler claim intake and appeal process; a single, comprehensive quality review process; and a simplified method for making disability decisions. From the claimant’s perspective, the redesigned process was to offer a range of options for filing a claim; provide a single point of contact; and have fewer decision points, as shown in figure 1.2. Page 21 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Figure 1.2: SSA’s Proposed Disability Claims Process Page 22 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Page 23 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction SSA had high expectations for its proposed redesigned process. The agency projected that the combined changes to the process would, by fiscal year 1997, result in a 25-percent improvement in productivity and customer service over projected fiscal year 1994 levels, and a further 25 percent by the end of fiscal year 2000—all without a decrease in decisional accuracy. SSA did not expect the overall redesigned process to alter total benefits paid to claimants, but it estimated that the changes would result in administrative cost savings of $704 million through fiscal year 2001, and an additional $305 million annually thereafter. After putting forth its broad vision, SSA issued in November 1994 a more detailed plan for developing, testing, and finally implementing proposed disability process improvements. The plan originally included 83 initiatives to be accomplished over 6 years. SSA recognized in its implementation plan that most, if not all, of the proposed process changes were interdependent, and that the development, testing, and implementation of related changes would need to be properly sequenced. For example, SSA recognized that all activities and associated benefits were dependent on improvements to its computer system, which were not expected to be completed until the end of the 6-year time frame. Progress on the Initial In 1996 and 1997, we issued several reports that raised concerns regarding Redesign Plan Was Slow SSA’s redesign effort. These concerns included, among other things, a lack of progress and demonstrable results. For example, we reported that SSA had not fully completed any of the 38 near-term initiatives it had hoped to accomplish in the first 2 years.16 As a result, SSA did not have any concrete results available to demonstrate the efficacy of its proposed initiatives. SSA’s slow progress was due in part to its overly ambitious redesign plan and the complexity of some of its redesign initiatives. We reported that SSA did not follow best practices when it decided to take on a large number of initiatives concurrently.17 Specifically, we reported that successful reengineering calls for focusing on a small number of initiatives at one time, whereas SSA decided to tackle 38 initiatives in the first 2 years of its redesign effort. Moreover, some of these initiatives were large in scope and very complex. For example, scheduled implementation of SSA’s large and complicated initiative for redesigning its computer system was delayed because of problems identified during testing. 16 GAO/HEHS-97-20, Dec. 20, 1996. 17 GAO/HEHS-97-20, Dec. 20, 1996. Page 24 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Some aspects of SSA’s redesign plan faced considerable opposition. As part of its redesign effort, SSA had identified over 100 individual groups—both internal and external to SSA—as having a stake in the process and whose involvement was, in many cases, critical to the entire disability claims process. These stakeholder groups—which included various SSA employee unions and associations, state entities and organizations, congressional committees, other federal agencies, and advocacy groups—had a wide variety of views on SSA’s plan, and some opposed specific initiatives. For example, SSA’s plan called for a new position—a disability claims manager (DCM)—that would combine the duties of field office and DDS personnel into one position. The DCM represented significant change to the current process, and SSA faced numerous challenges in obtaining stakeholder cooperation for this key initiative. In light of these difficulties and in order to increase SSA’s chance of success, we recommended in our December 1996 report that SSA reduce the scope of its redesign effort by focusing on those initiatives considered most crucial to improving the process and testing those initiatives together, in an integrated fashion, at a few sites. In another 1996 report, we recommended that, concurrent with the first phase of its DCM test, SSA test alternatives that we believed were more feasible and compare their relative costs and benefits with those of the DCM before deciding to increase the number of DCM test positions.18 Later, we supported SSA’s redesign efforts associated with its initiative to improve the consistency of disability decision-making and recommended, among other things, that SSA establish a performance goal for this key redesign initiative.19 As a result of our input, the overall lack of progress, and stakeholder SSA Scaled Back Its concerns, SSA reassessed its approach to redesign and issued a revised Redesign Plan plan in February 1997. The new plan focused on eight key initiatives, each one intended to effect a major change to the system.20 The plan also included updated tasks and milestones for each key initiative and expanded the time frame for the entire redesign project from 6 to 9 years, 18 SSA Disability Redesign: More Testing Needed to Assess Feasibility of New Claim Manager Position (GAO/HEHS-96-170, Sept. 27, 1996). 19 GAO/HEHS-97-102, Aug. 12, 1997. 20 Some initiatives in the original implementation plan were deferred. Still others, considered to be good business practices, were “institutionalized”; that is, SSA shifted responsibility for implementing them from the Disability Process Redesign Team to front-line components without further testing or development. Page 25 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction ending in 2003. The eight initiatives and their milestones are described in figure 1.3. Page 26 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Page 27 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction Figure 1.3: The Eight Initiatives and Milestones Under the February 1997 Redesign Plan Page 28 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction a See app. I for a complete description of the eight subinitiatives. As shown in figure 1.3, five of the eight initiatives had relatively near-term deadlines—that is, before the end of fiscal year 1998—for completing a key test or beginning implementation. Two of these initiatives involve Page 29 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction testing new positions and, if test results warrant, implementing new positions on a stand-alone basis—that is, independently of other, related initiatives. One new position, the single decision maker (SDM), would expand the DDS disability examiner’s authority to determine certain claims without relying on the DDS physician; the SDM would instead use the physician as a consultant on an as-needed basis. The SDM was expected to make the initial determination process faster and more efficient by eliminating handoffs to DDS physicians in those cases in which the appropriate determination was clear.21 Another new position, the adjudication officer (AO), would review cases that were appealed to the hearing level. The AO was to help claimants understand the appeals process and would have authority to grant disability benefits in cases in which it was clear that the claim merited a fully favorable decision. In all other cases, the AO was to make sure that all pertinent information was included in the case file and was fully explained, thus facilitating its use by the ALJ at the next level of appeal. By performing these tasks, the AO was expected to improve customer service and make the appeals process faster and more efficient. A third near-term initiative is the full process model (FPM) test. The FPM combines five proposed changes into a single test to investigate their interactive effects on creating a more efficient process and better customer service. The five tested changes are (1) creating the SDM position; (2) creating the AO position; (3) establishing a new predecision interview, in which the SDM would interview claimants when the evidence did not support a fully favorable determination in order to obtain any additional information before making the final determination; (4) eliminating the reconsideration step; and (5) eliminating the Appeals Council step—that is, removing the claimant’s option to request a review by the Appeals Council of an ALJ decision. The two other near-term initiatives—process unification and quality assurance—are considered essential elements for achieving correct decisions in the new disability claims process. The intent of the process unification initiative was to achieve similar results on similar cases at all stages of the process. To this end, SSA planned a number of activities, including conducting ongoing training; clarifying policies; and developing unified guidance, called a single presentation of policy, for making disability decisions across all levels of the process. SSA also planned to 21 Under both SSA’s original and revised plans, the responsibilities of two positions, the SDM and the field office representative, would eventually be combined into a new position, the DCM. This position would be the initial point of contact for claimants, as depicted in fig. 1.2. Page 30 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction complete eight additional subinitiatives—all designed to help reduce inconsistencies in decision-making between the DDS and ALJ levels. SSA’s quality assurance initiative included near-term activities in two areas. First, as part of each of the other major redesign initiatives, SSA planned to develop and test “in-line” quality assurance approaches—such as training, mentoring, and peer review—in order to build quality into the process before decisions are made. Second, SSA planned to develop and test a single “end-of-line” quality review mechanism that covered the entire adjudicatory process from beginning to end and provided data on problems or failures in a component’s in-line quality assurance process. Appendix I provides additional information on SSA’s five near-term redesign initiatives. The Chairman of the House Subcommittee on Social Security, Committee Objectives, Scope, on Ways and Means, asked us to (1) assess SSA’s efforts to redesign its and Methodology disability claims process and (2) identify any actions needed to better ensure future progress. We agreed to focus our work on the five initiatives in SSA’s scaled-back plan that have relatively near-term dates for testing, implementation, or both: the SDM, AO, FPM, process unification, and quality assurance initiatives. In assessing SSA’s redesign experience, we obtained documents from and interviewed SSA officials responsible for planning, managing, and evaluating redesign efforts. We visited several DDS and hearing office test sites and interviewed test participants and managers in Richmond, California; Brooklyn, New York; Raleigh, North Carolina; and Providence and Warwick, Rhode Island. We also interviewed SSA regional officials with responsibility for overseeing or coordinating redesign efforts within their regions as well as representatives of nine major stakeholder groups to obtain their views on SSA’s specific initiatives and general approach for redesign. Finally, we reviewed the literature and interviewed experts on business process reengineering. We conducted our work between August 1997 and November 1998 in accordance with generally accepted government auditing standards with the following exception: we did not independently verify agency data, including test data on redesign initiatives. We did obtain information from SSA on steps it took to obtain and verify the test data and any problems associated with them. We have noted our concerns regarding the validity and reliability of the data in the report, where appropriate. We obtained Page 31 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 1 Introduction comments from SSA officials responsible for the redesign tests, which we have summarized in chapter 4. Page 32 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design After 4 years of redesign efforts, SSA has made only limited progress toward improving its disability claims process. While narrowing the focus of its redesign plan has helped, SSA has continued to miss milestones and has not clearly demonstrated that the proposed initiatives would significantly improve the current process. As a result, SSA has had to defer service improvements and reduce estimated savings. The agency’s limited progress has resulted, in part, from SSA’s overly ambitious strategy for testing and implementing its redesign initiatives. Conducting a number of tests and other redesign activities simultaneously proved to be too difficult to keep on track. In addition, problems with SSA’s approach to designing and managing its tests of new initiatives contributed to marginal and inconclusive test results and made it more difficult for SSA to discern how a tested initiative would operate if implemented on a widespread basis. SSA has made only limited progress in improving its disability claims Progress Under process, despite having fewer initiatives in its revised redesign plan than in Revised Plan Has the original plan. The agency has not met most of its adjusted milestones Been Limited for testing and implementing its five near-term initiatives.22 Moreover, results from SSA’s stand-alone tests of two new decisionmaker positions, the SDM and the AO, were not compelling and did not support broader implementation. Therefore, SSA decided to wait for preliminary results of its integrated test, which has in fact produced some promising results. In addition, SSA has made progress under its process unification initiative, such as providing training and clarifying policy, and agency officials believe the actions taken thus far have had a positive effect on customer service. Overall, however, as a result of missed milestones and disappointing test results, SSA has deferred many other process improvements and reduced its redesign expectations for administrative savings. Delays Continue Under Even under its scaled-back plan, SSA continues to experience delays. As of Revised Plan October 1998, the agency was behind schedule on all five of the plan’s near-term initiatives. After more than 3 years of testing, SSA had yet to complete its test of the AO decisionmaker position and, for reasons discussed in the next section, SSA has delayed its decision on whether to implement both the SDM and AO decisionmaker positions. Also, SSA did not 22 While SSA has issued various testing and implementation schedules over time, this report compares SSA’s progress against the dates for testing and implementing initiatives laid out in its February 1997 plan. In many cases, these revised dates provided the agency with additional time to complete its planned activities. Page 33 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design complete its assessment of the FPM test results in fiscal year 1998 as scheduled. SSA has completed some of its planned activities under its process unification initiative, but it has missed other key implementation deadlines. The agency has clarified key policies and, since 1996, has issued policy instructions in the same format for all adjudicators. SSA has also provided an unprecedented training program involving 15,000 decisionmakers and quality reviewers from key components of the disability claims process and has adopted process unification principles for its ongoing training program by providing the same training to all adjudicators. However, SSA has experienced delays in several other planned activities. For example, the agency is behind schedule on a test to study the effect of requiring DDS adjudicators to more fully document the rationale they used in making particular disability determinations. SSA hopes this more detailed explanation will reduce decisional inconsistencies. SSA has begun work on its quality assurance initiative, but this effort has also been delayed. As part of its tests of other redesign initiatives, SSA has been exploring “in-line” quality assurance approaches—such as training and mentoring—that are intended to build quality into the process before decisions are made. SSA planned to institute these practices nationwide when it implemented the other redesign initiatives; however, delays in implementing the initiatives have delayed the widespread use of these quality assurance practices. In addition, SSA is more than a year behind in developing a single end-of-line review mechanism.23 The agency planned to develop one quality standard for its end-of-line reviews in fiscal year 1997 and to test its use in fiscal year 1998. However, as of the end of fiscal year 1998, SSA had not reached internal agreement on what that single quality review standard should be. Key milestones and the status of SSA’s five near-term initiatives are summarized in table 2.1. Additional information on SSA’s efforts to meet near-term milestones—including specific actions taken to date and the nature and extent of delays—can be found in appendix II. 23 As discussed in ch. 1, SSA quality reviewers currently use different criteria for reviewing disability decisions depending on the level at which the decision was made. Page 34 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design Table 2.1: Milestones and Status for Near-Term Initiatives Date for planned Milestones (through FY 1998) action Planned action Status as of October 1998 SDM initiative 4/1997 Complete phase I test and Completed 11/1997 evaluation. FY 1998 Begin implementation. Not started FY 1998/1999 Complete phase II test. Ongoing AO initiative FY 1997 Complete test. Not completed 1/1998 Begin nationwide rollout. Not started FPM initiative 1/1998 Complete test case selection. Completed 1/1998 FY 1998 Complete assessment of four Not completed of five process changes.a Process unification initiative FY 1997 Complete training of 15,000 Completed FY 1997 adjudicators and quality reviewers. FY 1997 Implement eight subinitiatives. Three completed (FY 1997 and 1998); five not completed Quality assurance initiative FY 1997 Develop end-of-line review Not completed mechanism. FY 1997 Develop in-line QA procedures Completed FY 1997 to support testing of AO and SDM initiatives. FY 1998 Test review mechanism. Not started FY 1998 Develop QA procedures to Not completed support testing of FPM and DCM redesign initiatives. a The four changes being assessed under FPM are the SDM, AO, predecision interview, and elimination of reconsideration. Sources: SSA planning documents and officials. Efficacy of Redesign As of October 1998, SSA had not clearly demonstrated that its proposed Initiatives Has Not Yet changes would achieve the desired improvements in the disability claims Been Fully Demonstrated process. SSA had expected the new SDM and AO positions to significantly improve the efficiency and processing time without sacrificing the quality of decision-making. However, results from the stand-alone tests of these positions have been largely disappointing and, in some cases, Page 35 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design inconclusive. As a result, SSA decided to postpone implementation decisions on these two initiatives until results from the agency’s integrated FPM test were available. As an example, SSA had hoped that permitting the SDM to make disability determinations independently, using the DDS physician only on an as-needed basis, would reduce the time spent on the determination process. However, early test results revealed that the SDM position would, on average, reduce by only 1 day the time claimants waited for an initial determination and by only 3.6 minutes the time personnel actually spent working on the case. Moreover, SDM determinations for certain impairment categories were less accurate than under the current system. However, early results from the test of the FPM initiative, where the SDM was tested with other process changes, have shown more promise for the SDM. SSA’s final evaluation of the FPM test for four of the five process changes will not be available before October 1999. Table 2.2 shows the final or most recent results of the tests of the three initiatives. Appendix III contains more detailed information on test results for these three initiatives. Page 36 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design Table 2.2: Results of Tested Redesign Initiatives Test results Redesign objectives SDM AO FPM Allow claims that Increased Increase in Increased allowances should be allowed at allowances (by 1.2 allowances made (by 4.7 percent) and the earliest possible percent), but earlier at the improved accuracy level. reduced accuracy appellate level was (mostly denial (mostly denial not demonstrated, determinations) at the determinations) at but improved initial level, but the initial level. accuracy of some improvements at the allowance appellate level not decisions was demonstrated. demonstrated. Make decisions as Marginal Did not achieve Improvement quickly as possible improvement (1 targets; demonstrated at the (that is, reduce days day faster) at initial improvement over DDS level due to needed to make a final level. the current elimination of decision). process not reconsideration step, demonstrated. but extent of this improvement has not been determined. Improvement at the appellate level, or overall, not demonstrated. Make process more Marginal Did not achieve Improvement efficient (for example, improvement targets; demonstrated at the by reducing task time, (3.6-minute improvement over DDS level due to administrative costs, or reduction in task the current elimination of both). time) at the initial process not reconsideration step, decision level. demonstrated. but extent of this improvement has not been determined. Improvement at the appellate level, or overall, not demonstrated. Make process Not tested. Mixed results: Customer input user-friendly. customer input neutral regarding shifted from predecision interview. positive to neutral over time. Provide a satisfying Generally positive Mixed results: test Generally positive for work environment for for SDM test participants were SDM and AO test employees. participants. more satisfied with participants. AO position than OHA staff were. Note: SDM test results were reported in Nov. 1997 and are final. AO test results are from different time periods and generally reflect the most recent and complete data available. FPM test results were reported in Oct. 1998 and are preliminary, because not all of the cases have been processed through the AO and ALJ levels. Source: SSA test evaluation documents. Page 37 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design SSA has not made enough progress on its two other near-term initiatives, process unification and quality assurance, to fully assess their efficacy. Although SSA has not completed many of its planned measures for these two initiatives, some of the early process unification actions may have had a positive effect on customer service. SSA reported that it has accurately paid benefits to approximately 90,000 people 500 days earlier in the process than otherwise might have been the case. While SSA generally did not test these process unification initiatives before implementing them, officials believe that the increase in allowances made earlier in the process is in large part due to the agency’s process unification efforts. At the same time, these officials noted that other factors can influence allowance rates. Therefore, without conducting carefully structured tests, it is difficult to isolate the effects of actions taken by the agency. Delays and Disappointing As a result of the delays and disappointing test results, SSA has decreased Test Results Led to projected administrative savings and postponed the anticipated date for Decreased Expectations realizing any savings. In 1997, SSA projected savings of 12,086 staff-years for fiscal years 1998 through 2002, resulting from implementing several for Redesign process changes. SSA planned to use some of these staff-year savings to help with other workloads. Instead, in 1998, SSA both decreased its savings projections and postponed the date it expected to realize savings, which changed its projected staff-year savings to 7,207 through fiscal year 2003. Table 2.3 shows how SSA’s projected staff-year savings changed from its 1998 to its 1999 President’s budget. Table 2.3: Projected Redesign Savings in Staff-Years for 1998 and 1999 Staff-year savings President’s Budgets 1998 President’s 1999 President’s Difference (1998 Fiscal year budget budget to 1999) 1998 (861)a (654) 207 1999 1,684 (741) (2,425) 2000 3,551 1,850 (1,701) 2001 3,683 2,236 (1,447) 2002 4,029 2,187 (1,842) b 2003 2,329 2,329 Total 12,086 7,207 (4,879) a Numbers in parentheses represent additional staff-years needed. b The 1998 President’s budget provided projected redesign savings for fiscal years 1998 through 2002. Source: SSA budget documents. Page 38 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design More importantly, test results have not provided a compelling case for SSA to make these changes and thereby improve customer service as quickly as it had hoped. Overall processing times have not significantly changed since the beginning of redesign; that is, while processing times have decreased at the initial level, they have increased at the ALJ level.24 On the other hand, more allowances are being made earlier in the process, and SSA attributes this to its process unification efforts, which were planned to improve customer service without significantly increasing the overall cost of providing benefits. SSA’s difficulties in achieving appreciable improvements in its disability Limited Progress Is claims process have been caused, in part, by the scope of SSA’s revised Related to SSA’s plan and the agency’s strategy for testing its proposed process changes. Testing and Much like its original plan, SSA’s February 1997 plan was designed to achieve quick and major improvements on many fronts simultaneously in Implementation response to the pressing problems with the claims process. However, as Strategy with the original plan, SSA’s revised plan proved to be too ambitious and difficult to manage within established time frames. Moreover, SSA’s decision to conduct stand-alone tests contributed to marginal SDM and AO test results, and weaknesses in how SSA designed and managed its AO test contributed to unreliable AO test results. Finally, difficulties SSA experienced with testing changes in an operational environment raised questions about how a tested initiative would operate if implemented. SSA’s Revised Plan Still Like its original plan, SSA’s revised plan was designed to provide a number Proved Too Ambitious of near-term, visible improvements, while also laying the foundation for long-term changes. To accomplish this, SSA acted to make progress on a number of fronts simultaneously. For example, hoping to alleviate growing workloads at the appellate level, SSA began testing and planned to implement the AO position independently of other initiatives, even though certain changes that could support the position, such as the redesigned computer system, were still being developed. SSA also began testing and planned to implement the SDM position by itself because SSA believed it could achieve quick and decisive improvements through this position. The agency believed these quick improvements would build momentum for redesign and increase stakeholder support. While the AO and SDM tests were still ongoing, SSA began its FPM test, which investigated the interactive 24 Between fiscal years 1994 and 1998, processing time decreased at the DDS level from 86.9 and 109.6 days (for the DI and SSI programs, respectively) to 72.6 and 75.4 days and increased at the ALJ level, where data for the two programs are combined, from 305 to 341 days. SSA does not currently measure overall processing time. Page 39 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design effects of five process changes together: the new SDM and AO decisionmaker positions, the predecision interview with the claimant, eliminating the reconsideration step, and eliminating the claimant’s option to request a review by the Appeals Council. In addition, SSA began testing and developing subinitiatives under its process unification, quality assurance, and the three remaining longer-term initiatives. Given the urgent need to fix the process, SSA considered this ambitious approach appropriate as well as consistent with reengineering theory. At that time, reengineering theory generally called for short time lines for testing and implementing major process changes. In addition to this multifaceted approach, SSA decided for several reasons to conduct its tests of the proposed redesign changes at many sites and to involve numerous participants. First, officials believed this approach would build trust among employees and other stakeholders, who feared that redesign would negatively affect them. Second, SSA believed it needed to use a large number of test cases to produce statistically valid information in key areas. For example, SSA wanted sufficient data to determine the impact of redesign initiatives on the accuracy of SDM determinations in each major category of impairment. Finally, SSA wanted to have enough data to demonstrate the impact that changes to the process would have on benefit outlays. SSA officials told us that Office of Management and Budget officials were concerned that the proposed changes to the claims process could result in large, unanticipated increases in benefit outlays. Because of the size of the disability programs, even a small increase in the percentage of claimants awarded benefits can result in a significant increase in program costs. For example, SSA officials have roughly estimated that a 1-percent increase in allowances in the disability programs for a period of 10 years could result in an increase of $11 billion in the total benefits paid to beneficiaries—that is, program costs—during that same period. As a result of SSA’s decision to conduct many tests simultaneously, at one point SSA was testing four near-term initiatives and training test participants for another, longer-term initiative, the DCM. These tests were being conducted at more than 100 sites and involved over 1,000 participants. Table 2.4 shows SSA’s testing schedule, including numbers of sites and participants. Page 40 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design Table 2.4: SSA’s Test Schedule and Number of Sites and Participants Time period Initiative and (in fiscal years) Number Number of participants phase 1996 1997 1998 of sites Decisionmakers Clerical staff SDM, phase I X X 10 93 12 SDM, phase II X X 30 337 44 AO X X X 23-26 191 110 FPM X X 11 134 22 Process unification, expanded rationales X 10 209 93a DCM, phase I X 33 213 75 Total 2 4 5 117-120 1,177 263 a The amount of clerical staff time devoted to this test varied. Note: While some tests were conducted in the same fiscal year, the exact starting and ending dates of each test varied. The number of test sites and participants also varied over time. The number of participants represents SSA’s best estimate and does not include indirect support staff, such as supervisors and case consultants. The expanded rationales test also included physician and supervisor participants, which are not shown in the table. According to SSA, test participants were doing work comparable to what they would have been doing if they had not been in a test. Source: GAO analysis of data provided by SSA officials. Despite SSA’s good intentions, its scaled-back plan still proved to be too ambitious, and the agency had difficulty keeping it on track. Conducting several large tests that overlapped in time consumed a great deal of management attention and resources. In addition to developing the test plan, implementing and monitoring the test, and collecting and analyzing data, each test involved negotiating and coordinating activities with test sites, test participants, employee unions, and other stakeholders. This large array of testing and evaluation activities made it difficult for SSA to stay on schedule and simultaneously maintain sufficient focus on other redesign efforts—such as process unification and quality assurance. Unrealistic milestones for specific initiatives also contributed to missed deadlines. For example, SSA allowed itself only 17 months to conduct the FPM test and assess the results, even though it can take up to 21 months for a test case to make its way through the entire disability claims process. In addition, SSA’s milestones for the eight process unification subinitiatives were probably too ambitious (they did not include sufficient time to conduct needed tests or make procedural changes), especially given the overall magnitude of SSA’s redesign efforts and the complexity of the Page 41 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design problems these subinitiatives are intended to address. Moreover, some of the factors that contributed to differences between decisions made by the DDS adjudicators and the ALJs have evolved over a number of years and involve sensitive legal issues. Finally, other competing workloads placed considerable strain on SSA’s ability to manage the overall redesign effort. Besides the redesign initiatives, disability program officials and staff had to cope with additional unanticipated duties and responsibilities. For example, legislation that reformed the nation’s welfare program in 1996 also required that children receiving benefits under the SSI program meet a stricter definition of disability than had been applied in the past. As a result, during fiscal year 1997, when many redesign initiatives were being tested, SSA’s disability staff also had to plan and execute a review of the eligibility of over 288,000 children receiving SSI benefits. Decision to Conduct SSA’s decision to test its AO and SDM initiatives independently from related Stand-Alone Tests initiatives contributed to the disappointing test results. SSA conducted Contributed to Marginal these stand-alone tests because it wanted to institute these two positions quickly. However, as initially envisioned, these initiatives were expected Results to result in process improvements and administrative savings in concert with other initiatives. Tested alone, these positions did not demonstrate potential for significantly improving the process. To illustrate, at the very early stages of its redesign effort, SSA developed expectations for AO productivity assuming that the AO would be operating in a completely redesigned environment. However, the AO test did not include supporting initiatives, such as a redesigned computer system, and, consequently, AO productivity was far below SSA’s expectations. Similarly, the SDM was expected to be operating in a redesigned environment that included, among other changes, the new responsibility of conducting a predecision interview with claimants. The results of the stand-alone SDM test indicated a decline in the accuracy of initial determinations; on the other hand, the integrated FPM test indicated that adding the predecision interview to the SDM’s responsibilities may improve accuracy, as compared with the current process.25 This improved accuracy may have resulted because SDMs collected more or better data during the predecision interview or because SDMs performed their job more thoroughly in preparation for a meeting with the claimant. While SSA could not have 25 The AO position continued to perform poorly under the integrated FPM test, which did not include the redesigned computer system either. Page 42 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design predicted the precise impact of not including a particular process change in its stand-alone tests, the agency understood from the outset of its redesign effort that proposed changes were closely linked and that they depended on each other—especially on computer supports—to dramatically improve the process. Overall, the decision to conduct stand-alone tests caused delays, did not result in the efficient use of resources, and did not achieve the agency’s goal of quickly building trust and enthusiasm among those who resisted the changes. For example, despite the improved performance of the SDM in the FPM test, pockets of opposition to the SDM, particularly among groups representing some DDS physicians, still existed. While groups that perceive themselves to be negatively affected by change may not be swayed on the basis of clear and positive test results, marginal or inconclusive results provide detractors with a firmer basis to oppose change. Test Design and The AO test suffered from a number of design and management flaws that Management Problems raised questions about the reliability of certain test results. For example, Contributed to Unreliable to ensure that AO sites were staffed with the best employees possible, SSA selected test participants from a national pool and temporarily relocated AO Test Results them to their preferred locations. Since the test lasted some time, many of these employees decided to return to their home units, and SSA had to replace them with new, less experienced employees. Replacing test participants created instability in the test environment that negatively affected the test results. In addition, SSA did not arrange for AOs to have necessary supports (such as computers, clerical assistance, supervisors, or feedback from ALJs), which contributed to poor results. Consequently, SSA took steps to refine the AO test and provided additional supports, including training and feedback, to test participants. Accuracy and productivity subsequently improved, although productivity has not improved to the level originally expected by SSA. Despite these improvements, other problems with how the test cases were handled made it difficult for SSA to assess the efficacy of the AO position. Under the proposed process, an AO cannot deny a claim, so, when an AO does not allow a case, the AO is then required to make sure that all pertinent information is properly arranged in the case file and to prepare a thorough explanation of all medical evidence so that the case can move expeditiously to an ALJ hearing. To fairly assess the impact that the AO had on processing time at the appellate level, SSA planned to compare cases prepared by AOs with a small group of control cases in which no AO had Page 43 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design been involved. The two groups were to be handled in a comparable manner; for example, both sets of cases were to be promptly scheduled for hearings. However, in many instances OHA staff did not follow instructions concerning how the cases were to be handled. Since the number of control cases was relatively small, when the improperly handled cases were excluded from the analysis, the number of useable control cases was too small to permit a valid comparison. In addition, SSA did not design its test to determine the overall impact of the AO-prepared cases on the quality of decisions at the next appellate level.26 Without reliable data on its control group or sufficient data on the impact on quality, SSA could not fully assess the effect of the AO position on the claims process. Valid Tests Were Difficult SSA’s other tests—of the SDM and FPM—suffered from design problems that to Conduct in SSA’s stemmed largely from difficulties with trying to conduct a test in a “real Operational Environment world” operational environment. While the SDM and FPM tests provided information and insight into the efficacy of these two concepts, operational limitations made it difficult for SSA to conduct a statistically valid test and conclusively demonstrate how a tested initiative would operate if implemented. To a lesser extent than with the AO test, SSA’s test of its SDM initiative also provided incomplete information and limited assurance that the initiative would perform as tested. For example, under the current process, 50 percent of DDS allowance determinations are reviewed by regional quality assurance staff, and errors are returned to the adjudicator for correction. However, under the SDM stand-alone test, 100 percent of all determinations—allowances and denials—were reviewed by SSA quality assurance staff and returned for correction. As a result, a large number of cases were returned to SDM adjudicators even though, on average, there was not a large difference in error rates between the SDM and the current process. SDM test participants and other DDS officials told us that this 100-percent review probably caused test participants to rely more heavily on agency physicians than they might have otherwise. In addition, because SSA does not have administrative control over state DDS programs—which are under the direction of state governors—the agency was not able to select a strictly random group of test sites or participants; nevertheless, SSA officials believe that the participant selection methods they used came as close to random as possible, given the present constraints. Moreover, 26 In cases in which the AO had developed the case but could not allow the claim, the AO would certify that it was fully developed and ready for hearing. Although SSA surveyed ALJ views on the quality of AO certifications, SSA did not assess the overall impact of AO decisions and certifications on the quality of appellate decisions. Page 44 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 2 Progress Under Revised Plan Hindered by Overly Ambitious Strategy and Problems With Test Design because workloads and production capacity varied at the sites, SSA could not dictate the number of test cases at each site and was therefore unable to distribute the test caseload in a representative manner. Finally, the test was not initially designed to collect data on test cases as they moved beyond the initial determination level to the appeals level—data that would have helped determine the impact of the SDM on overall appeal rates, processing time, efficiency, and quality of determinations. SSA has since modified its approach to collect some of this information. In designing the FPM test, SSA overcame some, but not all, of the problems experienced with previous tests. For example, SSA was able to persuade states that it believed were nationally representative, on the basis of an analysis of state characteristics, to participate in the test. SSA also decided to track cases through the entire disability claims process, rather than through the initial determination level only. To further ensure a sound test design, SSA hired a consulting firm to independently evaluate the design of the test. While the firm found the test design to be basically sound, it made several suggestions to improve the test and better ensure stakeholder confidence in the validity of the test results. SSA was not able to make all the recommended changes, however. For example, because of state union-management agreements, SSA was unable to obtain data on the qualifications of employees to ensure that test participants were representative of all employees, as recommended by the consultant. In addition, contrary to the consultant’s recommendation, SSA did not mitigate the impact of the 100-percent review of SDM determinations for quality, which may lead to some of the same problems experienced with the SDM stand-alone test. Page 45 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress While SSA has experienced, and continues to face, many difficulties with its redesign effort, the agency can still take actions to increase its chances for future progress. As SSA continues its redesign work, it has an opportunity to apply lessons learned from its 4 years of reengineering experience, as well as from other commonly accepted reengineering and management best practices. SSA has already begun to apply some lessons it has learned, such as strengthening executive oversight of its redesign effort. However, the extraordinary difficulty of the task at hand and the performance shortcomings previously experienced suggest that these steps might not be enough. Other fundamental changes in SSA’s approach will probably be necessary. In particular, although more focused than its original plan, SSA’s current redesign plan is still very large in scope and difficult to manage, and the successful completion of key initiatives will likely require that SSA scale back its near-term efforts even further. SSA can also modify its testing approach to avoid pitfalls encountered in the past. As it moves to implement changes that appeared efficacious in a testing environment, SSA can ensure that it has adequate performance measures and goals to assess changes to the process and to provide early warning of problems as well as adequate quality assurance processes to guard against unanticipated results. The need for SSA to improve its disability claims process continues today. Many Steps Remain SSA’s large pending workload persists, especially at the hearing level. The Under the Revised pending workload at the hearing level grew from 357,564 in 1993 to about Plan 483,712 in 1997. In addition, the average length of time it took to receive a hearing decision upon appeal also grew in the 1990s—from 238 days in 1993 to 386 in 1997. The dramatic growth in initial applications for disability benefits that contributed to these increases and exacerbated long-standing problems has ended. In fact, in recent years the number of individuals applying for disability benefits has declined, which has in turn helped reduce the 1998 appellate backlog to 384,313 and appellate processing time to 371 days. However, no one knows how long this decline will last. An economic downturn could increase unemployment and drive up demand for disability benefits, and the number of applications, at any time. Moreover, the number of applications for disability benefits can be dramatically affected by court cases and changes in the law, such as the possibility of congressional action to increase the retirement age. Finally, SSA expects claims to increase again beginning in 2000, when the eligibility age for full Social Security retirement benefits changes from 65 to 67 years, and more dramatically by 2010, as the baby boom generation approaches its disability-prone years. Taken together, present and future Page 46 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress workloads highlight the continuing pressure on SSA to move expeditiously to improve its disability claims process. Many steps remain to be taken under the agency’s February 1997 redesign plan. As of October 1998, the agency was continuing to test the AO, SDM, and FPM initiatives.27 Should SSA decide to implement any of these positions or process changes, it will face innumerable steps ahead. For example, SSA will need to seek changes in the law or develop new regulations for many of the changes it is considering, a time-consuming and multistep process. For some of the initiatives, such as the SDM, SSA will also need to provide for training, facilities, equipment, and various clerical and managerial supports. In some cases, SSA will need to develop plans for implementing changes in phases, such as installing new positions at a small number of sites each month. In addition, SSA must guard against unwanted effects that could result from making changes to one part of the process without adequately addressing their impact upon other parts. For example, should SSA decide to eliminate the reconsideration step, SSA will need to be aware of the possibility of, and take steps to guard against, the development of more backlogs at OHA caused by the speedier movement of cases through the process to that level. While SSA has made important progress, much remains to be accomplished on two other important near-term initiatives: process unification and quality assurance. For example, under its process unification initiative, SSA intends to review and revise established regulations to develop its planned single presentation of policy—a time-consuming task to which SSA has not yet been able to devote adequate resources. Also under process unification, SSA intends to continue providing systematic ongoing training to adjudicators at all decision-making levels and to continue work on several remaining subinitiatives. Under its quality assurance initiative, SSA still needs to ensure that adequate in-line quality assurance procedures are in place for any changes it makes to the process. SSA is also still trying to reach an agreement on a single “end-of-line” quality review mechanism for the whole disability claims process. Once agreement is reached, SSA will need to test this mechanism. As discussed in chapter 2, developing and testing initiatives can involve a substantial amount of time and effort and require the cooperation of numerous stakeholders. If SSA continues its redesign effort as planned, the agency has even more matters to contend with for its three longer-term initiatives: the DCM; a 27 As discussed in chapter 2, stand-alone testing for the AO has been extended to work on improving upon initial disappointing results; stand-alone testing continues for the SDM as a step toward implementation. In addition, evaluation of FPM test results has not been completed. Page 47 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress simplified methodology for making disability decisions; and the Redesigned Disability System (RDS), SSA’s new disability computer system. All three involve major operational changes and are the furthest from implementation. The DCM combines the duties of SSA field office personnel and state disability examiners and will require legislative changes before it can be implemented. As of October 1998, SSA was still conducting the first of three lengthy test phases that precede full implementation of the DCM.28 The simplified decision methodology initiative is still in the developmental stage, and much more research needs to be accomplished before SSA can begin laboratory testing. Finally, SSA is experiencing a number of problems with its proposed RDS, a system that is viewed as key to many of the planned process efficiencies. In January 1998, we reported that software development problems had delayed the scheduled implementation of RDS by more than 2 years.29 Later in 1998, we reported that SSA had experienced problems and delays in its RDS pilot effort initiated in August 1997 to assess the performance, costs, and benefits of RDS.30 For example, systems officials stated that, using RDS, the reported productivity of claims representatives in the SSA field office dropped. Systems officials also stated that because the RDS software had not performed as anticipated, SSA had engaged an independent contractor to evaluate and recommend options for proceeding with RDS. This effort is expected to further delay SSA’s national rollout of the new disability computer system. See table 3.1 for the steps that have yet to be taken under the revised plan. Table 3.1: Steps Remaining for the Eight Initiatives in SSA’s Initiative Steps remaining February 1997 Plan SDM Complete test and evaluation, prepare cost-benefit analysis, and make go/no go decision (also see FPM). If warranted, prepare a notice of proposed rulemaking (NPRM) and final regulations and conduct phased rollout (including training, facilities, equipment, and other support). To extend SDM to all cases, make legislative changes. AO Complete test and evaluation, prepare cost-benefit analysis, and make go/no go decision (also see FPM). If warranted, prepare NPRM and final regulations; make organizational decisions; and conduct phased rollout, including training, facilities, equipment, and other support. (continued) 28 The three phases are a preparation phase involving classroom and on-the-job training (Nov. 1997 to June 1999); a phase to formally test the concept without the support of RDS and the simplified decision methodology (June 1999 to Oct. 2000); and a preimplementation phase to assess the impact of implementation with RDS and the simplified decision methodology supports (Oct. 2000 to Oct. 2002). 29 Social Security Administration: Software Development Process Improvements Started but Work Remains (GAO/AIMD-98-39, Jan. 28, 1998). 30 Social Security Administration: Technical Performance Challenges Threaten Progress of Modernization (GAO/AIMD-98-136, June 19, 1998). Page 48 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress Initiative Steps remaining FPM For AO, SDM, PDI, and eliminating reconsideration, complete test and evaluation and make go/no go decisions. If warranted, prepare NPRM and final regulations for eliminating reconsideration and conduct phased rollout of PDI (including training and other support) and elimination of reconsideration. Complete test and evaluation for eliminating Appeals Council review. (Also see AO and SDM.) Process unification For single presentation of policy, begin consolidating established policies and regulations. For training, continue to develop and provide ongoing training to support process unification. For four pending subinitiatives: (1) clarify and finalize policies and procedures for weighing treating source opinion; (2) complete testing and implement expanded rationales; (3) draft final rules for weight of DDS medical consultant opinion; and (4) clarify and finalize procedures for assessing residual functional capacity for less than the full range of sedentary work. For two subinitiatives that SSA has to some extent implemented, the following steps remain: (1) continue oversight of new procedure for quality review of hearing decisions and (2) monitor new acquiescence action plan. Quality assurance For in-line QA, complete development and implement in-line QA processes for redesign initiatives. For end-of-line QA, reach agreement on a single standard for an end-of-line review mechanism and test and evaluate review mechanism. If warranted, implement end-of-line quality review mechanism. Simplified decision Complete development and studies of new methodology, conduct methodology lab tests of functional assessment instruments, integrate various research projects into a single paradigm for disability decision-making purposes, and establish “methodology lab” studies to determine the administrative impact of recommendations on the results of disability decisions. If warranted, publish NPRM and final rules and implement new decision methodology. RDS Contingent on results of consultant review, continue piloting and enhancing functionality of RDS to support redesign vision, incorporate disability applications and forms into SSA’s expanded Internet capabilities for use with third parties, identify additional opportunities for electronic medical evidence, and roll out RDS. DCM Complete first phase of testing. Prepare and develop an evaluation plan for second, formal test phase. Make go/no go decision on whether to enter into a third or preimplementation phase of testing with RDS and decision methodology supports. If warranted, incorporate RDS and decision support, prepare final regulations, and make any necessary legislative changes. Source: SSA. Even if SSA successfully tests and implements all of the redesign initiatives included in the February 1997 updated plan, it is unlikely that all of the Page 49 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress problems that gave rise to SSA’s redesign effort in the first place will be satisfactorily resolved. As we have noted, test results to date show only modest improvements in operations, and budgetary savings will not be as large as originally anticipated. Moreover, except for the AO initiative, most of SSA’s redesign efforts to date are focused on improving the process at the initial determination level, leaving problems at the ALJ level largely unresolved. These problems include length of processing times and the large number of backlogged cases at hearing offices, which are among the most pressing problems that SSA faces with the claims process and which require additional solutions. SSA has already taken actions to revise its approach and apply some SSA’s Actions to lessons learned from its early efforts with redesign, including formalizing Revise Its Approach high-level executive oversight, working to improve test design, and Have Been Helpful but rethinking its strategy for communicating with stakeholders. However, these efforts may not be enough to ensure success. Because of the unique Insufficient to Ensure barriers to change inherent in governmental operations, redesign is Success particularly challenging for government agencies, and SSA may need to consider additional changes in its approach to improve its chances of making tangible future progress. In its 1997 plan, SSA established a new management structure to oversee redesign efforts in order to make its senior managers more accountable and involved. Specifically, SSA centralized authority for redesign efforts by creating an Executive Committee for Disability Redesign, chaired by the principal deputy commissioner. Such high-level oversight is critical, given the organizational complexity of the disability claims process. It is also consistent with government and industry best practices, which provide that the individual in charge of a reengineering effort be responsible for the entire process and its performance.31 Strengthening executive oversight has already had a positive effect on the progress of redesign. For example, by promoting timely processing of cases for the FPM test, the Executive Committee has helped to expedite analysis of test results. SSA officials told us they believe that Executive Committee oversight has helped provide a new momentum by working to ensure that activities stay on schedule and that critical policy decisions receive sufficient and early high-level attention. 31 Michael Hammer and Steven A. Stanton, “The Reengineering Revolution,” Government Executive (Sept. 1995), p. 7A. Page 50 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress SSA is also applying some valuable lessons learned from conducting the AO test. Because the AO test results were inconclusive as a result of problems with the design and management of the test, SSA has taken greater care with the design and management of subsequent tests. For example, SSA enlisted the services of an independent consulting firm to review its proposals for both the FPM and DCM tests. Also, for the DCM test, which is similar to the AO test in that it is lengthy and involves testing the efficacy of a new position, SSA is taking steps to ensure test participants receive adequate training and support, and that the testing environment remains stable. SSA officials told us they have also learned a great deal about balancing the need for open communication with stakeholders with the need to keep initiatives on track and make tough and sometimes unpopular decisions. Effective stakeholder communication is an important area according to reengineering experts. Its importance was recently noted in a private sector survey of 102 private and government organizations that found that sending inconsistent signals and not communicating enough with stakeholders were among the five most serious mistakes top management sponsors made during a major change.32 However, communicating with stakeholders is different from obtaining consensus on proposed changes, a practice that can sometimes lead to management paralysis. The proposed changes in SSA’s redesign plan affect most aspects of the disability claims process, and it is unlikely that the agency can achieve across-the-board support from all parties affected by the change. Early on in its redesign efforts, SSA leadership took extraordinary steps to reach out to key stakeholders to build acceptance and consensus for its redesign initiatives. SSA officials told us they now understand that they cannot expect to satisfy all stakeholders and believe they lost valuable momentum early in the redesign effort trying to do so. Agency officials have continued with their efforts to communicate with various stakeholder groups, however, and our review showed that, although stakeholders do not unanimously support all of SSA’s redesign initiatives, many of the stakeholders we contacted were satisfied with the level of communication from SSA. Nevertheless, these positive efforts and lessons learned may prove insufficient for achieving appreciable progress. Even with strengthened executive oversight since February 1997, milestones have continued to slip. Compelling test results and improvements to the disability claims process have also proven elusive. With so much remaining to be accomplished, and many barriers to overcome, SSA will need to take 32 ProSci, Best Practices Report for Managing Change (Loveland, Colo.: ProSci, 1998). Page 51 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress additional steps to keep its redesign effort on track and achieve further improvements to the disability claims process. SSA is not the only government agency that has had trouble reengineering its operations. According to reengineering experts, many federal, state, and local agencies have failed in their reengineering efforts. One reason for this high degree of failure is the unique environment of the government workplace, which adds considerable complexity. For example, the flexibility to reengineer a process is often constrained by laws or regulations that require that processes follow certain procedures—such as the requirement, in some cases, that a physician participate in disability cases involving children or mental impairments. Also, government agencies, unlike their private sector counterparts, cannot choose their customers and stakeholders. Agencies must serve multiple customers and stakeholders, who often have competing interests. In addition, following government procedures, such as drafting and issuing new regulations and complying with civil service rules, makes it difficult to implement changes at the quick pace often considered vital for successful reengineering efforts. Finally, public agencies must also cope with frequent leadership turnover and changes in the public policy agenda. For example, as discussed in chapter 2, SSA faced several policy changes during the last few years, such as the need to redetermine the eligibility of thousands of children receiving SSI benefits, at the same time that the agency was trying to conduct large tests of process changes. According to experts in the field, reengineering requires sharp focus and Sharper Focus Is enormous discipline, and organizations are more likely to succeed if they Needed on Most concentrate their efforts on a small number of initiatives at any given time. Critical Areas One way of focusing a reengineering effort is by prioritizing process improvement objectives and identifying those initiatives most likely to achieve those objectives. Basic reengineering precepts suggest that an agency should decide which process or major subprocess should have highest priority for agency action. This decision should be based on selecting process changes that (1) have strong links to the agency’s mission and would have a high impact on customers, (2) are likely to provide a large return on invested resources, (3) enjoy a strong consensus, (4) are feasible given the available resources and infrastructure, or (5) can be achieved within a short period of time in order to gain experience in reengineering. Page 52 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress SSA’s own experience strongly underscores the need for focus. As discussed in chapter 1, SSA realized early on that it could not effectively manage the large number of initiatives in its original redesign plan within established time frames, and later that scaling back its plan in February 1997 was a step in the right direction. SSA’s experience was not unlike that of others. Early reengineering theory called for large systemwide changes over a short period of time; but experts now suggest that achieving significant change takes longer and costs more than generally believed several years ago. However, SSA has continued to miss milestones and, with much remaining to be accomplished, additional focus may be necessary to achieve significant and concrete improvements to the process. As we reported in December 1996, process unification, quality assurance, and enhanced information systems are among those initiatives most crucial to producing significant improvements in the process. Other initiatives could be explored on a limited basis or undertaken at a later date once progress was ensured for critical initiatives or when additional resources became available.33 Concern over the scope of SSA’s plan and the resources used for redesign activities was similarly expressed by the independent, bipartisan Social Security Advisory Board in an August 1998 report.34 The Board concluded that the costs of the redesign project were significant and could not be sustained indefinitely. The cost of SSA’s redesign efforts is difficult to calculate. According to SSA officials, the agency spent approximately $16.7 million from 1995 through 1998 on redesign activities—mostly on travel associated with relocating test participants around the country, but also on training, rent, supplies, and equipment.35 In addition to these expenditures, the Advisory Board pointed out that the redesign effort consumed the time and attention of a considerable number of the most experienced and knowledgeable staff within both SSA and the DDSs, diverting them from the routine disability claims process. In the context of constrained administrative resources, the Board advised that resources that had been diverted be returned as soon as possible to their usual functions so that SSA and the state agencies could fulfill their basic program responsibilities. 33 GAO/HEHS-97-20, Dec. 20, 1996. 34 Social Security Advisory Board, How SSA’s Disability Programs Can Be Improved (Washington, D.C.: Social Security Advisory Board, Aug. 1998). 35 This amount does not include the salaries of personnel participating in or managing the tests or the cost of developing the new decision methodology. Page 53 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress Prioritizing its key redesign objectives might help SSA to better focus its efforts. As discussed in chapter 1, SSA’s redesign effort currently has five key objectives: allowing claims that should be allowed at the earliest possible level and improving efficiency, speed, and customer and employee satisfaction. However, these objectives can work at cross purposes; an improvement in one area can result in a deterioration of performance in another. For example, focusing on efforts that speed up the process and improve efficiency might reduce the amount of attention given to developing evidence and documenting decisions. This, in turn, might result in incorrect allowances (or denials) earlier in the process. On the other hand, focusing on the objective of making the right decision at the earliest possible level could add time at the initial level, which might result in more accurate initial determinations and fewer appeals, which in turn might improve the speed and efficiency of the overall process. SSA officials told us that if they were to begin again, they would consider dividing the redesign effort into smaller, more manageable segments. This would be one way for the agency to better focus on specific initiatives and perhaps be able to achieve more visible near-term gains. In fact, SSA may end up taking this approach during the implementation phase by rolling out small segments of the redesign plan one at a time. Reengineering best practices, as well as SSA’s own experience to date, A Modified Testing suggest that modifications to SSA’s testing approach could help the agency Approach Could More to more efficiently demonstrate the likely result of proposed changes. Efficiently Provide Conducting smaller and more integrated tests could free up resources to address critical initiatives while effectively demonstrating the efficacy of Valuable Information interrelated changes. In addition, some of SSA’s redesign initiatives face considerable barriers to implementation because they represent significant change, affect jobs, or depend on other changes or supports to be effective. SSA could more effectively explore the viability of such initiatives—as well as of alternative approaches—on a small scale or wait until essential supports have been developed before investing significant resources in testing these initiatives. Small-Scale Testing Is an Many reengineering experts believe that entities undergoing reengineering, Effective Way to Initially such as SSA, should conduct small tests of proposed initiatives. Demonstrate Efficacy Reengineering best practices caution against moving directly from concept to large-scale testing or implementation and suggest that methods such as limited pilot tests and prototyping are cost-effective means for evaluating Page 54 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress the effectiveness and workability of proposed changes. As we recommended in our 1996 report on SSA’s reengineering effort, SSA would benefit from concentrating its efforts on first testing initiatives using a smaller, more manageable scope at only a few sites across the country.36 SSA’s own experience with the AO and SDM tests confirmed that small-scale testing is prudent. Significant resources and time were devoted to large-scale tests of the AO and SDM, only to discover that their efficacy in a stand-alone environment was marginal. The AO test in particular—which lacked good design, disciplined management, and key supports—proved costly and ineffective in proving the AO concept. SSA moved quickly from concept to large-scale testing because it wanted to definitively demonstrate the positive impact of these proposed changes so they could be immediately implemented. But test results did not support immediate implementation. Instead, the outcome has been continual testing that has drained agency resources and energies. In hindsight, SSA could have discovered the marginality of stand-alone initiatives with a much smaller commitment of resources. As noted in chapter 2, SSA officials continue to believe that the agency must conduct tests involving a large number of cases. Given SSA’s desire to collect a sufficiently large amount of data and move quickly to change the claims process, SSA officials believe their approach for the AO and SDM tests was correct, and that if the test results had been positive, all would be well. However, we believe that SSA took a costly risk that may have eroded support for the initiatives. SSA officials have said that in the future they would consider reducing the number of sites that they use in tests by concentrating test sites in a few states or within one SSA region to permit more efficient use of resources and easier test management and oversight. SSA’s current plans involve testing other initiatives, such as the DCM, on a large scale. The DCM test currently under way has a start-up cost of $20 million and involves 210 federal and state participants at 33 sites across the country. Given the uncertainties inherent in this new position, as well as SSA’s past experience with large-scale testing of the AO and SDM initiatives, SSA runs the risk of learning on a large and expensive scale that the DCM does not meet the agency’s redesign objectives. It would be more cost-effective to test this initiative on a small scale and move on to a large-scale test only if initial results suggested the potential for significant gains. In the event of unforeseen difficulties or poor test results, it would 36 GAO/HEHS-97-20, Dec. 20, 1996. Page 55 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress be easier and less costly to make any necessary adjustments to a small-scale test than to a larger one. Integrated Testing Is According to a reengineering expert we consulted, stand-alone testing of Important for Assessing interrelated initiatives is inefficient and unnecessary because it provides the Effectiveness of no synergy or learning across the whole process. In addition, as shown by reengineering research, effectively evaluating the overall impact of a Interrelated Initiatives redesign effort requires studying the entire business unit or process being reengineered. In fact, we recommended in our 1996 report that SSA combine key initiatives into an integrated process and test that process at a few sites. SSA’s experience confirms the importance of integrated testing. Projected benefits from reengineering were predicated on the assumption that most process changes and supporting initiatives would be operational simultaneously.37 However, as discussed in chapter 2, SSA has been testing initiatives independently and without the benefit of some key supports. SSA officials maintain that they have learned a great deal from the large-scale, stand-alone tests, such as how to better run a test. They also maintain that the stand-alone tests provided a baseline of information; for example, testing the SDM in a stand-alone environment provided data to compare with the SDM performance in an integrated environment. SSA officials also believe that the tests contributed to improved communication among operational units and opened the door for important cultural changes needed to support redesign. Although SSA may have learned from its stand-alone tests, these tests did not demonstrate dramatic improvements to the process or provide valuable insight on how the AO and SDM would ultimately work in concert with other initiatives. For example, only when SSA began the FPM test did it become apparent that the SDM might have performed differently if it had been tested in an integrated environment. Rather than conducting large-scale testing of individual initiatives, such as the SDM and AO, moving directly into integrated testing, even on a small scale, or waiting until key supports were in place, might have been more efficient. SSA has the opportunity to apply these lessons in future tests of initiatives. For example, the agency recently began testing expanded rationales —an effort designed to more fully document, at the initial level, the reasons a claim has been denied. These tests have been conducted outside of the FPM 37 According to SSA’s Nov. 1994 plan, most key initiatives were to be operational around 1998, except for the DCM, simplified decision methodology, and some features of RDS, because these initiatives needed much longer lead times. Page 56 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress test, even though expanded rationales are closely related to other process changes in the FPM. Officials are now taking steps to incorporate this feature into the FPM. SSA will be conducting small pilot tests in four states to gather information regarding the impact of expanded rationales when they are added to other FPM process changes. Folding the expanded rationales test into the FPM test will provide more valuable information on the efficacy of this change in the environment in which it was intended to be implemented. Similarly, a new simplified disability decision methodology and computer software support are considered essential to the success of the DCM position. However, since these important support initiatives are not scheduled to be available in time to meet the current schedule for testing the DCM, it is not clear what or how much SSA will learn from this test about the viability or effectiveness of the DCM in a redesigned environment. Alternative Redesign Reengineering best practices suggest that, before selecting a specific Processes Should Be Fully process change for implementation, an organization should develop Explored several possible alternatives to the existing work process and consider the costs and benefits of each. These alternatives should then be explored in order to (1) convincingly demonstrate the potential of each option to achieve the desired performance goals; (2) fully describe the types of technical and organizational changes necessary to support each goal; and (3) if possible, test key assumptions. Also, as part of a cost-benefit analysis, an agency should take into consideration any barriers to and risks in implementing each alternative. SSA might have avoided some of the problems currently being experienced with the AO initiative, which has engendered strong opposition, had other alternative work subprocesses also been explored on a small-scale basis before large-scale AO testing. Alternatives to the AO initiative for improving the appellate process exist, such as SSA’s temporary program to permit senior staff attorneys in hearing offices to allow benefits in clear-cut cases.38 However, SSA did not adequately assess the merits of the alternatives by obtaining concrete and comparable data on their relative costs, benefits, and risks. After 3 years of testing, SSA must decide whether to abandon the AO initiative, begin seriously exploring other solutions to pressing problems at the appellate level, or both. Compounding matters, opponents of the AO concept have pointed to its marginal test results to 38 In 1994, SSA began a plan to reduce the backlog of appealed cases. As part of this plan, SSA established an expanded prehearing conference, at which OHA’s senior staff attorneys were given quasi-judicial powers, or the authority to issue allowance decisions without an ALJ’s involvement or approval. Page 57 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress support their own favored, albeit untested, alternatives. SSA officials agreed that they did not fully prepare themselves for the possibility that their proposed changes might not work and thus did not adequately pursue alternatives earlier in the redesign process or develop contingency plans. SSA may still be able to apply this important lesson in a remaining area by more fully exploring feasible alternatives to the DCM initiative. As with the AO, the DCM initiative is facing some strong opposition and has perhaps even more barriers to full implementation standing in its way. According to one high-level SSA official, test results would have to be very compelling to support implementation of the DCM initiative. Nevertheless, SSA has begun a 3-year large-scale test of the DCM without adequately exploring feasible alternatives. For example, SSA could have—as we recommended in our 1996 report—systematically tested alternatives such as sequential interviewing to compare their relative effects on the process before beginning the large-scale DCM test.39 Instead of testing this concept, SSA allowed the individual operating units to decide whether or not they would adopt this approach. SSA officials believe few, if any, units are actively pursuing it. There is still time for SSA to explore such alternatives to the DCM while the agency conducts its protracted test. As of October 1998, SSA was considering widespread implementation of Implementation Will several changes to the disability claims process on the basis of some Require Careful promising results from its FPM test. While SSA has encountered Monitoring considerable challenges in testing its initiatives, the risk of further difficulty during their implementation is very high. The experience of other public and private organizations that have attempted business process reengineering strongly indicates that, when compared with developing or testing possible changes to a process, implementation of those changes is more difficult. Moreover, it is possible that certain process changes may not perform as expected outside the test environment. SSA, therefore, needs adequate performance goals and measures for key initiatives and objectives to monitor and assess the impact of any changes made to the process. SSA also needs an adequate quality assurance process in place to ensure the quality and accuracy of decisions. 39 Under sequential interviewing, after SSA field office personnel obtained the necessary nonmedical information from the claimant, they would refer the claimant to disability examiners, who would complete the medical portion of the application, often during the same interview. Claimants would be able to talk to both parties that worked on their case, and time lost in hand-offs between the two units could be minimized. Page 58 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress Implementing a New Experience has shown that implementation of a new process is extremely Process Is Difficult difficult and, compared with development and testing, is the most failure-prone phase of a reengineering effort. During implementation, an organization’s natural resistance to change must be overcome. According to a reengineering expert we consulted, many reengineering efforts fail because too little time and effort are allotted to implementation. The numerous issues that need to be considered and planned for include • identifying all tasks, time frames, and needed resources for an orderly transition; • structuring the rollout of the new process in a way reasonably suited to the nature of the process and the work and structure of the organization; • assigning roles and responsibilities for implementation to the individuals who will do the work of the new process; • providing a means for collecting and sharing information on implementation problems and solutions; and • providing for close monitoring during implementation. SSA’s implementation plans issued in 1994 and 1997 do not address many of the above considerations. For example, the plans do not address the key roles, responsibilities, and reporting relationships required by the new process. In our discussions with stakeholders, we found increasing anxiety over the fact that some key organizational decisions related to work space, which unit would be responsible for managing the proposed AO positions, and other infrastructure issues had not yet been made. Nor do the implementation plans address how SSA will monitor the process to ensure successful implementation and optimum improvements. Recognizing that its current implementation plan is lacking in many specifics, SSA plans to develop more detailed implementation plans as key decisions are made. Adequate Performance In order to be able to effectively monitor the results of its process changes Goals and Measures Are during implementation, SSA will need adequate performance goals and Needed to Closely Monitor measures. Researchers for the Harvard Business Review found that failure to measure a new process can be particularly damaging to a reengineering Results of Process effort because, without a comprehensive measurement system that can Changes track the new process’ performance, it is impossible to tell if implementation is succeeding or failing.40 A National Academy of Public Administration report similarly found that measuring and tracking performance continuously was one of six critical success factors in 40 Gene Hall, Jim Rosenthal, and Judy Wade, “How to Make Reengineering Really Work,” Harvard Business Review (Nov.-Dec. 1993), p. 129. Page 59 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress reengineering in the government sector.41 The report cites performance management as a key characteristic in successful organizations because it offers the only way for them to assess whether or not reengineering is achieving the results they desire. SSA currently collects a large amount of data related to the disability claims process, but these data could be improved or better tracked for the purpose of determining progress toward redesign goals. Key indicators that SSA uses or could use to measure progress are fragmented, incomplete, or entirely missing. For example, for its agencywide performance plan, SSA is using separate performance measures for disability claims processing times at the initial and appeal levels. This fragmented approach ignores the interrelationship between the two levels; that is, reducing processing time at the initial level might result in premature or poor determinations; cause more cases to be appealed; and, thus, cause overall processing times to increase. Conversely, implementing steps that result in a longer initial processing time but also permit earlier correct allowances could shorten the overall average processing time by reducing appeals. In addition, although SSA has said that process unification is the “cornerstone” in the foundation of the redesigned disability claims process, SSA’s performance plan does not contain a goal for this important initiative; rather, SSA continues to measure performance in a disjointed manner. SSA is collecting some appropriate data for its tests but still needs to make sure they are linked to the agency’s strategic goals and integrated into the agencywide performance measurement system. As stressed by the Chief Financial Officers Council, an organization composed of representatives of federal departments and agencies, government entities should integrate all reform activities, including reengineering, into the framework of the Government Performance and Results Act (commonly known as the Results Act). According to the Council, one of the reasons this is important is to ensure consistency and reduce duplication of effort.42 Our review of SSA’s fiscal year 1999 performance plan pointed out that SSA’s reengineering effort is not fully integrated into its Annual Performance Plan.43 Although the Plan noted SSA’s efforts to improve the disability 41 Sharon L. Caudle, Reengineering for Results: Keys to Success From Government Experience (Washington, D.C.: National Academy of Public Administration, 1995), p. 53. 42 Executive Guide: Effectively Implementing the Government Performance and Results Act (GAO/GGD-96-118, June 1996). 43 The Results Act: Observations on the Social Security Administration’s Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R, June 9, 1998). Page 60 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress claims process, the Plan did not include any useful discussion of SSA’s major initiative to completely redesign its disability claims process, nor did it indicate whether changes or improvements expected to result from this effort were factored into the performance measures or goals. Guarding Against SSA cannot be certain that its initiatives will perform the same under “real Unintended Results Is world” conditions as they did in an artificial test environment, and the Important agency will need to take additional steps to guard against the possibility of unintended results. For example, SSA’s test of the SDM included a quality review of all cases decided under the test, whereas currently, far fewer cases, most of which involve allowance determinations, are reviewed. In the absence of this 100-percent review, the SDM might perform differently, which could have a significant effect on the accuracy of determinations, the number of allowances and appeals, and overall benefit outlays. Possible unintended results could include inaccurate disability determinations, unanticipated increases in benefit outlays, and increased appellate workloads. When test results are marginal, there is a greater chance that expected process improvements might not materialize. SSA needs to be sure that, when implementing a change in the process such as the SDM, an adequate quality assurance process is in place to ensure that benefit eligibility decisions are accurate. Accuracy is important because incorrect decisions can result in wrongful benefit payments, unnecessary appeals, or hardship to the claimants caused by incorrect denials. Under its quality assurance initiative, SSA is seeking to build quality into the decision-making process using tools such as training, mentoring, peer review, and feedback. SSA has been exploring approaches to in-line quality assurance as part of its SDM phase II test, allowing individual test sites to set up their own processes. During implementation of the SDM, and in the absence of a uniform approach, SSA will need to take steps to ensure that individual state processes are sufficient to maintain quality. Ultimately, SSA will need to establish a final quality assurance process that will both identify systemic problems with case decisions and measure the success of SSA’s efforts to build quality into the process. As discussed in chapter 1, current reviews of DDS determinations and ALJ decisions are conducted in isolation from each other. SSA has recently instituted a review of ALJ decisions that will help identify inconsistencies in decision-making between the two levels. However, SSA has yet to develop a single quality review mechanism applicable to both levels. SSA has had particular difficulty getting its initial and appellate decision-making levels Page 61 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 3 Actions Needed to Improve Chances for Future Progress to agree on a consistent quality assurance process that cuts across all phases of the decision-making process, including reaching agreement on what constitutes a correct decision. Page 62 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 4 Conclusions, Recommendations, and Agency Comments and Our Evaluation More than 4 years after releasing its original redesign blueprint, SSA is still Conclusions struggling to make significant improvements to its disability claims process. While the agency has made some progress with process unification, SSA has missed many of its redesign milestones, and the results of early tests did not support implementation of specific proposed changes. The agency is still conducting a number of tests, including yet another large, nonintegrated test at numerous sites. Also, top agency officials would like to begin making some implementation decisions about new decisionmaker positions and other proposed changes. With so much left to do, SSA still has a window of opportunity, which will not be open for long, to apply some lessons learned to help the agency achieve important improvements to its disability claims process. (SSA is no longer experiencing a dramatic growth in applications for disability benefits, but the agency can expect applications to increase again as the baby boom generation ages or if the economy suffers a downturn.) SSA’s ability to learn from past experience will be an important ingredient in the success of future efforts. For example, the size of SSA’s tests and the scope of redesign initiatives slowed SSA’s progress under its original 1994 redesign plan. When the agency revised its redesign plan in 1997 to include fewer initiatives and increased executive oversight, similar problems continued to limit progress. Even this revised plan required the agency to move forward on a number of varied fronts simultaneously, and SSA continued to miss key milestones. Again, the agency may have underestimated the challenges of managing stakeholder input and keeping such an ambitious effort on course. Strong project oversight should continue, but it will probably not be enough to ensure timely progress. Therefore, SSA needs to further focus its efforts by prioritizing its objectives and concentrating its resources on the efforts most likely to achieve those objectives. Such efforts should include those that help to improve consistency in decision-making, ensure accurate results, and achieve large efficiencies through the use of technology. Past experience has shown that a large-scale test of an individual initiative, while providing an abundance of information on how well that initiative performs in isolation from other changes, does not clearly demonstrate how the initiative would function in a redesigned process and is not the most efficient and effective use of resources. Moreover, while SSA hoped that this testing approach would help gain the support of key stakeholders likely to be affected by the changes, it has not done so. To help free up resources and effectively demonstrate the efficacy of proposed changes, SSA should conduct relatively small tests that integrate several of the Page 63 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 4 Conclusions, Recommendations, and Agency Comments and Our Evaluation proposed changes to the process. Smaller tests will allow SSA to more efficiently identify promising concepts before moving to larger-scale testing or implementation. Integrated testing—testing related concepts together and with key supports in place—will help SSA to demonstrate whether proposed changes will perform as intended under the new process. SSA’sexperience with the AO test has also shown the risks inherent in devoting considerable time and resources to a single unproven approach or change. Results of AO tests have been consistently disappointing, and SSA now finds itself faced with the same long-standing problems the AO was intended to remedy without a tested alternative solution. Therefore, in the future, before investing significant time and resources on any initiative, SSA should explore feasible alternatives for changing the process on a small scale. For example, as we have recommended before, SSA should explore sequential interviewing as a feasible and less risky alternative to the controversial DCM position. Exploring alternatives and conducting small, integrated tests of related initiatives before making large investments are sound reengineering and management practices, the wisdom of which has been underscored by SSA’s experience to date. Since other organizations have found implementation of process changes to be the most failure-prone phase of a redesign effort, SSA is also likely to encounter numerous pitfalls as it attempts to effect process changes in such a complex environment. As a result, it is especially important for SSA to take action to closely monitor the results of changes it makes to the process and watch for early warnings of problems. It is possible that process changes may not operate as expected outside the test environment. It is also possible that some stakeholders who do not support specific changes may act to undermine their success. If process changes do not operate as expected, the results could include inaccurate decisions, unanticipated program costs, increased appellate workloads, and lack of improvement in service to the claimant. Therefore, SSA should immediately establish a comprehensive set of performance goals and measures—a set that cuts across the whole process and is also linked to SSA’s overall strategic and performance plans—in order to assess and monitor the results of changes to the process. Finally, SSA’s tests of process changes have provided only limited assurance that these changes would not degrade the quality of disability decisions. Specifically, SSA’s tests included artificial steps, such as a quality review of all test cases, that are not likely to be used outside the test Page 64 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 4 Conclusions, Recommendations, and Agency Comments and Our Evaluation environment. Quality is perhaps the most critical aspect of the decision-making process because each inappropriate disability decision does a disservice to claimants, taxpayers, or both. A wrongful denial burdens the claimant and could result in unnecessary administrative costs if the claimant appeals the decision, whereas a wrongful allowance results in a continuous stream of inappropriate benefit payments. Therefore, as changes are made to the process, SSA should ensure that it has a quality assurance process in place that both promotes and monitors the quality of disability decisions. As SSA proceeds with further exploration and testing of redesign initiatives Recommendations and considers implementation options, it should take the following steps to improve the likelihood of making key improvements to the disability claims process: • further focus resources on those initiatives, such as process unification, quality assurance, and computer support systems, that offer the greatest potential for achieving SSA’s most critical redesign objectives; • test promising concepts at a few sites in an integrated fashion; • establish key supports and explore feasible alternatives before committing significant resources toward the testing of specific initiatives, such as the DCM; • develop a comprehensive set of performance goals and measures to assess and monitor changes in the disability claims process; and • ensure that quality assurance processes are in place that both monitor and promote the quality of disability decisions. SSA mostly agreed with our report’s observations and the thrust of its Agency Comments recommendations. Specifically, SSA agreed that the tests conducted took and Our Evaluation longer than anticipated and did not result in the budgetary and operational efficiencies originally hoped for in the 1994 redesign plan. SSA also agreed that it should focus on those areas that will make the greatest contributions to improving the quality and timeliness of decisions. As we have recommended, SSA intends to pursue additional process unification and quality assurance activities. The agency also indicated it will pursue elements of the FPM that will significantly improve customer service. Finally, SSA agreed that systems technology must continue to be an important focus of resources. Page 65 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 4 Conclusions, Recommendations, and Agency Comments and Our Evaluation SSA took issue with our critique of its testing strategy. SSA believes that stand-alone and FPM testing were both needed to gather data and experience that are essential for making responsible decisions. Moreover, SSA believes that testing at fewer sites would not have provided the required information or allowed the agency to complete the tests in less time. While we understand the agency’s desire to conduct large tests in order to obtain statistically valid results, we continue to believe that exploring the efficacy of initiatives initially on a smaller scale before moving to large-scale testing or implementation would result in a better use of resources. Also, because the various initiatives are interdependent, we believe that integrated testing would provide more complete and useful information on how the initiatives will perform in the new process. SSA also stated that its current approach to testing the DCM is consistent with our concerns and recommendations, in that it recognizes and builds upon what SSA has learned from previous testing experiences. However, we still have reservations about SSA’s current approach to testing the DCM. First, SSA continues to test this new position on a rather large scale without having explored the position’s potential efficacy through prototyping or limited pilot testing. Second, SSA is testing this initiative without the benefit of the key supports (such as a new simplified disability decision methodology and computer software support) upon which its efficacy relies. Finally, SSA is moving forward with the DCM test without having explored the feasibility of alternative approaches. While agreeing to focus on certain key initiatives, SSA believes that changes to the decision-making process should precede major computer system changes to enable technological developments to be crafted in the manner most supportive of the new process. Similarly, SSA stated that changes to the decision-making process should precede the development of a new quality assurance process, the purpose of which will be to evaluate the quality of the new process. However, we believe that SSA can make substantial progress toward developing these critical supports before finalizing the process changes. For example, certain key aspects of SSA’s quality assurance initiative—such as ensuring the consistent application of policy across all levels of the process and developing agreement on what constitutes a correct decision—need not rely on final changes to the process of making a decision. Finally, SSA pointed out, and we agree, that the agency’s monitoring and evaluation systems currently capture a significant amount of data related to the disability claims process. However, as our report indicates, these Page 66 GAO/HEHS-99-25 Disability Claims Process Redesign Chapter 4 Conclusions, Recommendations, and Agency Comments and Our Evaluation data are not always translated into comprehensive and complete performance goals and measures that look at the efficiency and effectiveness of the process as a whole. As we asserted in our report, SSA’s use of separate performance measures for disability claims processing times at the initial and appeals levels in its agencywide performance plan ignores the interrelationship between the two levels, thereby reducing the usefulness of the performance measures. We also noted the lack of integration of SSA’s redesign objectives with those found in the agencywide performance plan. We believe SSA can do more to make better use of the large amount of data it collects through a carefully crafted set of performance goals and measures. Page 67 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules Five of the eight key initiatives in the Social Security Administration’s (SSA) February 1997 plan had near-term milestones; that is, those initiatives were to be tested or implemented before fiscal year 1999. At the time the February 1997 plan was issued, SSA had already begun testing two of the five initiatives—the adjudication officer (AO) and single decision maker (SDM) positions—and planned to begin implementing them in fiscal year 1998. Another initiative—the full process model (FPM)—involved a more complicated and lengthy test of several proposed changes to the process; SSA expected to assess and implement some of these changes earlier than others. The remaining two near-term initiatives—process unification and quality assurance—involve a number of actions aimed at improving the consistency and quality of decisions at all stages of the process. SSA expected to complete many planned activities under these two initiatives in time to support the other proposed changes to the process as they were implemented. The AO is a new position created both to facilitate the appeals process and Adjudication Officer to serve as the primary focal point for all prehearing activities when a hearing request is filed. For appealed cases, the AO would be responsible for (1) providing the claimants, claimants’ representatives, or both with an in-depth understanding of the hearing process; (2) providing unrepresented claimants with information about the hearing and their right to representation; (3) developing the record for a hearing (that is, obtaining additional evidence as warranted); (4) issuing fully favorable on-the-record decisions when supported by the evidence; and (5) narrowing issues and fully developing cases that are forwarded to the ALJ for hearing. According to the February 1997 plan, SSA planned to test the AO both in a stand-alone fashion, without system supports such as the new computer system in place, and, later, as part of the FPM test, which combined several redesign features into a single test. The AO was the first initiative to be tested because SSA felt that the AO position, if successfully tested and implemented, would help reduce the backlogs and lengthy processing times at the appellate level. Initially, the purpose of the stand-alone test was to quickly demonstrate the viability of the concept. However, SSA increased the number of test cases and added a control group for comparison purposes after the Office of Management and Budget raised concerns regarding the impact of process changes on benefit outlays. Page 68 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules The objectives for the stand-alone test were to determine whether the AO was an effective and cost-efficient method for (1) improving case processing time, (2) accelerating case development, and (3) increasing the ability of administrative law judges (ALJ) to adjudicate claims that require a hearing. In addition, testing was expected to demonstrate that the AO would not adversely affect benefit costs. Specifically, the test would determine the AO’s impact on benefit costs (measured by allowance and certification rates), decisional accuracy, productivity (measured by task time), customer service (measured by processing time), and employee reaction to the new process. The stand-alone AO test began in November 1995. Test sites included nine state disability determination service (DDS) sites and 16 federal sites spread among Office of Hearings and Appeals (OHA) hearing offices, SSA field offices, and processing centers. However, SSA was not satisfied with early test results and decided in February 1997 to postpone implementation while it refined and extended the test. Test refinements, designed to improve efficiency, processing time, and quality, included strengthening site management, improving clerical and computer support, and providing additional feedback and training to AOs. Refinements were to be accomplished within 120 days and to be followed by a second monitoring phase. According to the February 1997 plan, SSA expected to complete its evaluation of test data, make a decision for rollout, and develop and publish regulations for implementing the AO nationwide in fiscal year 1997. SSA planned to begin nationwide implementation in January 1998 and to complete the implementation process in fiscal year 1999. The SDM position was designed to make the initial disability determination Single Decision Maker process more effective and efficient by allowing the disability examiner to be solely responsible for making the determination. Currently, disability examiners and DDS physicians are jointly responsible for making the disability determination, and DDS physicians are required to sign disability determination forms and to certify all determinations. Under the SDM model, physicians would function as true consultants, providing information and advice only on cases referred to them by the SDM. Under its February 1997 plan, SSA planned to test the SDM position in both a stand-alone environment (without other, related process changes included in the test) and as part of the FPM (which would integrate several redesign Page 69 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules features into a single test). The SDM test was conducted early in SSA’s redesign effort, and in a stand-alone fashion, because SSA felt that with the SDM in place SSA could make some quick improvements to the process and thus build momentum and support for the overall redesign effort. SSA planned to test the SDM in a stand-alone fashion in two phases. The first phase would test the viability of the concept—that is, whether the SDM could improve productivity by reducing processing time and the cost of processing cases while maintaining the current level of accuracy in making disability determinations. More specifically, SSA would determine the impact of the SDM on allowance rates, decisional accuracy, processing time, productivity, and administrative costs. SSA planned to conduct this test in eight states in a tightly controlled test environment (that is, with 100-percent quality review of the test cases). As specified in the February 1997 plan, testing began in May 1996, and test evaluation was to be completed in fiscal year 1997. A decision to implement the SDM would be based on the results of this test. The second phase would test approaches to implementation under more “real world” circumstances, such as with lower levels of quality assurance review. Specifically, SSA was looking for approaches to implementation that would ensure the highest quality of customer service in terms of accuracy, productivity, processing time, and costs. In the February 1997 plan, the second phase was scheduled to begin in May 1997 and, according to other planning documents, to continue for at least 12 months. Contingent on positive test results, the February 1997 plan called for implementation of the SDM position by the close of fiscal year 1998. The SDM was not expected to handle all cases initially, because statutes and regulations currently require a DDS physician to review denials of claims involving mental impairments or children. SSA planned to pursue legislative changes in fiscal year 1999 that would permit the SDM to handle such cases and to fully implement the SDM position in fiscal year 2000. The FPM is not a single initiative but a combination of several redesign Full Process Model features to be tested as an integrated process. As noted by SSA, most, if not all, of the proposed process changes are interdependent, and this test was designed to assess the impact of combined changes. The redesign features included in the FPM test are (1) the SDM position, (2) a predecision interview by the SDM of the claimants in cases in which the evidence obtained does not support a fully favorable determination, (3) the Page 70 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules elimination of the reconsideration step, (4) the AO position, and (5) the elimination of the Appeals Council review step. The objective of the FPM test was to determine whether, or the extent to which, these redesign features could together produce an accurate and appropriately documented disability determination more efficiently than the current process. More specifically, the test would evaluate the impact of the combined features on allowance rates, decisional accuracy, appeal time, processing time, productivity, administrative costs, and employee satisfaction. The evaluation would also compare the performance of the SDM and the AO in an integrated test versus a stand-alone test. SSA’s February 1997 plan called for the FPM test to be conducted at 11 DDS sites in eight states and at two federal sites. SSA planned to complete its selection of 30,000 test cases for the FPM test by January 1998. Also in fiscal year 1998, SSA expected to complete its assessment of all redesign features except the elimination of the Appeals Council review step and to decide whether to institute these four features if test results supported these changes. In fiscal year 1999, SSA planned to implement the predecision interview and eliminate the reconsideration step. Also in fiscal year 1999, SSA expected to complete its evaluation of eliminating the Appeals Council step—which would take much longer because cases need time to get to and through the appellate process. SSA did not expect to fully implement the elimination of the Appeals Council review step until fiscal year 2001. The objective of this initiative was to achieve similar results on similar Process Unification cases at all stages of the process, through the consistent application of laws, regulations, and rulings. Through this initiative, SSA hoped to address long-standing inconsistencies in decision-making between the DDS and ALJ levels that have contributed to high rates of appeal and allowances at the hearing level. High appeal and ALJ allowance rates have, in turn, increased case backlogs and processing times at the hearing level. In addition, ALJ allowances have been used in class action law suits as an indication of DDS errors in initial determinations. Process unification is considered by many in SSA to be the most critical of SSA’s redesign initiatives. In SSA’s early redesign plan, the process unification initiative consisted of developing a single presentation of policy to replace the many vehicles used by SSA to convey policy. After further review of the problem, SSA expanded the scope of this initiative to include developing Social Security Page 71 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules Rulings (SSR)44 to clarify policy, providing training, and taking action on eight subinitiatives that address key differences between the DDS and ALJ decision-making processes. The objectives and planned activities for these process unification efforts follow. Single Presentation of Currently, different vehicles exist for conveying policy and procedural Policy guidance to decisionmakers at the different levels of the process. The objective of this effort is to develop a single presentation of policy that is binding on all decisionmakers and helps to ensure consistent application of policy. In July 1996, SSA began issuing all policy instructions in the exact same language to all adjudicators. SSA’s next steps involve addressing differences in policy instructions issued before July 1996. Under its February 1997 plan, SSA intended to develop a plan and procedures for combining existing adjudicative policy documents into a “single book.” According to one SSA official, integrating policy documents will be a lengthy process because SSA will want to review and revise some of its policies before integrating them. Social Security Rulings The objectives of this effort are to clarify policy in areas contributing to inconsistent decision-making and to support planned process unification training. In July 1996, in preparation for its first training initiative, SSA issued nine SSRs. SSA believes these SSRs address the most significant adjudicative issues: • applying federal circuit court and district court decisions, • clarifying when adjudicators must give the opinion of a treating physician controlling weight, • considering allegations of pain and other symptoms in determining whether an impairment is severe, • evaluating symptoms, • considering medical source’s opinions on issues that are the purview of only SSA, • considering DDS physicians’ findings of fact at the ALJ and Appeals Council levels, • assessing the credibility of an individual’s statements about pain or other symptoms, 44 SSRs are precedential court decisions and policy statements or interpretations that SSA has adopted as binding policy. SSRs are published under the authority of the commissioner of Social Security and are binding on all components of SSA. SSRs do not have the effect of law or regulations, and they may be based on case decisions made at any administrative level of adjudication, federal court decisions, commissioner’s decisions, opinions of the Office of the General Counsel, and other policy interpretations of the law and regulations. Page 72 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules • assessing the residual functional capacity (RFC) for initial claims;45 and • explaining the impact of an RFC assessment of less than the full range of sedentary work on an individual’s ability to do other work. Training The objective of this effort is to provide consistent training to adjudicators at all levels of the process in three of the most complex disability areas—assessment of symptoms, treatment of opinion evidence, and assessment of RFC—in order to help ensure consistent application of policy. At the time SSA issued its February 1997 plan, SSA had already begun training 15,000 disability adjudicators (including DDS examiners, physicians, ALJs, and quality assurance staff). SSA planned to conduct ongoing process unification training. Eight Subinitiatives SSA approved a second series of recommendations for process unification in April 1996 and a set of action plans for implementing these recommendations in July 1996. These recommendations involved eight subinitiatives. According to its February 1997 plan, SSA expected to implement the subinitiatives in fiscal year 1997. Although the February 1997 plan did not cite specific activities for these subinitiatives, some planned activities were specified in other planning documents.46 The purpose of these subinitiatives and a description of any planned activities relating to them are provided below. Subinitiative 1: Treating The objective of this subinitiative is to ensure that, in rendering a decision, Physicians’ Opinions adjudicators give appropriate weight to the opinions of the claimants’ treating physicians. Under SSA regulations, adjudicators must consider the opinions of treating physicians and must give controlling weight to those opinions regarding the nature and severity of the claimant’s impairment(s), provided that the opinions are well-supported by medically acceptable clinical and laboratory diagnostic techniques and not inconsistent with other substantial evidence in the record. However, an SSA study found that DDS adjudicators and ALJs respond differently to opinions of claimants’ physicians: ALJs appear to give more weight to the opinions of the treating physicians than do DDS adjudicators, who are more likely to assess these opinions in conjunction with other medical evidence in the file. 45 When medical evidence does not show that an applicant’s condition meets or equals the severity criteria in SSA’s Listing of Impairments, adjudicators assess the applicant’s RFC to determine whether he or she can perform past relevant or other work. 46 These planning documents are the July 1996 action plans and SSA’s February 1997 tactical plan. Page 73 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules As discussed above, SSA issued a ruling in July 1996 clarifying its policy on assigning appropriate weight to the opinions of treating physicians. In its planning documents, SSA indicated that it intended to develop other options for achieving the goal of this initiative. Subinitiative 2: Complete The purpose of subinitiative 2 is to more completely document subjective Documentation in DDS Denials and objective evidence in cases in which a DDS denies a claim. SSA requires DDS physicians to record explanations of their reasoning in assessing a claimant’s RFC. However, SSA has found that such explanations often are not well-documented. Improved explanations could be more useful to ALJ decision-making, especially since it is SSA’s policy that ALJs consider the opinion of the DDS physician in making an RFC assessment. To address subinitiative 2, SSA intended to prepare a program circular on documentation requirements and develop a revised quality assurance process to enforce those requirements by October 1996. Subinitiative 3: More Detailed The purpose of subinitiative 3 is to require more detailed explanations in Explanations in all reconsideration denials—that is, more detail on how the determination Reconsideration Denials was made, especially addressing issues of treating physicians’ opinions, subjective symptoms (pain), and RFC. As with initiative 2, improved explanations of reconsideration denials could be more useful to ALJ decision-making. For subinitiative 3, SSA intended to provide training and implement new instructions on providing more detailed explanations of reconsideration denials by October 1996. Upon implementing this initiative, SSA also planned to evaluate these changes and their implications for other redesign initiatives. Subinitiative 4: Remand The purpose of this subinitiative is to return or remand selected appealed Selected Hearing Cases cases to the DDS for consideration of new evidence introduced at ALJ hearings. One SSA study indicated that 76 percent of appealed cases contain new evidence—generally medical in nature—and that 27 percent of hearing awards hinge on the additional evidence. For cases in which the DDS determined that benefits were indeed warranted, remanding cases with new evidence would avoid the need for a more costly and time-consuming ALJ decision. For cases in which the DDS did not believe an award was justified, the DDS physician would provide a revised assessment of the case’s medical facts and return the case to the ALJ for hearing. Page 74 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules In its February 1997 tactical plan, SSA expected to return 100,000 cases—out of roughly 500,000 appealed cases per year—to the DDS for consideration of new evidence. Subinitiative 5: Weight of the The purpose of this subinitiative is to issue guidelines defining the specific DDS Physicians’ Opinions weight to be given by ALJs to the opinions or conclusions of the DDS physicians. SSA policy already requires ALJs to consider the opinions of DDS physicians when making RFC assessments. However, perceptions exist that ALJs have not sufficiently done so. SSA issued a ruling in July 1996 clarifying that ALJs must consider the findings of fact made by DDS physicians as expert opinion evidence. In its planning documents, SSA indicated that it intended to issue final regulations by April 1997 clarifying the weight to be given by ALJs to the DDS physicians’ opinions. SSA also intended to issue implementing instructions and conduct training on this subject by April 1997. Subinitiative 6: RFC for Less The objective of this subinitiative is to clarify regulatory language Than Sedentary Work regarding assessing a claimant’s RFC in cases in which the claimant is found capable of less than a full range of sedentary work. SSA has determined that differing DDS and ALJ assessments of a claimant’s capacity to function in the workplace are the primary reason for most ALJ awards. For example, for ALJ awards involving physical impairments, ALJs were significantly more likely than DDS physicians to find that applicants had very limited work capacity—that is, they could do “less than the full range of sedentary work.”47 Moreover, according to one study, ALJs were likely to arrive at different conclusions in this regard than DDS adjudicators even when presented with the same evidence. SSA issued a ruling in July 1996 clarifying SSA’s policies regarding RFCs of less than a full range of sedentary work. In its planning document, SSA stated that it intended to publish final regulations and issue operating instructions on handling RFCs of less than the full range of sedentary work by April 1997. Subinitiative 7: Acquiescence The objective of this subinitiative is to issue an SSR clarifying SSA’s Policy acquiescence policy. The Social Security Act provides procedures for claimants to appeal a final SSA decision to the federal district court level. In some cases, the circuit court decision may conflict with SSA’s interpretation of the act. In such cases, if SSA does not seek further court 47 From Sept. 1992 through Apr. 1995, two-thirds of ALJ awards involved cases that merited a “less than the full range of sedentary work” assessment—a classification that often leads to an award. Page 75 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules review or is unsuccessful on further appellate review, SSA must acquiesce to the decision of the circuit court. This can result in inconsistency in policy across the circuits if SSA does not take specific and timely steps to convey the new policy uniformly to all SSA components. SSA published an SSR in July 1996 clarifying its policy on acquiescence. In its planning documents, SSA indicated that it intended to develop an action plan for accelerated preparation and clearance of acquiescence rulings by October 1996. Subinitiative 8: Quality Review The objective of this subinitiative is to implement a quality review of ALJ of Hearing Decisions allowances under the Appeals Council’s own motion review authority. The current quality review mechanism for ALJs consists largely of Appeals Council reviews of denials at the request of claimants, which some argue causes a propensity for ALJs to allow cases. Moreover, quality reviews for the DDS and ALJ decision-making processes are conducted in isolation from each other, and there are no procedures in place for reconciling the differences between the processes. For example, DDS adjudicators do not receive feedback on why cases they denied were subsequently overturned at the appellate level. The new quality assurance process aims to provide more balance in reviewing ALJ awards and denials and to identify and reconcile factors that contribute to differences between DDS and ALJ decisions. In its planning documents, SSA stated that it intended to implement a process whereby disability examiners and physicians with SSA’s Office of Program Integrity Review48 would perform pre-effectuation reviews of ALJ allowances and forward cases they felt were unsupported to the Appeals Council. The Appeals Council would then, using its own motion review authority, review the case and decide whether to reverse the decision or remand the case to the ALJ. For cases in which the Appeals Council disagreed with the conclusions of the quality reviewers, the award would be allowed, but the case would be referred to an intercomponent panel that considers the need to clarify policies, issue new policies, or provide additional training. In its July 1996 action plan, SSA stated that it intended to implement the quality assurance process, including conducting training, by October 1996. Evaluation of this process to determine, for example, the need for regulatory change and implications for redesign activities, would be ongoing. 48 In 1998, the Office of Program Integrity Review changed its title to the Office of Quality Assurance and Performance Assessment, also called the Office of Quality Assurance. Page 76 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix I Descriptions of Near-Term Redesign Initiatives and Their Test and Implementation Schedules The purpose of this initiative is twofold: (1) to develop in-line quality Quality Assurance assurance processes (approaches designed to build quality into the process) and (2) to develop a single, end-of-line review mechanism that looks at the whole adjudicative process. In-line quality assurance processes would include procedures and tools—such as regular training, mentoring, peer review, and real-time feedback—to ensure the right decision is made the first time. The end-of-line quality assurance process would include a final review and feedback mechanism that employed the same standard of review for all cases and would become SSA’s primary vehicle for monitoring and measuring the success of each component’s front-line quality assurance process. For in-line quality assurance, SSA expected to develop and incorporate processes as part of its major redesign initiatives, including the AO position, the SDM position, other FPM concepts, process unification, and the disability claims manager position. Specifically, procedures and tools would be developed during testing of these initiatives. For example, the February 1997 plan called for quality assurance principles to be built into the SDM process during the second phase of SDM testing, beginning in May 1997. Participating test sites were asked to develop quality assurance programs for safeguarding the accuracy of decisions. In addition, SSA expected to continue development of quality assurance processes to support the other FPM concepts—which include the AO, predecision interview, and elimination of reconsideration—in fiscal year 1998. Finally, SSA planned to develop a strategy for fully deploying initiatives, and their quality assurance processes, in fiscal year 1999. For end-of-line quality assurance, according to its February 1997 plan, SSA expected to develop a prototype final review and feedback mechanism in fiscal year 1997, test this mechanism in fiscal year 1998, and evaluate and deploy it in fiscal year 1999. Page 77 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix II SSA’s Efforts to Meet Near-Term Milestones SSA was not able to meet many of its near-term milestones for testing and implementing key initiatives in its February 1997 plan. SSA had hoped to begin implementing both the AO and SDM positions in fiscal year 1998 but postponed the decision of whether or not to implement until fiscal year 1999. SSA also is not as far along with evaluating the results of its FPM test as it had earlier hoped, and further implementation delays are likely. SSA has taken some concrete steps on its process unification initiative, but most of the activities occurred behind schedule, and many have yet to be completed. SSA also missed most of its milestones under its quality assurance initiative. SSA missed milestones in its February 1997 plan for implementing the AO Implementation position. SSA planned to complete its stand-alone AO test—its test of the AO Delays for AO and position excluding other redesign changes—in fiscal year 1997. If test SDM Positions results warranted, SSA had planned to begin nationwide rollout of the new position in January 1998. However, as of October 1998, SSA was still testing the AO in a stand-alone fashion and had not yet produced a final evaluation. SSA was also testing the AO as part of its FPM test—which integrates the AO and other process changes—and waiting for results from the FPM test before making a decision on whether to implement the AO. SSA also missed its planned milestones for implementing the SDM position. As with the AO initiative, SSA delayed SDM implementation until it had more definitive results from the FPM test. SSA expected the FPM test to provide better insight into the efficacy of the SDM position in conjunction with other, related changes. SSA is behind schedule in assessing the results of its FPM test. SSA had Delays in Evaluating planned to assess four of the five FPM process changes—the AO, the SDM, FPM Test Results predecision interview, and elimination of reconsideration—in fiscal year 1998. While SSA did not expect to process all its test cases until November 1999, it believed that it could use a smaller sample of cases to speed up some of the analysis. Specifically, SSA planned to use the first 2 months of test cases to analyze the effect of the four process changes on accuracy and allowance rates at all levels and processing time at the DDS level.49 While this approach expedited the analysis, SSA was not able to complete 49 SSA believed these 2 months of cohort data would enable it to reliably assess the effect of FPM because each month includes a randomly selected and representative group of cases. Without this approach, SSA would have had to wait for all the cases to flow through the process to determine the ultimate impact on allowance rates and the related cost, as was done with the tests of the AO and SDM initiatives. Page 78 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix II SSA’s Efforts to Meet Near-Term Milestones its assessment by the close of fiscal year 1998. As of October 1998, SSA could assess the impact of three process changes—the SDM, predecision interview, and eliminating reconsideration—on the initial level of decision-making. However, data to assess their impact on the hearing level and for the overall process, as well as data to assess the AO piece, were still incomplete: 5 percent of the AO cases and 15 percent of the ALJ cases had not yet been processed. Moreover, these 2 months of sample cases did not allow SSA to fully assess the impact of the proposed changes. SSA used another sample of cases selected later in the test process to assess the impact of the proposed changes on productivity (task time) at all levels and processing time at the appellate level. This later sample was needed in order to give test participants time to adjust to their new positions so that their productivity could be fairly assessed. Data needed for this assessment are incomplete, with approximately 15 percent of the AOs’ cases, 49 percent of ALJs’ control cases, and 69 percent of ALJs’ test cases not yet processed. Therefore, as of October 1998, SSA’s assessment of the impact of process changes on productivity and processing time at the hearing level and for the overall process was still preliminary and incomplete. SSA has made important but incomplete progress with its process Mixed Progress With unification initiative to improve the consistency of disability decisions Process Unification between the initial and appellate levels. Concrete actions taken by SSA under this initiative included completing nationwide process unification training for 15,000 adjudicators and quality reviewers.50 This training brought together staff from different offices and adjudicative levels in an effort to give adjudicators consistent training and an opportunity to share differing viewpoints. SSA has stated that it has also incorporated process unification principles into its ongoing training program and has provided additional cross-component training to adjudicators at all levels. Regarding its goal of developing a single presentation of policy, SSA also compared the guidance used by disability examiners with that used by ALJs and identified small differences. Starting in July 1996, SSA has been publishing new policies that are identical for all levels of adjudicators. Although SSA has yet to make changes to more closely align its older policies and procedures, SSA believes that instructions issued since 1996 address a substantial segment of important policy areas. 50 Before updating its plan, SSA issued in July 1996 nine SSRs to clarify policy in areas identified as contributing to inconsistent decisions. Page 79 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix II SSA’s Efforts to Meet Near-Term Milestones SSA has completed three of eight subinitiatives under process unification, although two were completed somewhat later than planned. For one subinitiative, SSA issued a final regulation in May 1998 that clarified its process for acquiescing to—that is, modifying its policies as a result of—court decisions. In addition, SSA has developed an action plan to implement this regulation. For another subinitiative, SSA issued a regulation in July 1998 establishing a new procedure for reviewing OHA allowances in order to identify factors that contribute to differences between DDS and ALJ decisions. SSA instituted these important changes 1 year behind schedule. SSA stated that it completed a third subinitiative—ensuring complete documentation of evidence in cases in which the DDS denies the claim—through its early process unification training. However, SSA has not completed other actions included in its original plan, such as developing a program circular on documentation requirements and revising its quality assurance process to support those requirements. SSA has been pursuing this subinitiative in concert with its subinitiative that provides for more detailed explanations of reconsideration denials. The combination of these two subinitiatives has been referred to as the “expanded rationale initiative.” As of October 1998, SSA had not completed, and had made mixed progress with respect to, the other five subinitiatives. Final regulations that address two subinitiatives—concerning the weight of DDS physicians’ opinions and RFC for less than sedentary work—have been drafted, and SSA expects to issue them in final form in the near future. SSA’s expanded rationale subinitiative experienced a number of delays. After encountering difficulties with an initial pilot test in 1997, SSA changed the focus of its expanded rationale subinitiative from more fully documenting only denial determinations to more fully documenting allowances as well. The agency did not begin testing again until early 1998. After experiencing further problems with deciding what information to include in the rationale, SSA issued a notice of its intent to test this subinitiative as part of the FPM test, beginning on or about October 29, 1998. For another subinitiative, SSA has been unable to achieve the goal of, within 1 year, remanding 100,000 appealed cases that had new medical information from OHA to the DDS, where medical experts reside to evaluate and reexamine the cases. SSA began remanding cases on schedule in July 1997, but after 10 months it had only remanded 8,488 cases to the DDSs. According to SSA, new evidence was generally received too late to warrant remanding. Realizing that it could not reach its goal of 100,000, SSA changed the remanding criteria to assist OHA with processing old cases and plans to pursue the original remanding goals through the other process unification subinitiatives. Page 80 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix II SSA’s Efforts to Meet Near-Term Milestones Regarding its subinitiative on treating physicians’ opinions, SSA held a symposium in June 1998 at which experts surfaced related policy issues on this complex topic. In October 1998, SSA indicated it planned to develop a final rule. SSA missed key milestones for its quality assurance initiative as well. As Delays in Quality planned, SSA is developing “in-line” quality assurance procedures— Assurance approaches for building quality into the process at the beginning—as it tests various redesign process changes. For example, as part of its SDM preimplementation test, SSA asked state agencies to develop and test approaches for improving the quality of SDM determinations. SSA planned to implement these procedures at the same time that it implemented the SDM and other process changes. However, implementation of in-line quality assurance approaches for process changes such as the SDM has been delayed along with implementation of the process changes themselves. Moreover, as of October 1998, SSA had not assessed the impact that state agency approaches have had on SDM quality. In addition to in-line quality assurance efforts, SSA planned to develop one standard for its end-of-line quality reviews in fiscal year 1997 and to test its use in fiscal year 1998.51 However, as of the end of fiscal year 1998, the agency had not reached agreement on what that single standard for quality reviews should be. Moreover, as of October 1998, SSA had identified additional and very important issues and questions that should be addressed in the context of the quality assurance initiative, including when, how, and by whom an “end-of-line” review should be conducted; whether end-of-line quality reviews should be balanced with respect to percentage of allowance and denial decisions selected for review; whether the DDS performance standard for quality reviews should be changed; and whether in-line quality assurance approaches should be mandatory. 51 As discussed in ch. 1, SSA quality reviews reflect the fragmented approach to decision-making at the two levels. Page 81 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix III Results of SSA’s Tests SSA has not yet demonstrated that tested initiatives will significantly improve the process. For the three tests conducted to date, SSA collected data to assess whether the initiatives would help achieve its five redesign objectives.52 To test its expectation that, overall, redesign would not change total benefits paid to claimants, SSA also assessed the impact of tested initiatives on the overall allowance rate. Results from SSA’s stand-alone tests of the AO and SDM positions were disappointing and, in the case of the AO, also inconclusive. FPM test results are still preliminary but, as of October 1998, indicated some potential for improving the claims process at the DDS level. For example, the combination of three process changes—that is, the SDM, predecision interview, and elimination of reconsideration—appears to improve accuracy of initial determinations and to reduce processing time and administrative costs at the DDS level, although SSA has not yet fully analyzed the extent of the improvement. On the other hand, preliminary data on the AO’s performance in the FPM context were inconclusive, and SSA has yet to demonstrate any improvements at the appellate level. SSA has not yet determined the overall impact of the four process changes together on the process. Results from the stand-alone test of the AO position generally fell short of AO Stand-Alone Test SSA’s early targets, and SSA has not been able to demonstrate whether the Results Were AO position is meeting SSA’s redesign objectives. Regarding SSA’s objective Disappointing and of correctly allowing more claims at the earliest possible level, the AO position appears to have improved the accuracy of allowances but did not Inconclusive increase the number of allowances made earlier in the process. SSA had expected the AO to help meet this objective because each allowance made correctly by an AO would represent a claim that would not require a hearing. Early results from the AO test indicated that AOs were allowing 24 percent of cases. However, early AO allowances were not clearly more accurate than comparable decisions made under the current process. Comparable decisions are “on-the-record” allowances made by either ALJs or other hearing office staff for claims that are relatively clear-cut and do not require a hearing.53 However, after almost 3 years of testing—half of which involved increased training and feedback for test participants—AO 52 SSA’s five redesign objectives are to (1) allow claims that should be allowed at the earliest possible level; (2) make decisions as quickly as possible (reduce processing time, or the number of days needed to make a final decision); (3) make the process more efficient (reduce task time spent working directly on claims, administrative costs, or both); (4) make the process user-friendly; and (5) provide a satisfying work environment for employees. 53 A comparison with all allowances made at the appeals level would be misleading because AOs were expected to allow only those cases that were straightforward, whereas most allowances at the appeals level are made by ALJs and generally involve more difficult adjudicative issues. Page 82 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix III Results of SSA’s Tests accuracy has improved, and AO allowance decisions are currently more accurate than comparable decisions made under the current process.54 On the other hand, the percentage of claims allowed by the AOs has declined over time, and SSA has not demonstrated that the current level of early allowances made by the AO represents an improvement over the current process. According to SSA officials, as of October 1998, AOs had been allowing about 15 percent of appealed cases. Data we obtained from OHA indicated that, as of October 1998, ALJs and OHA staff had together allowed over 17 percent of appealed cases without a hearing. Test results for two other redesign objectives—reducing processing time and improving efficiency—fell short of SSA’s early targets and did not clearly demonstrate an improvement over the current process. For example, as of April 1998, AOs needed an average of 89 days to allow cases and 103 days to prepare them for a hearing, compared with SSA’s early targets of 45 and 60 days. In addition, SSA originally expected AOs to process an average of two cases a day, but instead AOs processed, on average, less than one case per day. Comparisons with SSA’s early targets are somewhat misleading, according to SSA, because these targets assumed that key supports for the AO would be in place, while the test did not include such supports. To make a fairer assessment, SSA selected a small group of control cases to be handled under the current process with whose times the AO processing and task times could be compared. However, as discussed in chapter 2, OHA staff did not handle AO and control cases as instructed.55 Without control cases, SSA could not determine with certainty whether, or the extent to which, the AO position reduced overall processing or task time.56 On a positive note, judges considered 85 percent of cases prepared by AOs to be fully developed and ready for a hearing. Test results for the other two redesign objectives—to make the process user-friendly and provide a satisfying work environment for employees—were somewhat mixed. An early focus group study showed 54 Different standards exist for reviewing the accuracy of disability decisions, and opinions differ regarding which standard of review is appropriate to apply. SSA used three standards to assess the relative accuracy of AO allowances and, as of September 1998, AO allowances were higher than comparable decisions under the current process for all three standards. 55 To fairly assess processing and task times, OHA staff were instructed to promptly schedule and hold hearings for both AO and control cases. 56 Comparing cases allowed or prepared by AOs with cases handled routinely under the current system is inappropriate. Unlike AO cases, cases handled under the current system are generally placed in a lengthy backlog of cases, which can increase both the processing and task times needed to reach a final disposition. Comparing AO allowances with allowances for cases without a hearing under the current process would be more appropriate; however, SSA has not collected the data needed to make such a comparison. Page 83 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix III Results of SSA’s Tests claimants preferred the AO process, but more recent survey data did not show any clear preference. Two-thirds of claimants’ attorneys surveyed in December 1997 viewed the AO process as an improvement over the traditional hearing process.57 With respect to improving the work environment, AO test participants surveyed by SSA generally liked the new process, but other OHA staff were more neutral about the AO initiative.58 Finally, test results indicated that the AO position would significantly increase overall allowance rates at the appellate level. Specifically, the combined rate of AO allowances and ALJ allowances based on AO-prepared cases was 56 percent as of March 1998, which is 4 percent higher than the overall allowance rate for the appellate level under the current process. Assuming no change in allowance rates at the initial level, such an increase in allowances at the appellate level would translate into an increase in overall benefit payments; however, SSA did not estimate the impact on benefit payments. Results of SSA’s stand-alone test of the SDM initiative were marginal and SDM Stand-Alone Test mixed and did not clearly indicate that, by itself, the SDM would Results Were Marginal significantly aid SSA in meeting its redesign objectives. SDM test results for and Mixed SSA’s objective to correctly allow more cases at the earliest level were mixed. Although the SDM allowance rate was 1.2 percent higher than that of the control group representing the current process, SDM accuracy was slightly lower overall and significantly lower for denial determinations. In particular, analysts identified nine impairments that were likely to be associated with determination errors, such as diabetes and asthma, for which the accuracy of SDM determinations was markedly lower than the accuracy of determinations made under the current process. Maintaining accuracy is critical because incorrect determinations can result in unnecessary costs. For example, SSA staff unofficially estimated that the relatively high number of inaccurate allowances by the SDM could result in inappropriate benefits payments of $65 million, which would outweigh any administrative cost savings. And inaccurate denials could cause unnecessary reconsiderations and appeals, resulting in higher administrative costs—which SSA estimated at $2 million—as well as inconvenience, or even financial hardship, for some claimants. On the other hand, SSA estimated that the increase in the number of allowances 57 However, a spokesperson representing an organization of SSA claimants’ attorneys that we contacted indicated that the group does not support SSA’s AO initiative. 58 Six internal stakeholder groups that we contacted had the following views of the AO initiative: two supported the AO concept, two opposed it, and two did not have a position but did have concerns about its effectiveness. Page 84 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix III Results of SSA’s Tests that the SDM made correctly could save $9.5 million as a result of fewer reconsiderations and appeals. The SDM’s contribution toward meeting two other redesign objectives was marginal. By making the SDM solely responsible for the initial disability determination, instead of jointly responsible along with a DDS physician, SSA expected the SDM to reduce processing time and improve efficiency.59 However, test results indicated that the SDM process saved only 1 day, compared with the current process. Test results also showed a marginal decrease in task time (time spent working directly on cases). Specifically, the DDS physician task time decreased by 7.1 minutes per case, offset somewhat by an increase in SDM task time by 3.5 minutes, for an overall decrease of 3.6 minutes per case. The potential administrative savings associated with this process improvement was estimated by SSA to be $16.1 million. Test results for the objective of providing a satisfying work environment for employees were generally positive. SSA’s survey of test participants indicated the following: DDS managers believed the SDM model to be an effective use of examiner and consultant resources, disability examiners generally liked the process, and most medical consultants believed that SDMs performed good work and required less input.60 SSA did not test whether the SDMs would make the process user-friendly because this process change is transparent to the customer. With respect to the SDM’s impact on overall benefit payments, SSA estimated that the increase in SDM allowances would translate into an increase of $2.1 billion in program costs. SSA did not measure the impact of SDM on the overall allowance rates—that is, by tracking SDM denial determinations that were appealed through to the final decision—and the associated program costs. In its phase II test of SDM, SSA is exploring different approaches for improving the accuracy, productivity, and processing time of the SDM. Preliminary data from this test suggest that SDM accuracy and processing time might be improving. However, as of June 1998, data from the phase II test were not weighted to produce reliable estimates of accuracy or productivity. 59 SSA expected the SDM to consult with DDS physicians on an as-needed basis so that physicians could concentrate on the more complex cases, thereby reducing task and processing time. 60 Key stakeholder groups we contacted also supported the SDM concept, with the exception of groups representing DDS physicians from two states. Page 85 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix III Results of SSA’s Tests As of October 1998, preliminary results of the FPM test showed promise for Preliminary FPM achieving some of SSA’s five redesign objectives at the DDS level of the Results Were process; however, SSA has not determined the extent of potential Promising but improvements. As of October 1998, SSA had not processed enough cases, and performed sufficient analysis, to demonstrate improvement at the Inconclusive appellate level. Preliminary results indicate that, with the addition of the predecision interview, the FPM brings SSA closer to its objective of correctly allowing more cases earlier in the process—that is, at the DDS level; however, progress toward this objective has not been demonstrated at the appellate level. For initial determinations made under FPM, accuracy improved and the allowance rate increased, compared with the current process.61 It is unclear whether the 100-percent review of initial determinations in the FPM test affected relative accuracy and allowance rates.62 Moreover, if allowances made at the reconsideration step are included in the analysis, the overall allowance rates at the DDS level under the current process and under FPM are similar: 36.8 and 36.5, respectively. SSA preliminary findings are less conclusive at the appellate level. As of October 1998, SSA had not completely processed all the test cases through the appellate level.63 Also, SSA did not completely assess the AO’s performance with respect to this objective. For example, SSA compared the accuracy of AO allowances with that of all other allowances made at the appellate level under the current process. As discussed previously, a more appropriate comparison would have been with those cases allowed under the current process—by either ALJs or other OHA staff—that did not require a hearing: that is, similarly clear-cut cases. With the elimination of reconsideration, the FPM appears to contribute to reductions in processing time (measured in days) and task time (measured in minutes or work-years saved) at the DDS level. This analysis is incomplete, however, and SSA has not determined the impact of changes at 61 Using a predecision interview, the SDM’s accuracy rate for initial determinations that resulted in denials was 2 percent higher than under the current process; the accuracy rate for allowances was comparable to that of the current process. In addition, the allowance rate was 4.7 percent higher than that of the current process. 62 All initial determinations in the FPM test were reviewed for accuracy, whereas the comparison group received its normal, lower level of review that focuses on allowance decisions. The difference in level and focus of the reviews could have affected the relative behavior of FPM test participants. 63 As discussed in app. II, SSA used two different groups of cases for its assessment of the FPM. As of Oct. 1998, 5 percent of the 2-month group of test cases used to assess accuracy and allowance rates had still not been completed by AOs, and 15 percent had not been completed by ALJs. For the different set of cases used to assess task time and processing time, as of Oct. 1998, 15 percent had not been completed by AOs, and more than 50 percent had not been completed by ALJs. Page 86 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix III Results of SSA’s Tests the appellate level or on the process overall. Because of the predecision interview, the SDM required more days and task time to reach a final determination, as compared with the current process.64 On the other hand, the elimination of reconsideration under FPM has potential for significantly lowering the overall number of days and task time needed to reach a final determination at the DDS level. As of October 1998, SSA estimated that FPM would save 2,600 staff-years valued at $185 million because of reduced task time spent on cases at the DDS level. However, SSA’s test results did not include an assessment of overall reductions in processing time at the DDS level. Moreover, SSA had not demonstrated the impact of FPM changes on processing and task time at the appellate level and on the process overall. SSA’s test results were incomplete, in part, because it had not processed all the test cases through the appeals process. In particular, SSA lacked a sufficient number of cases to assess the impact of FPM on task time at the appellate level. In addition, SSA results did not include a comparison of the overall processing and task time needed at the appellate level to reach a final decision under FPM—either by an AO or an ALJ using an AO-prepared case—with the time needed under the current process. Test results did not demonstrate whether FPM would make the process more user-friendly, although there were indications that FPM might provide a more satisfying work environment. Customer survey results indicated that customer satisfaction was greater for those awarded after a predecision interview than for those denied and that the predecision interview did not alter the overall satisfaction of claimants who were denied. The survey also revealed that claimants found the letter sent by SSA to inform them of the interview to be confusing. With respect to the work environment, a majority of FPM test participants indicated that they believed that the new process offered them the ability to serve the claimant better. In addition, most test participants felt that their new duties had a positive effect on their job satisfaction. Finally, preliminary test results indicated that the FPM might result in a higher overall allowance rate and benefit payments, although more cases need to be processed and evaluated to reach a final conclusion. As of October 1998, SSA estimated an increase of 2 percent in the overall allowance rate under FPM but had not yet estimated the impact on benefit payments. 64 With the predecision interview, the SDM required, on average, 11 more days to make initial allowances and 24 more days to make denials than under the current process. Overall, the SDM’s task time was 16.3 minutes longer than that of the current process for initial determinations. Page 87 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix IV Comments From the Social Security Administration Page 88 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix IV Comments From the Social Security Administration Page 89 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix IV Comments From the Social Security Administration Now on p. 54. Page 90 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix IV Comments From the Social Security Administration Page 91 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix IV Comments From the Social Security Administration Page 92 GAO/HEHS-99-25 Disability Claims Process Redesign Appendix V GAO Contacts and Staff Acknowledgments Kay Brown, Assistant Director, (202) 512-3674 GAO Contacts Michele Grgich, Evaluator-in-Charge, (415) 904-2183 In addition to those named above, Julie M. DeVault, Carlos J. Evora, John Staff M. Ortiz, and Robert T. Tomco made major contributions to collecting and Acknowledgments synthesizing data. Ms. DeVault, Mr. Tomco, and James P. Wright contributed significantly to writing this report, and Sharon L. Caudle and William E. Hutchinson provided valuable technical assistance throughout the review. Page 93 GAO/HEHS-99-25 Disability Claims Process Redesign Bibliography Brynjolfsson, Erik, and others. “The Matrix of Change.” Sloan Management Review (winter 1997), pp. 37-54. Caudle, Sharon L. Reengineering for Results: Keys to Success From Government Experience. Washington, D.C.: National Academy of Public Administration, 1995. _____. “Reengineering: Avoiding Becoming Lost in Space.” The Public Manager (spring 1998), pp. 27-30. Guha, Subo, and others. “Business Process Change and Organizational Performance: Exploring an Antecedent Model.” Journal of Management Information Systems, 14 (1) (summer 1997), pp. 119-54. Hall, Gene, Jim Rosenthal, and Judy Wade. “How to Make Reengineering Really Work.” Harvard Business Review (Nov.-Dec. 1993), pp. 119-31. Hammer, Michael, and James Champy. Reengineering the Corporation. New York: Harper Collins Publishers, 1993. _____, and Steven A. Stanton. “The Reengineering Revolution.” Government Executive (Sept. 1995), pp. 2A-12A. Linden, R.M. Seamless Government: A Practical Guide to Re-engineering in the Public Sector. San Francisco: Jossey-Bass Publishers, 1994. ProSci. Best Practices Report for Managing Change. Loveland, Colo.: ProSci, 1998. Secretary of Health and Human Services. Implementation of Section 304 (g) of Public Law 96-265, Social Security Disability Amendments of 1980 (the Bellmon Report). Washington, D.C.: HHS, Jan. 1982. Social Security Advisory Board. How SSA’s Disability Programs Can Be Improved. Washington, D.C.: Social Security Advisory Board, Aug. 1998. SSA, Office of Program Integrity Reviews. Findings of the Disability Hearings Quality Review Process. Washington, D.C.: SSA, Sept. 1994. Page 94 GAO/HEHS-99-25 Disability Claims Process Redesign Bibliography Page 95 GAO/HEHS-99-25 Disability Claims Process Redesign Related GAO Products Social Security Disability: SSA Making Progress in Conducting Continuing Disability Reviews (GAO/HEHS-98-198, Sept. 18, 1998). Social Security Reform: Raising Retirement Ages Improves Program Solvency but May Cause Hardship for Some (GAO/T-HEHS-98-207, July 15, 1998). Social Security Administration: Technical Performance Challenges Threaten Progress of Modernization (GAO/AIMD-98-136, June 19, 1998). The Results Act: Observations on the Social Security Administration’s Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R, June 9, 1998). Social Security Administration: Software Development Process Improvements Started but Work Remains (GAO/AIMD-98-39, Jan. 28, 1998). Social Security Disability: SSA Must Hold Itself Accountable for Continued Improvement in Decision-Making (GAO/HEHS-97-102, Aug. 12, 1997). Business Process Reengineering Assessment Guide (GAO/AIMD-10.1.15, Apr. 1997, Ver. 3). SSA:Significant Challenges Await New Commissioner (GAO/HEHS-97-53, Feb. 20, 1997). SSADisability Redesign: Focus Needed on Initiatives Most Crucial to Reducing Costs and Time (GAO/HEHS-97-20, Dec. 20, 1996). Appealed Disability Claims: Despite SSA’s Efforts, It Will Not Reach Backlog Reduction Goal (GAO/HEHS-97-28, Nov. 21, 1996). SSA Disability Redesign: More Testing Needed to Assess Feasibility of New Claim Manager Position (GAO/HEHS-96-170, Sept. 27, 1996). Social Security Disability: Backlog Reduction Efforts Under Way; Significant Challenges Remain (GAO/HEHS-96-87, July 11, 1996). Executive Guide: Effectively Implementing the Government Performance and Results Act (GAO/GGD-96-118, June 1996). (207007) Page 96 GAO/HEHS-99-25 Disability Claims Process Redesign Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
SSA Disability Redesign: Actions Needed to Enhance Future Progress
Published by the Government Accountability Office on 1999-03-12.
Below is a raw (and likely hideous) rendition of the original report. (PDF)