Electronic Disability Claims Processing: Social Security Administration's Accelerated Strategy Faces Significant Risks

Published by the Government Accountability Office on 2003-07-24.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                             United States General Accounting Office

GAO                          Testimony
                             Before the Subcommittee on Social
                             Security, Committee on Ways and Means,
                             House of Representatives

For Release on Delivery
Expected at 10:00 a.m. EDT
Thursday, July 24, 2003
                             ELECTRONIC DISABILITY
                             CLAIMS PROCESSING
                             Social Security
                             Accelerated Strategy Faces
                             Significant Risks
                             Statement of Linda D. Koontz, Director
                             Information Management Issues

                                                July 24, 2003

                                                ELECTRONIC DISABILITY CLAIMS

Highlights of GAO-03-984T, testimony            Social Security Administration’s
before the Subcommittee on Social
Security, Committee on Ways and Means,          Accelerated Strategy Faces Significant
House of Representatives

Providing benefits to disabled                  SSA’s goal to establish a more efficient, paperless disability claims
individuals is one of the Social                processing system is important, and one that could benefit millions. To
Security Administration’s most                  achieve this goal, SSA’s immediate focus is on developing an electronic
important service delivery                      folder to store claimant information and large volumes of medical images,
obligations—touching the lives of               files, and other documents that are currently maintained in paper folders,
about 10 million individuals. In
recent years, however, providing
                                                and then make this information accessible to all entities involved in
this benefit in a timely and efficient          disability determinations. SSA’s accelerated strategy calls for development
manner has become an increasing                 of this capability by January 2004 rather than in 2005, as originally planned.
challenge for the agency. This past             (See figure.)
January, in fact, GAO designated
SSA’s disability programs as high-              Since accelerating this effort, SSA has performed important tasks toward
risk.                                           establishing this initial electronic capability. Nonetheless, it has substantial
                                                work to accomplish in order to develop the technologically complex
Following a prior unsuccessful                  electronic folder and begin implementation by late next January.
attempt, the agency is now in the
midst of a major initiative to                  While responsive to the agency’s need for an operational system as soon as
automate its disability claims
functions, taking advantage of
                                                possible, SSA’s accelerated strategy involves risks. For example, pilot tests
technology to improve this service.             that are to provide important information about the electronic folder’s
Seeking immediate program                       performance are not expected until late December—just 1 month before its
improvements, SSA is using an                   planned implementation. In addition, a strategy for end-to-end testing to
accelerated approach—called                     demonstrate that the individual components will work together reliably has
AeDib—to develop an electronic                  not been completed. Further increasing the system’s vulnerability is that SSA
disability claims processing system.            has not yet comprehensively assessed project risks. Unless addressed, these
                                                factors could ultimately derail the initiative.
At the request of the
Subcommittee, GAO is currently                  While SSA has taken steps to involve key stakeholders in the systems
assessing the strategy that                     development process, officials in state Disability Determination Services
underlies SSA’s latest initiative to
                                                offices that we contacted expressed concerns that they had only limited
develop the electronic disability
system. For this testimony, GAO                 involvement in the development effort. They stated that their concerns were
was asked to discuss its key                    not adequately heard and considered in the decision-making process. Unless
observations to date regarding the              SSA addresses these issues, it cannot be assured of stakeholder agreement
AeDib initiative, including strategy,           with and full use of the system.
risks, and stakeholder involvement.
                                                History of SSA’s Efforts to Achieve an Electronic Disability Claims Processing System
GAO plans to discuss more fully
the results of this continuing
review in a subsequent report.


To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Linda Koontz at
(202) 512-6240 or koontzl@gao.gov.
Mr. Chairman and Members of the Subcommittee:

I am pleased to be here today to participate in your continuing dialogue on
the Social Security Administration’s (SSA) service-delivery capability. My
testimony focuses on a critical aspect of SSA’s overall goal—its ongoing
initiative to achieve an electronic disability claims process. As you know,
one of SSA’s most vital obligations is paying cash benefits to disabled
individuals under the Disability Insurance and Supplemental Security
Income programs. In 2002, the agency paid approximately $86 billion to
about 10 million disabled beneficiaries. Yet, over the years, it has become
an increasing challenge for SSA to ensure an acceptable level of service—
both in terms of quality and timeliness. This past January, in fact, we
reported SSA’s disability programs as high-risk.1

During testimony before this Subcommittee in May 2002, the
Commissioner of Social Security voiced concerns about the length of time
that the current disability process can take, and promised immediate
improvements. Among these improvements, she announced plans to
accelerate SSA’s initiative to develop an electronic disability claims process
by late January 2004 rather than late 2005 as initially planned. SSA’s
refocused project is known as the accelerated electronic disability

At your request, we are currently reviewing AeDib to assess SSA’s strategy
for developing the electronic disability claims process. My testimony will
discuss our key observations to date regarding the initiative, including
SSA’s (1) accelerated approach for and progress toward completing the
electronic disability system, (2) actions for ensuring the system’s
successful operations and protection against risks, and (3) consultation
with and support from key stakeholders. We plan to discuss more fully the
results of our ongoing review in a subsequent report to you.

In conducting this work, we analyzed relevant documentation describing
SSA’s plans and strategies for developing and implementing the AeDib
system and its progress in doing so. We reviewed technical documents
pertaining to the system development and interviewed appropriate SSA
officials to determine the extent to which the agency has followed its
software development guidance. We supplemented our analysis with

 U.S. General Accounting Office, Major Management Challenges and Program Risks:
Social Security Administration, GAO-03-117 (Washington, D.C.: January 2003).

Page 1                                                                  GAO-03-984T
                   interviews of SSA officials in the Offices of Disability Programs,
                   Operations, Systems, and Hearings and Appeals. In addition, we visited SSA
                   field offices in Delaware and Texas to observe disability claims intake
                   operations and obtain staff perspectives on the AeDib project. We also
                   conducted site visits at the Delaware, New York, Texas, and Wisconsin
                   Disability Determination Services (DDS) offices to observe disability
                   system pilot tests and discuss these offices’ involvement in planning and
                   implementing AeDib. Further, we surveyed staff in six other DDS offices,
                   and interviewed representatives of state and SSA employees and the
                   medical community. These included the National Council of Disability
                   Determination Directors, the American Federation of Government
                   Employees, and the American Health Information Management
                   Association. We performed our work to date in accordance with generally
                   accepted government auditing standards, from December 2002 through
                   July of this year.

Results in Brief   SSA’s goal of achieving an electronic disability claims process represents an
                   important, positive direction toward more efficient delivery of disability
                   payments to an increasing beneficiary population. In undertaking AeDib,
                   SSA’s immediate focus is on developing the capability to allow claimant
                   information and large volumes of medical images, files, and other
                   documents that are currently maintained in paper folders to be stored in
                   electronic folders, and then accessed, viewed, and shared by the disability
                   processing offices. Since announcing the accelerated initiative in May
                   2002, SSA has made progress toward attaining this capability, including
                   implementing initial automated claims-intake functions in its field offices.
                   Nonetheless, substantial work remains—the most crucial of which is
                   developing document management and scanning and imaging capabilities
                   that are fundamental to achieving the electronic folder.

                   SSA’s current strategy, however, involves risks that could jeopardize its
                   successful transition to an electronic disability process. A pilot test that
                   would determine whether technology supporting the electronic folder will
                   work as intended, is not expected to be completed until at least
                   December—just 1 month before SSA plans to begin implementing the
                   electronic folder to the disability offices—leaving the agency little time to
                   incorporate test results. The agency also does not currently plan to
                   perform end-to-end testing to demonstrate, prior to the national
                   implementation, how successfully the multiple components will operate
                   together to electronically process disability claims. Adding to the system’s
                   vulnerability is that SSA has not yet performed a comprehensive

                   Page 2                                                            GAO-03-984T
             assessment to identify and establish strategies for mitigating project risks
             that could result in cost, schedule, and performance shortfalls.

             Finally, SSA has not yet successfully resolved certain concerns among key
             disability stakeholders regarding the AeDib strategy. SSA officials maintain
             that they have involved stakeholders in developing AeDib through
             including them in working groups and steering committee meetings.
             However, state DDSs in particular, have significant concerns about how the
             system is being developed and implemented, and do not believe that their
             offices have been effectively involved with SSA in making key decisions
             about the initiative; they question whether this strategy will effectively
             support their business processes. Further, although physicians and other
             providers of medical evidence are critical to the disability process, SSA’s
             consultations with the medical community have thus far been limited and
             their representatives have concerns about electronically submitting
             evidence for disability determinations. Until SSA can ensure itself and all
             stakeholders that the concerns have been effectively considered and
             addressed—and the stakeholders view themselves as fully engaged in the
             initiative—the agency risks not having full acceptance and use of this vital
             service-delivery tool.

Background   The Disability Insurance and Supplemental Security Income programs are
             the nation’s largest providers of federal income assistance to disabled
             individuals, with SSA making payments of approximately $86 billion to
             about 10 million beneficiaries in 2002. The process through which SSA
             approves or denies disability benefits is complex and involves multiple
             partners at both the state and federal levels in determining a claimant’s
             eligibility. Within SSA, these include its 1,300 field offices, which serve as
             the initial point of contact for individuals applying for benefits, and the
             Office of Hearings and Appeals, which, at the request of claimants,
             reconsiders SSA’s decisions when benefits are denied.

             SSA also depends on 54 state Disability Determination Services (DDS)
             offices to help process claims under its disability insurance programs.2
             State DDSs provide crucial support to the initial disability claims process—
             one that accounts for most of SSA’s workload—through their role in
             determining an individual’s medical eligibility for disability benefits. DDSs

              DDSs are located in all 50 states, the District of Columbia, Guam, Puerto Rico, and the
             Virgin Islands.

             Page 3                                                                         GAO-03-984T
make decisions regarding disability claims in accordance with federal
regulations and policies; the federal government reimburses 100 percent of
all DDS costs in making disability determination decisions. Physicians and
other members of the medical community support the DDSs by providing
the medical evidence to evaluate disability claims.

The process begins when individuals apply for disability benefits at an SSA
field office, where determinations are made about whether they meet
nonmedical criteria for eligibility. The field office then forwards the
applications to the appropriate state DDS, where a disability examiner
collects the necessary medical evidence to make the initial determination
of whether the applicant meets the definition of disability. Once the
applicant’s medical eligibility is determined, the DDS forwards this
decision to SSA for final processing.

Claimants who are initially denied benefits can ask to have the DDS
reconsider its denial. If the decision remains unfavorable, the claimant can
request a hearing before a federal administrative law judge at an SSA
hearings office, and, if still dissatisfied, can request a review by SSA’s
Appeals Council. Upon exhausting these administrative remedies, the
individual may file a complaint in federal district court. Each level of
appeal, if undertaken, involves multi-step procedures for the collection of
evidence, information review, and decision making. Many individuals who
appeal SSA’s initial decision will wait a year or longer—perhaps up to 3
years—for a final decision.

To address concerns regarding the program’s efficiency, in 1992 SSA
initiated a plan to redesign the disability claims process, emphasizing the
use of automation to achieve an electronic (paperless) processing
capability. The automation project started in 1992 as the Modernized
Disability System, and was redesignated the Reengineered Disability
System (RDS) in 1994. RDS was to automate the entire disability claims
process—from the initial claims intake in the field office to the gathering
and evaluation of medical evidence at the state DDSs, to payment
execution in the field office or processing center, and including the
handling of appeals at the hearings offices. However, our prior work noted
that SSA had encountered problems with RDS during its initial pilot

Page 4                                                           GAO-03-984T
                         testing.3 For example, systems officials had stated that, using RDS, the
                         reported productivity of claims representatives in the SSA field offices
                         dropped. They noted that before the installation of RDS, each field office
                         claims representative processed approximately five case interviews per
                         day. After RDS was installed, each claims representative could process
                         only about three cases per day. As a result, following an evaluation by a
                         contractor, SSA suspended RDS in 1999 after approximately 7 years and
                         more than $71 million reportedly spent on the initiative.

                         In August 2000 SSA issued a management plan with a renewed call for
                         developing an electronic disability system by the end of 2005. The strategy
                         was to incorporate three components: an electronic disability intake
                         process that would include (1) a subset of the existing RDS software, (2)
                         the existing DDS claims process, and (3) a new system for the Office of
                         Hearings and Appeals. The management plan also provided for several pilot
                         projects to test the viability and performance of each project component.
                         SSA’s work on this effort occurred through the spring of 2002, at which time
                         the Commissioner announced that she had begun an accelerated initiative
                         to more quickly automate the disability claims process. The agency
                         anticipated that, with technologically advanced disability processing
                         offices, it could potentially realize benefits of more than $1 billion, at an
                         estimated cost of approximately $900 million, over the 10-year life of the
                         accelerated initiative.

AeDib’s Strategy Calls   In undertaking AeDib, SSA has embarked on a major initiative consisting of
                         multiple projects that are intended to move all partners in its disability
For Developing and       claims adjudication and review to an electronic business process. SSA
Integrating Multiple     envisions that AeDib will allow its disability components to stop relying on
                         paper folders to process claims and to develop new business processes
Disability System        using legacy systems and information contained in an electronic folder to
Projects                 move and process all of its work. In so doing, SSA anticipates that AeDib
                         will enable disability components to achieve processing efficiencies,
                         improve data completeness, reduce keying errors, and save time and

                          U.S. General Accounting Office, Social Security Administration: Technical and
                         Performance Challenges Threaten Progress of Modernization, GAO/AIMD-98-136
                         (Washington, D.C.: June 19, 1998).

                         Page 5                                                                    GAO-03-984T
The AeDib strategy focuses on developing the capability for claimant
information and large volumes of medical images, files, and other
documents that are currently maintained in paper folders to be stored in
electronic folders, and then accessed, viewed, and shared by the disability
processing offices. SSA is undertaking five key projects to support the

• An Electronic Disability Collect System to provide the capability for SSA
  field offices to electronically capture information about the claimant’s
  disability and collect this structured data in an electronic folder for use
  by the disability processing offices;

• A Document Management Architecture that will provide a data
  repository and scanning and imaging capabilities to allow claimant
  information and medical evidence to be captured, stored, indexed, and
  shared electronically between the disability processing offices.

• Internet applications that will provide the capability to obtain disability
  claims and medical information from the public via the Internet.

• A DDS systems migration and electronic folder interface that will
  migrate and enhance the existing case processing systems to allow the
  state disability determination services offices to operate on a common
  platform and prepare their legacy systems to share information in the
  electronic folder; and

• A Case Processing and Management System for the Office of Hearings
  and Appeals that will interface with the electronic folder and enable its
  staff to track, manage, and complete case-related tasks electronically.

According to SSA, the Electronic Disability Collect System and the
Document Management Architecture are the two fundamental elements
needed to achieve the electronic disability folder. By late January 2004,
SSA plans to have developed these two components. It also expects to have
completed five Internet disability applications, enhanced the DDS legacy
systems, and developed the software that will allow existing SSA and DDS
systems to interface with the electronic folder. However, SSA will not yet
have implemented the scanning and imaging capabilities and the interface
software to enable each disability processing office to access and use the
data contained in the electronic folder. SSA officials explained that, at the
end of next January, the agency plans to begin an 18-month rollout period,
in which it will implement the scanning and imaging capabilities and

Page 6                                                            GAO-03-984T
                       establish the necessary interfaces. SSA has drafted but not yet finalized the
                       implementation strategy for the rollout.

SSA Has Completed      SSA has performed several important project tasks since beginning the
                       accelerated initiative in 2002. For example, it has implemented limited
Important AeDib        claims-intake functionality as part of the Electronic Disability Collect
Tasks, But Much Work   System, and begun additional upgrades of this software. In addition, it has
                       developed two Internet applications for on-line forms to aid claimants in
Remains                filing for disability benefits and services. Further, to support electronic
                       disability processing, SSA is in the process of migrating and upgrading
                       hardware and case processing software to allow all of the 54 state DDSs to
                       operate on a common platform,4 and has begun developing software to
                       enable the DDS systems to interface with the electronic folder. SSA has
                       also performed some initial tasks for the Document Management
                       Architecture, including developing a system prototype, establishing
                       requirements for the scanning capability, and drafting a management plan
                       and training strategy.

                       Nonetheless, the agency still has a significant amount of work to
                       accomplish to achieve the electronic disability folder by the end of next
                       January. While substantial work remains for each of the AeDib
                       components, primary among SSA’s outstanding tasks is completing the
                       Document Management Architecture’s development, testing, and
                       installation at the agency’s National Computer Center. Table 1 illustrates
                       SSA’s progress through last June in accomplishing tasks included in the
                       AeDib initiative, along with the many critical actions still required to
                       develop and implement the electronic disability processing capability.

                        Thirty of the 54 state DDSs previously operated on a platform consisting of Wang hardware
                       and iLevy disability processing software. SSA is now moving all DDSs to an IBM series
                       platform in an attempt to achieve consistency among all DDS systems in processing
                       disability claims.

                       Page 7                                                                       GAO-03-984T
Table 1: Status of Tasks Involved in Developing the AeDib

                                                                                                            Key tasks to be
                                                                                                            completed during
                                                                                                            18-month national
AeDib           Tasks completed as of       Tasks to be completed by           Planned January 2004 project rollout (2/2004–
component       June 30, 2003               January 30, 2004                   status                       7/2005)
Electronic      Developed EDCS release      Develop electronic folder interface EDCS software v.6 operational None reported.
Disability      4.2.3                       requirements for AeDib legacy       in all SSA field offices. It will
Collection      Developed EDCS release      systems                             automate the disability interview
System (EDCS)   5.0                                                             process. Data will be
                Developed EDCS release      Develop software for version 6.X propagated to EDCS and /or the
                5.1                                                             electronic folder from SSA
                Drafted training strategy   • Complete design and legacy        mainframe systems and
                                              system support for v6.0           disability Internet applications.

                                            • Complete validation for V.6.01

                                            • Compete validation for V6.02

                                            • Complete design, legacy system
                                              support, and integration and
                                              environmental testing for V6.1

                                            • Validate software

                                            • Conduct integration and
                                              environmental testing

                                            • Release software to production

                                            • Train users

                                              Page 8                                                                   GAO-03-984T
(Continued From Previous Page)
                                                                                                          Key tasks to be
                                                                                                          completed during
                                                                                                          18-month national
AeDib           Tasks completed as of       Tasks to be completed by         Planned January 2004 project rollout (2/2004–
component       June 30, 2003               January 30, 2004                 status                       7/2005)
Document        Developed document          Develop procurement strategies  DMA infrastructure established   Ensure site
Management      imaging and                 Conduct performance engineering in the SSA National Computer     preparation for DMA
Architecture    management system           and tuning                      Center
(DMA)           prototype                                                                                    Roll out DMA
                                            Conduct validation                                               infrastructure (e.g.,
                Provided technical                                                                           casual scanning
                training to DMA staff       Conduct integration and                                          equipment, object
                                            environmental testing                                            repository servers,
                Developed management                                                                         scanning and
                approach and plan           Install pilots                                                   imaging servers, and
                                                                                                             fax servers)
                Developed DMA               Conduct pilot testing                                            Conduct process
                requirements                                                                                 evaluation
                                            Evaluate pilot results
                Acquired AeDib pilot
                infrastructure              Address any pilot issues

                Drafted training strategy   Setup production environment

                                            Procure AeDib infrastructure

                                            Establish object management

                                            Contract with outsourced
                                            scanning vendors for national
                                            scanning support

                                            Finalize training strategy

                                              Page 9                                                                  GAO-03-984T
(Continued From Previous Page)
                                                                                                         Key tasks to be
                                                                                                         completed during
                                                                                                         18-month national
AeDib           Tasks completed as of      Tasks to be completed by         Planned January 2004 project rollout (2/2004–
component       June 30, 2003              January 30, 2004                 status                       7/2005)
Internet        Developed and released     Complete Internet form 3820      Public will have Internet access     None reported.
disability      into production Internet   (child)                          to disability applicationsa
applications    form 3368 (disability
                report)                    • Validation                     • i3368 (disability report)

                Developed and released • Integration testing                • i827 (authorization to release
                into production Internet                                      information)
                form 827 (authorization to • Pre-release tasks
                release information)                                        • i3820 (child)
                                           Complete Internet form 3369
                Drafted training strategy (work history)                    • i3369 (work history)

                                           • Construction including         • i3441 (appeals)
                                             hardware, capacity
                                             management, security support   Data will be automatically
                                             activities                     generated to EDCS from the
                                                                            i3368 (disability report), i3820
                                           • Software development           (child), i3369 (work history), and
                                                                            i3441 (appeals).
                                           • Software validation

                                           • Integration testing

                                           • Pre-release tasks

                                           • Complete Internet form 3441

                                           • Construction including
                                             hardware, capacity
                                             management, security support

                                           • Software development

                                           • Validation

                                           • Integration testing

                                           • Pre-release tasks

                                             Page 10                                                                      GAO-03-984T
(Continued From Previous Page)
                                                                                                                                 Key tasks to be
                                                                                                                                 completed during
                                                                                                                                 18-month national
AeDib                   Tasks completed as of              Tasks to be completed by                 Planned January 2004 project rollout (2/2004–
component               June 30, 2003                      January 30, 2004                         status                       7/2005)
DDS AS/400        Contracted for AS/400                    Complete migration of iLevy              Enhanced legacy systems                Install DMA
migration and     migration and electronic                 software                                 prepared to interface with             infrastructure based
electronic folder folder interface                                                                  electronic folder                      on rollout schedule
interface                                                  Complete procurement of IBM                                                     and strategy
                  Installed AS/400                         AS/400 upgrades
                  hardware                                                                                                                 Complete changes to
                                                           Complete IBM install upgrades                                                   New York and
                        Provided core AS/400                                                                                               Nebraska legacy
                        training                           Complete AS/400 training                                                        systems to interface
                                                                                                                                           with electronic folder
                        Drafted training strategy          Install print servers
                                                           Complete software changes to
                                                           support electronic folder interface

                                                           Conduct process evaluation
Hearings and            Documented business                Complete post-demo                       CPMS stand-alone software              Complete
appeals Case            process description                requirements                             without management                     development of
Processing                                                                                          information functionality and          CPMS management
Management              Developed software                 Conduct system validation                prepared to interface with             information
System                  development plan                                                            electronic folder                      functionality
(CPMS)                                                     Conduct integration and
                        Developed pre-demo                 environmental testing                                                           Roll out CPMS
                        requirements                                                                                                       software to hearings
                                                           Establish CPMS software for pilot                                               and appeals sites
                        Drafted training strategy                                                                                          starting in March
                                                           Conduct pilot training                                                          2004

                                                           Conduct pilot                                                                   Conduct training

                                                           Begin pilot evaluation                                                          Install DMA
                                                           Begin to address any pilot issues

                                                           Finalize training strategy
Source: GAO analysis of SSA AeDib project documentation.
                                                              SSA reported that Disability Internet form i454 (Continuing Disability Review) is being revised and will
                                                             not be available by January 2004.

                                                             As the table reflects, SSA’s electronic disability claims process hinges on
                                                             accomplishing numerous critical tasks by the end of January 2004. In
                                                             discussing the overall progress of the initiative, SSA officials in the Offices
                                                             of Systems and Disability Programs acknowledge that the agency will be
                                                             severely challenged to accomplish all of the tasks planned for completion
                                                             by the end of January. Nonetheless, they believe that SSA will meet the
                                                             targeted project completion dates, stating that the agency has conducted

                                                             Page 11                                                                                   GAO-03-984T
                          the necessary analyses to ensure that the accelerated schedule can
                          accommodate the project’s scope.

Risks in Developing the   Beyond meeting an ambitious project implementation schedule, SSA must
                          ensure that the system it delivers successfully meets key business and
Electronic Disability     technical requirements for reliably exchanging data among disability
System Increase           processing components and is protected from errors and vulnerabilities
                          that can disrupt service. Accomplishing this necessitates that SSA conduct
AeDib’s Overall           complete and thorough testing to provide reasonable assurance that
Vulnerability             systems perform as intended. These include tests and evaluations of pilot
                          projects to obtain data on a system’s functional performance and end-to-
                          end tests to ensure that the interrelated systems will operate together
                          effectively. In addition, the success of the system will depend on the agency
                          identifying and mitigating critical project risks.

                          SSA plans to rely on pilot tests and evaluations to help guide business and
                          technical decisions about the electronic disability folder, including critical
                          decisions regarding the document management technology. For example,
                          SSA stated that the Document Management Architecture pilots will be used
                          to test electronic folder interface requirements and DDS site configurations
                          for AeDib national implementation. In addition, the pilots are expected to
                          test the business process and work flow associated with incorporating the
                          Document Management Architecture. SSA has stated that this information
                          is crucial for determining whether the technology selected for the
                          Document Management Architecture will adequately support the electronic

                          However, SSA may not be able to make timely and fully informed decisions
                          about the system based on the pilot test results. The pilot tests were to
                          begin this month, and some of the test results upon which decisions are to
                          be based are not expected to be available until the end of December at the
                          earliest,5 leaving little time to incorporate the results into the system that is
                          to be implemented by late January. Further, even when completed, the pilot
                          tests will provide only limited information about the electronic folder’s
                          functionality. SSA stated that they will not test certain essential aspects of
                          the folder usage, such as the DDS’s disability determination function. Thus,
                          whether SSA will have timely and complete information needed to make

                           SSA plans to conduct the pilot tests at three state DDS sites—North Carolina, Illinois, and
                          California—beginning this month. It plans to complete the tests in December.

                          Page 12                                                                        GAO-03-984T
decisions that are essential to developing and implementing the electronic
disability folder is questionable.

In addition, given the technological complexity of the AeDib project, the
need for end-to-end testing is substantial. Our prior work has noted the
need for such testing to ensure that interrelated systems that collectively
support a core business area or function will work as intended in a true
operational environment.6 End-to-end testing evaluates both the
functionality and performance of all systems components, enhancing an
organization’s ability to trust the system’s reliability. SSA’s development and
use of new electronic tools to integrate an electronic folder with its own
and DDS legacy systems, along with Web-based applications and the new
Document Management Architecture, elevates the importance of ensuring
that all parts will work together as intended.

However, the agency currently has not completed a test and evaluation
strategy to conduct end-to-end testing to demonstrate, before deployment,
that these systems will operate together successfully. They added that
conducting end-to-end testing would require delaying system
implementation to allow the time needed for a claim to be tested as it
moved through all of the disability components—a process that could take
up to 6 months to complete. However, determining that all AeDib
components can correctly process disability claims when integrated is vital
to SSA’s knowing whether the electronic disability system can perform as

Compounding AeDib’s vulnerability is that SSA has not yet undertaken a
comprehensive assessment of project risks to identify facts and
circumstances that increase the probability of failing to meet project
commitments, and taking steps to prevent this from occurring. Best
practices and federal guidance7 advocate risk management. To be
effective, risk management activities should be (1) based on documented
policies and procedures and (2) executed according to a written plan that
provides for identifying and prioritizing risks, developing and implementing
appropriate risk mitigation strategies, and tracking and reporting on

 U.S. General Accounting Office, Year 2000 Computing Crisis: FAA Is Making Progress
But Important Challenges Remain, GAO/T-AIMD/RCED-99-118 (Washington, D.C.: March
15, 1999).
 See, for example, Software Acquisition Capability Maturity ModelSM (CMU/SEI-99-TR-
002, April 1999); OMB Circular A-130 (November 30, 2000).

Page 13                                                                  GAO-03-984T
                       progress in implementing the strategies. By doing so, potential problems
                       can be avoided before they manifest themselves into cost, schedule and
                       performance shortfalls.

                       SSA has developed a risk management plan to guide the identification and
                       mitigation of risks, and based on that plan, has developed a high-level risk
                       assessment of program and project risks. The high-level assessment, which
                       SSA issued last February, identified 35 risks that the agency described as
                       general in nature and addressing only overall program management issues
                       related to the project’s costs, schedule, and hardware and software. For
                       example, one of the high-level risks stated that the overall availability of the
                       Document Management Architecture might not meet service-level
                       commitments. The related mitigation strategy stated that the agency
                       should continue to investigate various approaches to ensure the system’s

                       SSA has acknowledged the potential for greater risks given the electronic
                       case processing and technological capability required for AeDib. Further, in
                       response to our inquiries, its officials stated that the agency would conduct
                       and document a comprehensive assessment of project risks by June 30 of
                       this year. The officials added that AeDib project managers would be given
                       ultimate responsibility for ensuring that appropriate risk-mitigation
                       strategies existed and that SSA had tasked a contractor to work with the
                       managers to identify specific risks associated with each system
                       component. However, at this time, SSA is still without a comprehensive
                       assessment of risks that could affect the project. Until it has a sound
                       analysis and mitigation strategy for AeDib, SSA will not be in a position to
                       cost-effectively plan for and prevent circumstances that could impede a
                       successful project outcome.

Unresolved             Integral to AeDib’s success are disability process stakeholders that SSA
                       relies on to fulfill the program’s mission, including state disability
Stakeholder Concerns   determination officials and medical providers. As primary partners in the
Could Undermine        disability determination process, stakeholders can offer valuable and
                       much-needed insight regarding existing work processes and information
AeDib’s Success        technology needs, and their stake and participation in the systems
                       development initiative is essential for ensuring its acceptance and use. In
                       assessing lessons learned from SSA’s earlier attempt to implement the
                       failed Reengineered Disability System, Booz-Allen and Hamilton
                       recommended that SSA at all times keep key stakeholders involved in its
                       process to develop an electronic disability processing capability.

                       Page 14                                                             GAO-03-984T
SSA disability program and systems officials told us that the agency has
involved its various stakeholders in developing AeDib. They stated that the
agency has entered into memorandums of understanding for data sharing
with state DDSs, established work groups comprising DDS representatives
to obtain advice on development activities, and included these
stakeholders in steering committee meetings to keep them informed of the
project’s status. In addition, SSA stated, it has met with representatives of
major medical professional associations to seek their support for SSA’s
requests for releases of medical evidence.

However, officials that we contacted in nine of the ten DDS offices stated
that their concerns were not adequately heard and considered in the
decision-making process for the development of AeDib, despite the critical
and extensive role that states play in making disability determinations.
Because of this limited involvement, the National Council of Disability
Determination Directors, which represents the DDSs, stated that they were
concerned that SSA may be pursuing an automated disability strategy that
could negatively affect business operations by creating delays in the ability
to make decisions on disability cases. The DDS representatives stated that
SSA has not articulated a clear and cohesive vision of how the disability
components will work to achieve the AeDib goal and that decisions about
AeDib were being made without considering their perspectives. They
explained, for example, that SSA’s decision to use a scanning and imaging
vendor to whom medical providers would have to submit evidence would
introduce an additional step into the disability process, and might result in
DDSs’ not being able to effectively manage the critical information that
they need to make disability determinations. Further, they have questions
about how in the disability process evidence will be electronically stored,
noting that SSA has proposed, but not yet decided among, three possible
scenarios for establishing repositories to house medical evidence.

Last March, the National Council of Disability Determination Directors
made three suggestions to SSA aimed at allowing the DDSs to have greater
responsibility for this aspect of the disability business process. Among
their proposals was that DDSs (1) be allowed to manage the contractors
who will be responsible for scanning and imaging all records received from
medical providers; (2) have the choice of receiving electronic medical
evidence at a repository maintained at their sites rather than at remote,
centralized locations; and (3) be allowed to test the possibility of scanning
records after, rather than before, the DDS adjudicates a claim. According to
the council, this latter approach would ensure that the DDSs could make
timely and accurate disability determinations, while also allowing SSA the

Page 15                                                           GAO-03-984T
time to perfect the electronic business process and transition to the initial
case process. As of last week, however, SSA had not responded. For its
part, SSA stated that it is reviewing, but has not yet taken a position on, the
council’s proposals.

SSA’s consultation with the medical community (physicians and other
sources of medical evidence used to evaluate disability claims) also has
been limited. These stakeholders are critical, as they represent the basic
source of most of the information that states use to evaluate an individual’s
disability. One of the key savings that SSA anticipates from AeDib is based
on physicians and other medical sources electronically transmitting or
faxing medical evidence that is now mailed to the DDSs. SSA has estimated
that as much as 30 percent of all medical evidence could be faxed or
electronically received from these providers, with the majority of it being
faxed. In speaking with American Health Information Management
Association officials in Georgia and Wisconsin, however, they expressed
concern about the possibility that SSA will want medical providers to fax
evidence. They cited the voluminous nature of much of the medical
evidence that they send to the DDSs, and believe that faxing it would be too
costly and not secure.

Our review to date has not assessed the validity of the concerns expressed
by the stakeholders, or SSA’s responses to them. Nonetheless, as long as
such concerns exist, SSA must be diligent in pursuing a mutually agreed-
upon understanding with its stakeholders about its vision and plan of
action being pursued. SSA’s success in implementing AeDib depends
heavily on resolving all outstanding issues and concerns that could affect
the use and, ultimately, the outcome of the intended electronic capability.
Without stakeholders’ full and effective involvement in AeDib’s planning
and development, SSA cannot be assured that the system will satisfy
critical disability process requirements and be used as intended to achieve
desired processing efficiencies and improved delivery of services to

To summarize, Mr. Chairman, in moving toward an electronic disability
process, SSA has undertaken a positive and very necessary endeavor.
Having the means to more effectively and efficiently provide disability
benefits and services is essential to meeting the needs of a rapidly aging
and disabled population, and we applaud the Commissioner’s
determination and proactive pursuit of this service-delivery enhancement.

Page 16                                                             GAO-03-984T
                   Nonetheless, SSA’s accelerated strategy may involve risks of delivering a
                   system that will not sufficiently address its needs. The execution of critical
                   pilot tests that are not scheduled for completion until December or later,
                   coupled with the lack of planned end-to-end testing and a comprehensive
                   assessment of risks, may prevent SSA from delivering an information
                   technology capability based on sound and informed decision making.
                   Moreover, uncertainties about the successful outcome of this project are
                   exacerbated by concerns that key stakeholders in the disability process
                   continue to have. Given the importance of this project to SSA’s future
                   service-delivery capability, it is essential that the agency satisfy itself that
                   AeDib will perform as intended with minimal risk before it is deployed
                   nationwide. We will continue to monitor SSA’s progress on this initiative as
                   part of our ongoing review.

                   This concludes my statement. I would be happy to respond to any
                   questions that you or other members of the Subcommittee may have at this

GAO Contacts and   For information regarding this testimony, please contact Linda D. Koontz,
                   Director, or Valerie Melvin, Assistant Director, Information Management
Staff              Issues at (202) 512-6240. Other individuals making key contributions to this
Acknowledgments    testimony include Michael Alexander, Tonia D. Brown, Derrick Dicoi, and
                   Mary J. Dorsey.

(310361)           Page 17                                                             GAO-03-984T
This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.

Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to
                         e-mail alerts” under the “Order GAO Products” heading.

Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061

To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470

Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548