Office of Inspector General Loan Account Reporting System 03-01 August 2003 August 28, 2003 The Honorable Michael M. Reyna Chairman of the Board and Chief Executive Officer Farm Credit Administration 1501 Farm Credit Drive McLean, Virginia 22102-5090 Dear Mr. Reyna: The Office of the Inspector General completed an audit of the Loan Account Reporting System (LARS). The overall objective of this audit was to determine whether LARS is effectively utilized. Specific audit objectives included (1) determining the adequacy of LARS data; (2) determining whether information provided is duplicative; and (3) assessing LARS future data use. We found that LARS is not providing sufficient and efficient loan data to support the Agency’s mission. Specifically LARS data is deficient, the system is outmoded and system improvements is challenging. As a result LARS is under-utilized and without significant reengineering we question whether the system is effective in realizing Agency goals and mission. We conducted the audit in accordance with Government Auditing Standards issued by the Comptroller General for audits of Federal organizations, program, activities, and functions. We conducted fieldwork from February 2003 through May 2003. We provided a draft report to management on July 22, 2003. We conducted an exit conference and discussed the draft report with the Office of Policy and Analysis on July 23, 2003. The management response to the report indicated general agreement and is included. Where actions were presented to the Office of Inspector General that would resolve audit findings, the recommendation was changed to agreed upon action. If you have any questions about this audit, I would be pleased to meet with you at your convenience. Respectfully, Stephen G. Smith Inspector General TABLE OF CONTENTS BACKGROUND _________________________________________________________________________1 OBJECTIVES AND SCOPE _______________________________________________________________1 FINDINGS AND RECOMMENDATIONS______________________________________________________2 Information System Investment ___________________________________________________________ 2 Loan Account Reporting System Data ______________________________________________________ 2 Directing Resources ___________________________________________________________________ 4 Roles and Responsibilities _______________________________________________________________ 5 Appendix 1: LARS Survey Results _________________________________________________________7 MANAGEMENT'S RESPONSE BACKGROUND In 1986 the Loan Account Reporting System (LARS) became operational with the quarterly collection of detailed borrower data from the Farm Credit System (FCS) institutions. The LARS objective was to provide the Farm Credit Administration (Agency) with accurate loan information needed to successfully carry out its responsibilities in examining, regulating and supervising FCS institutions. Historically, LARS provided information that helped in the statistical selection of loans during on-site examinations and in the institution’s loan loss evaluation. In April 1987, after an Office of Examination (OE) review, LARS was revised. The modified system reduced the data elements to 30 common variables. These 30 variables were identified as critical to OE review responsibilities. In March 1993, LARS was further revised to 29 data elements. Since 1993 LARS has not been revised. In January 1998, the Office of Policy and Analysis (OPA) began managing LARS. LARS is primarily used for monitoring, analyzing policy, reporting purposes, and responding to Congressional and public inquiries. In Fiscal Year (FY)’03 the LARS redesign project began. According to the Information Resource Management (IRM) Project data the estimated redesign project cost is $99,000 for FY’03-04. The overall project objective is to provide FCS institution loan level data needed in conducting examinations, monitoring portfolio risk, modeling loan performance relationships and conducting risk analysis. According to the IRM Project data specific objectives include: • Converting the existing LARS data collection process into a more useful and easy-to-use electronic data source. • Providing continuous access to loan level trend data on a consistent basis. • Coordinating and working with the OE to develop a loan level data collection system that meets the needs for examination purposes including integrating this effort with the OE modernization project. • Reducing the reporting burden on FCS institutions by coordinating the Agency’s collection of loan level data. The project will span over two years. OPA has established a work group that includes OPA, OE, Office of the Chief Information Officer and Office of the General Counsel representatives. The work group’s first objective is to publish a notice to the public, with request for comments on how best to modify the existing LARS requirement (i.e., data elements). The work group plans to have the notice published in 2003. OBJECTIVES AND SCOPE The objective of this audit is to determine whether LARS is effectively utilized. Specific audit objectives include (1) determining the adequacy of LARS data; (2) determining whether information provided is duplicative; and (3) assessing LARS data future use. We reviewed LARS documentation dating from July 1990 through May 2003 including background information, feasibility studies, mission and operational needs and budget documents. We conducted interviews with Agency personnel responsible for maintaining LARS data and key personnel who use LARS data. In addition, we conducted a survey to assess satisfaction with the LARS -1- program. We reviewed 37 survey responses from Agency users. We also discussed the loan data collection system with another federal government regulatory agency. FINDINGS AND RECOMMENDATIONS LARS is not providing sufficient and efficient loan data to support the Agency’s mission. Specifically LARS data is deficient, the system is outmoded and system improvement is challenging. As a result, LARS is under-utilized and without significant reengineering, we question whether the system is effective in realizing Agency goals and mission. Information System Investment LARS is rooted in the past, aimed at earlier needs. Thus, LARS is poorly positioned to meet the Agency’s current and future goals. The LARS redesign project is focused on improving the Agency’s loan-level data collection process. According to the project manager, efforts will be focused on designing a process that is the greatest benefit to the Agency and its customers. The drive for realizing dramatic improvements should focus on fundamentally rethinking how the Agency works as a whole. The Agency must begin managing LARS as an information management system investment. By looking at LARS as an information management system investment, attention is placed on assessing and managing risks between LARS continued funding or focusing resources on other potential loan data collection systems. The decision should be based on an explicit set of criteria that answer questions such as: • Is LARS providing the Agency with essential loan data? • Are resources directed toward improvement that will achieve the greatest benefit? • Are the right people involved in the decision making process? By answering these types of questions the Agency can better assess which core processes need improvement to fulfill mission goals, satisfy customers and stakeholders’ needs, reduce costs and provide high quality products and services. Loan Account Reporting System Data LARS is an under-utilized loan data system. Most staff feel the system is no longer an effective tool in providing them with pertinent loan data information. Specifically, LARS data is not comprehensive, data accuracy is questionable and ease of use could be improved. To assess LARS we sent a survey to 87 employees who had access to the system. The survey received a 43% response rate. Based on survey results, users expressed dissatisfaction with the completeness, usefulness, relevance, accuracy, and ease of use. -2- LARS Survey Results 70% 60% 50% 40% Dissatisified 30% Some Dissatisifcation 20% 10% 0% y s s e se es es ac nc fU ln en ur a ev fu o et cc el se se pl A R Ea U om C More detailed survey results are in Appendix 1. Comprehensive Data. LARS does not provide comprehensive loan data information needed to meet the Agency’s current and future goals and mission. LARS has not been revised since 1993 therefore much of the information provided is based on past loan data needs. According to LARS users, the data collected needs a reassessment to improve comprehensiveness and relevance. Currently, LARS collects data on most FCS loans but not all. LARS excludes information on cooperative loans, participations and syndications. Participation and syndications are among the fastest growing areas in the FCS. According to users, to be effective LARS should collect data on all FCS loans. In addition, information should be pertinent to Agency needs. Most examiners dismiss LARS as a key information source because they believe LARS data is not useful. A senior manager stated that only 50% of LARS information is useful in making important management decisions. Economists and analysts also stated more basic borrower information would be helpful. Finally, a redesign should include useful information in future regulatory requirements. For example, LARS data needs to include useful Basel Capital Accord requirements information. The OPA is trying to improve LARS loan data information. The LARS redesign project includes an objective to determine the Agency’s loan data needs. As part of that objective emphasis will be placed on capturing all loans. We commend the OPA for taking actions to determine loan data needs as part of the redesign project. However, to ensure loan data information remains relevant in the future, OPA should perform annual reviews on LARS data. Accurate Data. To be an effective information management system, LARS must provide the Agency with accurate loan information. Without accurate information, data analysis could result in erroneous decisions and improper conclusions. LARS users believe the data definitions are sometimes interpreted differently resulting in discrepancies in reported data. In addition, some FCS institutions expressed concerns that they understood LARS definitions and requirements differently and therefore discrepancies may appear in data reported. For example, some FCS institutions have used various commodity codes to report the same commodity. This can result in -3- masking high commodity concentrations. These types of discrepancies undermine the data creditability and can result in misrepresented and interpreted information. LARS system managers have taken steps to improve data accuracy. For example, systematic checks are performed to ensure various data elements meet the definitions. In addition, quarterly comparison checks are run at the district levels comparing various results with current quarter results. If the comparison check indicates a significant change from previous quarters, a follow-up is conducted to determine if the change was accurate. However, because LARS data is used in significant Agency services such as responses to Congressional requests and Board briefings, FCS institutions should certify data accuracy and LARS data should be reviewed during on-site examinations. These checks will require the Agency to view LARS as a cross-functional process requiring resources from the OPA and OE. Ease of Use. Improvements are needed in the LARS program to make it user-friendly. Because of the large volume of information contained within LARS, users find it difficult to abstract data and time consuming to find specific information. For example, one district has approximately 274,000 records. Very few FCA staff are knowledgeable about how to write queries to produce reports from these records. In addition, without knowing where to look for information, searching and determining where data is can be time consuming. Users suggest that standard reports be available without writing queries and a user guide should be on-line. In addition, splitting LARS information into more usable tables, possibly by institution, could facilitate finding data. Also, LARS does not maintain historical data. Therefore, comparisons across time periods cannot be easily made. Because of the data volume maintained each quarter, the feasibility of storing more than one quarter of information on-line is limited. Alternatives to providing historical information must be considered. For example, summary reports that include pertinent historical information could be posted on the Plans and Reports database. Agreed Upon Actions 1. The Office of Policy and Analysis should establish a process to ensure LARS information remains relevant and useful. 2. The Office of Policy and Analysis should require FCS institutions to certify LARS data accuracy. 3. The Office of Policy and Analysis should coordinate with the Office of Examination to validate LARS data during on-site institution examinations. 4. The Office of Policy and Analysis should include in the redesign project the development of standard reports. LARS users should provide input on reports that would be useful. 5. The Office of Policy and Analysis should develop a LARS users’ manual and place the manual on-line. Directing Resources Agency resources should focus on projects that maximize technology potential to improve performance. Within the past 10 years, FCS institutions’ accounting systems have changed dramatically allowing examiners to have direct access to institutions’ loan data. Building on the -4- LARS concept, various offices within the OE have developed their own loan databases that are used during examinations of institutions in their districts. Over time these databases have become an integral part of the examination process because they are more comprehensive and accurate than LARS. However, the type of data collected and the way it is presented can vary from office to office. Because each institutions system varies, information is not standardized and system-wide loan analysis can be difficult. In addition, data security cannot be assured on loan information collected by examiners. FCS institutions report loan data in a consistent standardized format in LARS thereby making system-wide analysis easier. However, given LARS weaknesses, system-wide analysis using the data can be incomplete and questionable. Therefore, as part of the LARS redesign effort, the Agency should have a comprehensive understanding of whom its customers are and their needs and expectations. This is essential for improving the loan data collection process. Resources should be expended in system development that will have the greatest Agency impact. Before investing significant funds in redesigning LARS, consideration should be given to leveraging off existing loan databases to obtain necessary data. In addition, more emphasis needs to be placed on trying to integrate the Agency’s various loan data collection processes. According to the LARS system managers, the redesign project will include reviewing various loan level data collection processes to assess the risks, costs and benefits of each. Given this, prior to expending significant funds on redesigning LARS, Board members should be provided information on the various processes reviewed and the Chief Executive Officer should decide on the system that best meets the Agency goals and mission. Agreed Upon Actions 6. The Office of Policy and Analysis should report to the Farm Credit Administration Board on the risks, costs and benefits of possible loan data collection systems reviewed during the redesign project. The Chief Executive Officer should decide on the system that best meets the Agency goals and mission. 7. The Office of Policy and Analysis should demonstrate through performance measures how LARS is improving Agency operations and mission effectiveness. Roles and Responsibilities If the Agency decides to continue with the LARS redesign project, the chief executive officer needs to understand the various organizational changes involved. LARS system managers are working diligently to develop a project plan, however, given LARS decreased use and high user dissatisfaction, Agency staff may not be fully committed to a redesign project. Reengineering projects need support with the right roles, responsibility and skills. Historically, efforts to improve LARS have produced minimal results. For example, a LARS Feasibility Study conducted in 1991 addressed some of the same issues that remain today such as data accuracy and ease of use. Further, in 1994 a review was conducted to produce standard reports from LARS without much success. A significant amount of resources were wasted on these projects because they lacked the necessary Agency commitment to ensure their success. Roles. For a project to be a success the Agency must create ownership. Studies have shown that chief executives who played a strong leadership role in improving information technology programs have resulted in successful projects. Chief executives’ involvement helps reinforce information management decisions, accountability and focuses system development on activities that promote measurable mission outcomes. Without such accountability, it is easy for the project to become delayed and fail to reach the initial goal. -5- Responsibility. It is critical that the LARS redesign project incorporate the right staff into the planning process. Given the various offices involved in the Agency loan data collection process, work group members should include staff representation to facilitate communication and maintain momentum during project development. In addition, because the redesign project will require work process innovation, cutting across program and functional lines; accountability must also be aligned with a decision-making authority that is able to raise issues above existing stovepipes. Therefore, the LARS project oversight should be at a level that can eliminate bottlenecks and resolve thorny operational issues that can easily stall improvement initiatives. By aligning project responsibility with Agency authority, staff will work effectively to change processes. Technical skills. Because of the project intricacy, the work group needs to include Agency staff that has the technical skill and informational knowledge to assist with project development. Work group members need to include Agency staff knowledgeable in the various loan database systems already developed within the Agency. For example, work group members should include examiners who have developed the various OE loan database systems. In addition, the work group should include a member who is knowledgeable in institutions’ loan accounting systems. Further, the Office of the Chief Information Officer should be involved in the process early on to facilitate future development plans. If the Agency feels it lacks the necessary resources to carry out the project, consideration should be given to involving a consultant to help guide the Agency’s effort. However, before pursuing a consultant, the Agency should have a strong business case that the redesigning effort is the best alternative to meeting the Agency needs. In addition, the Agency should have enough of a skill base so it can retain leadership and control over the project. Agreed Upon Actions 8. The Office of Policy and Analysis should provide periodic reports on the LARS redesign effort to the Farm Credit Administration Board members. 9. As part of the LARS redesign project, the Office of Policy and Analysis should conduct an assessment to determine whether the Agency has adequate resources to support the project. The assessment should identify the necessary skills and knowledge needed to develop the project. In addition, a determination should be made on whether the Agency employees that possess those skills and knowledge will be available to assist with system development. -6- Appendix 1: LAR SURVEY RESULTS The chart below shows users’ responses to survey questions concerning LARS information and content. The survey was sent to 87 users who had access to LARS. We received 37 responses (43% response rate). Ten users who responded to the survey did not express an opinion on the issues listed below because they use other sources for loan-level data. The majority of those who did not express an opinion were examiners. Very Some Satisfied Satisfied Dissatisfaction Dissatisfied Completeness 0 5 10 12 Usefulness 0 6 12 9 Relevance 0 10 11 6 Accuracy 0 10 10 7 Ease of Use 0 12 11 4 -7- Management Response Farm Credit Administration 1501 Farm Credit Drive McLean, Virginia 22102-5090 (703) 883-4000 August 26, 2003 Mr. Stephen Smith, Inspector General Office of the Inspector General Farm Credit Administration 1501 Farm Credit Drive McLean, VA 22102-5090 Dear Mr. Smith We appreciate the opportunity to comment on the Office of The Inspector General's (GIG) audit report on LARS. Overall, the report is a reasonable discussion of the Farm Credit Administration's (agency) current use of LARS and the data it provides for policy analysis and supervisory monitoring purposes. We agree with the nine recommendations contained in the draft report, and will implement corrective actions as soon as possible. We also expect the report to add value to our work on the LARS Redesign effort already underway in the agency. We agree with the report's conclusions about the comprehensiveness of LARS data in providing detailed loan level information. The data limits of the current LARS are understood and recognized. For instance, the current LARS specification of loan level data excludes participations and loans made by CoBank, ACB. We also acknowledge the need to follow a comprehensive and inclusive approach to identifying changes to loan level data requirements of the agency as we move to re-engineer LARS. We also concur with the report's conclusion that LARS data needs to be accurate and that users currently believe the data is inaccurate. We also want to thank you for noting that we have taken steps to improve data accuracy. CPA will continue to work to implement cost- effective and efficient approaches to strengthening the accuracy and quality of LARS data. The report also cited concerns with LARS' ease of use and the need for standardized and historical reports. While we agree in general that such changes are needed to reach a broader user population, we wish to point out that users can easily access LARS data and download information for analysis and monitoring needs. LARS is fully integrated in the agency's information systems. However, we acknowledge that we can improve in this area. We plan to focus our LARS Redesign project on improving substantially the ease of accessing and using loan level information in summary and other reports. Meanwhile, we are considering simpler ways in making the LARS loan level information available to a broader user group. We agree with the report's conclusion that we need to review how we collect loan level data from Farm Credit System (System) institutions and collaborate closely with OE on their needs, including how they collect loan level data for examination purposes. OPA is committed to developing a loan level data collection system that is cost-effective and efficient. Fundamental to this commitment is reducing the reporting burden on the System and meeting the information needs of critical user groups. As we move forward with the LARS Redesign project, we are focused on re-engineering the current collection system by considering new approaches and possible avenues to coordinate loan level data collection agencywide. Throughout the LARS audit, OIG staff was courteous and responsive to OPA's questions and suggestions. We appreciate their efforts and look forward to working together as we move forward in implementing the recommendations contained in the report. Thank you and your staff for this valuable insight. I believe it will help us develop a more viable and useful LARS and contribute to the overall efficiency and effectiveness of the agency. Sincerely, /signed/ Michael V. Dunn, Director Office of Policy and Analysis -1- REPORT FRAUD WASTE ABUSE MISMANAGEMENT FARM CREDIT ADMINISTRATION INSPECTOR GENERAL OFFICE TOLL FREE 1-800/437-7322 Washington , DC Area 703/883-4316 E-Mail email@example.com 1501 Farm Credit Drive McLean, Virginia 22102-5090
Loan Account Reporting System
Published by the Farm Credit Administration, Office of Inspector General on 2003-08-28.
Below is a raw (and likely hideous) rendition of the original report. (PDF)