Posthearing Questions Related to Strategic Human Capital Management

Published by the Government Accountability Office on 2003-05-22.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                                                                  Comptroller General
                                                                                  of the United States
United States General Accounting Office
Washington, DC 20548

          May 22, 2003

          The Honorable George V. Voinovich
          The Honorable Richard J. Durbin
          Ranking Minority Member
          Subcommittee on Oversight of Government Management,
            the Federal Workforce, and the District of Columbia
          Committee on Governmental Affairs
          United States Senate

          The Honorable Jo Ann Davis
          The Honorable Danny Davis
          Ranking Minority Member
          Subcommittee on Civil Service and Agency Organization
          Committee on Government Reform
          House of Representatives

          Subject: Posthearing Questions Related to Strategic Human Capital Management

          On April 8, I testified before your Subcommittees at a hearing on “The Human Capital
          Challenge: Offering Solutions and Delivering Results”.1 This letter responds to
          requests from Chairman Voinovich, Chairwoman Davis, and Senator Carper that I
          provide answers to follow-up questions from the hearing. The questions, along with
          my responses, follow.

          Questions from Chairman Voinovich

          1. In your testimony, you discuss the need to preserve and share an agency’s
          institutional knowledge in light of the impending retirement wave by using
          various means of “phased retirement.” Proposed in my Federal Workforce
          Flexibility Act are provisions to make it more desirable for individuals to
          work part-time at the end of their career. What other options might
          Congress consider to ease this transition?

          There are a variety of options that should be considered; however, they all require
          further analysis to ensure they would be cost-effective in retaining the institutional
          knowledge of key employees as they approach and become eligible for retirement.

          U.S. General Accounting Office, Human Capital: Building on the Current Momentum to Address
          High-Risk Issues, GAO-03-637T (Washington, D.C.: Apr. 8, 2003).

                                                                        GAO-03-779R Human Capital
For example, one option would be to allow retirement eligibles to continue working
after retirement eligibility on a part-time basis and factor in that service when making
pension calculations. However, for those employees who are under the Civil Service
Retirement System, legislation would be needed to remove the so-called penalty on
their retirement annuity for working part time. Another option would be to allow
certain employees to retire and receive their annuity and continue to work part time
for up to a stated period of time (e.g., 1 – 2 years) without a pension offset. Both of
these options would help agencies address succession planning and knowledge
transfer challenges. In all cases, however, Congress should consider placing certain
criteria along with numerical or percentage of workforce caps on agencies’ use of
such authorities to prevent abuse.

2. GAO has been successful in implementing personnel reforms that clearly
emphasize the importance of human capital. Part of your success has been
achieved through GAO-specific legislation, but your agency also has been
successful in implementing governmentwide reforms. Unfortunately, other
agencies have not been as successful. As agencies begin to consider the
governmentwide flexibilities included in last year’s homeland security
legislation, please discuss in a bit more detail how GAO has been successful
in the past.

As noted in my statement for the hearing, we have reported on the capabilities that
agencies need to have in place to effectively use the human capital flexibilities and
authorities that Congress has provided. Our own approach to the use of the
authorities that Congress has provided us has been consistent with the practices of
leading organizations, as shown in figure 1.

 U.S. General Accounting Office, Human Capital: Effective Use of Flexibilities Can Assist Agencies
in Managing Their Workforces, GAO-03-2 (Washington, D.C.: Dec. 6, 2002).

Page 2                                                            GAO-03-779R Human Capital
Figure 1: Key Practices for Effective Use of Human Capital Flexibilities

In our specific situation, we developed data-driven business cases to demonstrate the
need for the authorities, how they would be used, and took steps to assure that
safeguards were in place to ensure their proper and fair use. We also have involved
our employees at all levels in the development and implementation of our human
capital policies and programs to improve the quality of those policies and programs
and build GAO-wide ownership for the changes we are making. Further, we have
used our human capital initiatives as part of, and consistent with, broader efforts we
have underway to instill a more results-oriented culture throughout GAO. Finally, we
monitor and evaluate our efforts and publicly report on our use of the authorities
both to assure our own accountability as well as to provide lessons for other

3. You have recommended the creation of a nonpolitical chief operating
officer at federal agencies. While your roundtable discussion did not achieve
consensus on the concept, it did lay out three “themes” that should be
considered. Would you like to elaborate a bit more on these themes and
what more Congress and agencies can do to facilitate this discussion?

GAO convened a roundtable on September 9, 2002, to discuss the Chief Operating
Officer (COO) concept and how it might apply within selected federal departments
and agencies as one strategy to address certain systemic federal governance and
management challenges. We reported that there was general agreement that the
following three themes provide a course for action.

 U.S. General Accounting Office, Highlights of a GAO Roundtable: The Chief Operating Officer
Concept: A Potential Strategy To Address Federal Governance Challenges, GAO-03-192SP
(Washington, D.C.: Oct. 4, 2002).

Page 3                                                          GAO-03-779R Human Capital
•   Elevate attention on management issues and transformational change.
•   Integrate various key management and transformation efforts.
•   Institutionalize accountability for addressing management issues and leading
    transformational change.

The participants also offered a number of ideas to help address agencies’
management weaknesses and drive transformational change. First, while there is no
“one size fits all” solution to address the challenges agencies face, the critical point is
to craft an approach in each case that (1) sets responsibility and accountability for
functional management issues and transformational change at an organizational level
appropriate for the types of reforms that are needed and (2) creates integrated
leadership responsibility in a single organizational position for key management
functions such as human capital, financial management, information technology,
acquisition sourcing strategies, and performance management as well as for
transformational change initiatives, if appropriate.

Second, participants suggested that Congress should make clear in statute the broad
responsibilities and qualifications for at least the senior official responsible for
management and transformation. Congress has taken this general approach with
other important management legislation, such as the Chief Financial Officers Act
(CFO), which requires CFOs to “possess demonstrated ability in general management
of, and knowledge of and extensive practical experience in financial management
practices in large governmental or business entities” and clearly lays out the CFOs’
responsibilities. By establishing the broad CFO responsibilities in statute, Congress
created a number of important advantages: unambiguous expectations for the
position, a professional approach, and an implicit set of qualification standards and

Third, participants also widely agreed that augmented accountability mechanisms are
needed to help assure the success of key management and transformational change
efforts. To help provide the continuing focused attention essential to successfully
completing multiyear transformational change, the important role that congressional
oversight has played and can play in fostering improvements was acknowledged at
the forum. Likewise, public reporting, such as the annual performance plans and
performance reports required by the Government Performance and Results Act and
audited financial statements under the CFO Act, can provide useful information on
agencies’ progress in meeting goals and addressing mission-critical management
challenges. The use of performance contracts for senior leaders was also recognized
as being a potentially important mechanism for clarifying expectations, monitoring
progress, and assessing accountability.

Questions from Chairwoman Davis

1. Far too often, when agency budgets are short, training is the first item cut.
The world’s top companies have a commitment to training because training
helps ensure the skills and performance necessary for an organization to
meet its goals. The Federal Workforce Flexibility Act, which was introduced
by both Senator Voinovich and myself, enhances the institutional manner in
which employees are trained in the federal government. How critical is

Page 4                                                      GAO-03-779R Human Capital
employee training to attract, retain and improve the performance of the
federal workforce?

A strategically sound training and development program is critical to both building
and retaining an effective federal workforce. Initiatives such as the Federal
Workforce Flexibility Act that Chairwoman Davis and Chairman Voinovich have
introduced are evidence that this area is beginning to receive the increased emphasis
and attention it warrants.

A learning environment, with opportunities to update and enhance an individual’s
skills and competencies, is an attractive feature for prospective and new employees
as well as for current and long-time employees. People want to work where they will
be able to continue to learn and grow as they develop their careers. As in the private
sector, effective training and development opportunities are an important element in
the package that the federal government needs to be able to offer in the “war for
talent.” Once employees are onboard, strategic training and development efforts can
help ensure that, as part of succession planning, new leaders are developed and ready
to take on new roles and assume greater responsibilities.

Training and development opportunities are also a tangible example of the need to
invest in human capital, both in terms of time and budgetary resources. The Federal
Workforce Flexibility Act calls for agencies to evaluate their training programs and
plans to ensure that they are linked to strategic and performance goals and contribute
to achieving the agency’s mission. This is a key step in making a business case and
demonstrating that training and development efforts, like other strategic human
capital management investments, are a vital and integrated part of agencies’ ability to
marshal, manage, and maintain the human capital needed to maximize government
performance and assure its accountability.

2. At our hearing on April 1, we talked about the crucial importance of
performance management improvements by federal agencies before a pay-
for-performance system will work well. At the same time, as the Deputy
Director of the Office of Personnel Management (OPM) pointed out last
week, the current system of lock-step raises provides no incentive for agency
managers to devote their attention to the difficult job of improving their
performance management. So we have a chicken-and-egg problem here. If
we wait for better appraisals before we get to pay for performance, we may
never get there. In your own experience at GAO, what amount of time
separated the first pay raises under a banded pay system and the first
appraisals under the new competency-based performance management
system? What approach would you recommend for this problem?

While there is growing agreement on the need to better link individual pay to
performance, experience has shown that moving too quickly—and prematurely—
significantly raises the risk of doing it wrong, which could severely set back the
current momentum. Thus, while it is imperative that we take steps to better link
employee pay to performance across the federal government, how it is done, when it
is done, and the basis on which it is done, can make all the difference in whether or
not such efforts are successful. In our own case, we began making significant

Page 5                                                  GAO-03-779R Human Capital
changes to our analyst performance management system in 2000 and made more
direct links between performance and pay as a result of the 2002 appraisal cycle. It
should be noted, that we had over a decade of experience in the use of pay banding
before we undertook our recent changes, so much of the needed organizational
infrastructure was already in place. With regard to executive branch agencies, the
key to progress over the next year is to create performance management systems that
are capable of supporting more performance based pay and other personnel
decisions. Such performance management systems are based on the practices used
by leading organizations to align individual performance with organizational success
and have the institutional infrastructures that provide adequate safeguards,
reasonable transparency, and appropriate accountability mechanisms.4 The critical
issue is not so much the length of time needed, but rather the extent to which agency
senior leadership is willing to provide the priority attention and targeted investments
needed to create and maintain a results-oriented performance management system.
Finally, in our view, appropriate systems and safeguards should be in place before
any additional pay for performance authority is actually implemented or
operationalized under this approach. Congress could authorize such authority while
providing that certain conditions must be met before the authority can be
implemented or operationalized.

3. Last week, when we asked the Deputy Director of OPM about the
desirability of eliminating the General Schedule altogether, he testified that
the Administration’s proposal for a $500 million Human Capital Performance
Fund was a “downpayment” on pay reform. Would you agree that the Human
Capital Performance Fund is a step in the right direction, if perhaps only a
baby step? Do you think the executive branch is capable of doing a good
enough job on performance appraisals to experiment on about one half of
one percent of pay? Maybe this is a way to get some energy into the agencies
when it comes to improving their performance management.

Modern, reliable, effective, and, as appropriate, validated performance management
systems with adequate safeguards, including reasonable transparency and
appropriate accountability mechanisms, must serve as the fundamental underpinning
of any successful results-oriented pay reform. Most executive branch agencies are a
long way from meeting this essential test.

To build the necessary performance management systems within agencies and to
create incentives for progress, I suggested in my testimony that Congress should
consider establishing a governmentwide fund where agencies, based on a sound
business case, could apply to OPM for funds to be used to modernize their
performance management systems and ensure that those systems have adequate
safeguards to prevent abuse. The basic idea is to provide for targeted investments
needed to prepare agencies to use their performance management systems as
strategic tools to achieve organizational results and drive cultural change. (If
successful, this approach to targeted investments could be expanded to foster and
support agencies’ related transformation efforts, including other aspects of the High

 U.S. General Accounting Office, Results-Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success, GAO-03-488 (Washington, D.C.: Mar. 14, 2003).

Page 6                                                          GAO-03-779R Human Capital
Performing Organization (HPO) concept recommended by the Commercial Activities

Congress should also consider providing the authority where whole agencies and/or
employee groups move to a pay for performance approach only after it has been
demonstrated to OPM that a modern, reliable, effective, and, as appropriate, validated
performance management system with adequate safeguards, including reasonable
transparency and appropriate accountability mechanisms, is in place to support pay
and related personnel decisions. In any case, Congress should consider establishing
statutory principles for the standards that an agency must have in place before OPM
can grant additional pay flexibilities. OPM, working with the Chief Human Capital
Officers’ Council, would issue guidance implementing legislatively-defined principles.

As a first step, the Senior Executive Service needs to lead the way in the federal
government’s effort to better link pay to performance. We have reported that there
are significant opportunities to strengthen efforts to hold senior executives
accountable for results.

4. I notice in your testimony that you believe OPM should play an aggressive
leadership role in the area of human capital management. If pay and
performance management systems are handed over to individual agencies so
they can tailor their systems to their particular needs and objectives, what
role in setting compensation policy would you foresee for OPM?

We have reported that OPM leadership is critical to accomplish its mission in a
decentralized human capital environment in which direct accountability for strategic
human capital management continues to shift to agencies. In particular, as noted
above, OPM should certify that an agency has a modern, effective, credible, and as
appropriate, validated performance management system in place before the agency is
granted the authority to better link pay to performance for broad-based employee
groups. In addition, OPM should gather, assess, and disseminate leading practices
from federal organizations on a full range of innovative human capital policies and
procedures, such as pay for performance. Further, OPM should build on its White
Paper to design and lead a broad research agenda to develop a more market-based
approach to federal pay.

5. What legislative changes do you think are most important for us to enact
in this Congress to improve the government’s human resources management
over both the short and long term? Or, if we could only get one change, what
should that be?

The need for results-oriented pay reform is one of the most pressing human capital
issues facing the federal government today. As implied in our answers above,
Congress could provide broad-based authority for broadbanding and pay for
performance along with specific statutory standards that would have to be met

  U.S. General Accounting Office, Results-Oriented Cultures: Using Balanced Expectations to Manage
Senior Executive Performance, GAO-02-966, (Washington, D.C.: Sept. 27, 2002).
  U.S. General Accounting Office, Major Management Challenges and Program Risks: Office of
Personnel Management, GAO-03-115 (Washington, D.C.: January 2003).

Page 7                                                          GAO-03-779R Human Capital
before such authorities can be operationalized. An agency should have to
demonstrate, and OPM should have to certify, that a modern, effective, credible, and,
as appropriate, validated performance management system with adequate safeguards,
including reasonable transparency and appropriate accountability mechanisms, is in
place to support more performance-based pay and related personnel decisions,
before the agency could implement a new system. OPM should be required to act on
any individual certifications within prescribed time frames (e.g., 30-60 days).

6. Similarly, where do you think we should most focus our oversight
attention in the area of human resources management?

Congress has had and will need to continue to have a central role in improving
agencies’ human capital approaches. First and foremost, congressional oversight is
important to ensure that agencies human capital plans and programs are integrated
with program missions and goals. Too often in the past, agencies’ program initiatives
were designed and implemented without due regard to the human capital
implications that these program decisions entail. In addition, Congressional
oversight is important to ensure that agencies effectively and properly use human
capital authorities and flexibilities that Congress has provided, for example those in
the Homeland Security Act. While agencies’ first priority should be to improve their
human capital management by using the authorities already available to them,
additional flexibilities may be appropriate where clear business cases have been

Questions from Senator Carper

1. In recent years, Congress has granted agencies like the Internal Revenue
Service and the Federal Aviation Administration some additional personnel
flexibilities aimed at meeting agency-specific problems. Just this year, the
National Aeronautics and Space Administration came to this committee with
its own set of problems and suggested some reforms aimed at recruiting
workers with advanced degrees in math and science. Would it be better for
Congress to address human capital issues on an agency-by-agency basis
rather than enacting broad reforms that may not be well suited for everyone?

We strongly support the concept of modernizing federal human capital policies,
including providing reasonable flexibility to management in this critical area provided
adequate safeguards are in place to prevent abuse. In this regard, we believe that
Congress should consider both governmentwide and selected agency changes to
address the pressing human capital issues confronting the federal government.
Agency-specific human capital reforms should be enacted to the extent that the
problems being addressed and the solutions offered are specific to a particular
agency (e.g., military personnel reforms for the Department of Defense). In addition,
targeted reforms should be considered in situations where additional testing or
piloting is needed for fundamental governmentwide reform.

In our view, it would be preferable to employ a governmentwide approach to address
certain flexibilities that have broad-based application (e.g., broadbanding, pay for
performance, part-time employment, reemployment annuities) and serious potential

Page 8                                                   GAO-03-779R Human Capital
implications for the civil service system, in general, and OPM, in particular. In these
situations, it may be prudent and preferable for Congress to provide such authorities
on a governmentwide basis and in a manner that ensures that appropriate
performance management systems and safeguards are in place before the new
authorities are implemented by the respective agency. This approach is not intended
to delay action on any individual agency’s efforts, but rather to accelerate needed
human capital reform throughout the federal government in a manner that ensures
reasonable consistency on key principles within the overall civilian workforce. This
approach also would provide agencies with reasonable flexibility while incorporating
key safeguards to help maximize the chances of success and minimize the chances of
abuse and failure. This approach would also help to maintain a level playing field
among federal agencies in competing for talent and avoid a further fragmentation of
civil service in key areas.

However, in all cases whether from a governmentwide authority or agency specific
legislation, in our view, such additional authorities should be implemented (or
operationalized) only when an agency has the institutional infrastructure in place to
make effective use of the new authorities. This institutional infrastructure includes,
at a minimum, a human capital planning process that integrates the agency’s human
capital policies, strategies, and programs with its program goals and mission, and
desired outcomes; the capabilities to effectively develop and implement a new human
capital system; and importantly, the existence of a modern, effective, credible, and
validated performance management system that includes adequate safeguards,
including reasonable transparency and appropriate accountability mechanisms, to
ensure the fair, effective, and nondiscriminatory implementation of the system.

2. In your view, how helpful can the streamlined hiring process authorized
through the Homeland Security Act be in helping agencies address their
personnel problems? How helpful can the other governmentwide personnel
provisions included in the bill be?

Much of the authority agency leaders need to manage human capital strategically is
already available under current laws and regulations, especially in connection with
modernizing existing performance appraisal and management systems. In that
regard, we believe that the hiring and other central provisions of the Homeland
Security Act will serve as important tools for agencies to use to address their hiring
and other human capital needs. For example, the Homeland Security Act included
significant provisions relating to categorical ranking when considering applicants,
direct hire authority, the creation of chief human capital officer (CHCO) positions
and a CHCO Council, an expanded voluntary early retirement and “buy-out”
authority, a requirement to discuss human capital approaches in Government
Performance and Results Act plans and reports, and a provision allowing executives
to receive their total performance bonus in the year in which it is awarded.

3. What are the specific safeguards against discrimination and abuse that you
believe need to be in place before an agency can implement a “pay for
performance” program?

To assist agencies in identifying available human capital flexibilities, OPM has published Human
Resources Flexibilities and Authorities in the Federal Government (Washington, D.C.: July 2001).

Page 9                                                           GAO-03-779R Human Capital
At the request of Representative Danny Davis, we developed an initial list of possible
safeguards for Congress to consider to help ensure that any pay for performance
systems in the government are fair, effective, and credible:

•   Assure that the agency’s performance management systems (1) link to the
    agency’s strategic plan, related goals, and desired outcomes and (2) result in
    meaningful distinctions in individual employee performance. This should include
    consideration of critical competencies and achievement of concrete results.

•   Involve employees, their representatives, and other stakeholders in the design of
    the system, including having employees directly involved in validating any related
    competencies, as appropriate.

•   Assure that certain predecisional internal safeguards exist to help achieve the
    consistency, equity, nondiscrimination, and nonpoliticization of the performance
    management process (e.g., independent reasonableness reviews by Human
    Capital Offices and/or Offices of Opportunity and Inclusiveness or their equivalent
    in connection with the establishment and implementation of a performance
    appraisal system, as well as reviews of performance rating decisions, pay
    determinations, and promotion actions before they are finalized to ensure that
    they are merit-based; internal grievance processes to address employee
    complaints; and pay panels whose membership is predominately made up of
    career officials who would consider the results of the performance appraisal
    process and other information in connection with final pay decisions).

•   Assure reasonable transparency and appropriate accountability mechanisms in
    connection with the results of the performance management process (e.g., publish
    overall results of performance management and pay decisions while protecting
    individual confidentiality and report periodically on internal assessments and
    employee survey results).

The above items should help serve as a starting point for Congress to consider in
crafting possible statutory safeguards for executive agencies’ performance
management systems. OPM would then issue guidance implementing the legislatively
defined safeguards.

4. What role should employees play in administering any “pay for
performance” initiative tested or put into place at the federal level?

We have reported that the involvement of employees both directly and indirectly is
crucial to the success of new initiatives, including implementing a pay for
performance system. Performance management systems are more effective when
employees perceive the process to be fair and the criteria to be clearly defined,
transparent, and consistently applied. Leading organizations have found that by

U.S. General Accounting Office, Results-Oriented Cultures: Insights for U.S. Agencies from Other
Countries’ Performance Management Initiatives, GAO-02-862 (Washington, D.C.: Aug. 2, 2002) and
Human Capital: Practices That Empowered and Involved Employees, GAO-01-1070 (Washington,
D.C.: Sept. 14, 2001).

Page 10                                                          GAO-03-779R Human Capital
actively involving employees, unions, or other employee associations when
developing results-oriented performance management systems, employee confidence
and belief in the fairness of incentive programs improves due to an understanding of
why certain employees were rewarded. To involve stakeholders and employees
when reforming their performance management systems, agencies should:

• Consult a Wide Range of Stakeholders Early in the Process.
• Obtain Feedback Directly from Employees. Directly asking employees to provide
  feedback on proposed changes in their performance management systems
  encourages a direct sense of involvement and buy-in, allows employees to express
  their views, and helps to validate the system to ensure that performance measures
  are appropriate.
• Engage Employee Unions or Associations. Effective labor-management relations
  help to achieve consensus on planned changes, avoid misunderstandings, and
  assist in the expeditious resolution of problems.

For additional information on our work on federal agency transformation efforts and
strategic human capital management, please contact me on 512-5500 or J. Christopher
Mihm, Director, Strategic Issues, at 512-6806 or at mihmj@gao.gov.

David M. Walker
Comptroller General
of the United States


Page 11                                                GAO-03-779R Human Capital
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.

                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.

Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061

To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470

                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.