26 August 1997
Source: http://www.hr.doe.gov/IG/IG-0407.txt

------------------------------------------------------------------- 
 
 
                          June 17, 1997
 
MEMORANDUM FOR THE SECRETARY
 
FROM:               John C. Layton
                    Inspector General
 
SUBJECT:            INFORMATION:  Report on "Audit of the
                    Department of Energy's Scientific and Technical 
                    Information Process"
 
 
BACKGROUND:
 
The Department of Energy has historically devoted significant
resources to fund research and development activities.  During FY
1995, approximately $5.7 billion was obligated for research and
development to management and operating contractors, and another
$1.8 billion was obligated through direct procurements,
cooperative agreements, and other financial instruments.
Scientific and technical information, in most instances, is the
only demonstrable result of this research.  The objective of this
audit was to determine whether the Department's field and
contractor sites had developed and maintained systems to manage
and track their scientific and technical information and whether
these systems ensured that the Department's Office of Scientific
and Technical Information received the work products resulting
from Department of Energy funded initiatives.
 
DISCUSSION:
 
The audit found that (1) the Department and its contractors had
not implemented systems to effectively identify, collect, and
disseminate scientific and technical information on a life-cycle
basis as required and (2) the Office of Scientific and Technical
Information was not receiving all scientific and technical
information generated by the contractors.  The Department and its
management and operating contractors did not identify at the
outset of research and development projects the deliverables that
would be conveyed to stakeholders.  Without a process to identify
anticipated deliverables, the Department could not adequately
track or monitor the overall success of its research and
development program or ensure broad-based dissemination of
scientific and technical information.
 
For direct procurements, the Department created the Technical
Information Monitoring System (TIMS) to provide life-cycle
management of scientific and technical information generated
under direct procurements.  However, Departmental personnel did
not fully maintain or utilize this system.  Procurement awards
were closed even though the system indicated that deliverables
had not been received.  In addition, almost one-third of the
active awards had delinquent scientific and technical information
products.
 
We recommended for management and operating contractor activities
that the Deputy Secretary direct Cognizant Secretarial Officers
to develop and implement a system to track scientific and
technical information on a life-cycle basis.  For direct
procurements, we recommended that the Deputy Secretary reiterate
to field activities the importance of managing scientific and
technical information in accordance with established policies and
procedures.  These recommendations are consistent with the intent
of the Government Performance and Results Act in that the
Department and its contractors (1) are accountable for scientific
and technical information and (2) need to establish scientific
and technical information performance measures as well as a life-
cycle tracking system.
 
In response to the audit recommendations, the Deputy Secretary
endorsed a plan proposed by the Director, Office of Energy
Research.  This plan calls for the Office of Scientific and
Technical Information to initiate and coordinate a strategic
planning process that will engage the Departmental community in
establishing goals and objectives for scientific and technical
information.  The plan also calls for the development of
performance measures and the establishment of a quality assurance
process.
 
Attachment
 
cc:  Deputy Secretary
       Under Secretary
                    
                    
                    
                    U.S. DEPARTMENT OF ENERGY
                   OFFICE OF INSPECTOR GENERAL
                                
                                
                                
                                
                                
                                
                                
                                
                                
               AUDIT OF THE DEPARTMENT OF ENERGY'S
                                
          SCIENTIFIC AND TECHNICAL INFORMATION PROCESS
 
 
 
 
 
 
 
The Office of Inspector General wants to make the distribution of
its reports as customer friendly and cost effective as possible.
 Therefore, this report will be available electronically through
    the Internet five to seven days after publication at the
                following alternative addresses:
                                
            Department of Energy Headquarters Gopher
                        gopher.hr.doe.gov
                                
         Department of Energy Headquarters Anonymous FTP
                       vm1.hqadmin.doe.gov
                                
  Department of Energy Human Resources Administration Home Page
               http://www.hr.doe.gov/ig
                                
  Your comments would be appreciated and can be provided on the
         Customer Response Form attached to the report.
                                
              This report can be obtained from the
                    U.S. Department of Energy
         Office of Scientific and Technical Information
                           P.O. Box 62
                   Oak Ridge, Tennessee  37831
 
 
 
 
Report No.:DOE/IG-0407         Capital Regional Audit Office
Date of Issue:  June 17, 1997  Germantown, MD  20874
 
               
               
               
               AUDIT OF THE DEPARTMENT OF ENERGY'S
          SCIENTIFIC AND TECHNICAL INFORMATION PROCESS
                                
                        TABLE OF CONTENTS
 
 
 
                                                            Page
 
SUMMARY . . . . . . . . . . . . ... . . . . . . . . . . . .  1
 
PART I -  APPROACH AND OVERVIEW . . . . . . . . . . . . . .  2
 
          Introduction . . . . . . . . . . . . . . . . . .   2
 
          Scope and Methodology . . . . . . . . . . . . .    2
 
          Background . . . . . . . . . . . . . . . . . . .   3
 
          Observations and Conclusions. . . . . . . . . . .  4
 
PART II - FINDINGS AND RECOMMENDATIONS . . . . . . . . . .   6
 
               1.  Management and Operating Contractor Technical
                   Information . . . . . . . . . . . . . .   6
 
               2.  Direct Procurement Technical 
                   Information .. . . .. . .. .. . . . . .  14
 
PART III -     MANAGEMENT AND AUDITOR COMMENTS . . . . . .  19
 
PART IV-  APPENDIX  . . . . . . . . . . . . . . . . . . .   22
 
               Deputy Secretary's Comments . . . . . . .    22
 
                  
                  
                  U.S. DEPARTMENT OF ENERGY
                 OFFICE OF INSPECTOR GENERAL
                  OFFICE OF AUDIT SERVICES
 
             AUDIT OF THE DEPARTMENT OF ENERGY'S
        SCIENTIFIC AND TECHNICAL INFORMATION PROCESS
                              
                              
Audit Report Number:  DOE/IG-0407
                           
                           
                           SUMMARY
 
 
     The Department of Energy obligated approximately $7.5
billion in FY 1995 to management and operating contractors,
grantees, and other financial instrument recipients for
research and development efforts.  A key product of research
and development work is scientific and technical
information.  Departmental policies and procedures require
the programs that fund the creation of scientific and
technical information to follow life-cycle management
practices.  The audit was performed to determine whether the
Department's field and contractor sites had developed and
maintained systems to manage and track their scientific and
technical information and whether these systems ensured that
the Department's Office of Scientific and Technical
Information received the work products resulting from
Department of Energy funded initiatives.
 
     The audit disclosed that the scientific and technical
information generated by management and operating
contractors was not managed on a life-cycle basis.
Implementation and execution problems also existed in the
collection and dissemination of products generated under
direct procurements.  These conditions existed because
expected deliverables for management and operating
contractors were not identified at the beginning of a
project.  For direct procurements, Departmental managers
were not aware of or bypassed established procedures.  In
both instances, effective and comprehensive quality
assurance processes had not been implemented.  As a
consequence, the Department was not in a position to know
whether it received value for its significant investment in
research and development or whether all scientific and
technical information resulting from these efforts received
the widest possible dissemination in the scientific
community.
 
     We recommended that the Deputy Secretary direct
Cognizant Secretarial Officers to implement a system to
track scientific and technical information on a life-cycle
basis.  For direct procurements, we recommended that the
Deputy Secretary reiterate to field activities the
importance of managing scientific and technical information
in accordance with established policies and procedures.
 
     (Signed)
_________________________                    
Office of Inspector General
 
                           PART I
                              
                    APPROACH AND OVERVIEW
                              
 
INTRODUCTION
 
     The Department of Energy has historically devoted
significant resources to fund research and development
activities.  The Department holds ownership and unlimited
rights to all scientific and technical information produced
by contractors.  This information, in most instances, is the
only demonstrable result of Department of Energy funded
research.  The Department, under its authorizing
legislation, is charged with the responsibility of
collecting and disseminating scientific and technical
information to promote scientific progress and public
understanding.
 
     This audit examined the Department's system for
managing scientific and technical information.  One
objective of the audit was to determine whether the
Department's field and contractor sites had developed and
maintained systems to track and manage their scientific and
technical information.  A second objective was to determine
whether these systems ensured that the Department's Office
of Scientific and Technical Information received the
scientific and technical information products resulting from
Department of Energy funded work.
 
SCOPE AND METHODOLOGY
 
     The audit was performed at three management and
operating contractor sites (Sandia National Laboratories,
the Yucca Mountain Site Characterization Project, and
Brookhaven National Laboratory) and at three Departmental
procurement activities (Federal Energy Technology Center
[FETC] - Morgantown Site, Headquarters' Procurement
Operations Office, and Oakland Operations Office).
Fieldwork was conducted from January through October 1996.
The sites selected for review were chosen based on the FY
1995 funds obligated to research and development activities
and the type of work performed.  Additional audit work was
performed at the Yucca Mountain Site Characterization Office
and the Office of Scientific and Technical Information
(OSTI) at Headquarters and Oak Ridge.
 
     To accomplish the audit objectives, scientific and
technical information policies and procedures were reviewed
to determine how management and operating contractors
managed scientific and technical information.  Contractor
systems were examined to determine if these systems provided
for life-cycle administration of technical information
products.  Data maintained in contractor systems for reports
and conferences were also compared to OSTI listings.
 
     For scientific and technical information generated as a
result of direct procurements, field office policies and
procedures were reviewed.  Detailed tests were performed at
Headquarters, Oakland, and the FETC - Morgantown Site using
random samples of FY 1995 active and inactive procurement
awards.  In this context, contract files were examined and
contractual reporting requirements were compared to
information in OSTI's Technical Information Monitoring
System (TIMS).  In addition, information in the TIMS was
reviewed to determine whether OSTI had received required
deliverables.
 
     As a part of the audit, discussions were held with
officials responsible for managing scientific and technical
information.  Headquarters, field office, and contractor
officials provided applicable guidance and directives and
supplied information on specific awards and management
systems.  Computer-generated data in OSTI's Report Holdings
File database and the TIMS were also reviewed to determine
whether these systems contained accurate information.  The
results of this analysis are discussed in Part II of this
report.
 
     The audit was made in accordance with generally
accepted Government auditing standards for performance
audits, which included tests of internal controls and
compliance with laws and regulations to the extent necessary
to satisfy the objectives of the audit.  Internal controls
were assessed with respect to the management of the
Department's scientific and technical information.  Because
the review of internal controls was limited, it would not
necessarily have disclosed all internal control and
compliance deficiencies that may have existed.
 
     The Director, Office of Energy Research, and officials
from the Offices of Procurement and Assistance Management
and Scientific and Technical Information waived the exit
conference.
 
BACKGROUND
 
     The Department of Energy is the fourth largest provider
of research and development resources within the Federal
Government.  During FY 1995, approximately $5.7 billion was
obligated for research and development to management and
operating contractors who operate Government-owned
scientific laboratories for the Department of Energy.
Another $1.8 billion was obligated through grants,
cooperative agreements, and prime contracts.  Thus, the
research and development mission represented about one-third
of the Department's overall funding.
 
     It is imperative that the Department's laboratory
contractors be accountable for research and development
expenditures and that the taxpayers receive fair value for
their investment in these programs.  To achieve this, the
Department must be able to demonstrate the results of its
research and development expenditures.  Scientific and
technical information is a key product of research and
development work.  Scientific and technical information is
defined as information in any format or medium that is
derived from scientific and technical studies; work; or
investigations that relate to research, development,
demonstration, and other specialized areas and includes
unclassified, classified, declassified, and sensitive
information.
 
     According to DOE Order 1430.1D, "Scientific and
Technical Information Management," Departmental programs
that fund the creation of scientific and technical
information must follow life-cycle management practices.
Life-cycle management is a process that is defined as
planning for, budgeting, producing, processing,
disseminating, and storing in a cost-effective manner
scientific and technical information to ensure widespread
use by U.S. industry and the general public.  The Department
identifies, collects, and disseminates scientific and
technical information through its Technical Information
Management Program.  OSTI is responsible for administering
the program and for providing centralized planning,
development, and utilization of scientific and technical
information.
 
     Prior reviews of the Department's scientific and
technical information program have identified difficulties
in collecting and ensuring the delivery of scientific and
technical information to the Department.  A review of the
Department's records management practices by the National
Archives Record Administration in 1988 noted "...the
tendency for scientists to consider the results of their
activities to be their personal property..." rather than
handle them as Federal records.  An Office of Inspector
General audit conducted in 1995 showed that the
Superconducting Super Collider Program had not provided the
Department with approximately 21 percent of the scientific
and technical information generated in connection with this
project.
 
     The legislative history of the Department of Energy
underscores the importance of managing and disseminating
scientific and technical information.  The Atomic Energy Act
of 1954, as amended, created a program for the dissemination
of unclassified information to encourage scientific and
industrial progress.  The Department of Energy Organization
Act of 1977 and other laws including the American Technology
Pre-eminence Act of 1991 and the Energy Policy Act of 1992
require the Department to make its information available and
disseminate the information resulting from its programs.
The American Technology Pre-eminence Act further requires
the Department to transfer information that results from
federally funded research and development activities to the
National Technical Information Service of the Department of
Commerce in a timely manner for dissemination to the private
sector, academia, state and local governments, and Federal
agencies.  Also, Title 44, U.S.C., states that Government
publications, which are publicly releasable, shall be made
available to the depository libraries through the
Superintendent of Documents, Government Printing Office, for
public information.
 
     Since inception, the Department has recognized its
central role in collecting and disseminating scientific and
technical information resulting from its extensive research
and development activities.  The Department of Energy's 1995
performance agreement with the President, for example,
states that the Department will be "...a major partner in
world class science and technology through its national
laboratories, research centers, university research, and its
educational and information dissemination programs."
[emphasis added]
 
OBSERVATIONS AND CONCLUSIONS
 
     Despite the Department's commitment to developing and
maintaining a "world class" research and development
program, the Department's process for managing scientific
and technical information associated with its $7.5 billion
research and development program was not fully effective.
Scientific and technical information generated by management
and operating contractors was not managed on a life-cycle
basis.  Implementation and execution problems also existed
in the collection and dissemination of products produced
under direct procurements (i.e., prime contracts, grants,
and other financial assistance awards).
 
     Although management and operating contractors received
the majority of the Department's research and development
dollars annually (in FY 1995 about $5.7 billion obligated
for management and operating contractors versus about $1.8
billion obligated for direct procurements), they had not
implemented systems to provide life-cycle management of
scientific and technical information.  Despite Departmental
requirements, the contractors at the locations included in
the audit did not identify at the outset of Department
funded research and development projects what deliverables
would be conveyed to stakeholders.  Without a process to
identify anticipated outputs or deliverables, the Department
could not adequately track or monitor the overall success of
its research and development program.  Further, the
Department was unable to ensure broad-based dissemination of
scientific and technical information products.  This
weakness was particularly noteworthy in FY 1996 because the
Department was not in a position to examine and fully
explain a 30 percent reduction in the number of scientific
and technical information products transmitted to OSTI.
 
     In contrast, the Department did utilize a life-cycle
system to manage scientific and technical information
generated under direct procurements.  The Department's
Technical Information Monitoring System was intended to
identify and anticipate scientific and technical information
deliverables at the time of the award, track progress
against reporting requirements, and record receipt of
completed products.  However, Departmental personnel did not
fully maintain and utilize this system.  The audit disclosed
that procurement awards were closed and contractors and
grantees received payment, despite the fact that the TIMS
showed that final deliverables had not been received.  The
TIMS also indicated that almost one-third of the
Department's $23.4 billion in active direct procurement
awards had delinquent scientific and technical information
deliverables.  Nearly 40 percent of these expected
deliverables were delinquent by 5 years or longer.  This is
described more fully on page 16 of this report.
 
     These conditions existed in part because of the low
priority placed on the identification, collection, and
dissemination of scientific and technical information.  To
correct the weaknesses noted, the Department needs to ensure
that its contractors establish life-cycle technical
information systems.  Performance measures for scientific
and technical information management also need to be
incorporated into the Department's contracts to improve
management of and accountability for information developed
under those contracts.  Further, quality assurance
procedures need to be strengthened to ensure the effective
collection and dissemination of scientific and technical
information.
 
     The Deputy Secretary, the Director, Office of Energy
Research, and the Office of Procurement and Assistance
Management concurred with the report's recommendations and
provided comments on the initial draft report.  These
comments have been incorporated into this report where
appropriate, and the Deputy Secretary's comments are
included verbatim in Part IV.  Part II of this report
provides details of the findings and Part III includes
detailed management and auditor comments.
                           
                           
                           PART II
                              
                FINDINGS AND RECOMMENDATIONS
                              
                              
1.  Management and Operating Contractor Technical
    Information
 
FINDING
 
     The Department and its management and operating
contractors are required to establish life-cycle systems to
identify, collect, and disseminate scientific and technical
products generated under Department of Energy funded
research and development activities and provide these
products to the Office of Scientific and Technical
Information (OSTI).  In FY 1995, about $5.7 billion was
obligated for the research and development activities of the
Department's management and operating contractors.  Although
the Department had initiated development of a new research
and development tracking system, effective mechanisms to
identify, collect, and disseminate scientific and technical
information had not been implemented.  The Department
neither utilized a life-cycle management process nor ensured
that all information generated by the Department's
management and operating contractors was provided to OSTI.
These conditions existed because expected deliverables were
not identified at the beginning of a project and an
effective quality review process had not been instituted.
As a consequence, the Department was not in a position to
know whether it received value for its significant
investment in research and development or whether
information emanating from these efforts received the widest
possible dissemination.
 
RECOMMENDATIONS
 
     We recommend that the Deputy Secretary direct Cognizant
Secretarial Officers to implement the existing system or
create a new research and development tracking system that:
 
        Identifies anticipated scientific and technical
        information deliverables when management and operating
        contractor task assignments or work authorizations are
        issued.
   
        Establishes processes to track scientific and technical
        information deliverables on a life-cycle basis that are
        integrated into Departmental systems.
 
     We recommend that the Deputy Assistant Secretary for
Procurement and Assistance Management ensure that
performance measures for scientific and technical
information are incorporated into performance-based
contracts for research and development activities.
     
     We recommend that the Director, Office of Energy
Research:
 
        Work with cognizant field office representatives to
        initiate "for cause" reviews, similar to those conducted
        under the Department's Business Management Oversight
        Program, to ensure that the Department's goal regarding the
        identification, collection, and dissemination of scientific
        and technical information is met.
   
        Perform future assessments of contractor scientific and
        technical information management under the Department's
        newly developed Technical Program Oversight Process.
 
MANAGEMENT REACTION
 
     The Deputy Secretary endorsed a plan proposed by the
Director, Office of Energy Research, to engage the Cognizant
Secretarial Officers and those in the field and laboratories
to work cooperatively to address the audit recommendations.
The Office of Procurement and Assistance Management agreed
to take responsibility for adding language to its
"Department of Energy Acquisition Guide" regarding the
inclusion of performance measures for scientific and
technical information in contracts.
 
 
                     DETAILS OF FINDING
 
 
SCIENTIFIC AND TECHNICAL INFORMATION REQUIREMENTS
 
     Department of Energy policy (DOE Order 1430.1D
"Scientific and Technical Information Management") requires
that scientific and technical information be managed on a
life-cycle basis.  Departmental program offices, the
cognizant field office, and the Headquarters program offices
must follow life-cycle information management practices to
ensure that the information is planned for, budgeted,
produced, processed, disseminated, and stored in a cost-
effective way to ensure its maximum utilization by all
customer segments including U.S. industry and the general
public.  The Department also requires that program elements
and contractors provide the results of completed scientific
and technical efforts to OSTI--the coordination point for
customer access to all Department of Energy scientific and
technical information.
 
     Under this system, the contractors that operate the
Department's laboratories play a key role in this process.
They are to:
 
   o Identify outputs in the form of anticipated
     deliverables when research and development tasks are
     authorized.
   
   o Track anticipated and actual deliverables.
   
   o Collect and transmit completed deliverables to OSTI.
 
     The Department's Guide for the Management of Scientific
and Technical Information states that "program/project
officials should specify information outputs that can be
measured."  At the initiation of a project or task,
anticipated products could include technical reports,
journal articles, professional publications, and
presentations and conference proceedings.  These
deliverables give Departmental managers a basis for
evaluating contractor performance in the research and
development field.  They also are an important vehicle used
by the Department and its laboratories to make their
technical findings available to the widest appropriate
audience in the scientific community.  Ultimately, the
deliverables are critical in determining if the scientific
work effort was of value in relation to the significant
taxpayer investment in the Department of Energy's research
and development program.
 
MANAGEMENT SYSTEMS
 
     Our review at Sandia, Brookhaven, the Yucca Mountain
Site Characterization Project, and corresponding discussions
with cognizant Departmental offices disclosed that effective
systems had not been established to manage the scientific
and technical information resulting from the research
accomplished at these facilities.  Improvements were needed
in identifying, collecting, and transmitting completed
scientific and technical products to OSTI.  The contractors
did not have technical information management systems that:
 
   o    documented anticipated project deliverables,
 
   o    identified ongoing scientific projects, or
   
   o    captured the periodic status or progress of the
        projects.
   
     The results of this audit raised concerns about the
ability of the Department to identify and track anticipated
versus actual technical deliverables.  A parallel situation
was addressed in a 1995 Office of Inspector General audit
report "Audit of Program Administration by the Office of
Energy Research" (DOE/IG-0376).  The report concluded that
the process used at that time within the Energy Research
laboratory complex did not provide the Department with a
method for determining if research and development schedules
were met, resources were properly used, deliverables were
provided as specified, or whether research was performed in
accordance with the Department's mission.  The report noted
that Energy Research officials were concerned that the
additional requirements to identify milestones, criteria, or
deliverables for management and operating contractors
"...could reduce desirable flexibility and stifle the
creativity of the researchers."
 
Anticipated Project Deliverables
 
     The Department's process for administering scientific
and technical information at its management and operating
contractors did not establish deliverables as part of the
initial task assignment as outlined in DOE Order 5700.C
"Work Authorization System."  Neither the Department nor its
laboratory contractors had the benefit of an initial
baseline against which research and development progress
could be measured.  This eliminated the ability of Sandia,
Brookhaven, Yucca Mountain, and the Department to identify
and monitor the progress of its scientific and technical
initiatives.
 
     Sandia, as an alternative, attempted to identify
scientific and technical information deliverables through a
document collection system.  The Laboratory assigned numbers
to potential information products when the principal
investigators or researcher notified Sandia's scientific and
technical information office that delivery of a technical
document was imminent.  However, even under this process,
which did not include baseline data, there was no assurance
that Sandia's scientific and technical office received all
promised deliverables.  The audit disclosed that 99 of 843
anticipated reports and conference papers assigned a
tracking number in FY 1995 had not been received as of May
1996.  Further, Sandia's system did not include followup
procedures, which would have provided an explanation as to
the status of the documents that researchers had indicated
would be produced.  Thus, it was possible that the documents
were published, submission of the deliverables was delayed,
or the anticipated deliverables were canceled without the
knowledge of the Department or Sandia's office of scientific
and technical information.  Because of the inherent
limitations in its process, Sandia could not confirm the
existence or nonexistence of its anticipated work products.
Instead, officials suggested that individual scientists
would have to be contacted to determine the status of each
project.
 
     Sandia subsequently adopted a new system similar to
that employed by Brookhaven.  Under this new approach,
identification numbers were assigned to research products
after the documents were submitted.  Once again, the revised
system did not require the establishment of anticipated
deliverables at the time of task or project approval.  Thus,
there was no baseline against which actual performance could
be evaluated.  [emphasis added]
 
Collection and Transmittal of Completed Technical
Information Products
 
     The collection and transmittal of completed scientific
and technical information products to OSTI is a critical
component of the life-cycle management process.  The audit
disclosed weaknesses at Sandia, Brookhaven, and at Yucca
Mountain in the transmittal of completed products to OSTI.
Both Sandia and Brookhaven maintained database systems that
were described previously in this report.  We found that
these systems needed improvement because their data on
completed scientific and technical information products did
not fully agree with data in OSTI's database.  And, we could
not find evidence that management had attempted to identify
or reconcile the inconsistencies.  At Yucca Mountain the
collection and dissemination of information products was
decentralized.  The Department did not have a mechanism in
place to ensure that technical deliverables prepared by the
Project's management and operating contractor were
transmitted to OSTI.
 
     Departmental and Contractor Database Systems
 
     Sandia and Brookhaven National Laboratories both
maintained databases of known scientific and technical
documents.  Inconsistencies were noted between the data
maintained in the contractors' databases and the data in
OSTI's system.  The OSTI system had no record of 3 percent
of Sandia's documents and 7 percent of Brookhaven's
documents.  Deliverables not listed in the OSTI database
included studies on "...Diagnosis of Cervical Pre-Cancer"
and "...Ischemic Acute Renal Failure."  It was difficult to
assess the substance of these studies and the impact of
their not being part of the OSTI database.  Although the
inconsistencies, in percentage terms, were small it was
clear that the Department's goal of providing the widest
possible dissemination of the results of its research and
development activities was not well served.  Other
researchers may not have had the benefit of the outcome of
this scientific inquiry.  The effective dissemination of
this information also assists the Department in ensuring
that it does not fund duplicative research.
 
     Yucca Mountain Site Characterization Project
 
     At Yucca Mountain, the management and operating
contractor did not maintain a system to record the
collection of scientific and technical information.  The
contractor relied on the Department's project office to
review and approve documents for publication and advise
authors on which documents should be submitted to OSTI.  No
followup procedures were implemented to ensure that the
documents were actually transmitted to OSTI.
 
     In FY 1995, the Department approved 176 Yucca Mountain
Site Characterization Project scientific and technical
information documents, prepared by management and operating
contractors, for publication.  Only 47 percent of these
documents were ever transmitted to OSTI.
 
     The Acting Director, Office of Civilian Radioactive
Waste Management, commented on the above audit observations.
He emphasized that the Yucca Mountain Site Characterization
Project conducts site-specific and engineering studies and
that the Project has in place management systems to plan,
produce, track, disseminate, and store scientific and
technical information.  The Acting Director, however, did
agree to enhance the Project's Planning and Control System
and ensure that documents identified for transmittal to OSTI
are forwarded in a timely manner.
 
LACK OF EMPHASIS ON TECHNICAL INFORMATION MANAGEMENT
 
     The Department's process for administering scientific
and technical information at its M&O contractors appears to
have been limited by:  (1) a reluctance to establish
deliverables as a part of the initial task assignment and
(2) an absence of full scope reviews of contractor technical
information management systems.  The Department has an
opportunity to correct these problems by adding performance
measures in future contracts relating to contractor
scientific and technical information deliverables.
 
Process Implementation
 
     The Department's ability to administer scientific and
technical information has been hindered by sporadic
implementation of its work authorization processes.  As in
DOE Order 5700.7C "Work Authorization System," a process is
established for identifying management and operating
contractor scientific and technical information products.
The Order requires that an approved work authorization
and/or field work proposal identify the schedule, results,
and product expected from the work; that Departmental
officials ensure that a description of research in progress
be provided to OSTI; and that a unique identifying number be
assigned to the project throughout its life.
 
     Although this Order is still in effect and OSTI has
actively encouraged its implementation, processes to
administer technical information generated by management and
operating contractors, according to OSTI, had not been
successful.  For example, an OSTI official stated that
several program organizations within the Department did not
use the fieldwork proposal system or provide unique
identifying numbers for their projects.  In another
instance, a Brookhaven official stated that they did not
believe it was necessary to identify deliverables since
there were few reasons for researchers to withhold the
results of their research efforts.
 
     In 1993, the Department was requested by the White
House, Office of Science and Technology Policy, to
participate in gathering information on research and
development activities supported by the Federal Government.
The Department was to electronically transmit information on
the allocation of its budget for non-R&D and R&D activities,
the substantive areas and stages of the research, and
project descriptions by laboratory.  The Department was not
able to provide complete information, and the information
that was provided was viewed as being unresponsive to the
request.  The importance of identifying and tracking
scientific and technical deliverables was apparent in this
exercise.
 
     To correct this situation, the Deputy Secretary
subsequently directed the development of the R&D Tracking
System.  Beginning with a call for data in 1996, each
laboratory was directed to identify the deliverable expected
for each approved R&D project and track each project with a
unique identifying number.  Successful implementation of
this system will be dependent upon the full support of
Departmental program managers and management and operating
contractors.
 
OSTI Reviews
 
     In 1995 examinations of contractor scientific and
technical information programs were incorporated into the
business management system reviews under the Department's
Business Management Oversight Pilot Project.  The Business
Management Pilot was oriented towards a business systems
review.
 
     Prior to the implementation of the Business Management
Pilot, OSTI conducted about 20 to 30 onsite Departmental and
management and operating contractor appraisals each year.
These reviews addressed three main objectives: an evaluation
of life-cycle management of scientific and technical
information resources, scientific and technical information
reporting, and compatibility with the Department's
scientific and technical information program.
 
     In contrast, the approach utilized in the Business
Management Oversight Program was substantially less
comprehensive, relying primarily on a process-based
evaluation.  Further, despite the significance of its
research and development activities, in FY 1996 the
Department conducted only three Business Management reviews
addressing scientific and technical information management.
 
     After its initial participation in the Business
Management Pilot in September 1995, OSTI proposed that the
Department improve management of its scientific and
technical information program by transferring responsibility
for reviews of the program to the Department's Technical
Program Oversight Process.  This annual technical review
process, to be implemented during calendar year 1996, was
intended to improve oversight of technical programs at
Departmental laboratories.  OSTI had not received a response
to its proposal as of the end of this audit.
 
     Regardless of the process chosen for the review of
scientific and technical information, it is essential for
the Department to ensure that its goals regarding the
identification, collection, and dissemination of scientific
and technical information are met.
 
Scientific and Technical Information as Performance Measures
 
     As the Department transitions to performance-based
contracting, managers will also need to develop performance
measures for scientific and technical information. These
performance measures or benchmarks assist in tracking
Departmental and contractor accomplishments to ensure, among
several goals, that the taxpayers receive full value for the
investment made in the Department's research and development
program.  The Department's contract reform team emphasized
this point when it concluded that:
 
      The Department must clearly state what we expect
      from our contractors and develop meaningful ways
      to measure whether they are meeting our
      expectations.
 
     Departmental officials have begun to develop "best-in-
class" measures for inclusion in new and renegotiated
contracts.  However, performance measures developed by OSTI
in 1995 for management of scientific and technical
information were not included in recently awarded
performance-based contracts.
 
INVESTMENT IN SCIENTIFIC AND TECHNICAL INFORMATION
 
     The Department invested over $7 billion in FY 1995 in
research and development activities.  Nonetheless, the
Department was not in a position to know whether it received
value, in the form of scientific and technical information,
for its investment.  The Department had neither a systematic
process to collect technical information products from its
management and operating contractors nor a mechanism to
compare actual scientific and technical information
accomplishments against expectations.
 
Scientific and Technical Information Products
 
     The significance of not having a system to manage
scientific and technical information was illustrated by the
recent drop in the number of technical information products
being delivered to OSTI.  Between FY 1995 and FY 1996, there
was a 30 percent reduction in the number of information
products transmitted to OSTI.  In FY 1996, OSTI received
14,254 documents, compared to 20,452 documents it received
in FY 1995.  Yet, during this period, the Department's
research and development budget remained relatively
constant.
 
     We questioned officials to determine the cause for the
significant reduction in documents transmitted to OSTI.
Although they had not performed a definitive trend analysis,
they speculated that this could have occurred due to a
changing budget, an increase in the number of journal
articles, or fewer conferences being held.  However, we
concluded that since appropriate benchmarks had not been
established for scientific and technical information
products, there was no way for Departmental officials to
determine if they received the appropriate number of
products or whether the sharp decline in the receipt of such
products was consistent with the Departmental and
congressional direction or expectations.
 
2.  Direct Procurement Technical Information
 
FINDING
 
     Departmental program managers and contracting personnel
are responsible for obtaining and promptly transmitting to
the Office of Scientific and Technical Information technical
deliverables generated under contracts, grants, and other
financial assistance instruments.  The Department
established a system to track and process this type of
information for direct procurements, but weaknesses existed
in its implementation.  Procurement instruments were closed
even though the OSTI database indicated that deliverables
had not been provided, and many other deliverables were not
being provided on a timely basis.  The Department's tracking
system at the time of the audit showed that almost one-third
of the Department's $23.4 billion in active direct
procurements had delinquent scientific and technical
deliverables.  These conditions existed because managers
were not aware of or bypassed controls, and a comprehensive
quality assurance process had not been established.  The
Department, as a consequence, could not be assured that all
scientific and technical information products were being
provided to its customers.
 
RECOMMENDATIONS
 
     We recommend that the Deputy Secretary reemphasize to
field activities the importance of processing scientific and
technical information in accordance with Departmental
policies and procedures, and require full implementation by
all Departmental elements.
 
     We recommend that the Director, Office of Scientific
and Technical Information, working in conjunction with the
Deputy Assistant Secretary for Procurement and Assistance
Management, institute a comprehensive quality assurance
process that ensures that all field activities are
fulfilling their responsibilities for collecting,
processing, and disseminating scientific and technical
information.
 
MANAGEMENT REACTION
 
     The Deputy Secretary agreed to reemphasize scientific
and technical information as a key deliverable of research
and development.  The Director, Office of Energy Research,
indicated that the Offices of Scientific and Technical
Information and Procurement and Assistance Management had
initiated discussions to develop a quality assurance
process.
 
 
                     DETAILS OF FINDING
 
 
TECHNICAL INFORMATION REPORTING REQUIREMENTS
 
     Departmental managers, in accordance with Department of
Energy Acquisition Regulation (DEAR) 935.010 "Scientific and
Technical Reports" and DOE Order 1332.2 "Uniform Reporting
System for Federal Assistance," are required to specify
scientific and technical information deliverables under
contracts, grants, and other financial assistance
instruments and ensure that these deliverables are promptly
provided to OSTI.  DOE Order 1430.1D "Scientific and
Technical Information Management" emphasizes the importance
of obtaining this information and states that procurement
instruments are not considered closed until the Head of the
cognizant Departmental element has ensured that the
deliverables have been received.
 
SCIENTIFIC AND TECHNICAL DELIVERABLES
 
     In contrast with the procedures used in administering
management and operating contracts, the Department developed
a life-cycle system to administer scientific and technical
information generated under prime contracts, grants, and
other financial assistance instruments.  Key components of
the system included identification of the deliverables;
reviews of completed products for patent, classification,
and sensitivity issues; tracking the status of information
products utilizing automated databases; and receipt and
storage of required scientific and technical deliverables by
OSTI.
 
     Although a comprehensive system had been established,
weaknesses existed in its implementation.  Awards were
closed without deliverables, and many scientific and
technical deliverables were not provided to OSTI on a timely
basis for dissemination.
 
Deliverables Not Received or Provided in a Timely Manner
 
     Between FY 1991 and FY 1995, Departmental records
indicated that 89 awards were closed even though required
scientific and technical information products had not been
provided to OSTI.  The value of the closed awards was $138
million.  In total, 389 separate deliverables required under
these procurement actions were not provided to OSTI.
 
     In addition, many deliverables due under prime
contracts, grants, and cooperative agreements were
delinquent according to Departmental records.  At the end of
FY 1995, the Department had 7,030 active and inactive multi-
year awards requiring scientific and technical information
deliverables.  The total value of these awards was $23.4
billion.  Of this amount, procurement actions valued at $7.3
billion had delinquent scientific and technical products.
 
     To determine the extent to which scientific and
technical information had not been provided to the
Department's central coordination point, an aging schedule
was prepared by OSTI of overdue annual and final reports.
The following table shows the results of this analysis for
the Headquarters, the FETC - Morgantown Site, and the
Oakland procurement offices.  These are three of the most
active procurement offices in the Department that handle
scientific and technical contracts, grants, and cooperative
agreements.
                              
    AGING SCHEDULE OF DELINQUENT ANNUAL AND FINAL REPORTS
 
 
                 HQ Proc.    FETC     Oakland     Total
                                                    
       0-1 yrs.     78        53        191        322
       1-2 yrs.     47        10         97        154
       2-3 yrs.     37        11         76        124
       3-4 yrs.     30        4          45         79
        5+ yrs.     97        12        315        424
                                                    
        TOTALS      289       90        724       1,103
 
In total, 1,103 scientific and technical information reports
were delinquent for contracts, grants, and cooperative
agreements issued by these three offices.  Of this total,
424 annual and final reports had been delinquent for 5 years
or more.  We believe that the extensive delays in receipt of
this data by OSTI minimize its usefulness to the Department
and, potentially, in the scientific community.
 
SYSTEM IMPLEMENTATION PROBLEMS
 
     OSTI did not receive many of these annual and final
reports because Departmental managers incorrectly processed
information, bypassed system controls, or did not understand
their role in processing scientific and technical
information.  Further, the Department did not have an
effective quality assurance process.
 
Controls Bypassed or Not Understood
 
     Various problems were noted in the collection and
transmittal of information to OSTI.  In some instances, the
delinquent reports had been received by a Departmental
element but had not been forwarded to OSTI.  For example,
FETC - Morgantown Site received several quarterly and final
reports during FY 1995 but had not provided them to the
Department's central repository for scientific and technical
information.  A Morgantown official explained that many of
these reports needed to go through a patent review process
and had to be released by the contracting officer's
representative before they went to OSTI.  This process often
delayed the transmittal of reports by anywhere from a few
months to over a year.
 
     In other cases, data contained in the Department's
Procurement Acquisition Data System (PADS) were not
accurate.  For example, an examination of 25 procurement
award files at Headquarters indicated that the scientific
and technical deliverables listed for 17 of these awards
were incorrect in the PADS.  PADS is the primary source of
information for the Department's scientific and technical
database (Technical Information Monitoring System--TIMS),
and OSTI is dependent upon PADS to monitor the generation
and collection of scientific and technical information.
Data errors in the PADS system, therefore, limited OSTI's
ability to perform its assigned task.
 
     Linkage between local database systems and the PADS and
OSTI systems was also weak.  Officials at FETC-Morgantown
Site waived reporting requirements and changed information
in their local system but did not update the Departmental
systems.  In another situation, officials at the Oakland
Operations Office, from FY 1991 to FY 1994, closed eight
awards valued at $7.8 million prior to obtaining required
clearance from the Office of Scientific and Technical
Information.
 
     Program managers also indicated that they had been
excluded from the process or were not aware of their
assigned responsibilities.  For example, a contract file for
a $240,000 award at Headquarters indicated that quarterly
technical progress reports and a final technical report were
due at the end of the award's performance period.  However,
none of the required reports had been sent to OSTI.  The
responsible procurement official stated that it was the
"responsibility of the program officials to obtain
scientific and technical information documents."  The
program official responded that she received the reports,
but she had never seen the reporting checklist.  The
official was of the opinion that it was the procurement
people who [were] responsible for obtaining scientific and
technical information.  According to DOE Order 1430.1D, both
program and procurement officials are responsible for
ensuring that this information is provided to OSTI.
 
Quality Assurance Program
 
     The absence of an effective quality assurance program
also contributed to problems in the receipt of technical
information products.  For example, a Departmental technical
information officer in Oakland was unable to correct the
problem of delinquent scientific and technical information
reporting.  At the Oakland Operations Office, a monthly
report was prepared by the scientific and technical
information officer and forwarded to procurement and program
officials to notify them of delinquent technical information
deliverables.  However, this process did not result in
improved receipt and transmittal of scientific and technical
information to OSTI.
 
     OSTI also performed limited examinations of scientific
and technical information statistics during its reviews at
field activities.  However, these reviews looked only at the
number of information products obtained, not at why
information was not being collected from contractors and
forwarded to OSTI for dissemination.
 
IMPACT OF DELINQUENT SCIENTIFIC AND TECHNICAL INFORMATION
 
     The Department is involved in major scientific efforts
to clean up the environment, pursue basic research, develop
alternative sources of energy, and research health and
environmental issues.  The value of the Department's
research and development effort rests not only in the
quality of the work, but also in the timely dissemination of
scientific and technical information to the public and
scientific community.  The success of these efforts and the
value of the research is not maximized when its availability
and dissemination is limited.  And, the Department's ability
to demonstrate positive results from its expenditures is
hindered.  Information that is not obtained from a
contractor or grantee, or information that resides in a
local procurement or programmatic office, does not enhance
or contribute to the expansion of scientific knowledge or
allow the Department to fulfill its research and development
mission.
                          PART III
 
               MANAGEMENT AND AUDITOR COMMENTS
                              
                              
     The Deputy Secretary of Energy, the Director, Office of
Energy Research, and the Office of Procurement and
Assistance Management agreed with the report's
recommendations.  A summary of management's comments and the
auditor's response follows.
 
Finding 1.  Management and Operating Contractor Technical
Information
 
     Recommendation 1.  The Deputy Secretary direct
Cognizant Secretarial Officers to implement the existing
system or create a new R&D tracking system that:  (1)
identifies anticipated scientific and technical information
deliverables when management and operating contractor
assignments or work authorizations are issued and (2)
establishes processes to track scientific and technical
information deliverables on a life-cycle basis that are
integrated into Departmental systems.
 
     Management Comments.  Concur.  The Deputy Secretary
agreed that the Department should emphasize scientific and
technical information as a key deliverable of research and
development and endorsed the plan from the Office of Energy
Research that directed Cognizant Secretarial Officers to
fully implement a tracking system to ensure life-cycle
management of R&D projects including the resulting
scientific and technical information.
 
     Auditor Comments.  Management comments are responsive
to the recommendation.
 
     Recommendation 2.  The Deputy Assistant Secretary for
Procurement and Assistance Management ensure that
performance measures for scientific and technical
information are incorporated into performance-based
contracts for research and development activities.
 
     Management Comments.  Concur.  The Office of
Procurement and Assistance Management agreed to fulfill its
contracting responsibility by developing appropriate
language concerning inclusion of performance measures for
scientific and technical information to be included in its
Acquisition Guide.  This is to convey management's
expectation that contracts contain performance measures;
however, development and approval of performance measures is
the responsibility of the cognizant program office under the
requirements of the Government Performance and Results Act.
 
     Auditor Comments.  While management concurred and
proposed several positive initiatives, its comments were not
fully responsive to the recommendation.  The Office of
Inspector General recognizes that it is the responsibility
of cognizant program and field offices to establish
scientific and technical information performance measures.
However, it is Procurement's responsibility to ensure,
through its contract approval process, that established
scientific and technical information performance measures
are contained in all applicable contracts.  Procurement's
response did not indicate what steps or actions it would
take to ensure that appropriate measures are included in
performance-based contracts.
 
     Recommendation 3.  The Director, Office of Energy
Research:  (1) work with cognizant field office
representatives to initiate "for cause" reviews, similar to
those conducted under the Department's Business Management
Oversight Program, to ensure that the Department's goal
regarding the identification, collection, and dissemination
of scientific and technical information is met and (2)
perform future assessments of contractor scientific and
technical information management under the Department's
newly developed Technical Program Oversight Process.
 
     Management Comments.  Concur.  The Office of Energy
Research indicated that the recommendations for scientific
and technical information complemented other management
changes currently underway at the Department, such as the
Government Performance and Results Act of 1993.  Part of
that reform is to establish performance-based management
with a focus on outcomes rather than processes.  The Office
of Energy Research plans to integrate steps for improvement
into the strategic planning process, with annual performance
plans and results reported beginning with the Fiscal Year
1999 budget cycle, consistent with the Department's overall
planning efforts.  The Office of Energy Research also agreed
to establish performance expectations that will be
incorporated in future technical program reviews with a goal
to incorporate scientific and technical information in
Fiscal Year 1998 program reviews.
 
     Auditor Comments.  Management's comments are generally
responsive to the recommendation.  However, the Office of
Energy Research needs to develop a detailed action plan on
how it will work with cognizant field office representatives
to initiate for cause reviews or perform future assessments
of scientific and technical information management by the
Department's contractors.
 
Finding 2.  Direct Procurement Technical Information
 
     Recommendation 1.  The Deputy Secretary reemphasize to
field activities the importance of processing scientific and
technical information in accordance with Departmental
policies and procedures, and require full implementation by
all Departmental elements.
 
     Management Comments.  Concur.  The Deputy Secretary
agreed that the Department should emphasize scientific and
technical information as a key deliverable of research and
development.  As a means of taking corrective action, the
Deputy Secretary endorsed the plan proposed by the Director,
Office of Energy Research, to have the Office of Scientific
and Technical Information initiate and coordinate a
strategic planning process to engage the Department's R&D
community in establishing goals, objectives, and measures
for scientific and technical information.  In addition, to
ensure implementation, performance expectations will be
established and incorporated in technical program reviews
with a goal to incorporate scientific and technical
information in Fiscal Year 1998.
 
     Auditor Comments.  Management comments are responsive
to the recommendation.
 
     Recommendation 2.  The Director, Office of Scientific
and Technical Information, working in conjunction with the
Deputy Assistant Secretary for Procurement and Assistance
Management, institute a comprehensive quality assurance
process that ensures that all field activities are
fulfilling their responsibilities for collecting,
processing, and disseminating scientific and technical
information.
 
     Management Comments.  The Director, Office of Energy
Research, concurred with the recommendation and indicated
that the Office of Scientific and Technical Information had
initiated communications with the Deputy Assistant Secretary
for Procurement and Assistance Management to institute a
quality assurance process.  A memorandum including a summary
of OSTI's experience with the field activities, citing
specific quality assurance issues would be provided to
Procurement and Assistance Management to develop a process
to address the needs related to scientific and technical
information.
 
     The Office of Procurement and Assistance Management
agreed to assist the Director of OSTI in effectively
communicating to the field offices to ensure that they
understand the requirement to provide deliverables to OSTI.
They will also assist OSTI in making their system more
accurate by securing timely deliverables through such
communications.
 
     Auditor Comments.  The comments of the Director, Office
of Energy Research, were responsive to the recommendation.
However, the Office of Procurement and Assistance
Management's response did not detail the actions it would
take towards instituting a comprehensive quality assurance
process that ensures that all field activities are
fulfilling their responsibilities for collecting,
processing, and disseminating scientific and technical
information.
                                                    
                                                    
                                                    APPENDIX
 
                           PART IV
 
                 DEPUTY SECRETARY'S COMMENTS
                              
                              
                    Department of Energy
                    Washington, DC  20585
                              
OFFICE OF THE
DEPUTY SECRETARY
 
MEMORANDUM FOR:     John C. Layton
                    Inspector General
 
FROM:               Charles B. Curtis
                    Deputy Secretary
 
SUBJECT:            Comments on the Initial Draft
                    Report on "Audit of the
                    Department of Energy's Scientific and
                    Technical Information Process"
                  
DATE:                         April 30, 1997
 
 
I appreciate being given the opportunity to offer my views
on the subject report.  It raises several important issues
related to the scientific and technical information
resulting from the Department's research and development
activities.
 
I agree that the Department should emphasize scientific and
technical information as a key deliverable of research and
development.  The value of the Department's research and
development effort relies on adequate information being made
available to the scientific community and the public,
thereby helping Americans to better understand what we do.
 
Further, as a means of taking corrective action, I endorse
the plan proposed by the Director of the Office of Energy
Research to have the Office of Scientific and Technical
Information conduct strategic planning sessions to address
these important issues.  By working cooperatively through
the Cognizant Secretarial Officers and those engaged in the
field and laboratories, a stronger, more effective
scientific and technical information process should result.
                             
                             
                             IG Report No.  DOE/IG-0407
                                                       
 
                                                
                              
                              
                   CUSTOMER RESPONSE FORM
                              
The Office of Inspector General has a continuing
interest in improving the usefulness of its products.
We wish to make our reports as responsive as possible
to our customers' requirements, and therefore ask that
you consider sharing your thoughts with us.  On the
back of this form, you may suggest improvements to
enhance the effectiveness of future reports.  Please
include answers to the following questions if they are
applicable to you:
 
     1.   What additional background information
          about the selection, scheduling, scope,
          or procedures of the audit or inspection
          would have been helpful to the reader in
          understanding this report?
 
     2.   What additional information related to
          findings and recommendations could have
          been included in this report to assist
          management in implementing corrective
          actions?
 
     3.   What format, stylistic, or organizational
          changes might have made this report's overall
          message more clear to the reader?
 
     4.   What additional actions could the Office of
          Inspector General have taken on the issues
          discussed in this report which would have
          been helpful?
 
Please include your name and telephone number so that
we may contact you should we have any questions about
your comments.
 
Name ____________________________
Date_____________________
 
Telephone _______________________
Organization_____________
 
When you have completed this form, you may telefax it
to the Office of Inspector General at (202) 586-0948,
or you may mail it to:
 
     Office of Inspector General (IG-1)
     U.S. Department of Energy
     Washington, D.C. 20585
     ATTN:  Customer Relations
 
If you wish to discuss this report or your comments
with a staff member of the Office of Inspector General,
please contact Wilma Slaughter at (202) 586-1924.
 
 
 
 
