Safety Software Quality Assurance Requirements

Parent Policy: Lifecycle Management for Information, Hardware, Software, and Services
Document #: 10.01.003.03

Overview

Safety Software is software whose degradation can have a direct effect on human safety (see full definition in Section F. below). Safety Software at Berkeley Lab must be appropriately, approved, controlled and tested. Our lifecycle management policy requires that Safety Software adhere to the quality assurance requirements in this document.

Management may adopt a subset of these requirements for software that is part of a safety chain, but where a non-software control protects human safety from software degradation.

Our SSQA approach sets four core requirements:

1. The process owner must consider the system as a whole in considering the risks, taking into
   account software and non-software components.
2. The Software must be documented to a level where users, developers, and those providing
   oversight can understand its functions.
3. Tests must be created and executed which clearly show that the software is performing as
   intended across a range of operating conditions. These tests must be repeated at any time that the
   environment or the software changes in a way that could create differences in behavior.
4. Changes to the software must be approved, documented, tested, and archived to provide for
   rigorous, continuous oversight

Safety software includes safety system software, safety and hazard analysis software and design software, and safety management and administrative control software, and performs a safety function as part of a system, structure or component (SSC), and is cited in either a DOE-approved safety analysis document (SAD) or a Lab Directorate-approved hazard analysis document (HAD).

• Safety and Hazard Analysis Software and Design Software is used to classify, design or analyze nuclear/radiological facilities.
• Safety Management and Administrative Controls Software performs a hazard control function in support of nuclear facility or radiological safety management programs or Technical Safety Requirements or other software that performs a control function necessary to provide adequate protection from nuclear facility or radiological hazards.

Applicability

[A] System Software

System Software includes operating systems such as Windows, Linux, and system utilities such as compilers, assemblers, translators, interpreters, query languages, word processing programs, spreadsheet programs, database managers, and graphing programs.

System Software is exempt from these requirements.

[B] Library (Data) Files

Library files e.g., nuclide library files used for Nondestructive Assay (NDA) containing vendor-supplied data shall be controlled by the qualified supplier. Change control is required for the initial library files when they are introduced into the LBNL program for installation, and when they are updated with qualified supplier changes.

Data files that do not affect accuracy or precision, or the quality and correctness of the information, are exempt from these requirements.

[C] Applications Developed Within Commercial-Off-The-Shelf (COTS) or System Software

Specific applications developed for use that fall under the applicability of this Plan, based on COTS or System Software (e.g., Microsoft (MS) Excel spreadsheets, MS-Access databases, detailed formulas, or macros) are not exempt from these requirements.

Calculations that are 100% validated by an alternate calculation method are exempt from these requirements.

[D] Acquired Software

Software that’s initial design, programming, and testing is performed by a third party. Supplier Software can be categorized as COTS, System Software, or Firmware and could be supplied by either a Qualified or Non-qualified Supplier.

D-1 COTS Software

COTS is short for Commercial Off-The-Shelf, an adjective that describes software products that are general purpose, ready-made and available for sale to the general public. COTS products are mass-produced, and are designed to be implemented easily into existing systems without the need for customization by the vendor. COTS excludes Sections A and B above.

COTS software is not exempt from these requirements.

D-2 Firmware

Firmware is vendor-supplied software that is included as an integral part of an instrument (e.g., programmed in a read-only memory ROM chip) and cannot be modified by another party.

Firmware that cannot be removed or updated is exempt from the requirements of this plan. Firmware that is updated by the vendor without respect to particular LBNL requirements is treated as COTS software above.

D-3 Firmware, modified

Firmware, modified, is vendor-supplied software that is included as an integral part of an instrument (e.g., programmable logic controller) and can be modified by another party. Firmware that is removable or updated will be treated as software if the specifications are created by LBNL.

Firmware that can be removed or updated is not exempt from these requirements.

D-5 Vendor-Developed Software

Vendor-Developed software is software that is developed for LBNL to perform a specific end-user function.

Vendor-Developed software is not exempt from these requirements.

D-5 LBNL-Developed Software

LBNL-Developed Software is software developed for use by LBNL to perform a specific end-user function.

LBNL-Developed Software is not exempt from these requirements.

Software Evaluation

New and modified safety software used in facilities will be evaluated to determine the applicable criteria of Software Quality Assurance.

Table 2, Software Evaluation Checklist, will be used for this evaluation.

Software Lifecycle

Changes to software will be uniquely identified and cataloged. An inventory of related systems will be maintained at the system-owner or functional-owner level. An inventory of safety software systems (broadly) will be maintained by the Office of Contract Assurance (OCA).

[A] Labeling System

A labeling system for configuration items shall be implemented by the responsible organization that:

• Uniquely identifies core configuration items.
• Identifies changes to configuration items by revision or version identifier.
• Provides the ability to uniquely identify each approved configuration of the revised software that is available for use.

[B] Change Control

Changes to software shall be systematically proposed, evaluated, documented and approved to ensure that the impact and rationale for making the change is carefully assessed prior to updating the software baseline. Changes to previously accepted software will be subject to the same level of control as the original software.

Information concerning approved changes will be transmitted to affected organizations if software functionality is impacted. A system of change management will be developed and documented. Software verification activities will be performed for the changes, as necessary, to ensure that the change is appropriate reflected in the software documentation and to ensure traceability is maintained. The degree of software validation will be commensurate with the nature and scope of change.

[C] Software Inventory

A software inventory of all applicable software/systems will be maintained that identifies the software name, version classification, , operating environment and the person and responsible organization for the software (i.e. the organization that owns and/or uses the software).

The Software Inventory List (SIL) is maintained as an institutional document by the Office of Contract Assurance, and will be updated by the responsible organization (i.e. the organization that owns and/or uses the software).

The SIL will be evaluated periodically by all responsible organizations, at least once annually, to ensure the data is accurate, current and complete.

[D] Software Problem Reporting

Problems with software, the potential impact of the problem and the corrective action(s) taken are documented by the responsible organization (e.g. may be the organization that owns and/or uses the software or the assigned software developer(s)).

[E] Software Removal and Retirement

The removal and retirement of software applications will be documented by the responsible organization (e.g. may be the organization that owns and/or uses the software or the assigned software developer(s)). Software covered by this policy must be archived in accordance with archive and records schedules for the safety-area they supported.

Software Documentation

[A] Verification and Validation Plan (V&VP)

V&V processes are used to determine if developed software products conform to their requirements, and whether the products from each development phase fulfill the requirements or conditions imposed by the previous phase (verification) and whether the final systems or components comply with specified requirements (validation). This includes analysis, evaluation, review, inspection, assessment, and testing of the software products and the processes that produced the products. Also, the software testing, validation, and verification processes apply when integrating purchased or customer-supplied software products into the developed product. The verification plan should document the verification tasks and the validation plan should document the validation tasks.

Each plan defines the verification and validation tasks and required inputs and outputs needed to maintain the appropriate software integrity level. It also provides a means of verifying the implementation of the requirements of the Requirements Document in the design as expressed in the Design Document and in the testing as expressed in the project’s test documentation.

If desired, the verification plan and validation plan may be packaged together in a single document (i.e. V&V P).

[B] Change Control Document

Change control documentation define the methods and facilities used to maintain, store, secure and document controlled versions and related artifacts of the software through all software life cycle phases.

[C] Requirements Document

The software requirements will be identified, documented and reviewed. The Requirements Document will specify requirements for a particular software product, program, or set of programs that perform certain functions in a specific environment. The Requirements Document may be written by the supplier (internal or external), the customer, or by both. The Requirements Document should address the basic issues of functionality, external interfaces, performance, attributes, and design constraints imposed on implementation. Each requirement will be specified in sufficient detail to permit the accomplishment of design and validation activities. Software Requirements will be traceable throughout the software development cycle, and a V&V Plan will be prepared after the requirements have been documented and approved.

[D] Design Document

The software design will be based on the software requirements and will be documented and reviewed. The Design Document specifies the overall structure of the software (control and data flow) and the reduction of the overall structure into physical solutions (e.g. algorithms, control logic, data structures). The Design Document should describe the components and subcomponents of the software design, including databases and internal interfaces. The Design Document may be prepared first as the architecture design but should be subsequently expanded to address additional detail. The design may necessitate the modification of the Requirements Document and the V&V Plans.

[E] Test Plan

Test Plans identifies the scope, approach, resources and schedule of the testing activities. They also identify the items and features that will be tested, the tests that will be performed, the people responsible for each test and any risks associated with the Plan.

Test Plans define the test approach and identifies the features that are covered by the design and its associated tests. This plan identifies the specific test cases and test procedures, if nay, required to accomplish the testing and specifies the acceptance criteria.

[F] Test Results

Test results are documented and reviewed. Test results describe the results of designated testing activities. Test results identify the summary of the results, any variances that occurred during the test, a summary of activities, the person(s) who performed the tests, the name of the reviewers of the test results.

[G] User Documentation

User documentation guides users in installing, operating, managing and maintaining software products. This does not include modifying source code. User documents should describe the data control inputs, input sequences, options, program limitations, error messages and corrective actions to correct those errors, and other essential information for the software product.

Software Reviews

Software Reviews are conducted at various stages of software development and may include managerial reviews, acquirer-supplier reviews, technical reviews, inspections, walk-throughs, and audits. Software reviews will be documented as well as further actions, implementation of those actions and verification of those actions, if applicable. The following are examples of possible reviews:

• A Requirements Review will be performed to assure the adequacy of the requirements, as appropriate.
• A Design Review will be performed to determine the acceptability of the detailed software designs as depicted in the detailed Design Description in satisfying the requirements of the Requirements Document, as appropriate.
• The V&VP Review will be performed to evaluate the adequacy and completeness of the verification and validation methods defined in the verification and validation plans.
• Other reviews and audits may include the user documentation review. This review is held to evaluate the adequacy (e.g., completeness, clarity, correctness, and usability) of the user documentation.

All reviews performed will be documented by the responsible organization.

Software Testing

Testing will be performed to ensure that the design as implemented in code to assure adherence to the requirements and to assure that the software produces the correct results for the test cases.

To evaluate technical adequacy, the software test case results can be compared to results from alternative test methods such as:

• Hand calculations
• Comparison with other validated software of similar purpose
• Calculations using comparable proven problems
• Empirical data and information from confirmed published data and correlations
• Manual inspections or qualitative checks not involving numerical manipulations (visual inspection of database reformatting or data plotting)

Tests may be performed by persons who did not implement the design who are technically competent.

Supplier Control

Applicable quality assurance requirements will be specified and the required vendor-supplier software documentation, plans, and procedures will be identified in software procurement documentation.

Procured software will be tested in accordance with documented and approved test plans using approved test case specification to ensure that the acquired software will perform satisfactorily in its operating environment. Test plans, and the results of the tests will be identified, documented and retained by the responsible organization.

The responsible organization will perform an acceptance test prior to use of procured software to verify the functional capability of the software and the acceptability of the supplier’s supporting documentation (e.g. user documentation, technical specifications and results of supplier testing). In cases of multiple components from a single vendor, a sampling process approved by the functional owner may be utilized. In some cases, then a representative sample may be one component.

Records

Software quality assurance documentation will be retained by the responsible organization in accordance with the records requirements outlined in the RPM.

Training

Training requirements needed to develop or use the software application, if required,
will be identified by the responsible organization

Graded Approach

Software Quality Assurance is performed using a graded approach, considering aspects such as software applications that are important to safety that may be included or associated with SSCs.

This graded approach takes into consideration that LBNL has multiple radiological and accelerator facilities that have been determined to need a documented and DOE- approved SAD or a Lab Directorate-approved HAD. Radiological and accelerator facilities where a documented SAD is required are considered to be medium to high hazard facilities, and LBNL applies the complete process detailed here to software which meets the definitions in Section 1.

- = Not required

X = Required

VD = Vendor-developed