Independent verification and validation (IV&V) guidelines for statement of work (SOW) content

GUIDELINE FOR IV&V STATEMENT OF WORK

Independent Verification and Validation (IV&V)
Guidelines for Statement of Work (SOW) Content

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 1 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK
TABLE OF CONTENTS
1. Introduction 2. GTA Perspective on IV&V 3. Conflict of Interest Exclusion 4. Contractor Capability 5. Key Personnel 6. Scope of Services
6.1 IV&V Project Management 6.2 Planning Oversight 6.3 Project Oversight 6.4 Quality Management 6.5 Training 6.6 Requirements Management 6.7 Operating Environment 6.8 Development Environment 6.9 Software Development 6.10 System and Acceptance Testing 6.11 Data Management 6.12 Operations Oversight 6.13 Other 7. Deliverables 8. IV&V Reporting Requirements 9. State Furnished Items 10. Travel 11. Points of Contact 12. Contract Period

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 2 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

1. Introduction
Verification and Validation (V&V) is a systems engineering discipline which helps the development organization build quality into the software during the software life cycle. Validation is concerned with checking that the software meets the user's needs, and Verification is concerned with checking that the system is well engineered.
Independent Verification and Validation (IV&V) is a set of Verification and Validation activities performed by an agency that is not under the control of the organization that is developing the software. IV&V services must be provided, managed and financed by organizations that are technically, managerially and financially independent of the development project. Technical independence requires that the IV&V does not use personnel who are involved in the development effort. Managerial independence requires that the IV&V effort be vested in an organization separate from the development and program management organizations. The IV&V must be able to submit to State and Federal management, the IV&V results and findings without any restrictions (e.g. without any prior review or approval from the development group). Financial independence requires that control of the IV&V budget be vested in an organization independent of the development organization.
The definition of activities included under IV&V is quite broad, including both technical and management activities. The Federal approach to IV&V differs considerably from standard IV&V, such as that described in the Institute of Electrical and Electronic Engineers Standard for Software Verification and Validation (IEEE Std 1012-1998). Federal IV&V does not require a continuous on-site presence or extensive testing. It instead requires periodic site visits to get a "snapshot" of a project's management and technical processes at pre-determined intervals (see Federal Perspective on IV&V).
This document defines the IV&V services required by the Georgia Technology Authority (GTA) in support of the GTA automation efforts. This definition includes the periodicity of the IV&V and a description of the activities to be performed.

2. GTA Perspective on IV&V

The Georgia Technology Authority's overall mission was re-crafted by the Georgia Legislature in 2005. The legislations in question are shown below:

BE IT ENACTED BY THE GENERAL ASSEMBLY OF GEORGIA: SECTION 1.

Title 50 of the Official Code of Georgia Annotated, relating to state government, is amended in Code Section 50-25-1, relating to the establishment of the Georgia Technology Authority, by striking the introductory language of subsection (c) preceding paragraph (1) thereof and inserting in its place the following:

"(c) The purpose of the authority shall be to provide for procurement of technology resources, technology enterprise management, and technology portfolio management as defined in this chapter, as well as the centralized marketing, provision, sale, and leasing, or execution of license agreements for access on line or in volume, of certain public information maintained in electronic format to the public, on such terms and conditions as may be determined to be in the best interest of the state in light of the following factors:"

O.C.G.A. Code Section 50-25-4;

"A BILL to be entitled an Act to amend Title 50 of the O.C.G.A., relating to state government, and Title 45 of the O.C.G.A., relating to public officers and employees, so as to provide for the substantial revision and transfer of certain powers, duties, and authority of the Department of Administrative Services, the Georgia Technology Authority, the Office of Planning and Budget, the Board of Regents of the University System of Georgia, the Department of Agriculture, the Department of Veterans Service, and the state accounting officer; to change certain provisions regarding the establishment,

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 3 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK
powers, purchasing authority, procedures, and limitations and vendor qualification of the Georgia Technology Authority."
Typically Independent Verification and Validation means a technical evaluation of a work product. Validation means making sure the right product or service is built / presented, and verification means making sure it is built / presented correctly.
Offerors to the Independent Verification & Validation (IV&V) Contractor Qualification (980-270020) solicitation need to be aware that the requirements of IV&V on the State's project do not necessarily conform to industry standard practices for IV&V as defined in the IEEE Standard for Software Verification and Validation (IEEE Standard 1012-1098). The Federal requirements for IV&V are, in fact, a subset of the full IV&V standard as defined by the IEEE Standard 1012-1098. The IEEE Standard describes software IV&V processes as generally determining if development products of a given activity conform to the requirements of that activity, and if the software satisfies the intended use and user needs. As defined in the IEEE standards, IV&V processes include activities such as assessment, analysis, evaluation, review, inspection, and testing of software products and processes. These IV&V processes further include assessing software in the context of the system, including the operational environment, hardware, interfacing software, operators and users. The IEEE standard seeks to assure that software IV&V is performed in parallel with software development, not at the conclusion of the software development.
However, the Federal requirements for IV&V on State automation projects are limited in their scope from the industry standard IEEE definition for IV&V in two key regards:
1. IV&V of the project is not considered to be an ongoing, integral process within the larger development project. Rather, it is considered to be a periodically performed adjunct activity that does not fall within the managerial oversight or control of the day-to-day operation of the Project's management structure, including any and all of its "umbrella" agencies. The IV&V Service Provider must maintain organizational independence and autonomy from the project's organization, and therefore has a reduced role from that normally associated with full IV&V services. Further, in some respects, the IV&V Service Provider can be viewed as performing a "Technology Audit." Offerors to this solicitation should not view their role as that of providing a "continuous presence" to the project, such as might be the case with Quality Control and Assurance services. Though such fulltime IV&V services could be deemed appropriate at some point in the future, it is not the intent of this guideline document to acquire such full-time IV&V resources for the project.
2. The IV&V Service Provider shall provide its detailed, structured reports of findings of deficiencies and recommendations for their remediation to the GTA. This reporting process, in accordance with GTA regulatory requirements, includes not only final report issuance, but all draft report submissions as well. Again, the intent of the GTA and State in acquiring an IV&V Service Provider under this procurement, unlike that which might be defined under the IEEE 1012-1098 standards for IV&V, is not to continually work with various project components to actively participate in the remediation of deficiencies and risks. Rather, the requirement for the IV&V Service Provider under this procurement calls for the offeror to provide periodic, independent analyses of the areas of responsibility as presented within the scope of services of this solicitation in order to identify, inform and educate project management as well as the cognizant state Office of any areas of weakness and risk to the project, as well as the proposed and recommended solutions for their remediation and/or mitigation.
Any offeror whose proposal suggests a constant presence on or within the project will likely find their costs unnecessarily higher than those of an offeror who proposes to accomplish the same mission

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 4 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK
(from IV&V review initiation to final report delivery and presentation) within well defined, periodic timeframes.
3. Conflict of Interest Exclusion
Any contractor (and its subcontractors) serving in the role of IV&V Service Contractor/Provider to the State Project is prohibited from soliciting, proposing or being awarded any project management, quality assurance, software design, development or other manner of implementation phase work (excluding IV&V services) on that Project. The effect of this is to preclude from this solicitation all contractors who are involved in, prepare, advise on, or have access to the creation and/or preparation of a Statement of Work (SOW) or Request for Proposal (RFP) or other solicitation vehicle. This also precludes those closely involved with the State staff responsible for such solicitation preparations.
4. Contractor Capability
The offeror must have a demonstrated ability to perform the following activities, which are the same as those stated in GTA 980-270020 (Independent Verification & Validation (IV&V) Contractor Qualification):
Develop a project work plan. The work plan may require direct submission to the cognizant Federal Office at the same time it is given to the State.
Review and make recommendations on both the management of the project, both State and vendor, and the technical aspects of the project. The results of this analysis may require direct submission to the cognizant Federal Office at the same time it is given to the State.
Consult with all stakeholders and assess the user involvement and buy-in regarding system functionality and the system's ability to meet program needs.
Conduct an analysis of past project performance (schedule, budget) sufficient to identify and make recommendations for improvement.
Provide a risk management assessment and capacity planning services.
Develop performance metrics which allow tracking of project completion against milestones set by the State.
At a more detailed level, the offeror must demonstrate the corporate knowledge and experience to:
Develop a project management plan, including recommendations for: adequate staff; staff skills, positions and abilities; equipment resources; training and facilities; and functional responsibility and authority within a structured project organization.
Analyze project management; evaluate project progress, resources, budget, schedules, work flow and reporting.
Review and analyze project management planning documents.
Review and analyze project software development documents.
Review and monitor development processes to ensure they are being documented, carried out, and analyzed for improvement.
Assess the project's Configuration Management function/organization by reviewing CM reports and making recommendations regarding appropriate processes and tools to manage system changes.
Perform a detailed review of project deliverables for accuracy, completeness, and adherence to contractual and functional requirements.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 5 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK
Perform a detailed review of the system documentation (Requirements, Design, Training, Test, and Management Plans, etc.) for accuracy and completeness.
Perform a detailed review of the software architecture for feasibility, consistency, and adherence to industry standards.
Inventory and review the application software for completeness and adherence to programming standards for the project.
Analyze application, network, hardware and software operating platform performance characteristics relative to expected/anticipated/contractually guaranteed results and industry standards/expectations.
Review the process for tracking of business and technical requirements to their source and review the process established during the planning phase for requirements traceability throughout the subsequent development/implementation phase. Review the traceability of system requirements to design, code, test, and training.
Assess and recommend improvement, as needed, to assure maintenance of a data center, including data center input to the project regarding operational and maintenance performance of the application.
Assess and recommend improvement, as needed, to assure software testing is being performed adequately through review of test plans or other documentation and through direct observation of testing where appropriate, including participation in and coordination of peer reviews.
Assess and recommend improvement, as needed, to assure appropriate user and developer training is planned and carried out.
Review system hardware and software configuration and report on any compatibility and obsolescence issues.
Review and analyze system capacity studies.
5. Key Personnel
Each proposal for IV&V services must include the experience and skills of the key personnel proposed for the IV&V analysis and specify by name the key personnel who actually will work on the project. The contractor and the GTA agree that the key personnel are critical to the performance of the contract and cannot be removed without GTA approval. The State has the right of refusal for any personnel assigned to these tasks.
After contract award, the IV&V provider shall secure written approval from GTA prior to making any changes to key personnel. The GTA will be notified, in writing, of any changes in the personnel assigned to these tasks. Qualifications for suggested staff changes should be comparable with those being replaced.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 6 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6. Scope of Services
The following section contains lists of individual IV&V activities. All activities in Section 6.1 through 6.13 are mandatory IV&V activities unless otherwise indicated. These activities are to be considered part of this solicitation and should be costed and scheduled in the IV&V Project Management Plan and reported on in the Initial and Periodic Reports.

6.1 IV&V Project Management IV&V PROJECT MANAGEMENT

TASK ITEM

TASK #

IV&V

IM-1

Management

Plan

Conduct

IM-2

Initial Review

TASK DESCRIPTION
As the first deliverable the IV&V provider shall develop an IV&V Management Plan. This plan shall describe the activities, personnel, schedule, standards, and methodology for conducting the IV&V reviews. (see Deliverables for more details)
Prepare and deliver an Initial IV&V report on the required activities. Report on status of each activity. (see Deliverables for more details)

Conduct Periodic Review(s)

IM-3

Management IM-4 Briefing

Prepare and deliver a Follow-up IV&V report on the required activities. Report on status of each activity and progress since the previous report. (see Deliverables for more details)
Prepare and deliver a formal presentation(s) on the status of the IV&V project. Presented as required, with at least ten (10) business days notice. No more than once a month. (see Deliverables for more details)

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 7 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.2 Planning Oversight

TASK ITEM

TASK # TASK DESCRIPTION

Procurement

PO-1 PO-2 PO-3 PO-4
PO-5 PO-6 PO-7

Verify the procurement strategy supports State and Federal project objectives.
Review and make recommendations on the solicitation documents relative to their ability to adequately inform potential vendors about project objectives, requirements, risks, etc.
Verify the evaluation criteria are consistent with project objectives and evaluation processes are consistently applied; verify all evaluation criteria is metrics based and clearly articulated within the solicitation documents.
Verify that the obligations of the vendor, sub-contractors and external staff (terms, conditions, statement of work, requirements, technical standards, performance standards, development milestones, acceptance criteria, delivery dates, etc.) are clearly defined. This includes verifying that performance metrics have been included that will allow tracking of project performance and progress against criteria set by the State.
Verify the final contract for the vendor team states that the vendor will participate in the IV&V process, being cooperative for coordination and communication of information.
Review and evaluate the PAPD(U)/IAPD(U) documents.
Review and evaluate the Cost Benefit Analysis to assess its reasonableness.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 8 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.3 Project Management

TASK ITEM

TASK # TASK DESCRIPTION

Project Sponsorship

PM-1 Assess and recommend improvement, as needed, to assure continuous executive stakeholder buy-in, participation, support and commitment, and that open pathways of communication exist among all stakeholders.

PM-2 Verify that executive sponsorship has bought-in to all changes which impact project objectives, cost, or schedule.

Management Assessment

PM-3 Verify and assess project management and organization, verify that lines of reporting and responsibility provide adequate technical and managerial oversight of the project.

PM-4 Evaluate project progress, resources, budget, schedules, work flow, and reporting.

PM-5 Assess coordination, communication and management to verify agencies and departments are not working independently of one another and following the communication plan.

Project Management

PM-6 Verify that a Project Management Plan is created and being followed. Evaluate the project management plans and procedures to verify that they are developed, communicated, implemented, monitored and complete.

PM-7 Evaluate project reporting plan and actual project reports to verify project status is accurately traced using project metrics.

PM-8 Verify milestones and completion dates are planned, monitored, and met.

PM-9

Verify the existence and institutionalization of an appropriate project issue tracking mechanism that documents issues as they arise, enables communication of issues to proper stakeholders, documents a mitigation strategy as appropriate, and tracks the issue to closure. This should include but is not limited to technical and development efforts.

PM-10 Evaluate the system's planned life-cycle development methodology or methodologies (waterfall, evolutionary spiral, rapid prototyping, incremental, etc.) to see if they are appropriate for the system being developed.

Business Process Reengineering

PM-11 Evaluate the project's ability and plans to redesign business systems to achieve improvements in critical measures of performance, such as cost, quality, service, and speed.

PM-12 Verify that the reengineering plan has the strategy, management backing, resources, skills and incentives necessary for effective change.

PM-13 Verify that resistance to change is anticipated and prepared for by using principles of change management at each step (such as excellent communication, participation, incentives) and having the appropriate leadership (executive pressure, vision, and actions) throughout the reengineering process.

Risk Management

PM-14 Verify that a Project Risk Management Plan is created and being followed. Evaluate the projects risk management plans and procedures to verify that risks are identified and quantified and that mitigation plans are developed,

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 9 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

TASK ITEM

TASK # TASK DESCRIPTION

communicated, implemented, monitored, and complete.

Change Management

PM-15 Verify that a Change Management Plan is created and being followed. Evaluate the change management plans and procedures to verify they are developed, communicated, implemented, monitored, and complete; and that resistance to change is anticipated and prepared for.

Communication Management

PM-16 Verify that a Communication Plan is created and being followed. Evaluate the communication plans and strategies to verify they support communications and work product sharing between all project stakeholders; and assess if communication plans and strategies are effective, implemented, monitored and complete.

Configuration Management

PM-17 Review and evaluate the configuration management (CM) plans and procedures associated with the development process.

PM-18 Verify that all critical development documents, including but not limited to requirements, design, code and JCL are maintained under an appropriate level of control.

PM-19 Verify that the processes and tools are in place to identify code versions and to rebuild system configurations from source code.

PM-20 Verify that appropriate source and object libraries are maintained for training, test, and production and that formal sign-off procedures are in place for approving deliverables.

PM-21 Verify that appropriate processes and tools are in place to manage system changes, including formal logging of change requests and the review, prioritization and timely scheduling of maintenance actions.

PM-22 Verify that mechanisms are in place to prevent unauthorized changes being made to the system and to prevent authorized changes from being made to the wrong version.

Project Estimating and Scheduling

PM-23 Evaluate and make recommendations on the estimating and scheduling process of the project to ensure that the project budget and resources are adequate for the work-breakdown structure and schedule.

PM-24 Review schedules to verify that adequate time and resources are assigned for planning, development, review, testing and rework.

PM-25 Examine historical data to determine if the project/department has been able to accurately estimate the time, labor and cost of software development efforts.

Project Personnel

PM-26 Examine the job assignments, skills, training and experience of the personnel involved in program development to verify that they are adequate for the development task.

PM-27 Evaluate the State's hiring plan for the project to verify that adequate human resources will be available for development and maintenance.

PM-28 Evaluate the State's personnel policies to verify that staff turnover will be minimized.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 10 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

TASK ITEM

TASK # TASK DESCRIPTION

Project Organization

PM-29 Verify that lines of reporting and responsibility provide adequate technical and managerial oversight of the project.

PM-30 Verify that the project's organizational structure supports training, process definition, independent Quality Assurance, Configuration Management, product evaluation, and any other functions critical for the projects success.

Subcontractors and External Staff

PM-31 Evaluate the use of sub-contractors or other external sources of project staff (such as IS staff from another State organization) in project development.
PM-32 Verify that the obligations of sub-contractors and external staff (terms, conditions, statement of work, requirements, standards, development milestones, acceptance criteria, delivery dates, etc.) are clearly defined.

PM-33 Verify that the subcontractors' software development methodology and product standards are compatible with the system's standards and environment.

PM-34 Verify that the subcontractor has and maintains the required skills, personnel, plans, resources, procedures and standards to meet their commitment. This will include examining the feasibility of any offsite support of the project

PM-35 Verify that any proprietary tools used by subcontractors do not restrict the future maintainability, portability, and reusability of the system.

State Oversight PM-36 Verify that State oversight is provided in the form of periodic status reviews and technical interchanges.

PM-37 Verify that the State has defined the technical and managerial inputs the subcontractor needs (reviews, approvals, requirements and interface clarifications, etc.) and has the resources to supply them on schedule.

PM-38 Verify that State staff has the ultimate responsibility for monitoring project cost and schedule.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 11 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.4 Quality Management

TASK ITEM

TASK # TASK DESCRIPTION

Quality Assurance

QA-1

Evaluate and make recommendations on the project's Quality Assurance plans, procedures and organization.

QA-2

Verify that QA has an appropriate level of independence from project management.

QA-3

Verify that the QA organization monitors the fidelity of all defined processes in all phases of the project.

QA-4

Verify that the quality of all products produced by the project is monitored by formal reviews and sign-offs.

QA-5

Verify that project self-evaluations are performed and that measures are continually taken to improve the process.

QA-6

Monitor the performance of the QA contractor by reviewing its processes and reports and performing spot checks of system documentation; assess findings and performance of the processes and reports.

QA-7

Verify that QA has an appropriate level of independence; evaluate and make recommendations on the project's Quality Assurance plans, procedures and organization.

QA-8

Verify that the QA vendor provides periodic assessment of the CMM activities of the project and that the project takes action to reach and maintain CMM Level __.

QA-9

Evaluate if appropriate mechanisms are in place for project self-evaluation and process improvement.

Process Definition and Product Standards

QA-10 QA-11

Review and make recommendations on all defined processes and product standards associated with the system development.
Verify that all major development processes are defined and that the defined and approved processes and standards are followed in development.

QA-12 Verify that the processes and standards are compatible with each other and with the system development methodology.

QA-13 Verify that all process definitions and standards are complete, clear, up-todate, consistent in format, and easily available to project personnel

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 12 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.5 Training

TASK ITEM

TASK #

User Training TR-1 and Documentation

TR-2

TR-3

TR-4

TR-5

Developer

TR-6

Training and

Documentation TR-7

TR-8

TR-9

TASK DESCRIPTION
Review and make recommendations on the training provided to system users. Verify sufficient knowledge transfer for maintenance and operation of the new system.
Verify that training for users is instructor-led and hands-on and is directly related to the business process and required job skills.
Verify that user-friendly training materials and help desk services are easily available to all users.
Verify that all necessary policy and process and documentation is easily available to users.
Verify that all training is given on-time and is evaluated and monitored for effectiveness, with additional training provided as needed.
Review and make recommendations on the training provided to system developers.
Verify that developer training is technically adequate, appropriate for the development phase, and available at appropriate times.
Verify that all necessary policy, process and standards documentation is easily available to developers.
Verify that all training is given on-time and is evaluated and monitored for effectiveness, with additional training provided as needed.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 13 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.6 Requirements Management

TASK ITEM

TASK # TASK DESCRIPTION

Requirements RM-1 Management

Evaluate and make recommendations on the project's process and procedures for managing requirements.

RM-2

Verify that system requirements are well-defined, understood and documented.

RM-3

Evaluate the allocation of system requirements to hardware and software requirements.

RM-4

Verify that software requirements can be traced through design, code and test phases to verify that the system performs as intended and contains no unnecessary software elements.

RM-5 Verify that requirements are under formal configuration control.

Security

RM-6

Requirements

Evaluate and make recommendations on project policies and procedures for ensuring that the system is secure and that the privacy of client data is maintained.

RM-7 Evaluate the projects restrictions on system and data access.

RM-8 Evaluate the projects security and risk analysis.

RM-9

Verify that processes and equipment are in place to back up client and project data and files and archive them safely at appropriate intervals.

Requirements RM-10 Analysis

Verify that an analysis of client, State and federal needs and objectives has been performed to verify that requirements of the system are well understood, well defined, and satisfy federal regulations.

RM-11

Verify that all stakeholders have been consulted to the desired functionality of the system, and that users have been involved in prototyping of the user interface.

RM-12 Verify that all stakeholders have bought-in to all changes which impact project objectives, cost, or schedule.

RM-13 Verify that performance requirements (e.g. timing, response time and throughput) satisfy user needs

RM-14 Verify that user's maintenance requirements for the system are completely specified

Interface

RM-15

Requirements

Verify that all system interfaces are exactly described, by medium and by function, including input/output control codes. data format, polarity, range, units, and frequency.

RM-16

Verify those approved interface documents are available and that appropriate relationships (such as interface working groups) are in place with all agencies and organizations supporting the interfaces.

Requirements RM-17 Allocation and Specification
RM-18

Verify that all system requirements have been allocated to a either a software or hardware subsystem.
Verify that requirements specifications have been developed for all hardware

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 14 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

Reverse Engineering

RM-19

and software subsystems in a sufficient level of detail to ensure successful implementation.
If a legacy system or a transfer system is or will be used in development, Verify that a well defined plan and process for reengineering the system is in place and is followed. The process, depending on the goals of the reuse/transfer, may include reverse engineering, code translation, redocumentation, restructuring, normalization, and re-targeting.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 15 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.7 Operating Environment

TASK ITEM

TASK # TASK DESCRIPTION

System Hardware

OE-1

Evaluate new and existing system hardware configurations to determine if their performance is adequate to meet existing and proposed system requirements.

OE-2

Determine if hardware is compatible with the State's existing processing environment, if it is maintainable, and if it is easily upgradeable. This evaluation will include, but is not limited to CPUs and other processors, memory, network connections and bandwidth, communication controllers, telecommunications systems (LAN/WAN), terminals, printers and storage devices.

OE-3

Evaluate current and projected vendor support of the hardware, as well as the State's hardware configuration management plans and procedures.

System Software

OE-4

Evaluate new and existing system software to determine if its capabilities are adequate to meet existing and proposed system requirements.

OE-5

Determine if the software is compatible with the State's existing hardware and software environment, if it is maintainable, and if it is easily upgradeable. This evaluation will include, but is not limited to, operating systems, middleware, and network software including communications and file-sharing protocols.

OE-6

Current and projected vendor support of the software will also be evaluated, as well as the States software acquisition plans and procedures.

Database Software

OE-7

Evaluate new and existing database products to determine if their capabilities are adequate to meet existing and proposed system requirements.

OE-8

Determine if the database's data format is easily convertible to other formats, if it supports the addition of new data items, if it is scaleable, if it is easily refreshable and if it is compatible with the State's existing hardware and software, including any on-line transaction processing (OLTP) environment.

OE-9

Evaluate any current and projected vendor support of the software, as well as the State's software acquisition plans and procedures.

System Capacity

OE-10 Evaluate the existing processing capacity of the system and verify that it is adequate for current statewide needs for both batch and on-line processing.

OE-11 Evaluate the historic availability and reliability of the system including the frequency and criticality of system failure.

OE-12 Evaluate the results of any volume testing or stress testing.

OE-13 Evaluate any existing measurement and capacity planning program and will evaluate the system's capacity to support future growth.

OE-14

Make recommendations on changes in processing hardware, storage, network systems, operating systems, COTS software, and software design to meet future growth and improve system performance.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 16 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.8 Development Environment

TASK ITEM

TASK # TASK DESCRIPTION

Development DE-1 Hardware

Evaluate new and existing development hardware configurations to determine if their performance is adequate to meet the needs of system development.

DE-2

Determine if hardware is maintainable, easily upgradeable, and compatible with the State's existing development and processing environment. This evaluation will include, but is not limited to CPUs and other processors, memory, network connections and bandwidth, communication controllers, telecommunications systems (LAN/WAN), terminals, printers and storage devices.

DE-3

Current and projected vendor support of the hardware will also be evaluated, as well as the State's hardware configuration management plans and procedures.

Development DE-4 Software

Evaluate new and existing development software to determine if its capabilities are adequate to meet system development requirements.

DE-5

Determine if the software is maintainable, easily upgradeable, and compatible with the State's existing hardware and software environment.

DE-6

Evaluate the environment as a whole to see if it shows a degree of integration compatible with good development. This evaluation will include, but is not limited to, operating systems, network software, CASE tools, project management software, configuration management software, compilers, cross-compilers, linkers, loaders, debuggers, editors, and reporting software.

DE-7

Language and compiler selection will be evaluated with regard to portability and reusability (ANSI standard language, non-standard extensions, etc.)

DE-8

Current and projected vendor support of the software will also be evaluated, as well as the States software acquisition plans and procedures.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 17 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.9 Software Development

TASK ITEM

TASK # TASK DESCRIPTION

High-Level Design

SD-1

Evaluate and make recommendations on existing high level design products to verify the design is workable, efficient, and satisfies all system and system interface requirements.

SD-2

Evaluated the design products for adherence to the project design methodology and standards.

SD-3

Evaluate the design and analysis process used to develop the design and make recommendations for improvements. Evaluate design standards, methodology and CASE tools used will be evaluated and make recommendations.

SD-4 Verify that design requirements can be traced back to system requirements.

SD-5

Verify that all design products are under configuration control and formally approved before detailed design begins.

Detailed Design

SD-6

Evaluate and make recommendations on existing detailed design products to verify that the design is workable, efficient, and satisfies all high level design requirements.

SD-7

The design products will also be evaluated for adherence to the project design methodology and standards.

SD-8

The design and analysis process used to develop the design will be evaluated and recommendations for improvements made.

SD-9

Design standards, methodology and CASE tools used will be evaluated and recommendations made.

SD-10 Verify that design requirements can be traced back to system requirements and high level design.

SD-11 Verify that all design products are under configuration control and formally approved before coding begins.

Job Control

SD-12 Perform an evaluation and make recommendations on existing job control and on the process for designing job control.

SD-13 Evaluate the system's division between batch and on-line processing with regard to system performance and data integrity.

SD-14 Evaluate batch jobs for appropriate scheduling, timing and internal and external dependencies.

SD-15 Evaluate the appropriate use of OS scheduling software.

SD-16 Verify that job control language scripts are under an appropriate level of configuration control.

Code

SD-17 Evaluate and make recommendations on the standards and process currently in place for code development.

SD-18 Evaluate the existing code base for portability and maintainability, taking software metrics including but not limited to modularity, complexity and

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 18 of 25

TASK ITEM Unit Test

GUIDELINE FOR IV&V STATEMENT OF WORK

TASK # SD-19 SD-20
SD-21 SD-22 SD-23 SD-24 SD-25

TASK DESCRIPTION
source and object size.
Code documentation will be evaluated for quality, completeness (including maintenance history) and accessibility.
Evaluate the coding standards and guidelines and the projects compliance with these standards and guidelines. This evaluation will include, but is not limited to, structure, documentation, modularity, naming conventions and format.
Verify that developed code is kept under appropriate configuration control and is easily accessible by developers.
Evaluate the project's use of software metrics in management and quality assurance.
Evaluate the plans, requirements, environment, tools, and procedures used for unit testing system modules.
Evaluate the level of test automation, interactive testing and interactive debugging available in the test environment.
Verify that an appropriate level of test coverage is achieved by the test process, that test results are verified, that the correct code configuration has been tested, and that the tests are appropriately documented.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 19 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.10 System and Acceptance Testing

TASK ITEM

TASK # TASK DESCRIPTION

System Integration Test

ST-1 ST-2

Evaluate the plans, requirements, environment, tools, and procedures used for integration testing of system modules.
Evaluate the level of automation and the availability of the system test environment.

ST-3

Verify that an appropriate level of test coverage is achieved by the test process, that test results are verified, that the correct code configuration has been tested, and that the tests are appropriately documented, including formal logging of errors found in testing.

ST-4

Verify that the test organization has an appropriate level of independence from the development organization.

Pilot Test

ST-5

Evaluate the plans, requirements, environment, tools, and procedures for pilot testing the system.

ST-6

Verify that a sufficient number and type of case scenarios are used to ensure comprehensive but manageable testing and that tests are run in a realistic, real-time environment.

ST-7

Verify that test scripts are complete, with step-by-step procedures, required pre-existing events or triggers, and expected results.

ST-8

Verify that test results are verified, that the correct code configuration has been used, and that the tests runs are appropriately documented, including formal logging of errors found in testing.

ST-9

Verify that the test organization has an appropriate level of independence from the development organization.

Interface Testing

ST-10 Evaluate interface testing plans and procedures for compliance with industry standards.

Acceptance ST-11 and Turnover

Acceptance procedures and acceptance criteria for each product must be defined, reviewed, and approved prior to test and the results of the test must be documented. Acceptance procedures must also address the process by which any software product that does not pass acceptance testing will be corrected.

ST-12 Verify that appropriate acceptance testing based on the defined acceptance criteria is performed satisfactorily before acceptance of software products.

ST-13 Verify that the acceptance test organization has an appropriate level of independence from the subcontractor.

ST-14

Verify that training in using the contractor-supplied software is be on-going throughout the development process, especially If the software is to be turned over to State staff for operation.

ST-15 Review and evaluate implementation plan.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 20 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.11 Data Management

TASK ITEM

TASK # TASK DESCRIPTION

Data Conversion

DM-1

Evaluate the State's existing and proposed plans, procedures and software for data conversion.

Database Design

DM-2
DM-3 DM-4 DM-5

Verify that procedures are in place and are being followed to review the completed data for completeness and accuracy and to perform data clean-up as required.
Determine conversion error rates and if the error rates are manageable.
Make recommendations on making the conversion process more efficient and on maintaining the integrity of data during the conversion.
Evaluate new and existing database designs to determine if they meet existing and proposed system requirements.

DM-6 DM-7 DM-8

Recommend improvements to existing designs to improve data integrity and system performance.
Evaluate the design for maintainability, scalability, refreshability, concurrence, normalization (where appropriate) and any other factors affecting performance and data integrity.
Evaluate the project's process for administering the database, including backup, recovery, performance analysis and control of data item creation.

6.12 Operations Oversight

TASK ITEM

TASK # TASK DESCRIPTION

Operational

OO-1

Change Tracking

Evaluate statewide system's change request and defect tracking processes.

OO-2

Evaluate implementation of the process activities and request volumes to determine if processes are effective and are being followed.

Customer & User OO-3 Operational Satisfaction

Evaluate user satisfaction with system to determine areas for improvement

Operational Goals OO-4 Evaluate impact of system on program goals and performance standards.

Operational

OO-5

Documentation

Evaluate operational plans and processes.

Operational Processes and Activity

OO-6

Evaluate implementation of the process activities including backup, disaster recovery and day-to-day operations to verify the processes are being followed.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 21 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

6.13 Other

TASK ITEM

TASK #

(Provide

OTH-1

subject name)

TASK DESCRIPTION (Insert project task(s) not previously addressed in Tasks selections above.)

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 22 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

7. Deliverables
The following table identifies the anticipated deliverables. The GTA reserves the right to request additional analyses, as needed. The IV&V vendor may suggest development of additional deliverables in specific areas. The State must authorize the need for any additional deliverables prior to their development.
Where applicable, the deliverable must be developed in accordance with IEEE (or substantially similar) standards using the State's templates and guides. If no applicable template exists the deliverable will be developed in a format similar to the existing template. When no applicable standard exists, the methodology and processes used in the analysis and creation of the deliverable must be briefed to the State prior to its use, and described in the final deliverable. All deliverables, standards, processes, plans, and applicable reference material should be available upon request of the State.
Copies of all deliverables will be delivered to the GTA. The GTA reserves the right to extend the due date if appropriate, due to document size, schedule or changes in scope. The IV&V Vendor must notify the GTA of an anticipated delay of a deliverable, as far in advance of the due date as possible. The Deliverables Acceptance process will be the default method of submitting deliverables for acceptance and payment.
It should also be understood that the deliverables will vary based on the IV&V Vendors entry into project phase.
If the entry is during planning and initiation the activities and reports will focused on the RFP development and award process. If award takes place the activities and reports will be focused on execution.
If entry is at project award the tasks will generally occur as listed below. Additional deliverables will be added in accordance with the customer's request.

TASK Develop IV&V Management Plan Conduct Initial Review
Stage Gate Reviews
Critical Panel Review

DELIVERABLE IV&V Management Plan
Initial Review Report
Periodic Review Report(s)
Formal presentations to the GTA executive Leadership and the Governors on the IV&V project.

TIME PERIOD
To be completed within __ days from the date the contractor is notified of contract approval
Review to commence within __ days from the date the contractor is notified of contract approval.
Report to be delivered __ days after the review commences.
Review to commence ___ months following the start of the previous review. Report to be delivered 60 days after the review commences.
These reviews and reports shall continue every ___ months.
Monthly

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 23 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

The deliverables to be provided during this contract are:
The IV&V Management Plan shall include project plan and a schedule with deliverables, as described in the Scope of Services.
The Initial Review Report shall evaluate the State's project in all the areas described in the Scope of Services. The report must provide project context and quantitative data on each area analyzed. It shall include detailed recommendations on how the State can improve its development process. The recommendations shall be prioritized so that the State knows what steps would be most beneficial to take next
The Periodic Review Report(s) shall reevaluate the health of the project at predefined stages of the project. All areas reviewed during the initial assessment will be reexamined and recommendations for improvement included.
The Critical Review Panel briefing shall be a formal presentation on a monthly basis to the GTA Executive Panel and the Governor's Critical Review Panel. The format for the presentation is a document referred to as the "Dash Board". In cases where a project is not on the critical panel the presentations will be given to GTA each month.
All deliverables shall be approved by the GTA in order for the task which produced them to be considered complete. In all cases, the payment to the IV&V provider shall be contingent upon GTA approval of deliverables. No review will be considered complete until the approved documentation is delivered to and reviewed by the GTA.
Each proposal for IV&V services must include descriptions of the actions that shall be taken to produce the deliverables and obtain GTA approval. In addition, each proposal must include a proposed format and content outline for each deliverable.
The GTA must approve, in writing, changes to milestones, deliverables or other material changes to the contract prior to implementation of changes.

8. IV&V Reporting Requirements

All deliverables will follow the Deliverables Acceptance Process.

DOCUMENT TYPE

FORMAT

Word Processing

Microsoft Word 2003

Spreadsheets

Microsoft Excel 2003

Graphics

Microsoft PowerPoint 2003

Project Management

Microsoft Project 2003

9. State Furnished Items
Workspace for up to three contractor staff while on-site at the projects for the duration of the contract. The workspace will include desk or tables, phone and access to the internet. The contractor is expected to have regular office space separate from the project site.
Access to project information, including, but not limited to, technical documentation and project status data.
Access to project and contractor personnel for information related to the project.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 24 of 25

GUIDELINE FOR IV&V STATEMENT OF WORK

The State is not responsible for providing clerical or administrative support to the vendor.

10. Travel The vendor is expected to include all travel and accommodations expense in the submitted proposal.

11. Points of Contact

The State points of contact for this SOW are:

NAME

PHONE(S)

FAX

E-MAIL

12. Contract Period
It is anticipated that the start date of these services will be __________ and the end date will be _________. Contract execution is contingent on prior Federal approval of the resulting IV&V contract, and upon the availability of state funding. Contract extensions may be negotiated on a twelve (12) month basis.

Version 1.0

This is a controlled document Refer to the GTA Web Site for the current version

Jun 1, 2009 Page 25 of 25