4.6.1 Test Planning

The CSC/PBFI Team will develop a test plan and schedule for this project. Among the initial documents to be delivered will be one that describes the overall testing concepts tied to a preliminary schedule. The intent of this step is to allow the CHART staff to become involved in the testing process from the outset and help form the direction of the testing so that the system meets all reasonable expectations.

A good example of a test plan for the software is the one associated with the CHART II Initial Operational Capability (C2IOC). This example offers some insight regarding the complexity of testing. For testing, the C2IOC System is decomposed into 11 Scenario Groups including:

Each Scenario Group is divided into two or more Subgroups. The Scenario Group called Configure System includes eight Subgroups¾ those required to configure groups, sites, ATM switches, processors, monitors, cameras, media access, and duration. Each Subgroup includes two or more user-oriented functional scenarios that invoke subsets of Use Cases previously defined by MDSHA. The C2IOC test scenarios are based on the use cases defined for the C2IOC system. Figure 4-44 depicts the system level use case diagram.

Figure 4-44. C2IOC System Level Use Case Diagram

Processes will be thoroughly tested to determine if they meet their intended functions and uses

Each sub-group exercises the use cases used for configuring the C2IOC system for a specific object. Figure 4-45 shows the use case diagram for configuring the C2IOC system.

 

 Figure 4-45. Use Case Diagram for Configuring the C2IOC System

Configure Groups will be tested using four scenarios

 

Configure Groups includes four scenarios to be tested as noted below.

  1. Get Group Privileges Scenario - This scenario tests if the UI displayed to Privileged CHART Operators and CHART operators is consistent with the privileges assigned to the individual’s Group.

  2. Add/Replace Group Privileges Scenario - This scenario tests if Privileged CHART Operators can add new Groups and replace existing Groups in accordance with their assigned privileges.

  3. Delete Group Privileges Scenario - This scenario tests if Privileged CHART Operators can delete existing Groups in accordance with their assigned privileges.

  4. Update Group Privileges Scenario - This scenario tests if Privileged CHART Operators can update existing Groups in accordance with their assigned privileges.

System Test Plan Development

Test plans will be developed for testing the various system components and the overall system. A specific test plan will be developed for a variety of individual testing activities. Each procedure will be reviewed by CHART Staff prior to execution. All of the plans will follow a similar format to promote the ease of understanding and review, including:

  1. Title Page - The title page will indicate the project name, type of test, submitted by, submitted to, date, and revision number.

  2. Test Procedure Specification Identifier - A description of the criteria under which the test has been developed and defines the document that is used to determine the basis for pass/fail criteria.

  3. Purpose - A description of the purpose of the test, reason for conducting the test, and any history relative to the testing process.

  4. Special Requirements - A description of any special requirements needed to conduct the test. This would include items such as simulators, computers, test equipment, assistance from CHART Staff, and lane closure, or other significant items relating to the execution of the test procedures.

  5. Step-by-Step Procedures - Each test procedure will include a description of the test, the function to be tested, and a step-by-step description how to execute the test. There will also be a sign-off block, which will include testers name, date of test, witnessed by, pass/fail, and a comment block.

Another example of a specific test plan that will be developed for a wide variety of CHART testing activities is noted below. This is a brief extract from the Northern Virginia Signal System test plan, developed by PBFI, that comprises over 130 pages.

System Detector Operations (Filtering Parameters – Low Volume)

This procedure will demonstrate the ability of the system to fail detectors that report a volume lower than the historical volume for that detector by an operator settable percentage.

Parameter_value is the settable parameter in the record "Fail_Det_Volume_Range" that gives the percentage variation the volume reported by the detector is allowed to vary from the historical volume for that detector before it is marked as failed by the system.

  1. Click "MISTÒ " icon

  2. Select "Database" menu

  3. Select "Edit Data"

  4. Select "System Parameters" (highlight, OK)

  5. Go to record named "Fail_Det_Volume_Range"

  6. Record the "Parameter_value" (Definition: this parameter is a percentage, when the volume returned differs from the historical volume by this percentage, the detector is marked failed)

  7. Click "Cancel"

  8. Click "Cancel"

  9. The field shall cause a low volume return (1) on a detector

  10. Verify that the failure appears in the failure status window. (The time for the failure to appear will be determined by the number of incorrect responses required to generate a failure which is a parameter set in the database)

Comments:________________________________________________________________________________________________________________________________________________________________

_____________________________________________________________________________________

Date:__________________________

Tested By (Name):________________________ (Company):_______________

Witnessed By (Name):_____________________ (Company):_______________

 

The project manager will keep records of all of the individual tests run on the project as a part of our documentation practices. A copy of all of the testing information will be provided to the CHART Staff throughout the implementation process.

Test Plan Review

All test plans and procedures developed will be submitted to CHART Staff for review and comment. The CSC/PBFI Team will meet with the CHART Staff to describe the test procedures, the objectives, and expected outcomes of the test. Comments forwarded by the CHART Staff review process will be incorporated into the test plan.