Test Summary Report Identifier By André Vondran - QA Analyst

Some type of unique company generated number to identify this summary report, its level

and the level of software that it is related to. Preferably the report level will be the same as the

related software level. The number may also identify whether the summary report is for the

entire project or a specific level of testing. This is to assist in coordinating software and testware

versions within configuration management.

Summary

Identify all relevant support materials so that the reader of the report knows which version

and release of the project/software is being reported on. It may be particularly important to

identify the specific version of an external package used in the testing, especially if a new release

occurred during the test cycle and was not included. The version/release information should

match the information contained in the configuration management system and may include the

following elements.

· Test Items – This should match the item definitions from the appropriate level test plan

that this report is covering. Any variance from the items specified in the test plan should

be identified. Elements from the features sections of the test plan (both included and

excluded) can also be included here or in a separate reference section.

· Environment – The environment and any variances for that identified in the test plan

should be verified here to ensure that the correct test set-up was used. This will help

avoid confusion when the product is released to production and will ensure that the test

environment matches the destination platform.

· References – Any documents that support this report and their location within the

configuration management system.

Variances

Document any changes or deviations from those areas agreed on in the reference documents,

especially in areas that may cause concern to the group accepting the test results. Include

references to any supporting documentation that covers the reasons for the deviations.

· From Test Plans or Specifications

· Reasons for Deviations

· Support materials and documents

· Change requests

· Enhancement requests

· Incident reports (incident left in by intent)

Comprehensiveness Assessment

Evaluation of the testing and test process in terms of the documented test objectives. This is

to assess the quality and effectiveness of testing so that assessment of the software can be viewed

correctly. Keep in mind that a coverage assessment only has meaning in relation to a known set

of initial test objectives.

Cover how effective testing was, and any weaknesses in the process, especially any

surprising trends and the causes of those deviations.

· Evaluation of test coverage

· Total objectives (by Inventory category)

· Objectives covered (depth and width)

· Objectives omitted and reason for omission

· Identification of uncovered attributes

· Surprising trends and test process changes to cover them

Summary of Results

Report on the overall status of the incidents. Focus should be on trends and patterns in the

process and not on specific individuals or teams. Avoid pure numbers, as numbers due not really

provide insight as to the nature and cause of problems. The focus should be on costs, impacts,

and trends; including any positive trends. This is where you begin to set the stage for the

evaluation of the test process, the quality of the testing and the software quality and can include

areas such as:

· Total Incidents

· By Severity and Priority

· By cost/failure impact

· Defect Patterns

· Open or Unresolved incidents

Evaluation

Based on the evaluation of the testing as documented in sections three (3) through (6) assess

the quality of the software. This should be an objective assessment of the failure likelihood and

overall quality in terms of the criteria specified in the appropriate level test plan. Each item

identified in Section 2 under test items should be covered in the evaluation.

Include Good news as well. If the application tested out with high quality in some areas

(even though others were no so good) be sure to state that as well.

· Limitations

· Incomplete or partial functions/features

· Dropped features (due to requirements change or defects)

· Failure Likelihood

· High or Medium risk areas

· Good quality areas or features

Summary of Activities

Cover the planned activities and the changes to those plans especially in areas where the

amount of actual effort greatly exceeded the planned effort. Include the reasons for the variances

and the possible impact on the testing staff. Major impacts to the testing staff will have possible

negative effects on follow-on projects or on the next project in line.

· Staff time used

· Hours per day/week

· Elapsed time versus staff time

· Is staff working excess hours per week

· Costs – planned versus actual

· Variances and the reasons for the change

· Changes to the project scope and direction

· Requirements and design changes

· Surprising defect trends

· Loss of personnel (development, test, etc.)

· Test environment availability and accuracy

Approvals

Provide a list and signature block for each approving authority. This should match the list of

names that approved the test plan in the first place. Those who agreed to the test plan need to

verify the results of the testing

André Vondran

QA Analyst

www.aqasol.com.ar