Historical Automated Test Runs

Each time one or more Test Cases is collected into a Test Set to be executed a Test Set entry is created. This Test Set shows all the Test Cases that have been run for this Draft. The details include:

  • Whether the Test Set Run passed or failed (did all Test Cases in the run pass?)

  • When the Test Set was executed

  • Who executed the Test Set

  • What Rave URL and environment the Test Set was run on

Test Sets are a holder for the execution and results of Test Cases. A Test Case can be included in more than one Test Sets but only the last execution of the Test Case counts toward whether it is passed or failed.

Changes to Test Cases

A Test Case can change over time. Imagine a scenario in which a Test Case named "TEST 1" is created and is run as part of a Test Set. In this run the Test Case passes. Later the Test Case is changed - the results of the last run of this Test Case should no longer count. A Test Case called "TEST 1" was run but it us no longer the same Test Case so the system shows this as "Changed" in the Test Set and does not count it as a pass for the Test Case.

Deleted Test Cases

Test Cases can change between runs but they can also be deleted. This can result in a Test Set containing a record of a Test Case result for a Test Case which no longer exists. In this case the Test Case is marked as "Deleted" in the Test Set but it is not removed from the historical record.