How does the Test Case Advisor work?

1. Folders, Forms and Fields

The Test Case Advisor examines the Edit Check steps to find Folders, Forms and Fields.

If the step specifies a Variable but not a Form and Field then the Test Case Advisor will select the first Form and Field (ordered by Ordinal), for that Variable.

If the Folder is not specified the Test Case Advisor will select a Folder using the following rules (applied in order):

if the Form is in the Default Matrix for the Draft then the Folder in the Default Matrix for that Form will be used

if the Form is used elsewhere in the Edit Check steps or actions, with a Folder specified, that Folder will be used

if it is the Primary Form for the Draft, then the 'SUBJECT' folder will be used

if the Form is used in the AddForm check action of another Edit Check in the Draft, then the Folder for that Check Action will be used (note: if the Check Action does not specify a Folder then the same rules will be applied to find a Folder)

if the Form is in a Matrix other than the Default Matrix then the Folder from that Matrix for that Form will be used

if any other Folder is specified in the Edit Check, then use it

if no Folder has been found, use the first Folder in the Draft (by Ordinal)

2. Test data values

The Test Case Advisor generates lists of possible values for each Field:

for Data Dictionary Fields the Test Case Advisor will select entries from the Data Dictionary. It will avoid 'Other specify' entries, unless the Code for that entry is used as a static value elsewhere in the Edit Check.

for numeric Fields (integer and floats) the Test Case Advisor will use the Non Conformant Lower and Upper ranges (if specified) as the minimum and maximum values, or will use 0 as the minimum value and the maximum value allowed by the Data Format as the maximum. The Test Case Advisor will then create a set of possible values using the minimum, maximum and values in between.

for date Fields the Test Case Advisor uses a starting point of 2 years before the current date. It then creates a set of dates around that point in time; 1 day before/after, 1 month before/after, 1 year before/after, 20 years before, 50 years before, 90 years before.

for time Fields the Test Case Advisor uses the current time and generates a list of times 1 second before/after, 1 minute before/after, 1 hour before/after

for text Fields the Test Case Advisor will use any static values in the Edit Check and an alphabetic string 'ABCDEFGHIJ'

the Test Case Advisor will not choose values which would make the Field non-conformant.

3. Solutions

The logic of the check steps and the test data values are then processed by a constraint solver. The constraint solver attempts to find a set of test data values which will 'solve' the check steps. In rare cases the constraint solver might not find a solution and an issue will be added to the Test Case, for manual resolution.

For Edit Checks, the constraint solver is run twice; once to find a solution which trigger the check actions and once to find a solution which will not trigger the check actions.

Why doesn't the Test Case Advisor generate all possible combinations of test values?

The two test scenarios suggested by the Test Case Advisor can be easily copied in the Test Case Editor if more combinations of data are desired for testing. The Test Case Advisor does not attempt to generate all possible combinations because the number of scenarios becomes surprisingly large.

For example, a common type of Edit Check might compare specific Inclusion/Exclusion criteria and the Eligibility Status:

Here there are 5 Fields each of which can have 3 values (Yes, No and empty) leading to 3^5 = 243 different combinations. Already this is more than anyone would probably want to test but this example was taken from a study which actually has 9 Inclusion critera and 36 Exclusion criteria leading to 8,862,938,119,652,501,095,929 possibilities!