Test Case Advisor

The TrialGrid Test Case Advisor suggests Test Cases for Edit Checks by analyzing the Check Steps and generating Test Steps which will trigger the Check Actions.

See also

For more information about the structure of Test Cases see Test Cases

Running the Test Case Advisor

The Test Case Advisor can be run for a single Edit Check by opening the Edit Check editor, expanding the side bar and clicking the 'Add' or 'Replace' button. 'Replace' will delete any existing Test Cases related to that Edit Check before creating a new one.

The Test Case Advisor can be run for one or more Edit Checks from the Edit Checks list page by selecting Edit Checks using the checkboxes in the right hand columns and then clicking the 'Test Case Advisor' button. This will create Test Cases for the selected Edit Checks. When creating Test Cases from the Edit Checks list page the following options are available:

EDC Role

A default EDC Role should be selected which will be suitable for running the Test Cases.

Subject Step

The step which will be used to create or select a subject in the Test Case(s), for example 'a subject exists'.

Delete existing test cases?

Existing Test Cases may be deleted by selecting the 'Delete Existing' option. If 'Delete Existing' is not selected then the Test Case Advisor will create Test Cases only for Edit Checks which do not already have a Test Case. Edit Checks which are inactive or invalid (for example, have no Check Actions defined) will not have Test Cases suggested.

Create scenarios with unknown dates?

If selected, the Test Case Advisor will create test scenarios which include unknown date entry. For example if a field AESTDAT has Rave format 'dd- MMM- yyyy' then the Test Case Advisor will create a scenario for unknown days (e.g. 'UN JAN 2020'), for unknown months (e.g. '10 UNK 2020') and for all unknowns (e.g. 'UN UNK 2020'). Scenarios with unknown dates will be added for each field with unknown date parts.

Note

Creating scenarios with unknown dates will increase the time taken to create the Test Cases.

Create additional scenarios?

Selecting this option will create:

• three scenarios for IsLessThan, IsLessThanOrEqualTo, IsEqualTo, IsGreaterThan, IsGreaterThanOrEqualTo steps. One value will be immediately below the comparison, one equal and one greater than.

• a positive and negative scenario for each AND and OR step, if the check logic allows these scenarios to be generated.

• a positive and negative scenario for each IsEmpty/IsNotEmpty step, if the check logic allows these scenarios to be generated.

Note

Creating additional scenarios will increase the time taken to create the Test Cases.

Note

Additional scenarios will not be created if there are issues with the initial scenarios preventing complete scenarios from being generated, such as a Custom Function in the Check Steps.

Create additional log records?

The Test Case advisor will create 2 scenarios if you select this option, if there are wildcarded log Fields in the Edit Check. If there are no log Fields, or a specific record position is used in the Edit Check, then this option will have no effect.

For example, for an Edit Check comparing 2 dates, 'AEENDAT < AESTDAT':

  Scenario: Check Actions are run
    When I enter and save data:
      | DataPoint        | Value       |
      | AE.AE.AEENDAT[1] | 26 SEP 2019 |
      | AE.AE.AESTDAT[1] | 27 SEP 2019 |

Then I should see the following query on "AE.AE.AESTDAT[1]"
  """
  Start date is after end date
  """

  Scenario: Check Actions are run (record position 2)
    When I enter and save data:
      | DataPoint        | Value       |
      | AE.AE.AEENDAT[2] | 26 SEP 2019 |
      | AE.AE.AESTDAT[2] | 27 SEP 2019 |

And I should see the following query on "AE.AE.AESTDAT[2]"
  """
  Start date is after end date
  """

Test Case Advisor Results

The Test Case Advisor uses the enter and save data entry step because that step simulates most closely a user entering data into Rave, making sure that custom functions and derivations on the form are triggered.

Test Cases created by the Test Case Advisor will be in one of these states:

Ready to run

For most Edit Checks the Test Case Advisor can suggest a valid Test Case containing two Scenarios; one which should cause the Check Actions to run and one which should not trigger the Check Actions. These Test Case should be runnable without requiring any modifications but we do recommend that all suggested Test Cases are reviewed to ensure the Edit Check is being tested optimally.

Informational notes will be added to the Scenario descriptions if the Test Case Advisor has:

• detected that a Field has a Default Value and so does not need to have a Data Entry step

• inserted data entry steps to make a Field visible. If the Edit Check references a Field which is not visible by default then the Test Case Advisor will look for an Edit Check with a SetDataPointVisible action for that Field and will then add the steps necessary to make the Field visible.

• inserted data entry steps to derive a Field value. If the Edit Check references a Field which is the target of a Derivation then the Test Case Advisor will analyze the Derivation and insert data entry steps to derive an appropriate value for the Field. Sometimes the Test Case Advisor will not be able to calculate values for the derived Field which will also trigger the Edit Check in which case an issue will be added and the Test Case will be marked as invalid.

The following example shows how a data entry step has been added for the Date of Birth and Date of Informed Consent fields which are used in Derivation "DER_AGE" to calculate the age. The Age Field is used in Edit Check "AGE_RANGE".

@EditCheck:AGE_RANGE
@Derivation:DER_AGE
@Form:DM
Feature: AGE_RANGE

  Background:
    Given I am logged in with role "Investigator"
    And a subject exists

  Scenario: Check Actions are run
    Note: Field "AGE" on Form "DM" is derived by Derivation "DER_AGE"
    When I enter and save data:
      | DataPoint              | Value       |
      | SCREENING.SUB.ICDAT[0] | 01 JAN 2019 |
    And I enter and save data:
      | DataPoint            | Value       |
      | SCREENING.DM.BRTHDAT | 01 JAN 1979 |
    Then I should see the following query on field "SCREENING.DM.AGE"
    """
    Subject age does not meet inclusion criteria
    """

Review recommended

For a small number of Edit Checks a valid Test Case can be suggested but the Test Case Advisor has made some assumptions which should be reviewed.

The assumptions are added as comments in the suggested Test Case. The assumptions include:

• If the Edit Check references a Variable, with no specific Field specified in the Edit Check then the Test Case Advisor must find an appropriate Field on which to do data entry. If there is only one Field associated with that Variable then the selection is obvious but if there are multiple Fields the Test Case Advisor will order them by Form Ordinal and Field Ordinal and select the first one (ie. the Form with the lowest Ordinal number and the Field on that Form which is associated with the Variable).

• If the Edit Check has one or more wildcarded Folders then the Test Case Advisor must find a suitable Folder for that Field and Form, using this strategy. Note that the same Folder must be selected for all wildcarded references because the Rave edit check engine will insert one Folder into all Folder wilcards when processing the edit check.

  • Are all of the Forms for wildcarded Folders present in the same Folder in the default Matrix?

  • Is it the Primary Form (in which case it will be in the SUBJECT Folder)?

  • Are all of the Forms for wildcarded Folders present in the same Folder in any Matrix, except an 'All Forms' matrix?

  • Is there an AddForm Check Action on another Edit Check which will help us find the appropriate Folder?

  • Is a specific Folder referenced elsewhere in the Edit Check which we can use?

  • If no Folder has been found in the above steps then take the first Folder, ie. the Folder with the lowest Ordinal. In this case a comment will be added to the Test Case.

The following example shows comments added for variable and folder selection:

@EditCheck:AGE_RANGE
@Derivation:DER_AGE
@Form:DM

# Using Field ICDAT on Form SUB for Variable CDAT
# Wildcard Folder replaced with SCREENING

Feature: AGE_RANGE

  Background:
    Given I am logged in with role "Investigator"
    And a subject exists

  Scenario: Check Actions are run
    Note: Field "AGE" on Form "DM" is derived by Derivation "DER_AGE"
    When I enter and save data:
      | DataPoint              | Value       |
      | SCREENING.SUB.ICDAT[0] | 01 JAN 2019 |
    And I enter and save data:
      | DataPoint            | Value       |
      | SCREENING.DM.BRTHDAT | 01 JAN 1979 |
    Then I should see the following query on field "SCREENING.DM.AGE"
    """
    Subject age does not meet inclusion criteria
    """

Requires attention

Some Edit Checks, mainly those using Custom Functions, cannot be created in a complete and valid state because the Test Case Advisor is unable to work out how the Edit Check should be triggered and/or what actions should be taken. For these Edit Checks the Test Case Advisor will create as much of the Test Case as it can and will list the issues which need to be addressed in a table at the start of each scenario. The list of issues should be reviewed and the appropriate steps added to the Test Case, after which the issues should be removed from the Test Case.

• The Test Case Advisor is unable to analyse the source code of Custom Functions and Dynamic Search Lists and so test steps must be added to take the appropriate actions for them. An 'always true' Custom Function, eg. one that simply does 'return true' does not require additional work in the Test Case.

• The Test Case Advisor might not be able to find a solution for some Edit Checks, either because there are no possible values which would trigger the Check Actions (for example, a Dictionary Field being compared against a static value which does not exist in the Dictionary) or in rare cases because the Test Case Advisor has not been able to find suitable values.

• A Field in a Data Entry step is View or Entry Restricted, at Field or Form level, to the default role. The Test Case Advisor will insert an additional log in step before the restricted Field which will need to be edited to select an appropriate EDC Role. Note that Fields which have a default value will not appear in the data entry step because Rave will automatically enter the default value, and so they will not require this additional log in step. Similarly Fields which are Derivation targets will not appear directly in data entry steps and do not require additional log in steps if they are view or entry restricted.

• Edit Checks using Logical Record Position will need additional work to add extra values in the appropriate logical sequence.

• The Test Case Advisor will not be able to find suitable values for all Derivations in which case this will be noted as an issue.

• If a Field is not visible by default and no Check Action could be found to make it visible then steps must be inserted to make it visible (probably by triggering a Custom Function in another Edit Check).

• If a Field and/or Form is inactive then the Edit Check and Test Case must be reviewed.

• File Upload and Signature controls are not supported by TrialGrid Test Cases. If an Edit Check references such a Field it will need to be manually tested.

• Some Check Actions are not supported by TrialGrid Test Cases and will need to be manually tested.

See also

For more information about the limitations of Test Cases see Automated Testing Limitations

The following example shows a Test Case where the Edit Check references a derived Field and the Test Case Advisor could not find suitable values. The data entry steps are blank and should be manually completed. The Edit Check has a Custom Function action which must also be changed to the appropriate steps:

After review the Test Case might look like this:

Labelling results

When running the Test Case Advisor for a batch of Edit Checks the resulting Test Cases can be labelled to make searching the results easier.

Object references

The Test Case Advisor will insert a cross-reference to the Edit Check, to the Form(s) referenced in Check Steps and Actions, to any Custom Functions used in the Edit Check and to any Derivations needed to populate derived Fields.

Links to Test Cases

When the Test Case Advisor has finished processing all selected Edit Checks, the result lines for each newly created Test Case will link to the Test Case. Clicking on the link will open the Test Case. The links will not be active until the Test Case Advisor has completed processing.