Test Cases

A Test Case represents a Script in the manual testing process: A list of steps which describe actions to perform and the expected results.

See also

For a full list of all the commands which Automated Testing understands see Automated Testing Step Reference

In TrialGrid Test Cases are written in a text form which is designed to be easy to read and write for TrialGrid users but which can also be executed by TrialGrid in an automated way. Tests have a fixed format which they must follow.

This example tests an Edit Check called "If CM not collected reason must be present"

 1 # Created by IJS on 2 JAN 2019
 2 @EditCheck:If Con Meds not collected reason must be present
 3 @Form:CM
 4 Feature: If Con Meds not collected reason must be present
 5
 6   This Test verifies that the CM Reason Not Done field is shown or hidden appropriately.
 7
 8   Background:
 9     Given I am logged in with role "Batch Upload"
10     And a subject exists
11     And form "CM" exists in folder "C1WK1"
12
13   Scenario: Check Actions are run
14     When I enter data:
15       | DataPoint          | Value |
16       | C1WK1.CM.CMSTAT[0] | No    |
17     Then I should see field "C1WK1.CM.CMREASND[0]"
18
19   Scenario: Check Actions are not run
20     When I enter data:
21       | DataPoint          | Value |
22       | C1WK1.CM.CMSTAT[0] | Yes   |
23     Then I should not see field "C1WK1.CM.CMREASND[0]"

The Test Case can also be viewed (not edited) with object names rather than OIDs by changing the 'Edit' toggle button to 'View'. The example would then be:

 1 @EditCheck:If Con Meds not collected reason must be present
 2 @Form:CM
 3 Feature: If CM not collected reason must be present
 4
 5   This Test verifies that the CM Reason Not Done field is shown or hidden appropriately.
 6
 7   Background:
 8
 9     Given I am logged in with role "Batch Upload"
10     And a subject exists
11     And form "Concomitant Medications" exists in folder "Cycle 1 Week 1"
12
13   Scenario: Check Actions are run
14
15     When I enter data:
16       | Folder         | Form                    | Field      | Value |
17       | Cycle 1 Week 1 | Concomitant Medications | Collected? | No    |
18
19     Then I should see field "Reason not collected" in Form "Concomitant Medications" in Folder "Cycle 1 Week 1"
20
21   Scenario: Check Actions are not run
22
23     When I enter data:
24       | Folder         | Form                    | Field      | Value |
25       | Cycle 1 Week 1 | Concomitant Medications | Collected? | Yes   |
26
27     Then I should not see field "Reason not collected" in Form "Concomitant Medications" in Folder "Cycle 1 Week 1"

Note

Comments are not displayed in Test Case view mode and there might be differences in the number of empty lines displayed.

Note

The maximum length of a Test Case is 1 million characters. If a Test Case contains template features such as 'for' loops the maximum length is 1 million characters and 20000 lines after processing any template control structures.

Comments (optional)

At Line 1 we see a comment. Comments start with a # and the rest of the line is ignored by the system. This makes it very useful for version declarations, author comments etc.

Note

if you wish to comment out a Test Case Template structures such as a 'for' loops, there is a slightly different syntax, see _test_case_templates.

Referencing Edit Checks and other Draft Objects (optional)

Line 2 and Line 3 are tags which show a link to other Draft objects. Using tags in this way creates a link between the object and this test which can be viewed in the editor sidebar for that object and which is also shown in object listings.

Linking to an Edit Check with the @EditCheck tag allows the system to calculate coverage metrics for which Edit Checks are covered by tests. These metrics can be shown on the Draft home page (see settings for Draft). The @EditCheck tag also gives the TrialGrid Test Case editor a hint that this Edit Check is being tested which allows it to check that any tests for OpenQuery actions reference the exact Query Text defined in the Edit Check Action.

Valid object reference tags are:

  • @CustomFunction:<Function Name>

  • @Derivation:<Derivation Name>

  • @EditCheck:<Edit Check Name>

  • @Form:<Form OID>

Tip

Click on the icon next to the line number in the editor to navigate direct to the referenced Object.

Note

It is good practice to make Test Cases specific to one goal - testing a single Edit Check for example. Structuring tests this way means that if a Test Case fails because it does not see an expected result the problem can be addressed and the test re-run. If a single Test Case tests 10 Edit Checks and the test fails on the 9th check then a lot of time will be wasted in the re-running.

Declaring the Feature (required)

Line 4 starts with "Feature:" and then a short name for the feature. The name is not important but the "Feature:" tells the system to start looking for test instructions and is a required part of the setup.

Line 6 is optional. It is a description of what the test is trying to achieve. You may also use comments for this purpose at any point in the test.

Declaring Background (optional)

Line 8 starts the definition of a set of instructions which will be repeated at the start of every test Scenario that follows. You do not have to have a Background section but it can make tests more readable. It is also possible to put the background steps into each Scenario but this makes the tests longer and less readable:

Feature: If CM not collected reason must be present

  This Test verifies that the CM Reason Not Done field is shown or hidden appropriately.

  Scenario: Check Actions are run
    Given I am logged in with role "Batch Upload"
    And a subject exists
    And form "CM" exists in folder "SUBJECT"
    When I enter data:
      | DataPoint            | Value |
      | SUBJECT.CM.CMSTAT[0] | No    |
    Then I should see field "SUBJECT.CM.CMREASND[0]"

  Scenario: Check Actions are not run
    Given I am logged in with role "Batch Upload"
    And a subject exists
    And form "CM" exists in folder "SUBJECT"
    When I enter data:
      | DataPoint            | Value |
      | SUBJECT.CM.CMSTAT[0] | Yes   |
    Then I should not see field "SUBJECT.CM.CMREASND[0]"

Declaring Test Steps (required)

Lines 9-11 are our first executable steps. These is an instruction to perform some actions:

10  Given I am logged in with role "Batch Upload"
11  And a subject exists
12  And form "CM" exists in folder "SUBJECT"

Line 9 is an instruction for the system to log into Rave using credentials (username/password) for the Batch Upload role which is provided when the tests are run. See Running Tests

Line 10 is an instruction to create a subject. A subject will be created in Rave using the Test Case name and the date/time of the run to make it easier to identify which subject is related to which test and when it was run.

Line 11 is an instruction to create Form "CM" in folder "SUBJECT" if it does not already exist. This may not be necessary if the Default Matrix adds the Form or if the test already contains steps which trigger the creation of the Form.

Test Step Instructions

Instructions start with one of the keywords "Given", "When", "Then", "And", "But" or "*". Each keyword has a set of associated instructions which it understands e.g. 'Given I am logged in with role "Batch Upload"' is valid but 'Then I am logged in with role "Batch Upload"' would not be recognized - see the full list of commands.

  • Given - Used to set up the environment for the test or to get the system into a state where tests can be performed. For example, to log into Rave, to create or select a Subject, to create Forms or add Folders etc

  • When - Used to initiate an action which is part of the test. For example, "When I enter data". This is the "doing" part of the test.

  • Then - Used to check that an action has occurred. Often has the form "Then I should see..." This is the "checking" part of the test.

The additional keywords "And", "But" and "*" can be used in place of any of the Given/When/Then keywords and help to make tests more readable. The following are actually all the same instructions:

# Repetitive and not easy to read
Given I am logged in with role "Batch Upload"
Given a subject exists
Given form "CM" exists in folder "SUBJECT"

# Easier to read
Given I am logged in with role "Batch Upload"
And a subject exists
And form "CM" exists in folder "SUBJECT"

# Odd but valid
Given I am logged in with role "Batch Upload"
But a subject exists
But form "CM" exists in folder "SUBJECT"

# Minimalist
* I am logged in with role "Batch Upload"
* a subject exists
* form "CM" exists in folder "SUBJECT"

The goal is to make the tests as readable as possible for humans so choose a style which works best for your organization.

Declaring Scenarios (required)

A Scenario represents a set of steps to test one thing and is a single session working with Medidata Rave. In order for a Scenario to test anything it will require some setup steps (Given..) such as logging into Rave, some action steps (When...) and then some steps which validate the results of those actions (Then...).

Note

When testing an Edit Check it is good practice to create one scenario for the positive test (the check fires) and one for the negative test (the check does not fire) because Scenarios pass or fail as a single unit. If you combine the steps for postive a negative tests in a single Scenario you won't know immediately if it failed to fire or if it fired when you didn't expect it to.

Our first scenario starts at line 14:

14   Scenario: Check Actions are run
15     When I enter data:
16       | DataPoint            | Value |
17       | SUBJECT.CM.CMSTAT[0] | No    |
18     Then I should see field "SUBJECT.CM.CMREASND[0]"

The Scenario declares what it is that it is trying to test in this case "Check Actions are run" - a postive test. Line 15 is our first action in the Scenario but before this action is taken any Background steps are first run so for this Scenario the system actually runs:

14   Scenario: Check Actions are run
15     # Background steps run first
16     Given I am logged in with role "Batch Upload"
17     And a subject exists
18     And form "CM" exists in folder "SUBJECT"
19
20     # Steps for this Scenario
21     When I enter data:
22       | DataPoint            | Value |
23       | SUBJECT.CM.CMSTAT[0] | No    |
24     Then I should see field "SUBJECT.CM.CMREASND[0]"

We already discussed the background steps but the "When I enter data:" step at line 21 above is an instruction to enter data into Rave using the table of data defined on lines 22 and 23.

21  When I enter data:
22    | DataPoint            | Value |
23    | SUBJECT.CM.CMSTAT[0] | No    |

The first column defines the DataPoint reference. This has the form:

FOLDER_OID . FORM_OID . FIELD_OID [ RECORDPOSITION ]

If you need to set a Folder repeat number or a Form repeat number you can also do that:

FOLDER_OID[2] . FORM_OID[1] . FIELD_OID [0]

The second column defines the value to be set for that Datapoint. Notice that in this case the value is "No". This is the User Data (what the user sees) from a Data Dictionary. TrialGrid can validate these values so that you don't enter a value that is not valid. An additional column can be added for Dictionaries with 'Specify' entries:

21  When I enter data:
22    | DataPoint            | Value | Specify       |
23    | SUBJECT.CM.CMSTAT[0] | Other | Not performed |

For a Checkbox use the values 1 or 0 (Yes and No respectively) and when entering numeric values you can also add a Units column which will be checked against the UnitDictionary for the Field.

  When I enter data:
    | DataPoint               | Value | Unit  |
    | SUBJECT.VS.SYSTOLIC[0]  | 120   | mmHg  |
    | SUBJECT.VS.DIASTOLIC[0] | 80    | mmHg  |

You can also enter data into multiple forms in a single "When I enter data" table and TrialGrid will enter the data and take a screenshot of each Form when the test is run.

Test Case Priority

A priority can be set for a Test Case. The priority is a number between 1 and 999999. When multiple Test Case are selected to be run the Test Cases will be run in order of priority, with priority 1 being the first to be run. Test Cases with the same, or no, priority will be run in alphanumeric order.