1tdc - Adding test cases for tdc
2
3Author: Lucas Bates - lucasb@mojatatu.com
4
5ADDING TEST CASES
6-----------------
7
8User-defined tests should be added by defining a separate JSON file.  This
9will help prevent conflicts when updating the repository. Refer to
10template.json for the required JSON format for test cases.
11
12Include the 'id' field, but do not assign a value. Running tdc with the -i
13option will generate a unique ID for that test case.
14
15tdc will recursively search the 'tc' subdirectory for .json files.  Any
16test case files you create in these directories will automatically be included.
17If you wish to store your custom test cases elsewhere, be sure to run tdc
18with the -f argument and the path to your file.
19
20Be aware of required escape characters in the JSON data - particularly when
21defining the match pattern. Refer to the tctests.json file for examples when
22in doubt.
23
24
25TEST CASE STRUCTURE
26-------------------
27
28Each test case has required data:
29
30id:           A unique alphanumeric value to identify a particular test case
31name:         Descriptive name that explains the command under test
32category:     A list of single-word descriptions covering what the command
33              under test is testing. Example: filter, actions, u32, gact, etc.
34setup:        The list of commands required to ensure the command under test
35              succeeds. For example: if testing a filter, the command to create
36              the qdisc would appear here.
37cmdUnderTest: The tc command being tested itself.
38expExitCode:  The code returned by the command under test upon its termination.
39              tdc will compare this value against the actual returned value.
40verifyCmd:    The tc command to be run to verify successful execution.
41              For example: if the command under test creates a gact action,
42              verifyCmd should be "$TC actions show action gact"
43matchPattern: A regular expression to be applied against the output of the
44              verifyCmd to prove the command under test succeeded. This pattern
45              should be as specific as possible so that a false positive is not
46              matched.
47matchCount:   How many times the regex in matchPattern should match. A value
48              of 0 is acceptable.
49teardown:     The list of commands to clean up after the test is completed.
50              The environment should be returned to the same state as when
51              this test was started: qdiscs deleted, actions flushed, etc.
52
53
54SETUP/TEARDOWN ERRORS
55---------------------
56
57If an error is detected during the setup/teardown process, execution of the
58tests will immediately stop with an error message and the namespace in which
59the tests are run will be destroyed. This is to prevent inaccurate results
60in the test cases.
61
62Repeated failures of the setup/teardown may indicate a problem with the test
63case, or possibly even a bug in one of the commands that are not being tested.
64
65It's possible to include acceptable exit codes with the setup/teardown command
66so that it doesn't halt the script for an error that doesn't matter. Turn the
67individual command into a list, with the command being first, followed by all
68acceptable exit codes for the command.
69
70