1#openbmc-automation
2
3Quickstart
4----------
5
6To run openbmc-automation first you need to install the prerequisite python
7packages which will help to invoke tests through tox.  Note that tox
8version 2.3.1 or greater is required.
9
10Install the python dependencies for tox
11```shell
12    $ easy_install tox
13    $ easy_install pip
14```
15
16Initilize the following environment variable which will used while testing
17```shell
18    $ export OPENBMC_HOST=<openbmc machine ip address>
19    $ export OPENBMC_PASSWORD=<openbmc password>
20    $ export OPENBMC_USERNAME=<openbmc username>
21    $ export OPENBMC_MODEL=[./data/Barreleye.py, ./data/Palmetto.py, etc]
22    $ export IPMI_COMMAND=<Dbus/External>
23    $ export IPMI_PASSWORD=<External IPMI password>
24```
25
26There are two different set of test suite existing based on the usage.
27The test suites are distinctly separated by directory as under
28    tests/
29    extended/
30
31`tests`: directory contains the general test cases
32
33`extended`: directory contains the use cases for new IP network testing,PDU,
34BIOS and BMC code update.
35
36```shell
37Use Following Variables for networking test cases
38===========================================================
39    $export NEW_BMC_IP=<openbmc machine ip address>
40    $export NEW_SUBNET_MASK=<openbmc new subnet mask>
41    $export NEW_GATEWAY=<openbmc new gateway>
42==========================================================
43
44    Use following parameters for PDU:
45    $ export PDU_IP=<PDU IP address>
46    $ export PDU_USERNAME=<PDU username>
47    $ export PDU_PASSWORD=<PDU password>
48    $ export PDU_TYPE=<PDU type>
49    $ export PDU_SLOT_NO=<SLOT number>
50
51    for PDU_TYPE we support only synaccess at the moment
52
53Use following variables for syslog test cases
54==========================================================
55    $ export SYSLOG_IP_ADDRESS=<remote syslog system ip>
56    $ export SYSLOG_PORT=<remote syslog system port>
57
58Use the following variables for Qemu test run
59==========================================================
60    $ export SSH_PORT=<ssh port number>
61    $ export HTTPS_PORT=<https port number>
62
63Use the following variables for BIOS update testing
64==========================================================
65    $ export PNOR_IMAGE_PATH=<path to>/<machine>.pnor
66
67```
68
69Run tests
70```shell
71    $ tox -e tests
72```
73
74How to test individual test
75```shell
76    One specific test
77    $ tox -e custom -- -t '"DIMM0 no fault"' tests/test_sensors.robot
78
79    No preset environment variables, one test case from one test suite
80    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- -t '"DIMM0 no fault"' tests/test_sensors.robot
81
82    No preset environment variables, one test suite  for a palmetto system
83    $ OPENBMC_HOST=x.x.x.x tox -e palmetto -- tests/test_sensors.robot
84
85    No preset environment variables, the entire test suite for a barreleye system
86    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- tests
87
88    No preset environment variables, the entire test suite excluding test
89    cases using argument file.
90    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/skip_test tests
91```
92
93It can also be run by pasing variables from the cli...
94```shell
95    Run one test suite using using pybot
96    $  pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v OPENBMC_MODEL:<model path> tests/test_time.robot
97
98    Run entire test suite using using pybot
99    $  pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v OPENBMC_MODEL:<model path> tests
100
101    Run entire test suite using external ipmitool
102    $  pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v IPMI_COMMAND:External -v IPMI_PASSWORD:PASSW0RD -v OPENBMC_MODEL:<model path> tests
103```
104
105Run extended tests
106```shell
107    Set the preset environment variables, run test suite for a barreleye system
108    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- extended/test_power_restore.robot
109
110    Similarly for Network, PDU and update BIOS
111
112    For BMC code update, download the system type *.all.tar image from https://openpower.xyz
113    and run as follows:
114
115    For Barreleye system
116    python -m robot -v OPENBMC_HOST:x.x.x.x -v FILE_PATH:downloaded_path/barreleye-xxxx.all.tar  extended/code_update/update_bmc.robot
117
118    For loop test (Default iteration is 10)
119    python -m robot -v OPENBMC_HOST:x.x.x.x -v OPENBMC_SYSTEMMODEL:xxxxxx -v ITERATION:n -v LOOP_TEST_COMMAND:xxxxxx extended/full_suite_regression.robot
120    Below is sample command using tox to test only fw version using Barreleye system for 5 times
121    OPENBMC_HOST=x.x.x.x  LOOP_TEST_COMMAND="--argumentfile test_lists/skip_test tests/test_fw_version.robot" ITERATION=5  OPENBMC_SYSTEMMODEL=barreleye tox -e barreleye -- ./extended/full_suite_regression.robot
122```
123
124Jenkins jobs tox commands
125```shell
126    HW CI tox command
127    Set the preset environment variables, run HW CI test for a barreleye system
128    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/HW_CI tests
129
130```
131