1#openbmc-automation
2
3Quickstart
4----------
5
6To run openbmc-automation first you need to install the prerequisite python
7packages which will help to invoke tests through tox.  Note that tox
8version 2.3.1 or greater is required.
9
10Install the python dependencies for tox
11```shell
12    $ easy_install tox
13    $ easy_install pip
14```
15
16Initialize the following environment variable which will be used during testing
17```shell
18    $ export OPENBMC_HOST=<openbmc machine ip address>
19    $ export OPENBMC_PASSWORD=<openbmc password>
20    $ export OPENBMC_USERNAME=<openbmc username>
21    $ export OPENBMC_MODEL=[./data/Barreleye.py, ./data/Palmetto.py, etc]
22    $ export IPMI_COMMAND=<Dbus/External>
23    $ export IPMI_PASSWORD=<External IPMI password>
24```
25
26There are two different set of test suite existing based on the usage.
27The test suites are distinctly separated by directory as under
28    tests/
29    extended/
30
31`tests`: directory contains the general test cases
32
33`extended`: directory contains the use cases for new IP network testing, PDU,
34BIOS and BMC code update.
35
36```shell
37Use Following Variables for networking test cases
38===========================================================
39    $export NEW_BMC_IP=<openbmc machine ip address>
40    $export NEW_SUBNET_MASK=<openbmc new subnet mask>
41    $export NEW_GATEWAY=<openbmc new gateway>
42==========================================================
43
44    Use following parameters for PDU:
45    $ export PDU_IP=<PDU IP address>
46    $ export PDU_USERNAME=<PDU username>
47    $ export PDU_PASSWORD=<PDU password>
48    $ export PDU_TYPE=<PDU type>
49    $ export PDU_SLOT_NO=<SLOT number>
50
51    for PDU_TYPE we support only synaccess at the moment
52
53Use following variables for syslog test cases
54==========================================================
55    $ export SYSLOG_IP_ADDRESS=<remote syslog system ip>
56    $ export SYSLOG_PORT=<remote syslog system port>
57
58Use the following variables for Qemu test run
59==========================================================
60    $ export SSH_PORT=<ssh port number>
61    $ export HTTPS_PORT=<https port number>
62
63Use the following variables for BIOS update testing
64==========================================================
65    $ export PNOR_IMAGE_PATH=<path to>/<machine>.pnor
66
67```
68
69Run tests
70```shell
71    $ tox -e tests
72```
73
74How to test individual test
75```shell
76    One specific test
77    $ tox -e custom -- -t '"DIMM0 no fault"' tests/test_sensors.robot
78
79    No preset environment variables, one test case from one test suite
80    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- -t '"DIMM0 no fault"' tests/test_sensors.robot
81
82    No preset environment variables, one test suite  for a palmetto system
83    $ OPENBMC_HOST=x.x.x.x tox -e palmetto -- tests/test_sensors.robot
84
85    No preset environment variables, the entire test suite for a barreleye system
86    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- tests
87
88    Default CI test bucket list:
89    No preset environment variables, the entire test suite excluding test
90    cases using argument file.
91    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/skip_test tests
92
93    Exclude test list for supported systems:
94    Barrleye:  test_lists/skip_test_barreleye
95    Palmetto:  test_lists/skip_test_palmetto
96    Witherspoon:  test_lists/skip_test_witherspoon
97```
98
99It can also be run by passing variables from the cli...
100```shell
101    Run one test suite using using pybot
102    $  pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v OPENBMC_MODEL:<model path> tests/test_time.robot
103
104    Run entire test suite using using pybot
105    $  pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v OPENBMC_MODEL:<model path> tests
106
107    Run entire test suite using external ipmitool
108    $  pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v IPMI_COMMAND:External -v IPMI_PASSWORD:PASSW0RD -v OPENBMC_MODEL:<model path> tests
109```
110
111Run extended tests
112```shell
113    Set the preset environment variables, run test suite for a barreleye system
114    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- extended/test_power_restore.robot
115
116    Similarly for Network, PDU and update BIOS
117
118    For BMC code update, download the system type *.all.tar image from https://openpower.xyz
119    and run as follows:
120
121    For Barreleye system
122    python -m robot -v OPENBMC_HOST:x.x.x.x -v FILE_PATH:downloaded_path/barreleye-xxxx.all.tar  extended/code_update/update_bmc.robot
123
124    For loop test (Default iteration is 10)
125    python -m robot -v OPENBMC_HOST:x.x.x.x -v OPENBMC_SYSTEMMODEL:xxxxxx -v ITERATION:n -v LOOP_TEST_COMMAND:xxxxxx extended/full_suite_regression.robot
126    Below is sample command using tox to test only fw version using Barreleye system for 5 times
127    OPENBMC_HOST=x.x.x.x  LOOP_TEST_COMMAND="--argumentfile test_lists/skip_test tests/test_fw_version.robot" ITERATION=5  OPENBMC_SYSTEMMODEL=barreleye tox -e barreleye -- ./extended/full_suite_regression.robot
128```
129
130Jenkins jobs tox commands
131```shell
132    HW CI tox command
133    Set the preset environment variables, run HW CI test for a barreleye system
134    $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/HW_CI tests
135
136```
137
138Template to be used to create new stand-alone python programs.
139```shell
140
141If you wish to create a new python stand-alone program, please copy bin/python_pgm_template to your new name and then begin your work.  Example:
142
143cd bin
144cp python_pgm_template my_new_program
145
146This template has much of your preliminary work done for you and it will help us all follow a similar structure.
147
148Features:
149- Help text and argparsing started for you.
150- Support for "stock" parameters like quiet, debug, test_mode.
151- exit_function and signal_handler defined.
152- validate_parms function pre-created.
153- main function follows conventional startup:
154
155    if not gen_get_options(parser, stock_list):
156        return False
157
158    if not validate_parms():
159        return False
160
161    qprint_pgm_header()
162
163    # Your code here.
164
165```
166
167Command to get GitHub issues (any GitHub repository) report in CSV format
168Note: On Prompt "Enter your GitHub Password:" enter your GitHub password.
169```shell
170python ./tools/github_issues_to_csv <github user> <github repo>
171
172Example for getting openbmc issues
173python ./tools/github_issues_to_csv <github user>  openbmc/openbmc
174
175Example for getting openbmc-test-automation issues
176python ./tools/github_issues_to_csv <github user>  openbmc/openbmc-test-automation
177```
178