openbmc/openbmc-test-automation/README.md

5 - DMTF Redfish
6 - Out-of-band IPMI
7 - SSH to BMC and Host OS
8 - [Legacy REST](https://github.com/openbmc/openbmc-test-automation/releases/tag/v4.0-stable)
12 - Power on/off
13 - Reboot Host
14 - Reset BMC
15 - Code update BMC and host
16 - Power management
17 - Fan controller
18 - HTX bootme
19 - XCAT execution
20 - Network
21 - IPMI support (generic and DCMI compliant)
22 - Factory reset
23 - RAS (Reliability, availability and serviceability)
24 - Web UI testing
25 - Secure boot
26 - SNMP (Simple Network Management Protocol)
27 - Remote Logging via Rsyslog
28 - LDAP (Lightweight Directory Access Protocol)
29 - Certificate
30 - Local User Management(Redfish/IPMI)
31 - DateTime
32 - Event Logging
33 - PLDM (Platform Level Data Model) via pldmtool
37 - SOL collection
38 - FFDC collection
39 - Error injection from host
43 - [Robot Framework Install Instruction](https://github.com/robotframework/robotframework/blob/maste…
45 - Miscellaneous Packages required to be installed for OpenBmc Automation.
54     $ pip install -r requirements.txt
60     $ pip install -r requirements_optional.txt
63 you'll find this file once your clone openbmc-test-automation repository.
68 [OpenBMC GUI Test Setup Guide](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/…
73     $ pip install -U tox
76 Installing expect (Ubuntu example):
79     $ sudo apt-get install expect
86 - [MAINTAINERS](OWNERS): OpenBMC test code maintainers information.
87 - [CONTRIBUTING.md](CONTRIBUTING.md): Coding guidelines.
88 - [Code Check Tools](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/code_stand…
90 - [Redfish Coding Guidelines](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/r…
92 - [REST-cheatsheet.md](https://github.com/openbmc/docs/blob/master/REST-cheatsheet.md):
95 - [REDFISH-cheatsheet.md](https://github.com/openbmc/docs/blob/master/REDFISH-cheatsheet.md):
97 - [README.md](https://github.com/openbmc/webui-vue/blob/master/README.md): Web
99 - [Redfish Request Via mTLS](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/re…
101 - [Corporate CLA and Individual CLA](https://github.com/openbmc/docs/blob/master/CONTRIBUTING.md#su…
106 - [OpenBMC Test Architecture](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/o…
108 - [Tools](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/openbmc_test_tools.md…
110 - [Code Update](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/code_update.md):
112 - [Certificate Generate](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/certif…
114 - [Boot Test](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/boot_test.md):
121 - POWER
122 - x86 systems running OpenBMC firmware stack.
128 - Download the openbmc-test-automation repository:
131   $ git clone https://github.com/openbmc/openbmc-test-automation
132   $ cd openbmc-test-automation
135 - Execute basic setup test run:
138   $ robot -v OPENBMC_HOST:xx.xx.xx.xx templates/test_openbmc_setup.robot
145 There are several sub-directories within the openbmc-test-automation base which
146 contain test suites, tools, templates, etc. These sub-directories are classified
151 `redfish/`: Contains the general test cases for OpenBMC stack functional
158 `gui/test/`: Contains test cases for testing web-based interface built on
161 `gui/gui_test/`: Contains test cases for testing web-based user interface built
170 `openpower/`: Contains test cases for an OpenPOWER based system.
186 `redfish/`: Contains test cases for DMTF Redfish-related feature supported on
189 `redfish/extended/`: Contains test cases for combined DMTF Redfish-related
193 `-v REDFISH_SUPPORT_TRANS_STATE:1` to force the test suites to execute in
198 To run openbmc-automation first you need to install the prerequisite Python
204 - Execute all test suites for `tests/`:
207   $ robot -v OPENBMC_HOST:xx.xx.xx.xx  tests
210 - Execute a test suite:
213   $ robot -v OPENBMC_HOST:xx.xx.xx.xx redfish/extended/test_basic_ci.robot
216 - Initialize the following test variables which will be used during test
235   -v OPENBMC_HOST:<openbmc machine IP address/hostname>
236   -v OPENBMC_USERNAME:<openbmc username>
237   -v OPENBMC_PASSWORD:<openbmc password>
240 - Testing in qemu:
252 …$ OPENBMC_HOST=x.x.x.x SSH_PORT=<port number> HTTPS_PORT=<port number> robot -A test_lists/QEMU_CI…
255 - Run tests:
261 - How to run an individual test:
264   $ tox -e default -- --include Power_On_Test  tests/test_basic_poweron.robot
267 - No preset environment variables, default configuration for all supported
271   $ OPENBMC_HOST=x.x.x.x tox -e default -- tests
274 - No preset environment variables, one test case from a test suite:
277   $ OPENBMC_HOST=x.x.x.x tox -e default -- --include Power_On_Test tests/test_basic_poweron.robot
280 - No preset environment variables, the entire test suite:
283   $ OPENBMC_HOST=x.x.x.x tox -e default -- tests
286 - No preset environment variables, the entire test suite excluding test cases
290   $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/skip_test tests
293 - Exclude test list for supported systems:
303   $ robot -v OPENBMC_HOST:xx.xx.xx.xx -A test_lists/skip_test_witherspoon tests/
306 - Run IPMI tests:
308   Running only out-of-band IPMI tests:
311 …$ robot -v IPMI_COMMAND:External -v OPENBMC_HOST:x.x.x.x --argumentfile test_lists/witherspoon/ski…
317 …$ robot -v IPMI_COMMAND:Inband -v OPENBMC_HOST:x.x.x.x -v OS_HOST:x.x.x.x -v OS_USERNAME:xxxx -v O…
320 - Run GUI tests:
326   $ robot -v OPENBMC_HOST:x.x.x.x -v GUI_BROWSER:gc -v GUI_MODE:header gui/test/
332   $ robot -v OPENBMC_HOST:x.x.x.x --argumentfile test_lists/BMC_WEB_CI gui/test/
335 - Run LDAP tests:
343 …ps://searchwindowsserver.techtarget.com/definition/Microsoft-Active-Directory-Domain-Services-AD-DS
350 …-v OPENBMC_HOST:x.x.x.x -v LDAP_SERVER_URI:<ldap(s)//LDAP Hostname / IP> -v LDAP_BIND_DN:<LDAP Bin…
353 - How to run CI and CT bucket test:
358   $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/HW_CI tests
364   $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/CT_basic_run tests
367 - Run extended tests:
369   For-loop test (default iteration is 10):
372 …$ robot -v OPENBMC_HOST:x.x.x.x -v OPENBMC_SYSTEMMODEL:xxxxxx -v ITERATION:n -v LOOP_TEST_COMMAND:…
378 …t_fw_version.robot" ITERATION=5 OPENBMC_SYSTEMMODEL=witherspoon tox -e witherspoon -- ./extended/f…
381 - Host CPU architecture
383   By default openbmc-test-automation framework assumes that host CPU is based on
385   `-v PLATFORM_ARCH_TYPE:x86` variable setting to your CLI commands or set an
394 - HW CI tox command:
397   $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/HW_CI tests