112e33526SPatrick Williams## Features of OpenBMC Test Automation 224c9f88dSGeorge Keishing 324c9f88dSGeorge Keishing**Interface Feature List** 412e33526SPatrick Williams 512e33526SPatrick Williams- DMTF Redfish 612e33526SPatrick Williams- Out-of-band IPMI 712e33526SPatrick Williams- SSH to BMC and Host OS 8*df972ffeSGeorge Keishing- [Legacy REST](https://github.com/openbmc/openbmc-test-automation/releases/tag/v4.0-stable) 924c9f88dSGeorge Keishing 1024c9f88dSGeorge Keishing**Key Feature List** 1112e33526SPatrick Williams 1212e33526SPatrick Williams- Power on/off 1312e33526SPatrick Williams- Reboot Host 1412e33526SPatrick Williams- Reset BMC 1512e33526SPatrick Williams- Code update BMC and host 1612e33526SPatrick Williams- Power management 1712e33526SPatrick Williams- Fan controller 1812e33526SPatrick Williams- HTX bootme 1912e33526SPatrick Williams- XCAT execution 2012e33526SPatrick Williams- Network 2112e33526SPatrick Williams- IPMI support (generic and DCMI compliant) 2212e33526SPatrick Williams- Factory reset 2312e33526SPatrick Williams- RAS (Reliability, availability and serviceability) 2412e33526SPatrick Williams- Web UI testing 2512e33526SPatrick Williams- Secure boot 2612e33526SPatrick Williams- SNMP (Simple Network Management Protocol) 2712e33526SPatrick Williams- Remote Logging via Rsyslog 2812e33526SPatrick Williams- LDAP (Lightweight Directory Access Protocol) 2912e33526SPatrick Williams- Certificate 3012e33526SPatrick Williams- Local User Management(Redfish/IPMI) 3112e33526SPatrick Williams- DateTime 3212e33526SPatrick Williams- Event Logging 3312e33526SPatrick Williams- PLDM (Platform Level Data Model) via pldmtool 3424c9f88dSGeorge Keishing 3524c9f88dSGeorge Keishing**Debugging Supported List** 3624c9f88dSGeorge Keishing 3712e33526SPatrick Williams- SOL collection 3812e33526SPatrick Williams- FFDC collection 3912e33526SPatrick Williams- Error injection from host 40b29d2e84SChris Austen 4112e33526SPatrick Williams## Installation Setup Guide 4212e33526SPatrick Williams 4312e33526SPatrick Williams- [Robot Framework Install Instruction](https://github.com/robotframework/robotframework/blob/master/INSTALL.rst) 4412e33526SPatrick Williams 4512e33526SPatrick Williams- Miscellaneous Packages required to be installed for OpenBmc Automation. 46a5c6e9d6SGeorge Keishing Install the packages and it's dependencies via `pip` 47a5c6e9d6SGeorge Keishing 4812e33526SPatrick WilliamsIf using Python 3.x, use the corresponding `pip3` to install packages. Note: 4912e33526SPatrick WilliamsOlder Python 2.x is not actively supported. 5012bbe002SGeorge Keishing 51a5c6e9d6SGeorge KeishingREST base packages: 5212e33526SPatrick Williams 53a5c6e9d6SGeorge Keishing``` 54a5c6e9d6SGeorge Keishing $ pip install -U requests 55a5c6e9d6SGeorge Keishing $ pip install -U robotframework-requests 561da95695SGeorge Keishing $ pip install -U robotframework-httplibrary 57a5c6e9d6SGeorge Keishing``` 58aba958dcSGeorge Keishing 5912e33526SPatrick WilliamsPython redfish library packages: For more detailed intstructions see 6012e33526SPatrick Williams[python-redfish-library](https://github.com/DMTF/python-redfish-library) 6112e33526SPatrick Williams 624e64eaddSGeorge Keishing``` 634e64eaddSGeorge Keishing $ pip install redfish 644e64eaddSGeorge Keishing``` 654e64eaddSGeorge Keishing 6612e33526SPatrick WilliamsSSH and SCP base packages: For more detailed installation instructions see 6712e33526SPatrick Williams[robotframework-sshlibrary](https://pypi.python.org/pypi/robotframework-sshlibrary) 6812e33526SPatrick Williams 69a5c6e9d6SGeorge Keishing``` 70a5c6e9d6SGeorge Keishing $ pip install robotframework-sshlibrary 71a5c6e9d6SGeorge Keishing $ pip install robotframework-scplibrary 72a5c6e9d6SGeorge Keishing``` 732de157e4SGeorge Keishing 7412bbe002SGeorge KeishingInstalling requirement dependencies: 7512e33526SPatrick Williams 7612bbe002SGeorge Keishing``` 7712bbe002SGeorge Keishing $ pip install -r requirements.txt 7812bbe002SGeorge Keishing``` 7912e33526SPatrick Williams 8012bbe002SGeorge Keishingyou'll find this file once your clone openbmc-test-automation repository. 8112bbe002SGeorge Keishing 828b1024d5SGeorge KeishingFor Web UI ( GUI ) Testing setup instructions see 838b1024d5SGeorge Keishing[OpenBMC GUI Test Setup Guide](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/gui_setup_reference.md) 848b1024d5SGeorge Keishing 85aba958dcSGeorge KeishingInstalling tox: 8612e33526SPatrick Williams 87aba958dcSGeorge Keishing``` 88aba958dcSGeorge Keishing $ pip install -U tox 89aba958dcSGeorge Keishing``` 90aba958dcSGeorge Keishing 91361200b4SMatt FischerInstalling expect (Ubuntu example): 9212e33526SPatrick Williams 9377ac2bd9SPeter D Phan``` 94361200b4SMatt Fischer $ sudo apt-get install expect 9577ac2bd9SPeter D Phan``` 9677ac2bd9SPeter D Phan 9712e33526SPatrick Williams## OpenBMC Test Development 982de157e4SGeorge Keishing 992de157e4SGeorge KeishingThese documents contain details on developing OpenBMC test code and debugging. 1002de157e4SGeorge Keishing 101448e0669SGeorge Keishing- [MAINTAINERS](OWNERS): OpenBMC test code maintainers information. 1022de157e4SGeorge Keishing- [CONTRIBUTING.md](CONTRIBUTING.md): Coding guidelines. 10312e33526SPatrick Williams- [Code Check Tools](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/code_standards_check.md): 10412e33526SPatrick Williams To check common code misspellings, syntax and standard checks. 10512e33526SPatrick Williams- [REST-cheatsheet.md](https://github.com/openbmc/docs/blob/master/REST-cheatsheet.md): 10612e33526SPatrick Williams Quick reference for some common curl commands required for legacy REST 10712e33526SPatrick Williams testing. 10812e33526SPatrick Williams- [REDFISH-cheatsheet.md](https://github.com/openbmc/docs/blob/master/REDFISH-cheatsheet.md): 10912e33526SPatrick Williams Quick reference for some common curl commands required for redfish testing. 110b82c848fSGunnar Mills- [README.md](https://github.com/openbmc/webui-vue/blob/master/README.md): Web 111b82c848fSGunnar Mills UI setup reference. 11212e33526SPatrick Williams- [Corporate CLA and Individual CLA](https://github.com/openbmc/docs/blob/master/CONTRIBUTING.md#submitting-changes-via-gerrit-server): 11312e33526SPatrick Williams Submitting changes via Gerrit server 1142de157e4SGeorge Keishing 11512e33526SPatrick Williams## OpenBMC Test Documentation 1168c7eec37SGeorge Keishing 11712e33526SPatrick Williams- [Tools](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/openbmc_test_tools.md): 11812e33526SPatrick Williams Reference information for helper tools. 11912e33526SPatrick Williams- [Code Update](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/code_update.md): 12012e33526SPatrick Williams Currently supported BMC and PNOR update. 12112e33526SPatrick Williams- [Certificate Generate](https://github.com/openbmc/openbmc-test-automation/blob/master/docs/certificate_generate.md): 12212e33526SPatrick Williams Steps to create and install CA signed certificate. 1238c7eec37SGeorge Keishing 12412e33526SPatrick Williams## Supported Systems Architecture 12587a3ff53SGeorge Keishing 12687a3ff53SGeorge KeishingOpenBMC test infrastructure is proven capable of running on: 12787a3ff53SGeorge Keishing 12812e33526SPatrick Williams- POWER 12912e33526SPatrick Williams- x86 systems running OpenBMC firmware stack. 13012e33526SPatrick Williams 13112e33526SPatrick Williams## Testing Setup Steps 1323df030faSGeorge Keishing 1333df030faSGeorge KeishingTo verify the installation setup is completed and ready to execute. 1343df030faSGeorge Keishing 13512e33526SPatrick Williams- Download the openbmc-test-automation repository: 136361200b4SMatt Fischer 1373df030faSGeorge Keishing ``` 1383df030faSGeorge Keishing $ git clone https://github.com/openbmc/openbmc-test-automation 1393df030faSGeorge Keishing $ cd openbmc-test-automation 1403df030faSGeorge Keishing ``` 141361200b4SMatt Fischer 14212e33526SPatrick Williams- Execute basic setup test run: 143361200b4SMatt Fischer 1443df030faSGeorge Keishing ``` 1453df030faSGeorge Keishing $ robot -v OPENBMC_HOST:xx.xx.xx.xx templates/test_openbmc_setup.robot 1463df030faSGeorge Keishing ``` 147361200b4SMatt Fischer 1483df030faSGeorge Keishing where xx.xx.xx.xx is the BMC hostname or IP. 149a5c6e9d6SGeorge Keishing 15012e33526SPatrick Williams## Test Layout 1515977d2aeSGeorge Keishing 1525977d2aeSGeorge KeishingThere are several sub-directories within the openbmc-test-automation base which 15312e33526SPatrick Williamscontain test suites, tools, templates, etc. These sub-directories are classified 15412e33526SPatrick Williamsas follows: 1555977d2aeSGeorge Keishing 156*df972ffeSGeorge Keishing`redfish/`: Contains the general test cases for OpenBMC stack functional 157*df972ffeSGeorge Keishingverification. 1585977d2aeSGeorge Keishing 159b7ec2790SGeorge Keishing`systest/`: Contains test cases for HTX bootme testing. 1605977d2aeSGeorge Keishing 1615977d2aeSGeorge Keishing`xcat/`: Contains test cases for XCAT automation. 1625977d2aeSGeorge Keishing 16312e33526SPatrick Williams`gui/test/`: Contains test cases for testing web-based interface built on 16412e33526SPatrick WilliamsAngularJS. 165038e0049SRahul Maheshwari 16612e33526SPatrick Williams`gui/gui_test/`: Contains test cases for testing web-based user interface built 16712e33526SPatrick Williamson Vue.js. 16829814f0eSGeorge Keishing 16912e33526SPatrick Williams`pldm/`: Contains test cases for platform management subsystem (base, bios, fru, 17012e33526SPatrick Williamsplatform, OEM). 171cb858f7aSGeorge Keishing 1720465b9cdSGeorge Keishing`snmp/`: Contains test cases for SNMP (Simple Network Management Protocol) 1730465b9cdSGeorge Keishingconfiguration testing. 1740465b9cdSGeorge Keishing 175*df972ffeSGeorge Keishing`openpower/`: Contains test cases for an OpenPOWER based system. 17638f566bbSGeorge Keishing 1775977d2aeSGeorge Keishing`tools/`: Contains various tools. 1785977d2aeSGeorge Keishing 1795977d2aeSGeorge Keishing`templates/`: Contains sample code examples and setup testing. 180acb3c1efSGeorge Keishing 18112e33526SPatrick Williams`test_list/`: Contains the argument files used for skipping test cases (e.g 18212e33526SPatrick Williams"skip_test", "skip_test_extended", etc.) or grouping them (e.g "HW_CI", 18312e33526SPatrick Williams"CT_basic_run", etc.). 184aba958dcSGeorge Keishing 18512e33526SPatrick Williams## Redfish Test Layout 186acb3c1efSGeorge Keishing 18712e33526SPatrick WilliamsOpenBMC is moving steadily towards DTMF Redfish, which is an open industry 18812e33526SPatrick Williamsstandard specification and schema that meets the expectations of end users for 18912e33526SPatrick Williamssimple, modern and secure management of scalable platform hardware. 190acb3c1efSGeorge Keishing 19112e33526SPatrick Williams`redfish/`: Contains test cases for DMTF Redfish-related feature supported on 19212e33526SPatrick WilliamsOpenBMC. 193acb3c1efSGeorge Keishing 194*df972ffeSGeorge Keishing`redfish/extended/`: Contains test cases for combined DMTF Redfish-related 195*df972ffeSGeorge Keishingfeature supported on OpenBMC. Some of the test will be deprecated. 1963292d4acSGeorge Keishing 19712e33526SPatrick WilliamsNote: Work in progress test development parameter 19812e33526SPatrick Williams`-v REDFISH_SUPPORT_TRANS_STATE:1` to force the test suites to execute in 19912e33526SPatrick Williamsredfish mode only. 200f32ee4cbSGeorge Keishing 20112e33526SPatrick Williams## Quickstart 202acb3c1efSGeorge Keishing 2035977d2aeSGeorge KeishingTo run openbmc-automation first you need to install the prerequisite Python 20412e33526SPatrick Williamspackages which will help to invoke tests through tox (Note that tox version 20512e33526SPatrick Williams2.3.1 or greater is required) or via Robot CLI command. 2065977d2aeSGeorge Keishing 2075977d2aeSGeorge Keishing**Robot Command Line** 2085977d2aeSGeorge Keishing 20912e33526SPatrick Williams- Execute all test suites for `tests/`: 21012e33526SPatrick Williams 2115977d2aeSGeorge Keishing ``` 2125977d2aeSGeorge Keishing $ robot -v OPENBMC_HOST:xx.xx.xx.xx tests 2135977d2aeSGeorge Keishing ``` 214aba958dcSGeorge Keishing 21512e33526SPatrick Williams- Execute a test suite: 21612e33526SPatrick Williams 2175977d2aeSGeorge Keishing ``` 218088da102SGeorge Keishing $ robot -v OPENBMC_HOST:xx.xx.xx.xx redfish/extended/test_basic_ci.robot 2195977d2aeSGeorge Keishing ``` 220b29d2e84SChris Austen 22112e33526SPatrick Williams- Initialize the following test variables which will be used during test 22212e33526SPatrick Williams execution: 22349a80604SGeorge Keishing 22449a80604SGeorge Keishing User can forward declare as environment variables: 22512e33526SPatrick Williams 226a7c46e39SGeorge Keishing ``` 22749a80604SGeorge Keishing $ export OPENBMC_HOST=<openbmc machine IP address/hostname> 2287a212a09SRahul Maheshwari $ export OPENBMC_USERNAME=<openbmc username> 22949a80604SGeorge Keishing $ export OPENBMC_PASSWORD=<openbmc password> 2307a212a09SRahul Maheshwari $ export IPMI_COMMAND=<Dbus/External> 23149a80604SGeorge Keishing ``` 23249a80604SGeorge Keishing 23349a80604SGeorge Keishing or 23449a80604SGeorge Keishing 23549a80604SGeorge Keishing User can input as robot variables as part of the CLI command: 23612e33526SPatrick Williams 23749a80604SGeorge Keishing ``` 23849a80604SGeorge Keishing -v OPENBMC_HOST:<openbmc machine IP address/hostname> 23949a80604SGeorge Keishing -v OPENBMC_USERNAME:<openbmc username> 24049a80604SGeorge Keishing -v OPENBMC_PASSWORD:<openbmc password> 241359f8039SGeorge Keishing ``` 242b29d2e84SChris Austen 243361200b4SMatt Fischer- Testing in qemu: 244361200b4SMatt Fischer 245361200b4SMatt Fischer Set extra environment variables: 24612e33526SPatrick Williams 247a7c46e39SGeorge Keishing ``` 2484d6c1dabSGeorge Keishing $ export SSH_PORT=<ssh port number> 2494d6c1dabSGeorge Keishing $ export HTTPS_PORT=<https port number> 250a7c46e39SGeorge Keishing ``` 251b29d2e84SChris Austen 252361200b4SMatt Fischer Run the QEMU CI test suite (not all tests will pass in qemu): 253361200b4SMatt Fischer 254361200b4SMatt Fischer ``` 255361200b4SMatt Fischer $ OPENBMC_HOST=x.x.x.x SSH_PORT=<port number> HTTPS_PORT=<port number> robot -A test_lists/QEMU_CI tests/ 256361200b4SMatt Fischer ``` 257361200b4SMatt Fischer 25812e33526SPatrick Williams- Run tests: 25912e33526SPatrick Williams 260a7c46e39SGeorge Keishing ``` 2615e021124SAlvin Wang $ tox tests 262b29d2e84SChris Austen ``` 263b29d2e84SChris Austen 264361200b4SMatt Fischer- How to run an individual test: 26512e33526SPatrick Williams 266a7c46e39SGeorge Keishing ``` 267a7c46e39SGeorge Keishing $ tox -e default -- --include Power_On_Test tests/test_basic_poweron.robot 268a7c46e39SGeorge Keishing ``` 26912e33526SPatrick Williams 270361200b4SMatt Fischer- No preset environment variables, default configuration for all supported 271a7c46e39SGeorge Keishing systems: 27212e33526SPatrick Williams 273a7c46e39SGeorge Keishing ``` 274a7c46e39SGeorge Keishing $ OPENBMC_HOST=x.x.x.x tox -e default -- tests 275a7c46e39SGeorge Keishing ``` 276a7c46e39SGeorge Keishing 277361200b4SMatt Fischer- No preset environment variables, one test case from a test suite: 27812e33526SPatrick Williams 27912e33526SPatrick Williams ``` 28012e33526SPatrick Williams $ OPENBMC_HOST=x.x.x.x tox -e default -- --include Power_On_Test tests/test_basic_poweron.robot 28112e33526SPatrick Williams ``` 28212e33526SPatrick Williams 283361200b4SMatt Fischer- No preset environment variables, the entire test suite: 28412e33526SPatrick Williams 28512e33526SPatrick Williams ``` 28612e33526SPatrick Williams $ OPENBMC_HOST=x.x.x.x tox -e default -- tests 28712e33526SPatrick Williams ``` 28812e33526SPatrick Williams 289361200b4SMatt Fischer- No preset environment variables, the entire test suite excluding test cases 29012e33526SPatrick Williams using argument file: 29112e33526SPatrick Williams 292a7c46e39SGeorge Keishing ``` 293a7c46e39SGeorge Keishing $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/skip_test tests 294a7c46e39SGeorge Keishing ``` 295ddf0cc2aSGeorge Keishing 296361200b4SMatt Fischer- Exclude test list for supported systems: 29712e33526SPatrick Williams 298a7c46e39SGeorge Keishing ``` 299ddf0cc2aSGeorge Keishing Palmetto: test_lists/skip_test_palmetto 300ddf0cc2aSGeorge Keishing Witherspoon: test_lists/skip_test_witherspoon 301b29d2e84SChris Austen ``` 302b29d2e84SChris Austen 303361200b4SMatt Fischer Using the exclude lists (example for Witherspoon) 304361200b4SMatt Fischer 305361200b4SMatt Fischer ``` 306361200b4SMatt Fischer $ robot -v OPENBMC_HOST:xx.xx.xx.xx -A test_lists/skip_test_witherspoon tests/ 307361200b4SMatt Fischer ``` 308361200b4SMatt Fischer 30912e33526SPatrick Williams- Run IPMI tests: 3107eea8ed0SRahul Maheshwari 3117eea8ed0SRahul Maheshwari Running only out-of-band IPMI tests: 31212e33526SPatrick Williams 3137eea8ed0SRahul Maheshwari ``` 3147eea8ed0SRahul Maheshwari $ robot -v IPMI_COMMAND:External -v OPENBMC_HOST:x.x.x.x --argumentfile test_lists/witherspoon/skip_inband_ipmi tests/ipmi/ 3157eea8ed0SRahul Maheshwari ``` 3167eea8ed0SRahul Maheshwari 3177eea8ed0SRahul Maheshwari Running only inband IPMI tests: 31812e33526SPatrick Williams 3197eea8ed0SRahul Maheshwari ``` 3207eea8ed0SRahul Maheshwari $ robot -v IPMI_COMMAND:Inband -v OPENBMC_HOST:x.x.x.x -v OS_HOST:x.x.x.x -v OS_USERNAME:xxxx -v OS_PASSWORD:xxxx --argumentfile test_lists/witherspoon/skip_oob_ipmi tests/ipmi/ 3217eea8ed0SRahul Maheshwari ``` 3227eea8ed0SRahul Maheshwari 32312e33526SPatrick Williams- Run GUI tests: 3247eea8ed0SRahul Maheshwari 32512e33526SPatrick Williams By default, GUI runs with Firefox browser and headless mode. Example with 326361200b4SMatt Fischer Chrome browser and header mode: 327dd38aef3SRahul Maheshwari 328dd38aef3SRahul Maheshwari ``` 329eeb93c31SRahul Maheshwari $ robot -v OPENBMC_HOST:x.x.x.x -v GUI_BROWSER:gc -v GUI_MODE:header gui/test/ 330dd38aef3SRahul Maheshwari ``` 331dd38aef3SRahul Maheshwari 3322bd32ff1SRahul Maheshwari Run GUI default CI test bucket: 33312e33526SPatrick Williams 3342bd32ff1SRahul Maheshwari ``` 335eeb93c31SRahul Maheshwari $ robot -v OPENBMC_HOST:x.x.x.x --argumentfile test_lists/BMC_WEB_CI gui/test/ 3362bd32ff1SRahul Maheshwari ``` 3372bd32ff1SRahul Maheshwari 33812e33526SPatrick Williams- Run LDAP tests: 3395d00a3c8SSivas SRR 34012e33526SPatrick Williams Before using LDAP test functions, be sure appropriate LDAP user(s) and 34112e33526SPatrick Williams group(s) have been created on your LDAP server. Note: There are multiple ways 34212e33526SPatrick Williams to create LDAP users / groups and all depend on your LDAP server. One common 34312e33526SPatrick Williams way for openldap is ldapadd / ldapmodify refer 34412e33526SPatrick Williams https://linux.die.net/man/1/ldapadd For ldapsearch, refer to 34512e33526SPatrick Williams "https://linux.die.net/man/1/ldapsearch". Microsoft ADS: refer to 34612e33526SPatrick Williams https://searchwindowsserver.techtarget.com/definition/Microsoft-Active-Directory-Domain-Services-AD-DS 347962ff298SSivas SRR 34812e33526SPatrick Williams Note: Currently, LDAP test automation for Redfish API is in progress. The 34912e33526SPatrick Williams format to invoke LDAP test is as follows: 35012e33526SPatrick Williams 3515d00a3c8SSivas SRR ``` 352962ff298SSivas SRR $ cd redfish/account_service/ 353962ff298SSivas SRR $ robot -v OPENBMC_HOST:x.x.x.x -v LDAP_SERVER_URI:<ldap(s)//LDAP Hostname / IP> -v LDAP_BIND_DN:<LDAP Bind DN> -v LDAP_BASE_DN:<LDAP Base DN> -v LDAP_BIND_DN_PASSWORD:<LDAP Bind password> -v LDAP_SEARCH_SCOPE:<LDAP search scope> -v LDAP_SERVER_TYPE:<LDAP server type> -v LDAP_USER:<LDAP user-id> -v LDAP_USER_PASSWORD:<LDAP PASSWORD> -v GROUP_NAME:<Group Name> -v GROUP_PRIVILEGE:<Privilege> ./test_ldap_configuration.robot 3545d00a3c8SSivas SRR ``` 3555d00a3c8SSivas SRR 35612e33526SPatrick Williams- How to run CI and CT bucket test: 357d7c26840SGeorge Keishing 358d7c26840SGeorge Keishing Default CI test bucket list: 35912e33526SPatrick Williams 360d7c26840SGeorge Keishing ``` 361d7c26840SGeorge Keishing $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/HW_CI tests 362d7c26840SGeorge Keishing ``` 363d7c26840SGeorge Keishing 364d7c26840SGeorge Keishing Default CI smoke test bucket list: 36512e33526SPatrick Williams 366d7c26840SGeorge Keishing ``` 367d7c26840SGeorge Keishing $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/CT_basic_run tests 368d7c26840SGeorge Keishing ``` 369d7c26840SGeorge Keishing 37012e33526SPatrick Williams- Run extended tests: 371aba958dcSGeorge Keishing 372aba958dcSGeorge Keishing For-loop test (default iteration is 10): 37312e33526SPatrick Williams 374aba958dcSGeorge Keishing ``` 375aba958dcSGeorge Keishing $ robot -v OPENBMC_HOST:x.x.x.x -v OPENBMC_SYSTEMMODEL:xxxxxx -v ITERATION:n -v LOOP_TEST_COMMAND:xxxxxx extended/full_suite_regression.robot 376aba958dcSGeorge Keishing ``` 377aba958dcSGeorge Keishing 378aba958dcSGeorge Keishing Example using tox testing a test suite for 5 iterations "witherspoon": 37912e33526SPatrick Williams 380aba958dcSGeorge Keishing ``` 381aba958dcSGeorge Keishing OPENBMC_HOST=x.x.x.x LOOP_TEST_COMMAND="tests/test_fw_version.robot" ITERATION=5 OPENBMC_SYSTEMMODEL=witherspoon tox -e witherspoon -- ./extended/full_suite_regression.robot 382aba958dcSGeorge Keishing ``` 383aba958dcSGeorge Keishing 38412e33526SPatrick Williams- Host CPU architecture 385d61a64baSKonstantin Aladyshev 38612e33526SPatrick Williams By default openbmc-test-automation framework assumes that host CPU is based on 38712e33526SPatrick Williams the POWER architecture. If your host CPU is x86 add 38812e33526SPatrick Williams `-v PLATFORM_ARCH_TYPE:x86` variable setting to your CLI commands or set an 38912e33526SPatrick Williams environment variable: 39012e33526SPatrick Williams 391c056c2bbSKonstantin Aladyshev ``` 392c056c2bbSKonstantin Aladyshev $ export PLATFORM_ARCH_TYPE=x86 393c056c2bbSKonstantin Aladyshev ``` 394d61a64baSKonstantin Aladyshev 395aba958dcSGeorge Keishing**Jenkins jobs tox commands** 39612e33526SPatrick Williams 39712e33526SPatrick Williams- HW CI tox command: 398361200b4SMatt Fischer 399aba958dcSGeorge Keishing ``` 400aba958dcSGeorge Keishing $ OPENBMC_HOST=x.x.x.x tox -e default -- --argumentfile test_lists/HW_CI tests 401aba958dcSGeorge Keishing ``` 402