1b29d2e84SChris Austen#openbmc-automation 2b29d2e84SChris Austen 3b29d2e84SChris AustenQuickstart 4b29d2e84SChris Austen---------- 5b29d2e84SChris Austen 6b29d2e84SChris AustenTo run openbmc-automation first you need to install the prerequisite python 74319d194SJay Azurinpackages which will help to invoke tests through tox. Note that tox 84319d194SJay Azurinversion 2.3.1 or greater is required. 9b29d2e84SChris Austen 10b29d2e84SChris AustenInstall the python dependencies for tox 11b29d2e84SChris Austen```shell 124319d194SJay Azurin $ easy_install tox 13b29d2e84SChris Austen $ easy_install pip 14b29d2e84SChris Austen``` 15b29d2e84SChris Austen 16ee1d38c3SGunnar MillsInitialize the following environment variable which will be used during testing 17b29d2e84SChris Austen```shell 18b29d2e84SChris Austen $ export OPENBMC_HOST=<openbmc machine ip address> 197a212a09SRahul Maheshwari $ export OPENBMC_PASSWORD=<openbmc password> 207a212a09SRahul Maheshwari $ export OPENBMC_USERNAME=<openbmc username> 21b29d2e84SChris Austen $ export OPENBMC_MODEL=[./data/Barreleye.py, ./data/Palmetto.py, etc] 227a212a09SRahul Maheshwari $ export IPMI_COMMAND=<Dbus/External> 237a212a09SRahul Maheshwari $ export IPMI_PASSWORD=<External IPMI password> 24359f8039SGeorge Keishing``` 25b29d2e84SChris Austen 26359f8039SGeorge KeishingThere are two different set of test suite existing based on the usage. 27359f8039SGeorge KeishingThe test suites are distinctly separated by directory as under 28359f8039SGeorge Keishing tests/ 29359f8039SGeorge Keishing extended/ 30b29d2e84SChris Austen 31359f8039SGeorge Keishing`tests`: directory contains the general test cases 32359f8039SGeorge Keishing 33359f8039SGeorge Keishing`extended`: directory contains the use cases for new IP network testing, PDU, 34359f8039SGeorge KeishingBIOS and BMC code update. 35359f8039SGeorge Keishing 36359f8039SGeorge Keishing```shell 37b29d2e84SChris AustenUse Following Variables for networking test cases 38b29d2e84SChris Austen=========================================================== 39b29d2e84SChris Austen $export NEW_BMC_IP=<openbmc machine ip address> 40b29d2e84SChris Austen $export NEW_SUBNET_MASK=<openbmc new subnet mask> 41b29d2e84SChris Austen $export NEW_GATEWAY=<openbmc new gateway> 42b29d2e84SChris Austen========================================================== 43b29d2e84SChris Austen 44b29d2e84SChris Austen Use following parameters for PDU: 45b29d2e84SChris Austen $ export PDU_IP=<PDU IP address> 46b29d2e84SChris Austen $ export PDU_USERNAME=<PDU username> 47b29d2e84SChris Austen $ export PDU_PASSWORD=<PDU password> 48b29d2e84SChris Austen $ export PDU_TYPE=<PDU type> 49b29d2e84SChris Austen $ export PDU_SLOT_NO=<SLOT number> 50b29d2e84SChris Austen 51b29d2e84SChris Austen for PDU_TYPE we support only synaccess at the moment 52b29d2e84SChris Austen 53b29d2e84SChris AustenUse following variables for syslog test cases 54b29d2e84SChris Austen========================================================== 55b29d2e84SChris Austen $ export SYSLOG_IP_ADDRESS=<remote syslog system ip> 56b29d2e84SChris Austen $ export SYSLOG_PORT=<remote syslog system port> 57b29d2e84SChris Austen 584d6c1dabSGeorge KeishingUse the following variables for Qemu test run 594d6c1dabSGeorge Keishing========================================================== 604d6c1dabSGeorge Keishing $ export SSH_PORT=<ssh port number> 614d6c1dabSGeorge Keishing $ export HTTPS_PORT=<https port number> 624d6c1dabSGeorge Keishing 63e4c52eb2SJay AzurinUse the following variables for BIOS update testing 64e4c52eb2SJay Azurin========================================================== 65e4c52eb2SJay Azurin $ export PNOR_IMAGE_PATH=<path to>/<machine>.pnor 66e4c52eb2SJay Azurin 67b29d2e84SChris Austen``` 68b29d2e84SChris Austen 69b29d2e84SChris AustenRun tests 70b29d2e84SChris Austen```shell 71b29d2e84SChris Austen $ tox -e tests 72b29d2e84SChris Austen``` 73b29d2e84SChris Austen 74b29d2e84SChris AustenHow to test individual test 75b29d2e84SChris Austen```shell 76b29d2e84SChris Austen One specific test 77b29d2e84SChris Austen $ tox -e custom -- -t '"DIMM0 no fault"' tests/test_sensors.robot 78b29d2e84SChris Austen 79b29d2e84SChris Austen No preset environment variables, one test case from one test suite 80b29d2e84SChris Austen $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- -t '"DIMM0 no fault"' tests/test_sensors.robot 81b29d2e84SChris Austen 82b29d2e84SChris Austen No preset environment variables, one test suite for a palmetto system 83b29d2e84SChris Austen $ OPENBMC_HOST=x.x.x.x tox -e palmetto -- tests/test_sensors.robot 84b29d2e84SChris Austen 85b29d2e84SChris Austen No preset environment variables, the entire test suite for a barreleye system 86b29d2e84SChris Austen $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- tests 8708fd617cSRahul Maheshwari 8808fd617cSRahul Maheshwari No preset environment variables, the entire test suite excluding test 89f8119103SRahul Maheshwari cases using argument file. 90f8119103SRahul Maheshwari $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/skip_test tests 91b29d2e84SChris Austen``` 92b29d2e84SChris Austen 93ee1d38c3SGunnar MillsIt can also be run by passing variables from the cli... 94b29d2e84SChris Austen```shell 957a212a09SRahul Maheshwari Run one test suite using using pybot 967a212a09SRahul Maheshwari $ pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v OPENBMC_MODEL:<model path> tests/test_time.robot 977a212a09SRahul Maheshwari 987a212a09SRahul Maheshwari Run entire test suite using using pybot 997a212a09SRahul Maheshwari $ pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v OPENBMC_MODEL:<model path> tests 1007a212a09SRahul Maheshwari 1017a212a09SRahul Maheshwari Run entire test suite using external ipmitool 1027a212a09SRahul Maheshwari $ pybot -v OPENBMC_HOST:<ip> -v OPENBMC_USERNAME:root -v OPENBMC_PASSWORD:0penBmc -v IPMI_COMMAND:External -v IPMI_PASSWORD:PASSW0RD -v OPENBMC_MODEL:<model path> tests 103b29d2e84SChris Austen``` 104359f8039SGeorge Keishing 105359f8039SGeorge KeishingRun extended tests 106359f8039SGeorge Keishing```shell 107359f8039SGeorge Keishing Set the preset environment variables, run test suite for a barreleye system 108359f8039SGeorge Keishing $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- extended/test_power_restore.robot 109359f8039SGeorge Keishing 110359f8039SGeorge Keishing Similarly for Network, PDU and update BIOS 111359f8039SGeorge Keishing 112359f8039SGeorge Keishing For BMC code update, download the system type *.all.tar image from https://openpower.xyz 113359f8039SGeorge Keishing and run as follows: 114359f8039SGeorge Keishing 115359f8039SGeorge Keishing For Barreleye system 116359f8039SGeorge Keishing python -m robot -v OPENBMC_HOST:x.x.x.x -v FILE_PATH:downloaded_path/barreleye-xxxx.all.tar extended/code_update/update_bmc.robot 117594a6c9dSSivas SRR 118594a6c9dSSivas SRR For loop test (Default iteration is 10) 119594a6c9dSSivas SRR python -m robot -v OPENBMC_HOST:x.x.x.x -v OPENBMC_SYSTEMMODEL:xxxxxx -v ITERATION:n -v LOOP_TEST_COMMAND:xxxxxx extended/full_suite_regression.robot 120594a6c9dSSivas SRR Below is sample command using tox to test only fw version using Barreleye system for 5 times 121594a6c9dSSivas SRR OPENBMC_HOST=x.x.x.x LOOP_TEST_COMMAND="--argumentfile test_lists/skip_test tests/test_fw_version.robot" ITERATION=5 OPENBMC_SYSTEMMODEL=barreleye tox -e barreleye -- ./extended/full_suite_regression.robot 122359f8039SGeorge Keishing``` 12397651c75SGeorge Keishing 12497651c75SGeorge KeishingJenkins jobs tox commands 12597651c75SGeorge Keishing```shell 12697651c75SGeorge Keishing HW CI tox command 12797651c75SGeorge Keishing Set the preset environment variables, run HW CI test for a barreleye system 128f8119103SRahul Maheshwari $ OPENBMC_HOST=x.x.x.x tox -e barreleye -- --argumentfile test_lists/HW_CI tests 12997651c75SGeorge Keishing 13097651c75SGeorge Keishing``` 1314cae87daSMichael Walsh 1324cae87daSMichael WalshTemplate to be used to create new stand-alone python programs. 1334cae87daSMichael Walsh```shell 1344cae87daSMichael Walsh 1354cae87daSMichael WalshIf you wish to create a new python stand-alone program, please copy bin/python_pgm_template to your new name and then begin your work. Example: 1364cae87daSMichael Walsh 1374cae87daSMichael Walshcd bin 1384cae87daSMichael Walshcp python_pgm_template my_new_program 1394cae87daSMichael Walsh 1404cae87daSMichael WalshThis template has much of your preliminary work done for you and it will help us all follow a similar structure. 1414cae87daSMichael Walsh 1424cae87daSMichael WalshFeatures: 1434cae87daSMichael Walsh- Help text and argparsing started for you. 1444cae87daSMichael Walsh- Support for "stock" parameters like quiet, debug, test_mode. 1454cae87daSMichael Walsh- exit_function and signal_handler defined. 1464cae87daSMichael Walsh- validate_parms function pre-created. 1474cae87daSMichael Walsh- main function follows conventional startup: 1484cae87daSMichael Walsh 1494cae87daSMichael Walsh if not gen_get_options(parser, stock_list): 1504cae87daSMichael Walsh return False 1514cae87daSMichael Walsh 1524cae87daSMichael Walsh if not validate_parms(): 1534cae87daSMichael Walsh return False 1544cae87daSMichael Walsh 1554cae87daSMichael Walsh qprint_pgm_header() 1564cae87daSMichael Walsh 1574cae87daSMichael Walsh # Your code here. 1584cae87daSMichael Walsh 1594cae87daSMichael Walsh``` 160a464a7ceSSivas SRR 161a464a7ceSSivas SRRCommand to get GitHub issues (any GitHub repository) report in CSV format 162*9fe1b12eSSivas SRRNote: On Prompt "Enter your GitHub Password:" enter your GitHub password. 163a464a7ceSSivas SRR```shell 164*9fe1b12eSSivas SRRpython ./tools/github_issues_to_csv <github user> <github repo> 165a464a7ceSSivas SRR 166a464a7ceSSivas SRRExample for getting openbmc issues 167*9fe1b12eSSivas SRRpython ./tools/github_issues_to_csv <github user> openbmc/openbmc 168a464a7ceSSivas SRR 169a464a7ceSSivas SRRExample for getting openbmc-test-automation issues 170*9fe1b12eSSivas SRRpython ./tools/github_issues_to_csv <github user> openbmc/openbmc-test-automation 171a464a7ceSSivas SRR``` 172