1*** Settings *** 2 3Documentation Utilities for Robot keywords. 4 5Resource ../lib/resource.txt 6Resource ../lib/rest_client.robot 7Resource ../lib/connection_client.robot 8Library String 9Library DateTime 10Library Process 11Library OperatingSystem 12Library gen_print.py 13Library gen_robot_print.py 14Library gen_cmd.py 15Library gen_robot_keyword.py 16Library bmc_ssh_utils.py 17Library utils.py 18Library var_funcs.py 19Library SCPLibrary WITH NAME scp 20 21*** Variables *** 22 23${pflash_cmd} /usr/sbin/pflash -r /dev/stdout -P VERSION 24${SYSTEM_SHUTDOWN_TIME} ${5} 25 26${dbuscmdBase} 27... dbus-send --system --print-reply --dest=${OPENBMC_BASE_DBUS}.settings.Host 28${dbuscmdGet} 29... ${SETTINGS_URI}host0 org.freedesktop.DBus.Properties.Get 30${dbuscmdString}= string:"xyz.openbmc_project.settings.Host" string: 31 32# Assign default value to QUIET for programs which may not define it. 33${QUIET} ${0} 34 35${bmc_mem_free_cmd}= free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f4 36${bmc_mem_total_cmd}= free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f2 37${bmc_cpu_usage_cmd}= top -n 1 | grep CPU: | cut -c 7-9 38${HOST_SETTING} ${SETTINGS_URI}host0 39 40# /run/initramfs/ro associate filesystem should be 100% full always 41${bmc_file_system_usage_cmd}= df -h | cut -c 52-54 | grep 100 | wc -l 42${total_pnor_ro_file_system_cmd}= df -h | grep /media/pnor-ro | wc -l 43${total_bmc_ro_file_system_cmd}= df -h | grep /media/rofs | wc -l 44 45${BOOT_TIME} ${0} 46${BOOT_COUNT} ${0} 47${count} ${0} 48${devicetree_base} /sys/firmware/devicetree/base/model 49 50# Initialize default debug value to 0. 51${DEBUG} ${0} 52 53${probe_cpu_tool_path} ${EXECDIR}/tools/ras/probe_cpus.sh 54${scom_addrs_tool_path} ${EXECDIR}/tools/ras/scom_addr_p9.sh 55${target_file_path} /root/ 56 57${default_tarball} ${EXECDIR}/obmc-phosphor-debug-tarball-witherspoon.tar.xz 58 59# These variables are used to straddle between new and old methods of setting 60# values. 61${boot_prog_method} ${EMPTY} 62${power_policy_setup} ${0} 63${bmc_power_policy_method} ${EMPTY} 64@{valid_power_policy_vars} RESTORE_LAST_STATE ALWAYS_POWER_ON 65... ALWAYS_POWER_OFF 66 67 68*** Keywords *** 69 70Check BMC Performance 71 [Documentation] Check BMC basic CPU Mem File system performance. 72 73 Check BMC CPU Performance 74 Check BMC Mem Performance 75 Check BMC File System Performance 76 77 78Verify PNOR Update 79 [Documentation] Verify that the PNOR is not corrupted. 80 # Example: 81 # FFS: Flash header not found. Code: 100 82 # Error 100 opening ffs ! 83 84 ${pnor_info}= BMC Execute Command ${pflash_cmd} 85 Should Not Contain Any ${pnor_info} Flash header not found Error 86 87 88Get BMC System Model 89 [Documentation] Get the BMC model from the device tree and return it. 90 91 ${bmc_model} ${stderr} ${rc}= BMC Execute Command 92 ... cat ${devicetree_base} | cut -d " " -f 1 return_stderr=True 93 ... test_mode=0 94 Should Be Empty ${stderr} 95 Should Not Be Empty ${bmc_model} msg=BMC model is empty. 96 [Return] ${bmc_model} 97 98 99Verify BMC System Model 100 [Documentation] Verify the BMC model with ${OPENBMC_MODEL}. 101 [Arguments] ${bmc_model} 102 103 # Description of argument(s): 104 # bmc_model System model (e.g. "witherspoon"). 105 106 ${tmp_bmc_model}= Fetch From Right ${OPENBMC_MODEL} / 107 ${tmp_bmc_model}= Fetch From Left ${tmp_bmc_model} . 108 ${ret}= Run Keyword And Return Status Should Contain ${bmc_model} 109 ... ${tmp_bmc_model} ignore_case=True 110 [Return] ${ret} 111 112 113Wait For Host To Ping 114 [Documentation] Wait for the given host to ping. 115 [Arguments] ${host} ${timeout}=${OPENBMC_REBOOT_TIMEOUT}min 116 ... ${interval}=5 sec 117 118 # Description of argument(s): 119 # host The host name or IP of the host to ping. 120 # timeout The amount of time after which ping attempts cease. 121 # This should be expressed in Robot Framework's time format 122 # (e.g. "10 seconds"). 123 # interval The amount of time in between attempts to ping. 124 # This should be expressed in Robot Framework's time format 125 # (e.g. "5 seconds"). 126 127 Wait Until Keyword Succeeds ${timeout} ${interval} Ping Host ${host} 128 129 130Ping Host 131 [Documentation] Ping the given host. 132 [Arguments] ${host} 133 134 # Description of argument(s): 135 # host The host name or IP of the host to ping. 136 137 Should Not Be Empty ${host} msg=No host provided 138 ${RC} ${output}= Run and return RC and Output ping -c 4 ${host} 139 Log RC: ${RC}\nOutput:\n${output} 140 Should be equal ${RC} ${0} 141 142 143Get Boot Progress 144 [Documentation] Get the boot progress and return it. 145 [Arguments] ${quiet}=${QUIET} 146 147 # Description of argument(s): 148 # quiet Indicates whether this keyword should run without any output to 149 # the console. 150 151 Set Boot Progress Method 152 ${state}= Run Keyword If '${boot_prog_method}' == 'New' 153 ... New Get Boot Progress quiet=${quiet} 154 ... ELSE 155 ... Old Get Boot Progress quiet=${quiet} 156 157 [Return] ${state} 158 159 160Set Boot Progress Method 161 [Documentation] Set the boot_prog_method to either 'Old' or 'New'. 162 163 # The boot progress data has moved from an 'org' location to an 'xyz' 164 # location. This keyword will determine whether the new method of getting 165 # the boot progress is valid and will set the global boot_prog_method 166 # variable accordingly. If boot_prog_method is already set (either by a 167 # prior call to this function or via a -v parm), this keyword will simply 168 # return. 169 170 # Note: There are interim builds that contain boot_progress in both the 171 # old and the new location values. It is nearly impossible for this 172 # keyword to determine whether the old boot_progress or the new one is 173 # active. When using such builds where the old boot_progress is active, 174 # the only recourse users will have is that they may specify 175 # -v boot_prog_method:Old to force old behavior on such builds. 176 177 Run Keyword If '${boot_prog_method}' != '${EMPTY}' Return From Keyword 178 179 ${new_status} ${new_value}= Run Keyword And Ignore Error 180 ... New Get Boot Progress 181 # If the new style read fails, the method must necessarily be "Old". 182 Run Keyword If '${new_status}' == 'PASS' 183 ... Run Keywords 184 ... Set Global Variable ${boot_prog_method} New AND 185 ... Rqpvars boot_prog_method AND 186 ... Return From Keyword 187 188 # Default method is "Old". 189 Set Global Variable ${boot_prog_method} Old 190 Rqpvars boot_prog_method 191 192 193Old Get Boot Progress 194 [Documentation] Get the boot progress the old way (via org location). 195 [Arguments] ${quiet}=${QUIET} 196 197 # Description of argument(s): 198 # quiet Indicates whether this keyword should run without any output to 199 # the console. 200 201 ${state}= Read Attribute ${OPENBMC_BASE_URI}sensors/host/BootProgress 202 ... value quiet=${quiet} 203 204 [Return] ${state} 205 206 207New Get Boot Progress 208 [Documentation] Get the boot progress the new way (via xyz location). 209 [Arguments] ${quiet}=${QUIET} 210 211 # Description of argument(s): 212 # quiet Indicates whether this keyword should run without any output to 213 # the console. 214 215 ${state}= Read Attribute ${HOST_STATE_URI} BootProgress quiet=${quiet} 216 217 [Return] ${state.rsplit('.', 1)[1]} 218 219 220Is Power On 221 [Documentation] Verify that the BMC chassis state is on. 222 ${state}= Get Power State 223 Should be equal ${state} ${1} 224 225 226Is Power Off 227 [Documentation] Verify that the BMC chassis state is off. 228 ${state}= Get Power State 229 Should be equal ${state} ${0} 230 231 232Initiate Power On 233 [Documentation] Initiates the power on and waits until the Is Power On 234 ... keyword returns that the power state has switched to on. 235 [Arguments] ${wait}=${1} 236 237 # Description of argument(s): 238 # wait Indicates whether to wait for a powered on state after issuing 239 # the power on command. 240 241 @{arglist}= Create List 242 ${args}= Create Dictionary data=@{arglist} 243 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ powerOn 244 ... data=${args} 245 should be equal as strings ${resp.status_code} ${HTTP_OK} 246 247 # Does caller want to wait for power on status? 248 Run Keyword If '${wait}' == '${0}' Return From Keyword 249 Wait Until Keyword Succeeds 3 min 10 sec Is Power On 250 251 252Initiate Power Off 253 [Documentation] Initiates the power off and waits until the Is Power Off 254 ... keyword returns that the power state has switched to off. 255 256 @{arglist}= Create List 257 ${args}= Create Dictionary data=@{arglist} 258 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ powerOff 259 ... data=${args} 260 should be equal as strings ${resp.status_code} ${HTTP_OK} 261 Wait Until Keyword Succeeds 1 min 10 sec Is Power Off 262 263 264Initiate OS Host Power Off 265 [Documentation] Initiate an OS reboot. 266 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 267 ... ${os_password}=${OS_PASSWORD} ${hard}=${0} 268 269 # Description of argument(s): 270 # os_host The DNS name or IP of the OS. 271 # os_username The username to be used to sign in to the OS. 272 # os_password The password to be used to sign in to the OS. 273 # hard Indicates whether to do a hard vs. soft power off. 274 275 ${time_string}= Run Keyword If ${hard} Set Variable ${SPACE}now 276 ... ELSE Set Variable ${EMPTY} 277 278 ${cmd_buf}= Run Keyword If '${os_username}' == 'root' 279 ... Set Variable shutdown${time_string} 280 ... ELSE 281 ... Set Variable echo ${os_password} | sudo -S shutdown${time_string} 282 283 ${output} ${stderr} ${rc}= OS Execute Command 284 ... ${cmd_buf} fork=${1} 285 286 287Initiate OS Host Reboot 288 [Documentation] Initiate an OS reboot. 289 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 290 ... ${os_password}=${OS_PASSWORD} 291 292 # Description of argument(s): 293 # os_host The host name or IP address of the OS. 294 # os_username The username to be used to sign in to the OS. 295 # os_password The password to be used to sign in to the OS. 296 297 ${cmd_buf}= Run Keyword If '${os_username}' == 'root' 298 ... Set Variable reboot 299 ... ELSE 300 ... Set Variable echo ${os_password} | sudo -S reboot 301 302 ${output} ${stderr} ${rc}= OS Execute Command 303 ... ${cmd_buf} fork=${1} 304 305 306Initiate Auto Reboot 307 [Documentation] Initiate an auto reboot. 308 [Arguments] ${milliseconds}=5000 309 310 # Description of argument(s): 311 # milliseconds The number of milliseconds for the watchdog timer. 312 313 # Set the auto reboot policy. 314 Set Auto Reboot ${1} 315 # Set the watchdog timer. 316 Trigger Host Watchdog Error ${milliseconds} 317 318 319Trigger Warm Reset 320 [Documentation] Initiate a warm reset. 321 322 log to console "Triggering warm reset" 323 ${data}= create dictionary data=@{EMPTY} 324 ${resp}= openbmc post request 325 ... ${OPENBMC_BASE_URI}control/bmc0/action/warmReset data=${data} 326 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 327 ${session_active}= Check If warmReset is Initiated 328 Run Keyword If '${session_active}' == '${True}' 329 ... Fail msg=warm reset didn't occur 330 331 Sleep ${SYSTEM_SHUTDOWN_TIME}min 332 Check If BMC Is Up 333 334 335Check OS 336 [Documentation] Attempts to ping the host OS and then checks that the host 337 ... OS is up by running an SSH command. 338 339 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 340 ... ${os_password}=${OS_PASSWORD} ${quiet}=${QUIET} 341 ... ${print_string}=${EMPTY} 342 [Teardown] SSHLibrary.Close Connection 343 344 # Description of argument(s): 345 # os_host The DNS name/IP of the OS host associated with our BMC. 346 # os_username The username to be used to sign on to the OS host. 347 # os_password The password to be used to sign on to the OS host. 348 # quiet Indicates whether this keyword should write to console. 349 # print_string A string to be printed before checking the OS. 350 351 rprint ${print_string} 352 353 # Attempt to ping the OS. Store the return code to check later. 354 ${ping_rc}= Run Keyword and Return Status Ping Host ${os_host} 355 356 SSHLibrary.Open connection ${os_host} 357 358 ${status} ${msg}= Run Keyword And Ignore Error Login ${os_username} 359 ... ${os_password} 360 ${err_msg1}= Sprint Error ${msg} 361 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1} 362 Run Keyword If '${status}' == 'FAIL' Fail msg=${err_msg} 363 ${output} ${stderr} ${rc}= Execute Command uptime return_stderr=True 364 ... return_rc=True 365 366 ${temp_msg}= Catenate Could not execute a command on the operating 367 ... system.\n 368 ${err_msg1}= Sprint Error ${temp_msg} 369 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1} 370 371 # If the return code returned by "Execute Command" is non-zero, this 372 # keyword will fail. 373 Should Be Equal ${rc} ${0} msg=${err_msg} 374 # We will likewise fail if there is any stderr data. 375 Should Be Empty ${stderr} 376 377 ${temp_msg}= Set Variable Could not ping the operating system.\n 378 ${err_msg1}= Sprint Error ${temp_msg} 379 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1} 380 # We will likewise fail if the OS did not ping, as we could SSH but not 381 # ping 382 Should Be Equal As Strings ${ping_rc} ${TRUE} msg=${err_msg} 383 384 385Wait for OS 386 [Documentation] Waits for the host OS to come up via calls to "Check OS". 387 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 388 ... ${os_password}=${OS_PASSWORD} ${timeout}=${OS_WAIT_TIMEOUT} 389 ... ${quiet}=${0} 390 [Teardown] rprintn 391 392 # Description of argument(s): 393 # os_host The DNS name or IP of the OS host associated with our 394 # BMC. 395 # os_username The username to be used to sign on to the OS host. 396 # os_password The password to be used to sign on to the OS host. 397 # timeout The timeout in seconds indicating how long you're 398 # willing to wait for the OS to respond. 399 # quiet Indicates whether this keyword should write to console. 400 401 # The interval to be used between calls to "Check OS". 402 ${interval}= Set Variable 5 403 404 ${message}= Catenate Checking every ${interval} seconds for up to 405 ... ${timeout} seconds for the operating system to communicate. 406 rqprint_timen ${message} 407 408 Wait Until Keyword Succeeds ${timeout} sec ${interval} Check OS 409 ... ${os_host} ${os_username} ${os_password} 410 ... print_string=\# 411 412 rqprintn 413 414 rqprint_timen The operating system is now communicating. 415 416 417Get Power State 418 [Documentation] Returns the power state as an integer. Either 0 or 1. 419 [Arguments] ${quiet}=${QUIET} 420 421 # Description of argument(s): 422 # quiet Indicates whether this keyword should run without any output to 423 # the console. 424 425 @{arglist}= Create List 426 ${args}= Create Dictionary data=@{arglist} 427 428 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ getPowerState 429 ... data=${args} quiet=${quiet} 430 Should be equal as strings ${resp.status_code} ${HTTP_OK} 431 ${content}= to json ${resp.content} 432 433 [Return] ${content["data"]} 434 435 436Clear BMC Gard Record 437 [Documentation] Clear gard records from the system. 438 439 @{arglist}= Create List 440 ${args}= Create Dictionary data=@{arglist} 441 ${resp}= Call Method 442 ... ${OPENPOWER_CONTROL}gard Reset data=${args} 443 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 444 445 446Copy PNOR to BMC 447 [Documentation] Copy the PNOR image to the BMC. 448 Import Library SCPLibrary WITH NAME scp 449 Open Connection for SCP 450 Log Copying ${PNOR_IMAGE_PATH} to /tmp 451 scp.Put File ${PNOR_IMAGE_PATH} /tmp 452 453 454Flash PNOR 455 [Documentation] Calls flash bios update method to flash PNOR image 456 [Arguments] ${pnor_image} 457 458 # Description of argument(s): 459 # pnor_image The filename and path of the PNOR image 460 # (e.g. "/home/image/zaius.pnor"). 461 462 @{arglist}= Create List ${pnor_image} 463 ${args}= Create Dictionary data=@{arglist} 464 ${resp}= Call Method /org/openbmc/control/flash/bios/ update 465 ... data=${args} 466 should be equal as strings ${resp.status_code} ${HTTP_OK} 467 Wait Until Keyword Succeeds 2 min 10 sec Is PNOR Flashing 468 469 470Get Flash BIOS Status 471 [Documentation] Returns the status of the flash BIOS API as a string. For 472 ... example 'Flashing', 'Flash Done', etc 473 ${data}= Read Properties /org/openbmc/control/flash/bios 474 [Return] ${data['status']} 475 476 477Is PNOR Flashing 478 [Documentation] Get BIOS 'Flashing' status. This indicates that PNOR 479 ... flashing has started. 480 ${status}= Get Flash BIOS Status 481 Should Contain ${status} Flashing 482 483 484Is PNOR Flash Done 485 [Documentation] Get BIOS 'Flash Done' status. This indicates that the 486 ... PNOR flashing has completed. 487 ${status}= Get Flash BIOS Status 488 should be equal as strings ${status} Flash Done 489 490 491Is OS Starting 492 [Documentation] Check if boot progress is OS starting. 493 ${boot_progress}= Get Boot Progress 494 Should Be Equal ${boot_progress} OSStart 495 496 497Is OS Off 498 [Documentation] Check if boot progress is "Off". 499 ${boot_progress}= Get Boot Progress 500 Should Be Equal ${boot_progress} Off 501 502 503Get Boot Progress To OS Starting State 504 [Documentation] Get the system to a boot progress state of 'FW Progress, 505 ... Starting OS'. 506 507 ${boot_progress}= Get Boot Progress 508 Run Keyword If '${boot_progress}' == 'OSStart' 509 ... Log Host is already in OS starting state 510 ... ELSE 511 ... Run Keywords Initiate Host PowerOff AND Initiate Host Boot 512 ... AND Wait Until Keyword Succeeds 10 min 10 sec Is OS Starting 513 514 515Verify Ping and REST Authentication 516 [Documentation] Verify ping and rest authentication. 517 ${l_ping}= Run Keyword And Return Status 518 ... Ping Host ${OPENBMC_HOST} 519 Run Keyword If '${l_ping}' == '${False}' 520 ... Fail msg=Ping Failed 521 522 ${l_rest}= Run Keyword And Return Status 523 ... Initialize OpenBMC 524 Run Keyword If '${l_rest}' == '${False}' 525 ... Fail msg=REST Authentication Failed 526 527 # Just to make sure the SSH is working for SCP 528 Open Connection And Log In 529 ${system} ${stderr}= Execute Command hostname return_stderr=True 530 Should Be Empty ${stderr} 531 532 533Check If BMC is Up 534 [Documentation] Wait for Host to be online. Checks every X seconds 535 ... interval for Y minutes and fails if timed out. 536 ... Default MAX timedout is 10 min, interval 10 seconds. 537 [Arguments] ${max_timeout}=${OPENBMC_REBOOT_TIMEOUT} min 538 ... ${interval}=10 sec 539 540 # Description of argument(s): 541 # max_timeout Maximum time to wait. 542 # This should be expressed in Robot Framework's time format 543 # (e.g. "10 minutes"). 544 # interfal Interval to wait between status checks. 545 # This should be expressed in Robot Framework's time format 546 # (e.g. "5 seconds"). 547 548 Wait Until Keyword Succeeds 549 ... ${max_timeout} ${interval} Verify Ping and REST Authentication 550 551 552Check If warmReset is Initiated 553 [Documentation] Ping would be still alive, so try SSH to connect 554 ... if fails the ports are down indicating reboot 555 ... is in progress 556 557 # Warm reset adds 3 seconds delay before forcing reboot 558 # To minimize race conditions, we wait for 7 seconds 559 Sleep 7s 560 ${alive}= Run Keyword and Return Status 561 ... Open Connection And Log In 562 Return From Keyword If '${alive}' == '${False}' ${False} 563 [Return] ${True} 564 565 566Flush REST Sessions 567 [Documentation] Removes all the active session objects 568 Delete All Sessions 569 570 571Initialize DBUS cmd 572 [Documentation] Initialize dbus string with property string to extract 573 [Arguments] ${boot_property} 574 575 # Description of argument(s): 576 # boot_property Property string. 577 578 ${cmd}= Catenate ${dbuscmdBase} ${dbuscmdGet} ${dbuscmdString} 579 ${cmd}= Catenate ${cmd}${boot_property} 580 Set Global Variable ${dbuscmd} ${cmd} 581 582 583Create OS Console File Path 584 [Documentation] Create OS console file path name and return it. 585 [Arguments] ${log_file_path}=${EMPTY} 586 587 # Description of arguments: 588 # file_path The caller's candidate value. If this value is ${EMPTY}, this 589 # keyword will compose a file path name. Otherwise, this 590 # keyword will use the caller's file_path value. In either 591 # case, the value will be returned. 592 593 ${status}= Run Keyword And Return Status Variable Should Exist ${TEST_NAME} 594 595 ${default_file_path}= Set Variable If ${status} == ${TRUE} 596 ... /tmp/${OPENBMC_HOST}_${TEST_NAME.replace(' ', '')}_os_console.txt 597 ... /tmp/${OPENBMC_HOST}_os_console.txt 598 599 ${log_file_path}= Set Variable If '${log_file_path}' == '${EMPTY}' 600 ... ${default_file_path} ${log_file_path} 601 602 [Return] ${log_file_path} 603 604 605Create OS Console Command String 606 [Documentation] Return a command string to start OS console logging. 607 608 # First make sure that the ssh_pw program is available. 609 ${cmd}= Catenate which ssh_pw 2>/dev/null || find 610 ... ${EXECDIR} -name 'ssh_pw' 611 612 Rdpissuing ${cmd} 613 ${rc} ${output}= Run And Return Rc And Output ${cmd} 614 Rdpvars rc output 615 616 Should Be Equal As Integers 0 ${rc} msg=Could not find ssh_pw. 617 618 ${ssh_pw_file_path}= Set Variable ${output} 619 620 ${cmd}= Catenate ${ssh_pw_file_path} ${OPENBMC_PASSWORD} -p 2200 621 ... -o "StrictHostKeyChecking no" ${OPENBMC_USERNAME}@${OPENBMC_HOST} 622 623 [Return] ${cmd} 624 625 626Get SOL Console Pid 627 [Documentation] Get the pid of the active SOL console job. 628 [Arguments] ${expect_running}=${0} 629 630 # Description of argument(s): 631 # expect_running If set and if no SOL console job is found, print debug 632 # info and fail. 633 634 # Find the pid of the active system console logging session (if any). 635 ${search_string}= Create OS Console Command String 636 # At least in some cases, ps output does not show double quotes so we must 637 # replace them in our search string with the regexes to indicate that they 638 # are optional. 639 ${search_string}= Replace String ${search_string} " ["]? 640 ${ps_cmd}= Catenate ps axwwo user,pid,cmd 641 ${cmd_buf}= Catenate echo $(${ps_cmd} | egrep '${search_string}' | 642 ... egrep -v grep | cut -c10-14) 643 Rdpissuing ${cmd_buf} 644 ${rc} ${os_con_pid}= Run And Return Rc And Output ${cmd_buf} 645 Rdpvars os_con_pid 646 # If rc is not zero it just means that there is no OS Console process 647 # running. 648 649 Return From Keyword If '${os_con_pid}' != '${EMPTY}' ${os_con_pid} 650 Return From Keyword If '${expect_running}' == '${0}' ${os_con_pid} 651 652 Cmd Fnc cat ${log_file_path} ; echo ; ${ps_cmd} quiet=${0} 653 ... print_output=${1} show_err=${1} 654 655 Should Not Be Empty ${os_con_pid} 656 657 658Stop SOL Console Logging 659 [Documentation] Stop system console logging and return log output. 660 [Arguments] ${log_file_path}=${EMPTY} 661 ... ${targ_file_path}=${EXECDIR}${/}logs${/} 662 ... ${return_data}=${1} 663 664 # If there are muliple system console processes, they will all be stopped. 665 # If there is no existing log file this keyword will return an error 666 # message to that effect (and write that message to targ_file_path, if 667 # specified). 668 # NOTE: This keyword will not fail if there is no running system console 669 # process. 670 671 # Description of arguments: 672 # log_file_path The file path that was used to call "Start SOL 673 # Console Logging". See that keyword (above) for details. 674 # targ_file_path If specified, the file path to which the source 675 # file path (i.e. "log_file_path") should be copied. 676 # return_data If this is set to ${1}, this keyword will return the SOL 677 # data to the caller as a unicode string. 678 679 ${log_file_path}= Create OS Console File Path ${log_file_path} 680 681 ${os_con_pid}= Get SOL Console Pid 682 683 ${cmd_buf}= Catenate kill -9 ${os_con_pid} 684 Run Keyword If '${os_con_pid}' != '${EMPTY}' Rdpissuing ${cmd_buf} 685 ${rc} ${output}= Run Keyword If '${os_con_pid}' != '${EMPTY}' 686 ... Run And Return Rc And Output ${cmd_buf} 687 Run Keyword If '${os_con_pid}' != '${EMPTY}' Rdpvars rc output 688 689 Run Keyword If '${targ_file_path}' != '${EMPTY}' 690 ... Run Keyword And Ignore Error 691 ... Copy File ${log_file_path} ${targ_file_path} 692 693 ${output}= Set Variable ${EMPTY} 694 ${loc_quiet}= Evaluate ${debug}^1 695 ${rc} ${output}= Run Keyword If '${return_data}' == '${1}' 696 ... Cmd Fnc cat ${log_file_path} 2>/dev/null quiet=${loc_quiet} 697 ... print_output=${0} show_err=${0} 698 699 [Return] ${output} 700 701 702Start SOL Console Logging 703 [Documentation] Start system console log to file. 704 [Arguments] ${log_file_path}=${EMPTY} ${return_data}=${1} 705 706 # This keyword will first call "Stop SOL Console Logging". Only then will 707 # it start SOL console logging. The data returned by "Stop SOL Console 708 # Logging" will in turn be returned by this keyword. 709 710 # Description of arguments: 711 # log_file_path The file path to which system console log data should be 712 # written. Note that this path is taken to be a location 713 # on the machine where this program is running rather than 714 # on the Open BMC system. 715 # return_data If this is set to ${1}, this keyword will return any SOL 716 # data to the caller as a unicode string. 717 718 ${log_file_path}= Create OS Console File Path ${log_file_path} 719 720 ${log_output}= Stop SOL Console Logging ${log_file_path} 721 ... return_data=${return_data} 722 723 # Validate by making sure we can create the file. Problems creating the 724 # file would not be noticed by the subsequent ssh command because we fork 725 # the command. 726 Create File ${log_file_path} 727 ${sub_cmd_buf}= Create OS Console Command String 728 # Routing stderr to stdout so that any startup error text will go to the 729 # output file. 730 # TODO: Doesn't work with tox so reverting temporarily. 731 # nohup detaches the process completely from our pty. 732 #${cmd_buf}= Catenate nohup ${sub_cmd_buf} &> ${log_file_path} & 733 ${cmd_buf}= Catenate ${sub_cmd_buf} > ${log_file_path} 2>&1 & 734 Rdpissuing ${cmd_buf} 735 ${rc} ${output}= Run And Return Rc And Output ${cmd_buf} 736 # Because we are forking this command, we essentially will never get a 737 # non-zero return code or any output. 738 Should Be Equal ${rc} ${0} 739 740 Wait Until Keyword Succeeds 10 seconds 0 seconds 741 ... Get SOL Console Pid ${1} 742 743 [Return] ${log_output} 744 745 746Get Time Stamp 747 [Documentation] Get the current time stamp data 748 ${cur_time}= Get Current Date result_format=%Y%m%d%H%M%S%f 749 [Return] ${cur_time} 750 751 752Start Journal Log 753 [Documentation] Start capturing journal log to a file in /tmp using 754 ... journalctl command. By default journal log is collected 755 ... at /tmp/journal_log else user input location. 756 ... The File is appended with datetime. 757 [Arguments] ${file_path}=/tmp/journal_log 758 759 # Description of arguments: 760 # file_path The file path of the journal file. 761 762 ${cur_time}= Get Time Stamp 763 Set Global Variable ${LOG_TIME} ${cur_time} 764 Start Command 765 ... journalctl -f > ${file_path}-${LOG_TIME} 766 Log Journal Log Started: ${file_path}-${LOG_TIME} 767 768 769Stop Journal Log 770 [Documentation] Stop journalctl process if its running. 771 ... By default return log from /tmp/journal_log else 772 ... user input location. 773 [Arguments] ${file_path}=/tmp/journal_log 774 775 # Description of arguments: 776 # file_path The file path of the journal file. 777 778 Open Connection And Log In 779 780 ${rc}= 781 ... Execute Command 782 ... ps ax | grep journalctl | grep -v grep 783 ... return_stdout=False return_rc=True 784 785 Return From Keyword If '${rc}' == '${1}' 786 ... No journal log process running 787 788 ${output} ${stderr}= 789 ... Execute Command killall journalctl 790 ... return_stderr=True 791 Should Be Empty ${stderr} 792 793 ${journal_log} ${stderr}= 794 ... Execute Command 795 ... cat ${file_path}-${LOG_TIME} 796 ... return_stderr=True 797 Should Be Empty ${stderr} 798 799 Log ${journal_log} 800 801 Execute Command rm ${file_path}-${LOG_TIME} 802 803 [Return] ${journal_log} 804 805 806Mac Address To Hex String 807 [Documentation] Converts MAC address into hex format. 808 ... Example 809 ... Given the following MAC: 00:01:6C:80:02:78 810 ... This keyword will return: 0x00 0x01 0x6C 0x80 0x02 0x78 811 ... Description of arguments: 812 ... i_macaddress MAC address in the following format 813 ... 00:01:6C:80:02:78 814 [Arguments] ${i_macaddress} 815 816 # Description of arguments: 817 # i_macaddress The MAC address. 818 819 ${mac_hex}= Catenate 0x${i_macaddress.replace(':', ' 0x')} 820 [Return] ${mac_hex} 821 822 823IP Address To Hex String 824 [Documentation] Converts IP address into hex format. 825 ... Example: 826 ... Given the following IP: 10.3.164.100 827 ... This keyword will return: 0xa 0x3 0xa4 0xa0 828 [Arguments] ${i_ipaddress} 829 830 # Description of arguments: 831 # i_macaddress The IP address in the format 10.10.10.10. 832 833 @{ip}= Split String ${i_ipaddress} . 834 ${index}= Set Variable ${0} 835 836 :FOR ${item} IN @{ip} 837 \ ${hex}= Convert To Hex ${item} prefix=0x lowercase=yes 838 \ Set List Value ${ip} ${index} ${hex} 839 \ ${index}= Set Variable ${index + 1} 840 ${ip_hex}= Catenate @{ip} 841 842 [Return] ${ip_hex} 843 844 845BMC CPU Performance Check 846 [Documentation] Minimal 10% of proc should be free in this instance 847 848 ${bmc_cpu_usage_output} ${stderr} ${rc}= BMC Execute Command 849 ... ${bmc_cpu_usage_cmd} 850 ${bmc_cpu_usage_output} ${stderr} ${rc}= BMC Execute Command 851 ... ${bmc_cpu_usage_cmd} 852 ${bmc_cpu_percentage}= Fetch From Left ${bmc_cpu_usage_output} % 853 Should be true ${bmc_cpu_percentage} < 90 854 855 856BMC Mem Performance Check 857 [Documentation] Minimal 10% of memory should be free in this instance 858 859 ${bmc_mem_free_output} ${stderr} ${rc}= BMC Execute Command 860 ... ${bmc_mem_free_cmd} 861 862 ${bmc_mem_total_output} ${stderr} ${rc}= BMC Execute Command 863 ... ${bmc_mem_total_cmd} 864 ${bmc_mem_free_output} ${stderr} ${rc}= BMC Execute Command 865 ... ${bmc_mem_free_cmd} 866 867 ${bmc_mem_total_output} ${stderr} ${rc}= BMC Execute Command 868 ... ${bmc_mem_total_cmd} 869 870 ${bmc_mem_percentage}= Evaluate ${bmc_mem_free_output}*100 871 ${bmc_mem_percentage}= Evaluate 872 ... ${bmc_mem_percentage}/${bmc_mem_total_output} 873 Should be true ${bmc_mem_percentage} > 10 874 875 876BMC File System Usage Check 877 [Documentation] Check the file system space. 4 file system should be 878 ... 100% full which is expected 879 # Filesystem Size Used Available Use% Mounted on 880 # /dev/root 14.4M 14.4M 0 100% / 881 # /dev/ubiblock0_0 14.4M 14.4M 0 100% /media/rofs-c9249b0e 882 # /dev/ubiblock8_0 19.6M 19.6M 0 100% /media/pnor-ro-8764baa3 883 # /dev/ubiblock4_0 14.4M 14.4M 0 100% /media/rofs-407816c 884 # /dev/ubiblock8_4 21.1M 21.1M 0 100% /media/pnor-ro-cecc64c4 885 ${bmc_fs_usage_output} ${stderr} ${rc}= BMC Execute Command 886 ... ${bmc_file_system_usage_cmd} 887 ${bmc_pnor_fs_usage_output} ${stderr} ${rc}= BMC Execute Command 888 ... ${total_pnor_ro_file_system_cmd} 889 ${bmc_bmc_fs_usage_output} ${stderr} ${rc}= BMC Execute Command 890 ... ${total_bmc_ro_file_system_cmd} 891 ${total_bmc_pnor_image}= Evaluate 892 ... ${bmc_pnor_fs_usage_output}+${bmc_bmc_fs_usage_output} 893 # Considering /dev/root also in total 100% used file system 894 ${total_full_fs}= Evaluate ${total_bmc_pnor_image}+1 895 Should Be True ${bmc_fs_usage_output}==${total_full_fs} 896 897 898Check BMC CPU Performance 899 [Documentation] Minimal 10% of proc should be free in 3 sample 900 :FOR ${var} IN Range 1 4 901 \ BMC CPU Performance check 902 903 904Check BMC Mem Performance 905 [Documentation] Minimal 10% of memory should be free 906 907 :FOR ${var} IN Range 1 4 908 \ BMC Mem Performance check 909 910 911Check BMC File System Performance 912 [Documentation] Check for file system usage for 4 times 913 914 :FOR ${var} IN Range 1 4 915 \ BMC File System Usage check 916 917 918Get URL List 919 [Documentation] Return list of URLs under given URL. 920 [Arguments] ${openbmc_url} 921 922 # Description of argument(s): 923 # openbmc_url URL for list operation (e.g. 924 # /xyz/openbmc_project/inventory). 925 926 ${url_list}= Read Properties ${openbmc_url}/list quiet=${1} 927 Sort List ${url_list} 928 929 [Return] ${url_list} 930 931 932Get Endpoint Paths 933 [Documentation] Returns all url paths ending with given endpoint 934 ... Example: 935 ... Given the following endpoint: cpu 936 ... This keyword will return: list of all urls ending with 937 ... cpu - 938 ... /org/openbmc/inventory/system/chassis/motherboard/cpu0, 939 ... /org/openbmc/inventory/system/chassis/motherboard/cpu1 940 [Arguments] ${path} ${endpoint} 941 942 # Description of arguments: 943 # path URL path for enumeration. 944 # endpoint Endpoint string (url path ending). 945 946 ${resp}= Read Properties ${path}/enumerate timeout=30 947 Log Dictionary ${resp} 948 949 ${list}= Get Dictionary Keys ${resp} 950 # For a given string, look for prefix and suffix for matching expression. 951 # Start of string followed by zero or more of any character followed by 952 # any digit or lower case character. 953 ${resp}= Get Matches ${list} regexp=^.*[0-9a-z_].${endpoint}[0-9a-z]*$ 954 955 [Return] ${resp} 956 957 958Check Zombie Process 959 [Documentation] Check if any defunct process exist or not on BMC 960 ${count} ${stderr} ${rc}= Execute Command ps -o stat | grep Z | wc -l 961 ... return_stderr=True return_rc=True 962 Should Be True ${count}==0 963 Should Be Empty ${stderr} 964 965 966Prune Journal Log 967 [Documentation] Prune archived journal logs. 968 [Arguments] ${vacuum_size}=1M 969 970 # This keyword can be used to prevent the journal 971 # log from filling up the /run filesystem. 972 # This command will retain only the latest logs 973 # of the user specified size. 974 975 # Description of argument(s): 976 # vacuum_size Size of journal. 977 978 Open Connection And Log In 979 ${output} ${stderr} ${rc}= 980 ... Execute Command 981 ... journalctl --vacuum-size=${vacuum_size} 982 ... return_stderr=True return_rc=True 983 984 Should Be Equal ${rc} ${0} msg=${stderr} 985 986 987Set BMC Power Policy 988 [Documentation] Set the given BMC power policy. 989 [Arguments] ${policy} 990 991 # Note that this function will translate the old style "RESTORE_LAST_STATE" 992 # policy to the new style "xyz.openbmc_project.Control.Power.RestorePolicy. 993 # Policy.Restore" for you. 994 995 # Description of argument(s): 996 # policy Power restore policy (e.g "RESTORE_LAST_STATE", 997 # ${RESTORE_LAST_STATE}). 998 999 # Set the bmc_power_policy_method to either 'Old' or 'New'. 1000 Set Power Policy Method 1001 # This translation helps bridge between old and new method for calling. 1002 ${policy}= Translate Power Policy Value ${policy} 1003 # Run the appropriate keyword. 1004 Run Key ${bmc_power_policy_method} Set Power Policy \ ${policy} 1005 ${currentPolicy}= Get System Power Policy 1006 Should Be Equal ${currentPolicy} ${policy} 1007 1008 1009New Set Power Policy 1010 [Documentation] Set the given BMC power policy (new method). 1011 [Arguments] ${policy} 1012 1013 # Description of argument(s): 1014 # policy Power restore policy (e.g. ${ALWAYS_POWER_OFF}). 1015 1016 ${valueDict}= Create Dictionary data=${policy} 1017 Write Attribute 1018 ... ${POWER_RESTORE_URI} PowerRestorePolicy data=${valueDict} 1019 1020 1021Old Set Power Policy 1022 [Documentation] Set the given BMC power policy (old method). 1023 [Arguments] ${policy} 1024 1025 # Description of argument(s): 1026 # policy Power restore policy (e.g. "ALWAYS_POWER_OFF"). 1027 1028 ${valueDict}= create dictionary data=${policy} 1029 Write Attribute ${HOST_SETTING} power_policy data=${valueDict} 1030 1031 1032Get System Power Policy 1033 [Documentation] Returns the BMC power policy. 1034 1035 # Set the bmc_power_policy_method to either 'Old' or 'New'. 1036 Set Power Policy Method 1037 ${cmd_buf}= Create List ${bmc_power_policy_method} Get Power Policy 1038 # Run the appropriate keyword. 1039 ${currentPolicy}= Run Keyword @{cmd_buf} 1040 1041 [Return] ${currentPolicy} 1042 1043 1044New Get Power Policy 1045 [Documentation] Returns the BMC power policy (new method). 1046 ${currentPolicy}= Read Attribute ${POWER_RESTORE_URI} PowerRestorePolicy 1047 1048 [Return] ${currentPolicy} 1049 1050 1051Old Get Power Policy 1052 [Documentation] Returns the BMC power policy (old method). 1053 ${currentPolicy}= Read Attribute ${HOST_SETTING} power_policy 1054 1055 [Return] ${currentPolicy} 1056 1057 1058Get Auto Reboot 1059 [Documentation] Returns auto reboot setting. 1060 ${setting}= Read Attribute ${CONTROL_HOST_URI}/auto_reboot AutoReboot 1061 1062 [Return] ${setting} 1063 1064 1065Set Auto Reboot 1066 [Documentation] Set the given auto reboot setting. 1067 [Arguments] ${setting} 1068 1069 # Description of argument(s): 1070 # setting The reboot setting, 1 for enabling and 0 for disabling. 1071 1072 ${valueDict}= Set Variable ${setting} 1073 ${data}= Create Dictionary data=${valueDict} 1074 Write Attribute ${CONTROL_HOST_URI}/auto_reboot AutoReboot data=${data} 1075 ${current_setting}= Get Auto Reboot 1076 Should Be Equal As Integers ${current_setting} ${setting} 1077 1078 1079Set BMC Reset Reference Time 1080 [Documentation] Set current boot time as a reference and increment 1081 ... boot count. 1082 1083 ${cur_btime}= Get BMC Boot Time 1084 Run Keyword If ${BOOT_TIME} == ${0} and ${BOOT_COUNT} == ${0} 1085 ... Set Global Variable ${BOOT_TIME} ${cur_btime} 1086 ... ELSE IF ${cur_btime} > ${BOOT_TIME} 1087 ... Run Keywords Set Global Variable ${BOOT_TIME} ${cur_btime} 1088 ... AND 1089 ... Set Global Variable ${BOOT_COUNT} ${BOOT_COUNT + 1} 1090 1091 1092Get BMC Boot Time 1093 [Documentation] Returns boot time from /proc/stat. 1094 1095 Open Connection And Log In 1096 ${output} ${stderr}= 1097 ... Execute Command egrep '^btime ' /proc/stat | cut -f 2 -d ' ' 1098 ... return_stderr=True 1099 Should Be Empty ${stderr} 1100 ${btime}= Convert To Integer ${output} 1101 [Return] ${btime} 1102 1103 1104Enable Core Dump On BMC 1105 [Documentation] Enable core dump collection. 1106 ${core_pattern} ${stderr} ${rc}= BMC Execute Command 1107 ... echo '/tmp/core_%e.%p' | tee /proc/sys/kernel/core_pattern 1108 Should Be Equal As Strings ${core_pattern} /tmp/core_%e.%p 1109 1110 1111Get Number Of BMC Core Dump Files 1112 [Documentation] Returns number of core dump files on BMC. 1113 Open Connection And Log In 1114 ${num_of_core_dump}= Execute Command 1115 ... ls /tmp/core* 2>/dev/null | wc -l 1116 [Return] ${num_of_core_dump} 1117 1118 1119Set Core Dump File Size Unlimited 1120 [Documentation] Set core dump file size to unlimited. 1121 BMC Execute Command ulimit -c unlimited 1122 1123 1124Check For Core Dumps 1125 [Documentation] Check for any core dumps exist. 1126 ${output}= Get Number Of BMC Core Dump Files 1127 Run Keyword If ${output} > 0 1128 ... Log **Warning** BMC core dump files exist level=WARN 1129 1130 1131Trigger Host Watchdog Error 1132 [Documentation] Inject host watchdog timeout error via REST. 1133 [Arguments] ${milliseconds}=1000 ${sleep_time}=5s 1134 1135 # Description of argument(s): 1136 # milliseconds The time watchdog timer value in milliseconds (e.g. 1000 = 1137 # 1 second). 1138 # sleep_time Time delay for host watchdog error to get injected. 1139 # Default is 5 seconds. 1140 1141 ${data}= Create Dictionary 1142 ... data=xyz.openbmc_project.State.Watchdog.Action.PowerCycle 1143 ${status} ${result}= Run Keyword And Ignore Error 1144 ... Read Attribute ${HOST_WATCHDOG_URI} ExpireAction 1145 Run Keyword If '${status}' == 'PASS' 1146 ... Write Attribute ${HOST_WATCHDOG_URI} ExpireAction data=${data} 1147 1148 ${data}= Create Dictionary data=${milliseconds} 1149 Write Attribute ${HOST_WATCHDOG_URI} Interval data=${data} 1150 1151 ${data}= Create Dictionary data=${True} 1152 Write Attribute ${HOST_WATCHDOG_URI} Enabled data=${data} 1153 1154 Sleep ${sleep_time} 1155 1156 1157Login To OS Host 1158 [Documentation] Login to OS Host and return the Login response code. 1159 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 1160 ... ${os_password}=${OS_PASSWORD} 1161 1162 # Description of arguments: 1163 # ${os_host} IP address of the OS Host. 1164 # ${os_username} OS Host Login user name. 1165 # ${os_password} OS Host Login passwrd. 1166 1167 ${os_state}= Run Keyword And Return Status Ping Host ${os_host} 1168 Run Keyword If '${os_state}' == 'False' 1169 ... Run Keywords Initiate Host Reboot AND 1170 ... Is Host Running AND 1171 ... Wait for OS ${os_host} ${os_username} ${os_password} 1172 1173 SSHLibrary.Open Connection ${os_host} 1174 ${resp}= Login ${os_username} ${os_password} 1175 [Return] ${resp} 1176 1177 1178Configure Initial Settings 1179 [Documentation] Restore old IP and route. 1180 ... This keyword requires initial settings viz IP address, 1181 ... Network Mask, default gatway and serial console IP and port 1182 ... information which should be provided in command line. 1183 1184 [Arguments] ${host}=${OPENBMC_HOST} ${mask}=${NET_MASK} 1185 ... ${gw_ip}=${GW_IP} 1186 1187 # Description of arguments: 1188 # host IP address of the OS Host. 1189 # mask Network mask. 1190 # gu_ip Gateway IP address or hostname. 1191 1192 # Open telnet connection and ignore the error, in case telnet session is 1193 # already opened by the program calling this keyword. 1194 Run Keyword And Ignore Error Open Telnet Connection to BMC Serial Console 1195 Telnet.write ifconfig eth0 ${host} netmask ${mask} 1196 Telnet.write route add default gw ${gw_ip} 1197 1198 1199Install Debug Tarball On BMC 1200 [Documentation] Copy the debug tar file to BMC and install. 1201 [Arguments] ${tarball_file_path}=${default_tarball} 1202 ... ${targ_tarball_dir_path}=/tmp/tarball/ 1203 1204 # Description of arguments: 1205 # tarball_file_path Path of the debug tarball file. 1206 # The tar file is downloaded from the build page 1207 # https://openpower.xyz/job/openbmc-build/ 1208 # obmc-phosphor-debug-tarball-witherspoon.tar.xz 1209 # 1210 # targ_tarball_dir_path The directory path where the tarball is to be 1211 # installed. 1212 1213 OperatingSystem.File Should Exist ${tarball_file_path} 1214 ... msg=${tarball_file_path} doesn't exist. 1215 1216 # Upload the file to BMC. 1217 Import Library SCPLibrary WITH NAME scp 1218 Open Connection for SCP 1219 scp.Put File ${tarball_file_path} /tmp/debug-tarball.tar.xz 1220 1221 # Create tarball directory and install. 1222 BMC Execute Command mkdir -p ${targ_tarball_dir_path} 1223 BMC Execute Command 1224 ... tar -xf /tmp/debug-tarball.tar.xz -C ${targ_tarball_dir_path} 1225 1226 # Remove the tarball file from BMC. 1227 BMC Execute Command rm -f /tmp/debug-tarball.tar.xz 1228 1229 1230Get BMC Boot Count 1231 [Documentation] Returns BMC boot count based on boot time. 1232 ${cur_btime}= Get BMC Boot Time 1233 1234 # Set global variable BOOT_TIME to current boot time if current boot time 1235 # is changed. Also increase value of global variable BOOT_COUNT by 1. 1236 Run Keyword If ${cur_btime} > ${BOOT_TIME} 1237 ... Run Keywords Set Global Variable ${BOOT_TIME} ${cur_btime} 1238 ... AND 1239 ... Set Global Variable ${BOOT_COUNT} ${BOOT_COUNT + 1} 1240 1241 [Return] ${BOOT_COUNT} 1242 1243 1244Set BMC Boot Count 1245 [Documentation] Set BMC boot count to given value. 1246 [Arguments] ${count} 1247 1248 # Description of arguments: 1249 # count boot count value. 1250 ${cur_btime}= Get BMC Boot Time 1251 1252 # Set global variable BOOT_COUNT to given value. 1253 Set Global Variable ${BOOT_COUNT} ${count} 1254 1255 # Set BOOT_TIME variable to current boot time. 1256 Set Global Variable ${BOOT_COUNT} ${count} 1257 1258 1259Get System LED State 1260 [Documentation] Return the state of given system LED. 1261 [Arguments] ${led_name} 1262 1263 # Description of argument(s): 1264 # led_name System LED name (e.g. heartbeat, identify, beep). 1265 1266 ${state}= Read Attribute ${LED_PHYSICAL_URI}${led_name} State 1267 [Return] ${state.rsplit('.', 1)[1]} 1268 1269 1270Set System LED State 1271 [Documentation] Set given system LED via REST. 1272 [Arguments] ${led_name} ${led_state} 1273 # Description of argument(s): 1274 # led_name System LED name (e.g. heartbeat, identify, beep). 1275 # led_state LED state to be set (e.g. On, Off). 1276 1277 ${args}= Create Dictionary 1278 ... data=xyz.openbmc_project.Led.Physical.Action.${led_state} 1279 Write Attribute ${LED_PHYSICAL_URI}${led_name} State data=${args} 1280 1281 Verify LED State ${led_name} ${led_state} 1282 1283 1284Verify LED State 1285 [Documentation] Checks if LED is in given state. 1286 [Arguments] ${led_name} ${led_state} 1287 # Description of argument(s): 1288 # led_name System LED name (e.g. heartbeat, identify, beep). 1289 # led_state LED state to be verified (e.g. On, Off). 1290 1291 ${state}= Get System LED State ${led_name} 1292 Should Be Equal ${state} ${led_state} 1293 1294 1295Get LED State XYZ 1296 [Documentation] Returns state of given LED. 1297 [Arguments] ${led_name} 1298 1299 # Description of argument(s): 1300 # led_name Name of LED. 1301 1302 ${state}= Read Attribute ${LED_GROUPS_URI}${led_name} Asserted 1303 # Returns the state of the LED, either On or Off. 1304 [Return] ${state} 1305 1306 1307Delete Error Logs 1308 [Documentation] Delete error logs. 1309 1310 # Check if error logs entries exist, if not return. 1311 ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${/}list quiet=${1} 1312 Return From Keyword If ${resp.status_code} == ${HTTP_NOT_FOUND} 1313 1314 # Get the list of error logs entries and delete them all. 1315 ${elog_entries}= Get URL List ${BMC_LOGGING_ENTRY} 1316 :FOR ${entry} IN @{elog_entries} 1317 \ Delete Error Log Entry ${entry} 1318 1319 1320Delete Error Log Entry 1321 [Documentation] Delete error log entry. 1322 [Arguments] ${entry_path} 1323 1324 # Description of argument(s): 1325 # entry_path Delete an error log entry. 1326 # Ex. /xyz/openbmc_project/logging/entry/1 1327 1328 # Skip delete if entry URI is a callout. 1329 # Example: /xyz/openbmc_project/logging/entry/1/callout 1330 Return From Keyword If '${entry_path.rsplit('/', 1)[1]}' == 'callout' 1331 1332 ${data}= Create Dictionary data=@{EMPTY} 1333 ${resp}= Openbmc Delete Request ${entry_path} data=${data} 1334 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 1335 1336 1337Delete All Error Logs 1338 [Documentation] Delete all error log entries using "DeleteAll" interface. 1339 1340 ${data}= Create Dictionary data=@{EMPTY} 1341 ${resp}= Openbmc Post Request ${BMC_LOGGING_URI}action/DeleteAll 1342 ... data=${data} 1343 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 1344 1345 1346Get BMC Version 1347 [Documentation] Returns BMC version from /etc/os-release. 1348 ... e.g. "v1.99.6-141-ge662190" 1349 1350 ${cmd}= Set Variable grep ^VERSION_ID= /etc/os-release | cut -f 2 -d '=' 1351 ${output} ${stderr} ${rc}= BMC Execute Command ${cmd} 1352 [Return] ${output} 1353 1354 1355Get PNOR Version 1356 [Documentation] Returns the PNOR version from the BMC. 1357 1358 ${pnor_attrs}= Get PNOR Attributes 1359 [Return] ${pnor_attrs['version']} 1360 1361 1362Get PNOR Attributes 1363 [Documentation] Return PNOR software attributes as a dictionary. 1364 1365 # This keyword parses /var/lib/phosphor-software-manager/pnor/ro/pnor.toc 1366 # into key/value pairs. 1367 1368 ${outbuf} ${stderr} ${rc}= BMC Execute Command 1369 ... cat /var/lib/phosphor-software-manager/pnor/ro/pnor.toc 1370 ${pnor_attrs}= Key Value Outbuf To Dict ${outbuf} delim== 1371 1372 [Return] ${pnor_attrs} 1373 1374 1375Get Elog URL List 1376 [Documentation] Return error log entry list of URLs. 1377 1378 ${url_list}= Read Properties /xyz/openbmc_project/logging/entry/ 1379 Sort List ${url_list} 1380 [Return] ${url_list} 1381 1382 1383Read Turbo Setting Via REST 1384 [Documentation] Return turbo setting via REST. 1385 # Returns 1 if TurboAllowed, 0 if not. 1386 1387 ${turbo_setting}= Read Attribute 1388 ... ${CONTROL_HOST_URI}turbo_allowed TurboAllowed 1389 [Return] ${turbo_setting} 1390 1391 1392Set Turbo Setting Via REST 1393 [Documentation] Set turbo setting via REST. 1394 [Arguments] ${setting} ${verify}=${False} 1395 1396 # Description of argument(s): 1397 # setting State to set TurboAllowed, 1=allowed, 0=not allowed. 1398 # verify If True, read the TurboAllowed setting to confirm. 1399 1400 ${data}= Create Dictionary data=${${setting}} 1401 Write Attribute ${CONTROL_HOST_URI}turbo_allowed TurboAllowed 1402 ... verify=${verify} data=${data} 1403 1404 1405Set Control Boot Mode 1406 [Documentation] Set given boot mode on the boot object path attribute. 1407 [Arguments] ${boot_path} ${boot_mode} 1408 1409 # Description of argument(s): 1410 # boot_path Boot object path. 1411 # Example: 1412 # /xyz/openbmc_project/control/host0/boot 1413 # /xyz/openbmc_project/control/host0/boot/one_time 1414 # boot_mode Boot mode which need to be set. 1415 # Example: 1416 # "xyz.openbmc_project.Control.Boot.Mode.Modes.Regular" 1417 1418 ${valueDict}= Create Dictionary data=${boot_mode} 1419 Write Attribute ${boot_path} BootMode data=${valueDict} 1420 1421 1422Copy Address Translation Utils To HOST OS 1423 [Documentation] Copy address translation utils to host OS. 1424 1425 OperatingSystem.File Should Exist ${probe_cpu_tool_path} 1426 ... msg=${probe_cpu_tool_path} doesn't exist. 1427 OperatingSystem.File Should Exist ${probe_cpu_tool_path} 1428 ... msg=${probe_cpu_tool_path} doesn't exist. 1429 1430 scp.Open connection ${OS_HOST} username=${OS_USERNAME} 1431 ... password=${OS_PASSWORD} 1432 scp.Put File ${probe_cpu_tool_path} ${target_file_path} 1433 scp.Put File ${scom_addrs_tool_path} ${target_file_path} 1434 1435 1436Verify BMC RTC And UTC Time Drift 1437 [Documentation] Verify that the RTC and UTC time difference is less than 1438 ... the given time_drift_max. 1439 [Arguments] ${time_diff_max}=${10} 1440 1441 # Description of argument(s): 1442 # time_diff_max The max allowable RTC and UTC time difference in seconds. 1443 1444 # Example: 1445 # time_dict: 1446 # [local_time]: Fri 2017-11-03 152756 UTC 1447 # [local_time_seconds]: 1509740876 1448 # [universal_time]: Fri 2017-11-03 152756 UTC 1449 # [universal_time_seconds]: 1509740876 1450 # [rtc_time]: Fri 2016-05-20 163403 1451 # [rtc_time_seconds]: 1463780043 1452 # [time_zone]: n/a (UTC, +0000) 1453 # [network_time_on]: yes 1454 # [ntp_synchronized]: no 1455 # [rtc_in_local_tz]: no 1456 1457 ${time}= Get BMC Date Time 1458 ${time_diff}= Evaluate 1459 ... ${time['universal_time_seconds']} - ${time['rtc_time_seconds']} 1460 Should Be True ${time_diff} < ${time_diff_max} 1461 1462 1463Watchdog Object Should Exist 1464 [Documentation] Check that watchdog object exists. 1465 1466 ${resp}= OpenBMC Get Request ${WATCHDOG_URI}host0 1467 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 1468 ... msg=Expected watchdog object does not exist. 1469 1470 1471Validate IP On BMC 1472 [Documentation] Validate IP address is present in set of IP addresses. 1473 [Arguments] ${ip_address} ${ip_data} 1474 1475 # Description of argument(s): 1476 # ip_address IP address to check (e.g. xx.xx.xx.xx). 1477 # ip_data Set of the IP addresses present. 1478 1479 Should Contain Match ${ip_data} ${ip_address}/* 1480 ... msg=${ip_address} not found in the list provided. 1481 1482 1483Remove Journald Logs 1484 [Documentation] Remove all journald logs and restart service. 1485 1486 ${cmd}= Catenate systemctl stop systemd-journald.service && 1487 ... rm -rf /var/log/journal && systemctl start systemd-journald.service 1488 1489 BMC Execute Command ${cmd} 1490 1491 1492Verify Identify LED State 1493 [Documentation] Verify the identify LED state 1494 ... matches caller's expectations. 1495 [Arguments] ${expected_state} 1496 1497 # Description of argument(s): 1498 # expected_state The LED state expected by the caller ("Blink" or "Off"). 1499 1500 ${resp}= Read Attribute ${LED_PHYSICAL_URI}/front_id State 1501 Should Be Equal ${resp} 1502 ... xyz.openbmc_project.Led.Physical.Action.${expected_state} 1503 ... msg=Unexpected LED state. 1504 1505 ${resp}= Read Attribute ${LED_PHYSICAL_URI}/rear_id State 1506 Should Be Equal ${resp} 1507 ... xyz.openbmc_project.Led.Physical.Action.${expected_state} 1508 ... msg=Unexpected LED state. 1509 1510 1511Verify The Attribute 1512 [Documentation] Verify the given attribute. 1513 [Arguments] ${uri} ${attribute_name} ${attribute_value} 1514 1515 # Description of argument(s): 1516 # uri URI path 1517 # (e.g. "/xyz/openbmc_project/control/host0/TPMEnable"). 1518 # attribute_name Name of attribute to be verified (e.g. "TPMEnable"). 1519 # attribute_value The expected value of attribute (e.g. "1", "0", etc.) 1520 1521 ${output}= Read Attribute ${uri} ${attribute_name} 1522 Should Be Equal ${attribute_value} ${output} 1523 ... msg=Attribute "${attribute_name} does not have the expected value. 1524 1525 1526Get BMC Flash Chip Boot Side 1527 [Documentation] Return the BMC flash chip boot side. 1528 1529 # Example: 1530 # 0 - indicates chip select is current side. 1531 # 32 - indicates chip select is alternate side. 1532 1533 ${boot_side} ${stderr} ${rc}= BMC Execute Command 1534 ... cat /sys/class/watchdog/watchdog1/bootstatus 1535 1536 [Return] ${boot_side} 1537 1538 1539Check For Regex In Journald 1540 [Documentation] Parse the journal log and check for regex string. 1541 [Arguments] ${regex}=${ERROR_REGEX} ${error_check}=${0} 1542 1543 ${journal_log} ${stderr} ${rc}= BMC Execute Command 1544 ... journalctl --no-pager | egrep '${regex}' ignore_err=1 1545 1546 Run Keyword If ${error_check} == ${0} 1547 ... Should Be Empty ${journal_log} 1548 ... ELSE 1549 ... Should Not Be Empty ${journal_log} 1550