1*** Settings *** 2 3Documentation Utilities for Robot keywords. 4 5Resource ../lib/resource.txt 6Resource ../lib/rest_client.robot 7Resource ../lib/connection_client.robot 8Library String 9Library DateTime 10Library Process 11Library OperatingSystem 12Library gen_print.py 13Library gen_robot_print.py 14Library gen_cmd.py 15Library gen_robot_keyword.py 16Library bmc_ssh_utils.py 17Library utils.py 18Library var_funcs.py 19Library SCPLibrary WITH NAME scp 20 21*** Variables *** 22 23${pflash_cmd} /usr/sbin/pflash -r /dev/stdout -P VERSION 24${SYSTEM_SHUTDOWN_TIME} ${5} 25 26${dbuscmdBase} 27... dbus-send --system --print-reply --dest=${OPENBMC_BASE_DBUS}.settings.Host 28${dbuscmdGet} 29... ${SETTINGS_URI}host0 org.freedesktop.DBus.Properties.Get 30${dbuscmdString}= string:"xyz.openbmc_project.settings.Host" string: 31 32# Assign default value to QUIET for programs which may not define it. 33${QUIET} ${0} 34 35${bmc_mem_free_cmd}= free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f4 36${bmc_mem_total_cmd}= free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f2 37${bmc_cpu_usage_cmd}= top -n 1 | grep CPU: | cut -c 7-9 38${HOST_SETTING} ${SETTINGS_URI}host0 39 40# /run/initramfs/ro associate filesystem should be 100% full always 41${bmc_file_system_usage_cmd}= df -h | cut -c 52-54 | grep 100 | wc -l 42${total_pnor_ro_file_system_cmd}= df -h | grep /media/pnor-ro | wc -l 43${total_bmc_ro_file_system_cmd}= df -h | grep /media/rofs | wc -l 44 45${BOOT_TIME} ${0} 46${BOOT_COUNT} ${0} 47${count} ${0} 48${devicetree_base} /sys/firmware/devicetree/base/model 49 50# Initialize default debug value to 0. 51${DEBUG} ${0} 52 53${probe_cpu_tool_path} ${EXECDIR}/tools/ras/probe_cpus.sh 54${scom_addrs_tool_path} ${EXECDIR}/tools/ras/scom_addr_p9.sh 55${target_file_path} /root/ 56 57${default_tarball} ${EXECDIR}/obmc-phosphor-debug-tarball-witherspoon.tar.xz 58 59# These variables are used to straddle between new and old methods of setting 60# values. 61${boot_prog_method} ${EMPTY} 62${power_policy_setup} ${0} 63${bmc_power_policy_method} ${EMPTY} 64@{valid_power_policy_vars} RESTORE_LAST_STATE ALWAYS_POWER_ON 65... ALWAYS_POWER_OFF 66 67 68*** Keywords *** 69 70Check BMC Performance 71 [Documentation] Check BMC basic CPU Mem File system performance. 72 73 Check BMC CPU Performance 74 Check BMC Mem Performance 75 Check BMC File System Performance 76 77 78Verify PNOR Update 79 [Documentation] Verify that the PNOR is not corrupted. 80 # Example: 81 # FFS: Flash header not found. Code: 100 82 # Error 100 opening ffs ! 83 84 Open Connection And Log In 85 ${pnor_info}= Execute Command On BMC ${pflash_cmd} 86 Should Not Contain Any ${pnor_info} Flash header not found Error 87 88 89Get BMC System Model 90 [Documentation] Get the BMC model from the device tree and return it. 91 92 ${bmc_model} ${stderr} ${rc}= BMC Execute Command 93 ... cat ${devicetree_base} | cut -d " " -f 1 return_stderr=True 94 ... test_mode=0 95 Should Be Empty ${stderr} 96 Should Not Be Empty ${bmc_model} msg=BMC model is empty. 97 [Return] ${bmc_model} 98 99 100Verify BMC System Model 101 [Documentation] Verify the BMC model with ${OPENBMC_MODEL}. 102 [Arguments] ${bmc_model} 103 104 # Description of argument(s): 105 # bmc_model System model (e.g. "witherspoon"). 106 107 ${tmp_bmc_model}= Fetch From Right ${OPENBMC_MODEL} / 108 ${tmp_bmc_model}= Fetch From Left ${tmp_bmc_model} . 109 ${ret}= Run Keyword And Return Status Should Contain ${bmc_model} 110 ... ${tmp_bmc_model} ignore_case=True 111 [Return] ${ret} 112 113 114Wait For Host To Ping 115 [Arguments] ${host} ${timeout}=${OPENBMC_REBOOT_TIMEOUT}min 116 ... ${interval}=5 sec 117 118 # Description of argument(s): 119 # host The host name or IP of the host to ping. 120 # timeout The amount of time after which ping attempts cease. 121 # This should be expressed in Robot Framework's time format 122 # (e.g. "10 seconds"). 123 # interval The amount of time in between attempts to ping. 124 # This should be expressed in Robot Framework's time format 125 # (e.g. "5 seconds"). 126 127 Wait Until Keyword Succeeds ${timeout} ${interval} Ping Host ${host} 128 129 130Ping Host 131 [Arguments] ${host} 132 133 # Description of argument(s): 134 # host The host name or IP of the host to ping. 135 136 Should Not Be Empty ${host} msg=No host provided 137 ${RC} ${output}= Run and return RC and Output ping -c 4 ${host} 138 Log RC: ${RC}\nOutput:\n${output} 139 Should be equal ${RC} ${0} 140 141 142Get Boot Progress 143 [Documentation] Get the boot progress and return it. 144 [Arguments] ${quiet}=${QUIET} 145 146 # Description of argument(s): 147 # quiet Indicates whether this keyword should run without any output to 148 # the console. 149 150 Set Boot Progress Method 151 ${state}= Run Keyword If '${boot_prog_method}' == 'New' 152 ... New Get Boot Progress quiet=${quiet} 153 ... ELSE 154 ... Old Get Boot Progress quiet=${quiet} 155 156 [Return] ${state} 157 158 159Set Boot Progress Method 160 [Documentation] Set the boot_prog_method to either 'Old' or 'New'. 161 162 # The boot progress data has moved from an 'org' location to an 'xyz' 163 # location. This keyword will determine whether the new method of getting 164 # the boot progress is valid and will set the global boot_prog_method 165 # variable accordingly. If boot_prog_method is already set (either by a 166 # prior call to this function or via a -v parm), this keyword will simply 167 # return. 168 169 # Note: There are interim builds that contain boot_progress in both the 170 # old and the new location values. It is nearly impossible for this 171 # keyword to determine whether the old boot_progress or the new one is 172 # active. When using such builds where the old boot_progress is active, 173 # the only recourse users will have is that they may specify 174 # -v boot_prog_method:Old to force old behavior on such builds. 175 176 Run Keyword If '${boot_prog_method}' != '${EMPTY}' Return From Keyword 177 178 ${new_status} ${new_value}= Run Keyword And Ignore Error 179 ... New Get Boot Progress 180 # If the new style read fails, the method must necessarily be "Old". 181 Run Keyword If '${new_status}' == 'PASS' 182 ... Run Keywords 183 ... Set Global Variable ${boot_prog_method} New AND 184 ... Rqpvars boot_prog_method AND 185 ... Return From Keyword 186 187 # Default method is "Old". 188 Set Global Variable ${boot_prog_method} Old 189 Rqpvars boot_prog_method 190 191 192Old Get Boot Progress 193 [Documentation] Get the boot progress the old way (via org location). 194 [Arguments] ${quiet}=${QUIET} 195 196 # Description of argument(s): 197 # quiet Indicates whether this keyword should run without any output to 198 # the console. 199 200 ${state}= Read Attribute ${OPENBMC_BASE_URI}sensors/host/BootProgress 201 ... value quiet=${quiet} 202 203 [Return] ${state} 204 205 206New Get Boot Progress 207 [Documentation] Get the boot progress the new way (via xyz location). 208 [Arguments] ${quiet}=${QUIET} 209 210 # Description of argument(s): 211 # quiet Indicates whether this keyword should run without any output to 212 # the console. 213 214 ${state}= Read Attribute ${HOST_STATE_URI} BootProgress quiet=${quiet} 215 216 [Return] ${state.rsplit('.', 1)[1]} 217 218 219Is Power On 220 ${state}= Get Power State 221 Should be equal ${state} ${1} 222 223 224Is Power Off 225 ${state}= Get Power State 226 Should be equal ${state} ${0} 227 228 229Initiate Power On 230 [Documentation] Initiates the power on and waits until the Is Power On 231 ... keyword returns that the power state has switched to on. 232 [Arguments] ${wait}=${1} 233 234 # Description of argument(s): 235 # wait Indicates whether to wait for a powered on state after issuing 236 # the power on command. 237 238 @{arglist}= Create List 239 ${args}= Create Dictionary data=@{arglist} 240 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ powerOn 241 ... data=${args} 242 should be equal as strings ${resp.status_code} ${HTTP_OK} 243 244 # Does caller want to wait for power on status? 245 Run Keyword If '${wait}' == '${0}' Return From Keyword 246 Wait Until Keyword Succeeds 3 min 10 sec Is Power On 247 248 249Initiate Power Off 250 [Documentation] Initiates the power off and waits until the Is Power Off 251 ... keyword returns that the power state has switched to off. 252 253 @{arglist}= Create List 254 ${args}= Create Dictionary data=@{arglist} 255 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ powerOff 256 ... data=${args} 257 should be equal as strings ${resp.status_code} ${HTTP_OK} 258 Wait Until Keyword Succeeds 1 min 10 sec Is Power Off 259 260 261Initiate OS Host Power Off 262 [Documentation] Initiate an OS reboot. 263 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 264 ... ${os_password}=${OS_PASSWORD} ${hard}=${0} 265 266 # Description of argument(s): 267 # os_host The DNS name or IP of the OS. 268 # os_username The username to be used to sign in to the OS. 269 # os_password The password to be used to sign in to the OS. 270 # hard Indicates whether to do a hard vs. soft power off. 271 272 ${time_string}= Run Keyword If ${hard} Set Variable ${SPACE}now 273 ... ELSE Set Variable ${EMPTY} 274 275 ${cmd_buf}= Run Keyword If '${os_username}' == 'root' 276 ... Set Variable shutdown${time_string} 277 ... ELSE 278 ... Set Variable echo ${os_password} | sudo -S shutdown${time_string} 279 280 ${output} ${stderr} ${rc}= OS Execute Command 281 ... ${cmd_buf} fork=${1} 282 283 284Initiate OS Host Reboot 285 [Documentation] Initiate an OS reboot. 286 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 287 ... ${os_password}=${OS_PASSWORD} 288 289 # Description of argument(s): 290 # os_host The host name or IP address of the OS. 291 # os_username The username to be used to sign in to the OS. 292 # os_password The password to be used to sign in to the OS. 293 294 ${cmd_buf}= Run Keyword If '${os_username}' == 'root' 295 ... Set Variable reboot 296 ... ELSE 297 ... Set Variable echo ${os_password} | sudo -S reboot 298 299 ${output} ${stderr} ${rc}= OS Execute Command 300 ... ${cmd_buf} fork=${1} 301 302 303Initiate Auto Reboot 304 [Documentation] Initiate an auto reboot. 305 [Arguments] ${milliseconds}=5000 306 307 # Description of argument(s): 308 # milliseconds The number of milliseconds for the watchdog timer. 309 310 # Set the auto reboot policy. 311 Set Auto Reboot ${1} 312 # Set the watchdog timer. 313 Trigger Host Watchdog Error ${milliseconds} 314 315 316Trigger Warm Reset 317 [Documentation] Initiate a warm reset. 318 319 log to console "Triggering warm reset" 320 ${data}= create dictionary data=@{EMPTY} 321 ${resp}= openbmc post request 322 ... ${OPENBMC_BASE_URI}control/bmc0/action/warmReset data=${data} 323 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 324 ${session_active}= Check If warmReset is Initiated 325 Run Keyword If '${session_active}' == '${True}' 326 ... Fail msg=warm reset didn't occur 327 328 Sleep ${SYSTEM_SHUTDOWN_TIME}min 329 Check If BMC Is Up 330 331 332Check OS 333 [Documentation] Attempts to ping the host OS and then checks that the host 334 ... OS is up by running an SSH command. 335 336 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 337 ... ${os_password}=${OS_PASSWORD} ${quiet}=${QUIET} 338 ... ${print_string}=${EMPTY} 339 [Teardown] SSHLibrary.Close Connection 340 341 # Description of argument(s): 342 # os_host The DNS name/IP of the OS host associated with our BMC. 343 # os_username The username to be used to sign on to the OS host. 344 # os_password The password to be used to sign on to the OS host. 345 # quiet Indicates whether this keyword should write to console. 346 # print_string A string to be printed before checking the OS. 347 348 rprint ${print_string} 349 350 # Attempt to ping the OS. Store the return code to check later. 351 ${ping_rc}= Run Keyword and Return Status Ping Host ${os_host} 352 353 SSHLibrary.Open connection ${os_host} 354 355 ${status} ${msg}= Run Keyword And Ignore Error Login ${os_username} 356 ... ${os_password} 357 ${err_msg1}= Sprint Error ${msg} 358 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1} 359 Run Keyword If '${status}' == 'FAIL' Fail msg=${err_msg} 360 ${output} ${stderr} ${rc}= Execute Command uptime return_stderr=True 361 ... return_rc=True 362 363 ${temp_msg}= Catenate Could not execute a command on the operating 364 ... system.\n 365 ${err_msg1}= Sprint Error ${temp_msg} 366 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1} 367 368 # If the return code returned by "Execute Command" is non-zero, this 369 # keyword will fail. 370 Should Be Equal ${rc} ${0} msg=${err_msg} 371 # We will likewise fail if there is any stderr data. 372 Should Be Empty ${stderr} 373 374 ${temp_msg}= Set Variable Could not ping the operating system.\n 375 ${err_msg1}= Sprint Error ${temp_msg} 376 ${err_msg}= Catenate SEPARATOR= \n ${err_msg1} 377 # We will likewise fail if the OS did not ping, as we could SSH but not 378 # ping 379 Should Be Equal As Strings ${ping_rc} ${TRUE} msg=${err_msg} 380 381 382Wait for OS 383 [Documentation] Waits for the host OS to come up via calls to "Check OS". 384 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 385 ... ${os_password}=${OS_PASSWORD} ${timeout}=${OS_WAIT_TIMEOUT} 386 ... ${quiet}=${0} 387 [Teardown] rprintn 388 389 # Description of argument(s): 390 # os_host The DNS name or IP of the OS host associated with our 391 # BMC. 392 # os_username The username to be used to sign on to the OS host. 393 # os_password The password to be used to sign on to the OS host. 394 # timeout The timeout in seconds indicating how long you're 395 # willing to wait for the OS to respond. 396 # quiet Indicates whether this keyword should write to console. 397 398 # The interval to be used between calls to "Check OS". 399 ${interval}= Set Variable 5 400 401 ${message}= Catenate Checking every ${interval} seconds for up to 402 ... ${timeout} seconds for the operating system to communicate. 403 rqprint_timen ${message} 404 405 Wait Until Keyword Succeeds ${timeout} sec ${interval} Check OS 406 ... ${os_host} ${os_username} ${os_password} 407 ... print_string=\# 408 409 rqprintn 410 411 rqprint_timen The operating system is now communicating. 412 413 414Get Power State 415 [Documentation] Returns the power state as an integer. Either 0 or 1. 416 [Arguments] ${quiet}=${QUIET} 417 418 # Description of argument(s): 419 # quiet Indicates whether this keyword should run without any output to 420 # the console. 421 422 @{arglist}= Create List 423 ${args}= Create Dictionary data=@{arglist} 424 425 ${resp}= Call Method ${OPENBMC_BASE_URI}control/chassis0/ getPowerState 426 ... data=${args} quiet=${quiet} 427 Should be equal as strings ${resp.status_code} ${HTTP_OK} 428 ${content}= to json ${resp.content} 429 430 [Return] ${content["data"]} 431 432 433Clear BMC Gard Record 434 [Documentation] Clear gard records from the system. 435 436 @{arglist}= Create List 437 ${args}= Create Dictionary data=@{arglist} 438 ${resp}= Call Method 439 ... ${OPENPOWER_CONTROL}gard Reset data=${args} 440 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 441 442 443Copy PNOR to BMC 444 Import Library SCPLibrary WITH NAME scp 445 Open Connection for SCP 446 Log Copying ${PNOR_IMAGE_PATH} to /tmp 447 scp.Put File ${PNOR_IMAGE_PATH} /tmp 448 449 450Flash PNOR 451 [Documentation] Calls flash bios update method to flash PNOR image 452 [Arguments] ${pnor_image} 453 454 # Description of argument(s): 455 # pnor_image The filename and path of the PNOR image 456 # (e.g. "/home/image/zaius.pnor"). 457 458 @{arglist}= Create List ${pnor_image} 459 ${args}= Create Dictionary data=@{arglist} 460 ${resp}= Call Method /org/openbmc/control/flash/bios/ update 461 ... data=${args} 462 should be equal as strings ${resp.status_code} ${HTTP_OK} 463 Wait Until Keyword Succeeds 2 min 10 sec Is PNOR Flashing 464 465 466Get Flash BIOS Status 467 [Documentation] Returns the status of the flash BIOS API as a string. For 468 ... example 'Flashing', 'Flash Done', etc 469 ${data}= Read Properties /org/openbmc/control/flash/bios 470 [Return] ${data['status']} 471 472 473Is PNOR Flashing 474 [Documentation] Get BIOS 'Flashing' status. This indicates that PNOR 475 ... flashing has started. 476 ${status}= Get Flash BIOS Status 477 Should Contain ${status} Flashing 478 479 480Is PNOR Flash Done 481 [Documentation] Get BIOS 'Flash Done' status. This indicates that the 482 ... PNOR flashing has completed. 483 ${status}= Get Flash BIOS Status 484 should be equal as strings ${status} Flash Done 485 486 487Is OS Starting 488 [Documentation] Check if boot progress is OS starting. 489 ${boot_progress}= Get Boot Progress 490 Should Be Equal ${boot_progress} OSStart 491 492 493Is OS Off 494 [Documentation] Check if boot progress is "Off". 495 ${boot_progress}= Get Boot Progress 496 Should Be Equal ${boot_progress} Off 497 498 499Get Boot Progress To OS Starting State 500 [Documentation] Get the system to a boot progress state of 'FW Progress, 501 ... Starting OS'. 502 503 ${boot_progress}= Get Boot Progress 504 Run Keyword If '${boot_progress}' == 'OSStart' 505 ... Log Host is already in OS starting state 506 ... ELSE 507 ... Run Keywords Initiate Host PowerOff AND Initiate Host Boot 508 ... AND Wait Until Keyword Succeeds 10 min 10 sec Is OS Starting 509 510 511Verify Ping and REST Authentication 512 ${l_ping}= Run Keyword And Return Status 513 ... Ping Host ${OPENBMC_HOST} 514 Run Keyword If '${l_ping}' == '${False}' 515 ... Fail msg=Ping Failed 516 517 ${l_rest}= Run Keyword And Return Status 518 ... Initialize OpenBMC 519 Run Keyword If '${l_rest}' == '${False}' 520 ... Fail msg=REST Authentication Failed 521 522 # Just to make sure the SSH is working for SCP 523 Open Connection And Log In 524 ${system} ${stderr}= Execute Command hostname return_stderr=True 525 Should Be Empty ${stderr} 526 527 528Check If BMC is Up 529 [Documentation] Wait for Host to be online. Checks every X seconds 530 ... interval for Y minutes and fails if timed out. 531 ... Default MAX timedout is 10 min, interval 10 seconds. 532 [Arguments] ${max_timeout}=${OPENBMC_REBOOT_TIMEOUT} min 533 ... ${interval}=10 sec 534 535 # Description of argument(s): 536 # max_timeout Maximum time to wait. 537 # This should be expressed in Robot Framework's time format 538 # (e.g. "10 minutes"). 539 # interfal Interval to wait between status checks. 540 # This should be expressed in Robot Framework's time format 541 # (e.g. "5 seconds"). 542 543 Wait Until Keyword Succeeds 544 ... ${max_timeout} ${interval} Verify Ping and REST Authentication 545 546 547Check If warmReset is Initiated 548 [Documentation] Ping would be still alive, so try SSH to connect 549 ... if fails the ports are down indicating reboot 550 ... is in progress 551 552 # Warm reset adds 3 seconds delay before forcing reboot 553 # To minimize race conditions, we wait for 7 seconds 554 Sleep 7s 555 ${alive}= Run Keyword and Return Status 556 ... Open Connection And Log In 557 Return From Keyword If '${alive}' == '${False}' ${False} 558 [Return] ${True} 559 560 561Flush REST Sessions 562 [Documentation] Removes all the active session objects 563 Delete All Sessions 564 565 566Initialize DBUS cmd 567 [Documentation] Initialize dbus string with property string to extract 568 [Arguments] ${boot_property} 569 570 # Description of argument(s): 571 # boot_property Property string. 572 573 ${cmd}= Catenate ${dbuscmdBase} ${dbuscmdGet} ${dbuscmdString} 574 ${cmd}= Catenate ${cmd}${boot_property} 575 Set Global Variable ${dbuscmd} ${cmd} 576 577 578Create OS Console File Path 579 [Documentation] Create OS console file path name and return it. 580 [Arguments] ${log_file_path}=${EMPTY} 581 582 # Description of arguments: 583 # file_path The caller's candidate value. If this value is ${EMPTY}, this 584 # keyword will compose a file path name. Otherwise, this 585 # keyword will use the caller's file_path value. In either 586 # case, the value will be returned. 587 588 ${default_file_path}= Catenate /tmp/${OPENBMC_HOST}_os_console.txt 589 ${log_file_path}= Set Variable If '${log_file_path}' == '${EMPTY}' 590 ... ${default_file_path} ${log_file_path} 591 592 [Return] ${log_file_path} 593 594 595Create OS Console Command String 596 [Documentation] Return a command string to start OS console logging. 597 598 # First make sure that the ssh_pw program is available. 599 ${cmd}= Catenate which ssh_pw 2>/dev/null || find 600 ... ${EXECDIR} -name 'ssh_pw' 601 602 Rdpissuing ${cmd} 603 ${rc} ${output}= Run And Return Rc And Output ${cmd} 604 Rdpvars rc output 605 606 Should Be Equal As Integers 0 ${rc} msg=Could not find ssh_pw. 607 608 ${ssh_pw_file_path}= Set Variable ${output} 609 610 ${cmd}= Catenate ${ssh_pw_file_path} ${OPENBMC_PASSWORD} -p 2200 611 ... -o "StrictHostKeyChecking no" ${OPENBMC_USERNAME}@${OPENBMC_HOST} 612 613 [Return] ${cmd} 614 615 616Get SOL Console Pid 617 [Documentation] Get the pid of the active SOL console job. 618 [Arguments] ${expect_running}=${0} 619 620 # Description of argument(s): 621 # expect_running If set and if no SOL console job is found, print debug 622 # info and fail. 623 624 # Find the pid of the active system console logging session (if any). 625 ${search_string}= Create OS Console Command String 626 # At least in some cases, ps output does not show double quotes so we must 627 # replace them in our search string with the regexes to indicate that they 628 # are optional. 629 ${search_string}= Replace String ${search_string} " ["]? 630 ${ps_cmd}= Catenate ps axwwo user,pid,cmd 631 ${cmd_buf}= Catenate echo $(${ps_cmd} | egrep '${search_string}' | 632 ... egrep -v grep | cut -c10-14) 633 Rdpissuing ${cmd_buf} 634 ${rc} ${os_con_pid}= Run And Return Rc And Output ${cmd_buf} 635 Rdpvars os_con_pid 636 # If rc is not zero it just means that there is no OS Console process 637 # running. 638 639 Return From Keyword If '${os_con_pid}' != '${EMPTY}' ${os_con_pid} 640 Return From Keyword If '${expect_running}' == '${0}' ${os_con_pid} 641 642 Cmd Fnc cat ${log_file_path} ; echo ; ${ps_cmd} quiet=${0} 643 ... print_output=${1} show_err=${1} 644 645 Should Not Be Empty ${os_con_pid} 646 647 648Stop SOL Console Logging 649 [Documentation] Stop system console logging and return log output. 650 [Arguments] ${log_file_path}=${EMPTY} 651 ... ${targ_file_path}=${EXECDIR}${/}logs${/} 652 ... ${return_data}=${1} 653 654 # If there are muliple system console processes, they will all be stopped. 655 # If there is no existing log file this keyword will return an error 656 # message to that effect (and write that message to targ_file_path, if 657 # specified). 658 # NOTE: This keyword will not fail if there is no running system console 659 # process. 660 661 # Description of arguments: 662 # log_file_path The file path that was used to call "Start SOL 663 # Console Logging". See that keyword (above) for details. 664 # targ_file_path If specified, the file path to which the source 665 # file path (i.e. "log_file_path") should be copied. 666 # return_data If this is set to ${1}, this keyword will return the SOL 667 # data to the caller as a unicode string. 668 669 ${log_file_path}= Create OS Console File Path ${log_file_path} 670 671 ${os_con_pid}= Get SOL Console Pid 672 673 ${cmd_buf}= Catenate kill -9 ${os_con_pid} 674 Run Keyword If '${os_con_pid}' != '${EMPTY}' Rdpissuing ${cmd_buf} 675 ${rc} ${output}= Run Keyword If '${os_con_pid}' != '${EMPTY}' 676 ... Run And Return Rc And Output ${cmd_buf} 677 Run Keyword If '${os_con_pid}' != '${EMPTY}' Rdpvars rc output 678 679 Run Keyword If '${targ_file_path}' != '${EMPTY}' 680 ... Run Keyword And Ignore Error 681 ... Copy File ${log_file_path} ${targ_file_path} 682 683 ${output}= Set Variable ${EMPTY} 684 ${loc_quiet}= Evaluate ${debug}^1 685 ${rc} ${output}= Run Keyword If '${return_data}' == '${1}' 686 ... Cmd Fnc cat ${log_file_path} 2>/dev/null quiet=${loc_quiet} 687 ... print_output=${0} show_err=${0} 688 689 [Return] ${output} 690 691 692Start SOL Console Logging 693 [Documentation] Start system console log to file. 694 [Arguments] ${log_file_path}=${EMPTY} ${return_data}=${1} 695 696 # This keyword will first call "Stop SOL Console Logging". Only then will 697 # it start SOL console logging. The data returned by "Stop SOL Console 698 # Logging" will in turn be returned by this keyword. 699 700 # Description of arguments: 701 # log_file_path The file path to which system console log data should be 702 # written. Note that this path is taken to be a location 703 # on the machine where this program is running rather than 704 # on the Open BMC system. 705 # return_data If this is set to ${1}, this keyword will return any SOL 706 # data to the caller as a unicode string. 707 708 ${log_file_path}= Create OS Console File Path ${log_file_path} 709 710 ${log_output}= Stop SOL Console Logging ${log_file_path} 711 ... return_data=${return_data} 712 713 # Validate by making sure we can create the file. Problems creating the 714 # file would not be noticed by the subsequent ssh command because we fork 715 # the command. 716 Create File ${log_file_path} 717 ${sub_cmd_buf}= Create OS Console Command String 718 # Routing stderr to stdout so that any startup error text will go to the 719 # output file. 720 # TODO: Doesn't work with tox so reverting temporarily. 721 # nohup detaches the process completely from our pty. 722 #${cmd_buf}= Catenate nohup ${sub_cmd_buf} &> ${log_file_path} & 723 ${cmd_buf}= Catenate ${sub_cmd_buf} > ${log_file_path} 2>&1 & 724 Rdpissuing ${cmd_buf} 725 ${rc} ${output}= Run And Return Rc And Output ${cmd_buf} 726 # Because we are forking this command, we essentially will never get a 727 # non-zero return code or any output. 728 Should Be Equal ${rc} ${0} 729 730 Wait Until Keyword Succeeds 10 seconds 0 seconds 731 ... Get SOL Console Pid ${1} 732 733 [Return] ${log_output} 734 735 736Get Time Stamp 737 [Documentation] Get the current time stamp data 738 ${cur_time}= Get Current Date result_format=%Y%m%d%H%M%S%f 739 [Return] ${cur_time} 740 741 742Start Journal Log 743 [Documentation] Start capturing journal log to a file in /tmp using 744 ... journalctl command. By default journal log is collected 745 ... at /tmp/journal_log else user input location. 746 ... The File is appended with datetime. 747 [Arguments] ${file_path}=/tmp/journal_log 748 749 # Description of arguments: 750 # file_path The file path of the journal file. 751 752 ${cur_time}= Get Time Stamp 753 Set Global Variable ${LOG_TIME} ${cur_time} 754 Start Command 755 ... journalctl -f > ${file_path}-${LOG_TIME} 756 Log Journal Log Started: ${file_path}-${LOG_TIME} 757 758 759Stop Journal Log 760 [Documentation] Stop journalctl process if its running. 761 ... By default return log from /tmp/journal_log else 762 ... user input location. 763 [Arguments] ${file_path}=/tmp/journal_log 764 765 # Description of arguments: 766 # file_path The file path of the journal file. 767 768 Open Connection And Log In 769 770 ${rc}= 771 ... Execute Command 772 ... ps ax | grep journalctl | grep -v grep 773 ... return_stdout=False return_rc=True 774 775 Return From Keyword If '${rc}' == '${1}' 776 ... No journal log process running 777 778 ${output} ${stderr}= 779 ... Execute Command killall journalctl 780 ... return_stderr=True 781 Should Be Empty ${stderr} 782 783 ${journal_log} ${stderr}= 784 ... Execute Command 785 ... cat ${file_path}-${LOG_TIME} 786 ... return_stderr=True 787 Should Be Empty ${stderr} 788 789 Log ${journal_log} 790 791 Execute Command rm ${file_path}-${LOG_TIME} 792 793 [Return] ${journal_log} 794 795 796Mac Address To Hex String 797 [Documentation] Converts MAC address into hex format. 798 ... Example 799 ... Given the following MAC: 00:01:6C:80:02:78 800 ... This keyword will return: 0x00 0x01 0x6C 0x80 0x02 0x78 801 ... Description of arguments: 802 ... i_macaddress MAC address in the following format 803 ... 00:01:6C:80:02:78 804 [Arguments] ${i_macaddress} 805 806 # Description of arguments: 807 # i_macaddress The MAC address. 808 809 ${mac_hex}= Catenate 0x${i_macaddress.replace(':', ' 0x')} 810 [Return] ${mac_hex} 811 812 813IP Address To Hex String 814 [Documentation] Converts IP address into hex format. 815 ... Example: 816 ... Given the following IP: 10.3.164.100 817 ... This keyword will return: 0xa 0x3 0xa4 0xa0 818 [Arguments] ${i_ipaddress} 819 820 # Description of arguments: 821 # i_macaddress The IP address in the format 10.10.10.10. 822 823 @{ip}= Split String ${i_ipaddress} . 824 ${index}= Set Variable ${0} 825 826 :FOR ${item} IN @{ip} 827 \ ${hex}= Convert To Hex ${item} prefix=0x lowercase=yes 828 \ Set List Value ${ip} ${index} ${hex} 829 \ ${index}= Set Variable ${index + 1} 830 ${ip_hex}= Catenate @{ip} 831 832 [Return] ${ip_hex} 833 834 835BMC CPU Performance Check 836 [Documentation] Minimal 10% of proc should be free in this instance 837 838 ${bmc_cpu_usage_output} ${stderr} ${rc}= BMC Execute Command 839 ... ${bmc_cpu_usage_cmd} 840 ${bmc_cpu_usage_output} ${stderr} ${rc}= BMC Execute Command 841 ... ${bmc_cpu_usage_cmd} 842 ${bmc_cpu_percentage}= Fetch From Left ${bmc_cpu_usage_output} % 843 Should be true ${bmc_cpu_percentage} < 90 844 845 846BMC Mem Performance Check 847 [Documentation] Minimal 10% of memory should be free in this instance 848 849 ${bmc_mem_free_output} ${stderr} ${rc}= BMC Execute Command 850 ... ${bmc_mem_free_cmd} 851 852 ${bmc_mem_total_output} ${stderr} ${rc}= BMC Execute Command 853 ... ${bmc_mem_total_cmd} 854 ${bmc_mem_free_output} ${stderr} ${rc}= BMC Execute Command 855 ... ${bmc_mem_free_cmd} 856 857 ${bmc_mem_total_output} ${stderr} ${rc}= BMC Execute Command 858 ... ${bmc_mem_total_cmd} 859 860 ${bmc_mem_percentage}= Evaluate ${bmc_mem_free_output}*100 861 ${bmc_mem_percentage}= Evaluate 862 ... ${bmc_mem_percentage}/${bmc_mem_total_output} 863 Should be true ${bmc_mem_percentage} > 10 864 865 866BMC File System Usage Check 867 [Documentation] Check the file system space. 4 file system should be 868 ... 100% full which is expected 869 # Filesystem Size Used Available Use% Mounted on 870 # /dev/root 14.4M 14.4M 0 100% / 871 # /dev/ubiblock0_0 14.4M 14.4M 0 100% /media/rofs-c9249b0e 872 # /dev/ubiblock8_0 19.6M 19.6M 0 100% /media/pnor-ro-8764baa3 873 # /dev/ubiblock4_0 14.4M 14.4M 0 100% /media/rofs-407816c 874 # /dev/ubiblock8_4 21.1M 21.1M 0 100% /media/pnor-ro-cecc64c4 875 ${bmc_fs_usage_output} ${stderr} ${rc}= BMC Execute Command 876 ... ${bmc_file_system_usage_cmd} 877 ${bmc_pnor_fs_usage_output} ${stderr} ${rc}= BMC Execute Command 878 ... ${total_pnor_ro_file_system_cmd} 879 ${bmc_bmc_fs_usage_output} ${stderr} ${rc}= BMC Execute Command 880 ... ${total_bmc_ro_file_system_cmd} 881 ${total_bmc_pnor_image}= Evaluate 882 ... ${bmc_pnor_fs_usage_output}+${bmc_bmc_fs_usage_output} 883 # Considering /dev/root also in total 100% used file system 884 ${total_full_fs}= Evaluate ${total_bmc_pnor_image}+1 885 Should Be True ${bmc_fs_usage_output}==${total_full_fs} 886 887 888Check BMC CPU Performance 889 [Documentation] Minimal 10% of proc should be free in 3 sample 890 :FOR ${var} IN Range 1 4 891 \ BMC CPU Performance check 892 893 894Check BMC Mem Performance 895 [Documentation] Minimal 10% of memory should be free 896 897 :FOR ${var} IN Range 1 4 898 \ BMC Mem Performance check 899 900 901Check BMC File System Performance 902 [Documentation] Check for file system usage for 4 times 903 904 :FOR ${var} IN Range 1 4 905 \ BMC File System Usage check 906 907 908Get URL List 909 [Documentation] Return list of URLs under given URL. 910 [Arguments] ${openbmc_url} 911 912 # Description of argument(s): 913 # openbmc_url URL for list operation (e.g. 914 # /xyz/openbmc_project/inventory). 915 916 ${url_list}= Read Properties ${openbmc_url}/list quiet=${1} 917 Sort List ${url_list} 918 919 [Return] ${url_list} 920 921 922Get Endpoint Paths 923 [Documentation] Returns all url paths ending with given endpoint 924 ... Example: 925 ... Given the following endpoint: cpu 926 ... This keyword will return: list of all urls ending with 927 ... cpu - 928 ... /org/openbmc/inventory/system/chassis/motherboard/cpu0, 929 ... /org/openbmc/inventory/system/chassis/motherboard/cpu1 930 [Arguments] ${path} ${endpoint} 931 932 # Description of arguments: 933 # path URL path for enumeration. 934 # endpoint Endpoint string (url path ending). 935 936 ${resp}= Read Properties ${path}/enumerate timeout=30 937 Log Dictionary ${resp} 938 939 ${list}= Get Dictionary Keys ${resp} 940 # For a given string, look for prefix and suffix for matching expression. 941 # Start of string followed by zero or more of any character followed by 942 # any digit or lower case character. 943 ${resp}= Get Matches ${list} regexp=^.*[0-9a-z_].${endpoint}[0-9a-z]*$ 944 945 [Return] ${resp} 946 947 948Check Zombie Process 949 [Documentation] Check if any defunct process exist or not on BMC 950 ${count} ${stderr} ${rc}= Execute Command ps -o stat | grep Z | wc -l 951 ... return_stderr=True return_rc=True 952 Should Be True ${count}==0 953 Should Be Empty ${stderr} 954 955 956Prune Journal Log 957 [Documentation] Prune archived journal logs. 958 [Arguments] ${vacuum_size}=1M 959 960 # This keyword can be used to prevent the journal 961 # log from filling up the /run filesystem. 962 # This command will retain only the latest logs 963 # of the user specified size. 964 965 # Description of argument(s): 966 # vacuum_size Size of journal. 967 968 Open Connection And Log In 969 ${output} ${stderr} ${rc}= 970 ... Execute Command 971 ... journalctl --vacuum-size=${vacuum_size} 972 ... return_stderr=True return_rc=True 973 974 Should Be Equal ${rc} ${0} msg=${stderr} 975 976 977Set BMC Power Policy 978 [Documentation] Set the given BMC power policy. 979 [Arguments] ${policy} 980 981 # Note that this function will translate the old style "RESTORE_LAST_STATE" 982 # policy to the new style "xyz.openbmc_project.Control.Power.RestorePolicy. 983 # Policy.Restore" for you. 984 985 # Description of argument(s): 986 # policy Power restore policy (e.g "RESTORE_LAST_STATE", 987 # ${RESTORE_LAST_STATE}). 988 989 # Set the bmc_power_policy_method to either 'Old' or 'New'. 990 Set Power Policy Method 991 # This translation helps bridge between old and new method for calling. 992 ${policy}= Translate Power Policy Value ${policy} 993 # Run the appropriate keyword. 994 Run Key ${bmc_power_policy_method} Set Power Policy \ ${policy} 995 ${currentPolicy}= Get System Power Policy 996 Should Be Equal ${currentPolicy} ${policy} 997 998 999New Set Power Policy 1000 [Documentation] Set the given BMC power policy (new method). 1001 [Arguments] ${policy} 1002 1003 # Description of argument(s): 1004 # policy Power restore policy (e.g. ${ALWAYS_POWER_OFF}). 1005 1006 ${valueDict}= Create Dictionary data=${policy} 1007 Write Attribute 1008 ... ${POWER_RESTORE_URI} PowerRestorePolicy data=${valueDict} 1009 1010 1011Old Set Power Policy 1012 [Documentation] Set the given BMC power policy (old method). 1013 [Arguments] ${policy} 1014 1015 # Description of argument(s): 1016 # policy Power restore policy (e.g. "ALWAYS_POWER_OFF"). 1017 1018 ${valueDict}= create dictionary data=${policy} 1019 Write Attribute ${HOST_SETTING} power_policy data=${valueDict} 1020 1021 1022Get System Power Policy 1023 [Documentation] Returns the BMC power policy. 1024 1025 # Set the bmc_power_policy_method to either 'Old' or 'New'. 1026 Set Power Policy Method 1027 ${cmd_buf}= Create List ${bmc_power_policy_method} Get Power Policy 1028 # Run the appropriate keyword. 1029 ${currentPolicy}= Run Keyword @{cmd_buf} 1030 1031 [Return] ${currentPolicy} 1032 1033 1034New Get Power Policy 1035 [Documentation] Returns the BMC power policy (new method). 1036 ${currentPolicy}= Read Attribute ${POWER_RESTORE_URI} PowerRestorePolicy 1037 1038 [Return] ${currentPolicy} 1039 1040 1041Old Get Power Policy 1042 [Documentation] Returns the BMC power policy (old method). 1043 ${currentPolicy}= Read Attribute ${HOST_SETTING} power_policy 1044 1045 [Return] ${currentPolicy} 1046 1047 1048Get Auto Reboot 1049 [Documentation] Returns auto reboot setting. 1050 ${setting}= Read Attribute ${CONTROL_HOST_URI}/auto_reboot AutoReboot 1051 1052 [Return] ${setting} 1053 1054 1055Set Auto Reboot 1056 [Documentation] Set the given auto reboot setting. 1057 [Arguments] ${setting} 1058 1059 # Description of argument(s): 1060 # setting The reboot setting, 1 for enabling and 0 for disabling. 1061 1062 ${valueDict}= Set Variable ${setting} 1063 ${data}= Create Dictionary data=${valueDict} 1064 Write Attribute ${CONTROL_HOST_URI}/auto_reboot AutoReboot data=${data} 1065 ${current_setting}= Get Auto Reboot 1066 Should Be Equal As Integers ${current_setting} ${setting} 1067 1068 1069Set BMC Reset Reference Time 1070 [Documentation] Set current boot time as a reference and increment 1071 ... boot count. 1072 1073 ${cur_btime}= Get BMC Boot Time 1074 Run Keyword If ${BOOT_TIME} == ${0} and ${BOOT_COUNT} == ${0} 1075 ... Set Global Variable ${BOOT_TIME} ${cur_btime} 1076 ... ELSE IF ${cur_btime} > ${BOOT_TIME} 1077 ... Run Keywords Set Global Variable ${BOOT_TIME} ${cur_btime} 1078 ... AND 1079 ... Set Global Variable ${BOOT_COUNT} ${BOOT_COUNT + 1} 1080 1081 1082Get BMC Boot Time 1083 [Documentation] Returns boot time from /proc/stat. 1084 1085 Open Connection And Log In 1086 ${output} ${stderr}= 1087 ... Execute Command egrep '^btime ' /proc/stat | cut -f 2 -d ' ' 1088 ... return_stderr=True 1089 Should Be Empty ${stderr} 1090 ${btime}= Convert To Integer ${output} 1091 [Return] ${btime} 1092 1093 1094Execute Command On BMC 1095 [Documentation] Execute given command on BMC and return output. 1096 [Arguments] ${command} 1097 1098 # Description of argument(s): 1099 # command The command to execute on the BMC. 1100 1101 ${stdout} ${stderr}= Execute Command ${command} return_stderr=True 1102 Should Be Empty ${stderr} 1103 [Return] ${stdout} 1104 1105 1106Enable Core Dump On BMC 1107 [Documentation] Enable core dump collection. 1108 Open Connection And Log In 1109 ${core_pattern}= Execute Command On BMC 1110 ... echo '/tmp/core_%e.%p' | tee /proc/sys/kernel/core_pattern 1111 Should Be Equal As Strings ${core_pattern} /tmp/core_%e.%p 1112 1113 1114Get Number Of BMC Core Dump Files 1115 [Documentation] Returns number of core dump files on BMC. 1116 Open Connection And Log In 1117 ${num_of_core_dump}= Execute Command 1118 ... ls /tmp/core* 2>/dev/null | wc -l 1119 [Return] ${num_of_core_dump} 1120 1121 1122Set Core Dump File Size Unlimited 1123 [Documentation] Set core dump file size to unlimited. 1124 Open Connection And Log In 1125 Execute Command On BMC 1126 ... ulimit -c unlimited 1127 1128 1129Check For Core Dumps 1130 [Documentation] Check for any core dumps exist. 1131 ${output}= Get Number Of BMC Core Dump Files 1132 Run Keyword If ${output} > 0 1133 ... Log **Warning** BMC core dump files exist level=WARN 1134 1135 1136Trigger Host Watchdog Error 1137 [Documentation] Inject host watchdog timeout error via REST. 1138 [Arguments] ${milliseconds}=1000 ${sleep_time}=5s 1139 1140 # Description of argument(s): 1141 # milliseconds The time watchdog timer value in milliseconds (e.g. 1000 = 1142 # 1 second). 1143 # sleep_time Time delay for host watchdog error to get injected. 1144 # Default is 5 seconds. 1145 1146 ${data}= Create Dictionary 1147 ... data=xyz.openbmc_project.State.Watchdog.Action.PowerCycle 1148 ${status} ${result}= Run Keyword And Ignore Error 1149 ... Read Attribute ${HOST_WATCHDOG_URI} ExpireAction 1150 Run Keyword If '${status}' == 'PASS' 1151 ... Write Attribute ${HOST_WATCHDOG_URI} ExpireAction data=${data} 1152 1153 ${data}= Create Dictionary data=${milliseconds} 1154 Write Attribute ${HOST_WATCHDOG_URI} Interval data=${data} 1155 1156 ${data}= Create Dictionary data=${True} 1157 Write Attribute ${HOST_WATCHDOG_URI} Enabled data=${data} 1158 1159 Sleep ${sleep_time} 1160 1161 1162Login To OS Host 1163 [Documentation] Login to OS Host and return the Login response code. 1164 [Arguments] ${os_host}=${OS_HOST} ${os_username}=${OS_USERNAME} 1165 ... ${os_password}=${OS_PASSWORD} 1166 1167 # Description of arguments: 1168 # ${os_host} IP address of the OS Host. 1169 # ${os_username} OS Host Login user name. 1170 # ${os_password} OS Host Login passwrd. 1171 1172 ${os_state}= Run Keyword And Return Status Ping Host ${os_host} 1173 Run Keyword If '${os_state}' == 'False' 1174 ... Run Keywords Initiate Host Reboot AND 1175 ... Is Host Running AND 1176 ... Wait for OS ${os_host} ${os_username} ${os_password} 1177 1178 SSHLibrary.Open Connection ${os_host} 1179 ${resp}= Login ${os_username} ${os_password} 1180 [Return] ${resp} 1181 1182 1183Configure Initial Settings 1184 [Documentation] Restore old IP and route. 1185 ... This keyword requires initial settings viz IP address, 1186 ... Network Mask, default gatway and serial console IP and port 1187 ... information which should be provided in command line. 1188 1189 [Arguments] ${host}=${OPENBMC_HOST} ${mask}=${NET_MASK} 1190 ... ${gw_ip}=${GW_IP} 1191 1192 # Description of arguments: 1193 # host IP address of the OS Host. 1194 # mask Network mask. 1195 # gu_ip Gateway IP address or hostname. 1196 1197 # Open telnet connection and ignore the error, in case telnet session is 1198 # already opened by the program calling this keyword. 1199 Run Keyword And Ignore Error Open Telnet Connection to BMC Serial Console 1200 Telnet.write ifconfig eth0 ${host} netmask ${mask} 1201 Telnet.write route add default gw ${gw_ip} 1202 1203 1204Install Debug Tarball On BMC 1205 [Documentation] Copy the debug tar file to BMC and install. 1206 [Arguments] ${tarball_file_path}=${default_tarball} 1207 ... ${targ_tarball_dir_path}=/tmp/tarball/ 1208 1209 # Description of arguments: 1210 # tarball_file_path Path of the debug tarball file. 1211 # The tar file is downloaded from the build page 1212 # https://openpower.xyz/job/openbmc-build/ 1213 # obmc-phosphor-debug-tarball-witherspoon.tar.xz 1214 # 1215 # targ_tarball_dir_path The directory path where the tarball is to be 1216 # installed. 1217 1218 OperatingSystem.File Should Exist ${tarball_file_path} 1219 ... msg=${tarball_file_path} doesn't exist. 1220 1221 # Upload the file to BMC. 1222 Import Library SCPLibrary WITH NAME scp 1223 Open Connection for SCP 1224 scp.Put File ${tarball_file_path} /tmp/debug-tarball.tar.xz 1225 1226 # Create tarball directory and install. 1227 BMC Execute Command mkdir -p ${targ_tarball_dir_path} 1228 BMC Execute Command 1229 ... tar -xf /tmp/debug-tarball.tar.xz -C ${targ_tarball_dir_path} 1230 1231 # Remove the tarball file from BMC. 1232 BMC Execute Command rm -f /tmp/debug-tarball.tar.xz 1233 1234 1235Get BMC Boot Count 1236 [Documentation] Returns BMC boot count based on boot time. 1237 ${cur_btime}= Get BMC Boot Time 1238 1239 # Set global variable BOOT_TIME to current boot time if current boot time 1240 # is changed. Also increase value of global variable BOOT_COUNT by 1. 1241 Run Keyword If ${cur_btime} > ${BOOT_TIME} 1242 ... Run Keywords Set Global Variable ${BOOT_TIME} ${cur_btime} 1243 ... AND 1244 ... Set Global Variable ${BOOT_COUNT} ${BOOT_COUNT + 1} 1245 1246 [Return] ${BOOT_COUNT} 1247 1248 1249Set BMC Boot Count 1250 [Documentation] Set BMC boot count to given value. 1251 [Arguments] ${count} 1252 1253 # Description of arguments: 1254 # count boot count value. 1255 ${cur_btime}= Get BMC Boot Time 1256 1257 # Set global variable BOOT_COUNT to given value. 1258 Set Global Variable ${BOOT_COUNT} ${count} 1259 1260 # Set BOOT_TIME variable to current boot time. 1261 Set Global Variable ${BOOT_COUNT} ${count} 1262 1263 1264Get System LED State 1265 [Documentation] Return the state of given system LED. 1266 [Arguments] ${led_name} 1267 1268 # Description of argument(s): 1269 # led_name System LED name (e.g. heartbeat, identify, beep). 1270 1271 ${state}= Read Attribute ${LED_PHYSICAL_URI}${led_name} State 1272 [Return] ${state.rsplit('.', 1)[1]} 1273 1274 1275Set System LED State 1276 [Documentation] Set given system LED via REST. 1277 [Arguments] ${led_name} ${led_state} 1278 # Description of argument(s): 1279 # led_name System LED name (e.g. heartbeat, identify, beep). 1280 # led_state LED state to be set (e.g. On, Off). 1281 1282 ${args}= Create Dictionary 1283 ... data=xyz.openbmc_project.Led.Physical.Action.${led_state} 1284 Write Attribute ${LED_PHYSICAL_URI}${led_name} State data=${args} 1285 1286 Verify LED State ${led_name} ${led_state} 1287 1288 1289Verify LED State 1290 [Documentation] Checks if LED is in given state. 1291 [Arguments] ${led_name} ${led_state} 1292 # Description of argument(s): 1293 # led_name System LED name (e.g. heartbeat, identify, beep). 1294 # led_state LED state to be verified (e.g. On, Off). 1295 1296 ${state}= Get System LED State ${led_name} 1297 Should Be Equal ${state} ${led_state} 1298 1299 1300Get LED State XYZ 1301 [Documentation] Returns state of given LED. 1302 [Arguments] ${led_name} 1303 1304 # Description of argument(s): 1305 # led_name Name of LED. 1306 1307 ${state}= Read Attribute ${LED_GROUPS_URI}${led_name} Asserted 1308 # Returns the state of the LED, either On or Off. 1309 [Return] ${state} 1310 1311 1312Delete Error Logs 1313 [Documentation] Delete error logs. 1314 1315 # Check if error logs entries exist, if not return. 1316 ${resp}= OpenBMC Get Request ${BMC_LOGGING_ENTRY}${/}list quiet=${1} 1317 Return From Keyword If ${resp.status_code} == ${HTTP_NOT_FOUND} 1318 1319 # Get the list of error logs entries and delete them all. 1320 ${elog_entries}= Get URL List ${BMC_LOGGING_ENTRY} 1321 :FOR ${entry} IN @{elog_entries} 1322 \ Delete Error Log Entry ${entry} 1323 1324 1325Delete Error Log Entry 1326 [Documentation] Delete error log entry. 1327 [Arguments] ${entry_path} 1328 1329 # Description of argument(s): 1330 # entry_path Delete an error log entry. 1331 # Ex. /xyz/openbmc_project/logging/entry/1 1332 1333 # Skip delete if entry URI is a callout. 1334 # Example: /xyz/openbmc_project/logging/entry/1/callout 1335 Return From Keyword If '${entry_path.rsplit('/', 1)[1]}' == 'callout' 1336 1337 ${data}= Create Dictionary data=@{EMPTY} 1338 ${resp}= Openbmc Delete Request ${entry_path} data=${data} 1339 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 1340 1341 1342Delete All Error Logs 1343 [Documentation] Delete all error log entries using "DeleteAll" interface. 1344 1345 ${data}= Create Dictionary data=@{EMPTY} 1346 ${resp}= Openbmc Post Request ${BMC_LOGGING_URI}action/DeleteAll 1347 ... data=${data} 1348 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 1349 1350 1351Get BMC Version 1352 [Documentation] Returns BMC version from /etc/os-release. 1353 ... e.g. "v1.99.6-141-ge662190" 1354 1355 Open Connection And Log In 1356 ${cmd}= Set Variable grep ^VERSION_ID= /etc/os-release | cut -f 2 -d '=' 1357 ${output}= Execute Command On BMC ${cmd} 1358 [Return] ${output} 1359 1360 1361Get PNOR Version 1362 [Documentation] Returns the PNOR version from the BMC. 1363 1364 ${pnor_attrs}= Get PNOR Attributes 1365 [Return] ${pnor_attrs['version']} 1366 1367 1368Get PNOR Attributes 1369 [Documentation] Return PNOR software attributes as a dictionary. 1370 1371 # This keyword parses /var/lib/phosphor-software-manager/pnor/ro/pnor.toc 1372 # into key/value pairs. 1373 1374 ${outbuf} ${stderr} ${rc}= BMC Execute Command 1375 ... cat /var/lib/phosphor-software-manager/pnor/ro/pnor.toc 1376 ${pnor_attrs}= Key Value Outbuf To Dict ${outbuf} delim== 1377 1378 [Return] ${pnor_attrs} 1379 1380 1381Get Elog URL List 1382 [Documentation] Return error log entry list of URLs. 1383 1384 ${url_list}= Read Properties /xyz/openbmc_project/logging/entry/ 1385 Sort List ${url_list} 1386 [Return] ${url_list} 1387 1388 1389Read Turbo Setting Via REST 1390 [Documentation] Return turbo setting via REST. 1391 # Returns 1 if TurboAllowed, 0 if not. 1392 1393 ${turbo_setting}= Read Attribute 1394 ... ${CONTROL_HOST_URI}turbo_allowed TurboAllowed 1395 [Return] ${turbo_setting} 1396 1397 1398 1399Set Turbo Setting Via REST 1400 [Documentation] Set turbo setting via REST. 1401 [Arguments] ${setting} 1402 1403 # Description of argument(s): 1404 # setting Value which needs to be set.(i.e. False or True). 1405 1406 ${data}= Create Dictionary data=${${setting}} 1407 Write Attribute ${CONTROL_HOST_URI}turbo_allowed TurboAllowed 1408 ... data=${data} 1409 1410 1411 1412Set Control Boot Mode 1413 [Documentation] Set given boot mode on the boot object path attribute. 1414 [Arguments] ${boot_path} ${boot_mode} 1415 1416 # Description of argument(s): 1417 # boot_path Boot object path. 1418 # Example: 1419 # /xyz/openbmc_project/control/host0/boot 1420 # /xyz/openbmc_project/control/host0/boot/one_time 1421 # boot_mode Boot mode which need to be set. 1422 # Example: 1423 # "xyz.openbmc_project.Control.Boot.Mode.Modes.Regular" 1424 1425 ${valueDict}= Create Dictionary data=${boot_mode} 1426 Write Attribute ${boot_path} BootMode data=${valueDict} 1427 1428 1429Copy Address Translation Utils To HOST OS 1430 [Documentation] Copy address translation utils to host OS. 1431 1432 OperatingSystem.File Should Exist ${probe_cpu_tool_path} 1433 ... msg=${probe_cpu_tool_path} doesn't exist. 1434 OperatingSystem.File Should Exist ${probe_cpu_tool_path} 1435 ... msg=${probe_cpu_tool_path} doesn't exist. 1436 1437 scp.Open connection ${OS_HOST} username=${OS_USERNAME} 1438 ... password=${OS_PASSWORD} 1439 scp.Put File ${probe_cpu_tool_path} ${target_file_path} 1440 scp.Put File ${scom_addrs_tool_path} ${target_file_path} 1441 1442 1443Verify BMC RTC And UTC Time Drift 1444 [Documentation] Verify that the RTC and UTC time difference is less than 1445 ... the given time_drift_max. 1446 [Arguments] ${time_diff_max}=${10} 1447 1448 # Description of argument(s): 1449 # time_diff_max The max allowable RTC and UTC time difference in seconds. 1450 1451 # Example: 1452 # time_dict: 1453 # [local_time]: Fri 2017-11-03 152756 UTC 1454 # [local_time_seconds]: 1509740876 1455 # [universal_time]: Fri 2017-11-03 152756 UTC 1456 # [universal_time_seconds]: 1509740876 1457 # [rtc_time]: Fri 2016-05-20 163403 1458 # [rtc_time_seconds]: 1463780043 1459 # [time_zone]: n/a (UTC, +0000) 1460 # [network_time_on]: yes 1461 # [ntp_synchronized]: no 1462 # [rtc_in_local_tz]: no 1463 1464 ${time}= Get BMC Date Time 1465 ${time_diff}= Evaluate 1466 ... ${time['universal_time_seconds']} - ${time['rtc_time_seconds']} 1467 Should Be True ${time_diff} < ${time_diff_max} 1468 1469 1470Watchdog Object Should Exist 1471 [Documentation] Check that watchdog object exists. 1472 1473 ${resp}= OpenBMC Get Request ${WATCHDOG_URI}host0 1474 Should Be Equal As Strings ${resp.status_code} ${HTTP_OK} 1475 ... msg=Expected watchdog object does not exist. 1476 1477 1478Validate IP On BMC 1479 [Documentation] Validate IP address is present in set of IP addresses. 1480 [Arguments] ${ip_address} ${ip_data} 1481 1482 # Description of argument(s): 1483 # ip_address IP address to check (e.g. xx.xx.xx.xx). 1484 # ip_data Set of the IP addresses present. 1485 1486 Should Contain Match ${ip_data} ${ip_address}/* 1487 ... msg=${ip_address} not found in the list provided. 1488 1489 1490Remove Journald Logs 1491 [Documentation] Remove all journald logs and restart service. 1492 1493 ${cmd}= Catenate systemctl stop systemd-journald.service && 1494 ... rm -rf /var/log/journal && systemctl start systemd-journald.service 1495 1496 BMC Execute Command ${cmd} 1497 1498 1499Verify Identify LED State 1500 [Documentation] Verify the identify LED state 1501 ... matches caller's expectations. 1502 [Arguments] ${expected_state} 1503 1504 # Description of argument(s): 1505 # expected_state The LED state expected by the caller ("Blink" or "Off"). 1506 1507 ${resp}= Read Attribute ${LED_PHYSICAL_URI}/front_id State 1508 Should Be Equal ${resp} 1509 ... xyz.openbmc_project.Led.Physical.Action.${expected_state} 1510 ... msg=Unexpected LED state. 1511 1512 ${resp}= Read Attribute ${LED_PHYSICAL_URI}/rear_id State 1513 Should Be Equal ${resp} 1514 ... xyz.openbmc_project.Led.Physical.Action.${expected_state} 1515 ... msg=Unexpected LED state. 1516 1517 1518Verify The Attribute 1519 [Arguments] ${uri} ${attribute_name} ${attribute_value} 1520 1521 # Description of argument(s): 1522 # uri URI path 1523 # (e.g. "/xyz/openbmc_project/control/host0/TPMEnable"). 1524 # attribute_name Name of attribute to be verified (e.g. "TPMEnable"). 1525 # attribute_value The expected value of attribute (e.g. "1", "0", etc.) 1526 1527 ${output}= Read Attribute ${uri} ${attribute_name} 1528 Should Be Equal ${attribute_value} ${output} 1529 ... msg=Attribute "${attribute_name} does not have the expected value. 1530