1*** Settings ***
2Documentation  Test for HW CI.
3
4Library             DateTime
5
6Resource            ../../lib/utils.robot
7Resource            ../../lib/ipmi_client.robot
8Resource            ../../lib/boot_utils.robot
9Resource            ../../lib/openbmc_ffdc.robot
10Resource            ../../lib/bmc_redfish_resource.robot
11
12Test Setup          Test Setup Execution
13Test Teardown       FFDC On Test Case Fail
14
15*** Variables ***
16
17# Error strings to check from journald.
18${ERROR_REGEX}     SEGV|core-dump|FAILURE|Failed to start|Found ordering cycle
19${STANDBY_REGEX}   Startup finished in
20
21${SKIP_ERROR}      ${EMPTY}
22
23# 3 minutes standby boot time.
24${startup_time_threshold}  180
25
26*** Test Cases ***
27
28Verify Application Services Running At Standby
29    [Documentation]  Check if there are services that have not completed.
30    [Tags]  Verify_Application_Services_Running_At_Standby
31
32    # Application services running on the BMC are not tightly coupled.
33    # At standby, there shouldn't be any pending job waiting to complete.
34    # Examples:
35    # Failure o/p:
36    # root@witherspoon:~# systemctl list-jobs --no-pager | cat
37    #    JOB UNIT                                     TYPE  STATE
38    # 35151 xyz.openbmc_project.ObjectMapper.service start running
39    # 1 jobs listed.
40    #
41    # Success o/p:
42    # root@witherspoon:~# systemctl list-jobs --no-pager | cat
43    # No jobs running.
44
45    Redfish Hard Power Off
46    ${stdout}  ${stderr}  ${rc}=  BMC Execute Command
47    ...  systemctl list-jobs --no-pager | cat
48    Should Be Equal As Strings  ${stdout}  No jobs running.
49
50
51Verify Front And Rear LED At Standby
52    [Documentation]  Front and Rear LED should be off at standby.
53    [Tags]  Verify_Front_And_Rear_LED_At_Standby
54
55    Redfish Power Off  stack_mode=skip  quiet=1
56    Verify Identify LED State  ${0}
57
58
59Check For Application Failures
60    [Documentation]  Parse the journal log and check for failures.
61    [Tags]  Check_For_Application_Failures
62
63    Check For Regex In Journald  ${ERROR_REGEX}  error_check=${0}  boot=-b  filter_string=${SKIP_ERROR}
64
65
66Verify Uptime Average Against Threshold
67    [Documentation]  Compare BMC average boot time to a constant threshold.
68    [Tags]  Verify_Uptime_Average_Against_Threshold
69
70    Redfish OBMC Reboot (off)
71
72    Wait Until Keyword Succeeds
73    ...  1 min  30 sec  Check BMC Uptime Journald
74
75
76Test SSH And IPMI Connections
77    [Documentation]  Try SSH and IPMI commands to verify each connection.
78    [Tags]  Test_SSH_And_IPMI_Connections
79
80    BMC Execute Command  true
81    Run IPMI Standard Command  chassis status
82
83
84*** Keywords ***
85
86Test Setup Execution
87   [Documentation]  Do test case setup tasks.
88
89   Printn
90   Redfish.Login
91
92
93Check BMC Uptime Journald
94    [Documentation]  Check BMC journald uptime entry.
95
96    # Example output:
97    # Startup finished in 10.074s (kernel) + 2min 23.506s (userspace) = 2min 33.581s.
98    ${startup_time}  ${stderr}  ${rc}=  BMC Execute Command
99    ...  journalctl --no-pager | egrep '${STANDBY_REGEX}' | tail -1
100    Should Not Be Empty  ${startup_time}
101
102    # Example time conversion:
103    # Get the "2min 33.581s" string total time taken to reach standby.
104    # Convert time "2min 33.581s" to unit 153.581.
105    ${startup_time}=  Convert Time  ${startup_time.split("= ",1)[1].strip(".")}
106
107    Should Be True  ${startup_time} < ${startup_time_threshold}
108    ...  msg=${startup_time} greater than threshold value of ${startup_time_threshold}.
109