1*** Settings *** 2 3Documentation Test BMC dump functionality of OpenBMC. 4 5Resource ../../lib/bmc_redfish_resource.robot 6Resource ../../lib/boot_utils.robot 7Resource ../../lib/dump_utils.robot 8Resource ../../lib/openbmc_ffdc.robot 9 10Suite Setup Redfish.Login 11Test Setup Redfish Delete All BMC Dumps 12Test Teardown Test Teardown Execution 13 14*** Variables *** 15 16# Total size of the dump in kilo bytes 17${BMC_DUMP_TOTAL_SIZE} ${1024} 18 19# Minimum space required for one bmc dump in kilo bytes 20${BMC_DUMP_MIN_SPACE_REQD} ${20} 21${MAX_DUMP_COUNT} ${20} 22${BMC_DUMP_COLLECTOR_PATH} /var/lib/phosphor-debug-collector/dumps 23 24*** Test Cases *** 25 26Verify Error Response For Already Deleted Dump Id 27 [Documentation] Delete non existing BMC dump and expect an error. 28 [Tags] Verify_Error_Response_For_Already_Deleted_Dump_Id 29 30 Redfish Power Off stack_mode=skip 31 ${dump_id}= Create User Initiated BMC Dump Via Redfish 32 Redfish Delete BMC Dump ${dump_id} 33 Run Keyword And Expect Error ValueError: * Redfish Delete BMC Dump ${dump_id} 34 35 36Verify User Initiated BMC Dump When Host Powered Off 37 [Documentation] Create user initiated BMC dump at host off state and 38 ... verify dump entry for it. 39 [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Powered_Off 40 41 Redfish Power Off stack_mode=skip 42 ${dump_id}= Create User Initiated BMC Dump Via Redfish 43 ${dump_entries}= Get BMC Dump Entries 44 Length Should Be ${dump_entries} 1 45 List Should Contain Value ${dump_entries} ${dump_id} 46 47 48Verify User Initiated BMC Dump Size 49 [Documentation] Verify user initiated BMC dump size is under 20 MB. 50 [Tags] Verify_User_Initiated_BMC_Dump_Size 51 52 ${dump_id}= Create User Initiated BMC Dump Via Redfish 53 ${resp}= Redfish.Get Properties /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_id} 54 55 # Example of response from above Redfish GET request. 56 # "@odata.type": "#LogEntry.v1_7_0.LogEntry", 57 # "AdditionalDataSizeBytes": 31644, 58 # "AdditionalDataURI": "/redfish/v1/Managers/bmc/LogServices/Dump/attachment/9", 59 # "Created": "2020-10-23T06:32:53+00:00", 60 # "DiagnosticDataType": "Manager", 61 # "EntryType": "Event", 62 # "Id": "9", 63 # "Name": "BMC Dump Entry" 64 65 # Max size for dump is 20 MB = 20x1024x1024 Byte. 66 Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520 67 68 69Verify Multiple BMC Dump Creation 70 [Documentation] Verify that multiple BMC dumps can be created one after 71 ... another successfully. 72 [Tags] Verify_Multiple_BMC_Dump_Creation 73 74 ${dump_count}= Evaluate random.randint(5, 10) modules=random 75 FOR ${INDEX} IN 1 ${dump_count} 76 Create User Initiated BMC Dump Via Redfish 77 END 78 79 80Verify BMC Dump Default Location In BMC 81 [Documentation] Verify that BMC dump is created in its default location of BMC. 82 [Tags] Verify_BMC_Dump_Default_Location_In_BMC 83 84 Redfish Delete All BMC Dumps 85 ${dump_id}= Create User Initiated BMC Dump Via Redfish 86 ${dump_file} ${stderr} ${rc}= BMC Execute Command 87 ... ls ${BMC_DUMP_COLLECTOR_PATH}/${dump_id} 88 Should Be True ${rc} == 0 89 Should Start With ${dump_file} BMCDUMP 90 91 92Verify User Initiated BMC Dump When Host Booted 93 [Documentation] Create user initiated BMC dump at host booted state and 94 ... verify dump entry for it. 95 [Tags] Verify_User_Initiated_BMC_Dump_When_Host_Booted 96 97 Redfish Power On stack_mode=skip 98 ${dump_id}= Create User Initiated BMC Dump Via Redfish 99 ${dump_entries}= Get BMC Dump Entries 100 Length Should Be ${dump_entries} 1 101 List Should Contain Value ${dump_entries} ${dump_id} 102 103 104Verify Dump Persistency On Dump Service Restart 105 [Documentation] Create user dump, restart dump manager service and verify dump 106 ... persistency. 107 [Tags] Verify_Dump_Persistency_On_Dump_Service_Restart 108 109 Create User Initiated BMC Dump Via Redfish 110 ${dump_entries_before}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries 111 112 # Restart dump service. 113 BMC Execute Command systemctl restart xyz.openbmc_project.Dump.Manager.service 114 Sleep 10s reason=Wait for BMC dump service to restart properly 115 116 ${dump_entries_after}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries 117 Lists Should Be Equal ${dump_entries_before} ${dump_entries_after} 118 119 120Verify Dump Persistency On BMC Reset 121 [Documentation] Create user dump, reset BMC and verify dump persistency. 122 [Tags] Verify_Dump_Persistency_On_BMC_Reset 123 124 # Power off host so that dump is not offloaded to host OS. 125 Redfish Power Off stack_mode=skip 126 127 Create User Initiated BMC Dump Via Redfish 128 ${dump_entries_before}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries 129 130 # Reset BMC. 131 OBMC Reboot (off) stack_mode=skip 132 133 ${dump_entries_after}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries 134 Lists Should Be Equal ${dump_entries_before} ${dump_entries_after} 135 136 137Delete User Initiated BMC Dump And Verify 138 [Documentation] Delete user initiated BMC dump and verify. 139 [Tags] Delete_User_Initiated_BMC_Dump_And_Verify 140 141 ${dump_id}= Create User Initiated BMC Dump Via Redfish 142 Redfish Delete BMC Dump ${dump_id} 143 144 ${dump_entries}= Get BMC Dump Entries 145 Should Be Empty ${dump_entries} 146 147 148Delete All User Initiated BMC Dumps And Verify 149 [Documentation] Delete all user initiated BMC dumps and verify. 150 [Tags] Delete_All_User_Initiated_BMC_Dumps_And_Verify 151 152 # Power off host so that dump is not offloaded to host OS. 153 Redfish Power Off stack_mode=skip 154 155 # Create some BMC dump. 156 Create User Initiated BMC Dump Via Redfish 157 Create User Initiated BMC Dump Via Redfish 158 159 Redfish Delete All BMC Dumps 160 ${dump_entries}= Get BMC Dump Entries 161 Should Be Empty ${dump_entries} 162 163 164Create Two User Initiated BMC Dumps 165 [Documentation] Create two user initiated BMC dumps. 166 [Tags] Create_Two_User_Initiated_BMC_Dumps 167 168 ${dump_id1}= Create User Initiated BMC Dump Via Redfish 169 ${dump_id2}= Create User Initiated BMC Dump Via Redfish 170 171 ${dump_entries}= Get BMC Dump Entries 172 Length Should Be ${dump_entries} 2 173 Should Contain ${dump_entries} ${dump_id1} 174 Should Contain ${dump_entries} ${dump_id2} 175 176 177Create Two User Initiated BMC Dumps And Delete One 178 [Documentation] Create two dumps and delete the first. 179 [Tags] Create_Two_User_Initiated_BMC_Dumps_And_Delete_One 180 181 ${dump_id1}= Create User Initiated BMC Dump Via Redfish 182 ${dump_id2}= Create User Initiated BMC Dump Via Redfish 183 184 Redfish Delete BMC Dump ${dump_id1} 185 186 ${dump_entries}= Get BMC Dump Entries 187 Length Should Be ${dump_entries} 1 188 List Should Contain Value ${dump_entries} ${dump_id2} 189 190 191Create And Delete User Initiated BMC Dump Multiple Times 192 [Documentation] Create and delete user initiated BMC dump multiple times. 193 [Tags] Create_And_Delete_User_Initiated_BMC_Dump_Multiple_Times 194 195 FOR ${INDEX} IN 1 10 196 ${dump_id}= Create User Initiated BMC Dump Via Redfish 197 Redfish Delete BMC Dump ${dump_id} 198 END 199 200 201Verify Maximum BMC Dump Creation 202 [Documentation] Create maximum BMC dump and verify error when dump runs out of space. 203 [Tags] Verify_Maximum_BMC_Dump_Creation 204 [Teardown] Redfish Delete All BMC Dumps 205 206 # Maximum allowed space for dump is 1024 KB. BMC typically hold 8-14 dumps 207 # before running out of this dump space. So trying to create dumps in 20 208 # iterations to run out of space. 209 # User can key in the Maximum allowed space for bmc dump and how many iteration. 210 FOR ${n} IN RANGE 0 ${MAX_DUMP_COUNT} 211 Create User Initiated BMC Dump Via Redfish 212 ${dump_space}= Get Disk Usage For Dumps 213 Exit For Loop If ${dump_space} >= (${BMC_DUMP_TOTAL_SIZE} - ${BMC_DUMP_MIN_SPACE_REQD}) 214 END 215 216 # Check error while creating dump when dump size is full. 217 ${payload}= Create Dictionary DiagnosticDataType=Manager 218 Redfish.Post /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData 219 ... body=${payload} valid_status_codes=[${HTTP_INTERNAL_SERVER_ERROR}] 220 221 222Verify BMC Core Dump When Host Powered Off 223 [Documentation] Verify BMC core dump after application crash at host powered off state. 224 [Tags] Verify_BMC_Core_Dump_When_Host_Powered_Off 225 226 Redfish Power Off stack_mode=skip 227 228 # Ensure all dumps are cleaned out. 229 Redfish Delete All BMC Dumps 230 Trigger Core Dump 231 232 # Verify that BMC dump is available. 233 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available 234 235 236Verify Core Dump Size 237 [Documentation] Verify BMC core dump size is under 20 MB. 238 [Tags] Verify_Core_Dump_Size 239 240 Redfish Power Off stack_mode=skip 241 242 # Ensure all dumps are cleaned out. 243 Redfish Delete All BMC Dumps 244 Trigger Core Dump 245 246 # Verify that BMC dump is available. 247 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available 248 ${dump_entries}= Get BMC Dump Entries 249 ${resp}= Redfish.Get Properties 250 ... /redfish/v1/Managers/bmc/LogServices/Dump/Entries/${dump_entries[0]} 251 252 # Max size for dump is 20 MB = 20x1024x1024 Byte. 253 Should Be True 0 < ${resp["AdditionalDataSizeBytes"]} < 20971520 254 255 256Verify Error While Initiating BMC Dump During Dumping State 257 [Documentation] Verify error while initiating BMC dump during dumping state. 258 [Tags] Verify_Error_While_Initiating_BMC_Dump_During_Dumping_State 259 260 ${task_id}= Create User Initiated BMC Dump Via Redfish ${1} 261 262 # Check error while initiating BMC dump while dump in progress. 263 ${payload}= Create Dictionary DiagnosticDataType=Manager 264 Redfish.Post 265 ... /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData 266 ... body=${payload} valid_status_codes=[${HTTP_SERVICE_UNAVAILABLE}] 267 268 # Wait for above initiated dump to complete. Otherwise, on going dump would impact next test. 269 Wait Until Keyword Succeeds 5 min 15 sec Check Task Completion ${task_id} 270 271 272Verify BMC Dump Create Errors While Another BMC Dump In Progress 273 [Documentation] Verify BMC dump creation error until older BMC dump completion. 274 [Tags] Verify_BMC_Dump_Create_Errors_While_Another_BMC_Dump_In_Progress 275 276 # Initiate a BMC dump that returns without completion. 277 ${task_id}= Create User Initiated BMC Dump Via Redfish ${1} 278 279 # Now continue to initiate multiple dump request which is not expected to be accepted 280 # till earlier BMC dump task is completed. A limit is set to avoid risk of infinite loop. 281 ${payload}= Create Dictionary DiagnosticDataType=Manager 282 WHILE True limit=1000 283 ${task_dict}= Redfish.Get Properties /redfish/v1/TaskService/Tasks/${task_id} 284 IF '${task_dict['TaskState']}' == 'Completed' BREAK 285 Redfish.Post 286 ... /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData 287 ... body=${payload} valid_status_codes=[${HTTP_SERVICE_UNAVAILABLE}] 288 END 289 290 # The next BMC dump initiation request should be accepted as earlier dump is completed. 291 ${resp}= Redfish.Post 292 ... /redfish/v1/Managers/bmc/LogServices/Dump/Actions/LogService.CollectDiagnosticData 293 ... body=${payload} valid_status_codes=[${HTTP_ACCEPTED}] 294 295 # Wait for above initiated dump to complete. Otherwise, on going dump would impact next test. 296 Wait Until Keyword Succeeds 5 min 15 sec Check Task Completion ${resp.dict['Id']} 297 298 299Verify Core Dump After Terminating Dump Manager Service 300 [Documentation] Verify initiate core dumps and kill Phosphor-dump-manager. 301 [Tags] Verify_Core_Dump_After_Terminating_Dump_Manager_Service 302 303 Redfish Power Off stack_mode=skip 304 305 # Remove all available dumps in BMC. 306 Redfish Delete All BMC Dumps 307 308 # Find the pid of the phosphor-dump-manage process and kill it. 309 ${cmd_buf}= Catenate kill -s SEGV $(pgrep phosphor-dump-manager) 310 ${cmd_output} ${stderr} ${rc}= BMC Execute Command ${cmd_buf} 311 Should Be Equal As Integers ${rc} ${0} 312 313 # Verify that BMC dump is available. 314 Wait Until Keyword Succeeds 2 min 10 sec Is BMC Dump Available 315 316 # Verifing that there is only one dump. 317 ${dump_entries}= Get BMC Dump Entries 318 ${length}= Get length ${dump_entries} 319 Should Be Equal As Integers ${length} ${1} 320 321 322*** Keywords *** 323 324Get BMC Dump Entries 325 [Documentation] Return BMC dump ids list. 326 327 ${dump_uris}= redfish_utils.get_member_list /redfish/v1/Managers/bmc/LogServices/Dump/Entries 328 ${dump_ids}= Create List 329 330 FOR ${dump_uri} IN @{dump_uris} 331 ${dump_id}= Fetch From Right ${dump_uri} / 332 Append To List ${dump_ids} ${dump_id} 333 END 334 335 [Return] ${dump_ids} 336 337 338Is BMC Dump Available 339 [Documentation] Verify if BMC dump is available. 340 341 ${dump_entries}= Get BMC Dump Entries 342 343 # Verifying that BMC dump is available. 344 ${length}= Get length ${dump_entries} 345 Should Be True 0 < ${length} 346 347 348Get Disk Usage For Dumps 349 [Documentation] Return disk usage in kilobyte for BMC dumps. 350 351 ${usage_output} ${stderr} ${rc}= BMC Execute Command du -s ${BMC_DUMP_COLLECTOR_PATH} 352 353 # Example of output from above BMC cli command. 354 # $ du -s /var/lib/phosphor-debug-collector/dumps 355 # 516 /var/lib/phosphor-debug-collector/dumps 356 357 ${usage_output}= Fetch From Left ${usage_output} / 358 ${usage_output}= Convert To Integer ${usage_output} 359 360 [return] ${usage_output} 361 362 363Test Teardown Execution 364 [Documentation] Do test teardown operation. 365 366 FFDC On Test Case Fail 367 Close All Connections 368