Add Redfish interoperability validator

This change is a proposal to add Redfish interop validator
which is also a DMTF tool to validate the Redfish service's
conformance. This validator validate the

Add Redfish interoperability validator

This change is a proposal to add Redfish interop validator
which is also a DMTF tool to validate the Redfish service's
conformance. This validator validate the Redfish service
based on an interoperability profile given to it. I've also
added a profile for validating the minimal requirements now,
we could populate more requirements based on this profile.
The benefit of bringing this tool is the user could use
this tool to validate their own implementation with their
own interoperability profile.

Signed-off-by: Jordan Chen <jordanchen@google.com>
Change-Id: Ic00738c349276c6874a4f850cad71b56f9aa8582

show more ...

Fix missing kewyord in update service suite.

Note: This is still a mix of both REST legacy and redfish.

Redfish Signed Code Update :: BMC/Host signed code update over fun... | FAIL |
Several failu

Fix missing kewyord in update service suite.

Note: This is still a mix of both REST legacy and redfish.

Redfish Signed Code Update :: BMC/Host signed code update over fun... | FAIL |
Several failures occurred:

1) No keyword with name 'Reboot BMC And Verify BMC Image' found.

2) No keyword with name 'Poweron Host And Verify Host Image' found.


Change-Id: I86bcaa101e25ca3ec5f1182ea4bad92a8d26aaad
Signed-off-by: Sushil Singh <susilsi7@in.ibm.com>

show more ...

Moved below keyword from certificate suite to utils

- Modify BMC Date

Change-Id: I9abafec83f78373aeff0c14653edf612cfd542ea
Signed-off-by: Rahul Maheshwari <rahulmaheshwari@in.ibm.com>

Add support to code update same image on current and alternate side

Changes:
- Add new test case to perform code update.

Future commit will need few more checks on version.

Change-Id: Iada7030

Add support to code update same image on current and alternate side

Changes:
- Add new test case to perform code update.

Future commit will need few more checks on version.

Change-Id: Iada7030baf7dd48e396dbb89892f590d17768962
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Fix tag mismatch in test suites

Fixes the following suites:

$ awk -f check_robot_tags.awk redfish/*/* openpower/*/*
--- redfish/account_service/test_user_account.robot:
Redfish Create and Verify

Fix tag mismatch in test suites

Fixes the following suites:

$ awk -f check_robot_tags.awk redfish/*/* openpower/*/*
--- redfish/account_service/test_user_account.robot:
Redfish Create and Verify Admin User
Redfish_Create_and_Verify_Users

--- openpower/test_timed_power_on.robot:
Set And Return Timer For Power ON
Set_Time_For_Power_ON

Change-Id: I6fa3d4f323233b81cccae0ad68de28523cdb48db
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Fixed user ID can be blocked

The keyword IPMI Create Random User Plus Password And Privilege
always gave the new user a user ID of 2. This results in a fail
if another user (e.g. root, or other adm

Fixed user ID can be blocked

The keyword IPMI Create Random User Plus Password And Privilege
always gave the new user a user ID of 2. This results in a fail
if another user (e.g. root, or other admin) already occupies
user ID=2.

Instead, keyword Find Free User Id is applied to generate a random
user ID by selecting from those user IDs that are not occupied.
If no user ID is free, the test fails and reports
"A free user ID could not be found."

Signed-off-by: Gene Ratzlaff <generatz@gmail.com>
Change-Id: I60d6332541814f5cc88674ed33bfc57d8e6afe45

show more ...

Duplicate testcase name removal from test suite

+ redfish/account_service/test_user_account.robot
E: 103, 0: Duplicate testcase name 'Verify Redfish Operator User with Wrong Password' (DuplicateTest

Duplicate testcase name removal from test suite

+ redfish/account_service/test_user_account.robot
E: 103, 0: Duplicate testcase name 'Verify Redfish Operator User with Wrong Password' (DuplicateTestNames)

Change-Id: I81aefd2d6c40802a811b22d357ce2b58f5ed5b6a
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Seperated different users test into multiple test cases

Change-Id: I170187b47cc877c833ec998f1af4b8e20009fbd9
Signed-off-by: Rahul Maheshwari <rahulmaheshwari@in.ibm.com>

systems: eventlog: fix Host state cannot match expected match_state

Symptom:
Boot failures exceed the boot failure threshold. The states no longer match.
state:
[boot_progress]:

systems: eventlog: fix Host state cannot match expected match_state

Symptom:
Boot failures exceed the boot failure threshold. The states no longer match.
state:
[boot_progress]: Unspecified
[host]: Running
[operating_system]: Inactive
match_state:
[boot_progress]: FW Progress, Starting OS|OSStart
[operating_system]: Standby
[host]: Running

Root cause:
In test "test_event_logging.robot" that use "Trigger Host Watchdog Error" cause
Host be rebooted by watchdog. However, in "test_led_indicator_asserted.robot"
that will call "Redfish Power ${pre_req_state}" to keep Host at previous state
then perform the test. Sometimes, this results the host report is not be booted
even if it actually is. Eventually, cause test case confusion then return error

Solution:
For fixing this false alarm wait few seconds for Host be booted properly

Tested:
Run robot test redfish/systems/eventlog/test_event_logging.robot then
run redfish/systems/test_led_indicator_asserted.robot continuity

Signed-off-by: Tim Lee <timlee660101@gmail.com>
Change-Id: I2f7e93df9b430cc889d8f0b09ac86b9acaf039df

show more ...

Moved a keyword from certificate suite to utils

Change-Id: I51935be6e1cf16cbffc75f7ccb3d05ceaf692a59
Signed-off-by: Rahul Maheshwari <rahulmaheshwari@in.ibm.com>

Added new testcases to protocol settings sub menu

- Enable SSH and IPMI protocol and verify SSH and IPMI commands work
- Disable SSH and IPMI protocol and verify SSH and IPMI commands stop w

Added new testcases to protocol settings sub menu

- Enable SSH and IPMI protocol and verify SSH and IPMI commands work
- Disable SSH and IPMI protocol and verify SSH and IPMI commands stop working
- Enable SSH and Disable IPMI and verify SSH command works & IPMI command stop working

Change-Id: I19d41694a24708fccfffd90aa7ce275a6d9ca69e
Signed-off-by: meghagn <Megha.GN@ibm.com>

show more ...

Replace CSP plugin-types directive with object-src

The HTTP Content-Security-Policy (CSP) plugin-types directive has been
marked as deprecated and suggests set object-src none to disallow
plugins.

Replace CSP plugin-types directive with object-src

The HTTP Content-Security-Policy (CSP) plugin-types directive has been
marked as deprecated and suggests set object-src none to disallow
plugins.

Signed-off-by: Brian Ma <chma0@nuvoton.com>
Change-Id: Ie532deeb8d8cb0d1de275b5d864bd20dc5f11ce7

show more ...

Update test documentation

Change-Id: I79271d5dde7380a2472ce859861c44f87ae6f41c
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

fix System event log entry is not empty

Symptom:
When running full automation test cases, the test case
"Reboot And Check Ambient Temperature Records Are Valid" sometimes got failed.
The error log i

fix System event log entry is not empty

Symptom:
When running full automation test cases, the test case
"Reboot And Check Ambient Temperature Records Are Valid" sometimes got failed.
The error log is "System event log entry is not empty".

Root cause:
The keyword "Get Thermal Records and Verify" will check whether there is any
error event log when checking ambient temperature records are valid.
However, all temperature records are valid in test result.
But, seems there are other unrelated system error event logs already exist.
Thus, cause this keyword return failed.

Solution:
Prevent this unexpected redfish event log when doing some test items.
We should clear redfish event log to make sure the error event is not cause
by current test item.

Tested:
Run robot test redfish/systems/test_thermal_ambient_temperatures.robot.

Signed-off-by: Tim Lee <timlee660101@gmail.com>
Change-Id: Id1d9d9bd2307f51be67e284f64373622b147cc5c

show more ...

systems: thermal: use redfish error log checking

Symptom:
Auto test got failed from redfish/systems/test_thermal_ambient_temperatures.robot
Error is "Unexpected BMC error log(s) present.: 200 != 404

systems: thermal: use redfish error log checking

Symptom:
Auto test got failed from redfish/systems/test_thermal_ambient_temperatures.robot
Error is "Unexpected BMC error log(s) present.: 200 != 404" and result is FAIL
Get Thermal Records and Verify | FAIL |
Reboot And Check Ambient Temperature Records Are Valid | FAIL |

Root cause:
In Test Setup that already change to use redfish error log purging.
But, "Get Thermal Records and Verify" still using legacy method
"Error Logs Should Not Exist" then cause unexpected error log present.

Solution:
Using redfish error log checking "Event Log Should Not Exist" instead.

Tested:
Run robot test redfish/systems/test_thermal_ambient_temperatures.robot.
Get Thermal Records and Verify | PASS |
Reboot And Check Ambient Temperature Records Are Valid | PASS |

Signed-off-by: Tim Lee <timlee660101@gmail.com>
Change-Id: I6d68001fb04abb989d492cc14198677e4c540132

show more ...

Fix for non-root IPMI user delete in redfish account service

Change-Id: I77c550d2c106ac1a514334c3084a4d734bf71e0d
Signed-off-by: Ashwini Chandrappa <Ashwini.Chandrappa777@ibm.com>

Added comment to refer code_update.md file
to generate bad firmware image for testing

Change-Id: Id96f018383d88f974686e1916a64c672861170b0
Signed-off-by: Sushil Singh <susilsi7@in.ibm.com>

Add Test teardown directive for code update suite

Changes:

- Added test teardown
- Added checks for input parameter needed

Resolves openbmc/openbmc-test-automation#2183

Change-Id: I96

Add Test teardown directive for code update suite

Changes:

- Added test teardown
- Added checks for input parameter needed

Resolves openbmc/openbmc-test-automation#2183

Change-Id: I96f56b76bfc7ac38c26a21adc1d0491fdc949a92
Signed-off-by: Sushil Singh <susilsi7@in.ibm.com>

show more ...

More Bytes MAC Address Valid

Change in Test Case as Valid scenario

Change-Id: I85f77e24caf76086245ffa0387cc5d2d1942c39f
Signed-off-by: Sweta Potthuri <spotthur@in.ibm.com>

Fix for usr/sbin/faillock file not found failure.

Added check for faillock file, if present it execute faillock cmd
else execute pam_tally2 cmd to reset the accountlock.

Signed-off-by: Nagarjun B

Fix for usr/sbin/faillock file not found failure.

Added check for faillock file, if present it execute faillock cmd
else execute pam_tally2 cmd to reset the accountlock.

Signed-off-by: Nagarjun B <nagarjunb@ami.com>
Change-Id: I2df6dd9e729be149e9631b844e8197915f0fba69

show more ...

Move test suite test_occ_power.robot to openpower

The OCC is the OpenPOWER On Chip Controller.

Resolves openbmc/openbmc-test-automation#2182

Change-Id: I8484dcef0e08d1fd53c397cb38bce973513e3f15
S

Move test suite test_occ_power.robot to openpower

The OCC is the OpenPOWER On Chip Controller.

Resolves openbmc/openbmc-test-automation#2182

Change-Id: I8484dcef0e08d1fd53c397cb38bce973513e3f15
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Move test suite test_bmc_devicetree.robot to openpower

This is BMC specific but cater to openpower systems like witherspoon.

Resolves openbmc/openbmc-test-automation#2181

Change-Id: Ic3cfc9cb06e

Move test suite test_bmc_devicetree.robot to openpower

This is BMC specific but cater to openpower systems like witherspoon.

Resolves openbmc/openbmc-test-automation#2181

Change-Id: Ic3cfc9cb06e26076ca2b33399078ef47d2e3e0d0
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Update Redfish-Reference-Checker and Redfish-Service-Validator suites

Changes:
- lib/dmtf_tools_utils.robot to return the RC
- Update test code to handle rc returnning

Resolves openbmc/ope

Update Redfish-Reference-Checker and Redfish-Service-Validator suites

Changes:
- lib/dmtf_tools_utils.robot to return the RC
- Update test code to handle rc returnning

Resolves openbmc/openbmc-test-automation#2178
Resolves openbmc/openbmc-test-automation#2179

Change-Id: Ia9c6f3ea10ed2a25c31cf9c8af84680a441fc954
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Update DMTF Refish use case checker test suite

Changes:
- Add logic to print errors in test suite
- Update the directory where log is collected
- Fix typo for IPMI suite

Resolves o

Update DMTF Refish use case checker test suite

Changes:
- Add logic to print errors in test suite
- Update the directory where log is collected
- Fix typo for IPMI suite

Resolves openbmc/openbmc-test-automation#2177

Change-Id: Ice7bb4a7b154529e904a501802ef22cf583963ed
Signed-off-by: George Keishing <gkeishin@in.ibm.com>

show more ...

Fix Tag mismatch and typo in the test suite

Change-Id: I3c01d1604198cc9cd3825a6e09c445fcba1bc5a3
Signed-off-by: George Keishing <gkeishin@in.ibm.com>