Aggregation: Generate collections from schema

In aggregation we need to account for when a satellite BMC supports a
resource collection that the aggregating BMC does not. We need to
add links in re

Aggregation: Generate collections from schema

In aggregation we need to account for when a satellite BMC supports a
resource collection that the aggregating BMC does not. We need to
add links in responses from upstream resources to indicate that these
satellite only resources exist. These top level collections do not
always have the location format of /redfish/v1/<collection_id>. We
determine all of those from the schema rather than hardcoding a few
choice exceptions that do not fit that format such as
/redfish/v1/UpdateService/FirmwareInventory.

This patch is the first step in accomplishing this. We parse all
xml schema files in order to determine what are the top level
collection URIs. For URIs that are upstream from top level collections
we'll be able to derive what URIs could appear in the response which
are also on the same path to a top level collection.

The xml files for resources that aren't currently supported by bmcweb
are ignored when creating the schema index and later deleted once the
above parsing completes.

Later patches will use this information to make sure that we are
properly handling all schema defined top level collections.

Tested:
Compiles with aggregation enabled

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: Icff825f4e4e9d84c96254561350f82839abdc074

show more ...

Use owning strings for aggregator URI

Clang correctly notes that, because the segments() object is an rvalue,
and is destroyed after the std::string_view is used on the next line,
this is technicall

Use owning strings for aggregator URI

Clang correctly notes that, because the segments() object is an rvalue,
and is destroyed after the std::string_view is used on the next line,
this is technically undefined behavior. Make these use owning
std::strings instead of std::string_view to avoid the lifetime problem.

Tested:
Enabled aggregation without including a satellite config. Sending a
GET request to /redfish/v1/Chassis/5B247A_Test returned a 404 due to
hitting one of the sections of changed code. The other section is not
currently reachable, but uses the same type of change.

Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: I1b2b2ad444e2718c0cb167f4506ea1d0915b3a5b
Signed-off-by: Carson Labrado <clabrado@google.com>

show more ...

Prepare for boost::url upgrade

The new boost URL now interops properly with std::string_view, which is
great, and cleans up a bunch of mediocre code to convert one to another.
It has also been pulle

Prepare for boost::url upgrade

The new boost URL now interops properly with std::string_view, which is
great, and cleans up a bunch of mediocre code to convert one to another.
It has also been pulled into boost-proper, so we no longer need a
boost-url dependency that's separate.

Unfortunately, boost url makes these improvements by changing
boost::string_view for boost::urls::const_string, which causes us to
have some compile errors on the missing type.

The bulk of these changes fall into a couple categories, and have to be
executed in one commit.
string() is replaced with buffer() on the url and url_view types
boost::string_view is replaced by std::string_view for many times, in
many cases removing a temporary that we had in the code previously.

Tested: Code compiles with boost 1.81.0 beta.
Redfish service validator passes.
Pretty good unit test coverage for URL-specific use cases.

Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: I8d3dc89b53d1cc390887fe53605d4867f75f76fd

show more ...

Aggregation: Detect and fix all URI properties

There are a number of properties of Type "string (uri)" for which we
do not currently support adding prefixes. This patch adds support
for all existin

Aggregation: Detect and fix all URI properties

There are a number of properties of Type "string (uri)" for which we
do not currently support adding prefixes. This patch adds support
for all existing URI properties which are missed by the existing
implementation.

This change will be needed by future patches which will expand
aggregation support to all top level collections defined by the
schema. Those collections that are not currently supported include
properties whose URIs should be fixed, but would be missed by the
existing implementation.

Tested:
New unit test passes.
URI properties are still handled correctly.

```shell
curl localhost/redfish/v1/Chassis/5B247A_<chassisID>
{
"@odata.id": "/redfish/v1/Chassis/5B247A_<chassisID>",
"@odata.type": "#Chassis.v1_16_0.Chassis",
"Actions": {
"#Chassis.Reset": {
"@Redfish.ActionInfo": "/redfish/v1/Chassis/5B247A_<chassisID>/ResetActionInfo",
"target": "/redfish/v1/Chassis/5B247A_<chassisID>/Actions/Chassis.Reset"
}
},
...
}
```

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: I3b3e06ee3191564d266598f7bc9f1641e6fcb333

show more ...

Aggregation: Fix header clobbering

We attempt to sanitize the response body of the asyncResp object as
the first step of copying over the satellite response into the
object. This is clobbering the

Aggregation: Fix header clobbering

We attempt to sanitize the response body of the asyncResp object as
the first step of copying over the satellite response into the
object. This is clobbering the "OData-Version" from the response
header.

We can completely remove this step since the body will already be
empty.

Tested:
OData-Version is still present in the header when querying a satellite
resource
curl -s -D- localhost/redfish/v1/Chassis/5B247A_test | grep OData
OData-Version: 4.0

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: I82f54ae9b990504a3236b7a1273c794b1ee6f090

show more ...

treewide: change EM's ObjectManager path

EntityManager moves its ObjectManager in commit [1], this patch is to
change accordingly. Please see [1] for why we made that change.

[1] https://gerrit.ope

treewide: change EM's ObjectManager path

EntityManager moves its ObjectManager in commit [1], this patch is to
change accordingly. Please see [1] for why we made that change.

[1] https://gerrit.openbmc.org/c/openbmc/entity-manager/+/57279

Tested: code compiles.

Signed-off-by: Nan Zhou <nanzhoumails@gmail.com>
Change-Id: Idf5657463d54022f53d12e860483d06b5c5c7ccd

show more ...

Aggregation: Ignore JsonSchemas and fix prefixes

We don't want to aggregate JsonSchemas as-is since it can introduce
problems related to inconsistent versions between the aggregating and
satellite B

Aggregation: Ignore JsonSchemas and fix prefixes

We don't want to aggregate JsonSchemas as-is since it can introduce
problems related to inconsistent versions between the aggregating and
satellite BMCs. For now we will just assume that the aggregating BMC
will match all satellite BMCs in terms of schemas and versions.

There was also an edge case where we are not adding prefixes to "Uri"
keys. These are used by Registries resources. Now we make a case-
insensitive check to see if a key ends with "uri" in order to
determine if we need to add the prefix to the resource ID.

Tested:
Requests to /redfish/v1/JsonSchemas only show schemas on the
aggregating BMC.

Responses from /redfish/v1/Registries/<id> now contain the
aggregation prefix in the value associated with the "Uri" key.
~# curl localhost/redfish/v1/Registries/5B247A_TaskEvent
{
...
"Location": [
{
...
"Uri": "/redfish/v1/Registries/5B247A_TaskEvent/TaskEvent"
}
],
...
}

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: I935785740c05ad0ac3e8c682a72ae1d1419054a8

show more ...

Aggregation: Improve handling of certain requests

This patch cleans up a few edge cases that aren't handled properly.

We need to break out of the aggregation code earlier when there are
no satellit

Aggregation: Improve handling of certain requests

This patch cleans up a few edge cases that aren't handled properly.

We need to break out of the aggregation code earlier when there are
no satellite configs. The logs are showing mixed messages of
Aggregation not being enabled due to no found satellite configs
followed by processing the request anyway until we fail to actually
find a satellite BMC to forward the request to.

When we don't have any satellite configs, but a request is sent to
what should be a valid satellite URI such as
/redfish/v1/Chassis/5B247A_ChassisID then we need to make sure we
return a 404 within the aggregation code since we won't locally
handle the request. We don't have to worry about collection
requests since by design we will also locally handle the request.

This patch is also prep to allow forwarding non-GET requests to
resources that are not supported by BMCWeb. The aggregation code
will get to handle all such requests and we need to make sure that
we do not forward non-GET requests to top level collections.

Tested:
Without any satellite configs the aggregation code exited before
it began trying to send a request to all satellites for
/redfish/v1/Chassis. The same occurred for a request for a satellite
resource. In the latter case the aggregation code also returned a
404.

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: Idd1a71ebb485a77795ba47b873624c8e53c36a4c

show more ...

Add SSL support for http_client (EventService)

This commit adds the initial SSL support for http_client which can be
used for sending asynchronous Events/MetricReports to subscribed Event
Listener s

Add SSL support for http_client (EventService)

This commit adds the initial SSL support for http_client which can be
used for sending asynchronous Events/MetricReports to subscribed Event
Listener servers over secure channel.

Current implementation of http client only works for http protocol.
With current implementation, http client can be configured to work
with secure http (HTTPS). As part of implementation it adds the SSL
handshake mechanism and enforces the peer ceritificate verification.

The http-client uses the cipher suites which are supported by mozilla
browser and as recommended by OWASP. For better security enforcement
its disables the SSLv2, SSLv3, TLSv1, TLSv1.1 as described in below
OWASP cheetsheet.

It is validated with RootCA certificate(PEM) for now. Adding support
for different certificates can be looked in future as need arises.

[1]: https://cheatsheetseries.owasp.org/cheatsheets/TLS_Cipher_String_Cheat_Sheet.html

Tested:
- Created new subscription with SSL destination(https) and confirmed
that events are seen on EventListener side.
URI: /redfish/v1/EventService/Subscriptions
Method: POST
Body:
{
"Context": "CustomText",
"Destination": "https://<IP>:4000/service/collector/event_logs",
"EventFormatType": "Event",
"DeliveryRetryPolicy": "RetryForever",
"Protocol": "Redfish"
}

- Unit tested the non-SSL connection by disabling the check in code
(Note: EventService blocks all Non-SSL destinations). Verified that
all events are properly shown on EventListener.
URI: /redfish/v1/EventService/Subscriptions
Method: POST
Body:
{
"Context": "CustomText",
"Destination": "http://<IP>:4001/service/collector/event_logs",
"EventFormatType": "Event",
"Protocol": "Redfish"
}

- Combined above two tests and verified both SSL & Non-SSL work fine in
congention.

- Created subscription with different URI paths on same IP, Port and
protocol and verified that events sent as expected.

Change-Id: I13b2fc942c9ce6c55cd7348aae1e088a3f3d7fd9
Signed-off-by: AppaRao Puli <apparao.puli@intel.com>
Signed-off-by: Ed Tanous <edtanous@google.com>

show more ...

Aggregation: Reduce Retry Policy

Reduces the number of retry attempts to 1. The aggregating BMC
cannot generate a response until the retry policy has been exhausted.
We want to minimize the amount

Aggregation: Reduce Retry Policy

Reduces the number of retry attempts to 1. The aggregating BMC
cannot generate a response until the retry policy has been exhausted.
We want to minimize the amount of time it takes for the aggregating
BMC to respond in the event of an unreachable satellite BMC.

Also explicity sets Redfish Aggregation's retry policy action as
"TerminateAfterRetries". Previously it relied on this being the
default action.

Tested:
Requests sent to unreachable satellite BMC only attempted to resend
a single time.

Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: If1819389affc96f49908d586459b03b1bb2689c2

show more ...

Redfish Aggregation: Aggregate Collections

Adds aggregation support for resource collections that take the form
of "/redfish/v1/<resource collection>". Collection URIs are
identified by the precens

Redfish Aggregation: Aggregate Collections

Adds aggregation support for resource collections that take the form
of "/redfish/v1/<resource collection>". Collection URIs are
identified by the precense of a "Members" array in the response.

Resources from satellite BMCs are added to the "Members" array of
the response and the "Members@odata.count" value is updated to
denote the new array size.

These satellite resource URIs that are added also include the
prefix associated with that satellite.

Note that as a first step this patch assumes a single satellite BMC.
There are some potential race conditions that could occur for setups
with multiple satellite BMCs. This has been commented in the code
and is better left to its own patch.

Tested:
Queried various collection URIs and the aggregated resources
appeared in the response's "Members" array.

Querying 'localhost:80/redfish/v1/Chassis?$expand=.($levels=1)'
resulted in $expand correctly returning the outputs from querying
the URIs of all local and satellite Chassis resources. This would
have failed if the satellite Chassis resources were omitted from the
"Members" array or the satellite's prefix was not correctly added to
the URI.

Also queried a collection URI that only existed on the satellite BMC.
The AsyncResp was completely overwritten by the response from the
satellite BMC.

Queries to non-collection URIs resulted in no attempts to add
satellite responses to the AsyncResp.

Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: I3b379cd57e5a121eb4a344d88fc8e43170ca78a6

show more ...

Redfish Aggregation: Fixup aggregated URIs

URIs in the responses returned with Redfish Aggregation enabled will
potentially be incorrect since ones from satellite BMCs will not
include the associate

Redfish Aggregation: Fixup aggregated URIs

URIs in the responses returned with Redfish Aggregation enabled will
potentially be incorrect since ones from satellite BMCs will not
include the associated prefix such as "5B247A_" in the resource ID
portion of the URIs.

This patch fixes those links so that they include their BMC's
associated prefix. Note that a future patch will be needed to add
prefixes to aggregated resources that would appear under collection
URIs such as "/redfish/v1/Chassis".

Tested:
Requests were sent to URIs associated with the aggregating BMC and a
satellite BMC denoted as "5B247A". The URIs in the responses
were successfully updated such that "5B247A_" was added for
satellite resources.

Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: Ib4f976fab1ca1e8603f7cf55292732ffb71cd03e

show more ...

Redfish Aggregation: Router to satellite resources

Adds ability to route requests to either native resources or
resources that belong to a satellite BMC as part of Redfish
Aggregation. A prefix in

Redfish Aggregation: Router to satellite resources

Adds ability to route requests to either native resources or
resources that belong to a satellite BMC as part of Redfish
Aggregation. A prefix in the URI denotes if the resource is actually
from a satellite BMC. Prefixes are only used to denote satellite
resources. The URI of resources on the local/aggregating BMC will
remain unchanged.

Prefixes are separated from the resource ID by an underscore. This
means that underscores cannot be used in the prefix name itself.
The prefixes used by satellite BMCs are revealed via D-Bus as well as
the config information needed to connect to that BMC.

Requests for satellite resources will not be handled locally.

Care should be taken to not name any local resources in a way that
could cause a collision (e.g. having a Chassis object named
"aggregated0_1U" on the aggregating BMC).

The patch only covers routing requests. Requests to collection URIs
like /redfish/v1/Chassis will only return resources on the local BMC.
A future patch will cover adding satellite resources to collections.

Also note that URIs returned in the responses will not have the proper
prefix included. Fixing these URIs will be addressed in future
patches.

A number of TODO comments are included in the code to indicate that
this functionality (collections and URI fixup) still needs to be
implemented.

Example URIs w/o Redfish Aggregation:
/redfish/v1/Chassis/1U/
/redfish/v1/Systems/system/
/redfish/v1/Managers/bmc/

Example URIs after enabling Redfish Aggregation if the associated
resources are located on the local/aggregating BMC:
/redfish/v1/Chassis/1U/
/redfish/v1/Systems/system/
/redfish/v1/Managers/bmc/

Example URIs if resources are instead located on a satellite BMC
named "aggregated0":
/redfish/v1/Chassis/aggregated0_1U/
/redfish/v1/Systems/aggregated0_system/
/redfish/v1/Managers/aggregated0_bmc/

Tested:
I was able to query supported resources located on the local BMC
as well as on a satellite BMC. Requests with unknown prefixes return
a 404. Requests to resource collections only return the resources
that are located on the aggregating BMC.

Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: I87a3deb730bda95e72ecd3144ea40b0e5ee7d491

show more ...

Aggregation: Prepare for routing requests

We do not want to allow a HW config to set its own prefix since that
results in HW choosing and hardcoding resource URIs. Removes using
"Name" from the sat

Aggregation: Prepare for routing requests

We do not want to allow a HW config to set its own prefix since that
results in HW choosing and hardcoding resource URIs. Removes using
"Name" from the satellite config as the config's prefix.

For now assume there will be no more than one satellite bmc. We will
always assign that config to be "aggregated0". If more than one
config is present then we will not attempt to forward any requests.
In a future patch we will add support for aggregating multiple
satellite BMCs. The aggregator will be responsible for assigning the
prefixes to each satellite.

When we receive a request we parse the resource ID to see if it
begins with "aggregated" and thus should be forwarded to a satellite
BMC. In those cases we should not locally handle the request. We
return a 500 error, but in a future patch that will be replaced by
the actual code to forward the request to the appropriate satellite.

Requests for resource collections need to be both handled locally and
forwarded. Place holders are added for where the forwarding will
occur. A future patch will add that functionality.

Tested:
Exposed two configs in an entity-manager json:
"Exposes": [
{
"Hostname": "127.0.0.1",
"Port": "443",
"Name": "Sat1",
"Type": "SatelliteController",
"AuthType": "None"
},
{
"Hostname": "127.0.0.1",
"Port": "444",
"Name": "Sat2",
"Type": "SatelliteController",
"AuthType": "None"
},

It produced an error that only one satellite is supported and as a
result both configs were ignored. I removed the second config and
that resulted in the first (and only) config being added as
"aggregated0".

Requests for local resources were ignored by the aggregation code.
Requests for collections hit the forward collection endpoints and
return local results.

500 returned for satellite resources such as:
/redfish/v1/Chassis/aggregated0_Fake
/redfish/v1/UpdateService/FirmwareInventory/aggregated0_Fake
/redfish/v1/UpdateService/SoftwareInventory/aggregated0_Fake

Change-Id: I5c860c01534e7d5b1a37c95f75be5b3c1f695816
Signed-off-by: Carson Labrado <clabrado@google.com>
Signed-off-by: Ed Tanous <edtanous@google.com>

show more ...

clang: fix extra semicolon

Failed with -Wextra-semi.

Tested: no -Wextra-semi when build with clang++

Signed-off-by: Nan Zhou <nanzhoumails@gmail.com>
Change-Id: Ib0c0b3a2b0fcfe0e415987baa18f810e4b

clang: fix extra semicolon

Failed with -Wextra-semi.

Tested: no -Wextra-semi when build with clang++

Signed-off-by: Nan Zhou <nanzhoumails@gmail.com>
Change-Id: Ib0c0b3a2b0fcfe0e415987baa18f810e4b19b89f

show more ...

bmcweb: Set Retry Policy Valid Response Codes

Allows individual retry policies to specify what HTTP response codes
are considered valid. Sets functions for the EventService and
Redfish Aggregation

bmcweb: Set Retry Policy Valid Response Codes

Allows individual retry policies to specify what HTTP response codes
are considered valid. Sets functions for the EventService and
Redfish Aggregation retry policies. Those functions expect a
response code and return an error code based on what the response
code is.

This change is needed because EventService only considers 2XX codes
to be valid. Any code outside of that range would trigger a retry
attempt. Redfish Aggregation by design will need to return
errors outside of that range such as 404. It should not retry to
send a message when it receives a 404 from a satellite BMC.

Right now 404 is the only error code that is handled differently
between the services. Going forward, Redfish Aggregation will
likely want to allow other error codes as its functionality is
expanded.

Tested:
Used Redfish-Event-Listener with ssh port forwarding to create 3
subscriptions. I then closed the ssh connection and sent a test
event. Bmcweb made 3 retry attempts for each subscription. At
that point the max retry amount (as defined by EventService) was
reached and bmcweb stop attempting to resend the messages.

There were no errors when the Redfish-Event-Listener was correctly
connected. Test events resulted in messages being sent for each
subscription.

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: Ifdfaf638d28982ed18998f3ca05280a288e0020a

show more ...

Try to fix the lambda formatting issue

clang-tidy has a setting, LambdaBodyIndentation, which it says:
"For callback-heavy code, it may improve readability to have the
signature indented two levels

Try to fix the lambda formatting issue

clang-tidy has a setting, LambdaBodyIndentation, which it says:
"For callback-heavy code, it may improve readability to have the
signature indented two levels and to use OuterScope."

bmcweb is very callback heavy code. Try to enable it and see if that
improves things. There are many cases where the length of a lambda call
will change, and reindent the entire lambda function. This is really
bad for code reviews, as it's difficult to see the lines changed. This
commit should resolve it. This does have the downside of reindenting a
lot of functions, which is unfortunate, but probably worth it in the
long run.

All changes except for the .clang-format file were made by the robot.

Tested: Code compiles, whitespace changes only.

Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: Ib4aa2f1391fada981febd25b67dcdb9143827f43

show more ...

bmcweb: Fetch Satellite Config from D-Bus

Adds a RedfishAggregator class which is able to pull configuration
information from D-Bus for Satellite BMCs. These BMCs will be
aggregated by Redfish Aggr

bmcweb: Fetch Satellite Config from D-Bus

Adds a RedfishAggregator class which is able to pull configuration
information from D-Bus for Satellite BMCs. These BMCs will be
aggregated by Redfish Aggregation. Also added is a new compiler
option which will be used to enable Redfish Aggregation.

This patch only allows configurations with unencrypted and
unauthenticated satellite BMC communication. Support for encryption
and authentication willneed to be added in future patches.

Note that this patch does not actually use the config information
after it has been fetched. That functionality will be added in
future patches.

Tested:
I made this example config information available on D-Bus
busctl introspect xyz.openbmc_project.EntityManager \
/xyz/openbmc_project/inventory/system/board/SatelliteBMC/aggregated0 \
xyz.openbmc_project.Configuration.SatelliteController
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
.AuthType property s "None" emits-change
.Hostname property s "127.0.0.1" emits-change
.Name property s "aggregated0" emits-change
.Port property t 443 emits-change
.Type property s "SatelliteController" emits-change

That information was picked up by the changes in this CL:
[DEBUG "redfish_aggregator.hpp":80] Found Satellite Controller at /xyz/openbmc_project/inventory/system/board/SatelliteBMC/aggregated0
[DEBUG "redfish_aggregator.hpp":209] Added satellite config aggregated0 at http://127.0.0.1:443
[DEBUG "redfish_aggregator.hpp":52] Redfish Aggregation enabled with 1 satellite BMCs
[DEBUG "redfish_aggregator.hpp":21] There were 1 satellite configs found at startup

Signed-off-by: Carson Labrado <clabrado@google.com>
Change-Id: Ib5eee2c93aeb209157191055975c127759d73627

show more ...