Using AsyncResp everywhere

Get the core using AsyncResp everywhere, and not have each individual handler
creating its own object.We can call app.handle() without fear of the response
getting ended a

Using AsyncResp everywhere

Get the core using AsyncResp everywhere, and not have each individual handler
creating its own object.We can call app.handle() without fear of the response
getting ended after the first tree is done populating.
Don't use res.end() anymore.

Tested:
1. Validator passed.

Signed-off-by: zhanghaicheng <zhanghch05@inspur.com>
Change-Id: I867367ce4a0caf8c4b3f4e07e06c11feed0782e8

show more ...

Fix nlohmann::json::dump calls

The nlohmann::json::dump call needs to be called with specific arguments
to avoid throwing in failure cases. http connection already does this
properly, but a bunch o

Fix nlohmann::json::dump calls

The nlohmann::json::dump call needs to be called with specific arguments
to avoid throwing in failure cases. http connection already does this
properly, but a bunch of code has snuck in (mostly in redfish) that
ignores this, and calls it incorrectly. This can potentially lead to a
crash if the wrong thing throws on invalid UTF8 characters.

This audits the whole codebase, and replaces every dump() call with the
correct dump(2, ' ', true, nlohmann::json::error_handler_t::replace)
call. For correct output, the callers should expect no change, and in
practice, this would require injecting non-utf8 characters into the
BMC.

Tested:
Ran several of the endpoints/error conditions in question, including
some of the error cases. Observed correct responses. I don't know of a
security issue that would allow injecting invalid utf8 into the BMC, but
in theory if it were possible, this would prevent a crash.

Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: I4a15b8e260e3db129bc20484ade4ed5449f75ad0

show more ...

Add MutualExclusiveProperties registry

Add MutualExclusiveProperties message registry entry
and error message.
As per redfish specification, "RegistryPrefixes" and
"MessageIds" are mutually exclusiv

Add MutualExclusiveProperties registry

Add MutualExclusiveProperties message registry entry
and error message.
As per redfish specification, "RegistryPrefixes" and
"MessageIds" are mutually exclusive. So add check for
same in EventService and return MutualExclusiveProperties
error message.

Tested:
- Create subscription failed with error(bad request)
when the request body contain both "RegistryPrefixes"
and "MessageIds".

Change-Id: I4c14f946977bce2ced8a7f96eb85855117fde9a8
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

Revert http::Request::socket() callback

Details on why this revert is needed are here.

https://lists.ozlabs.org/pipermail/openbmc/2020-August/022478.html

Appu and Ravi still have not commented.

I

Revert http::Request::socket() callback

Details on why this revert is needed are here.

https://lists.ozlabs.org/pipermail/openbmc/2020-August/022478.html

Appu and Ravi still have not commented.

It should be noted, this also causes a memory leak in http connection,
where connections refuse to be freed, because of a bad usage of
shared_from_this.

This code wasn't very well thought through, and needs rearchitected to
not break the unit testability of bmcweb, nor cause memory leaks.

https://github.com/openbmc/bmcweb/blob/218bd4746130aac22366968c8c9a34a929e45a3d/http/http_connection.h#L351

Is the memory leak in question.

Specifically, this reverts:
The /attachment download in LogServices. This needs reimplemented
properly, but is an OEM property, so it shouldn't be a big deal to
revert, and shouldn't break our redfish compliance.

The IpAddress property in SessionService. I have no idea why this was
injected, and it's functionally incorrect. IpAddresses are not related
to a session, and IP addresses can change over the course of a session,
so this property is already broken as written. I suspect the author
really wanted RedfishEvent type logging, but that was too complex, so
they half implemented this.

Redfish SSE properties. This needs to be reimplemented similar to the
patchset here:
https://gerrit.openbmc-project.xyz/c/openbmc/bmcweb/+/13948
Where the ownership of the HTTP connection does not leave the http
framework. As written, the SSE implementation causes ownership issues,
as there's no clear delineation of the ownership between HttpConnection
and the SSE framework.

Tested:
On current master, running this command:
wget -O- --no-http-keep-alive --no-check-certificate https://{bmc
hostname}:18080/redfish/v1
Which should download the service root, then immediately close and
destroy the connection, prints:

(2020-08-28 16:55:24) [DEBUG "routing.h":1258] Matched rule
'/redfish/v1/' 2 / 4
(2020-08-28 16:55:24) [DEBUG "http_response.h":130] calling completion
handler
(2020-08-28 16:55:24) [DEBUG "http_response.h":133] completion handler
was valid
(2020-08-28 16:55:24) [INFO "http_connection.h":429] Response: 0x1e1ee28
/redfish/v1 200 keepalive=0
(2020-08-28 16:55:24) [DEBUG "timer_queue.h":48] timer add inside:
0x1d3d1a8 7
(2020-08-28 16:55:24) [DEBUG "http_connection.h":751] 0x1e1ee28 timer
added: 0x1d3d1a8 7
(2020-08-28 16:55:24) [DEBUG "http_connection.h":655] 0x1e1ee28 doWrite
(2020-08-28 16:55:24) [DEBUG "http_connection.h":663] 0x1e1ee28
async_write 1555 bytes
(2020-08-28 16:55:24) [DEBUG "http_connection.h":697] 0x1e1ee28 timer
cancelled: 0x1d3d1a8 7
(2020-08-28 16:55:24) [DEBUG "http_connection.h":676] 0x1e1ee28 from
write(1)

Then stops. Note, that the connection was not destroyed, and has
leaked. Once this patchset is added, the connection closes and destroys
properly, and doesn't leak, so it prints the above, but also prints.

(2020-08-28 16:27:10) [DEBUG "http_connection.h":305] 0x1d15c90
Connection closed, total 1

Ran Redfish service validator. Saw one unrelated failure due to UUID,
all other things pass.

Signed-off-by: Ed Tanous <ed@tanous.net>
Change-Id: I18686037bf58f20389d31facc0d77020274d38a1

show more ...

Enable unused variable warnings and resolve

This commit enables the "unused variables" warning in clang. Throughout
this, it did point out several issues that would've been functional
bugs, so I th

Enable unused variable warnings and resolve

This commit enables the "unused variables" warning in clang. Throughout
this, it did point out several issues that would've been functional
bugs, so I think it was worthwhile. It also cleaned up several unused
variable from old constructs that no longer exist.

Tested:
Built with clang. Code no longer emits warnings.

Downloaded bmcweb to system and pulled up the webui, observed webui
loads and logs in properly.

Change-Id: I51505f4222cc147d6f2b87b14d7e2ac4a74cafa8
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Enable clang warnings

This commit enables clang warnings, and fixes all warnings that were
found. Most of these fall into a couple categories:

Variable shadow issues were fixed by renaming variabl

Enable clang warnings

This commit enables clang warnings, and fixes all warnings that were
found. Most of these fall into a couple categories:

Variable shadow issues were fixed by renaming variables

unused parameter warnings were resolved by either checking error codes
that had been ignored, or removing the name of the variable from the
scope.

Other various warnings were fixed in the best way I was able to come up
with.

Note, the redfish Node class is especially insidious, as it causes all
imlementers to have variables for parameters, regardless of whether or
not they are used. Deprecating the Node class is on my list of things
to do, as it adds extra overhead, and in general isn't a useful
abstraction. For now, I have simply fixed all the handlers.

Tested:
Added the current meta-clang meta layer into bblayers.conf, and added
TOOLCHAIN_pn-bmcweb = "clang" to my local.conf

Signed-off-by: Ed Tanous <ed@tanous.net>
Change-Id: Ia75b94010359170159c703e535d1c1af182fe700

show more ...

Remove middlewares

Middlewares, while kinda cool from an academic standpoint, make our
build times even worse than they already are. Given that we only really
use 1 real middleware today (token aut

Remove middlewares

Middlewares, while kinda cool from an academic standpoint, make our
build times even worse than they already are. Given that we only really
use 1 real middleware today (token auth) and it needs to move into the
parser mode anyway (for security limiting buffer sizes), we might as well
use this as an opportunity to delete some code.

Some other things that happen:
1. Persistent data now moves out of the crow namespace
2. App is no longer a template
3. All request_routes implementations no longer become templates. This
should be a decent (unmeasured) win on compile times.

This commit was part of a commit previously called "various cleanups".
This separates ONLY the middleware deletion part of that.

Note, this also deletes about 400 lines of hard to understand code.

Change-Id: I4c19e25491a153a2aa2e4ef46fc797bcb5b3581a
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Fix: MetricReportDefinitions filter not working

The metric reports are not sending when user configures
the MetricReportDefinitions filter. This is of odata json
object type. Corrected code to prope

Fix: MetricReportDefinitions filter not working

The metric reports are not sending when user configures
the MetricReportDefinitions filter. This is of odata json
object type. Corrected code to properly handle odata type
object and store it as string array to make filters faster.

Tested:
- Created metric report EventService subscription type
with MetricReportDefinitions and events properly sent to
Event listener.

Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>
Change-Id: If96564219da7d38a2ee5e415b89824ba25cd686d

show more ...

Remove QueryString

QueryString is an error-prone library that was
leftover from crow. Replace it with boost::url,
a header only library based and written by the
one of the authors of boost beast.

T

Remove QueryString

QueryString is an error-prone library that was
leftover from crow. Replace it with boost::url,
a header only library based and written by the
one of the authors of boost beast.

Tested: Verified logging paging still worked
as expected

Change-Id: I47c225089aa7d0f7d2299142f91806294f879381
Signed-off-by: James Feist <james.feist@linux.intel.com>

show more ...

EventService: Add the parameter ResourceTypes to subscription

This commit supports sending the ResourceTypes list while subscribing
to the events. The "Task" resource is added as a supported type to

EventService: Add the parameter ResourceTypes to subscription

This commit supports sending the ResourceTypes list while subscribing
to the events. The "Task" resource is added as a supported type to receive
the task life cycle events.
For IBM's management console along with the Task resource, the support
is provided to subscribe to the "IBMConfigFile" ResourceType to receive
events while creating/updating the ConfigFiles.

Tested by:
1. GET https://${bmc}/redfish/v1/EventService
2. Create subscription :
POST https://${bmc}/redfish/v1/EventService/Subscriptions
-d '{"Destination" : <>, "Protocol":"Redfish", "ResourceTypes": ["Task"]}'
3. GET https://${bmc}/redfish/v1/EventService/Subscriptions/<id>
3. Redfish validator was run successfully

Signed-off-by: Sunitha Harish <sunharis@in.ibm.com>
Change-Id: Ibaf3f4f5f005a1beedf0a1cd049ae11d93a3af36

show more ...

Codespell redfish-core spelling fixes

These spelling errors were found using
https://github.com/codespell-project/codespell
Tested: Top commit (along with this) was built and ran against
val

Codespell redfish-core spelling fixes

These spelling errors were found using
https://github.com/codespell-project/codespell
Tested: Top commit (along with this) was built and ran against
validator.
Change-Id: Ic9dce27b1de8567eedf7753164ef564d3aedf8ca
Signed-off-by: Gunnar Mills <gmills@us.ibm.com>

show more ...

Support the $filter query params for SSE stream

This commit adds the support for $filter query
paramater in the URI for SSE stream.

Method: GET
URI: /redfish/v1/EventService/Subscriptions/SSE?
$fil

Support the $filter query params for SSE stream

This commit adds the support for $filter query
paramater in the URI for SSE stream.

Method: GET
URI: /redfish/v1/EventService/Subscriptions/SSE?
$filter=(MessageIds%20eq%20DCPowerOn) or
(MessageIds%20eq%20DCPowerOn)

Tested:
- From browser sent request using SSE URI along with filter query param
- query params were read and parsed successfully.
- used SubmitTestEvent and could see test events coming to BMC.
- Ran redfish validator successfully.
- Performed GET on Subscription collections and Subscription/<id> URI
and checked for valid data.

Change-Id: Ie18546749495175ede918ab933ff8dd1d65b775f
Signed-off-by: Ayushi Smriti <smriti.ayushi@linux.intel.com>
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

EventService: Support for Server Sent Events(SSE)

Add support for Server Sent Events(SSE)
Filters support is not part of this commit.

Tested:
- GET on URI /redfish/v1/EventService/Subscriptions/SS

EventService: Support for Server Sent Events(SSE)

Add support for Server Sent Events(SSE)
Filters support is not part of this commit.

Tested:
- GET on URI /redfish/v1/EventService/Subscriptions/SSE/
from chrome browser, can see all BMC Events on browser.
- Redfish validator is successful.

Change-Id: Icd10cdad20c4529f64c97b67d46f2e4a7e0c329c
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

EventService: Add retry configuration support

This commit is to pass configuration parameters: retry attempts,
retry interval secs and retry policy to http client and take
required delivery retry po

EventService: Add retry configuration support

This commit is to pass configuration parameters: retry attempts,
retry interval secs and retry policy to http client and take
required delivery retry policy action.

Also, perform async wait for retryTimeoutInterval before each
retry attempts.

Tested:
- Set and verified config properties by sending PATCH req on
EventService and EventDestination uri.
- Verified the appropriate delivery retry policy action block reached.
- Verified the async_wait logic by triggering retry case depending
failed state of connection.
- could see a wait for timeout interval before next retry.

Signed-off-by: Ayushi Smriti <smriti.ayushi@linux.intel.com>
Change-Id: Id1366fca59dc9e6543c553bfe5df95a59f468bc7
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

EventService: Add enable/disable support

Add EventService enable/disable support.
When EventService is enabled
- Check for no of event log subscribers and then
only process events async sending.

EventService: Add enable/disable support

Add EventService enable/disable support.
When EventService is enabled
- Check for no of event log subscribers and then
only process events async sending.
- Check for no of metric report subscribers
and register for metric report signal.
When EventService is disabled
- Discard the inotify event for redfish logs.
- Unregister the metric report signal.

Tested:
- Modified ServieEnabled, DeliveryRetryAttempts,
DeliveryRetryInterval values using patch and
it reflects on subsequent gets.
- Above mentioned functionality tested with Service
enabled & disabled modifications.
- Ran redfish validator successfully.

Change-Id: Id049860a89d3040d859ac8907e7bad5b4209b73d
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

EventService: Add MetricReport support

Add Telemetry metric report support to EventService.
- Adding MetricReport support to schema implemenation.
- Dynamically register and unregister the metric

EventService: Add MetricReport support

Add Telemetry metric report support to EventService.
- Adding MetricReport support to schema implemenation.
- Dynamically register and unregister the metric report
signal.
- Reads Telemtry data using D-Bus calls.
- Filter the metric reports depending on user configured
MetricReportDefinition.
- Format the Telemetry readings as per MetricReport schema.
- Send the formatted data to the client.

Tested:
- HTTP client successfully received asynchronous metric
reports data.
- valdiated the register and unregister by adding and
deleting subscriptions.
- Ran Redfish validator successfully.

Change-Id: I7b59ac3ecad169a7959a800730dbc2fe85baf068
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

EventService: SubmitTestEvent initial support

This commit adds SubmitTestEvent initial support to
send out test events to subscribers.

URI:
/redfish/v1/EventService/Actions/EventService.SubmitTestE

EventService: SubmitTestEvent initial support

This commit adds SubmitTestEvent initial support to
send out test events to subscribers.

URI:
/redfish/v1/EventService/Actions/EventService.SubmitTestEvent

Tested:
- Client subscribed to event listener via destination uri.
After sending POST request on SubmitTestEvent uri, could see
generated test event.
- Successfully ran the redfish validator.
Counter({'metadataNamespaces': 1739, 'pass': 26,
'skipOptional': 22, 'serviceNamespaces': 3, 'passGet': 3,
'passAction': 1})
Validation has succeeded.

Change-Id: I16e02c1977e99af39317070567196767ac7c7400
Signed-off-by: Ayushi Smriti <smriti.ayushi@linux.intel.com>

show more ...

EventService: Manager and subscriptions support

Add EventService Manager which will manage all the
EventService configuration and subscriptions. This
includes API for add or update or delete subscri

EventService: Manager and subscriptions support

Add EventService Manager which will manage all the
EventService configuration and subscriptions. This
includes API for add or update or delete subscriptions
along with other supported API support. Also includes
http connection open and send event code using
"push style eventing".
Added BMCWEB_INSECURE_HTTP_PUSH_STYLE_EVENTING
build flag to enable/disable http push style eventing
which is not a secure channel.

Tested:
- Tested along with other patches such as http
client and Event log support, SubmitTestEvent
and its works.
- Ran Redfish validation successfully.

Change-Id: Ie4687e4cbfabd525b7a8ad4e615510f034edc6e9
Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>

show more ...

Redfish EventService schema implementation

Add Redfish EventService schema support for
EventService - GET and PATCH methods.
EventDestinationCollections - GET and POST methods.
EventDestination - GE

Redfish EventService schema implementation

Add Redfish EventService schema support for
EventService - GET and PATCH methods.
EventDestinationCollections - GET and POST methods.
EventDestination - GET, PATCH and DELETE methods.

URI's:
/redfish/v1/EventService
/redfish/v1/EventService/Subscriptions
/redfish/v1/EventService/Subscriptions/<id>

Tested:
- Validated all default event config data using GET.
- Validated supported/unsupported properties change.
- Validated range parameters for retry and timeout.
- Added new subscription using POST and validated using GET.
- Modified subscription using PATCH and validated.
- Validated delete subscription.
- Validated negative case for eventTypes, RegistryPrefixes,
mandate properties for POST etc.
- Successfully ran the redfish validator tool.
Counter({'metadataNamespaces': 1739, 'pass': 24,
'skipOptional': 23, 'passGet': 3,
'serviceNamespaces': 3})
Validation has succeeded.

Signed-off-by: AppaRao Puli <apparao.puli@linux.intel.com>
Change-Id: I220de2cb85e73124753d95b7ee311f1c30e2cce4

show more ...