Break out sse into a compile unit

Verify similar to beb96b0 Break out websockets

Break out the SSE functions into a separate compile unit. This allows
the SSE sockets in beast to be compiled separ

Break out sse into a compile unit

Verify similar to beb96b0 Break out websockets

Break out the SSE functions into a separate compile unit. This allows
the SSE sockets in beast to be compiled separately, which significantly
reduces the overall compile time by a few seconds. Code is identical
with the exceptions of minor header definitions to convert header-only
to compile unit.

Change-Id: I5aae4f17cbd2badf75b3e0bb644a2309f6300663
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Implement http2 TODO

To support HTTP2 simultaneously on http and https connections, the HTTP
connection classes formerly took the socket as a template option,
allowing passing ssl::stream<tcp::socke

Implement http2 TODO

To support HTTP2 simultaneously on http and https connections, the HTTP
connection classes formerly took the socket as a template option,
allowing passing ssl::stream<tcp::socket> or simply tcp socket. With
the addition of the multiple-sockets option, this would cause two copies
of the template to be instantiated, increasing both compile times and
binary size.

This commit applies the same logic to http2connection as was applied to
HTTPConnection, adding an http type parameter to the constructor, which
allows switching between adapter and adapter.next_level() on each read
or write operation. In compiled code, this means that the connection
classes are only specialized once.

Tested:
When configured for one of each http and https socket and http2
curl --http2 http://<ip>/redfish/v1
succeeds
curl --http2 https://<ip>/redfish/v1 succeeds

Change-Id: I8f33796edd5874d5b93d10a3f253cfadd4f6d7a4
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Enable HTTP additional sockets

This commit attempts to add the concept of an SSL detector from beast,
and add the capability into bmcweb. This allows directing multiple
socket files to the bmcweb i

Enable HTTP additional sockets

This commit attempts to add the concept of an SSL detector from beast,
and add the capability into bmcweb. This allows directing multiple
socket files to the bmcweb instance, and bmcweb will automatically sort
out whether or not they're SSL, and give the correct response. This
allows users to plug in erroneous urls like "https://mybmc:80" and they
will forward and work correctly.

Some key design points:
The HTTP side of bmcweb implements the exact same http headers as the
HTTPS side, with the exception of HSTS, which is explicitly disallowed.
This is for consistency and security.

The above allows bmcweb builds to "select" the appropriate security
posture (http, https, or both) for a given channel using the
FileDescriptorName field within a socket file. Items ending in:
both: Will support both HTTPS and HTTP redirect to HTTPS
https: Will support HTTPS only
http: will support HTTP only

Given the flexibility in bind statements, this allows administrators to
support essentially any security posture they like. The openbmc
defaults are:
HTTPS + Redirect on both ports 443 and port 80 if http-redirect is
enabled

And HTTPS only if http-redirect is disabled.

This commit adds the following meson options that each take an array of
strings, indexex on the port.
additional-ports
Adds additional ports that bmcweb should listen to. This is always
required when adding new ports.

additional-protocol
Specifies 'http', 'https', or 'both' for whether or not tls is enfoced
on this socket. 'both' allows bmcweb to detect whether a user has
specified tls or not on a given connection and give the correct
response.

additional-bind-to-device
Accepts values that fill the SO_BINDTODEVICE flag in systemd/linux,
and allows binding to a specific device

additional-auth
Accepts values of 'auth' or 'noauth' that determines whether this
socket should apply the normal authentication routines, or treat the
socket as unauthenticated.

Tested:
Previous commits ran the below tests.
Ran the server with options enabled. Tried:
```
curl -vvvv --insecure --user root:0penBmc http://192.168.7.2/redfish/v1/Managers/bmc
* Trying 192.168.7.2:80...
* Connected to 192.168.7.2 (192.168.7.2) port 80 (#0)
* Server auth using Basic with user 'root'
> GET /redfish/v1/Managers/bmc HTTP/1.1
> Host: 192.168.7.2
> Authorization: Basic cm9vdDowcGVuQm1j
> User-Agent: curl/7.72.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Location: https://192.168.7.2
< X-Frame-Options: DENY
< Pragma: no-cache
< Cache-Control: no-Store,no-Cache
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Content-Security-Policy: default-src 'none'; img-src 'self' data:; font-src 'self'; style-src 'self'; script-src 'self'; connect-src 'self' wss:
< Date: Fri, 08 Jan 2021 01:43:49 GMT
< Connection: close
< Content-Length: 0
<
* Closing connection 0
```

Observe above:
webserver returned 301 redirect.
webserver returned the appropriate security headers
webserver immediately closed the connection.

The same test above over https:// returns the values as expected

Loaded the webui to test static file hosting. Webui logs in and works
as expected.

Used the scripts/websocket_test.py to verify that websockets work.
Sensors report as expected.

Change-Id: Ib5733bbe5473fed6e0e27c56cdead0bffedf2993
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Implement urlsafe base64 decode

base64 decoding comes in two flavors, "normal" which we already
implement, and "url safe" which modifies the alphabet to create base64
encodings that are safe to use

Implement urlsafe base64 decode

base64 decoding comes in two flavors, "normal" which we already
implement, and "url safe" which modifies the alphabet to create base64
encodings that are safe to use in filenames and urls. Functionally this
just involves swapping two characters with underscore and minus in the
encode/decode table. To avoid duplicating a lot of code, this commit
refactors the base64 tables to be generated at compile time.

Tested: Included unit tests pass. No usage until next commit.

Change-Id: I71724fd2e04000f115c22a40d382d411986d7b39
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

clang-format: update latest spec and reformat

Copy the latest format file from the docs repository and apply.

Change-Id: I2f0b9d0fb6e01ed36a2f34c750ba52de3b6d15d1
Signed-off-by: Patrick Williams <p

clang-format: update latest spec and reformat

Copy the latest format file from the docs repository and apply.

Change-Id: I2f0b9d0fb6e01ed36a2f34c750ba52de3b6d15d1
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

show more ...

Fix includes

Clang-tidy misc-include-cleaner appears to now be enforcing
significantly more headers than previously. That is overall a good
thing, but forces us to fix some issues. This commit is

Fix includes

Clang-tidy misc-include-cleaner appears to now be enforcing
significantly more headers than previously. That is overall a good
thing, but forces us to fix some issues. This commit is largely just
taking the clang-recommended fixes and checking them in. Subsequent
patches will fix the more unique issues.

Note, that a number of new ignores are added into the .clang-tidy file.
These can be cleaned up over time as they're understood. The majority
are places where boost includes a impl/x.hpp and x.hpp, but expects you
to use the later. include-cleaner opts for the impl, but it isn't clear
why.

Change-Id: Id3fdd7ee6df6c33b2fd35626898523048dd51bfb
Signed-off-by: Ed Tanous <etanous@nvidia.com>
Signed-off-by: Gunnar Mills <gmills@us.ibm.com>

show more ...

Use SPDX identifiers

SPDX identifiers are simpler, and reduce the amount of cruft we have in
code files. They are recommended by linux foundation, and therefore we
should do as they allow.

This pa

Use SPDX identifiers

SPDX identifiers are simpler, and reduce the amount of cruft we have in
code files. They are recommended by linux foundation, and therefore we
should do as they allow.

This patchset does not intend to modify any intent on any existing
copyrights or licenses, only to standardize their inclusion.

[1] https://www.linuxfoundation.org/blog/blog/copyright-notices-in-open-source-software-projects

Change-Id: I935c7c0156caa78fc368c929cebd0f068031e830
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Move isJSONContentType to content-type parser

Previously this function was based on a basic string comparison. This
is fine, but found several inconsistencies, like not handling spaces in
the appro

Move isJSONContentType to content-type parser

Previously this function was based on a basic string comparison. This
is fine, but found several inconsistencies, like not handling spaces in
the appropriate places.

This commit creates a new function getContentType, using the new parsing
infrastructure. As doing this, it showed that the existing parser
functions were not handling case insensitive compares for the mime type.
While this is technically not required, it's something we unit test for,
and relatively easy to add.

Note, that because this parser ignores charset, this moves charset=ascii
from something that previously failed, to something that now succeeds.
This is expected.

Tested: Unit tests pass. Good coverage

Change-Id: I825a72862135b62112ee504ab0d9ead9d6796354
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Fix SSE firing order

When sending last-event-id, the previous events were being received
before the the header was completed. This is because the open handler is
being called before the connect call

Fix SSE firing order

When sending last-event-id, the previous events were being received
before the the header was completed. This is because the open handler is
being called before the connect call was in place, so you get:

Connection starts
open handler called
sendEvent() called from open handler
sendSSEHeader() called.

This results in a spec violation.

Tested:
curl --user root:0penBmc -vvv -s --no-buffer -k -N -H 'Accept:
text/event-stream' -H 'Last-Event-Id: 4' -X GET
https://localhost:8000/redfish/v1/EventService/SSE

Now succeeds, and wireshark dumps show the header is being sent
correctly. Note that previously this command would fail unless http0.9
header was set.

Unit test coverage for this path without last-event-id passes.

Change-Id: I44bb6eedbcbdc727b257646ec55e808157231f75
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Fix clang-tidy

Change-Id: Iefe1b695b86a640d8dfaafd1f77f374fa34246de
Signed-off-by: Ed Tanous <etanous@nvidia.com>

clang-tidy: add misc-include-cleaner fixes

Fix the following clang-tidy errors:
```
../redfish-core/src/filter_expr_executor.cpp:102:21: error: no header providing "nlohmann::json" is directly inclu

clang-tidy: add misc-include-cleaner fixes

Fix the following clang-tidy errors:
```
../redfish-core/src/filter_expr_executor.cpp:102:21: error: no header providing "nlohmann::json" is directly included [misc-include-cleaner,-warnings-as-errors]
7 | const nlohmann::json& body;
```

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I2e0d66bb35c1010607b9795d00b3321dc20d6d65

show more ...

Rename sendEvent

There are currently 3 function prototypes that hold the name
"sendEvent". This makes them hard to search for, and even though they
take different arguments, and are attached to dif

Rename sendEvent

There are currently 3 function prototypes that hold the name
"sendEvent". This makes them hard to search for, and even though they
take different arguments, and are attached to different classes, they're
still difficult to trace.

Rename two of the classes.

Tested: Code compiles. Rename only.

Change-Id: I5df9c690ba0ca8ebe19c73fc0848e9c3ef4d52f7
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Remove IWYU pragmas

These were added as part of
d5c80ad9c07b94465d8ea62d2b6f87c30cac765e: test treewide: iwyu

Since then, Nan hasn't been very active on the project, and to my
knowledge, since the

Remove IWYU pragmas

These were added as part of
d5c80ad9c07b94465d8ea62d2b6f87c30cac765e: test treewide: iwyu

Since then, Nan hasn't been very active on the project, and to my
knowledge, since the initial run, we've never used IWYU again.

clang-include-cleaner seems to work well without needing these pragmas,
and is what we're using, even if it's less useful than IWYU.

Remove all mention of IWYU.

Tested: Code compiles.

Change-Id: I06feedeeac9a114f5bdec81d59ca83223efd8aa7
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Fix includes

This commit is automatically generated by enabling clang-include-fixer.

Tested: Code compiles.

Change-Id: I475d7b9d43e95bbdeeaadf11905d3b2a60aa8ef3
Signed-off-by: Ed Tanous <etanous@n

Fix includes

This commit is automatically generated by enabling clang-include-fixer.

Tested: Code compiles.

Change-Id: I475d7b9d43e95bbdeeaadf11905d3b2a60aa8ef3
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

clang-format: re-format for clang-18

clang-format-18 isn't compatible with the clang-format-17 output, so we
need to reformat the code with the latest version. The way clang-18
handles lambda forma

clang-format: re-format for clang-18

clang-format-18 isn't compatible with the clang-format-17 output, so we
need to reformat the code with the latest version. The way clang-18
handles lambda formatting also changed, so we have made changes to the
organization default style format to better handle lambda formatting.

See I5e08687e696dd240402a2780158664b7113def0e for updated style.
See Iea0776aaa7edd483fa395e23de25ebf5a6288f71 for clang-18 enablement.

Change-Id: Iceec1dc95b6c908ec6c21fb40093de9dd18bf11a
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

show more ...

Add SSE filter param support

The Redfish spec require filtering of SSE entries to be supported.
This commit rearranges the code, and implements SSE sorting as well
as support for Last-Event-Id. To

Add SSE filter param support

The Redfish spec require filtering of SSE entries to be supported.
This commit rearranges the code, and implements SSE sorting as well
as support for Last-Event-Id. To do this it adds a dependency on
boost circular_buffer.

Tested:

SSE connections succeed. Show filtered results.

Change-Id: I7aeb266fc40471519674c7b65cd5cc4625019e68
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Generate metadata at runtime

In the initial implementation of metadata indexing the bmc knew at
compile time what schemas it could potentially publish. bmcweb took the
approach of adding all schema

Generate metadata at runtime

In the initial implementation of metadata indexing the bmc knew at
compile time what schemas it could potentially publish. bmcweb took the
approach of adding all schemas of all versions to the $metadata
resource. Since that was made, two major changes have happened.
First, Redfish has added significantly more versions of each schema, as
well as significantly more schemas to the point where the metadata index
is now 213KB. While this file compresses fairly well, the size is
obvious from the large amount of time that redfish service validator
takes to parse the schemas, compared to actually acquiring BMC redfish
resources.
Second, aggregation was added, where an aggregated Redfish service might
implement any number of schemas, including OEM ones.

In an effort to fix this, this patch takes the compile-time algorithm in
update_schemas.py, and moves it into bmcweb itself, parsing the files on
disk as needed on demand. This has some immediate benefits; First, is
that now schemas can be potentially installed from anywhere, not only
from within the bmcweb build, and they will be resolved at runtime.

Second, patches that want to add support for a given schema need to only
symlink the schema into the correct folder, without needing to rerun
update_schemas.py. This saves time in review.

Finally, this opens to door to reducing the schema versions present in
the metadata to the unique set of only what this bmcweb instance, and
its aggregated BMCs expose.

Tested: Redfish service validator passes. Need A/B checking to verify
the file is byte for byte the same.
GET /redfish/v1/$metadata returns what looks like sane results, with a
correct content-type.
Unit tests require the use of TemporaryFileHandle, so that class is
moved into a more general folder, outside of test/http.

Change-Id: I326159099c6b6c4056023b2e173c5f074ed88ce1
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Manage Request with shared_ptr

This is an attempt to solve a class of use-after-move bugs on the
Request objects which have popped up several times. This more clearly
identifies code which owns the

Manage Request with shared_ptr

This is an attempt to solve a class of use-after-move bugs on the
Request objects which have popped up several times. This more clearly
identifies code which owns the Request objects and has a need to keep it
alive. Currently it's just the `Connection` (or `HTTP2Connection`)
(which needs to access Request headers while sending the response), and
the `validatePrivilege()` function (which needs to temporarily own the
Request while doing an asynchronous D-Bus call). Route handlers are
provided a non-owning `Request&` for immediate use and required to not
hold the `Request&` for future use.

Tested: Redfish validator passes (with a few unrelated fails).
Redfish URLs are sent to a browser as HTML instead of raw JSON.

Change-Id: Id581fda90b6bceddd08a5dc7ff0a04b91e7394bf
Signed-off-by: Jonathan Doman <jonathan.doman@intel.com>
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Break out formatters

In the change made to move to std::format, we defined some custom type
formatters in logging.hpp. This had the unintended effect of making
all compile units pull in the majorit

Break out formatters

In the change made to move to std::format, we defined some custom type
formatters in logging.hpp. This had the unintended effect of making
all compile units pull in the majority of boost::url, and nlohmann::json
as includes.

This commit breaks out boost and json formatters into their own separate
includes.

Tested: Code compiles. Logging changes only.

Change-Id: I6a788533169f10e19130a1910cd3be0cc729b020
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Implement a Content-Security-Policy TODO

This TODO has been in bmcweb for a very long time. Implement it.

W3 sets rules for what security policies apply to which content
types[1]. Reading through

Implement a Content-Security-Policy TODO

This TODO has been in bmcweb for a very long time. Implement it.

W3 sets rules for what security policies apply to which content
types[1]. Reading through this, essentially CSP should only apply to
HTML files.

Tested: Unit tests pass. Webui loads properly. Chrome network window
Shows headers show up as expected.

[1] https://www.w3.org/TR/CSP2/#which-policy-applies

Change-Id: I5467d0373832668763c72a66da2a8872e07bfb58
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Add missing headers

Most of these were found by breaking every redfish class handler into
its own compile unit:

When that's done, these missing headers become compile errors. We
should just fix the

Add missing headers

Most of these were found by breaking every redfish class handler into
its own compile unit:

When that's done, these missing headers become compile errors. We
should just fix them.

In addition, this allows us to enable automatic header checking in
clang-tidy using misc-header-cleaner. Because the compiler can now
"see" all the defines, it no longer tries to remove headers that it
thinks are unused.

[1] https://github.com/openbmc/bmcweb/commit/4fdee9e39e9f03122ee16a6fb251a380681f56ac

Tested: Code compiles.

Change-Id: Ifa27ac4a512362b7ded7cc3068648dc4aea6ad7b
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Remove OpenSSL warnings ignore

If we include OpenSSL in extern "C" blocks consistently, c++ warnings no
longer appear. This means we can remove the special case from meson.

Tested: Code compiles w

Remove OpenSSL warnings ignore

If we include OpenSSL in extern "C" blocks consistently, c++ warnings no
longer appear. This means we can remove the special case from meson.

Tested: Code compiles when built locally on an ubuntu 22.04 system.

Change-Id: I5add4113b32cd88b7fdd874174c845425a7c287a
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Fix a couple includes to use <> instead of ""

Tested: Code compiles.

Change-Id: Ifb254b703b6193a1faf68a54ad823dc807f92514
Signed-off-by: Ed Tanous <ed@tanous.net>

Simplify router

Now that we only support string types in the router we no longer need to
build a "Tag" to be used for constructing argument types. Now, we can
just track the number of arguments, wh

Simplify router

Now that we only support string types in the router we no longer need to
build a "Tag" to be used for constructing argument types. Now, we can
just track the number of arguments, which simplifies the code
significantly, and removes the need to convert to and from the tag to
parameter counts.

This in turn deletes a lot of code in the router, removing the need for
tracking tag types.

Tested: Redfish service validator passes. Unit tests pass.

Change-Id: Ide1d665dc1984552681e8c05952b38073d5e32dd
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

Create TemporaryFileHandle class

TemporaryFileHandle class is used to create temp files in the
filesystem, and hold a handle to them until the class goes out of scope,
at which time they can be remo

Create TemporaryFileHandle class

TemporaryFileHandle class is used to create temp files in the
filesystem, and hold a handle to them until the class goes out of scope,
at which time they can be removed. It replaces makeFile(), which was
not RAII safe if an exception gets thrown, and could potentially leave
files in the filesystem if the tests fail.

Tested: Unit tests pass

Change-Id: I03eb0d342a6cd7b78115a8c42be9175f30c4ccd0
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...