message: ensure support for flat-map

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Id50c1e5a4dbf86a16668f1a51c00a82fd2319c73

use unpack syntax

Rather than defining a variable and then reading it from a message,
we also supports directly unpack-ing from the message. Use this
syntax instead as it is more efficient and succ

use unpack syntax

Rather than defining a variable and then reading it from a message,
we also supports directly unpack-ing from the message. Use this
syntax instead as it is more efficient and succinct.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I2ce2aa3e6a43d8866ba9b3920dd3165b7b6b9f6e

show more ...

Allow appending std::vector<bool>

std::vector<bool> when done with a for loop returns the type of
std::_bit_reference, which doesn't have a direct conversion to the
underlying boolean type.

Rather

Allow appending std::vector<bool>

std::vector<bool> when done with a for loop returns the type of
std::_bit_reference, which doesn't have a direct conversion to the
underlying boolean type.

Rather than relying on the type returned by begin(), rely on
T::value_type to specifically get the type in the container.

There isn't a particular use for this, but
https://gerrit.openbmc.org/c/openbmc/entity-manager/+/81474

Is trying to consolidate code, and having all types use the same
pack/unpack code is advantageous to avoid special cases. I don't
know of a place we're actually packing/unpacking arrays of bool.

Tested: Booted gb200nvl-bmc. Services launched, no new crashes seen.

Change-Id: I07fc150a190ae44f7bdc90531ad2a4ce7f47e0a1
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Remove grouping for append

These APIs are fairly complex. Similar to what was done for the
read_single class of values, simplify these using c++17 template
expressions.

Tested: Booted gb200nvl-bmc

Remove grouping for append

These APIs are fairly complex. Similar to what was done for the
read_single class of values, simplify these using c++17 template
expressions.

Tested: Booted gb200nvl-bmc. Services launched, no new crashes seen.

Change-Id: If2604113efa578062890f11d18ecca7a9f249e9c
Signed-off-by: Ed Tanous <etanous@nvidia.com>

show more ...

Use sd-bus array methods for read and append

Currently sdbusplus uses sd_bus_message_read_basic and
message_append_basic for decoding and encoding of arrays. For large
arrays, this imposes a signif

Use sd-bus array methods for read and append

Currently sdbusplus uses sd_bus_message_read_basic and
message_append_basic for decoding and encoding of arrays. For large
arrays, this imposes a significant overhead in calling the library each
time. This leads to some extreme performance degradation in some cases,
for example, passing a 4K bytes data array over sdbusplus interface now
takes 4096 + 2 calls of sd_bus_message_read_basic and
message_append_basic, this will consume about 10ms CPU time on a Aspeed
2600 platform for each package on both send and receive side.

While in this case, a DBus interface design should likely opt for using
an FD rather than an array of bytes, this isn't a reason to not optimize
this case.

sd-bus, in version 240 added methods to deal with this performance
degradation, namely sd_bus_message_read_array and
sd_bus_message_append_array, which each only require a single call to
load values into the array using pointers and lengths.

This patchset is based on the one submitted here:
https://gerrit.openbmc.org/c/openbmc/sdbusplus/+/68614

But opts for a different approach to a number of technical details on
how it accomplishes the result, including changing the underlying sd-bus
calls used to utilize ones that don't require sdbus internal malloc,
avoiding unit tests ASAN issues in the process.

This commit adds support for utilizing the above methods when doing
calls to the library if the following requirements are met:
1. The container is contiguous (using the std::contiguous_iterator
concept)
2. The container represents a list of values of type int/uint 8,16,32,
or 64 bits wide. Note, per the sd-bus documentation, arrays of bool are
explicitly not supported (presumably because the internal representation
of a bool array for both libc++ and sd-bus is implementation defined).

To accomplish this, the arrays handling for contiguous arrays is moved
to use concepts, simplifying the code significantly, and allowing the
use of std::contiguous_iterator.

Change-Id: I3bd9f97ed160835e8c30c302b80553a1d450bbf9
Signed-off-by: Ed Tanous <ed@tanous.net>
Signed-off-by: Yongbing Chen <yongbingchen@google.com>

show more ...

Prepare for one shot array reading

With the patch changing the behavior for arrays of numbers, we want
these tests to continue testing the iteration array read paths. In the
next commit, std::vecto

Prepare for one shot array reading

With the patch changing the behavior for arrays of numbers, we want
these tests to continue testing the iteration array read paths. In the
next commit, std::vector<uint8_t> will no longer be using iteration
paths, so change the unit test to use strings instead. Next commit will
add back equivalent coverage.

Change-Id: I7484ac54b6f964596b5e914734999e6441b5e9c7
Signed-off-by: Ed Tanous <ed@tanous.net>

show more ...

message: Fix unpack void type

The template unpack<void> is deduced as one argument instead of 0, and
results in below compile error:
```
error: variable or field ‘r’ declared void
error: return-stat

message: Fix unpack void type

The template unpack<void> is deduced as one argument instead of 0, and
results in below compile error:
```
error: variable or field ‘r’ declared void
error: return-statement with a value, in function returning ‘void’ [-fpermissive]
```

Fix this by checking void if `sizeof...(Args) == 1` and return void.
Also add a unit test case to make sure the code compiles.

Change-Id: I74cca180783645496863393be37215f1d6f4ca02
Signed-off-by: Lei YU <yulei.sh@bytedance.com>

show more ...

clang-format: re-format for clang-18

clang-format-18 isn't compatible with the clang-format-17 output, so we
need to reformat the code with the latest version. The way clang-18
handles lambda forma

clang-format: re-format for clang-18

clang-format-18 isn't compatible with the clang-format-17 output, so we
need to reformat the code with the latest version. The way clang-18
handles lambda formatting also changed, so we have made changes to the
organization default style format to better handle lambda formatting.

See I5e08687e696dd240402a2780158664b7113def0e for updated style.
See Iea0776aaa7edd483fa395e23de25ebf5a6288f71 for clang-18 enablement.

Change-Id: I4f63258febea27dae710c252033b9151e02be7e8
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

show more ...

test: message: read: add structured binding test for unpack

Add test case for structured binding syntax from unpack, which might
be used as a example in implementations.

Signed-off-by: Patrick Will

test: message: read: add structured binding test for unpack

Add test case for structured binding syntax from unpack, which might
be used as a example in implementations.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I342fffebad3d6e9a58fe5ab57bbf13019e1b0e6f

show more ...

clang-format: copy latest and re-format

clang-format-17 has some backwards incompatible changes that require
additional settings for best compatibility and re-running the formatter.
Copy the latest

clang-format: copy latest and re-format

clang-format-17 has some backwards incompatible changes that require
additional settings for best compatibility and re-running the formatter.
Copy the latest .clang-format from the docs repository and reformat the
repository.

Change-Id: I1d7d35c8035993df4c164bfb055d3be476d3ea84
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

show more ...

test: message: read: add test for unix_fd

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I602b3d2d44dbd74916aa73d018b37ff0cafb210d

clang-format: copy latest and re-format

clang-format-16 has some backwards incompatible changes that require
additional settings for best compatibility and re-running the formatter.
Copy the latest

clang-format: copy latest and re-format

clang-format-16 has some backwards incompatible changes that require
additional settings for best compatibility and re-running the formatter.
Copy the latest .clang-format from the docs repository and reformat the
repository.

Change-Id: I5daa012bf76924eb7a7d22ed31b6b77ad2f723df
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>

show more ...

Remove some extra semi-colons in tests

These snuck in here and cause clang warnings. Fix them.

Change-Id: I526cfee1ebc799eae48b92e790273f31462c655f
Signed-off-by: Ed Tanous <edtanous@google.com>

Add support for appending std::string_view

std::string_view is used more in the project now that c++17 is
available. It should be allowed as a base type in serialization of dbus
interfaces.

To avo

Add support for appending std::string_view

std::string_view is used more in the project now that c++17 is
available. It should be allowed as a base type in serialization of dbus
interfaces.

To avoid an extra string copy, this function makes use of
sd_bus_message_append_string_iovec, which allows appending in string
pieces that might not be null terminated. This function is piped
through the test framework, and interfaces to match.

Change-Id: Iee3e2cea9759fa9759cec98ab30c12c822aa3b73
Signed-off-by: Ed Tanous <edtanous@google.com>

show more ...

message: add unpack method

Add an unpack method that allows reading from the message as
r-values. This simplifies the pattern:

```
foo f{};
bar b{};
msg.read(f,b);

// Can now be w

message: add unpack method

Add an unpack method that allows reading from the message as
r-values. This simplifies the pattern:

```
foo f{};
bar b{};
msg.read(f,b);

// Can now be written as...

auto [f, b] = msg.unpack<foo, bar>();
```

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ic2ae7f2c52d41702b8c7c3af6a2efb21558a7579

show more ...

message: append: support span

Change how we detect the ability to iterate over a type so that
it supports additional types, such as span. Add test case to cover
span. Add some static_asserts in th

message: append: support span

Change how we detect the ability to iterate over a type so that
it supports additional types, such as span. Add test case to cover
span. Add some static_asserts in the test cases to ensure we are
triggering the correct add_multiple/add_single paths for a few
assorted types.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I42ff496ab58af613940c9ceb979c71f468f125fc

show more ...

bus: shorten bus type

Create an alias `sdbusplus::bus_t` to `sdbusplus::bus::bus` to reduce
duplication.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I9f17892eed5112d4946949861488

bus: shorten bus type

Create an alias `sdbusplus::bus_t` to `sdbusplus::bus::bus` to reduce
duplication.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I9f17892eed5112d49469498614884c02bba86011

show more ...

message: shorten message type

Create an alias `sdbusplus::message_t` to `sdbusplus::message::message`
to reduce duplication.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ib400b12f

message: shorten message type

Create an alias `sdbusplus::message_t` to `sdbusplus::message::message`
to reduce duplication.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ib400b12fe4a412c0c0c3d26a88f3fae46445cfa8

show more ...

native_types: string_path_wrapper: Don't escape all strings

Not all strings require escaping when appending them to a path. This
allows us to now append `openbmc_project` to `/xyz` and have
expected

native_types: string_path_wrapper: Don't escape all strings

Not all strings require escaping when appending them to a path. This
allows us to now append `openbmc_project` to `/xyz` and have
expected behavior.

Change-Id: I49a2ee100455cff4067dcf3df61f3e145231eeba
Signed-off-by: William A. Kennington III <wak@google.com>

show more ...

native_types: Reduce append allocation

This builds the string from left to right, instead of dealing with
string inserts, optimizing for allocations. Uses our own hex encoder
instead of relying on s

native_types: Reduce append allocation

This builds the string from left to right, instead of dealing with
string inserts, optimizing for allocations. Uses our own hex encoder
instead of relying on systemd libraries. This maintains explicit
compatability with the systemd encoding scheme.

Change-Id: Ia6c2f4fb19984f549c6bac2d67d0b5d610022038
Signed-off-by: William A. Kennington III <wak@google.com>

show more ...

test/message/types: Convert ASSERTs to EXPECTs

None of the comparisons made in this test prevent the continuation of
the test. This makes it easier to debug changes since all errors will
now be repo

test/message/types: Convert ASSERTs to EXPECTs

None of the comparisons made in this test prevent the continuation of
the test. This makes it easier to debug changes since all errors will
now be reported instead of early termination.

Change-Id: Ifd849f5a7585ff13cb314980a1128e0f126723b6
Signed-off-by: William A. Kennington III <wak@google.com>

show more ...

message: Add call_async method

This makes it possible to perform an async method call that is agnostic
to the event loop running the sd_bus state machine.

Change-Id: I32bc0fdf89c44cc6bab1c4622b143d

message: Add call_async method

This makes it possible to perform an async method call that is agnostic
to the event loop running the sd_bus state machine.

Change-Id: I32bc0fdf89c44cc6bab1c4622b143d6e06098659
Signed-off-by: William A. Kennington III <wak@google.com>

show more ...

Fix #60 by adding encoding/decoding rules

This commit attempts to make encoding more compatible with things within
openbmc that don't encode paths per the systemd method. It does this by
forcing th

Fix #60 by adding encoding/decoding rules

This commit attempts to make encoding more compatible with things within
openbmc that don't encode paths per the systemd method. It does this by
forcing the first character of every path segment to be encoded, so /abc
would be encoded as /_61bc. This is then used as a mechanism to
determine if the path needs to be decoded per systemds rules.

The decode mechanisms are also modified to check whether the first
element is encoded before decoding the portion.

Looking for input on whether this is an OK path. The hope would be that
we slowly transition the system over to using the encoding mechanisms
that can handle all ascii characters, and can round trip through
encode/decode without loss. The hope is that this patch can be reversed
at some point in the future.

As an aside, I tried reverting some of the patches, but they're somewhat
ungainly to revert at this point, and would require reverting some
bmcweb patches, which, if this patch is determined to be bad, I'm happy
to do, but I'm hoping this will get us something in the interim

Tested:
unit tests added and updated to cover some of the cases. Unit tests
pass.

Change-Id: Ie9c8cd0627b8b086e8b0bb2287f28dd86eb98ee9
Signed-off-by: Ed Tanous <edtanous@google.com>

show more ...

Implement operator /= for object_path

This probably should've been implemented as part of
fa0fa3b0fa5a9d59c37b6edbc53cc21b7c2a16b0 Implement path encoding and
decoding but alas, it was not. This co

Implement operator /= for object_path

This probably should've been implemented as part of
fa0fa3b0fa5a9d59c37b6edbc53cc21b7c2a16b0 Implement path encoding and
decoding but alas, it was not. This commit adds the relatively simple
operator/= overloads to object_path. This allows calling code to do
operations like:

object_path myPath("/foo");
myPath /= "bar";

Which would result in the object path /foo/bar.

In the implementation, this actually just calls into operator/, and does
a relatively naive replace of the content in the new object. Given how
the sd_bus_path_encode API is organized, there doesn't appear to be a
way to call into it and "append" to an existing thing, so constructing a
new string and assigning it to the current object was the best I could
come up with in terms of efficiency.

This includes overloads for both const char* and const std::string&,
both of which have essentially the same behavior.

Tested:
New unit tests included in commit, and pass.

Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: Idfb7f7f2dc94ea3773841c9ae62cd671327db2cd

show more ...

Implement path encoding and decoding

sdbus has methods for encoding and decoding paths in a lossless way.
This commit adds those methods to object_path in such a way that those
functions are called

Implement path encoding and decoding

sdbus has methods for encoding and decoding paths in a lossless way.
This commit adds those methods to object_path in such a way that those
functions are called implicitly when building strings, the hope being
that any API choosing to use object_path need not write its own escaping
functions, and said escaping will happen implicitly.

The explicit changes to make this happen are.
1. filename() now calls into sd_bus_path_decode, rather than writing its
own method. This means that the strings returned from the method now no
longer need decoding explicitly, and can largely be used as-is. For
paths that did not require escaping, the behavior should be identical.
For paths that did require escaping, this method will now return the raw
version of the path. Existing usages should remain the same, and impact
is unlikely, given that most sdbusplus users don't explicitly escape
strings, and filename() is relatively new.

2. a operator/ method is added to the object_path object. Similar to
operator/ on std::filesystem::path, this allows generating a new
object_path from an existing one, by adding a new leaf node to the path.
Functionally this allows nearly any value to be escaped into a dbus
path. An example of a common usage of this might be

object_path myPath = "/";
object_path myNewPath = myPath / "Foo-bar";

The above line will implicitly escape the - in the name. Overloads for
both const std::string& and const char* have been added.

One motivating example where the above would be useful is to remove the
escapePathForDbus method in bmcweb.
https://github.com/openbmc/bmcweb/blob/88b3dd12851cd7bdd4b5c065ba99f40feafb775e/include/dbus_utility.hpp#L47

and the similarly named method in dbus-sensors
https://github.com/openbmc/dbus-sensors/blob/6cb732a31b7664089124b00e806311768bc24a87/src/SensorPaths.cpp#L49
both written by the same author (not me), and centralize them in
sdbusplus where they belong.

Functionally this commit also includes a new utility/memory.hpp, that
allows for C-style RAII using freep, mfree and the _cleanup_free_
attributes. These were largely copied as-is from alloc-util.h in
systemd on latest patchset recommendations.

Tested:
Unit tests have been updated to include all new code paths, and some
common items that might need escaped. Unit tests pass.

Signed-off-by: Ed Tanous <edtanous@google.com>
Change-Id: If77b8bc5f75709b59353570a9c14383cac44dcd4

show more ...