autojson: handle recursion

In order to allow the configurations directory to have vendor
subdirectories, modify the script slightly for directory traversal
to handle subdirectories.

Tested:

After

autojson: handle recursion

In order to allow the configurations directory to have vendor
subdirectories, modify the script slightly for directory traversal
to handle subdirectories.

Tested:

After performing a `git mv`, the subdirectory files show up in the
`format-code` output.
```
...
formatting file configurations/meta/terminus_2x100g_nic_tsff.json
...
```

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ib8cea8053d558c60e0a89dd6dcddbc4bc0d970fb

show more ...

jsonschema: update validator to the latest draft

The JSON Schema "draft 7" is now old and the latest is
"2020-12". Switch the validator object we use to be the
latest version.

Signed-off-by: Patri

jsonschema: update validator to the latest draft

The JSON Schema "draft 7" is now old and the latest is
"2020-12". Switch the validator object we use to be the
latest version.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I90a0d9b8669bade600ca3d48041f31727e410e14

show more ...

scripts/validate_configs.py: Ensure schemas are validated as well

The quick'n'dirty method of validating a schema is via
`jsonschema.validate()`. From `help(jsonschema.validate)` we have:

> :func

scripts/validate_configs.py: Ensure schemas are validated as well

The quick'n'dirty method of validating a schema is via
`jsonschema.validate()`. From `help(jsonschema.validate)` we have:

> :func:`validate` will first verify that the provided schema is
> itself valid, since not doing so can lead to less obvious error
> messages and fail in less obvious or consistent ways.
>
> If you know you have a valid schema already, especially if you
> intend to validate multiple instances with the same schema, you
> likely would prefer using the `Validator.validate` method directly
> on a specific validator (e.g. ``Draft7Validator.validate``).

The implication is that explicitly instantiating a validator does not
validate the provided schema.

The implementation of validate_configs.py directly instantiates the
Draft7Validator() in order to provide a RefResolver. However, it does
not do the required work of validating the provided schema.

Rework the code instantiating the validator to validate the schema prior
to use.

Tested:

```
$ git revert -n $(git log --grep 'I2b8102e7d6047c8a7e624e182465065268fb8ad9' --format='%H') && ./scripts/validate_configs.py -v -k; git reset --hard
Traceback (most recent call last):
File "/home/andrew/src/openbmc.org/openbmc/entity-manager/origin/./scripts/validate_configs.py", line 187, in <module>
main()
File "/home/andrew/src/openbmc.org/openbmc/entity-manager/origin/./scripts/validate_configs.py", line 138, in main
spec.check_schema(schema)
File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 204, in check_schema
raise exceptions.SchemaError.create_from(error)
jsonschema.exceptions.SchemaError: ['The schema for an entity manager configuration file. An entity ', 'mananger configuration file can consist of a single object, or an ', 'array of objects. Objects must be of type EMConfig.'] is not of type 'string'

Failed validating 'type' in metaschema['properties']['description']:
{'type': 'string'}

On schema['description']:
['The schema for an entity manager configuration file. An entity ',
'mananger configuration file can consist of a single object, or an ',
'array of objects. Objects must be of type EMConfig.']
HEAD is now at cecbd8129645 scripts/validate_configs.py: Ensure schemas are validated as well
```

Change-Id: Ib4d41c0f7a07c1ea4b9a483893969f21870a50e1
Signed-off-by: Andrew Jeffery <andrew@codeconstruct.com.au>

show more ...

Support C-style comments for configuration JSON parsing

1. Add and set ignore_comment to true to all nlohmann::json::parse().
2. Add remove_c_comments() in `validate_configs.py` to remove C-style

Support C-style comments for configuration JSON parsing

1. Add and set ignore_comment to true to all nlohmann::json::parse().
2. Add remove_c_comments() in `validate_configs.py` to remove C-style
comments before loading.
3. Attempt to reformat comments in the `autojson.py` taking liberal
short-cuts which are documented in the script.

Supported comment examples:

- Single-line style comments
```
{
// Single-line style comment (new line)
"Key": "Value" // Single-line comment (end of content)
}
```

- Multi-line style comments
```
{
/* Multi-line style comment */
/*
* Multi-line style comments
*/
}
```

Tested on harma system with manual applied patch below, which contains
a c-style comment in harma-pttv.json file.
Link: https://gerrit.openbmc.org/c/openbmc/entity-manager/+/67469/25

- scripts/autojson.py
Run autojson.py on harma-pttv.json, the output as same as original file.

- scripts/validate_configs.py
Run validate_configs.py passed.

- EntityManager service
EntityManager service loads and probes harma-pttv.json successfully.
```
root@harma:~# busctl introspect xyz.openbmc_project.EntityManager \
> /xyz/openbmc_project/inventory/system/board/Harma_PTTV \
> xyz.openbmc_project.Inventory.Item.Board
NAME TYPE SIGNATURE RESULT/VALUE FLAGS
.Name property s "Harma PTTV" emits-change
.Probe property s "xyz.openbmc_project.FruDevice({\'BOA... emits-change
.Type property s "Board" emits-change
```

Signed-off-by: Potin Lai <potin.lai@quantatw.com>
Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ib235f2aa6a724615dc4c8184577f57abda8e17a6

show more ...

format: reformat with latest openbmc-build-scripts

Run all the latest formatters from openbmc-build-scripts. Disable
prettier on the configurations subdirectory because these are
script-formatted a

format: reformat with latest openbmc-build-scripts

Run all the latest formatters from openbmc-build-scripts. Disable
prettier on the configurations subdirectory because these are
script-formatted already with the repo-local `format-code` extension.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ifa4f2ec6688d60fde2a5f42efc96b6046e9be642

show more ...

python: fix flake8 warnings

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: If040c21bb65b613d75574d3154daa35c88c43f76

Remove mux-closing detritus

It's been a while since we had python in the build; this script is
obsolete, as is the reference to it in the FruDevice service file.

Signed-off-by: Zev Weiss <zev@bewil

Remove mux-closing detritus

It's been a while since we had python in the build; this script is
obsolete, as is the reference to it in the FruDevice service file.

Signed-off-by: Zev Weiss <zev@bewilderbeest.net>
Change-Id: If0bb9a4ff7c30d2e5b9ca370e11918ab36c820f8

show more ...

scripts: use lower_snake_case for filenames

To be consistent with lower_snake_case source code filenames, use
lower_snake_case for scripts file names as well.

run-ci.sh is not renamed because openb

scripts: use lower_snake_case for filenames

To be consistent with lower_snake_case source code filenames, use
lower_snake_case for scripts file names as well.

run-ci.sh is not renamed because openbmc-build-scripts specifically
looks for run-ci.sh or run-ci.

Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>
Change-Id: I4535bac41375d2c78e71da5020cb5e2410ff3eaa

show more ...

autojson.py: add newline character before the end of file

POSIX standard defines line as something terminating with the newline
character. So the last line of a text file should also have that.

Pop

autojson.py: add newline character before the end of file

POSIX standard defines line as something terminating with the newline
character. So the last line of a text file should also have that.

Popular editors add it automatically so to avoid unnecessary noise in
the diffs between manually edited files and something that went through
autojson.py it should add it as well.

Signed-off-by: Paul Fertser <fercerpav@gmail.com>
Change-Id: I1e04b85eb41b8a7d00d506f49f994c6b40eb5fdf

show more ...

Revert "autojson.py: Check the sensor name"

This reverts commit 086f19d287fdc7d7be9c52ad9aee7704e4f339f5.

The recent script change causes errors.
formatting file configurations/1Ux16 Riser.json
Tra

Revert "autojson.py: Check the sensor name"

This reverts commit 086f19d287fdc7d7be9c52ad9aee7704e4f339f5.

The recent script change causes errors.
formatting file configurations/1Ux16 Riser.json
Traceback (most recent call last):
File "/build/openbmcup/build/workspace/sources/entity-manager/scripts/autojson.py", line 30, in <module>
nl = [s['Name'] for s in j['Exposes']]
TypeError: list indices must be integers or slices, not str

Considering the files this fail on have been around for a while, it's
clear they were never tested.

Change-Id: Ibda22e9ce43d1ba8e7a3c02609ef4d135ebb3044

show more ...

autojson.py: Check the sensor name

We may be careless and write down duplicate sensor names
in the json, which will cause the entity-manager to
generate duplicate dbus paths and coredump.

This chan

autojson.py: Check the sensor name

We may be careless and write down duplicate sensor names
in the json, which will cause the entity-manager to
generate duplicate dbus paths and coredump.

This change counts the sensor name and outputs a red alert
message if there are duplicates

Signed-off-by: John Wang <wangzhiqiang.bj@bytedance.com>
Change-Id: I3363e744d7e0ddf8cb8894e120368a45af4f8de9

show more ...

run-ci: modify permissions to execute

Per discussion in Ib420c69e88e88ca0a50863c83ad43fac750d3e7a, we
are going to require these scripts to have execute permission.

Signed-off-by: Patrick Williams

run-ci: modify permissions to execute

Per discussion in Ib420c69e88e88ca0a50863c83ad43fac750d3e7a, we
are going to require these scripts to have execute permission.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I04fefb6a10b195ae2782014bf305d2cf80f706b3

show more ...

test: run validation script from CI

Run the schema validation script when in the OpenBMC continuous
integration environment. This ensures that both the schema and EM JSON
documents are always valid

test: run validation script from CI

Run the schema validation script when in the OpenBMC continuous
integration environment. This ensures that both the schema and EM JSON
documents are always valid.

A handful of schema are known to fail - add an xfail file until those
can be fixed up.

Change-Id: Ifabfcce0f7953cd0e3bd5489bb0bbc2ce3e6989d
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>

show more ...

scripts: add configuration validation script

Add a script to validate configurations against the schema. Use
https://github.com/Julian/jsonschema to perform the json validation.

The script is inte

scripts: add configuration validation script

Add a script to validate configurations against the schema. Use
https://github.com/Julian/jsonschema to perform the json validation.

The script is intended to be run from a continuous integration
environment or by configuration/schema developers. A key
assumption/feature of the script is that its users will always prefer to
resolve relative references to the local filesystem. As such, the
script computes a base URI that points to the filesystem and instructs
the validator to use that in place of whatever base_uri it derives from
the global $id attribute. For additional reading see:

https://json-schema.org/understanding-json-schema/structuring.html#the-id-property
https://github.com/Julian/jsonschema/issues/98

Without any options the script assumes it is being run from an
entity-manager source distribution and attempts to find the schema and
configuration files relative to the location of the script.

Alternatively, the script can validate arbitrary json files against
arbitrary schema:

./validate-configs.py -s foo.schema.json -c test1.json -c test2.json

By default the validation stops as soon as a configuration does not
validate. Use -k to override this behavior and validate as many
configurations as possible.

Provide an option to instruct the script to ignore a list of
configurations that are expected to fail validation to be used in
continuous integration setups - similar in concept to xfail mechanisms
provided by most build systems with unit test support.

Change-Id: I7d67a54993a6d5e00daf552d9d350c80411b997b
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>

show more ...

autojson: don't check non-json files

Skip files that don't have a json suffix so other files (like makefiles)
can be placed in the configurations directory.

Change-Id: Ibdb4d29a6a024762d3f90ca410d1

autojson: don't check non-json files

Skip files that don't have a json suffix so other files (like makefiles)
can be placed in the configurations directory.

Change-Id: Ibdb4d29a6a024762d3f90ca410d11815fe641070
Signed-off-by: Brad Bishop <bradleyb@fuzziesquirrel.com>

show more ...

Add build step to verify json formatting

This will run autojson during the build to verify
that all files are formatted correctly.

Tested: Before this change build failed, after
fixing json file, i

Add build step to verify json formatting

This will run autojson during the build to verify
that all files are formatted correctly.

Tested: Before this change build failed, after
fixing json file, it passed

Change-Id: I119cc898536a972bb7a248143b40d695f880ff2f
Signed-off-by: James Feist <james.feist@linux.intel.com>

show more ...

Sort Exposes Alphabetically By Type

This makes it much easier to find components of
a specific type because they will be grouped together.

Tested: Sensor list was still the same

Change-Id: Iab15be

Sort Exposes Alphabetically By Type

This makes it much easier to find components of
a specific type because they will be grouped together.

Tested: Sensor list was still the same

Change-Id: Iab15be34e003be1aac120864c18a80eaaa9b391c
Signed-off-by: James Feist <james.feist@linux.intel.com>

show more ...

Move closeMuxes script to a regex

This is to allow muxes of any type to be closed correctly on restart.

Change-Id: Ib0e15d3ca6309e7a297c910bded2ce172d16f956
Signed-off-by: Ed Tanous <ed.tanous@inte

Move closeMuxes script to a regex

This is to allow muxes of any type to be closed correctly on restart.

Change-Id: Ib0e15d3ca6309e7a297c910bded2ce172d16f956
Signed-off-by: Ed Tanous <ed.tanous@intel.com>

show more ...

Rename all properties to UpperCamelCase

Upstream dbus standard is UpperCamelCase, start matching and
check in json schema to start failing builds that don't match.

Change-Id: I2d2c6fec5a5a0b07c88c8

Rename all properties to UpperCamelCase

Upstream dbus standard is UpperCamelCase, start matching and
check in json schema to start failing builds that don't match.

Change-Id: I2d2c6fec5a5a0b07c88c843257fa7c88d420c4e5
Signed-off-by: James Feist <james.feist@linux.intel.com>

show more ...

Add close muxes script

We have a problem that muxes can be left open
on reboot, so that when we scan for devices they
are left open. Add this script as a mitigation until
we can come up with a bette

Add close muxes script

We have a problem that muxes can be left open
on reboot, so that when we scan for devices they
are left open. Add this script as a mitigation until
we can come up with a better idea.

Change-Id: I74143bb0624188394ed67110813671ca68698806
Signed-off-by: James Feist <james.feist@linux.intel.com>

show more ...

Move components from proving-ground

Move all needed components from proving ground to this
repo. Some clean up was done in json files to fix probes
as well as some slight modification to readme.

Ch

Move components from proving-ground

Move all needed components from proving ground to this
repo. Some clean up was done in json files to fix probes
as well as some slight modification to readme.

Change-Id: I05b7f6459704640c4850420a4573d157500d0aff
Signed-off-by: James Feist <james.feist@linux.intel.com>

show more ...