build-unit-test-docker: update ubuntu to hirsute

The latest Ubuntu "Hirsute Hippo" was released on 2021-04-22.
Switch our CI builds to use it.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Ch

build-unit-test-docker: update ubuntu to hirsute

The latest Ubuntu "Hirsute Hippo" was released on 2021-04-22.
Switch our CI builds to use it.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: If41dfd2247f0513bc297c4789cb051f7f69cbec4

show more ...

build-unit-test-docker: cereal: get upstream c++20 fixes

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Iaa03418fdad4405f2f102c283bfaafcdad10c042

build-unit-test-docker: fix default docker in comments

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ie5f34b4886105e378d049d4e056c3cdb138b81a4

Remove libevdev2-dbgsym

Removing this package appears to resolve an issue where the docker unit
test script fails due to missing a dependency:
libevdev2-dbgsym : Depends: libevdev2 (= 1.9.1+dfsg-1)

Remove libevdev2-dbgsym

Removing this package appears to resolve an issue where the docker unit
test script fails due to missing a dependency:
libevdev2-dbgsym : Depends: libevdev2 (= 1.9.1+dfsg-1)

Signed-off-by: Zane Shelley <zshelle@us.ibm.com>
Change-Id: Id04eea803bbdcfb6ac959f8b441cc8b4b0a29b92

show more ...

build-unit-test-docker: boost: switch mirror to yocto

The official download location for boost, bintray, often runs out
of bandwidth and gives errors when we download from it. Switch
our download l

build-unit-test-docker: boost: switch mirror to yocto

The official download location for boost, bintray, often runs out
of bandwidth and gives errors when we download from it. Switch
our download location to be the yoctoproject mirror instead, which
seems to be more stable.

The downside of this is that we can only use Boost releases which
are already incorporated into a Yocto commit, but we really should
only do that anyhow.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ia028ed18d7ef4b9d2a0cc3ca351cee27073ad6bc

show more ...

build-unit-test-docker: add bogus echo to force update keyserver

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I7fe918532f0af4bfac6cd782b6fd3ff9a4763944

Spell check on commit message support for CI

- This commit would add the codespell package while
building the base container.

- The intention is that, we maintain two dictionaries
1. That comes

Spell check on commit message support for CI

- This commit would add the codespell package while
building the base container.

- The intention is that, we maintain two dictionaries
1. That comes directly with codespell
- We should also download the latest dictionary while
while building the base container.
2. An openbmc spellings dictionary
- We should be able to add words and spellings that are
openbmc related in this dictionary over time.

- When the commit message have words that the dictionary does not have,
the CI proceeds further without any rejection. The CI only scores a -1 if
it sees a word wrongly spelled from it knowledge from the dictionary.

Tested by:

1. when commit message have mistakes: It breaks & scores a -1
It also highlights the spelling mistake and the correct spelling
to be used.

2. when commit message does not have mistakes : it does not break &
proceeds further for compilation.

Signed-off-by: Manojkiran Eda <manojkiran.eda@gmail.com>
Change-Id: I87278597876b6e2b0a59e297cde020631491b5fd

show more ...

ensure unit test ci can write /usr/local

Certain repos like openpower-debug-collector generate their error files
on the fly during build time. These are installed into
/usr/local/share/phosphor-dbus

ensure unit test ci can write /usr/local

Certain repos like openpower-debug-collector generate their error files
on the fly during build time. These are installed into
/usr/local/share/phosphor-dbus-yaml/ sub-directories.

Without this change, this is happening:

/home/jenkins-op/workspace/ci-repository/openbmc/openpower-debug-collector > make -j 80 -l 80 -O
cp: cannot create regular file '/usr/local/share/phosphor-dbus-yaml/yaml/org/open_power/Host/Boot.errors.yaml': Permission denied

Since this is just a docker container and /usr/local is not persisted,
give permission to all of /usr/local of the ID running the script.

Tested:
- Verified openpower-debug-collector repo now passes CI

Change-Id: Iea1e89f017e9128b0c538392302b177b7c7d600d
Signed-off-by: Andrew Geissler <geissonator@yahoo.com>

show more ...

Bump the Meson version to 0.57.1

- Meson version 0.57 now supports multi thread support during
the link time optimization, so our compilations can get much
better for repos like bmcweb which use

Bump the Meson version to 0.57.1

- Meson version 0.57 now supports multi thread support during
the link time optimization, so our compilations can get much
better for repos like bmcweb which uses the LTO.

- For more details check the below Meson issue:
https://github.com/mesonbuild/meson/issues/7820

- With the latest poky update (90fd73cb556c0db84e45514413a68f9c5e1aea22),
Meson in yocto had already moved to 0.57.1, so we can bump up the
version used in CI too.

Signed-off-by: Manojkiran Eda <manojkiran.eda@gmail.com>
Change-Id: I0114fe8498f9624ad8bce0ce909267d1412cad7b

show more ...

build-unit-test-docker: make clang go to 11

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Iab3c2729cbf0c1682ad4ef1edbe619c6d270a27c

Upgrade to ubuntu groovy (21.04)

Ubuntu 21.04 looks in terms of dependencies looks much more like the
current bmc builds today. Namely, it includes systemd 246.

Tested:
Build sdbusplus with these

Upgrade to ubuntu groovy (21.04)

Ubuntu 21.04 looks in terms of dependencies looks much more like the
current bmc builds today. Namely, it includes systemd 246.

Tested:
Build sdbusplus with these changes pre-rebase. Build succeeded.

Change-Id: Id52d6c5ee6a11a4af70d2bf789b306d069eb3971
Signed-off-by: Ed Tanous <edtanous@google.com>

show more ...

build-unit-test-docker: prevent exception in thread start-up

Add a lock to allow all the threads to start before they start
running, so that one thread doesn't 'join' on its dependency
before the de

build-unit-test-docker: prevent exception in thread start-up

Add a lock to allow all the threads to start before they start
running, so that one thread doesn't 'join' on its dependency
before the dependency starts. This shows as an exception:

Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "./scripts/build-unit-test-docker", line 353, in run
deppkg.join()
File "/usr/lib/python3.8/threading.py", line 1006, in join
raise RuntimeError("cannot join thread before it is started")

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I7594ac393f310085eb8328a0394493c01b78add3

show more ...

build-unit-test-docker: use Gerrit revision if available

If Jenkins is triggering us to run CI on a package that is included
in the Docker image itself, use the requested revision-under-test as
part

build-unit-test-docker: use Gerrit revision if available

If Jenkins is triggering us to run CI on a package that is included
in the Docker image itself, use the requested revision-under-test as
part of the Docker image build process.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I61a0a249123800aba860ab1535e96f5cdb2ec63f

show more ...

build-unit-test-docker: Fix https_proxy

The https_proxy was not correctly set and if build with proxy settings,
it gives below error:

base: WARNING: Retrying (Retry(total=4, connect=None, read=

build-unit-test-docker: Fix https_proxy

The https_proxy was not correctly set and if build with proxy settings,
it gives below error:

base: WARNING: Retrying (Retry(total=4, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'ProxyError('Cannot connect to proxy.',
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection
object at 0x7efe419e99d0>: Failed to establish a new connection:
[Errno -2] Name or service not known'))': /simple/inflection/
RAN: /usr/bin/docker build --build-arg http_proxy=http://<myproxy>
--build-arg 'https_proxy={https_proxy}' --network=host --force-rm
--no-cache=false -t
openbmc/ubuntu-unit-test-base:2021-W07-773f263e895d8ed6 -

Tested: Fix the https_proxy string and verify the CI passes.

Signed-off-by: Lei YU <yulei.sh@bytedance.com>
Change-Id: I87f543d33747b4957af83990f80bd8e832f4e5e3

show more ...

build-unit-test-docker: switch json to use cmake

nlohmann/json supports CMake and we use it in the bitbake
recipe, so use it here as well. The bitbake recipe has a
'install_append' to symlink the f

build-unit-test-docker: switch json to use cmake

nlohmann/json supports CMake and we use it in the bitbake
recipe, so use it here as well. The bitbake recipe has a
'install_append' to symlink the final header directly into
'{includedir}/json.hpp' so we need to continue to do the
same here.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ibf61a7a471216c43427109907a54e81dde1ae8fc

show more ...

build-unit-test-docker: pythonic refactoring

Do a bunch of clean-up to the script to better / modern Python
practices.

* Document the PackageDef dictionary.

* Add type hinting so that 'myp

build-unit-test-docker: pythonic refactoring

Do a bunch of clean-up to the script to better / modern Python
practices.

* Document the PackageDef dictionary.

* Add type hinting so that 'mypy' static analysis passes (and
code self-documents better).

* Create a Package class to encapsulate everything related to
package stage builds and a Docker namespace to encapsulate
Docker operations.

* Overall better documentation.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I841d844aa9401889d428ed55c10bee6c1b0a7109

show more ...

build-unit-test-docker: update numerous packages

- Update packages per new bitbake recipes.
- Favor using the tag-name rather than a git hash, when available.
- Linux headers now requires rsync.

Si

build-unit-test-docker: update numerous packages

- Update packages per new bitbake recipes.
- Favor using the tag-name rather than a git hash, when available.
- Linux headers now requires rsync.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I90d045fa7ef69067ddf09685dfde3265cc1e948f

show more ...

build-unit-test-docker: create hashed tags for final image

Create a tag for the final image in the same format as the stage
images. Display this tag to stdout at the end of the docker build
so that

build-unit-test-docker: create hashed tags for final image

Create a tag for the final image in the same format as the stage
images. Display this tag to stdout at the end of the docker build
so that other scripts can consume it via calls like:
$(scripts/build-unit-test-docker)

Modify `run-unit-test-docker.sh` to get the docker image from
`build-unit-test-docker` rather than a static image name. This
ensures that the docker image used in unit testing is the exact
image requested to be built. (Soon this will allow us to incorporate
Gerrit changes into the docker image itself.)

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Idd7a9a7b0157f1c520b33b3ec7ac0d5b52018d9e

show more ...

build-unit-test-docker: invalidate base image weekly

Docker will cache an image built with the exact same commands,
so we only get latest Ubuntu packages rarely. Add a date-based
echo command into

build-unit-test-docker: invalidate base image weekly

Docker will cache an image built with the exact same commands,
so we only get latest Ubuntu packages rarely. Add a date-based
echo command into the early stages of the base image build so
that we regularly get updates to all the packages in the base
image.

This invalidation, via echo, is only performed when running under
a Jenkins-like context. The purpose of this distinction is so that
we are more likely to cache failures induced by an underlying OS
update in CI, but when developers run this script locally they rarely
see an invalidation that is not due to a code package update.

Add an environment variable option "FORCE_DOCKER_BUILD" which can be
set by developers to bypass the Docker cache and force all images
to be rebuilt, just in case they really want to do that sort of thing.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Iec92ebada513975d0806099083384dfca623f144

show more ...

build-unit-test-docker: build docker stages in parallel

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I1899b2cacdf8f668c3f1226959d1b5a044b5fc6c

build-unit-test-docker: split docker stages into invocations

Rather than a single large Dockerfile, split each stage into its
own Dockerfile with its own invocation. This is in preparation
for buil

build-unit-test-docker: split docker stages into invocations

Rather than a single large Dockerfile, split each stage into its
own Dockerfile with its own invocation. This is in preparation
for building the docker stages in parallel.

Since multiple invocations of this script could be running in
parallel, possibly with different content, this requires creating
well-named tags for each stage. Create these tags as a combination
of the current date and the content that went into the stage
Dockerfile. The date is used so that we can refresh the base
image more regularly.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: I77ac232f1603b6ce74c55f0f9466482ff30eb32e

show more ...

build-unit-test-docker: switch cmake to ninja backend

Ninja is faster and what is used by Yocto.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Icbb17ab466e03844b97c27869646cc580f77

build-unit-test-docker: switch cmake to ninja backend

Ninja is faster and what is used by Yocto.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Icbb17ab466e03844b97c27869646cc580f77c558

show more ...

build-unit-test-docker: generate package directives

Generate the docker build directives for all packages by
performing a depth-first dependency search.

Signed-off-by: Patrick Williams <patrick@stw

build-unit-test-docker: generate package directives

Generate the docker build directives for all packages by
performing a depth-first dependency search.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ia3b6778e0f15bd9d5c4cff5e5e7bf2a620c0e156

show more ...

build-unit-test-docker: templatize build directives

Create 'pkg_build' function that generates the docker RUN template
necessary to build all the packages.

Signed-off-by: Patrick Williams <patrick@

build-unit-test-docker: templatize build directives

Create 'pkg_build' function that generates the docker RUN template
necessary to build all the packages.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ie7e7a3b3e78c67e6d97a83f3b43c3c54d10b77e6

show more ...

build-unit-test-docker: add dependencies to package def

Add dependencies to the package definition for each package and remove
the manual "COPY" commands to create the staged-dependencies.
Eventuall

build-unit-test-docker: add dependencies to package def

Add dependencies to the package definition for each package and remove
the manual "COPY" commands to create the staged-dependencies.
Eventually this will also be used to parallelize the docker build
process.

Signed-off-by: Patrick Williams <patrick@stwcx.xyz>
Change-Id: Ifb7e26fccea62ae9bc8d450a91d1fc4d9ef0db06

show more ...