42bd9936 | 28-Jan-2021 |
Adriana Kobylak <anoo@us.ibm.com> |
meson: Make verify-full-signature stand-alone
The new verify-full-signature config option requires that the legacy config option verify-signature to be also enabled or it's a no-op. Having to specif
meson: Make verify-full-signature stand-alone
The new verify-full-signature config option requires that the legacy config option verify-signature to be also enabled or it's a no-op. Having to specify two different options to enable the full signature verification is not user friendly.
Allow the verify-full-signature to turn on signature checking the same as the verify-signature option, and add a comment that the new verify-full-signature is the preferred option instead of the legacy verify-signature.
Tested: Verified that compiling with -Dverify-full-signature=enabled defined WANT_SIGNATURE_VERIFY and WANT_SIGNATURE_FULL_VERIFY. Compiling with -Dverify-signature=enabled only enabled WANT_SIGNATURE_VERIFY. And either option compiled the image_verify.cpp file.
Change-Id: Ib3a674a51d7a3d4a122a6ceb35f1695831314867 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
878377d3 | 20-Jan-2021 |
Adriana Kobylak <anoo@us.ibm.com> |
Revert "Factory Reset: Wait for env variables"
This reverts commit 60f5ccfd5ab0fc8cedc3a2bf5f5adcab77318b7d.
There's no need to sleep in order to wait for the systemd service file to hopefully fini
Revert "Factory Reset: Wait for env variables"
This reverts commit 60f5ccfd5ab0fc8cedc3a2bf5f5adcab77318b7d.
There's no need to sleep in order to wait for the systemd service file to hopefully finish. The factory reset helper functions now executes in a child process and are waited on.
Tested: Added debug log messages to verify that the reset function returned until the environment variable was set.
Change-Id: I22a137c9440c9e971d9606c5efca3fc14d44c3eb Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
56ec62e2 | 20-Jan-2021 |
Adriana Kobylak <anoo@us.ibm.com> |
reset: Call execute() instead of systemd service
The obmc-flash-bmc-setenv@.service file calls a script that runs the fw_setenv command. Call the fw_setenv command directly as an effort to move away
reset: Call execute() instead of systemd service
The obmc-flash-bmc-setenv@.service file calls a script that runs the fw_setenv command. Call the fw_setenv command directly as an effort to move away from systemd service calls which are async calls and are harder to monitor for completion.
The only difference is that the script calls fw_setenv twice because for certain variables such as field mode, the code parses the actual environment file to get the value, and if the system supports a redundant environment, then fw_setenv is needed to be called twice so that the updated value is reflected in both redundant copies and the code parses the value correctly.
Factory reset is not a value that is parsed by this code, so a single call is enough.
Tested: Verified in witherspoon that the rwreset environment variable was set and factory reset occurred upon reboot.
Change-Id: I76691b1c8c78aa9232074d10d435a3e944b1cd9a Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
8a5ccbb5 | 20-Jan-2021 |
Adriana Kobylak <anoo@us.ibm.com> |
utils: Add execute command
As issue openbmc/phosphor-bmc-code-mgmt#6 describes, the code makes use of the systemd services to execute commands via a script, but calls to systemd service files are as
utils: Add execute command
As issue openbmc/phosphor-bmc-code-mgmt#6 describes, the code makes use of the systemd services to execute commands via a script, but calls to systemd service files are async, so if there is a need to wait for the service to finish executing, workarounds like sleep commands have been added to the code, ex: https://github.com/openbmc/phosphor-bmc-code-mgmt/commit/60f5ccfd5ab0fc8cedc3a2bf5f5adcab77318b7d
Create a function that would execute a command in a child process so that it's possible to wait for it to finish. In addition, errors can be more obvious by checking the return of the execute function and taking action to notify the caller of the error instead of relying on system unit dependencies to run recovery actions on error.
Tested: Called new execute function from the code and verified it was successful. Ex: utils::execute("/bin/mkdir", "/tmp/test-execute");
Change-Id: I81a6aa0a50276abb6aba40196a214629ec9baa13 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
0a06e97f | 16-Dec-2020 |
George Liu <liuxiwei@inspur.com> |
Add support for tarball contents signature
Currently only supported to allow optional image files in BMC tarball. In order to ensure that the contents of the tarball are the expected ones as a full
Add support for tarball contents signature
Currently only supported to allow optional image files in BMC tarball. In order to ensure that the contents of the tarball are the expected ones as a full package, an additional signature file has been created for all the signature files in the tarball, (ex: image-full.sig = hash(file1.sig + file2.sig + file3.sig...)). Need to check the existence of the file and the signature verification passed.
Also, added unit test case for the mergeFiles method.
Tested: Enable `WANT_SIGNATURE_FULL_VERIFY` and ran the following command:
curl -k -H "X-Auth-Token: $token" -H "Content-Type: application/octet-stream" -X POST -T obmc-phosphor-image-fp5280g2.static.mtd.tar
https://${bmc}/redfish/v1/UpdateService { "@odata.id": "/redfish/v1/TaskService/Tasks/1", "@odata.type": "#Task.v1_4_3.Task", "Id": "1", "TaskState": "Running", "TaskStatus": "OK" }
And Log output: `Successfully completed Signature vaildation.`
Signed-off-by: George Liu <liuxiwei@inspur.com> Change-Id: I0e658b9dd90ea405a9c8292f29183ab516a0fa31
show more ...
|
574f94bc | 05-Jan-2021 |
Henry Tian <tianxiaofeng@bytedance.com> |
image_verify: Support full image
The code only supports verify the static.mtd.tar, but not static.mtd.all.tar.
This change adds the signature verify for the full tarball.
Tested: Update .static.mt
image_verify: Support full image
The code only supports verify the static.mtd.tar, but not static.mtd.all.tar.
This change adds the signature verify for the full tarball.
Tested: Update .static.mtd.all.tar with Redfish API OK. Update .static.mtd.tar with Redfish API OK.
Signed-off-by: Henry Tian <tianxiaofeng@bytedance.com> Signed-off-by: Lei YU <yulei.sh@bytedance.com> Change-Id: I58a47d76ef090644cc1e870077f438b8768f6aa9
show more ...
|
837de881 | 03-Nov-2020 |
Lei YU <yulei.sh@bytedance.com> |
flash-bios: Do not delete image in case of failure
The code was to delete the BIOS image after the BIOS code update is finished no matter if the update is successful or not.
Change the logic to not
flash-bios: Do not delete image in case of failure
The code was to delete the BIOS image after the BIOS code update is finished no matter if the update is successful or not.
Change the logic to not delete the image when it fails, so that we could examine what's wrong and retry the update.
To delete the image, use DELETE API.
Tested: Verify the image is not deleted when BIOS code update fails.
Signed-off-by: Lei YU <yulei.sh@bytedance.com> Change-Id: Ic4d985b0e48bc134f710590aa06098b1e15b1c27
show more ...
|
9b8816f5 | 02-Dec-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
obmc-flash-bmc: Specify full path for mktemp
Some eMMC systems report the following error when updating u-boot:
Dec 02 18:14:40 openbmc systemd[1]: Starting Write image 3ca0afa1 to BMC storage... D
obmc-flash-bmc: Specify full path for mktemp
Some eMMC systems report the following error when updating u-boot:
Dec 02 18:14:40 openbmc systemd[1]: Starting Write image 3ca0afa1 to BMC storage... Dec 02 18:14:41 openbmc obmc-flash-bmc[507]: mktemp: (null): Read-only file system Dec 02 18:14:41 openbmc obmc-flash-bmc[508]: dd: can't open '': No such file or directory Dec 02 18:14:41 openbmc obmc-flash-bmc[510]: /usr/bin/obmc-flash-bmc: line 462: : No such file or directory
It appears that the TMPDIR variable is not set in that environment, even though that a manual mktemp command works, so may be tied to the script running in a bash shell.
In any case, specify the full path for the mktemp directory so that it creates the file in /tmp/ and doesn't attempt to create it in the running rootfs directory which is read-only.
Tested: Error was not seeing in the failing system and u-boot was updated.
Change-Id: I407a1e4f0d185d8fbb1e4f939bf8283246299a2d Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
62f3820b | 29-Sep-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Update u-boot if needed
Calculate a hash over the u-boot image file and the mmc device where u-boot is stored to determine if it needs to be updated.
Tested: Verified that after the mmc device
mmc: Update u-boot if needed
Calculate a hash over the u-boot image file and the mmc device where u-boot is stored to determine if it needs to be updated.
Tested: Verified that after the mmc device is updated, an update to the same image does not rewrite the device where u-boot is.
Change-Id: I9f3cb8cc639512c02030b7be34845a67f3e15319 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
34077dc9 | 11-Sep-2020 |
Gunnar Mills <gmills@us.ibm.com> |
Fix flash-layout.md link
31de159f moved flash-layout.md to architecture/
Change-Id: I023e0a8c061b5efa74a7c850a6249627f03bf536 Signed-off-by: Gunnar Mills <gmills@us.ibm.com> |
d148b4f0 | 27-Aug-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Remove hostfw patches on update
Remove the hostfw patch for the alternate side during code update since that's the side that's been updated and the patches may not be compatible with the new fi
mmc: Remove hostfw patches on update
Remove the hostfw patch for the alternate side during code update since that's the side that's been updated and the patches may not be compatible with the new firmware level.
Change-Id: I95eda772334673890b772587bfc89698fefbd4c6 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
13e09911 | 31-Aug-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
image_manager: Delete image dir if same version exists
If the uploaded version already exists, no version D-Bus object is created, but the directory in /tmp/ with the image files remain even though
image_manager: Delete image dir if same version exists
If the uploaded version already exists, no version D-Bus object is created, but the directory in /tmp/ with the image files remain even though there is no use for it, also there is no way for the user to delete it except rebooting the BMC. This takes up space and may cause subsequent uploads to fail with out-of-space. Delete it if the version already exists.
Tested: Uploading the same image deletes the dir in /tmp.
Change-Id: I9518f8132c18c2e7a20e330e8f894caa7bab6236 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
c31943e1 | 29-Jul-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Remove: Sleep to allow service to finish
There's not currently a good solution for waiting for systemd service files before returning from a function. The way to achieve this is to add a sleep
mmc: Remove: Sleep to allow service to finish
There's not currently a good solution for waiting for systemd service files before returning from a function. The way to achieve this is to add a sleep before returning.
Need to add a sleep for the mmc remove function to allow the device to be erased before the update, otherwise the delete and update may run at the same time causing corruption.
This race condition is not seeing in ubi because the update creates a new volume for the image, while in the background the old volume is being erased, although this could still cause issues if space is tight and the delete is not finished by the time the new image is written, but this issue hasn't been seen so focusing on mmc only for this commit.
Tested: Verified no corruption was seen on the tacoma system where the issue was reported.
Change-Id: I89eeb97b357cc6a0eb0bf01aa9ac4147cab6168a Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
8e9ccfe7 | 18-Nov-2019 |
Bright Cheng <bright_cheng@wiwynn.com> |
Add support for full BMC FW flash image
Add BMC flash file name list for full flash image. Save the information for which images are being updated.
Tested: Update '.static.mtd.all.tar' with redfish
Add support for full BMC FW flash image
Add BMC flash file name list for full flash image. Save the information for which images are being updated.
Tested: Update '.static.mtd.all.tar' with redfish API ( UpdateService.SimpleUpdate). Verified the code update works well.
Change-Id: Icb47e518db61a8d17998179aed328d0cf56db6f5 Signed-off-by: Bright Cheng <bright_cheng@wiwynn.com> Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
21dad047 | 26-Jun-2020 |
Miguel Gomez <mgomez@mx1.ibm.com> |
BMC Minimum ship Level
This code compares the BMC_MSL defined at compile time, It will parse the new fw (explicit at MANIFEST file) and use regex to evaluate it and compare BMC_MSL against version
BMC Minimum ship Level
This code compares the BMC_MSL defined at compile time, It will parse the new fw (explicit at MANIFEST file) and use regex to evaluate it and compare BMC_MSL against version from MANIFEST. If newer or equal it will apply, otherwise it will fail, preventing activation operation.
Tested: regex-bmc-msl="([a-z]+[0-9]{2})+([0-9]+).([0-9]+).([0-9]+)" fw-package="version=fw1010.00-28.4-0-ge611abca21" bmc-msl="fw1010.00-27" proceeds with activation... bmc-msl="fw1010.00-29" returns:
Jul 15 20:35:45 tacoma1z-w81 phosphor-image-updater[766]: BMC Minimum Ship Level NOT met
Jul 15 20:35:45 tacoma1z-w81 phosphor-image-updater[766]: A system component has a software version that is incompatible as determined by the implementation and needs to be updated....
Signed-off-by: Miguel Gomez <mgomez@mx1.ibm.com> Change-Id: I0ab0eba7c7c89f38ca698aa3e369aa50797edb07
show more ...
|
5312d854 | 18-Jul-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Enhance get running partition label function
The mount -l command may not show the filesystem label after a code update since the this filesystem label is part of the image when it gets built.
mmc: Enhance get running partition label function
The mount -l command may not show the filesystem label after a code update since the this filesystem label is part of the image when it gets built. Therefore determine the running partition via the device then map it to the partition label (partlabel) which doesn't change. Also return the label only if found, don't assume that because 'a' didn't match that the device should be 'b'.
Tested: Verified the right label was found.
Change-Id: Id50223e21e9c6cc280f98f2b928d37a36de48cc9 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
8c5209d3 | 12-Jul-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Add support for update/delete hostfw
The hostfw image file can be delivered in the code update tarball. If it is, then update it (copy the file to the hostfw partition directory.
To delete it,
mmc: Add support for update/delete hostfw
The hostfw image file can be delivered in the code update tarball. If it is, then update it (copy the file to the hostfw partition directory.
To delete it, need to first unmount the secondary (non- running) hostfw mount, then delete the file.
Change-Id: I352dd017114d58b79f9e624983841eafce5badf1 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
73609bb5 | 18-Jun-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
Add support for optional images
Add support to allow optional image files in the BMC tarball. This can be used for example to add a host bios file in a "System" image, or for elements that as best p
Add support for optional images
Add support to allow optional image files in the BMC tarball. This can be used for example to add a host bios file in a "System" image, or for elements that as best practice should not be updated unless necessary like for a bug fix, such as the U-Boot SPL.
This commit provides the ability to add these optional images from a list of pre-defined names. These files will then go through the same checks as the default image files such as checking the files exist and the signature verification (if enabled) passes.
Change-Id: I304b4e28c776db4a51537613888b4e11824cab88 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
d7fbc1e3 | 06-Jun-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Add mount services
The iterm updater relies on the versions being mounted in the MEDIA_DIR in order to create the D-Bus objects when the BMC first starts up.
Create two services: one to mount
mmc: Add mount services
The iterm updater relies on the versions being mounted in the MEDIA_DIR in order to create the D-Bus objects when the BMC first starts up.
Create two services: one to mount the emmc rootfs partitions so that the version D-Bus objects get created, and another one to unmount them after the item updater has run because there's no need to keep the mounts around, actually they shouldn't be mounted during a code update. The UBI layout does an unmount at the start of an update, but it's less risky to just unmount once the mount is no longer needed in case the update is done manually by starting the update service file or directly calling obmc-flash-bmc.
These services get started automatically by adding a dependency to the item updater service: "WantedBy=xyz.openbmc_project.Software.BMC.Updater.service" The mount has a "Before" directive and the unmount has an "After" directive. This allows the item updater to have the mounts available for processing and they are unmounted once the processing is complete.
Also add a dash "-" to ExecStart to ignore errors: https://www.freedesktop.org/software/systemd/man/systemd.service.html#ExecStart= This is needed since the non-running filesystem may had been invalidated during a Delete operation. This directive still logs an error but does not put the service file in Failed state: Ex:
Jul 11 16:36:07 rainier systemd[1]: Starting Mount BMC rofs volumes after a reboot... Jul 11 16:36:07 rainier sh[423]: mount: /media/rofs-b: wrong fs type, bad option, bad superblock on /dev/mmcblk0p5, missing codepage or helper program, or other error. Jul 11 16:36:07 rainier systemd[1]: obmc-flash-mmc-mount.service: Succeeded. Jul 11 16:36:07 rainier systemd[1]: Finished Mount BMC rofs volumes after a reboot. Jul 11 16:36:07 rainier systemd[1]: Started OpenBMC Software Update Manager. Jul 11 16:36:07 rainier systemd[1]: Starting Unmount BMC rofs volumes... Jul 11 16:36:08 rainier umount[427]: umount: /media/rofs-b: no mount point specified. Jul 11 16:36:08 rainier systemd[1]: obmc-flash-mmc-umount.service: Succeeded. Jul 11 16:36:08 rainier systemd[1]: Finished Unmount BMC rofs volumes.
Tested: Verified the D-Bus version objects are created after BMC reboot, and the item updater service starts successfully after a Delete operation.
Change-Id: Ic0c83391a2879b8508bf5aa9f18941590d96352d Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
34124351 | 22-May-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Set updated version to primary
Once the update is successful, mark the version as primary so that it boots from the updated version upon BMC reboot.
Add a sleep to wait for the service file to
mmc: Set updated version to primary
Once the update is successful, mark the version as primary so that it boots from the updated version upon BMC reboot.
Add a sleep to wait for the service file to complete setting the primary version. Otherwise, the BMC could mark a Delete or Priority value change as complete and the service is not done yet, and if the BMC is rebooted it could try to boot from a non-existent version.
As backgroung, reference issue openbmc/openbmc#2857 that attempted to create a 'wait for service' function but was not successful. This could be investigated further at a later time, which would benefit other functions like Factory Reset that are also using sleep as a workaround to wait for systemd service files.
Tested: Verified the version was set to primary during code update and delete, and that 3s passed (service file finished in about 1s) before the delete/update continued.
Change-Id: I4f9afdb020d8cc7c18cfdafe468dbff2dc8046c1 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
70f5bc04 | 13-May-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
mmc: Update and remove functions for the kernel and rootfs
Add support for update and remove of eMMC images.
The code update tarball contains an image-kernel and image-rofs files. These can be flas
mmc: Update and remove functions for the kernel and rootfs
Add support for update and remove of eMMC images.
The code update tarball contains an image-kernel and image-rofs files. These can be flashed directly to the eMMC partitions since they're filesystem files compressed with zstd:
https://gerrit.openbmc-project.xyz/c/openbmc/meta-phosphor/+/30781 https://gerrit.openbmc-project.xyz/c/openbmc/meta-phosphor/+/34334
The image-rofs contains the BMC rootfs files, and the image-kernel contains the fitImage file which is loaded by U-Boot.
Tested: Verified the non-running rofs and kernel partitions were updated.
Change-Id: Ic983dec1df389d56f11f12dc2e82589d1a2b9dcc Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
d1a55adc | 11-Jun-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
item_updater: erase: Call resetUbootEnvVars() first
During erase, move the resetUbootEnvVars() call from the end to the beginning so that the BMC points to a valid image to boot from. If resetUbootE
item_updater: erase: Call resetUbootEnvVars() first
During erase, move the resetUbootEnvVars() call from the end to the beginning so that the BMC points to a valid image to boot from. If resetUbootEnvVars() is called after the image is actually deleted from the BMC flash, there'd be a time window where the BMC would be pointing to a non-existent image to boot from.
Change-Id: I98c69033ee0e9c6e8752200f65f54cfe0f56d130 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
f383d27a | 16-Jun-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
item_updater: Ignore duplicate version ids
If the partitions or devices where the BMC images reside are flashed with the same image file, for example during manufacturing process, we'd end up with d
item_updater: Ignore duplicate version ids
If the partitions or devices where the BMC images reside are flashed with the same image file, for example during manufacturing process, we'd end up with duplicate version ids. Ignore that version for now and treat it as non-existent. A subsequent code update would rewrite that partition.
There is a request about being able to support an Accept/Commit operation which would write the current version to the other BMC versions, so that for example a system with versions that contain a security vulnerability can be updated to the fixed version but also overwrite any other old versions to prevent the system from booting from an old version. So eventually we'll add support to show the system with 2 BMC versions that are the same.
Tested: Flashed an eMMC with the same image on both BMC partitions and verified only one d-bus object was created and the software updater didn't crash.
Change-Id: I95c5ed968df074ff56da3f80c07a95505fbdf2cd Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
716cd78d | 08-Jun-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
processBMCImage: Dynamically calculate version id
With the UBI layout, the name of the mount directories have the version id because the UBI volume names have the id so it's easy to get.
With other
processBMCImage: Dynamically calculate version id
With the UBI layout, the name of the mount directories have the version id because the UBI volume names have the id so it's easy to get.
With other layouts where static partition names are used, the id is not part of the partition name so it can only be calculated from the version string, which requires to mount the filesystem to read it from the os-release file.
Change the logic to have processBMCImage() calculate the id from the version string for each mounted filesystem instead of the mount directory name, as long as the mount directory name starts with BMC_ROFS_PREFIX, ex: /media/rofs-a. The current UBI mount directory names rofs-<id> continue to work, just the directory name is not used anymore to get the id.
If the os-release file is not found or does not contain a version string, assume it's corrupted and attempt to get the id from the mount directory name.
Tested: Verified in witherspoon that the d-bus objects are created with the expected version ids.
Change-Id: Iff0103c9cbd060c7fb49ff29c48a72f2873f0f32 Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|
436be816 | 17-Jun-2020 |
Adriana Kobylak <anoo@us.ibm.com> |
build: Rename .service files to service.in
Rename the .service files to .service.in to allow service files to support config variable substitution. Ex, this line in .in: ExecStart=/usr/bin/obmc-fl
build: Rename .service files to service.in
Rename the .service files to .service.in to allow service files to support config variable substitution. Ex, this line in .in: ExecStart=/usr/bin/obmc-flash-bmc ubiro @IMG_UPLOAD_DIR@ would become this in .service file: ExecStart=/usr/bin/obmc-flash-bmc ubiro "/tmp/images"
This allows the service files to use the user defined config values instead of making assumptions that the default is used.
This commit only renames the service files. The subsequent commit will make use of this feature.
In addition, the subdir meson.build files were deleted with this commit. The reason is that the service files need to be moved to the main meson.build to be able to read the conf file, and without the service files, the subdir meson.build files are just a couple lines long so there's not really a reason to have a separate meson.build file.
Tested: Verified the service files were installed as before.
Change-Id: I29190d1f0664352fbee8a0ada9de959fd0269d6e Signed-off-by: Adriana Kobylak <anoo@us.ibm.com>
show more ...
|