xref: /openbmc/bmcweb/TESTING.md (revision 8ece0e45) 
1# OpenBMC Webserver Testing
2
3This doc describes what type of functional testing, which contributors should
4perform before requesting code reviews, and tips for each type. Contributors
5should add types of testing they performed and their results as "Tested" footers
6in commit messages.
7
8## General Methodology
9
10There are a variety of ways to develop and test bmcweb software changes. Here
11are the steps for using the SDK and QEMU.
12
13- Follow all
14  [development environment setup](https://github.com/openbmc/docs/blob/master/development/dev-environment.md)
15  directions in the development environment setup document. This will get QEMU
16  started up and you in the SDK environment.
17- Follow all of the
18  [gerrit setup](https://github.com/openbmc/docs/blob/master/development/gerrit-setup.md)
19  directions in the gerrit setup document.
20- Clone bmcweb from gerrit
21
22  ```sh
23  git clone ssh://openbmc.gerrit/openbmc/bmcweb/
24  ```
25
26- Follow directions in
27  [README.md](https://github.com/openbmc/bmcweb#configuration) to compile
28
29- Reduce binary size by stripping it when ready for testing
30
31  ```sh
32  arm-openbmc-linux-gnueabi-strip bmcweb
33  ```
34
35  **Note:** Stripping is not required and having the debug symbols could be
36  useful depending on your testing. Leaving them will drastically increase your
37  transfer time to the BMC.
38
39- Copy your bmcweb you want to test to /tmp/ in QEMU
40
41  ```sh
42  scp -P 2222 bmcweb root@127.0.0.1:/tmp/
43  ```
44
45  **Special Notes:** The address and port shown here (127.0.0.1 and 2222)
46  reaches the QEMU session you set up in your development environment as
47  described above.
48
49- Stop bmcweb service within your QEMU session
50
51  ```sh
52  systemctl stop bmcweb
53  ```
54
55  **Note:** bmcweb supports being started directly in parallel with the bmcweb
56  running as a service. The standalone bmcweb will be available on port 18080.
57  An advantage of this is you can compare between the two easily for testing. In
58  QEMU you would need to open up port 18080 when starting QEMU. Your curl
59  commands would need to use 18080 to communicate.
60
61- If running within a system that has read-only /usr/ filesystem, issue the
62  following commands one time per QEMU boot to make the filesystem writable
63
64  ```sh
65  mkdir -p /var/persist/usr
66  mkdir -p /var/persist/work/usr
67  mount -t overlay -o lowerdir=/usr,upperdir=/var/persist/usr,workdir=/var/persist/work/usr overlay /usr
68  ```
69
70- Remove the existing bmcweb from the filesystem in QEMU
71
72  ```sh
73  rm /usr/bin/bmcweb
74  ```
75
76- Link to your new bmcweb in /tmp/
77
78  ```sh
79  ln -sf /tmp/bmcweb /usr/bin/bmcweb
80  ```
81
82- Test your changes. bmcweb will be started automatically upon your first REST
83  or Redfish command
84
85  ```sh
86  curl -c cjar -b cjar -k -H "Content-Type: application/json" -X POST https://127.0.0.1:2443/login -d "{\"data\": [ \"root\", \"0penBmc\" ] }"
87  curl -c cjar -b cjar -k -X GET https://127.0.0.1:2443/xyz/openbmc_project/state/bmc0
88  ```
89
90- Stop the bmcweb service and scp new file over to /tmp/ each time you want to
91  retest a change.
92
93  See the [REST](https://github.com/openbmc/docs/blob/master/REST-cheatsheet.md)
94  and
95  [Redfish](https://github.com/openbmc/docs/blob/master/REDFISH-cheatsheet.md)
96  cheatsheets for valid commands.
97
98## Types of Changes to Run
99
100A committer should run tests that exercise all paths changed in the patchset. If
101making changes to the http core, that are expected to effect all types of
102routes, testing one route of each class (websocket, HTTP get, HTTP post) is
103required.
104
105## Typical Types of Changes
106
107### Basic Redfish Robot Test on QEMU
108
109Run the
110[upstream Robot QEMU test](https://github.com/openbmc/openbmc-build-scripts/blob/master/run-qemu-robot-test.sh).
111This test is performed automatically when bumping SRCREV. Ensuring this test
112passing makes your CL less likely to be rolled back while bumping SRCREV of
113bmcweb.
114
115### Websocket
116
117Turn on the `rest` meson option which provides a websocket route.
118
119```bash
120# run the websocket testing script and verify results
121$ python scripts/websocket_test.py  --host 1.2.3.4:443 --ssl
122```
123
124### Redfish Validator
125
126Committers are required to run the
127[Redfish Validator](https://github.com/DMTF/Redfish-Service-Validator.git)
128anytime they make a change to the GET behavior of the redfish tree. The test
129must run on real hardware since the resource tree will be more complete.
130
131```bash
132$ git clone https://github.com/DMTF/Redfish-Service-Validator.git
133
134# run validator and inspect the report for failures
135$ python3 RedfishServiceValidator.py \
136  --auth Session -i https://1.2.3.4:443 \
137  -u root -p 0penBmc
138```
139
140Your change should not introduce any new validator errors. Please include
141something to the effect of "Redfish service validator passing" in your commit
142message.
143
144### Error Status
145
146Test error status for your newly added resources or core codes, e.g., 4xx client
147errors, 5xx server errors.
148