xref: /openbmc/bmcweb/TESTING.md (revision 4cee35e7) 
1# OpenBMC Webserver Testing
2
3This doc describes what type of functional testing, which contributors should
4perform before requesting code reviews, and tips for each type. Contributors
5should add types of testing they performed and their results as "Tested" footers
6in commit messages.
7
8## General Methodology
9There are a variety of ways to develop and test bmcweb software changes.
10Here are the steps for using the SDK and QEMU.
11
12- Follow all [development environment setup](https://github.com/openbmc/docs/blob/master/development/dev-environment.md)
13directions in the development environment setup document. This will get
14QEMU started up and you in the SDK environment.
15- Follow all of the [gerrit setup](https://github.com/openbmc/docs/blob/master/development/gerrit-setup.md)
16directions in the gerrit setup document.
17- Clone bmcweb from gerrit
18  ```
19  git clone ssh://openbmc.gerrit/openbmc/bmcweb/
20  ```
21
22- Follow directions in [README.md](https://github.com/openbmc/bmcweb#configuration) to compile
23
24- Reduce binary size by stripping it when ready for testing
25  ```
26  arm-openbmc-linux-gnueabi-strip bmcweb
27  ```
28  **Note:** Stripping is not required and having the debug symbols could be
29  useful depending on your testing. Leaving them will drastically increase
30  your transfer time to the BMC.
31
32- Copy your bmcweb you want to test to /tmp/ in QEMU
33  ```
34  scp -P 2222 bmcweb root@127.0.0.1:/tmp/
35  ```
36  **Special Notes:**
37  The address and port shown here (127.0.0.1 and 2222) reaches the QEMU session
38  you set up in your development environment as described above.
39
40- Stop bmcweb service within your QEMU session
41  ```
42  systemctl stop bmcweb
43  ```
44  **Note:** bmcweb supports being started directly in parallel with the bmcweb
45  running as a service. The standalone bmcweb will be available on port 18080.
46  An advantage of this is you can compare between the two easily for testing.
47  In QEMU you would need to open up port 18080 when starting QEMU. Your curl
48  commands would need to use 18080 to communicate.
49
50- If running within a system that has read-only /usr/ filesystem, issue
51the following commands one time per QEMU boot to make the filesystem
52writeable
53  ```
54  mkdir -p /var/persist/usr
55  mkdir -p /var/persist/work/usr
56  mount -t overlay -o lowerdir=/usr,upperdir=/var/persist/usr,workdir=/var/persist/work/usr overlay /usr
57  ```
58
59- Remove the existing bmcweb from the filesystem in QEMU
60  ```
61  rm /usr/bin/bmcweb
62  ```
63
64- Link to your new bmcweb in /tmp/
65  ```
66  ln -sf /tmp/bmcweb /usr/bin/bmcweb
67  ```
68
69- Test your changes. bmcweb will be started automatically upon your
70first REST or Redfish command
71  ```
72  curl -c cjar -b cjar -k -X POST https://127.0.0.1:2443/login -d "{\"data\": [ \"root\", \"0penBmc\" ] }"
73  curl -c cjar -b cjar -k -X GET https://127.0.0.1:2443/xyz/openbmc_project/state/bmc0
74  ```
75
76- Stop the bmcweb service and scp new file over to /tmp/ each time you
77want to retest a change.
78
79  See the [REST](https://github.com/openbmc/docs/blob/master/REST-cheatsheet.md)
80  and [Redfish](https://github.com/openbmc/docs/blob/master/REDFISH-cheatsheet.md) cheatsheets for valid commands.
81
82
83## Types of Changes to Run
84A committer should run tests that exercise all paths changed in the patchset.
85If making changes to the http core, that are expected to effect all types of
86routes, testing one route of each class (websocket, HTTP get, HTTP post) is
87required.
88
89## Typical Types of Changes
90
91### Basic Redfish Robot Test on QEMU
92
93Run the [upstream Robot QEMU test](https://github.com/openbmc/openbmc-build-scripts/blob/master/run-qemu-robot-test.sh).
94This test is performed automatically when bumping SRCREV. Ensuring this test
95passing makes your CL less likely to be rolled back while bumping SRCREV of
96bmcweb.
97
98### Websocket
99
100Turn on the `rest` meson option which provides a websocket route.
101
102```bash
103# run the websocket testing script and verify results
104$ python scripts/websocket_test.py  --host 1.2.3.4:443 --ssl
105```
106
107### Redfish Validator
108
109Commiters are required to run the [Redfish Validator](https://github.com/DMTF/Redfish-Service-Validator.git)
110anytime they make a change to the GET behavior of the redfish tree. The test
111must run on real hardware since the resource tree will be more complete.
112
113```bash
114$ git clone https://github.com/DMTF/Redfish-Service-Validator.git
115
116# run validator and inspect the report for failures
117$ python3 RedfishServiceValidator.py \
118  --auth Session -i https://1.2.3.4:443 \
119  -u root -p 0penBmc
120```
121
122Your change should not introduce any new validator errors. Please include
123something to the effect of "Redfish service validator passing" in your commit
124message.
125
126### Error Status
127
128Test error status for your newly added resources or core codes, e.g., 4xx client
129errors, 5xx server errors.
130