1# Running Toaster's browser-based test suite
2
3These tests require Selenium to be installed in your Python environment.
4
5The simplest way to install this is via pip3:
6
7  pip3 install selenium==2.53.2
8
9Note that if you use other versions of Selenium, some of the tests (such as
10tests.browser.test_js_unit_tests.TestJsUnitTests) may fail, as these rely on
11a Selenium test report with a version-specific format.
12
13To run tests against Chrome:
14
15* Download chromedriver for your host OS from
16  https://sites.google.com/a/chromium.org/chromedriver/downloads
17* On *nix systems, put chromedriver on PATH
18* On Windows, put chromedriver.exe in the same directory as chrome.exe
19
20To run tests against PhantomJS (headless):
21--NOTE - Selenium seems to be deprecating support for this mode ---
22* Download and install PhantomJS:
23  http://phantomjs.org/download.html
24* On *nix systems, put phantomjs on PATH
25* Not tested on Windows
26
27To run tests against Firefox, you may need to install the Marionette driver,
28depending on how new your version of Firefox is. One clue that you need to do
29this is if you see an exception like:
30
31  selenium.common.exceptions.WebDriverException: Message: The browser
32  appears to have exited before we could connect. If you specified
33  a log_file in the FirefoxBinary constructor, check it for details.
34
35See https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver
36for installation instructions. Ensure that the Marionette executable (renamed
37as wires on Linux or wires.exe on Windows) is on your PATH; and use "marionette"
38as the browser string passed via TOASTER_TESTS_BROWSER (see below).
39
40(Note: The Toaster tests have been checked against Firefox 47 with the
41Marionette driver.)
42
43The test cases will instantiate a Selenium driver set by the
44TOASTER_TESTS_BROWSER environment variable, or Chrome if this is not specified.
45
46To run tests against the Selenium Firefox Docker container:
47More explanation is located at https://wiki.yoctoproject.org/wiki/TipsAndTricks/TestingToasterWithContainers
48* Run the Selenium container:
49  ** docker run -it --rm=true -p 5900:5900 -p 4444:4444 --name=selenium selenium/standalone-firefox-debug:2.53.0
50  *** 5900 is the default vnc port. If you are runing a vnc server on your machine map a different port e.g. -p 6900:5900 and connect vnc client to 127.0.0.1:6900
51  *** 4444 is the default selenium sever port.
52* Run the tests
53  ** TOASTER_TESTS_BROWSER=http://127.0.0.1:4444/wd/hub TOASTER_TESTS_URL=http://172.17.0.1:8000 ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser
54  ** TOASTER_TESTS_BROWSER=remote TOASTER_REMOTE_HUB=http://127.0.0.1:4444/wd/hub ./bitbake/lib/toaster/manage.py test --liveserver=172.17.0.1:8000 tests.browser
55  *** TOASTER_REMOTE_HUB - This is the address for the Selenium Remote Web Driver hub. Assuming you ran the contianer with -p 4444:4444 it will be http://127.0.0.1:4444/wd/hub.
56  ***  --liveserver=xxx tells Django to run the test server on an interface and port reachable by both host and container.
57  **** 172.17.0.1 is the default docker bridge on linux, viewable from inside and outside the contianers. Find it with "ip -4 addr show dev docker0"
58* connect to the vnc server to see the tests if you would like
59   ** xtightvncviewer 127.0.0.1:5900
60   ** note, you need to wait for the test container to come up before this can connect.
61
62Available drivers:
63
64* chrome (default)
65* firefox
66* marionette (for newer Firefoxes)
67* ie
68* phantomjs (deprecated)
69* remote
70
71e.g. to run the test suite with phantomjs where you have phantomjs installed
72in /home/me/apps/phantomjs:
73
74PATH=/home/me/apps/phantomjs/bin:$PATH TOASTER_TESTS_BROWSER=phantomjs manage.py test tests.browser
75