Merge tag 'dm-pull-29oct19' of git://git.denx.de/u-boot-dm

[oweals/u-boot.git] / test / py / README.md

diff --git a/test/py/README.md b/test/py/README.md
index ea4b66a41ccb84a202567332560c99ddda85acc5..3cbe01b73e2838c6586746a8f7b8baf115025b77 100644 (file)
--- a/test/py/README.md
+++ b/test/py/README.md
@@ -11,7 +11,7 @@ results. Advantages of this approach are:
   U-Boot; there can be no disconnect.
 - There is no need to write or embed test-related code into U-Boot itself.
   It is asserted that writing test-related code in Python is simpler and more
-  flexible that writing it all in C.
+  flexible than writing it all in C.
 - It is reasonably simple to interact with U-Boot in this way.
 
 ## Requirements
@@ -21,18 +21,26 @@ involves executing some binary and interacting with its stdin/stdout. You will
 need to implement various "hook" scripts that are called by the test suite at
 the appropriate time.
 
-On Debian or Debian-like distributions, the following packages are required.
-Some packages are required to execute any test, and others only for specific
-tests. Similar package names should exist in other distributions.
+In order to run the testsuite at a minimum we require that both python3 and
+pip for python3 be installed.  All of the required python modules are
+described in the requirements.txt file in this directory and can be installed
+with the command ```pip install -r requirements.txt```
 
-| Package        | Version tested (Ubuntu 14.04) |
-| -------------- | ----------------------------- |
-| python         | 2.7.5-5ubuntu3                |
-| python-pytest  | 2.5.1-1                       |
-| gdisk          | 0.8.8-1ubuntu0.1              |
-| dfu-util       | 0.5-1                         |
-| dtc            | 1.4.0+dfsg-1                  |
-| openssl        | 1.0.1f-1ubuntu2.22            |
+In order to execute certain tests on their supported platforms other tools
+will be required.  The following is an incomplete list:
+
+| Package        |
+| -------------- |
+| gdisk          |
+| dfu-util       |
+| dtc            |
+| openssl        |
+| sudo OR guestmount |
+| e2fsprogs      |
+| dosfstools     |
+
+Please use the apporirate commands for your distribution to match these tools
+up with the package that provides them.
 
 The test script supports either:
 
@@ -44,18 +52,16 @@ The test script supports either:
 
 ### Using `virtualenv` to provide requirements
 
-Older distributions (e.g. Ubuntu 10.04) may not provide all the required
-packages, or may provide versions that are too old to run the test suite. One
-can use the Python `virtualenv` script to locally install more up-to-date
-versions of the required packages without interfering with the OS installation.
-For example:
+The recommended way to run the test suite, in order to ensure reproducibility
+is to use `virtualenv` to set up the necessary environment.  This can be done
+via the following commands:
 
 ```bash
 $ cd /path/to/u-boot
-$ sudo apt-get install python python-virtualenv
-$ virtualenv venv
+$ sudo apt-get install python3 python3-virtualenv
+$ virtualenv -p /usr/bin/python3 venv
 $ . ./venv/bin/activate
-$ pip install pytest
+$ pip install -r test/py/requirements.txt
 ```
 
 ## Testing sandbox
@@ -150,7 +156,7 @@ processing.
   option takes a single argument which is used to filter test names. Simple
   logical operators are supported. For example:
   - `'ums'` runs only tests with "ums" in their name.
-  - ``ut_dm'` runs only tests with "ut_dm" in their name. Note that in this
+  - `'ut_dm'` runs only tests with "ut_dm" in their name. Note that in this
     case, "ut_dm" is a parameter to a test rather than the test name. The full
     test name is e.g. "test_ut[ut_dm_leak]".
   - `'not reset'` runs everything except tests with "reset" in their name.
@@ -183,7 +189,7 @@ The following environment variables are set when running hook scripts:
 - `UBOOT_TEST_PY_DIR` the full path to `test/py/` in the source directory.
 - `UBOOT_BUILD_DIR` the U-Boot build directory.
 - `UBOOT_RESULT_DIR` the test result directory.
-- `UBOOT_PERSISTENT_DATA_DIR` the test peristent data directory.
+- `UBOOT_PERSISTENT_DATA_DIR` the test persistent data directory.
 
 #### `u-boot-test-console`
 
@@ -222,7 +228,7 @@ the following cases:
   from there. Use of this feature will reduce wear on the board's flash, so
   may be preferable if available, and if cold boot testing of U-Boot is not
   required. If this feature is used, the `u-boot-test-reset` script should
-  peform this download, since the board could conceivably be reset multiple
+  perform this download, since the board could conceivably be reset multiple
   times in a single test run.
 
 It is up to the user to determine if those situations exist, and to code this
@@ -309,6 +315,7 @@ instances of:
 
 - `buildconfig.get(...`
 - `@pytest.mark.buildconfigspec(...`
+- `@pytest.mark.notbuildconfigspec(...`
 
 ### Complete invocation example
 
@@ -320,7 +327,7 @@ If U-Boot has already been built:
 
 ```bash
 PATH=$HOME/ubtest/bin:$PATH \
-    PYTHONPATH=${HOME}/ubtest/py:${PYTHONPATH} \
+    PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \
     ./test/py/test.py --bd seaboard
 ```
 
@@ -331,7 +338,7 @@ follow:
 ```bash
 CROSS_COMPILE=arm-none-eabi- \
     PATH=$HOME/ubtest/bin:$PATH \
-    PYTHONPATH=${HOME}/ubtest/py:${PYTHONPATH} \
+    PYTHONPATH=${HOME}/ubtest/py/${HOSTNAME}:${PYTHONPATH} \
     ./test/py/test.py --bd seaboard --build
 ```