Eric Lee / smarc-uboot

29
Download as
Browse Code ยป

Commit f6349c3c4cd334148637c83bcfb6017b195102f5

Authored by Simon Glass
Committed by Tom Rini
1 parent 022885cb9c
Exists in v2017.01-smarct4x and in 25 other branches 8qm-imx_v2020.04_5.4.70_2.3.0, emb_lf_v2022.04, pitx_8mp_lf_v2020.04, smarc-8m-android-10.0.0_2.6.0, smarc-8m-android-11.0.0_2.0.0, smarc-8mp-android-11.0.0_2.0.0, smarc-imx6_v2018.03_4.14.98_2.0.0_ga, smarc-imx7_v2017.03_4.9.11_1.0.0_ga, smarc-imx7_v2018.03_4.14.98_2.0.0_ga, smarc-imx_v2017.03_4.9.11_1.0.0_ga, smarc-imx_v2017.03_4.9.88_2.0.0_ga, smarc-imx_v2017.03_o8.1.0_1.3.0_8m, smarc-imx_v2018.03_4.14.78_1.0.0_ga, smarc-n7.1.2_2.0.0-ga, smarc_8m-imx_v2018.03_4.14.98_2.0.0_ga, smarc_8m-imx_v2019.04_4.19.35_1.1.0, smarc_8m_00d0-imx_v2018.03_4.14.98_2.0.0_ga, smarc_8mm-imx_v2018.03_4.14.98_2.0.0_ga, smarc_8mm-imx_v2019.04_4.19.35_1.1.0, smarc_8mm-imx_v2020.04_5.4.24_2.1.0, smarc_8mp_lf_v2020.04, smarc_8mq-imx_v2020.04_5.4.24_2.1.0, smarc_8mq_lf_v2020.04, v2017.01-smarct3x, v2017.01-smarct3x-emmc

test: Add a README 

Add a few notes about how testing works in U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>

Showing 1 changed file with 82 additions and 0 deletions Side-by-side Diff

  1 +Testing in U-Boot
  2 +=================
  3 +
  4 +U-Boot has a large amount of code. This file describes how this code is
  5 +tested and what tests you should write when adding a new feature.
  6 +
  7 +
  8 +Sandbox
  9 +-------
  10 +U-Boot can be built as a user-space application (e.g. for Linux). This
  11 +allows test to be executed without needing target hardware. The 'sandbox'
  12 +target provides this feature and it is widely used in tests.
  13 +
  14 +
  15 +Pytest Suite
  16 +------------
  17 +
  18 +Many tests are available using the pytest suite, in test/py. This can run
  19 +either on sandbox or on real hardware. It relies on the U-Boot console to
  20 +inject test commands and check the result. It is slower to run than C code,
  21 +but provides the ability to unify lots of test and summarise their results.
  22 +
  23 +You can run the tests on sandbox with:
  24 +
  25 + ./test/py/test.py --bd sandbox --build
  26 +
  27 +This will produce HTML output in build-sandbox/test-log.html
  28 +
  29 +See test/py/README.md for more information about the pytest suite.
  30 +
  31 +
  32 +tbot
  33 +----
  34 +
  35 +Tbot provides a way to execute tests on target hardware. It is intended for
  36 +trying out both U-Boot and Linux (and potentially other software) on a
  37 +number of boards automatically. It can be used to create a continuous test
  38 +environment. See tools/tbot/README for more information.
  39 +
  40 +
  41 +Ad-hoc tests
  42 +------------
  43 +
  44 +There are several ad-hoc tests which run outside the pytest environment:
  45 +
  46 + test/fs - File system test (shell script)
  47 + test/image - FIT and lagacy image tests (shell script and Python)
  48 + test/stdint - A test that stdint.h can be used in U-Boot (shell script)
  49 + trace - Test for the tracing feature (shell script)
  50 + vboot - Test for verified boot (shell script)
  51 +
  52 +The above should be converted to run as part of the pytest suite.
  53 +
  54 +
  55 +When to write tests
  56 +-------------------
  57 +
  58 +If you add code to U-Boot without a test you are taking a risk. Even if you
  59 +perform thorough manual testing at the time of submission, it may break when
  60 +future changes are made to U-Boot. It may even break when applied to mainline,
  61 +if other changes interact with it. A good mindset is that untested code
  62 +probably doesn't work and should be deleted.
  63 +
  64 +You can assume that the Pytest suite will be run before patches are accepted
  65 +to mainline, so this provides protection against future breakage.
  66 +
  67 +On the other hand there is quite a bit of code that is not covered with tests,
  68 +or is covered sparingly. So here are some suggestions:
  69 +
  70 +- If you are adding a new uclass, add a sandbox driver and a test that uses it
  71 +- If you are modifying code covered by an existing test, add a new test case
  72 + to cover your changes
  73 +- If the code you are modifying has not tests, consider writing one. Even a
  74 + very basic test is useful, and may be picked up and enhanced by others. It
  75 + is much easier to add onto a test - writing a new large test can seem
  76 + daunting to most contributors.
  77 +
  78 +
  79 +Future work
  80 +-----------
  81 +
  82 +Converting existing shell scripts into pytest tests.