assertions_severity_levels.qbk 3.2 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970
  1. [/
  2. / Copyright (c) 2003 Boost.Test contributors
  3. /
  4. / Distributed under the Boost Software License, Version 1.0. (See accompanying
  5. / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
  6. /]
  7. [section:tools_assertion_severity_level Assertion severity level]
  8. There are three *levels* of assertions and all the testing tools are supplied in these three flavours/levels. These levels
  9. have different meaning on the consistency of the test case:
  10. * `REQUIRE` which implements a *requirements* : this is a strong condition for the operations following the assertion to be valid.
  11. This type of assertions should be used when a pre-condition for running the test is not met or when the test-case cannot continue.
  12. If such as assertion fails, the test case execution stops immediately, and the test-case is flagged as /failed/.
  13. * `CHECK` for standard *checks*: this is the most commonly used assertion level. If the statement evaluates to `false`, the test case is
  14. flagged as failed but its execution continues.
  15. * `WARN` which stands for *warnings*: this is an assertion providing information. The test case execution continues and a warning message is logged.
  16. The warning does not change the success status of a test case. This level of assertion can be used
  17. to validate aspects less important then correctness: performance, portability, usability, etc.
  18. For example:
  19. * [link boost_test.utf_reference.testing_tool_ref.assertion_boost_level_throw `BOOST_REQUIRE_THROW`], __BOOST_TEST_REQUIRE__
  20. * `BOOST_CHECK_THROW`, `BOOST_TEST` [footnote __BOOST_TEST__ is equivalent to `BOOST_TEST_CHECK`]
  21. * `BOOST_WARN_THROW`, `BOOST_TEST_WARN`
  22. These three levels of assertions are filtered by the framework and reported into the test log and output:
  23. # If an assertion designated by the tool passes, confirmation message can be printed in log output
  24. [footnote to manage what messages appear in the test log stream, set the proper [link boost_test.utf_reference.rt_param_reference.log_level `log_level`]].
  25. # If an assertion designated by the tool fails, the following will happen, depending on the assertion level
  26. [footnote in some cases log message can be slightly different to reflect failed tool specifics, see [link boost_test.testing_tools.reports here]]:
  27. [table:assertions_severity_levels Assertions severity levels
  28. [
  29. [Level]
  30. [Test log content]
  31. [Errors counter]
  32. [Test execution]
  33. ]
  34. [
  35. [WARN]
  36. [warning in `<test-case-name>`: condition `<assertion description>` is not satisfied]
  37. [not affected]
  38. [continues]
  39. ]
  40. [
  41. [CHECK]
  42. [error in `<test-case-name>`: test `<assertion description>` failed]
  43. [increased]
  44. [continues]
  45. ]
  46. [
  47. [REQUIRE]
  48. [fatal error in `<test-case-name>`: critical test `<assertion description>` failed]
  49. [increased]
  50. [aborts]
  51. ]
  52. ]
  53. The granularity of the report depends on the current [link boost_test.utf_reference.rt_param_reference.log_level log level] and
  54. [link boost_test.utf_reference.rt_param_reference.report_level report level].
  55. [note in the above table, the ['test execution] is related to the current test case ['only]. Hence ['"aborts"] means
  56. that the current test case is aborted, but other test cases in the test tree are still executed.]
  57. [endsect] [/ assertions severity level]