123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236 |
- <html>
- <head>
- <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
- <title>Assertion severity level</title>
- <link rel="stylesheet" href="../../boostbook.css" type="text/css">
- <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
- <link rel="home" href="../../index.html" title="Boost.Test">
- <link rel="up" href="../testing_tools.html" title="Writing unit tests">
- <link rel="prev" href="../testing_tools.html" title="Writing unit tests">
- <link rel="next" href="boost_test_universal_macro.html" title="BOOST_TEST: universal and general purpose assertions">
- </head>
- <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
- <table cellpadding="2" width="100%"><tr>
- <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../boost.png"></td>
- <td align="center"><a href="../../../../../../index.html">Home</a></td>
- <td align="center"><a href="../../../../../../libs/libraries.htm">Libraries</a></td>
- <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
- <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
- <td align="center"><a href="../../../../../../more/index.htm">More</a></td>
- </tr></table>
- <hr>
- <div class="spirit-nav">
- <a accesskey="p" href="../testing_tools.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
- </div>
- <div class="section">
- <div class="titlepage"><div><div><h3 class="title">
- <a name="boost_test.testing_tools.tools_assertion_severity_level"></a><a class="link" href="tools_assertion_severity_level.html" title="Assertion severity level">Assertion
- severity level</a>
- </h3></div></div></div>
- <p>
- There are three <span class="bold"><strong>levels</strong></span> of assertions and
- all the testing tools are supplied in these three flavours/levels. These
- levels have different meaning on the consistency of the test case:
- </p>
- <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
- <li class="listitem">
- <code class="computeroutput"><span class="identifier">REQUIRE</span></code> which implements
- a <span class="bold"><strong>requirements</strong></span> : this is a strong condition
- for the operations following the assertion to be valid. This type of
- assertions should be used when a pre-condition for running the test is
- not met or when the test-case cannot continue. If such as assertion fails,
- the test case execution stops immediately, and the test-case is flagged
- as <span class="emphasis"><em>failed</em></span>.
- </li>
- <li class="listitem">
- <code class="computeroutput"><span class="identifier">CHECK</span></code> for standard <span class="bold"><strong>checks</strong></span>: this is the most commonly used assertion
- level. If the statement evaluates to <code class="computeroutput"><span class="keyword">false</span></code>,
- the test case is flagged as failed but its execution continues.
- </li>
- <li class="listitem">
- <code class="computeroutput"><span class="identifier">WARN</span></code> which stands for
- <span class="bold"><strong>warnings</strong></span>: this is an assertion providing
- information. The test case execution continues and a warning message
- is logged. The warning does not change the success status of a test case.
- This level of assertion can be used to validate aspects less important
- then correctness: performance, portability, usability, etc.
- </li>
- </ul></div>
- <p>
- For example:
- </p>
- <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
- <li class="listitem">
- <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_level_throw.html" title="BOOST_<level>_THROW"><code class="computeroutput"><span class="identifier">BOOST_REQUIRE_THROW</span></code></a>, <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST_REQUIRE</span></code></a>
- </li>
- <li class="listitem">
- <code class="computeroutput"><span class="identifier">BOOST_CHECK_THROW</span></code>, <code class="computeroutput"><span class="identifier">BOOST_TEST</span></code> <a href="#ftn.boost_test.testing_tools.tools_assertion_severity_level.f0" class="footnote" name="boost_test.testing_tools.tools_assertion_severity_level.f0"><sup class="footnote">[6]</sup></a>
- </li>
- <li class="listitem">
- <code class="computeroutput"><span class="identifier">BOOST_WARN_THROW</span></code>, <code class="computeroutput"><span class="identifier">BOOST_TEST_WARN</span></code>
- </li>
- </ul></div>
- <p>
- These three levels of assertions are filtered by the framework and reported
- into the test log and output:
- </p>
- <div class="orderedlist"><ol class="orderedlist" type="1">
- <li class="listitem">
- If an assertion designated by the tool passes, confirmation message can
- be printed in log output <a href="#ftn.boost_test.testing_tools.tools_assertion_severity_level.f1" class="footnote" name="boost_test.testing_tools.tools_assertion_severity_level.f1"><sup class="footnote">[7]</sup></a>.
- </li>
- <li class="listitem">
- If an assertion designated by the tool fails, the following will happen,
- depending on the assertion level <a href="#ftn.boost_test.testing_tools.tools_assertion_severity_level.f2" class="footnote" name="boost_test.testing_tools.tools_assertion_severity_level.f2"><sup class="footnote">[8]</sup></a>:
- </li>
- </ol></div>
- <div class="table">
- <a name="boost_test.testing_tools.tools_assertion_severity_level.assertions_severity_levels"></a><p class="title"><b>Table 3. Assertions severity levels</b></p>
- <div class="table-contents"><table class="table" summary="Assertions severity levels">
- <colgroup>
- <col>
- <col>
- <col>
- <col>
- </colgroup>
- <thead><tr>
- <th>
- <p>
- Level
- </p>
- </th>
- <th>
- <p>
- Test log content
- </p>
- </th>
- <th>
- <p>
- Errors counter
- </p>
- </th>
- <th>
- <p>
- Test execution
- </p>
- </th>
- </tr></thead>
- <tbody>
- <tr>
- <td>
- <p>
- WARN
- </p>
- </td>
- <td>
- <p>
- warning in <code class="computeroutput"><span class="special"><</span><span class="identifier">test</span><span class="special">-</span><span class="keyword">case</span><span class="special">-</span><span class="identifier">name</span><span class="special">></span></code>: condition <code class="computeroutput"><span class="special"><</span><span class="identifier">assertion</span> <span class="identifier">description</span><span class="special">></span></code> is not satisfied
- </p>
- </td>
- <td>
- <p>
- not affected
- </p>
- </td>
- <td>
- <p>
- continues
- </p>
- </td>
- </tr>
- <tr>
- <td>
- <p>
- CHECK
- </p>
- </td>
- <td>
- <p>
- error in <code class="computeroutput"><span class="special"><</span><span class="identifier">test</span><span class="special">-</span><span class="keyword">case</span><span class="special">-</span><span class="identifier">name</span><span class="special">></span></code>: test <code class="computeroutput"><span class="special"><</span><span class="identifier">assertion</span> <span class="identifier">description</span><span class="special">></span></code> failed
- </p>
- </td>
- <td>
- <p>
- increased
- </p>
- </td>
- <td>
- <p>
- continues
- </p>
- </td>
- </tr>
- <tr>
- <td>
- <p>
- REQUIRE
- </p>
- </td>
- <td>
- <p>
- fatal error in <code class="computeroutput"><span class="special"><</span><span class="identifier">test</span><span class="special">-</span><span class="keyword">case</span><span class="special">-</span><span class="identifier">name</span><span class="special">></span></code>:
- critical test <code class="computeroutput"><span class="special"><</span><span class="identifier">assertion</span> <span class="identifier">description</span><span class="special">></span></code> failed
- </p>
- </td>
- <td>
- <p>
- increased
- </p>
- </td>
- <td>
- <p>
- aborts
- </p>
- </td>
- </tr>
- </tbody>
- </table></div>
- </div>
- <br class="table-break"><p>
- The granularity of the report depends on the current <a class="link" href="../utf_reference/rt_param_reference/log_level.html" title="log_level">log
- level</a> and <a class="link" href="../utf_reference/rt_param_reference/report_level.html" title="report_level">report
- level</a>.
- </p>
- <div class="note"><table border="0" summary="Note">
- <tr>
- <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../doc/src/images/note.png"></td>
- <th align="left">Note</th>
- </tr>
- <tr><td align="left" valign="top"><p>
- in the above table, the <span class="emphasis"><em>test execution</em></span> is related
- to the current test case <span class="emphasis"><em>only</em></span>. Hence <span class="emphasis"><em>"aborts"</em></span>
- means that the current test case is aborted, but other test cases in the
- test tree are still executed.
- </p></td></tr>
- </table></div>
- <div class="footnotes">
- <br><hr style="width:100; text-align:left;margin-left: 0">
- <div id="ftn.boost_test.testing_tools.tools_assertion_severity_level.f0" class="footnote"><p><a href="#boost_test.testing_tools.tools_assertion_severity_level.f0" class="para"><sup class="para">[6] </sup></a>
- <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> is equivalent to
- <code class="computeroutput"><span class="identifier">BOOST_TEST_CHECK</span></code>
- </p></div>
- <div id="ftn.boost_test.testing_tools.tools_assertion_severity_level.f1" class="footnote"><p><a href="#boost_test.testing_tools.tools_assertion_severity_level.f1" class="para"><sup class="para">[7] </sup></a>
- to manage what messages appear in the test log stream, set the proper
- <a class="link" href="../utf_reference/rt_param_reference/log_level.html" title="log_level"><code class="computeroutput"><span class="identifier">log_level</span></code></a>
- </p></div>
- <div id="ftn.boost_test.testing_tools.tools_assertion_severity_level.f2" class="footnote"><p><a href="#boost_test.testing_tools.tools_assertion_severity_level.f2" class="para"><sup class="para">[8] </sup></a>
- in some cases log message can be slightly different to reflect failed
- tool specifics, see <a class="link" href="reports.html" title="Reported information">here</a>
- </p></div>
- </div>
- </div>
- <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
- <td align="left"></td>
- <td align="right"><div class="copyright-footer">Copyright © 2001-2019 Boost.Test
- contributors<p>
- Distributed under the Boost Software License, Version 1.0. (See accompanying
- file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
- </p>
- </div></td>
- </tr></table>
- <hr>
- <div class="spirit-nav">
- <a accesskey="p" href="../testing_tools.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
- </div>
- </body>
- </html>
|