123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258 |
- <html>
- <head>
- <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
- <title>Reported information</title>
- <link rel="stylesheet" href="../../boostbook.css" type="text/css">
- <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
- <link rel="home" href="../../index.html" title="Boost.Test">
- <link rel="up" href="../testing_tools.html" title="Writing unit tests">
- <link rel="prev" href="boost_test_universal_macro.html" title="BOOST_TEST: universal and general purpose assertions">
- <link rel="next" href="extended_comparison.html" title="Extended comparisons support">
- </head>
- <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
- <table cellpadding="2" width="100%"><tr>
- <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../boost.png"></td>
- <td align="center"><a href="../../../../../../index.html">Home</a></td>
- <td align="center"><a href="../../../../../../libs/libraries.htm">Libraries</a></td>
- <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
- <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
- <td align="center"><a href="../../../../../../more/index.htm">More</a></td>
- </tr></table>
- <hr>
- <div class="spirit-nav">
- <a accesskey="p" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="extended_comparison.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
- </div>
- <div class="section">
- <div class="titlepage"><div><div><h3 class="title">
- <a name="boost_test.testing_tools.reports"></a><a class="link" href="reports.html" title="Reported information">Reported information</a>
- </h3></div></div></div>
- <h4>
- <a name="boost_test.testing_tools.reports.h0"></a>
- <span class="phrase"><a name="boost_test.testing_tools.reports.failure_message_why"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.failure_message_why">Failure message,
- why?</a>
- </h4>
- <p>
- When an assertion fails, a message is logged containing:
- </p>
- <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
- <li class="listitem">
- the body of the statement that failed
- </li>
- <li class="listitem">
- the name of the file and the line of the failed assertion
- </li>
- <li class="listitem">
- the name of the test case containing this assertion
- </li>
- </ul></div>
- <p>
- The purpose of all these information is to isolate as quickly as possible
- the test that failed from the others. The <span class="bold"><strong>feedback</strong></span>
- that the execution of the test case provides is an important cue, for the
- following reasons:
- </p>
- <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
- <li class="listitem">
- within the scheme of a continuous build/test, the logs available from
- the server contain this information, which points to a particular statement
- in the code
- </li>
- <li class="listitem">
- the <span class="bold"><strong>cost</strong></span> for reproducing an error is
- induced by the following steps:
- <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; ">
- <li class="listitem">
- identify the test module that failed in case there are many
- </li>
- <li class="listitem">
- compile and run the test module to reproduce the error
- </li>
- <li class="listitem">
- identify the line of the code that failed,
- </li>
- <li class="listitem">
- fix the test directly if all the information is enough, or start
- a debug session
- </li>
- </ul></div>
- </li>
- </ul></div>
- <p>
- We can see from the scheme above that reproduction of an error is <span class="emphasis"><em>costly</em></span>,
- since usually one tends to reproduce the error, which in turn induces at
- least the compilation of the test module. Also, a hidden cost is the lookup
- at the line of code that contains the failing statement, which triggers a
- sequence of back and forth lookup between the log on one hand and the code
- on the other hand.
- </p>
- <p>
- The information extracted from the logs suggests the following fact:
- </p>
- <div class="tip"><table border="0" summary="Tip">
- <tr>
- <td rowspan="2" align="center" valign="top" width="25"><img alt="[Tip]" src="../../../../../../doc/src/images/tip.png"></td>
- <th align="left">Tip</th>
- </tr>
- <tr><td align="left" valign="top"><p>
- Richness of the information contained in the logs is a key for the rapid
- understanding and the resolution of a failed statement
- </p></td></tr>
- </table></div>
- <h4>
- <a name="boost_test.testing_tools.reports.h1"></a>
- <span class="phrase"><a name="boost_test.testing_tools.reports.default_reporting"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.default_reporting">Default
- reporting</a>
- </h4>
- <p>
- When an assertion fails, <a class="link" href="../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> reports details and values
- on the operands of <code class="computeroutput"><span class="identifier">statement</span></code>
- that lead to the failure.
- </p>
- <h6>
- <a name="boost_test.testing_tools.reports.h2"></a>
- <span class="phrase"><a name="boost_test.testing_tools.reports.example_descr"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.example_descr">Example:
- BOOST_TEST reporting</a>
- </h6>
- <div class="informaltable"><table class="table">
- <colgroup><col></colgroup>
- <thead><tr><th>
- <p>
- Code
- </p>
- </th></tr></thead>
- <tbody><tr><td>
- <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">boost_test_macro3</span>
- <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
- <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span> <span class="identifier">test_op_reportings</span> <span class="special">)</span>
- <span class="special">{</span>
- <span class="keyword">int</span> <span class="identifier">a</span> <span class="special">=</span> <span class="number">13</span><span class="special">,</span> <span class="identifier">b</span> <span class="special">=</span> <span class="number">12</span><span class="special">;</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">==</span> <span class="identifier">b</span><span class="special">);</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special"><</span> <span class="identifier">b</span><span class="special">);</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span> <span class="special"><</span> <span class="identifier">b</span><span class="special">);</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">b</span> <span class="special">></span> <span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span><span class="special">);</span>
- <span class="special">}</span>
- </pre>
- </td></tr></tbody>
- </table></div>
- <div class="informaltable"><table class="table">
- <colgroup><col></colgroup>
- <thead><tr><th>
- <p>
- Output
- </p>
- </th></tr></thead>
- <tbody><tr><td>
- <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">></span> <span class="special">./</span><span class="identifier">boost_test_macro3</span> <span class="special">--</span><span class="identifier">log_level</span><span class="special">=</span><span class="identifier">all</span>
- <span class="identifier">Running</span> <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span><span class="special">...</span>
- <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_macro3"</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_op_reportings"</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">15</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">a</span> <span class="special">==</span> <span class="identifier">b</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">13</span> <span class="special">!=</span> <span class="number">12</span><span class="special">]</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">16</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">a</span> <span class="special"><</span> <span class="identifier">b</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">13</span> <span class="special">>=</span> <span class="number">12</span><span class="special">]</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">17</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span> <span class="special"><</span> <span class="identifier">b</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">13</span> <span class="special">-</span> <span class="number">1</span> <span class="special">>=</span> <span class="number">12</span><span class="special">]</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">18</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_op_reportings"</span><span class="special">:</span> <span class="identifier">check</span> <span class="identifier">b</span> <span class="special">></span> <span class="identifier">a</span> <span class="special">-</span> <span class="number">1</span> <span class="identifier">has</span> <span class="identifier">failed</span> <span class="special">[</span><span class="number">12</span> <span class="special"><=</span> <span class="number">12</span><span class="special">]</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_op_reportings"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">484u</span><span class="identifier">s</span>
- <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_macro3"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">588u</span><span class="identifier">s</span>
- <span class="special">***</span> <span class="number">2</span> <span class="identifier">failures</span> <span class="identifier">are</span> <span class="identifier">detected</span> <span class="identifier">in</span> <span class="identifier">the</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_macro3"</span>
- </pre>
- </td></tr></tbody>
- </table></div>
- <p>
- In the above example, the values of the operands are reported for inspection,
- which is more valuable as a copy of the full statement. However, we can observe
- that they are not treated symmetrically:
- </p>
- <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
- <li class="listitem">
- "<code class="computeroutput"><span class="identifier">a</span> <span class="special">-</span>
- <span class="number">1</span> <span class="special"><</span>
- <span class="identifier">b</span></code>" reports <code class="computeroutput"><span class="string">"13 - 1 >= 12"</span> <span class="identifier">failed</span></code>
- </li>
- <li class="listitem">
- "<code class="computeroutput"><span class="identifier">b</span> <span class="special">></span>
- <span class="identifier">a</span> <span class="special">-</span>
- <span class="number">1</span></code>" reports <code class="computeroutput"><span class="string">"12
- <= 12"</span> <span class="identifier">failed</span></code>
- </li>
- </ul></div>
- <p>
- More details on how the <span class="emphasis"><em>Unit Test Framework</em></span> parses the
- statement are given in <a class="link" href="internal_details.html" title="BOOST_TEST: details on expressions">this</a>
- section.
- </p>
- <h4>
- <a name="boost_test.testing_tools.reports.h3"></a>
- <span class="phrase"><a name="boost_test.testing_tools.reports.custom_messages"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.custom_messages">Custom
- messages</a>
- </h4>
- <p>
- While perfectly exact and precise, the file name, test case name, line number
- of a failed statement carry an information that is partial with regards to
- the meaning of the failed statement. Sometimes these information are not
- informative enough. The <code class="computeroutput"><span class="identifier">BOOST_TEST</span></code>
- macro let you override the default message by the use of a second argument,
- as shown on the following example.
- </p>
- <h6>
- <a name="boost_test.testing_tools.reports.h4"></a>
- <span class="phrase"><a name="boost_test.testing_tools.reports.example_descr0"></a></span><a class="link" href="reports.html#boost_test.testing_tools.reports.example_descr0">Example:
- BOOST_TEST optional failure message</a>
- </h6>
- <div class="informaltable"><table class="table">
- <colgroup><col></colgroup>
- <thead><tr><th>
- <p>
- Code
- </p>
- </th></tr></thead>
- <tbody><tr><td>
- <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="preprocessor">#define</span> <span class="identifier">BOOST_TEST_MODULE</span> <span class="identifier">boost_test_message</span>
- <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">included</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
- <span class="identifier">BOOST_AUTO_TEST_CASE</span><span class="special">(</span> <span class="identifier">test_message</span> <span class="special">)</span>
- <span class="special">{</span>
- <span class="keyword">const</span> <span class="keyword">int</span> <span class="identifier">a</span><span class="special">(</span><span class="number">1</span><span class="special">),</span> <span class="identifier">b</span><span class="special">(</span><span class="number">2</span><span class="special">);</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">==</span> <span class="identifier">b</span><span class="special">,</span> <span class="string">"a should be equal to b: "</span> <span class="special"><<</span> <span class="identifier">a</span> <span class="special"><<</span> <span class="string">"!="</span> <span class="special"><<</span> <span class="identifier">b</span><span class="special">);</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span><span class="identifier">a</span> <span class="special">!=</span> <span class="number">10</span><span class="special">,</span> <span class="string">"value of a="</span> <span class="special"><<</span> <span class="identifier">a</span><span class="special">);</span>
- <span class="special">}</span>
- </pre>
- </td></tr></tbody>
- </table></div>
- <div class="informaltable"><table class="table">
- <colgroup><col></colgroup>
- <thead><tr><th>
- <p>
- Output
- </p>
- </th></tr></thead>
- <tbody><tr><td>
- <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="special">></span> <span class="special">./</span><span class="identifier">boost_test_message</span> <span class="special">--</span><span class="identifier">log_level</span><span class="special">=</span><span class="identifier">all</span>
- <span class="identifier">Running</span> <span class="number">1</span> <span class="identifier">test</span> <span class="keyword">case</span><span class="special">...</span>
- <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_message"</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Entering</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_message"</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">15</span><span class="special">):</span> <span class="identifier">error</span><span class="special">:</span> <span class="identifier">in</span> <span class="string">"test_message"</span><span class="special">:</span> <span class="identifier">a</span> <span class="identifier">should</span> <span class="identifier">be</span> <span class="identifier">equal</span> <span class="identifier">to</span> <span class="identifier">b</span><span class="special">:</span> <span class="number">1</span><span class="special">!=</span><span class="number">2</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">16</span><span class="special">):</span> <span class="identifier">info</span><span class="special">:</span> <span class="identifier">check</span> <span class="char">'value of a=1'</span> <span class="identifier">has</span> <span class="identifier">passed</span>
- <span class="identifier">test</span><span class="special">.</span><span class="identifier">cpp</span><span class="special">(</span><span class="number">12</span><span class="special">):</span> <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="keyword">case</span> <span class="string">"test_message"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">219u</span><span class="identifier">s</span>
- <span class="identifier">Leaving</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_message"</span><span class="special">;</span> <span class="identifier">testing</span> <span class="identifier">time</span><span class="special">:</span> <span class="number">318u</span><span class="identifier">s</span>
- <span class="special">***</span> <span class="number">1</span> <span class="identifier">failure</span> <span class="identifier">is</span> <span class="identifier">detected</span> <span class="identifier">in</span> <span class="identifier">the</span> <span class="identifier">test</span> <span class="identifier">module</span> <span class="string">"boost_test_message"</span>
- </pre>
- </td></tr></tbody>
- </table></div>
- </div>
- <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
- <td align="left"></td>
- <td align="right"><div class="copyright-footer">Copyright © 2001-2019 Boost.Test
- contributors<p>
- Distributed under the Boost Software License, Version 1.0. (See accompanying
- file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
- </p>
- </div></td>
- </tr></table>
- <hr>
- <div class="spirit-nav">
- <a accesskey="p" href="boost_test_universal_macro.html"><img src="../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../testing_tools.html"><img src="../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="extended_comparison.html"><img src="../../../../../../doc/src/images/next.png" alt="Next"></a>
- </div>
- </body>
- </html>
|