123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226 |
- <html>
- <head>
- <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
- <title>A testing framework, what for?</title>
- <link rel="stylesheet" href="../../../boostbook.css" type="text/css">
- <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
- <link rel="home" href="../../../index.html" title="Boost.Test">
- <link rel="up" href="../tutorials.html" title="Tutorials">
- <link rel="prev" href="bt_and_tdd.html" title="Test driven development with Boost.Test">
- <link rel="next" href="../web_wisdom.html" title="Web Wisdom">
- </head>
- <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
- <table cellpadding="2" width="100%"><tr>
- <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td>
- <td align="center"><a href="../../../../../../../index.html">Home</a></td>
- <td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td>
- <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
- <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
- <td align="center"><a href="../../../../../../../more/index.htm">More</a></td>
- </tr></table>
- <hr>
- <div class="spirit-nav">
- <a accesskey="p" href="bt_and_tdd.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../web_wisdom.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
- </div>
- <div class="section">
- <div class="titlepage"><div><div><h4 class="title">
- <a name="boost_test.practical_usage_recommendations.tutorials.hello"></a><a class="link" href="hello.html" title="A testing framework, what for?">A
- testing framework, what for?</a>
- </h4></div></div></div>
- <p>
- How should a test program report errors? Displaying an error message is
- an obvious possibility:
- </p>
- <pre class="programlisting"><span class="keyword">if</span><span class="special">(</span> <span class="identifier">something_bad_detected</span> <span class="special">)</span>
- <span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span> <span class="special"><<</span> <span class="string">"something bad has been detected"</span> <span class="special"><<</span> <span class="identifier">std</span><span class="special">::</span><span class="identifier">endl</span><span class="special">;</span>
- </pre>
- <p>
- But that requires inspection of the program's output after each run to
- determine if an error occurred. Since test programs are often run as part
- of a regression test suite, human inspection of output to detect error
- messages is time consuming and unreliable. Test frameworks like GNU/expect
- can do the inspections automatically, but are overly complex for simple
- testing.
- </p>
- <p>
- A better simple way to report errors is for the test program to return
- <code class="computeroutput"><span class="identifier">EXIT_SUCCESS</span></code> (normally
- 0) if the test program completes satisfactorily, and <code class="computeroutput"><span class="identifier">EXIT_FAILURE</span></code>
- if an error is detected. This allows a simple regression test script to
- automatically and unambiguously detect success or failure. Further appropriate
- actions such as creating an HTML table or emailing an alert can be taken
- by the script, and can be modified as desired without having to change
- the actual C++ test programs.
- </p>
- <p>
- A testing protocol based on a policy of test programs returning <code class="computeroutput"><span class="identifier">EXIT_SUCCESS</span></code> or <code class="computeroutput"><span class="identifier">EXIT_FAILURE</span></code>
- does not require any supporting tools; the C++ language and standard library
- are sufficient. The programmer must remember, however, to catch all exceptions
- and convert them to program exits with non-zero return codes. The programmer
- must also remember to not use the standard library <code class="computeroutput"><span class="identifier">assert</span><span class="special">()</span></code> macro for test code, because on some
- systems it results in undesirable side effects like a message requiring
- manual intervention.
- </p>
- <p>
- The Boost Test Library's Unit Test Framework is designed to automate those
- tasks. The library supplied <code class="computeroutput"><span class="identifier">main</span><span class="special">()</span></code> relieves users from messy error detection
- and reporting duties. Users could use supplied testing tools to perform
- complex validation tasks. Let's take a look on the following simple test
- program:
- </p>
- <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">my_class</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
- <span class="keyword">int</span> <span class="identifier">main</span><span class="special">(</span> <span class="keyword">int</span><span class="special">,</span> <span class="keyword">char</span><span class="special">*</span> <span class="special">[]</span> <span class="special">)</span>
- <span class="special">{</span>
- <span class="identifier">my_class</span> <span class="identifier">test_object</span><span class="special">(</span> <span class="string">"qwerty"</span> <span class="special">);</span>
- <span class="keyword">return</span> <span class="identifier">test_object</span><span class="special">.</span><span class="identifier">is_valid</span><span class="special">()</span> <span class="special">?</span> <span class="identifier">EXIT_SUCCESS</span> <span class="special">:</span> <span class="identifier">EXIT_FAILURE</span><span class="special">;</span>
- <span class="special">}</span>
- </pre>
- <p>
- There are several issues with above test.
- </p>
- <div class="orderedlist"><ol class="orderedlist" type="1">
- <li class="listitem">
- You need to convert <code class="computeroutput"><span class="identifier">is_valid</span></code>
- result in proper result code.
- </li>
- <li class="listitem">
- Would exception happen in test_object construction of method <code class="computeroutput"><span class="identifier">is_valid</span></code> invocation, the program
- will crash.
- </li>
- <li class="listitem">
- You won't see any output, would you run this test manually.
- </li>
- </ol></div>
- <p>
- The <span class="emphasis"><em>Unit Test Framework</em></span> solves all these issues. To
- integrate with it above program needs to be changed to:
- </p>
- <pre class="programlisting"><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">my_class</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
- <span class="preprocessor">#define</span> <a class="link" href="../../utf_reference/link_references/link_boost_test_module_macro.html" title="BOOST_TEST_MODULE"><code class="computeroutput"><span class="identifier">BOOST_TEST_MODULE</span></code></a> <span class="identifier">MyTest</span>
- <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
- <a class="link" href="../../utf_reference/test_org_reference/test_org_boost_auto_test_case.html" title="BOOST_AUTO_TEST_CASE"><code class="computeroutput"><span class="identifier">BOOST_AUTO_TEST_CASE</span></code></a><span class="special">(</span> <span class="identifier">my_test</span> <span class="special">)</span>
- <span class="special">{</span>
- <span class="identifier">my_class</span> <span class="identifier">test_object</span><span class="special">(</span> <span class="string">"qwerty"</span> <span class="special">);</span>
- <span class="identifier">BOOST_TEST</span><span class="special">(</span> <span class="identifier">test_object</span><span class="special">.</span><span class="identifier">is_valid</span><span class="special">()</span> <span class="special">);</span>
- <span class="special">}</span>
- </pre>
- <p>
- Now, you not only receive uniform result code, even in case of exception,
- but also nicely formatted output from <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> tool, would you choose
- to see it. Is there any other ways to perform checks? The following example
- test program shows several different ways to detect and report an error
- in the <code class="computeroutput"><span class="identifier">add</span><span class="special">()</span></code>
- function.
- </p>
- <pre class="programlisting"><span class="preprocessor">#define</span> <a class="link" href="../../utf_reference/link_references/link_boost_test_module_macro.html" title="BOOST_TEST_MODULE"><code class="computeroutput"><span class="identifier">BOOST_TEST_MODULE</span></code></a> <span class="identifier">MyTest</span>
- <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">test</span><span class="special">/</span><span class="identifier">unit_test</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span>
- <span class="keyword">int</span> <span class="identifier">add</span><span class="special">(</span> <span class="keyword">int</span> <span class="identifier">i</span><span class="special">,</span> <span class="keyword">int</span> <span class="identifier">j</span> <span class="special">)</span> <span class="special">{</span> <span class="keyword">return</span> <span class="identifier">i</span> <span class="special">+</span> <span class="identifier">j</span><span class="special">;</span> <span class="special">}</span>
- <a class="link" href="../../utf_reference/test_org_reference/test_org_boost_auto_test_case.html" title="BOOST_AUTO_TEST_CASE"><code class="computeroutput"><span class="identifier">BOOST_AUTO_TEST_CASE</span></code></a><span class="special">(</span><span class="identifier">my_test</span><span class="special">)</span>
- <span class="special">{</span>
- <span class="comment">// six ways to detect and report the same error:</span>
- <span class="comment">// continues on error</span>
- <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a><span class="special">(</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">==</span> <span class="number">4</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c0" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c1"><img src="../../../../../../../doc/src/images/callouts/1.png" alt="1" border="0"></a>
- <span class="comment">// throws on error</span>
- <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST_REQUIRE</span></code></a><span class="special">(</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">==</span> <span class="number">4</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c2" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c3"><img src="../../../../../../../doc/src/images/callouts/2.png" alt="2" border="0"></a>
- <span class="comment">//continues on error</span>
- <span class="keyword">if</span> <span class="special">(</span><span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">!=</span> <span class="number">4</span><span class="special">)</span>
- <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_error.html" title="BOOST_ERROR"><code class="computeroutput"><span class="identifier">BOOST_ERROR</span></code></a><span class="special">(</span> <span class="string">"Ouch..."</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c4" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c5"><img src="../../../../../../../doc/src/images/callouts/3.png" alt="3" border="0"></a>
- <span class="comment">// throws on error</span>
- <span class="keyword">if</span> <span class="special">(</span><span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">!=</span> <span class="number">4</span><span class="special">)</span>
- <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_fail.html" title="BOOST_FAIL"><code class="computeroutput"><span class="identifier">BOOST_FAIL</span></code></a><span class="special">(</span> <span class="string">"Ouch..."</span> <span class="special">);</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c6" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c7"><img src="../../../../../../../doc/src/images/callouts/4.png" alt="4" border="0"></a>
- <span class="comment">// throws on error</span>
- <span class="keyword">if</span> <span class="special">(</span><span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">!=</span> <span class="number">4</span><span class="special">)</span>
- <span class="keyword">throw</span> <span class="string">"Ouch..."</span><span class="special">;</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c8" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c9"><img src="../../../../../../../doc/src/images/callouts/5.png" alt="5" border="0"></a>
- <span class="comment">// continues on error</span>
- <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a><span class="special">(</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">)</span> <span class="special">==</span> <span class="number">4</span><span class="special">,</span> <a class="co" name="boost_test.practical_usage_recommendations.tutorials.hello.c10" href="hello.html#boost_test.practical_usage_recommendations.tutorials.hello.c11"><img src="../../../../../../../doc/src/images/callouts/6.png" alt="6" border="0"></a>
- <span class="string">"2 plus 2 is not 4 but "</span> <span class="special"><<</span> <span class="identifier">add</span><span class="special">(</span><span class="number">2</span><span class="special">,</span> <span class="number">2</span><span class="special">));</span>
- <span class="special">}</span>
- </pre>
- <div class="calloutlist"><table border="0" summary="Callout list">
- <tr>
- <td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c1"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c0"><img src="../../../../../../../doc/src/images/callouts/1.png" alt="1" border="0"></a> </p></td>
- <td valign="top" align="left"><p>
- This approach uses tool <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a>, which displays
- an error message (by default on <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">cout</span></code>)
- that includes the expression that failed, as well as the values on
- the two side of the equation, the source file name, and the source
- file line number. It also increments the error count. At program termination,
- the error count will be displayed automatically by the <span class="emphasis"><em>Unit
- Test Framework</em></span>.
- </p></td>
- </tr>
- <tr>
- <td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c3"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c2"><img src="../../../../../../../doc/src/images/callouts/2.png" alt="2" border="0"></a> </p></td>
- <td valign="top" align="left"><p>
- This approach uses tool <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST_REQUIRE</span></code></a>, is similar
- to approach #1, except that after displaying the error, an exception
- is thrown, to be caught by the <span class="emphasis"><em>Unit Test Framework</em></span>.
- This approach is suitable when writing an explicit test program, and
- the error would be so severe as to make further testing impractical.
- </p></td>
- </tr>
- <tr>
- <td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c5"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c4"><img src="../../../../../../../doc/src/images/callouts/3.png" alt="3" border="0"></a> </p></td>
- <td valign="top" align="left"><p>
- This approach is similar to approach #1, except that the error detection
- and error reporting are coded separately. This is most useful when
- the specific condition being tested requires several independent statements
- and/or is not indicative of the reason for failure.
- </p></td>
- </tr>
- <tr>
- <td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c7"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c6"><img src="../../../../../../../doc/src/images/callouts/4.png" alt="4" border="0"></a> </p></td>
- <td valign="top" align="left"><p>
- This approach is similar to approach #2, except that the error detection
- and error reporting are coded separately. This is most useful when
- the specific condition being tested requires several independent statements
- and/or is not indicative of the reason for failure.
- </p></td>
- </tr>
- <tr>
- <td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c9"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c8"><img src="../../../../../../../doc/src/images/callouts/5.png" alt="5" border="0"></a> </p></td>
- <td valign="top" align="left"><p>
- This approach throws an exception, which will be caught and reported
- by the <span class="emphasis"><em>Unit Test Framework</em></span>. The error message
- displayed when the exception is caught will be most meaningful if the
- exception is derived from <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">exception</span></code>,
- or is a <code class="computeroutput"><span class="keyword">char</span><span class="special">*</span></code>
- or <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">string</span></code>.
- </p></td>
- </tr>
- <tr>
- <td width="5%" valign="top" align="left"><p><a name="boost_test.practical_usage_recommendations.tutorials.hello.c11"></a><a href="#boost_test.practical_usage_recommendations.tutorials.hello.c10"><img src="../../../../../../../doc/src/images/callouts/6.png" alt="6" border="0"></a> </p></td>
- <td valign="top" align="left"><p>
- This approach uses tool <a class="link" href="../../utf_reference/testing_tool_ref/assertion_boost_test_universal_macro.html" title="BOOST_TEST"><code class="computeroutput"><span class="identifier">BOOST_TEST</span></code></a> with additional
- message argument, is similar to approach #1, except that similar to
- the approach #3 displays an alternative error message specified as
- a second argument.
- </p></td>
- </tr>
- </table></div>
- </div>
- <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
- <td align="left"></td>
- <td align="right"><div class="copyright-footer">Copyright © 2001-2019 Boost.Test
- contributors<p>
- Distributed under the Boost Software License, Version 1.0. (See accompanying
- file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
- </p>
- </div></td>
- </tr></table>
- <hr>
- <div class="spirit-nav">
- <a accesskey="p" href="bt_and_tdd.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../web_wisdom.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
- </div>
- </body>
- </html>
|