tests_organization.html 11 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182
  1. <html>
  2. <head>
  3. <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
  4. <title>Declaring and organizing tests</title>
  5. <link rel="stylesheet" href="../boostbook.css" type="text/css">
  6. <meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
  7. <link rel="home" href="../index.html" title="Boost.Test">
  8. <link rel="up" href="../index.html" title="Boost.Test">
  9. <link rel="prev" href="usage_variants.html" title="Usage variants">
  10. <link rel="next" href="tests_organization/test_cases.html" title="Test cases">
  11. </head>
  12. <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
  13. <table cellpadding="2" width="100%"><tr>
  14. <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../boost.png"></td>
  15. <td align="center"><a href="../../../../../index.html">Home</a></td>
  16. <td align="center"><a href="../../../../../libs/libraries.htm">Libraries</a></td>
  17. <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
  18. <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
  19. <td align="center"><a href="../../../../../more/index.htm">More</a></td>
  20. </tr></table>
  21. <hr>
  22. <div class="spirit-nav">
  23. <a accesskey="p" href="usage_variants.html"><img src="../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../index.html"><img src="../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tests_organization/test_cases.html"><img src="../../../../../doc/src/images/next.png" alt="Next"></a>
  24. </div>
  25. <div class="section">
  26. <div class="titlepage"><div><div><h2 class="title" style="clear: both">
  27. <a name="boost_test.tests_organization"></a><a class="link" href="tests_organization.html" title="Declaring and organizing tests">Declaring and organizing
  28. tests</a>
  29. </h2></div></div></div>
  30. <div class="toc"><dl class="toc">
  31. <dt><span class="section"><a href="tests_organization/test_cases.html">Test cases</a></span></dt>
  32. <dd><dl>
  33. <dt><span class="section"><a href="tests_organization/test_cases/test_organization_nullary.html">Test
  34. cases without parameters</a></span></dt>
  35. <dt><span class="section"><a href="tests_organization/test_cases/test_case_generation.html">Data-driven
  36. test cases</a></span></dt>
  37. <dd><dl>
  38. <dt><span class="section"><a href="tests_organization/test_cases/test_case_generation/datasets.html">Datasets</a></span></dt>
  39. <dt><span class="section"><a href="tests_organization/test_cases/test_case_generation/datasets_auto_registration.html">Declaring
  40. and registering test cases with datasets</a></span></dt>
  41. <dt><span class="section"><a href="tests_organization/test_cases/test_case_generation/operations.html">Operations
  42. on dataset</a></span></dt>
  43. <dt><span class="section"><a href="tests_organization/test_cases/test_case_generation/generators.html">Datasets
  44. generators</a></span></dt>
  45. </dl></dd>
  46. <dt><span class="section"><a href="tests_organization/test_cases/test_organization_templates.html">Template
  47. test cases</a></span></dt>
  48. <dt><span class="section"><a href="tests_organization/test_cases/param_test.html">Parametrized
  49. test cases</a></span></dt>
  50. </dl></dd>
  51. <dt><span class="section"><a href="tests_organization/test_tree.html">Test tree</a></span></dt>
  52. <dd><dl>
  53. <dt><span class="section"><a href="tests_organization/test_tree/test_suite.html">Test
  54. suite</a></span></dt>
  55. <dt><span class="section"><a href="tests_organization/test_tree/master_test_suite.html">Master
  56. test suite</a></span></dt>
  57. <dt><span class="section"><a href="tests_organization/test_tree/test_naming.html">Tests
  58. naming</a></span></dt>
  59. <dt><span class="section"><a href="tests_organization/test_tree/test_tree_content.html">Test
  60. tree content</a></span></dt>
  61. </dl></dd>
  62. <dt><span class="section"><a href="tests_organization/decorators.html">Decorators</a></span></dt>
  63. <dd><dl>
  64. <dt><span class="section"><a href="tests_organization/decorators/suite_level_decorators.html">Suite-level
  65. decorators</a></span></dt>
  66. <dt><span class="section"><a href="tests_organization/decorators/explicit_decorator_declaration.html">Explicit
  67. decorator declaration</a></span></dt>
  68. </dl></dd>
  69. <dt><span class="section"><a href="tests_organization/fixtures.html">Fixtures</a></span></dt>
  70. <dd><dl>
  71. <dt><span class="section"><a href="tests_organization/fixtures/models.html">Fixture
  72. models</a></span></dt>
  73. <dt><span class="section"><a href="tests_organization/fixtures/case.html">Test case
  74. fixture</a></span></dt>
  75. <dt><span class="section"><a href="tests_organization/fixtures/per_test_suite_fixture.html">Test
  76. suite entry/exit fixture</a></span></dt>
  77. <dt><span class="section"><a href="tests_organization/fixtures/global.html">Global
  78. fixture</a></span></dt>
  79. </dl></dd>
  80. <dt><span class="section"><a href="tests_organization/tests_dependencies.html">Managing
  81. test dependencies</a></span></dt>
  82. <dt><span class="section"><a href="tests_organization/tests_grouping.html">Grouping
  83. tests into logical units by labels</a></span></dt>
  84. <dt><span class="section"><a href="tests_organization/enabling.html">Enabling or disabling
  85. test unit execution</a></span></dt>
  86. <dt><span class="section"><a href="tests_organization/semantic.html">Adding semantic
  87. to a test</a></span></dt>
  88. <dt><span class="section"><a href="tests_organization/summary_of_the_api_for_declaring.html">Summary
  89. of the API for declaring and organizing tests</a></span></dt>
  90. </dl></div>
  91. <p>
  92. If you look at many legacy test modules, big chance is that it's implemented
  93. as one big test function that consists of a mixture of check and output statements.
  94. Is there anything wrong with it? Yes. There are various disadvantages in single
  95. test function approach:
  96. </p>
  97. <div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
  98. <li class="listitem">
  99. One big function tends to become really difficult to manage if the number
  100. of checks exceeds a reasonable limit (true for any large function). What
  101. is tested and where - who knows?
  102. </li>
  103. <li class="listitem">
  104. Many checks require similar preparations. This results in code repetitions
  105. within the test function.
  106. </li>
  107. <li class="listitem">
  108. If a fatal error or an exception is caused by any checks within the test
  109. function the rest of tests are skipped and there is no way to prevent this.
  110. </li>
  111. <li class="listitem">
  112. No way to perform only checks for a particular subsystem of the tested
  113. unit.
  114. </li>
  115. <li class="listitem">
  116. No summary of how different subsystems of the tested unit performed under
  117. in the test.
  118. </li>
  119. </ul></div>
  120. <p>
  121. The above points should make it clear that it's preferable to split a <a class="link" href="section_glossary.html#ref_test_module">test module</a> into smaller <span class="bold"><strong>units</strong></span>.
  122. These units are the <span class="bold"><strong>test cases</strong></span>, the <span class="bold"><strong>test suites</strong></span> and the <span class="bold"><strong>fixtures</strong></span>.
  123. </p>
  124. <div class="variablelist">
  125. <p class="title"><b>Subjects covered by this section</b></p>
  126. <dl class="variablelist">
  127. <dt><span class="term">Declaration</span></dt>
  128. <dd><p>
  129. The <span class="emphasis"><em>Unit Test Framework</em></span> supports several methods
  130. for declaring a <a class="link" href="tests_organization/test_cases.html" title="Test cases">test
  131. case</a>. Test cases can be implemented using free function like syntax
  132. or based on actual free function, function object, that can be defined
  133. with or without parameters/data, or as template functions to be run against
  134. various types.
  135. </p></dd>
  136. <dt><span class="term">Organization</span></dt>
  137. <dd><p>
  138. The <span class="emphasis"><em>Unit Test Framework</em></span> provides facilities to group
  139. several test cases into <a class="link" href="tests_organization/test_tree/test_suite.html" title="Test suite">test
  140. suites</a>. The test suites can be nested, and the set of test suites
  141. and test cases defines the <a class="link" href="tests_organization/test_tree.html" title="Test tree">test
  142. tree</a>, where the leaves are the test cases. Besides hierarchical
  143. structure the <span class="emphasis"><em>Unit Test Framework</em></span> allows you to
  144. organize the test tree using <a class="link" href="tests_organization/tests_grouping.html" title="Grouping tests into logical units by labels">logical
  145. grouping</a> and <a class="link" href="tests_organization/tests_dependencies.html" title="Managing test dependencies">dependencies</a>
  146. and provides you with controls to utilize the defined test tree organization
  147. the way you want (eg. from command line).
  148. </p></dd>
  149. <dt><span class="term">Attributes</span></dt>
  150. <dd><p>
  151. It is possible to specify test unit <span class="emphasis"><em>attributes</em></span> by
  152. using <a class="link" href="tests_organization/decorators.html" title="Decorators">decorators</a>.
  153. Attributes are used for a fine grained control over various aspects of
  154. test module execution, such as <span class="emphasis"><em>logical grouping</em></span>,
  155. <span class="emphasis"><em>dependencies</em></span>, <span class="emphasis"><em>expected failures</em></span>,
  156. etc.
  157. </p></dd>
  158. <dt><span class="term">Setup/teardown test unit actions</span></dt>
  159. <dd><p>
  160. When several tests shares the same set-up (environment, test data preparation,
  161. etc.), the preparation and cleanup code may be factorized in <a class="link" href="tests_organization/fixtures.html" title="Fixtures">fixtures</a>.
  162. In the <span class="emphasis"><em>Unit Test Framework</em></span>, fixtures can be associated
  163. to test cases, test suites or globally to the test module.
  164. </p></dd>
  165. </dl>
  166. </div>
  167. </div>
  168. <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
  169. <td align="left"></td>
  170. <td align="right"><div class="copyright-footer">Copyright &#169; 2001-2019 Boost.Test
  171. contributors<p>
  172. Distributed under the Boost Software License, Version 1.0. (See accompanying
  173. file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
  174. </p>
  175. </div></td>
  176. </tr></table>
  177. <hr>
  178. <div class="spirit-nav">
  179. <a accesskey="p" href="usage_variants.html"><img src="../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../index.html"><img src="../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tests_organization/test_cases.html"><img src="../../../../../doc/src/images/next.png" alt="Next"></a>
  180. </div>
  181. </body>
  182. </html>