1_http_message.qbk 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361
  1. [/
  2. Copyright (c) 2016-2019 Vinnie Falco (vinnie dot falco at gmail dot com)
  3. Distributed under the Boost Software License, Version 1.0. (See accompanying
  4. file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
  5. Official repository: https://github.com/boostorg/beast
  6. <iframe width="560" height="315" src="https://www.youtube.com/embed/WsUnnYEKPnI?rel=0" frameborder="0" allowfullscreen></iframe>
  7. ]
  8. [section:http_message_container HTTP Message Container __video__]
  9. The following video presentation was delivered at
  10. [@https://cppcon.org/ CppCon]
  11. in 2017. The presentation provides a simplified explanation of the design
  12. process for the HTTP message container used in Beast. The slides and code
  13. used are available in the
  14. [@https://github.com/vinniefalco/CppCon2017 GitHub repository].
  15. [/ "Make Classes Great Again! (Using Concepts for Customization Points)"]
  16. [block'''
  17. <mediaobject>
  18. <videoobject>
  19. <videodata fileref="https://www.youtube.com/embed/WsUnnYEKPnI?rel=0"
  20. align="center" contentwidth="560" contentdepth="315"/>
  21. </videoobject>
  22. </mediaobject>
  23. ''']
  24. In this section we describe the problem of modeling HTTP messages and explain
  25. how the library arrived at its solution, with a discussion of the benefits
  26. and drawbacks of the design choices. The goal for creating a message model
  27. is to create a container with value semantics, possibly movable and/or
  28. copyable, that contains all the information needed to serialize, or all
  29. of the information captured during parsing. More formally, given:
  30. * `m` is an instance of an HTTP message container
  31. * `x` is a series of octets describing a valid HTTP message in
  32. the serialized format described in __rfc7230__.
  33. * `S(m)` is a serialization function which produces a series of octets
  34. from a message container.
  35. * `P(x)` is a parsing function which produces a message container from
  36. a series of octets.
  37. These relations are true:
  38. * `S(m) == x`
  39. * `P(S(m)) == m`
  40. We would also like our message container to have customization points
  41. permitting the following: allocator awareness, user-defined containers
  42. to represent header fields, and user-defined types and algorithms to
  43. represent the body. And finally, because requests and responses have
  44. different fields in the ['start-line], we would like the containers for
  45. requests and responses to be represented by different types for function
  46. overloading.
  47. Here is our first attempt at declaring some message containers:
  48. [table
  49. [[
  50. ```
  51. /// An HTTP request
  52. template<class Fields, class Body>
  53. struct request
  54. {
  55. int version;
  56. std::string method;
  57. std::string target;
  58. Fields fields;
  59. typename Body::value_type body;
  60. };
  61. ```
  62. ][
  63. ```
  64. /// An HTTP response
  65. template<class Fields, class Body>
  66. struct response
  67. {
  68. int version;
  69. int status;
  70. std::string reason;
  71. Fields fields;
  72. typename Body::value_type body;
  73. };
  74. ```
  75. ]]
  76. ]
  77. These containers are capable of representing everything in the model
  78. of HTTP requests and responses described in __rfc7230__. Request and
  79. response objects are different types. The user can choose the container
  80. used to represent the fields. And the user can choose the [*Body] type,
  81. which is a concept defining not only the type of `body` member but also
  82. the algorithms used to transfer information in and out of that member
  83. when performing serialization and parsing.
  84. However, a problem arises. How do we write a function which can accept
  85. an object that is either a request or a response? As written, the only
  86. obvious solution is to make the message a template type. Additional traits
  87. classes would then be needed to make sure that the passed object has a
  88. valid type which meets the requirements. These unnecessary complexities
  89. are bypassed by making each container a partial specialization:
  90. ```
  91. /// An HTTP message
  92. template<bool isRequest, class Fields, class Body>
  93. struct message;
  94. /// An HTTP request
  95. template<class Fields, class Body>
  96. struct message<true, Fields, Body>
  97. {
  98. int version;
  99. std::string method;
  100. std::string target;
  101. Fields fields;
  102. typename Body::value_type body;
  103. };
  104. /// An HTTP response
  105. template<bool isRequest, class Fields, class Body>
  106. struct message<false, Fields, Body>
  107. {
  108. int version;
  109. int status;
  110. std::string reason;
  111. Fields fields;
  112. typename Body::value_type body;
  113. };
  114. ```
  115. Now we can declare a function which takes any message as a parameter:
  116. ```
  117. template<bool isRequest, class Fields, class Body>
  118. void f(message<isRequest, Fields, Body>& msg);
  119. ```
  120. This function can manipulate the fields common to requests and responses.
  121. If it needs to access the other fields, it can use overloads with
  122. partial specialization, or in C++17 a `constexpr` expression:
  123. ```
  124. template<bool isRequest, class Fields, class Body>
  125. void f(message<isRequest, Fields, Body>& msg)
  126. {
  127. if constexpr(isRequest)
  128. {
  129. // call msg.method(), msg.target()
  130. }
  131. else
  132. {
  133. // call msg.result(), msg.reason()
  134. }
  135. }
  136. ```
  137. Often, in non-trivial HTTP applications, we want to read the HTTP header
  138. and examine its contents before choosing a type for [*Body]. To accomplish
  139. this, there needs to be a way to model the header portion of a message.
  140. And we'd like to do this in a way that allows functions which take the
  141. header as a parameter, to also accept a type representing the whole
  142. message (the function will see just the header part). This suggests
  143. inheritance, by splitting a new base class off of the message:
  144. ```
  145. /// An HTTP message header
  146. template<bool isRequest, class Fields>
  147. struct header;
  148. ```
  149. Code which accesses the fields has to laboriously mention the `fields`
  150. member, so we'll not only make `header` a base class but we'll make
  151. a quality of life improvement and derive the header from the fields
  152. for notational convenience. In order to properly support all forms
  153. of construction of [*Fields] there will need to be a set of suitable
  154. constructor overloads (not shown):
  155. ```
  156. /// An HTTP request header
  157. template<class Fields>
  158. struct header<true, Fields> : Fields
  159. {
  160. int version;
  161. std::string method;
  162. std::string target;
  163. };
  164. /// An HTTP response header
  165. template<class Fields>
  166. struct header<false, Fields> : Fields
  167. {
  168. int version;
  169. int status;
  170. std::string reason;
  171. };
  172. /// An HTTP message
  173. template<bool isRequest, class Fields, class Body>
  174. struct message : header<isRequest, Fields>
  175. {
  176. typename Body::value_type body;
  177. /// Construct from a `header`
  178. message(header<isRequest, Fields>&& h);
  179. };
  180. ```
  181. Note that the `message` class now has a constructor allowing messages
  182. to be constructed from a similarly typed `header`. This handles the case
  183. where the user already has the header and wants to make a commitment to the
  184. type for [*Body]. A function can be declared which accepts any header:
  185. ```
  186. template<bool isRequest, class Fields>
  187. void f(header<isRequest, Fields>& msg);
  188. ```
  189. Until now we have not given significant consideration to the constructors
  190. of the `message` class. But to achieve all our goals we will need to make
  191. sure that there are enough constructor overloads to not only provide for
  192. the special copy and move members if the instantiated types support it,
  193. but also allow the fields container and body container to be constructed
  194. with arbitrary variadic lists of parameters. This allows the container
  195. to fully support allocators.
  196. The solution used in the library is to treat the message like a `std::pair`
  197. for the purposes of construction, except that instead of `first` and `second`
  198. we have the `Fields` base class and `message::body` member. This means that
  199. single-argument constructors for those fields should be accessible as they
  200. are with `std::pair`, and that a mechanism identical to the pair's use of
  201. `std::piecewise_construct` should be provided. Those constructors are too
  202. complex to repeat here, but interested readers can view the declarations
  203. in the corresponding header file.
  204. There is now significant progress with our message container but a stumbling
  205. block remains. There is no way to control the allocator for the `std::string`
  206. members. We could add an allocator to the template parameter list of the
  207. header and message classes, use it for those strings. This is unsatisfying
  208. because of the combinatorial explosion of constructor variations needed to
  209. support the scheme. It also means that request messages could have [*four]
  210. different allocators: two for the fields and body, and two for the method
  211. and target strings. A better solution is needed.
  212. To get around this we make an interface modification and then add
  213. a requirement to the [*Fields] type. First, the interface change:
  214. ```
  215. /// An HTTP request header
  216. template<class Fields>
  217. struct header<true, Fields> : Fields
  218. {
  219. int version;
  220. verb method() const;
  221. string_view method_string() const;
  222. void method(verb);
  223. void method(string_view);
  224. string_view target(); const;
  225. void target(string_view);
  226. private:
  227. verb method_;
  228. };
  229. /// An HTTP response header
  230. template<class Fields>
  231. struct header<false, Fields> : Fields
  232. {
  233. int version;
  234. int result;
  235. string_view reason() const;
  236. void reason(string_view);
  237. };
  238. ```
  239. The start-line data members are replaced by traditional accessors
  240. using non-owning references to string buffers. The method is stored
  241. using a simple integer instead of the entire string, for the case
  242. where the method is recognized from the set of known verb strings.
  243. Now we add a requirement to the fields type: management of the
  244. corresponding string is delegated to the [*Fields] container, which can
  245. already be allocator aware and constructed with the necessary allocator
  246. parameter via the provided constructor overloads for `message`. The
  247. delegation implementation looks like this (only the response header
  248. specialization is shown):
  249. ```
  250. /// An HTTP response header
  251. template<class Fields>
  252. struct header<false, Fields> : Fields
  253. {
  254. int version;
  255. int status;
  256. string_view
  257. reason() const
  258. {
  259. return this->reason_impl(); // protected member of Fields
  260. }
  261. void
  262. reason(string_view s)
  263. {
  264. this->reason_impl(s); // protected member of Fields
  265. }
  266. };
  267. ```
  268. Now that we've accomplished our initial goals and more, there are a few
  269. more quality of life improvements to make. Users will choose different
  270. types for `Body` far more often than they will for `Fields`. Thus, we
  271. swap the order of these types and provide a default. Then, we provide
  272. type aliases for requests and responses to soften the impact of using
  273. `bool` to choose the specialization:
  274. ```
  275. /// An HTTP header
  276. template<bool isRequest, class Body, class Fields = fields>
  277. struct header;
  278. /// An HTTP message
  279. template<bool isRequest, class Body, class Fields = fields>
  280. struct message;
  281. /// An HTTP request
  282. template<class Body, class Fields = fields>
  283. using request = message<true, Body, Fields>;
  284. /// An HTTP response
  285. template<class Body, class Fields = fields>
  286. using response = message<false, Body, Fields>;
  287. ```
  288. This allows concise specification for the common cases, while
  289. allowing for maximum customization for edge cases:
  290. ```
  291. request<string_body> req;
  292. response<file_body> res;
  293. ```
  294. This container is also capable of representing complete HTTP/2 messages.
  295. Not because it was explicitly designed for, but because the IETF wanted to
  296. preserve message compatibility with HTTP/1. Aside from version specific
  297. fields such as Connection, the contents of HTTP/1 and HTTP/2 messages are
  298. identical even though their serialized representation is considerably
  299. different. The message model presented in this library is ready for HTTP/2.
  300. In conclusion, this representation for the message container is well thought
  301. out, provides comprehensive flexibility, and avoids the necessity of defining
  302. additional traits classes. User declarations of functions that accept headers
  303. or messages as parameters are easy to write in a variety of ways to accomplish
  304. different results, without forcing cumbersome SFINAE declarations everywhere.
  305. [endsect]