123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264 |
- [/
- Copyright (c) 2016-2019 Vinnie Falco (vinnie dot falco at gmail dot com)
- Distributed under the Boost Software License, Version 1.0. (See accompanying
- file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
- Official repository: https://github.com/boostorg/beast
- ]
- [section Chunked Encoding]
- For message payloads whose size is not known ahead of time, HTTP
- version 1.1 defines the
- [@https://tools.ietf.org/html/rfc7230#section-4.1 ['chunked]]
- transfer coding. This coding consists of zero or more
- [@https://tools.ietf.org/html/rfc7230#section-4.1 ['chunked bodies]],
- followed by a
- [@https://tools.ietf.org/html/rfc7230#section-4.1 ['last chunk]].
- Each chunked body may contain optional application-defined, connection-specific
- [@https://tools.ietf.org/html/rfc7230#section-4.1.1 ['chunk-extensions]].
- The last chunk may contain additional HTTP field values in a section
- of the last chunk called a
- [@https://tools.ietf.org/html/rfc7230#section-4.1.2 ['chunk-trailer]].
- The field values are "promised" in the header as a comma delimited list
- of field names in the
- [@https://tools.ietf.org/html/rfc7230#section-4.4 [*Trailer]]
- field value. Clients indicate their willingness to accept trailers
- by including the "trailers" token in the
- [@https://tools.ietf.org/html/rfc7230#section-4.3 [*TE]]
- field value.
- [heading Serializing Chunks]
- The __serializer__ automatically applies the chunked transfer encoding
- when a message returns `true` from
- [link beast.ref.boost__beast__http__message.chunked.overload1 `message::chunked`].
- The boundaries between chunks emitted by the serializer are implementation
- defined. Chunk extensions and trailers are omitted. Applications which
- need precise control over the chunk boundaries, extensions, and trailers
- may use a set of helper classes which enable manual emission of message
- payloads using chunk encoding.
- To use these helper classes, first serialize the header portion of the
- message using the standard interface. Then prepare the buffers, chunk
- extensions, and desired trailers, and use them with these helpers:
- [table Chunking Helpers
- [[Name][Description]]
- [
- [[link beast.ref.boost__beast__http__chunk_body `chunk_body`]]
- [
- A buffer sequence representing a complete chunk body.
- ]
- ][
- [[link beast.ref.boost__beast__http__chunk_crlf `chunk_crlf`]]
- [
- A buffer sequence representing the CRLF (`"\r\n"`) delimiter.
- This class is used when the caller desires to emit the
- chunk body in two or more individual stream operations.
- ]
- ][
- [
- [link beast.ref.boost__beast__http__chunk_extensions `chunk_extensions`]
- [link beast.ref.boost__beast__http__basic_chunk_extensions `basic_chunk_extensions`]
- ]
- [
- This is a simple, allocating container which lets callers
- easily build up a set of chunk extensions.
- ]
- ][
- [[link beast.ref.boost__beast__http__chunk_header `chunk_header`]]
- [
- A buffer sequence representing a hex-encoded chunk size,
- followed by an optional set of chunk extensions, including
- the terminating CRLF (`"\r\n"`) delimiter which precedes
- the chunk body. This class is used when the caller desires
- to emit the chunk body in two or more individual stream
- operations.
- ]
- ][
- [[link beast.ref.boost__beast__http__chunk_last `chunk_last`]]
- [
- A buffer sequence representing a last chunk. The last chunk
- indicates the end of the chunked message payload, and may
- contain optional trailer fields.
- ]
- ][
- [
- [link beast.ref.boost__beast__http__make_chunk `make_chunk`]
- [link beast.ref.boost__beast__http__make_chunk_last `make_chunk_last`]
- ]
- [
- These helper functions are used to construct a chunk
- or last chunk directly at call sites.
- ]
- ]]
- We demonstrate the use of these objects first by declaring a function
- which returns the next buffer sequence to use as a chunk body:
- [http_snippet_17]
- This example demonstrates sending a complete chunked message payload
- manually. No chunk extensions or trailers are emitted:
- [http_snippet_18]
- The following code sends additional chunks, and sets chunk extensions
- using the helper container. The container automatically quotes values
- in the serialized output when necessary:
- [http_snippet_19]
- Callers can take over the generation and management of the extensions
- buffer by passing a non-owning string. Note that this requires the
- string contents to adhere to the correct syntax for chunk extensions,
- including the needed double quotes for values which contain spaces:
- [http_snippet_20]
- The next code sample emits a chunked response which promises two
- trailer fields and delivers them in the last chunk. The implementation
- allocates memory using the default or a passed-in allocator to hold
- the state information required to serialize the trailer:
- [http_snippet_21]
- Using a custom allocator to serialize the last chunk:
- [http_snippet_22]
- Alternatively, callers can take over the generation and lifetime
- management of the serialized trailer fields by passing in a non-owning
- string:
- [http_snippet_23]
- For the ultimate level of control, a caller can manually compose the
- chunk itself by first emitting a header with the correct chunk body
- size, and then by emitting the chunk body in multiple calls to the
- stream write function. In this case the caller is responsible for
- also emitting the terminating CRLF (`"\r\n"`):
- [http_snippet_24]
- [heading Parsing Chunks]
- The __parser__ automatically removes the chunked transfer coding when
- it is the last encoding in the list. However, it also discards the
- chunk extensions and does not provide a way to determine the boundaries
- between chunks. Advanced applications which need to access the chunk
- extensions or read complete individual chunks may use a callback
- interface provided by __parser__:
- [table Chunking Parse Callbacks
- [[Name][Description]]
- [
- [[link beast.ref.boost__beast__http__parser.on_chunk_header `on_chunk_header`]]
- [
- Set a callback to be invoked on each chunk header.
- The callback will be invoked once for every chunk in the message
- payload, as well as once for the last chunk. The invocation
- happens after the chunk header is available but before any body
- octets have been parsed.
- The extensions are provided in raw, validated form, use
- [link beast.ref.boost__beast__http__basic_chunk_extensions.parse `chunk_extensions::parse`]
- to parse the extensions into a structured container for easier access.
- The implementation type-erases the callback without requiring
- a dynamic allocation. For this reason, the callback object is
- passed by a non-constant reference.
- The function object will be called with this equivalent signature:
- ```
- void
- callback(
- std::uint64_t size, // Size of the chunk, zero for the last chunk
- string_view extensions, // The chunk-extensions in raw form
- error_code& ec); // May be set by the callback to indicate an error
- ```
- ]
- ][
- [[link beast.ref.boost__beast__http__parser.on_chunk_body `on_chunk_body`]]
- [
- Set a callback to be invoked on chunk body data.
- The callback will be invoked one or more times to provide
- buffers corresponding to the chunk body for the current chunk.
- The callback receives the number of octets remaining in this
- chunk body including the octets in the buffer provided.
- The callback must return the number of octets actually consumed.
- Any octets not consumed will be presented again in a subsequent
- invocation of the callback.
- The implementation type-erases the callback without requiring
- a dynamic allocation. For this reason, the callback object is
- passed by a non-constant reference.
- The function object will be called with this equivalent signature:
- ```
- std::size_t
- callback(
- std::uint64_t remain, // Octets remaining in this chunk, includes `body`
- string_view body, // A buffer holding some or all of the remainder of the chunk body
- error_code& ec); // May be set by the callback to indicate an error
- ```
- ]
- ]]
- This example will read a message header from the stream, and then manually
- read each chunk. It recognizes the chunk boundaries and outputs the contents
- of each chunk as it comes in. Any chunk extensions are printed, each extension
- on its own line. Finally, any trailers promised in the header are printed.
- [example_chunk_parsing]
- Given the HTTP response as input on the left, the output of the function shown
- above is shown on the right:
- [table Chunk Parsing Example Output
- [[Input][Output]]
- [
- [
- ```
- HTTP/1.1 200 OK\r\n
- Server: test\r\n
- Trailer: Expires, Content-MD5\r\n
- Transfer-Encoding: chunked\r\n
- \r\n
- 5\r\n
- First\r\n
- d;quality=1.0\r\n
- Hello, world!\r\n
- e;file=abc.txt;quality=0.7\r\n
- The Next Chunk\r\n
- 8;last\r\n
- Last one\r\n
- 0\r\n
- Expires: never\r\n
- Content-MD5: f4a5c16584f03d90\r\n
- \r\n
- ```
- ]
- [
- ```
- Chunk Body: First
- Extension: quality = 1.0
- Chunk Body: Hello, world!
- Extension: file = abc.txt
- Extension: quality = 0.7
- Chunk Body: The Next Chunk
- Extension: last
- Chunk Body: Last one
- Expires: never
- Content-MD5: f4a5c16584f03d90
- ```
- ]
- ]]
- [endsect]
|