123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231 |
- [/==============================================================================
- Copyright (C) 2001-2011 Hartmut Kaiser
- Copyright (C) 2001-2011 Joel de Guzman
- Distributed under the Boost Software License, Version 1.0. (See accompanying
- file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
- ===============================================================================/]
- [section:lexer_concepts Lexer Concepts]
- __lex__ components fall into a couple of generalized __concepts__. The
- /Lexer/ is the most fundamental concept. All __lex__ components are
- models of the /Lexer/ concept. /PrimitiveLexer/, /UnaryLexer/,
- and /NaryLexer/ are all refinements of the /Lexer/ concept.
- The following sections provide details on these concepts.
- [/////////////////////////////////////////////////////////////////////////////]
- [section Lexer]
- [heading Description]
- The /Lexer/ is the most fundamental concept. A Lexer has a member
- function, `collect`, that accepts a token definition container `Def`, and a
- the name of the lexer state the token definitions of the lexer component need
- to be added to (a string). It doesn't return anything (return type is `void`).
- Each Lexer can represent a specific pattern or algorithm, or it
- can be a more complex lexer component formed as a composition of other Lexer's.
- Additionally, a Lexer exposes a member `add_actions`, that accepts the token
- definition container `Def`, while returning nothing (again, the returned type
- is `void`).
- [variablelist Notation
- [[`l`] [A `Lexer`.]]
- [[`L`] [A `Lexer` type.]]
- [[`Def`] [A token definition container type.]]
- [[`State`] [A type used to represent lexer state names.]]
- ]
- [heading Valid Expressions]
- In the expressions below, the behavior of the lexer component, `l`, is left
- unspecified in the base `Lexer` concept. These are specified in subsequent,
- more refined concepts and by the actual models thereof.
- For any Lexer the following expressions must be valid:
- [table
- [[Expression] [Semantics] [Return type]]
- [[`l.collect(def, state, targetstate)`]
- [Add all token definitions provided
- by this Lexer instance to the lexer
- state `state` of the token definition
- container `def`. After matching this token, the
- lexer should be switched into the state
- `targetstate` (optional)] [`void`]]
- [[`l.add_actions(def)`] [Add all semantic actions provided
- by this Lexer instance to the token
- definition container `def`.] [`void`]]
- ]
- [heading Type Expressions]
- [table
- [[Expression] [Description]]
- [[`traits::is_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if
- a certain type, `L` is a Lexer, `mpl::false_`
- otherwise (See __mpl_boolean_constant__).]]
- ]
- [heading Postcondition]
- Upon return from `l.collect` the following post conditions should hold:
- * On return, `def` holds all token definitions defined in the Lexer, `l`. This
- includes all Lexer's contained inside `l`.
- Upon return from `l.add_actions` the following post conditions should hold:
- * On return, `def` holds all semantic actions correctly associated with the
- corresponding token definitions as defined in the Lexer, `l`. This
- includes all semantic actions defined by the Lexer's contained inside `l`.
- [heading Models]
- All lexer components in __lex__ are models of the /Lexer/ concept.
- [endsect] [/ Lexer Concept]
- [/////////////////////////////////////////////////////////////////////////////]
- [section PrimitiveLexer]
- [heading Description]
- /PrimitiveLexer/ is the most basic building block that the client uses
- to build more complex lexer components.
- [heading Refinement of]
- [:__lexer_concept__]
- [heading Type Expressions]
- [table
- [[Expression] [Description]]
- [[`traits::is_primitive_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if
- a certain type, `L`, is a PrimitiveLexer, `mpl::false_`
- otherwise (See __mpl_boolean_constant__).]]
- ]
- [heading Models]
- The following lexer components conform to this model:
- * character literals (i.e. `'x'`), `char_`,
- * string literals (`"abc"`), `std::basic_string<>`, `string`
- __fixme__ Add more links to /PrimitiveLexer/ models here.
- [endsect] [/ PrimitiveLexer Concept]
- [/////////////////////////////////////////////////////////////////////////////]
- [section UnaryLexer]
- [heading Description]
- /UnaryLexer/ is a composite lexer component that has a single subject. The
- UnaryLexer may change the behavior of its subject following the
- __delegate_pattern__.
- [heading Refinement of]
- [:__lexer_concept__]
- [variablelist Notation
- [[`l`] [A UnaryLexer.]]
- [[`L`] [A UnaryLexer type.]]
- ]
- [heading Valid Expressions]
- In addition to the requirements defined in __lexer_concept__, for any
- UnaryLexer the following must be met:
- [table
- [[Expression] [Semantics] [Return type]]
- [[`l.subject`] [Subject lexer component.] [__lexer_concept__]]
- ]
- [heading Type Expressions]
- [table
- [[Expression] [Description]]
- [[`L::subject_type`] [The subject lexer component type.]]
- [[`traits::is_unary_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if
- a certain type, `L` is a UnaryLexer, `mpl::false_`
- otherwise (See __mpl_boolean_constant__).]]
- ]
- [heading Invariants]
- For any UnaryLexer, `L`, the following invariant always holds:
- * `traits::is_lexer<L::subject_type>::type` evaluates to `mpl::true_`
- [heading Models]
- The following lexer components conform to this model:
- * action lexer component (allowing to attach semantic actions)
- __fixme__ Add more links to models of UnaryLexer concept
- [endsect] [/ UnaryLexer Concept]
- [/////////////////////////////////////////////////////////////////////////////]
- [section NaryLexer]
- [heading Description]
- /NaryLexer/ is a composite lexer component that has one or more subjects. The
- NaryLexer allows its subjects to be treated in the same way as a single
- instance of a __lexer_concept__ following the __composite_pattern__.
- [heading Refinement of]
- [:__lexer_concept__]
- [variablelist Notation
- [[`l`] [A NaryLexer.]]
- [[`L`] [A NaryLexer type.]]
- ]
- [heading Valid Expressions]
- In addition to the requirements defined in __lexer_concept__, for any
- NaryLexer the following must be met:
- [table
- [[Expression] [Semantics] [Return type]]
- [[`l.elements`] [The tuple of elements.] [A __fusion__ Sequence of __lexer_concept__ types.]]
- ]
- [heading Type Expressions]
- [table
- [[Expression] [Description]]
- [[`l.elements_type`] [Elements tuple type.]]
- [[`traits::is_nary_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if
- a certain type, `L` is a NaryLexer, `mpl::false_`
- otherwise (See __mpl_boolean_constant__).]]
- ]
- [heading Invariants]
- For each element, `E`, in any NaryLexer, `L`, the following
- invariant always holds:
-
- * `traits::is_lexer<E>::type` evaluates to `mpl::true_`
- [heading Models]
- The following lexer components conform to this model:
- * lexer sequence component
- __fixme__ Add more links to models of NaryLexer concept
- [endsect] [/ NaryLexer Concept]
- [endsect]
|