This class provides a C++‐standards‐conforming input iterator over the tokens in the input string suppled at construction (along with the designation of soft and hard delimiter characters) to a Tokenizer object. Tokens are returned, using a bslstl::StringRef ‐‐ by value ‐‐ that means the iterated references remain valid until the underlying input string itself is modified or destroyed. Note that all iterators are invalidated whenever the input string in the parent Tokenizer change.

Synopsis

Declared in <bdlb_tokenizer.h>

class TokenizerIterator;

Type Aliases

Name

Description

difference_type

Signed type used to represent distances between iterators.

iterator_category

Tag identifying this type as an input iterator.

pointer

Proxy type returned by operator‐>.

reference

Reference type returned by operator*.

value_type

Element type yielded by this iterator (bslstl::StringRef).

Member Functions

Name

Description

TokenizerIterator [constructor]

Create a TokenizerIterator object having the value of the specified origin iterator.

operator=

Assign to this object the value of the specified rhs iterator, and return a reference providing modifiable access to this object.

operator*

Return a reference to the non‐modifiable current token (i.e., maximal sequence of non‐delimiter characters) in the input string. The returned reference remains valid so long as the underlying input is not modified or destroyed ‐‐ irrespective of the state (or existence) of this object. The behavior is undefined unless the iteration state of this object is initially valid, or if the underlying input has been modified or destroyed since this object was created.

operator++

Advance the iteration state of this object to refer to the next token in the underlying input sequence, and return a reference providing modifiable access to this object. The behavior is undefined unless the iteration state of this object is initially valid, or if the underlying input has been modified or destroyed since this object was created.

operator‐>

Return a proxy object containing the non‐modifiable current token (i.e., maximal sequence of non‐delimiter characters) in the input string. The returned proxy remains valid so long as the underlying input is not modified or destroyed ‐‐ irrespective of the state (or existence) of this object. The behavior is undefined unless the iteration state of this object is initially valid, or if the underlying input has been modified or destroyed since this object was created.

Friends

Name

Description

BloombergLP::bdlb::operator!=

Return true if the specified lhs and rhs objects do not have the same value, and false otherwise. The behaviour is undefined unless the iterators returned by the same Tokenizer object, or if the underlying input has been modified or destroyed since any of those objects were created.

BloombergLP::bdlb::operator==

Return true if the specified lhs and rhs objects have the same value, and false otherwise. Two TokenizerIterator objects have the same value if both of them are pointing to the same token within the same tokenized string or if they both point past the tokenized string. The behaviour is undefined unless the iterators returned by the same Tokenizer object, or if the underlying input has been modified or destroyed since any of those objects were created.

BloombergLP::bdlb::Tokenizer

This class provides (read‐only) sequential access to tokens delimited by two user‐supplied character sets consisting, respectively, of soft and hard delimiters characters. Access to the previous and current (trailing) delimiter, as well as to the current token itself, is provided efficiently via bslstl::StringRef.

Non-Member Functions

Name

Description

operator++

Advance the iteration state of the specified object to refer to the next token in the underlying input sequence, and return a copy of this object prior advancing the iteration state. The behavior is undefined unless the iteration state of this object is initially valid, or if the underlying input has been modified or destroyed since this object was created.

Created with MrDocs