[#BloombergLP-bdlb-TokenizerIterator]
= xref:BloombergLP.adoc[BloombergLP]::xref:BloombergLP/bdlb.adoc[bdlb]::TokenizerIterator
:relfileprefix: ../../
:mrdocs:

Input iterator over the tokens of a bound `Tokenizer` input string&period;

== Synopsis

Declared in `&lt;bdlb&lowbar;tokenizer&period;h&gt;`

[source,cpp,subs="verbatim,replacements,macros,-callouts"]
----
class TokenizerIterator;
----

== Description

This class provides a C&plus;&plus;&hyphen;standards&hyphen;conforming input iterator over the tokens in the input string suppled at construction (along with the designation of _soft_ and _hard_ delimiter characters) to a `Tokenizer` object&period; Tokens are returned, using a `bslstl&colon;&colon;StringRef` &hyphen;&hyphen; by value &hyphen;&hyphen; that means the iterated references remain valid until the underlying input string itself is modified or destroyed&period; Note that all iterators are invalidated whenever the input string in the parent `Tokenizer` change&period;

== Type Aliases

[cols="1,4"]
|===
| Name| Description
| xref:BloombergLP/bdlb/TokenizerIterator/difference_type.adoc[`difference&lowbar;type`] 
| Signed type used to represent distances between iterators&period;
| xref:BloombergLP/bdlb/TokenizerIterator/iterator_category.adoc[`iterator&lowbar;category`] 
| Tag identifying this type as an input iterator&period;
| xref:BloombergLP/bdlb/TokenizerIterator/pointer.adoc[`pointer`] 
| Proxy type returned by `operator&hyphen;&gt;`&period;
| xref:BloombergLP/bdlb/TokenizerIterator/reference.adoc[`reference`] 
| Reference type returned by `operator&ast;`&period;
| xref:BloombergLP/bdlb/TokenizerIterator/value_type.adoc[`value&lowbar;type`] 
| Element type yielded by this iterator (`bslstl&colon;&colon;StringRef`)&period;
|===


== Member Functions

[cols="1,4"]
|===
| Name| Description
| xref:BloombergLP/bdlb/TokenizerIterator/2constructor-0e.adoc[`TokenizerIterator`]         [.small]#[constructor]#
| Constructors
| xref:BloombergLP/bdlb/TokenizerIterator/operator_assign.adoc[`operator&equals;`] 
| Assign to this object the value of the specified `rhs` iterator, and return a reference providing modifiable access to this object&period;
| xref:BloombergLP/bdlb/TokenizerIterator/operator_star.adoc[`operator&ast;`] 
| Return a reference to the non&hyphen;modifiable current token&period;
| xref:BloombergLP/bdlb/TokenizerIterator/operator_inc.adoc[`operator&plus;&plus;`] 
| Advance this iterator to the next token and return `&ast;this`&period;
| xref:BloombergLP/bdlb/TokenizerIterator/operator_ptr.adoc[`operator&hyphen;&gt;`] 
| Return a proxy holding the non&hyphen;modifiable current token&period;
|===


== Friends

[cols="1,4"]
|===
| Name| Description
| `xref:BloombergLP/bdlb/operator_not_eq-098.adoc[BloombergLP::bdlb::operator!&equals;]`
| Return `true` if `lhs` and `rhs` do not refer to the same token position&period;
| `xref:BloombergLP/bdlb/operator_eq-026.adoc[BloombergLP::bdlb::operator&equals;&equals;]`
| Return `true` if `lhs` and `rhs` refer to the same token position&period;
| `xref:BloombergLP/bdlb/Tokenizer.adoc[BloombergLP::bdlb::Tokenizer]`
| Provides sequential read&hyphen;only access to tokens with soft and hard delimiters&period;
|===

== Non-Member Functions

[cols="1,4"]
|===
| Name| Description
| xref:BloombergLP/bdlb/operator_inc-07.adoc[`operator&plus;&plus;`]
| Advance `object` to the next token and return a copy of its prior value&period;
|===


[.small]#Created with https://www.mrdocs.com[MrDocs]#
