[#BloombergLP-bdlb-TokenizerIterator]
= xref:BloombergLP.adoc[BloombergLP]::xref:BloombergLP/bdlb.adoc[bdlb]::TokenizerIterator
:relfileprefix: ../../
:mrdocs:

Input iterator over the tokens of a bound `Tokenizer` input string.

== Synopsis

Declared in `<bdlb_tokenizer.h>`

[source,cpp,subs="verbatim,replacements,macros,-callouts"]
----
class TokenizerIterator;
----

== Description

This class provides a C++‐standards‐conforming input iterator over the tokens in the input string suppled at construction (along with the designation of _soft_ and _hard_ delimiter characters) to a `Tokenizer` object. Tokens are returned, using a `bslstl::StringRef` ‐‐ by value ‐‐ that means the iterated references remain valid until the underlying input string itself is modified or destroyed. Note that all iterators are invalidated whenever the input string in the parent `Tokenizer` change.

== Type Aliases

[cols="1,4"]
|===
| Name| Description
| xref:BloombergLP/bdlb/TokenizerIterator/difference_type.adoc[`difference_type`] 
| Signed type used to represent distances between iterators.
| xref:BloombergLP/bdlb/TokenizerIterator/iterator_category.adoc[`iterator_category`] 
| Tag identifying this type as an input iterator.
| xref:BloombergLP/bdlb/TokenizerIterator/pointer.adoc[`pointer`] 
| Proxy type returned by `operator‐>`.
| xref:BloombergLP/bdlb/TokenizerIterator/reference.adoc[`reference`] 
| Reference type returned by `operator*`.
| xref:BloombergLP/bdlb/TokenizerIterator/value_type.adoc[`value_type`] 
| Element type yielded by this iterator (`bslstl::StringRef`).
|===


== Member Functions

[cols="1,4"]
|===
| Name| Description
| xref:BloombergLP/bdlb/TokenizerIterator/2constructor-0e.adoc[`TokenizerIterator`]         [.small]#[constructor]#
| Constructors
| xref:BloombergLP/bdlb/TokenizerIterator/operator_assign.adoc[`operator=`] 
| Assign to this object the value of the specified `rhs` iterator, and return a reference providing modifiable access to this object.
| xref:BloombergLP/bdlb/TokenizerIterator/operator_star.adoc[`operator*`] 
| Return a reference to the non‐modifiable current token.
| xref:BloombergLP/bdlb/TokenizerIterator/operator_inc.adoc[`operator++`] 
| Advance this iterator to the next token and return `*this`.
| xref:BloombergLP/bdlb/TokenizerIterator/operator_ptr.adoc[`operator‐>`] 
| Return a proxy holding the non‐modifiable current token.
|===


== Friends

[cols="1,4"]
|===
| Name| Description
| `xref:BloombergLP/bdlb/operator_not_eq-098.adoc[BloombergLP::bdlb::operator!=]`
| Return `true` if `lhs` and `rhs` do not refer to the same token position.
| `xref:BloombergLP/bdlb/operator_eq-026.adoc[BloombergLP::bdlb::operator==]`
| Return `true` if `lhs` and `rhs` refer to the same token position.
| `xref:BloombergLP/bdlb/Tokenizer.adoc[BloombergLP::bdlb::Tokenizer]`
| Provides sequential read‐only access to tokens with soft and hard delimiters.
|===

== Non-Member Functions

[cols="1,4"]
|===
| Name| Description
| xref:BloombergLP/bdlb/operator_inc-07.adoc[`operator++`]
| Advance `object` to the next token and return a copy of its prior value.
|===


[.small]#Created with https://www.mrdocs.com[MrDocs]#