[#BloombergLP-bdlb-Tokenizer-2constructor-08]
= xref:BloombergLP.adoc[BloombergLP]::xref:BloombergLP/bdlb.adoc[bdlb]::xref:BloombergLP/bdlb/Tokenizer.adoc[Tokenizer]::Tokenizer
:relfileprefix: ../../../
:mrdocs:

Constructors

== Synopses

Declared in `<bdlb_tokenizer.h>`

Create a `Tokenizer` bound to `input` using the specified soft delimiters.


[source,cpp,subs="verbatim,replacements,macros,-callouts"]
----
xref:BloombergLP/bdlb/Tokenizer/2constructor-06.adoc[Tokenizer](
    std::string_view const& input,
    std::string_view const& soft);
----

[.small]#xref:BloombergLP/bdlb/Tokenizer/2constructor-06.adoc[_» more..._]#

Create a `Tokenizer` bound to `input` using the specified soft delimiters.


[source,cpp,subs="verbatim,replacements,macros,-callouts"]
----
xref:BloombergLP/bdlb/Tokenizer/2constructor-0a.adoc[Tokenizer](
    char const* input,
    std::string_view const& soft);
----

[.small]#xref:BloombergLP/bdlb/Tokenizer/2constructor-0a.adoc[_» more..._]#

Create a `Tokenizer` bound to `input` using soft and hard delimiters.


[source,cpp,subs="verbatim,replacements,macros,-callouts"]
----
xref:BloombergLP/bdlb/Tokenizer/2constructor-0f.adoc[Tokenizer](
    std::string_view const& input,
    std::string_view const& soft,
    std::string_view const& hard);
----

[.small]#xref:BloombergLP/bdlb/Tokenizer/2constructor-0f.adoc[_» more..._]#

Create a `Tokenizer` bound to `input` using soft and hard delimiters.


[source,cpp,subs="verbatim,replacements,macros,-callouts"]
----
xref:BloombergLP/bdlb/Tokenizer/2constructor-09.adoc[Tokenizer](
    char const* input,
    std::string_view const& soft,
    std::string_view const& hard);
----

[.small]#xref:BloombergLP/bdlb/Tokenizer/2constructor-09.adoc[_» more..._]#

== Parameters

[cols="1,4"]
|===
| Name| Description
| *input*
| character sequence to tokenize
| *soft*
| set of unique soft delimiter characters
| *hard*
| set of unique hard delimiter characters
|===


[.small]#Created with https://www.mrdocs.com[MrDocs]#