BloombergLP::bdlb::Tokenizer_Data::Tokenizer

Constructors

Synopses

Declared in <bdlb_tokenizer.h>

Create a Tokenizer_Data from the specified soft delimiters.

explicit
Tokenizer_Data(std::string_view const& softDelimiters);

Create a Tokenizer_Data from soft and hard delimiter sets.

Tokenizer_Data(
    std::string_view const& softDelimiters,
    std::string_view const& hardDelimiters);

Name

Description

softDelimiters

characters treated as soft delimiters

hardDelimiters

characters treated as hard delimiters

Created with MrDocs