This component‐private class is used to hold delimiter information. Each Tokenizer object will have, as a private data member, an object of this class, and will pass the address of that member to the (private) constructor of each TokenizerIterator object it issues: ` +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ | ,‐‐‐‐‐‐‐‐‐‐‐‐‐‐. | | ( Tokenizer_Data ) | | `‐‐‐‐‐‐‐‐‐‐‐‐‐‐'\ | | | | | | ,‐‐‐‐*‐‐‐‐‐‐‐‐‐‐‐‐. | | | ( TokenizerIterator ) | | | /`‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐' | | | / | | ,‐‐‐‐*‐‐o‐. | | ( Tokenizer ) | | `‐‐‐‐‐‐‐‐‐' | +‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐+ bdlb_tokenizer `

Synopsis

Declared in <bdlb_tokenizer.h>

class Tokenizer_Data;

Member Functions

Name

Description

Tokenizer_Data [constructor]

Constructors

inputType

Return the input type of the specified character: 0 for token, 1 for soft delimiter, 2 for hard delimiter.

Created with MrDocs