Extract one token from Source using the characters in Delimiters.
Declared in <llvm/ADT/StringExtras.h>
std::pair<StringRef, StringRef>
getToken(
StringRef Source,
StringRef Delimiters = " \t\n\v\f\r");
This function extracts one token from source, ignoring any leading characters that appear in the Delimiters string, and ending the token at any of the characters that appear in the Delimiters string. If there are no tokens in the source string, an empty string is returned. The function returns a pair containing the extracted token and the remaining tail string.
A pair of the extracted token and the remaining tail.
| Name | Description |
|---|---|
| Source | text to tokenize |
| Delimiters | characters treated as separators |