llvm::getToken

Extract one token from Source using the characters in Delimiters.

Synopsis

Declared in <llvm/ADT/StringExtras.h>

std::pair<StringRef, StringRef>
getToken(
    StringRef Source,
    StringRef Delimiters = " \t\n\v\f\r");

Description

This function extracts one token from source, ignoring any leading characters that appear in the Delimiters string, and ending the token at any of the characters that appear in the Delimiters string. If there are no tokens in the source string, an empty string is returned. The function returns a pair containing the extracted token and the remaining tail string.

Return Value

A pair of the extracted token and the remaining tail.

Parameters

NameDescription
Sourcetext to tokenize
Delimiterscharacters treated as separators