RAD Studio |  |

Tokens
See Also
Collapse All

Tokens are word-like units recognized by a language. The compiler recognizes six classes of tokens.

Here is the formal definition of a token:

- keyword
- identifier
- constant
- string-literal
- operator
- punctuator (also known as separators)

As the source code is scanned, tokens are extracted in such a way that the longest possible token from the character sequence is selected. For example, external would be parsed as a single identifier, rather than as the keyword extern followed by the identifier al.

See Token Pasting with ## for a description of token pasting.

See Also

Lexical Elements

Whitespace

Copyright(C) 2009 Embarcadero Technologies, Inc. All Rights Reserved.
What do you think about this topic? Send feedback!