|

Ionflux::Tools::Utf8TokenType Struct Reference
[String tokenizer]

A token type with UTF-8 support. More...

#include <Utf8TokenTypeMap.hpp>

List of all members.

Public Attributes
int	typeID
	Numeric token type ID.
std::string	validChars
	UTF-8 encoded sequence of valid characters.
unsigned int	maxChars
	Maximum number of characters (use 0 for unlimited).
Static Public Attributes
static const int	USERTYPE_ID = 100
	First free token type ID for user types.
static const int	INVALID_ID = -1
	Type ID for invalid tokens.
static const int	EMPTY_ID = 0
	Type ID for empty tokens.

---

Detailed Description

A token type with UTF-8 support.

---

Member Data Documentation

const int Ionflux::Tools::Utf8TokenType::EMPTY_ID = 0 [static]

Type ID for empty tokens.

const int Ionflux::Tools::Utf8TokenType::INVALID_ID = -1 [static]

Type ID for invalid tokens.

unsigned int Ionflux::Tools::Utf8TokenType::maxChars

Maximum number of characters (use 0 for unlimited).

int Ionflux::Tools::Utf8TokenType::typeID

Numeric token type ID.

const int Ionflux::Tools::Utf8TokenType::USERTYPE_ID = 100 [static]

First free token type ID for user types.

std::string Ionflux::Tools::Utf8TokenType::validChars

UTF-8 encoded sequence of valid characters.

---

The documentation for this struct was generated from the following files:

• Utf8TokenTypeMap.hpp
• Utf8TokenTypeMap.cpp

---