identifiers can now contain hyphens, with the following restrictions:
* an identifier cannot start or end with a hyphen.
* an identifier cannot contain more than one hyphen in a row.
kebab-case identifiers can be used for type and variable names, as well
as message identifiers and labels.
to avoid ambiguity, the lexer now enforces whitespace around most binary
operators (with a few exceptions, such as semicolons). trying to
compile a "compact" arithmetic expression, such as
y=1+2
will now result in a "missing whitespace" error.
the var keyword allows greater control over what scope a
particular variable exists in. it clarifies whether a new
variable is being defined or an existing variable is being
assigned to. it will also facilitate the implementation of
global variables.
if this function returns false, any attempt to read tokens from the lexer will cause another line of input to be retrieved from the lexer's line source.
the state stack is used to track whether a string is currently being read,
what kind of string it is, and whether or not we are in an interpolation
within that string.
