10.2.4 Calc++ Scanner
The Flex scanner first includes the driver declaration, then the
parser's to get the set of defined tokens.
| | %{ /* -*- C++ -*- */
# include <cstdlib>
# include <errno.h>
# include <limits.h>
# include <string>
# include "calc++-driver.hh"
# include "calc++-parser.hh"
/* Work around an incompatibility in flex (at least versions
2.5.31 through 2.5.33): it generates code that does
not conform to C89. See Debian bug 333231
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */
# undef yywrap
# define yywrap() 1
/* By default yylex returns int, we use token_type.
Unfortunately yyterminate by default returns 0, which is
not of token_type. */
#define yyterminate() return token::END
%}
|
Because there is no #include-like feature we don't need
yywrap, we don't need unput either, and we parse an
actual file, this is not an interactive session with the user.
Finally we enable the scanner tracing features.
| | %option noyywrap nounput batch debug
|
Abbreviations allow for more readable rules.
| | id [a-zA-Z][a-zA-Z_0-9]*
int [0-9]+
blank [ \t]
|
The following paragraph suffices to track locations accurately. Each
time yylex is invoked, the begin position is moved onto the end
position. Then when a pattern is matched, the end position is
advanced of its width. In case it matched ends of lines, the end
cursor is adjusted, and each time blanks are matched, the begin cursor
is moved onto the end cursor to effectively ignore the blanks
preceding tokens. Comments would be treated equally.
| | %{
# define YY_USER_ACTION yylloc->columns (yyleng);
%}
%%
%{
yylloc->step ();
%}
{blank}+ yylloc->step ();
[\n]+ yylloc->lines (yyleng); yylloc->step ();
|
The rules are simple, just note the use of the driver to report errors.
It is convenient to use a typedef to shorten
yy::calcxx_parser::token::identifier into
token::identifier for instance.
| | %{
typedef yy::calcxx_parser::token token;
%}
/* Convert ints to the actual type of tokens. */
[-+*/] return yy::calcxx_parser::token_type (yytext[0]);
":=" return token::ASSIGN;
{int} {
errno = 0;
long n = strtol (yytext, NULL, 10);
if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
driver.error (*yylloc, "integer is out of range");
yylval->ival = n;
return token::NUMBER;
}
{id} yylval->sval = new std::string (yytext); return token::IDENTIFIER;
. driver.error (*yylloc, "invalid character");
%%
|
Finally, because the scanner related driver's member function depend
on the scanner's data, it is simpler to implement them in this file.
| | void
calcxx_driver::scan_begin ()
{
yy_flex_debug = trace_scanning;
if (!(yyin = fopen (file.c_str (), "r")))
error (std::string ("cannot open ") + file);
}
void
calcxx_driver::scan_end ()
{
fclose (yyin);
}
|
