11.2 How Can I Reset the Parser

The following phenomenon has several symptoms, resulting in the following typical questions:

I invoke yyparse several times, and on correct input it works
properly; but when a parse error is found, all the other calls fail
too.  How can I reset the error flag of yyparse?

or

My parser includes support for an ‘#include’-like feature, in
which case I run yyparse from yyparse.  This fails
although I did specify %define api.pure.

These problems typically come not from Bison itself, but from Lex-generated scanners. Because these scanners use large buffers for speed, they might not notice a change of input file. As a demonstration, consider the following source file, ‘first-line.l’:

%{
#include <stdio.h>
#include <stdlib.h>
%}
%%
.*\n    ECHO; return 1;
%%
int
yyparse (char const *file)
{
  yyin = fopen (file, "r");
  if (!yyin)
    exit (2);
  /* One token only.  */
  yylex ();
  if (fclose (yyin) != 0)
    exit (3);
  return 0;
}

int
main (void)
{
  yyparse ("input");
  yyparse ("input");
  return 0;
}

If the file ‘input’ contains

input:1: Hello,
input:2: World!

then instead of getting the first line twice, you get:

$ flex -ofirst-line.c first-line.l
$ gcc  -ofirst-line   first-line.c -ll
$ ./first-line
input:1: Hello,
input:2: World!

Therefore, whenever you change yyin, you must tell the Lex-generated scanner to discard its current buffer and switch to the new one. This depends upon your implementation of Lex; see its documentation for more. For Flex, it suffices to call ‘YY_FLUSH_BUFFER’ after each change to yyin. If your Flex-generated scanner needs to read from several input streams to handle features like include files, you might consider using Flex functions like ‘yy_switch_to_buffer’ that manipulate multiple input buffers.

If your Flex-generated scanner uses start conditions (see (flex)Start conditions section `Start conditions' in The Flex Manual), you might also want to reset the scanner's state, i.e., go back to the initial start condition, through a call to ‘BEGIN (0)’.