To: jimmey@lexis-nexis.com (Jimmey Todd)
Subject: Re: Flex performance question
In-reply-to: Your message of Thu, 19 Feb 1998 11:01:17 PST.
Date: Thu, 19 Feb 1998 15:42:25 PST
From: Vern Paxson <vern>

So there are several problems.

First, to go fast, you want to match as much text as possible, which
your scanners don't in the case that what they're scanning is *not*
a <RN> tag.  So you want a rule like:

	[^<]+

Second, C++ scanners are particularly slow if they're interactive,
which they are by default.  Using -B speeds it up by a factor of 3-4
on my workstation.

Third, C++ scanners that use the istream interface are slow, because
of how poorly implemented istream's are.  I built two versions of
the following scanner:

	%%
	.*\n
	.*
	%%

and the C version inhales a 2.5MB file on my workstation in 0.8 seconds.
The C++ istream version, using -B, takes 3.8 seconds.

		Vern