File: autoconf.info,  Node: Using Autotest,  Next: FAQ,  Prev: Obsolete Constructs,  Up: Top

19 Generating Test Suites with Autotest
***************************************

     *N.B.: This section describes a feature which is still
     stabilizing.  Although we believe that Autotest is useful as-is, this
     documentation describes an interface which might change in the future:
     do not depend upon Autotest without subscribing to the Autoconf mailing
     lists.*

   It is paradoxical that portable projects depend on nonportable tools
to run their test suite.  Autoconf by itself is the paragon of this
problem: although it aims at perfectly portability, up to 2.13 its test
suite was using DejaGNU, a rich and complex testing framework, but which
is far from being standard on Posix systems.  Worse yet, it was likely
to be missing on the most fragile platforms, the very platforms that are
most likely to torture Autoconf and exhibit deficiencies.

   To circumvent this problem, many package maintainers have developed
their own testing framework, based on simple shell scripts whose sole
outputs are exit status values describing whether the test succeeded.
Most of these tests share common patterns, and this can result in lots
of duplicated code and tedious maintenance.

   Following exactly the same reasoning that yielded to the inception of
Autoconf, Autotest provides a test suite generation framework, based on
M4 macros building a portable shell script.  The suite itself is
equipped with automatic logging and tracing facilities which greatly
diminish the interaction with bug reporters, and simple timing reports.

   Autoconf itself has been using Autotest for years, and we do attest
that it has considerably improved the strength of the test suite and the
quality of bug reports.  Other projects are known to use some generation
of Autotest, such as Bison, GNU Wdiff, GNU Tar, each of them with
different needs, and this usage has validated Autotest as a general
testing framework.

   Nonetheless, compared to DejaGNU, Autotest is inadequate for
interactive tool testing, which is probably its main limitation.

* Menu:

* Using an Autotest Test Suite::  Autotest and the user
* Writing Testsuites::          Autotest macros
* testsuite Invocation::        Running ‘testsuite’ scripts
* Making testsuite Scripts::    Using autom4te to create ‘testsuite’