File: autoconf.info,  Node: Making testsuite Scripts,  Prev: testsuite Invocation,  Up: Using Autotest

19.4 Making ‘testsuite’ Scripts
===============================

For putting Autotest into movement, you need some configuration and
makefile machinery.  We recommend, at least if your package uses deep or
shallow hierarchies, that you use ‘tests/’ as the name of the directory
holding all your tests and their makefile.  Here is a check list of
things to do, followed by an example, taking into consideration whether
you are also using Automake.

   − Make sure to create the file ‘package.m4’, which defines the
     identity of the package.  It must define ‘AT_PACKAGE_STRING’, the
     full signature of the package, and ‘AT_PACKAGE_BUGREPORT’, the
     address to which bug reports should be sent.  For sake of
     completeness, we suggest that you also define ‘AT_PACKAGE_NAME’,
     ‘AT_PACKAGE_TARNAME’, ‘AT_PACKAGE_VERSION’, and ‘AT_PACKAGE_URL’.
     *Note Initializing configure::, for a description of these
     variables.  Be sure to distribute ‘package.m4’ and to put it into
     the source hierarchy: the test suite ought to be shipped!  See
     below for an example.

   − Invoke ‘AC_CONFIG_TESTDIR’ in your ‘configure.ac’.

      -- Macro: AC_CONFIG_TESTDIR (DIRECTORY, [TEST-PATH = DIRECTORY])
          An Autotest test suite is to be configured in DIRECTORY.  This
          macro causes ‘DIRECTORY/atconfig’ to be created by
          ‘config.status’ and sets the default ‘AUTOTEST_PATH’ to
          TEST-PATH (*note testsuite Invocation::).

   − Still within ‘configure.ac’, as appropriate, ensure that some
     ‘AC_CONFIG_FILES’ command includes substitution for
     ‘tests/atlocal’.

   − Also within your ‘configure.ac’, arrange for the ‘AUTOM4TE’
     variable to be set.

   − The appropriate ‘Makefile’ should be modified so the validation in
     your package is triggered by ‘make check’.

   The following example demonstrates the above checklist, first by
assuming that you are using Automake (see below for tweaks to make to
get the same results without Automake).  Begin by adding the following
lines to your ‘configure.ac’:

     # Initialize the test suite.
     AC_CONFIG_TESTDIR([tests])
     AC_CONFIG_FILES([tests/Makefile tests/atlocal])
     AM_MISSING_PROG([AUTOM4TE], [autom4te])

   Next, add the following lines to your ‘tests/Makefile.am’, in order
to link ‘make check’ with a validation suite.

     # The ':;' works around a Bash 3.2 bug when the output is not writable.
     $(srcdir)/package.m4: $(top_srcdir)/configure.ac
             :;{ \
               echo '# Signature of the current package.' && \
               echo 'm4_define([AT_PACKAGE_NAME],' && \
               echo '  [$(PACKAGE_NAME)])' && \
               echo 'm4_define([AT_PACKAGE_TARNAME],' && \
               echo '  [$(PACKAGE_TARNAME)])' && \
               echo 'm4_define([AT_PACKAGE_VERSION],' && \
               echo '  [$(PACKAGE_VERSION)])' && \
               echo 'm4_define([AT_PACKAGE_STRING],' && \
               echo '  [$(PACKAGE_STRING)])' && \
               echo 'm4_define([AT_PACKAGE_BUGREPORT],' && \
               echo '  [$(PACKAGE_BUGREPORT)])'; \
               echo 'm4_define([AT_PACKAGE_URL],' && \
               echo '  [$(PACKAGE_URL)])'; \
             } >'$(srcdir)/package.m4'

     EXTRA_DIST = testsuite.at $(srcdir)/package.m4 $(TESTSUITE) atlocal.in
     TESTSUITE = $(srcdir)/testsuite

     check-local: atconfig atlocal $(TESTSUITE)
             $(SHELL) '$(TESTSUITE)' $(TESTSUITEFLAGS)

     installcheck-local: atconfig atlocal $(TESTSUITE)
             $(SHELL) '$(TESTSUITE)' AUTOTEST_PATH='$(bindir)' \
               $(TESTSUITEFLAGS)

     clean-local:
             test ! -f '$(TESTSUITE)' || \
              $(SHELL) '$(TESTSUITE)' --clean

     AUTOTEST = $(AUTOM4TE) --language=autotest
     $(TESTSUITE): $(srcdir)/testsuite.at $(srcdir)/package.m4
             $(AUTOTEST) -I '$(srcdir)' -o $@.tmp $@.at
             mv $@.tmp $@

   Note that the built testsuite is distributed; this is necessary
because users might not have Autoconf installed, and thus would not be
able to rebuild it.  Likewise, the use of Automake's ‘AM_MISSING_PROG’
will arrange for the definition of ‘$AUTOM4TE’ within the Makefile to
provide the user with a nicer error message if they modify a source file
to the testsuite, and accidentally trigger the rebuild rules.

   You might want to list explicitly the dependencies, i.e., the list of
the files ‘testsuite.at’ includes.

   If you don't use Automake, you should make the following tweaks.  In
your ‘configure.ac’, replace the ‘AM_MISSING_PROG’ line above with
‘AC_PATH_PROG([AUTOM4TE], [autom4te], [false])’.  You are welcome to
also try using the ‘missing’ script from the Automake project instead of
‘false’, to try to get a nicer error message when the user modifies
prerequisites but did not have Autoconf installed, but at that point you
may be better off using Automake.  Then, take the code suggested above
for ‘tests/Makefile.am’ and place it in your ‘tests/Makefile.in’
instead.  Add code to your ‘tests/Makefile.in’ to ensure that
‘$(EXTRA_DIST)’ files are distributed, as well as adding the following
additional lines to prepare the set of needed Makefile variables:

     subdir = tests
     PACKAGE_NAME = @PACKAGE_NAME@
     PACKAGE_TARNAME = @PACKAGE_TARNAME@
     PACKAGE_VERSION = @PACKAGE_VERSION@
     PACKAGE_STRING = @PACKAGE_STRING@
     PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@
     PACKAGE_URL = @PACKAGE_URL@
     AUTOM4TE = @AUTOM4TE@

     atconfig: $(top_builddir)/config.status
             cd $(top_builddir) && \
                $(SHELL) ./config.status $(subdir)/$@

     atlocal: $(srcdir)/atlocal.in $(top_builddir)/config.status
             cd $(top_builddir) && \
                $(SHELL) ./config.status $(subdir)/$@

   Using the above example (with or without Automake), and assuming you
were careful to not initialize ‘TESTSUITEFLAGS’ within your makefile,
you can now fine-tune test suite execution at runtime by altering this
variable, for example:

     make check TESTSUITEFLAGS='-v -d -x 75 -k AC_PROG_CC CFLAGS=-g'