7.1 Writing tests for a new tool

In general, the best way to learn how to write (code or even prose) is to read something similar. This principle applies to test cases and to test suites. Unfortunately, well-established test suites have a way of developing their own conventions: as test writers become more experienced with DejaGnu and with Tcl, they accumulate more utilities, and take advantage of more and more features of expect and Tcl in general.

Inspecting such established test suites may make the prospect of creating an entirely new test suite appear overwhelming. Nevertheless, it is quite straightforward to get a new test suite going.

There is one test suite that is guaranteed not to grow more elaborate over time: both it and the tool it tests were created expressly to illustrate what it takes to get started with DejaGnu. The ‘example/’ directory of the DejaGnu distribution contains both an interactive tool called calc, and a test suite for it. Reading this test suite, and experimenting with it, is a good way to supplement the information in this section. (Thanks to Robert Lupton for creating calc and its test suite—and also the first version of this section of the manual!)

To help orient you further in this task, here is an outline of the steps to begin building a test suite for a program example.

1. Create or select a directory to contain your new collection of tests. Change to that directory (shown here as testsuite):

   eg$ cd testsuite/

2. Create a ‘configure.in’ file in this directory, to control configuration-dependent choices for your tests. So far as DejaGnu is concerned, the important thing is to set a value for the variable target_abbrev; this value is the link to the init file you will write soon. (For simplicity, we assume the environment is Unix, and use ‘unix’ as the value.)

   What else is needed in ‘configure.in’ depends on the requirements of your tool, your intended test environments, and which configure system you use. This example is a minimal configure.in for use with Cygnus Configure. (For an alternative based on the FSF autoconf system, see the calc example distributed with DejaGnu.) Replace example with the name of your program:

   # This file is a shell script fragment # for use with Cygnus configure. srctrigger="example.0" srcname="The DejaGnu example tests" # per-host: # per-target: # everything defaults to unix for a target target_abbrev=unix # post-target:

3. Create ‘Makefile.in’, the source file used by configure to build your ‘Makefile’. Its leading section should as usual contain the values that configure may override:

   srcdir = . prefix = /usr/local exec_prefix = $(prefix) bindir = $(exec_prefix)/bin libdir = $(exec_prefix)/lib tooldir = $(libdir)/$(target_alias) datadir = $(exec_prefix)/lib/dejagnu RUNTEST = runtest RUNTESTFLAGS = FLAGS_TO_PASS = #### host, target, site specific Makefile frags come in here.

   This should be followed by the standard targets at your site. To begin with, they need not do anything—for example, these definitions will do:

   all: info: install-info: install: uninstall: clean: -rm -f *~ core *.info*

   It is also a good idea to make sure your ‘Makefile’ can rebuild itself if ‘Makefile.in’ changes, with a target like this (which works for either Cygnus or FSF Configure):

   Makefile : $(srcdir)/Makefile.in $(host_makefile_frag) \ $(target_makefile_frag) $(SHELL) ./config.status

   You also need to include two targets important to DejaGnu: check, to run the tests, and site.exp, to set up the Tcl copies of configuration-dependent values. The check target must run ‘runtest --tool example’:

   check: site.exp all $(RUNTEST) $(RUNTESTFLAGS) $(FLAGS_TO_PASS) \ --tool example --srcdir $(srcdir)

   The site.exp target should usually set up (among other things!) a Tcl variable for the name of your program:

   site.exp: ./config.status Makefile @echo "Making a new config file..." -@rm -f ./tmp? @touch site.exp -@mv site.exp site.bak @echo "## these variables are automatically\ generated by make ##" > ./tmp0 @echo "# Do not edit here. If you wish to\ override these values" >> ./tmp0 @echo "# add them to the last section" >> ./tmp0 @echo "set host_os ${host_os}" >> ./tmp0 @echo "set host_alias ${host_alias}" >> ./tmp0 @echo "set host_cpu ${host_cpu}" >> ./tmp0 @echo "set host_vendor ${host_vendor}" >> ./tmp0 @echo "set target_os ${target_os}" >> ./tmp0 @echo "set target_alias ${target_alias}" >> ./tmp0 @echo "set target_cpu ${target_cpu}" >> ./tmp0 @echo "set target_vendor ${target_vendor}" >> ./tmp0 @echo "set host_triplet ${host_canonical}" >> ./tmp0 @echo "set target_triplet ${target_canonical}">>./tmp0 @echo "set tool binutils" >> ./tmp0 @echo "set srcdir ${srcdir}" >> ./tmp0 @echo "set objdir `pwd`" >> ./tmp0 @echo "set examplename example" >> ./tmp0 @echo "## All variables above are generated by\ configure. Do Not Edit ##" >> ./tmp0 @cat ./tmp0 > site.exp @sed < site.bak \ -e '1,/^## All variables above are.*##/ d' \ >> site.exp -@rm -f ./tmp?

4. Create a directory (in ‘testsuite/’) called ‘config/’:

   eg$ mkdir config

5. Make an init file in this directory; its name must start with the target_abbrev value, so call it ‘config/unix.exp’. This is the file that contains the target-dependent procedures; fortunately, most of them do not have to do very much in order for runtest to run.

   If example is not interactive, you can get away with this minimal ‘unix.exp’ to begin with:

   proc foo_exit {} {} proc foo_version {} {}

   If example is interactive, however, you might as well define a start routine and invoke it by using an init file like this:

   proc foo_exit {} {} proc foo_version {} {} proc foo_start {} { global examplename spawn $examplename expect { -re "" {} } } foo_start

6. Create a directory whose name begins with your tool’s name, to contain tests:

   eg$ mkdir example.0

7. Create a sample test file in ‘example.0’. Its name must end with ‘.exp’; you can use ‘first-try.exp’ To begin with, just write there a line of Tcl code to issue a message:

   send_user "Testing: one, two...\n"

8. Back in the ‘testsuite/’ (top level) directory, run

   eg$ configure

   (You may have to specify more of a path, if a suitable configure is not available in your execution path.)

9. You are now ready to triumphantly type ‘make check’ or ‘runtest --tool example’. You should see something like this:

   Test Run By rhl on Fri Jan 29 16:25:44 EST 1993 === example tests === Running ./example.0/first-try.exp ... Testing: one, two... === example Summary ===

   There is no output in the summary, because so far the example does not call any of the procedures that establish a test outcome.

10. Begin writing some real tests. For an interactive tool, you should probably write a real exit routine in fairly short order; in any case, you should also write a real version routine soon.