10.3 Linking with dlopened modules

When, say, an interpreter application uses dlopened modules to extend the list of methods it provides, an obvious abstraction for the maintainers of the interpreter is to have all methods (including the built in ones supplied with the interpreter) accessed through dlopen. For one thing, the dlopening functionality will be tested even during routine invocations. For another, only one subsystem has to be written for getting methods into the interpreter.

The downside of this abstraction is, of course, that environments that provide only static linkage can’t even load the intrinsic interpreter methods. Not so! We can statically link those methods by dlpreopening them.

Unfortunately, since platforms such as AIX and cygwin require that all library symbols must be resolved at compile time, the interpreter maintainers will need to provide a library to both its own dlpreopened modules, and third-party modules loaded by dlopen. In itself, that is not so bad, except that the interpreter too must provide those same symbols otherwise it will be impossible to resolve all the symbols required by the modules as they are loaded. Things are even worse if the code that loads the modules for the interpreter is itself in a library – and that is usually the case for any non-trivial application. Modern platforms take care of this by automatically loading all of a module’s dependency libraries as the module is loaded (libltdl can do this even on platforms that can’t do it by themselves). In the end, this leads to problems with duplicated symbols and prevents modules from loading, and prevents the application from compiling when modules are preloaded.

,-------------.    ,------------------.    ,-----------------.
| Interpreter |---->     Module------------>   Third-party   |
`-------------'    |     Loader       |    |Dlopened Modules |
                   |        |         |    `-----------------'
                   |,-------v--------.|             |
                   ||  Dlpreopened   ||             |
                   ||    Modules     ||             |
                   |`----------------'|             |
                   |        |         |             |
                   |,-------v--------.|    ,--------v--------.
                   ||Module Interface||    |Module Interface |
                   ||    Library     ||    |     Library     |
                   |`----------------'|    `-----------------'
                   `------------------'

Libtool has the concept of weak library interfaces to circumvent this problem. Recall that the code that dlopens method-provider modules for the interpreter application resides in a library: All of the modules and the dlopener library itself should be linked against the common library that resolves the module symbols at compile time. To guard against duplicate symbol definitions, and for dlpreopened modules to work at all in this scenario, the dlopener library must declare that it provides a weak library interface to the common symbols in the library it shares with the modules. That way, when libtool links the Module Loader library with some Dlpreopened Modules that were in turn linked against the Module Interface Library, it knows that the Module Loader provides an already loaded Module Interface Library to resolve symbols for the Dlpreopened Modules, and doesn’t ask the compiler driver to link an identical Module Interface Library dependency library too.

In conjunction with Automake, the ‘Makefile.am’ for the Module Loader might look like this:

lib_LTLIBRARIES = libinterface.la libloader.la

libinterface_la_SOURCES = interface.c interface.h
libinterface_la_LDFLAGS = -version-info 3:2:1

libloader_la_SOURCES    = loader.c
libloader_la_LDFLAGS    = -weak libinterface.la \
                          -version-info 3:2:1 \
                          -dlpreopen ../modules/intrinsics.la
libloader_la_LIBADD     = $(libinterface_la_OBJECTS)

And the ‘Makefile.am’ for the ‘intrinsics.la’ module in a sibling ‘modules’ directory might look like this:

AM_CPPFLAGS             = -I$(srcdir)/../libloader
AM_LDFLAGS              = -no-undefined -module -avoid-version \
                          -export-dynamic

noinst_LTLIBRARIES      = intrinsics.la

intrinsics_la_LIBADD    = ../libloader/libinterface.la

../libloader/libinterface.la:
        cd ../libloader && $(MAKE) $(AM_MAKEFLAGS) libinterface.la

For a more complex example, see the sources of ‘libltdl’ in the Libtool distribution, which is built with the help of the ‘-weak’ option.

This document was generated on December 1, 2011 using texi2html 5.0.