Cross-Version Extensions with Stubs

Dynamic languages usually emphasize adding extensions to the core language, Tcl probably more so than others. Extensions let you add all kinds of features and capabilities that would otherwise need to be included in the core language. In Tcl, these new features often appear as new Tcl commands, otherwise indistinguishable from the built-in commands.

The problem. Extensions need to use facilities of the core language library, for parsing arguments, working with interpreters, and so on. This means they need to link against the core libraries. But if you're linking against the library to call particular functions, that means you need to compile your extension against the exact version of the library you're using. An extension built for one version of the core language won't work on another version without recompiling it.

That makes it harder for people to upgrade, since they need to worry about upgrading all their extensions, and usually means that they need to recompile each piece individually. One way that many languages mediate this inconvenience is through so-called "batteries included" distributions, which bundle up compiled versions of the core language as well as lots and lots of popular extensions, so you just upgrade the whole thing at once, recompiling only those pieces that aren't in the distribution. Tcl has several such distributions.

Version independence with stubs. Tcl also has a better way to solve this problem, that makes it possible to take a Tcl extension compiled with one version of Tcl and use it with different versions, all without recompiling. Not only that, it removes all the operating system specific and compiler specific headaches associated with dynamic linking.

This makes upgrading remarkably easy, and is truly a wonderfully convenient and time saving feature for Tcl programmers. It uses a simple, but incredibly clever mechanism called stubs.

How stubs works. The basic idea is that rather than linking directly into functions scattered throughout the Tcl library, the extension links to a single function called Tcl_InitStubs. That function populates a function table with the addresses of all the functions exported by the Tcl library. The extension then is compiled to call functions via its own function table, rather than directly. As long as existing entries in the function table don't change between versions, the underlying implementation of the library can continue to change (and functions can be added to the table), the extension will continue to work.

The nice thing is, this is all completely transparent to extension writers and users. They simply need to remember to call Tcl_InitStubs at the beginning of their code. The header files and implementation of the Tcl library take care of all the rest, and the Tcl core developers ensure the exported function table stays backwards compatible across versions.

There are of course ways to restrict what versions of Tcl an extension can be loaded into (e.g. at least 8.4.5) to account for new additions to the core, and compatibility across major versions (e.g. 8.x to 9.x) isn't guaranteed. But all in all, stubs makes the daily job of using extensions with Tcl a breeze.