StrategoXT 0.11

Released July 16, 2004

Download

See the installation instructions if you are not familiar with the standard installation procedure of tarballs or RPMs.

Source tar.gz

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11

Source RPM

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11

Redhat Linux RPM

Redhat 8.0:

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11

Redhat 9.0:

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11

SuSE Linux RPM

SuSE 8.2:

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11

Mac OS X binaries

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11
• aterm-2.2 + sdf2-bundle-2.2 + strategoxt-0.11

Microsoft Windows Cygwin binaries

• aterm-2.2
• sdf2-bundle-2.2
• strategoxt-0.11
• aterm-2.2 + sdf2-bundle-2.2 + strategoxt-0.11

License

StrategoXT is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

Summary of Changes

• Separate compilation is now ready for use

• Shared libraries in the Stratego runtime

• Cygwin and Mac OS X Support (really!)

• Various deployment improvements
  • Reduced the size of distributions
  • Check dependencies during configure

• Upgrade of SDF packages (pgen 2.2)

• Support for XML Processing
  • Generate or transform XML
  • Interoperability with XML tools

• Various new or improved tools:
  • pack-sdf can now include modules from syntax definitions
  • sdf2parenthesize improves the support for implementing pretty-printers
  • sdf2rtg complains about missing start symbols

• New strategies in the Stratego library:
  • generate unique names with a prefix
  • stateful sets
  • make (temporary) directories
  • filtering text files

• Various improvements of XTC:
  • Construction of build-time XTC repositories
  • Compilation of XTC tools with stand-alone strc

• Global choice operator is now much easier to use.

• Many bug-fixes, e.g.
  • fixed a bug that causes crashes in large XTC tools
  • improvements of new dynamic rules
  • mutual recursive lets

Separate Compilation

The basic implementation of separate compilation of Stratego programs has been available from StrategoXT 0.9.4, but the feature has always been experimental. However, it was already clear that separate compilation drastically reduces the compilation time of Stratego programs. The open issues in the implementation of separate compilation, which are mostly configuration related, have all been resolved in this release, StrategoXT 0.11. Therefore, separate compilation of Stratego is now ready for use.

First, AutoXT has been extended with support for separate compilation. Because of the support in AutoXT, using separate compilation in your packages is very easy. Second, separate compilation is now also supported for XTC tools, for which it is particularly useful. Third, the stand-alone Stratego compiler strc now accepts flags for separate compilation. This is very convenient for small tests programs, or just for testing separate compilation.

The exact details of using separate compilation are explained in a how to available at the Stratego website (documentation section).

(Eelco Visser and Martin Bravenboer)

Shared libraries in the Stratego runtime

In StrategoXT 0.11 the Stratego Runtime System (srts) uses Libtool to build libraries. This means that shared libraries are created if the platform supports this. Since linkers by default prefer shared libraries over static libraries, your programs will be linked against the shared libraries of the Stratego runtime. This reduces the size of the executable and the code of the library will be shared by all Stratego programs.

Using shared libraries introduces the usual problems of locating these libraries when a program is executed. This might cause problems for users of your packages: if you do not change the way your programs are linked, then at most systems the shared libraries will not be found if the package is installed at a non-standard location. To solve this issue, we advice to use Libtool for linking, since Libtool adds the path of a dynamic library to the search path of an executable. For an overview of all options to avoid or prefer shared libraries, see the how to use shared libraries at the Stratego website.

(Martin Bravenboer)

Cygwin and Mac OS X Support

For StrategoXT 0.10 we announced support for Mac OS X. Unfortunately, this claim soon appeared to be poorly verified: we received several reports of failures at Mac OS X. For StrategoXT 0.11, we have worked on solving all these issues and can now safely re-announce support for Mac OS X. StrategoXT 0.11 has been compiled and checked at two different Mac OS X 10.3.4 systems with Apple's GCC 3.3 (unpatched). Older versions of GCC will probably not work. From now on, binary distributions for Mac OS X will be available at the release page.

Various problems had to be fixed in order to support Mac OS X:

• A very creepy issue with optimizations performed by GCC has been worked-around in the implementation of the some traversal primitive.

• Various issues with separate compilation at Mac OS X have been resolved. For example, initializers of dynamic libraries are by default invoked in a lazy way at Mac OS X, which Stratego separate compilation does not allow. This issue has been fixed in AutoXT by passing a flag to the linker at Darwin systems.

• The stand-alone Stratego compiler now passes more standard flags to GCC.

(Eelco Dolstra, Martin Bravenboer and Dick Eimers)

Cygwin support has also been verified. Unfortunately, separate compilation is currently not supported at Cygwin, due to limitations of Microsoft Windows DLLs. As a result, static linking is used for the separately compiled SSL and executables are rather large. We hope to fix this in a future release. From StrategoXT 0.11 binary distributions for Cygwin will be available at the release pages.

(Eelco Dolstra and Martin Bravenboer)

Various Deployment Improvements

Besides the support for Mac OS X and Cygwin, the deployment of StrategoXT has been improved at several other points. First, the distributions of StrategoXT 0.11 are much smaller then the distributions of the last few releases. A source distribution is now about 9 MB, which used to be 14 MB. This has been achieved by (1) not distributing bootstrapped sources of programs that are only used in a check, (2) not distributing parse tables, (3) reduction of the size of bootstrapped sources by heavy use of separate compilation in StrategoXT, (4) removing some tools that are obsolete or not working properly.

Second, the dependencies of StrategoXT are now finally checked in the configure script. If the provided configuration is incorrect, then the configure will fail and explain where the tools were expected in the configuration provided (or not) by the user. The configure scripts checks for the ATerm-gcc library in the aterm package, which reflects the requirement to configure this package with the --with-gcc flag. The StrategoXT configure script now also checks for several tools that are part of the packages of the sdf2-bundle.

The checks have been implemented as macros (XT_PKG_ATERM and XT_PKG_SDF) in AutoXT, so you can use them in your package as well. The macro XT_PKG_STRATEGOXT can be used if you want to check that the provided configuration for StrategoXT is correct. This macro checks that some essential tools in StrategoXT are available at the specified prefix. If you want to, then you can also invoke the checks for the specific program directly: they are available as XT_PROG_ macros in AutoXT.

(Martin Bravenboer)

Upgrade of SDF Packages

StrategoXT 0.11 uses a new version of the SDF language and its implementation. The SDF packages are collected in the sdf2-bundle 2.2, which is available at the release page of StrategoXT 0.11.

Important: the new version of SDF contains a new construct for declaring start symbols of the syntax definition. In older versions of SDF there was no way to declare start symbols. Instead, all declared sorts were implicitly added as start symbols. You must add this new explicit declaration of start-symbols to your syntax definition, otherwise the generated parse table will not parse anything at all. For example, if you want Module to be a start symbol of your syntax definition, then you should add a declaration context-free start-symbols Module to you syntax definition. You can also declare more complex start symbols, for example lists. Declaring all sorts (in a sorts construct)is now encouraged, since the parse table generator is then able to do more static checking of the syntax definition and warn you about suspicious constructs.

(Martin Bravenboer)

Support for XML Processing

StrategoXT 0.11 has a new sub-package: xml-front. This package contains an SDF syntax definition for XML, a pretty-printer for XML, an embedding of XML in Stratego, and various tools for XML processing. The syntax definition of XML and the accompanying tools can be used for the implementation of XML transformations, for generating XML (e.g. XHTML web pages) or for interoperability with external XML-based tools. How to use the tools of xml-front for XML applications will be explained in some howtos at the Stratego Wiki. The first one will explain the embedding of XML in Stratego, which is in particular useful for generating XML in Stratego programs.

(Martin Bravenboer)

New and Improved Tools

Many tools in StrategoXT have been improved. Some highlights:

pack-sdf now supports the "-Idef Foo.def" flag. This flag can be used to import SDF modules (.sdf) from an SDF syntax definition (.def). The order of mixed -I and -Idef arguments is respected.

sdf2parenthesize is a new tool that generates a Stratego transformation tool that adds the necessary parentheses to an abstract syntax tree. The information is obtained from an SDF syntax definition. The tool handles associativity, relative priorities, and groups of associative productions. The tool has successfully been applied to improve the pretty-printers of Stratego in stratego-front and Java in JavaFront. The tool is explained in more detail at the Tools web: sdf2parenthesize.

sdf2rtg, which is used for the generation of Stratego signatures, now uses the real start symbols of the SDF syntax definition and not the declared sorts. sdf2rtg also complains if the syntax definition contains no start symbols at all.

(Martin Bravenboer)

New strategies in the Stratego library

The Stratego Standard Library has been extended with several useful strategies.

The newname strategy is a variant of the new strategy. It generates unique strings, just like new, but it also accepts a prefix that will be part of the generated string. By default, the numbering is also done per prefix. For example, if you apply newname three times to the string "foo", then the results will be "foo_0", "foo_1" and "foo_2". If newname is applied to "bar" after this, then the result will be "bar_0", not "bar_4". Thus, The newname strategy is very useful for generating more user-friendly, unique names in a program transformation.

(Arthur van Dam)

The new sets module provides support for efficient, stateful sets, comparable to stateful hashtables in the tables module. Like hashtables, the set implementation is based on an implementation of this data structure in the ATerm library.

(Martin Bravenboer)

The posix-file module has been extended with the directory primitives mkdir and rmdir. The new strategy filemode returns the mode of a file. The strategy mkdtemp can be used to created temporary directories, similar to mkstemp.

The module char-io has been extended with the strategy filter-text-file for filtering character sequences from text files. This strategy is especially useful if you need to filter a file in a way that depends on characters in multiple lines of the input. filter-text-file is parameterized with two strategies: a strategy for transforming the current 'chunk' of characters and a strategy to commit the current chunk.

(Eelco Visser)

Various improvements of XTC

In StrategoXT 0.11 AutoXT constructs build-time XTC repositories. Such a build-time XTC repository refers to files in the build tree, instead of the installation directory. The build-time repository is named XTC and is created in the top-level directory of the build tree. Build-time repositories can be used to invoke tools in the build tree at build- or check-time. Thus, build-time XTC repositories can be used to support a 'make check' without having the package installed. Indeed, the StrategoShell and JavaFront have already been changed to use the build-time repositories and now fully support a pre-install check and 'make distcheck'. For StrategoXT a 'make install' is still required before you can do a 'make check'.

From StrategoXT 0.11, XTC tools can be compiled with stand-alone Stratego compiler. Until now, XTC tools could only be compiled in complete AutoXT-based packages. From this release, strc takes a flag --xtc-repo to set the default XTC repository when invoking strc as a full program compiler. This argument has no meaning if strc is instructed to produce only C code. The definition of a default XTC repository with the --xtc-repo flag is optional: if you leave it out, the tool will have no default XTC repository and will use the environment variable XTC_REPOSITORY or a repository that is passed to the tool when it is invoked. In this way, it is now much easier to implement ad-hoc Stratego programs that invoke other tools using XTC. It might also be useful for users that are new to XTC and just want to experiment without setting up an entire package.

(Martin Bravenboer)

Global choice operator

The global choice operator is now much easier to use. The global choice operator ++ was introduced in Stratego 0.7, but to use it, StrategoXT had to be configured with the location of the CPL. If the CPL was available, then it was used for all programs compiled with this installation of StrategoXT. In StrategoXT 0.11 we have moved the choice for the implementation of choice-points to link-time. This is now much more flexible and easy to use: the new version of the CPL for StrategoXT (will be released in a few days) can be installed after the installation of StrategoXT and for each program you can decide if you want to use the implementation of global choices in the CPL. If a program uses the global choice, but it is not linked with the CPL, then linking will fail.

If the CPL is installed and registered in the XTC repository of StrategoXT, then the Stratego compiler accepts a new argument --cpl, which will link the program with the CPL. The argument is not visible if the CPL has not been registered in the XTC repository that is used by strc. The new configuration for the CPL and the new strc option for linking with the CPL, make it very easy to experiment with the global choice implementation.

(Martin Bravenboer)

Many bug-fixes

In StrategoXT 0.11 many bugs have been fixed in tools, the library, and the compiler. Some highlights:

• fixed a bug in the implementation of mkstemp that causes segmentation faults in large programs that use this strategy. In particular, all XTC tools in 0.9.4, 0.9.5 and 0.10 are affected. The segmentation fault occurs during garbage collection of ATerms by the ATerm library.

• Several bugs have been fixed in the new implementation of dynamic rules, which was introduced in StrategoXT 0.10. Also, the performance of scope labeling has been improved by applying the new stateful sets.

• In the backend of the Stratego compiler a bug has been fixed in the generation of code for mutual recursive strategies declared locally in a let. The backend of strc now declares all local functions before defining them.

(Martin Bravenboer and Arthur van Dam)

Detailed List of Issues

The full list of issues closed in this release is available at:

• http://www.stratego-language.org/Stratego/StrategoRelease011Issues

Bugs and Known Problems

See our issue tracking systems for reports about (open) bugs:

• https://catamaran.labs.cs.uu.nl/jira/

For an overview of features planned for future releases:

• Release plan: http://www.stratego-language.org/Stratego/ReleasePlan
• To do: http://www.stratego-language.org/Stratego/ToDo

Please report any problems with installation or bugs in the implementation to our issue tracking system. Please check the existing issues to see if a report about the problem was already submitted.

Contributions

Developments, beta tests, and bug fixes were carried out by

• Martin Bravenboer
• Arthur van Dam
• Valentin David
• Eelco Dolstra
• Dick Eimers
• Kyle Ross
• Clement Vasseur
• Rob Vermaas
• Eelco Visser

Thanks a lot to the people working on the Transformers project. Their immediate feedback to changes in StrategoXT is very useful.