Rascal, the new kid on the block

For those of you who have been following our commit messages it is clear: we are furiously working on a new system called "Rascal". But why? And what is it about? In this and subsequent messages I will try to clarify this. Why a new language? The success in solving automatic analysis and transformation problems achieved by a language like ASF+SDF makes it clear that with the increasing scale and complexity of the problems that are being solved, more language support would be welcome. We are still excited about the many good features of ASF+SDF like

• Fully user-definable syntax.
• Conditional rewriting.
• Modular language definitions.
• Fast execution.

In recent years, we have been experimenting with a tiny new language called RScript, it provides a typed relational calculus and is very useful for doing program analysis. Inspired by this, we asked ourselves the question: is it possible to keep the features of ASF+SDF and integrate them with RScript? Would this be a starting point for designing the additonal language features we are looking for? The briefest possible summary is therefore: Rascal = ASF+SDF+RScript This means that we are adding:

• A new type system, including definitions for abstract data types, higher-order functions, and more.
• New data types: lists, sets, relations.
• New control structures, i.e. comprehensions, loops, execptions.

Of course, while designing such a new language we can profit from the extensive experience the ASF+SDF user community has gained while solving analysis and transformation problems. Some of the items on the wish list are:

• Concrete and abstract datatypes.
• More efficient processing of grammar modules.
• More features for resolving ambiguities.
• Seemless integration with a host language like Java.
• Seemless integration with an IDE like Eclipse.

Typical problems to be solved:

• Concise and efficient refactoring of Java programs.
• Finding software bugs.
• Large scale software conversions and transformations.

Now that the initial requirements are becoming clear, you may wonder how Rascal looks like. That will be the topic of a next message.

On the difference between a release and a release candidate

Question: Is ASF+SDF Meta-Environment 2.0.1-RC4 newer or older than ASF+SDF Meta-Environment 2.0.1?

We have added the RC postfix - release candidate - to indicate a preview of a version that is to be released after a number of pre-releases. Version 2.0.1 is thus the final version after the unfinished pre-releases 2.0.1-RC1 through 2.0.1-RC4.

We expect to release small bugfixes on 2.0.1 this year, called 2.0.x with increasing numbers for x. Non-backward compatible changes, or otherwise de-stabilizing changes will lead to 2.y versions. We are working on 3.z in parallel.

ASF+SDF Meta-Environment 2.0.1 released

We have released The ASF+SDF Meta-Environment 2.0.1!

The 2.x release has been coming for a while. The release candidates RC1, RC2 and RC3 have been used in several local projects and in courses for the Master Software Engineering at the UvA. After some bug fixing we are now happy to announce version 2.0.1.

Please download bundled source distributions or binary installers for linux here.

The ASF+SDF Meta-Environment 2.0.1. is a programming language workbench, featuring advanced syntax definition and term rewriting functionality, as well as IDE and visualization features. Among other applications, it is used to implement Domain Specific Languages, Software Analyses and Source-to-source Transformations.

The 2.x release has many features, which we will not list here. Please refer to the extensive documentation.

This is a high-level overview:

• Integrated support for syntax definition in SDF, including
  • modular syntax definitions (any context-free grammar) in SDF
  • an SDF static checker
  • automatic parser generation
  • generated syntax editors with configurable syntax highlighting
  • parse tree and parse forest visualizations
• Integrated support for source analysis and transformation in ASF+SDF, including
  • ASF+SDF, featuring conditional term rewriting with concrete syntax for source code patterns, syntax safe transformation and generation of source code and automated tree traversal.
  • ASF editors, with syntax highlighting.
  • ASF static checker
  • ASF interpreter, featuring 'hifi' source-to-source transformation (i.e. no loss of source code comments)
  • ASF compiler, extremely fast compiled term rewriting systems
  • Automatic IDE integration for language support programmed in ASF+SDF:
    • Syntax highlighting
    • Static checkers
    • Reverse engineering and software visualization
    • SVG drawing
  • Integrated ASF step-wise debugger.

For users of the 1.5.x series of The Meta-Environment, these are the major differences:

• The Meta-Environment integrates editors in tabbed windows, instead of interfacing with Emacs or ViM
• ASF+SDF now features fully structured lexical constructor functions, and is therefore now completely "syntax safe".
• The ASF static checker detects more errors and generates warnings for common programming errors
• There is a lot of online documentation now!

What the future holds?

Development and maintenance on the 2.x series will continue for a while. In parallel, the Meta-Environment team is working on the following:

• "Rascal" - a domain specific language for integrated software analysis and transformation
• Facilitating source code fact extraction using "DeFacto"
• Contributing to Eclipse IMP, a platform for IDE construction
• Port of The Meta-Environment to Eclipse IMP
• SDF ambiguity diagnostics

Using The Meta-Environment for Model Driven Engineering

During the last week of August, the 16th Joint Smalltalk Conference of the European Smalltalk User Group (ESUG) was held in Amsterdam, at CWI. As part of the "meta-modelling" day (Tuesday) I gave a talk on how the ASF+SDF Meta-Environment could be used for Model Driven Engineering (MDE).

During the discussion there seemed to be some interest as to how the Meta-Environment relates to OMeta, developed by Alessandro Warth. In this post I would like to clarify some of the distinctions.

First of all OMeta uses a grammar formalism that is based on parsing expression grammars (PEGs), whereas the Meta-Environment is based on the syntax definition formalism (SDF), which allows the specification of arbitrary context-free grammars (CFGs). As a result, in Ometa, you won't risk writing ambiguous grammars (that is, grammars which allow multiple derivations for the same string). However, you will have to mould your language's grammar until it fits the PEG framework. In SDF you can write your grammar much more declaratively, and use the full power of CFGs to describe your language. As a consequence, there is a risk of ambiguity. SDF provides a number of disambiguation constructs to amend this situation (follow restrictions, priorities, rejects).

Another difference between OMeta and the Meta-Environment is how languages are given semantics. In the Meta-Environment, all computation is done using (source-to-source) transformation. This means that grammars do not have semantic actions that are executed during parsing. Instead, a transformation engine can be provided with a set of rewrite rules that transforms one language to another. Interpretation, compilation, type checking etc. are all implemented as source-to-source transformations. In OMeta, on the other hand, grammar rules are decorated with semantic actions that are executed during parsing. These actions are written in an (general purpose) OMeta host language (which is a parameter of OMeta).

If you look at these distinctions, one could say that OMeta is better at defining (small) interpreters that require a general purpose language and a full-fledged host environment. On the other hand, the Meta-Environment is better fit for constructing compilers and code generation because compilers are essentially (source-to-source) transformations. In addition, the generality of the parsing technology behind the Meta-Environment allows it to be used for other task as well: analysis and transformation of legacy systems. Currently, when implementing a Domain Specific Language (DSL) using the Meta-Environment one has to resort to code generation for integration with the environment (e.g. for dealing with stuff like files, databases, GUI libraries, sockets etc. — anything that cannot be easily expressed as source-to-source transformation...). It would be interesting to see if the parsing power of SDF could be combined with the(much more dynamic and late-bound) interpretation model of OMeta to make the development of DSLs more light-weight.