Day 4

On the last day of the first part, you will add pretty-printing and syntactic editor services to your MiniJava editor.

Pretty Print Tables

The Box Text Formatting Language

The Box language is used in Spoofax as a language-independent intermediate representation. The input language dependent parts and the output format dependent parts are connected through this intermediate representation.

Box is a mark-up language to describe the intended layout of text and is used in pretty print tables. A Box expression is constructed by composing sub-boxes using Box operators. These operators specify the relative ordering of boxes. Examples of Box operators are the H and V operator which format boxes horizontally and vertically, respectively.

The exact formatting of each Box operator can be customized using Box options. For example, to control the horizontal layout between boxes the H operator supports the hs space option.

Tip: A detailed description of Box, including a description of all available Box operators, can be found in Merijn de Jong's PhD thesis (Chapter 4).

Anatomy of a Pretty Print Table

Pretty-print tables are used to specify how language constructs have to be pretty-printed. They use Box as language to specify formatting of language constructs. Spoofax generates a pretty-print table from your syntax definition. You can find it in syntax/MiniJava.generated.pp. It contains mappings from constructor names to Box expressions. For example, for the SDF production

Exp "+" Exp -> Exp {cons("Plus")}

A pretty-print entry looks like:

Plus -- H hs=1 [ _1 "+" _2]

Tip: Pretty-print tables are ordered such that pretty-print rules occuring first take preceedence over overlapping pretty-print rules defined later.

To be able to specify formattings for all nested constructs that are allowed in SDF productions, so called selectors are used in pretty-print tables to refer to specific parts of an SDF production and to define a formatting for them. For example, the SDF prodcution

Exp "." ID "(" {Exp ","}* ")" -> Exp {cons("Call")}

contains a nested symbol {Exp ","}*. To specify a formatting for this production, two pretty-print entries can be used:

Call                   -- _1 KW["."] _2 KW["("] _3 KW[")"],
Call.3:iter-star-sep   -- _1 KW[","],

A selector consists of a constructor name followed by a list of number and type tuples. A number selects a particular subtree of a constructor application, the type denotes the type of the selected construct (sequence, optional, separated list etc.). Specifying both the number of a subtree and its type allows unambiguous selection of subtrees and makes pretty-print tables easier to understand.

Below we summarize which selector types are available:

opt

For SDF optionals S?.

iter

For non-empty SDF lists S+.

iter-star

For possible empty SDF lists S*.

iter-sep

For SDF separator lists {S1 S2}+. Observe that the symbol S1 and the separator S2 are ordinary subtrees of the iter-sep construct which can be referred to as first and second subtree, respectively.

iter-star-sep

For SDF separator lists {S1 S2}*. Its symbol S1 and separator S2 can be refered to as first and second subtree.

alt

For SDF alternatives S1 | S2 | S3. According to the SDF syntax, alternatives are binary operators. The pretty-printer flattens all subsequent alternatives. Pretty-print rules can be specified for each alternative individually by specifying the number of each alternative.

seq

For SDF groupings (S1 S2 S3).

Customising Pretty Print Tables

Generated pretty-print tables can easiliy be customised by overruling them in additional pretty-print tables. You can define your own pretty-print table in syntax/MiniJava.pp, which is initially empty. Here you should add your own rules to improve parts of the generated table.

Tip: Spoofax will fallback to the generated table whenever there is no rule in your custom table. Thus, you should define only rules which are different from the generated ones.

Editor Integration

To test your pretty-printer, you need to define a builder and a corresponding strategy. Here you make your first contact to Stratego, Spoofax' language for term rewriting. Add the following rewrite rule to trans/minijava.str:

pretty-print:
  (selected, position, ast, path, project-path) -> (filename, text)
  where
    filename := <guarantee-extension(|"pp.mjv")> path ;
    text     := <pp-minijava-string> selected

This rule follows Spoofax' convention for strategies which implement editor services. On the left-hand site, it matches a tuple of the selected node, its position in the ast, the path of the current file and the project path. On the right-hand site, it instantiates a pair, consisting of a filename and the designated text of the file. Both variables are bound in the where clause. The file name is derived from the path of the current file, while the content of the file is a pretty-printed version of the selected AST node. Therefor, a strategy pp-minijava-string is applied to the node. You can see its definition, which is generated by Spoofax, in lib/editor-common.generated.str:

  pp-minijavasummer-string =  
    <ast2abox(|[<import-term(include/MiniJava.generated.pp.af)>, <import-term(include/MiniJava.pp.af)>])> selected => box ;
    <box2text-string(|100)> box => text

This strategy imports both pretty-printing tables, applies them to turn the AST node into boxes, and then turns these boxes into a string with a maximum line width of 100 characters. Now we can hook our strategy into the editor, making pretty-printing available in the Transform menu. Add the following builder definition to editor/MiniJava-Builders.esv:

builder : "Pretty-print syntax" = pretty-print (openeditor) (realtime) (meta) (source)

This rule defines a builder, its label in the Transform menu, and its implementation strategy pretty-print. Annotations can be used for different variants of builders. (openeditor) ensures that a new editor window is opened for the result. (realtime) requires this editor to be updated whenever the content in the original editor changes. (meta) restricts the builder to be only available to the language engineer, but not to the language user. While you can invoke the builder, people who install your MiniJava plugin cannot. Finally, (source) tells Spoofax to run the builder on an unanalysed (and also not desugared) AST.

Tip: While manual builders need to be invoked from the Transform menu, Spoofax also supports automatic builders which are triggered by saving a file.

Presentational editor services

Presentational editor services such as syntax highlighting, code folding, and the outline view are defined in esv files in the editor directory. You need to specify editor services for your MiniJava editor in the following files:

* syntax highlighting: editor/MiniJava-Colorer.esv, * outline view: editor/MiniJava-Outliner.esv, * code folding: editor/MiniJava-Folding.esv, * code templates: editor/MiniJava-Completion.esv.

These files import files which were generated from your grammar.

Tip: In the generated files you also find some documentation about editor service specifications.

Syntax Highlighting

Default syntax highlighting behaviour is derived based on the literals and lexical syntax in the grammar. The colours used for this derived behavior are specified in the generated MiniJava-Colorer.generated.esv descriptor:

It specifies a color for keywords (alphanumeric literals in the grammar), operators (non-alphanumeric literals), strings (lexical sorts that allow spaces), numbers (lexical numeric sorts), and identifiers (other lexical sorts). The default highlighting works well, but can be customized in the MiniJava-Colorer.esv file. Custom colouring rules for the context-free elements need to specify the sort together with specific colours for some or all of its constructors. Other colouring rules can override the colors for literals and lexical sorts, and can specify background colors, colors for regions of code rather than single productions, and more. You can now try to come up with an own highlighting scheme for MiniJava.

Tip: We will not grade the aesthetics of your highlighting scheme. Feel free to experiment and try different features.

Code Folding and Outline View

Code folding and the outline view are specified by selecting grammar productions that should be made foldable or shown in the outline view. The following picture illustrates some folding rules for another Spoofax project:

Spoofax uses heuristics to automatically derive a generated folding descriptor, based on the logical nesting structure in the syntax of the language. Currently, productions rules that have a lexical identifier and child elements are included in this descriptor. While not perfect, the heuristic provides a good starting point for a new folding definition. Any undesired definitions in the generated file can be disabled by using the (disabled) annotation in the custom specification. The (folded) annotation can be used for constructs that should be folded automatically.

You should enable content folding for class and method declarations as well as for if-, while- and block statements. In the same way, you can then specify the outline outline view showing classes with their fields and methods, including parameters, variable declarations and types.

%GS% Challenge: The grammar from the book inhibits a nice outline of the main class and the main method. Try to change your grammar in a way, that you can specify an outline for the main class and main method. Try to make it as similar as possible to the one for ordinary classes and methods.

Code Completion Templates

Syntactic content completion provides users with completion suggestions based purely on static, syntactic templates. For example

  completion template Statement:
    "while (" <e> ") {\n\t" <b> "\n}"

is a syntactic completion rule for while loops. Such rules are composed of static strings and placeholder expressions. Static strings allow for precise control of the pre- sentation and are enclosed by double quotes. They can use \n for newlines or \t for one indentation level (following the user’s tab/space configuration). Placeholder expressions are indicated by angular brackets. The editor automatically moves the cursor to these expressions once the user selects a completion proposal, allowing the expressions to be filled in as the user continues typing. Define your own completion rules for code completion on language constructs which you consider most useful.

%GS% Challenge: Instead of specifying pretty-print tables and code completion rules separately, you might switch to the Template Language. In this language, you can define the syntax of a language, pretty-printing, and syntactic code completion in a single definition formalism. .

-- GuidoWachsmuth - 03 Oct 2012