Enter Xtext

Xtext is an Eclipse framework for implementing programming languages and DSLs. It lets you implement languages quickly, and most of all, it covers all aspects of a complete language infrastructure, starting from the parser, code generator, or interpreter, up to a complete Eclipse IDE integration (with all the typical IDE features we discussed previously).

The really amazing thing about Xtext is that to start a DSL implementation, it only needs a grammar specification similar to ANTLR; it does not need to annotate the rules with actions to build the AST, since the creation of the AST (and the Java classes to store the AST) is handled automatically by Xtext itself. Starting from this specification, Xtext will automatically generate all the mechanisms sketched previously. It will generate the lexer, the parser, the AST model, the construction of the AST to represent the parsed program, and the Eclipse editor with all the IDE features!

Xtext comes with good and smart default implementations for all these aspects, and indeed most of these defaults will surely fit your needs. However, every single aspect can be customized by the programmer.

With all these features, Xtext is easy to use, it produces a professional result quickly, and it is even fun to use.

Installing Xtext

Xtext is an Eclipse framework, thus it can be installed into your Eclipse installation using the update site as follows:

http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases

Just copy this URL into the dialog you get when you navigate to Help | Install new software... in the textbox Work with and press Enter; after some time (required to contact the update site), you will be presented with lots of possible features to install. The important features to install are Xtend SDK 2.4.2 and Xtext SDK 2.4.2.

Alternatively, an Eclipse distribution for DSL developers based on Xtext is also available from the main Eclipse downloads page, http://www.eclipse.org/downloads, called Eclipse IDE for Java and DSL Developers.

Note

At the time of writing this book, the current version of Xtext was 2.4.2, and this is the version used in this book.

Let's try Xtext

Hopefully, by now you should be eager of seeing for yourself what Xtext can do! In this section we will briefly present the steps to write your first Xtext project and see what you get. Do not worry if you have no clue about most of the things you will see in this demo; they will be explained in the coming chapters.

1. Start Eclipse and navigate to File | New | Project...; in the dialog, navigate to the Xtext category and select Xtext Project.

   

2. In the next dialog you can leave all the defaults, but uncheck the option Create SDK feature project (we will use the Create SDK feature project feature later in Chapter 11, Building and Releasing).

   

   The wizard will create three projects and will open the file MyDsl.xtext, which is the grammar definition of the new DSL we are about to implement. You do not need to understand all the details of this file's contents for the moment. But if you understood how the grammar definitions work from the examples in the previous sections, you might have an idea of what this DSL does. It accepts lines starting with the keyword Hello followed by an identifier, then followed by !.

   

3. Now it is time to start the first Xtext generation, so navigate to the file MyDsl.xtext in the project org.xtext.example.mydsl, right-click on it, and navigate to Run As | Generate Xtext Artifacts. The output of the generation will be shown in the Console view. You will note that (only for the first invocation) you will be prompted with a question in the console:

   *ATTENTION* It is recommended to use the ANTLR 3 parser generator (BSD licence - http://www.antlr.org/license.html).Do you agree to download it (size 1MB) from 'http://download.itemis.com/antlr-generator-3.2.0.jar'? (type 'y' or 'n' and hit enter)

4. You should type y and press Enter so that this JAR will be downloaded and stored in your project once and for all (this file cannot be delivered together with Xtext installation: due to license problems, it is not compatible with the Eclipse Public License). Wait for that file to be downloaded, and once you read Done in the console, the code generation phase is finished, and you will note that the three projects now contain much more code. Of course, you will have to wait for Eclipse to build the projects.

5. Your DSL implementation is now ready to be tested! Since what the wizard created for you are Eclipse plug-in projects, you need to start a new Eclipse instance to see your implementation in action. Before you start the new Eclipse instance, you must make sure that the launch configuration has enough PermGen size, otherwise you will experience " out of memory " errors. You need to specify this VM argument in your launch configuration: -XX:MaxPermSize=256m; alternatively, you can simply use the launch configuration that Xtext created for you in your project org.xtext.example.mydsl, so right-click on that project and navigate to Run As | Run Configurations...; in the dialog, you can see Launch Runtime Eclipse under Eclipse Application; select that and click on Run.

   

6. A new Eclipse instance will be run and a new workbench will appear. In this instance, your DSL implementation is available; so let's create a new General project (call it, for instance, sample). Inside this project create a new file; the name of the file is not important, but the file extension must be mydsl (remember that this was the extension we chose in the Xtext new project wizard). As soon as the file is created it will also be opened in a text editor and you will be asked to add the Xtext nature to your project. You should accept that to make your DSL editor work correctly in Eclipse.

7. Now try all the things that Xtext created for you! The editor features syntax highlighting (you can see that by default Xtext DSLs are already set up to deal with Java-like comments like // and /* */), immediate error feedback (with error markers only in the relevant parts of the file), outline view (which is automatically synchronized with the elements in the text editor), and code completion. All of these features automatically generated starting from a grammar specification file.

   

This short demo should have convinced you about the powerful features of Xtext (implementing the same features manually would require a huge amount of work). The result of the code generated by Xtext is so close to what Eclipse provides you for Java that your DSLs implemented in Xtext will be of high quality and will provide the users with all the IDE benefits.