Chapter 10. Scoping

Usually, the first aspect you need to customize in your DSL implementation in Xtext is the validator. Typically, the second aspect you need to customize is scoping , which is the main mechanism behind visibility and cross-reference resolution. As soon as the DSL needs some constructs for structuring the code, a custom scoping implementation is required. In particular, scoping and typing are often strictly connected and interdependent especially for object-oriented languages. For this reason, in this chapter, we will continue developing the SmallJava DSL introduced in the previous chapter. We will describe both local and global scoping and explain how to customize scoping using SmallJava as a case study.

This chapter will cover the following topics:

• A detailed description of the Xtext mechanisms for local and global scoping

• How to customize the local scoping in an object-oriented language

• How to customize the global scoping with the concepts of packages and imports

• How to provide...

Cross-reference resolution in Xtext

Cross-reference resolution involves several mechanisms. In this section, we introduce the main concepts behind these mechanisms and describe how they interact. We will also write tests to get familiar with cross-reference resolution.

Containments and cross-references

Xtext relies on EMF for the in-memory representation of a parsed program, thus, it is necessary to understand how cross-references are represented in EMF.

In EMF, when a feature is not a datatype (string, integer, and so on), it is a reference, that is, an instance of EReference. A containment reference defines a stronger type of association. The association is stronger regarding the lifecycle. The referenced object is contained in the referring object, called the container. In particular, an object can have only one container. If you delete the container, all its contents are also automatically deleted. For non-containment references, the referenced object is stored somewhere else, for example...

Custom scoping

As soon as a DSL introduces more involved features, such as nested blocks or inheritance relations, the scope provider must be customized according to the semantics of the language.

The Xtext generator generates an Xtend stub class for implementing a custom scope provider. In the SmallJava DSL, it is SmallJavaScopeProvider.

In order to customize the scope provider we redefine the method getScope, and we manually check the reference and the context's class, for example:

override getScope(EObject context, EReference reference) {
  if (reference == SmallJavaPackage.eINSTANCE.SJSymbolRef_Symbol) {
    if (context instanceof ...)
      ...
  } else if (reference ==
        SmallJavaPackage.eINSTANCE.SJMemberSelection_Member) {
      ...
  } else {
    super.getScope(context, reference)
  }
}

Scope for blocks

The default scope provider can be too permissive. For example, in SmallJava, it allows an expression to refer to a variable declaration that is defined after that expression. As...

Global scoping

Xtext has a default mechanism for global scoping that allows you to refer to elements defined in a different file, possibly in a different project of the workspace; in particular, it uses the dependencies of the Eclipse projects. For Java projects, it uses the classpath of the projects. Of course, this mechanism relies on the global index.

Note

Global scoping is implied by the fact that the default scoping mechanism always relies on an outer scope that consists of the visible object descriptions in the index.

With the default configuration in the MWE2, this mechanism for global scoping works out of the box. You can experiment with a project with some SmallJava files. You will see that you can refer to the SmallJava classes defined in another file; content assist works accordingly.

Before proceeding to the use of global scoping, it is worthwhile to learn how to write JUnit tests that concern several input programs.

As hinted in the section The index, when running in a plain Java...

Providing a library

Our implementation of SmallJava does not yet allow to make references to types such as Object, String, Integer, and Boolean We could use these to declare variables initialized with constant expressions. In this section, we show how to create a library with predefined types.

One might be tempted to hardcode these classes/types directly in the grammar, but this is not the best approach. There are many reasons for not doing that; mostly, that the grammar should deal with syntax only. Moreover, if we hardcoded, for example, Object in the grammar, we would only be able to use it as a type, but what if we wanted Object to have some methods? We would not be able to express that in the grammar.

Instead, we will follow the library approach (see also the article Zarnekow 2012-b). Our language implementation will provide a library with some classes, for example, Object, String, and so on, just like Java does. Since Xtext deals with EMF models, this library could consist of any EMF...

Classes of the same package

Just like in Java, SmallJava classes should be able to refer to external classes in the same package without importing the package. However, this is not yet the case in the current implementation. For example, given the following two SmallJava files, they are not able to refer to each other without an explicit import, although they are in the same package:

// first file
package my.pack;

class A {
  B b;
}

// second file
package my.pack;

class B extends A {}

To solve this problem, we need to go back to our custom SmallJavaImportedNamespaceAwareLocalScopeProvider that we implemented in section Default imports. The idea is to customize internalGetImportedNamespaceResolvers so that when the context is a SmallJava program then we add an implicit import of the same package of the current SmallJava program, if the program has a package.

To do that, we create an ImportNormalizer. We have already used ImportNormalizer in section Default imports. This is the implementation...

Dealing with super

As a final feature, we add the mechanism for invoking the implementation of a method in the superclass using the keyword super. Note that super should be used only as the receiver of a member selection expression, that is, it cannot be passed as the argument of a method. Following the practice "loose grammar, strict validation", we do not impose this at the grammar level. Thus, we add the rule for super as a terminal expression:

SJTerminalExpression returns SJExpression:
   ...
   {SJSuper} 'super' | ...

We add a validator rule that checks the correct super usage:

public static val WRONG_SUPER_USAGE =
  ISSUE_CODE_PREFIX + "WrongSuperUsage"

@Check def void checkSuper(SJSuper s) {
  if (s.eContainingFeature !=
        SmallJavaPackage.eINSTANCE.SJMemberSelection_Receiver)
    error("'super' can be used only as member selection receiver",
      null, WRONG_SUPER_USAGE)
}

Thanks to the way we implemented the scope provider, in order to make members of the superclass visible...

What to put in the index?

As explained earlier in this chapter, everything that can be given a name will have a corresponding entry in the index; moreover, by default, each element of the index can be referred through its fully qualified name. However, only the references that use the qualified name syntax can refer to these elements using the index. In SmallJava, only classes can be referred with qualified names.

The index is also used by Xtext to keep track of dependencies among files and to determine when to rebuild other files when a file changes.

Therefore, it makes no sense to index those elements that cannot be referred from other files. In our DSL, this means that it does not make sense to index variables, since they can only be accessed from a method in the containing class. Instead, we leave the methods and their parameters in the index, because we want the other files using a method of an external class to be notified if the method changes name or parameters. Although the presence...

Additional automatic features

Xtext makes use of the index to automatically provide many additional IDE features for your DSL. Some examples are shown in the next screenshot. For example, you can mark occurrences of any named element by toggling the corresponding toolbar button. In the following screenshot, it is the one right on top of the Plug-in Development perspective button, marked with (1). The markers will be evident both in the editor and in its right-hand side ruler. This feature is based on the IResourceDescription instances stored in the index; they contain information about cross-references, possibly to other resources. Furthermore, by right-clicking on an element in the editor, you can choose the menu References, and in the Search view, you can see all the files in your project that reference the selected element. In the following screenshot, we selected the method predicate in the file example.smalljava, and the view shows all its occurrences also in the other file example2...

Providing a project wizard

It is nice to provide the clients of your DSL a project wizard that creates an Eclipse project, sets its source folders, for example, src and src-gen and adds the needed dependencies.

Xtext can generate such a project wizard for you, in the .ui project, if you enable this fragment in the MWE2 file in the StandardLanguage section:

newProjectWizardForEclipse = {
  generate = true
}

After running the MWE2 workflow, you must merge manually the plugin.xml and the plugin.xml_gen.

The generated project wizard will be available in the Eclipse New Project dialog, in the Xtext category.

The above MWE2 fragment will generate the classes for the wizard in the src-gen folder of the .ui project. Moreover, in the src folder, it will generate a stub class, in this example it is SmallJavaNewProjectWizardInitialContents, which you can use to generate some initial contents in the project created by the wizard. In our case, we will generate a simple SmallJava class.

The generated wizard...

Summary

In this chapter we described scoping, which is the main mechanism behind visibility and cross-reference resolution. In particular, scoping and typing are often strictly connected and interdependent especially for object-oriented languages. We described both local and global scoping, and we showed how to customize these mechanisms using the SmallJava DSL as a case study.

In the next chapter we will show how you can create an Eclipse update site for your DSL implementation; this way, other users can easily install it in Eclipse. Moreover, we will describe the features provided by Xtext for building and testing DSLs headlessly, for example. in a continuous integration system.