Metaprogramming and the Groovy MOP

Exclusive offer: get 50% off this eBook here
Groovy for Domain-Specific Languages

Groovy for Domain-Specific Languages — Save 50%

Extend and enhance your Java applications with Domain Specific Languages in Groovy

$26.99    $13.50
by Fergal Dearle | May 2010 | Java Open Source

In this article by Fergal Dearle, author of the book Groovy for Domain-Specific Languages, we will cover the inner workings of Groovy's Meta Object Protocol (MOP).

(For more resources on Groovy DSL, see here.)

In a nutshell, the term metaprogramming refers to writing code that can dynamically change its behavior at runtime. A Meta-Object Protocol (MOP) refers to the capabilities in a dynamic language that enable metaprogramming. In Groovy, the MOP consists of four distinct capabilities within the language: reflection, metaclasses, categories, and expandos.

The MOP is at the core of what makes Groovy so useful for defining DSLs. The MOP is what allows us to bend the language in different ways in order to meet our needs, by changing the behavior of classes on the fly. This section will guide you through the capabilities of MOP.

Reflection

To use Java reflection, we first need to access the Class object for any Java object in which are interested through its getClass() method. Using the returned Class object, we can query everything from the list of methods or fields of the class to the modifiers that the class was declared with. Below, we see some of the ways that we can access a Class object in Java and the methods we can use to inspect the class at runtime.

import java.lang.reflect.Field;
import java.lang.reflect.Method;

public class Reflection {
public static void main(String[] args) {
String s = new String();
Class sClazz = s.getClass();
Package _package = sClazz.getPackage();
System.out.println("Package for String class: ");
System.out.println(" " + _package.getName());
Class oClazz = Object.class;
System.out.println("All methods of Object class:");
Method[] methods = oClazz.getMethods();
for(int i = 0;i < methods.length;i++)
System.out.println(" " + methods[i].getName());
try {
Class iClazz = Class.forName("java.lang.Integer");
Field[] fields = iClazz.getDeclaredFields();
System.out.println("All fields of Integer class:");
for(int i = 0; i < fields.length;i++)
System.out.println(" " + fields[i].getName());
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
}

We can access the Class object from an instance by calling its Object.getClass() method. If we don't have an instance of the class to hand, we can get the Class object by using .class after the class name, for example, String.class. Alternatively, we can call the static Class.forName, passing to it a fully-qualified class name.

Class has numerous methods, such as getPackage(), getMethods(), and getDeclaredFields() that allow us to interrogate the Class object for details about the Java class under inspection. The preceding example will output various details about String, Integer, and Double.

Metaprogramming and the Groovy MOP

Groovy Reflection shortcuts

Groovy, as we would expect by now, provides shortcuts that let us reflect classes easily. In Groovy, we can shortcut the getClass() method as a property access .class, so we can access the class object in the same way whether we are using the class name or an instance. We can treat the .class as a String, and print it directly without calling Class.getName(), as follows:

Metaprogramming and the Groovy MOP

The variable greeting is declared with a dynamic type, but has the type java.lang.String after the "Hello" String is assigned to it. Classes are first class objects in Groovy so we can assign String to a variable. When we do this, the object that is assigned is of type java.lang.Class. However, it describes the String class itself, so printing will report java.lang.String.

Groovy also provides shortcuts for accessing packages, methods, fields, and just about all other reflection details that we need from a class. We can access these straight off the class identifier, as follows:

println "Package for String class"
println " " + String.package
println "All methods of Object class:"
Object.methods.each { println " " + it }
println "All fields of Integer class:"
Integer.fields.each { println " " + it }

Incredibly, these six lines of code do all of the same work as the 30 lines in our Java example. If we look at the preceding code, it contains nothing that is more complicated than it needs to be. Referencing String.package to get the Java package of a class is as succinct as you can make it. As usual, String.methods and String.fields return Groovy collections, so we can apply a closure to each element with the each method. What's more, the Groovy version outputs a lot more useful detail about the package, methods, and fields.

Metaprogramming and the Groovy MOP

When using an instance of an object, we can use the same shortcuts through the class field of the instance.

def greeting = "Hello"
assert greeting.class.package == String.package

Expandos

An Expando is a dynamic representation of a typical Groovy bean. Expandos support typical get and set style bean access but in addition to this they will accept gets and sets to arbitrary properties. If we try to access, a non-existing property, the Expando does not mind and instead of causing an exception it will return null. If we set a non-existent property, the Expando will add that property and set the value. In order to create an Expando, we instantiate an object of class groovy.util.Expando.

def customer = new Expando()

assert customer.properties == [:]

assert customer.id == null

assert customer.properties == [:]

customer.id = 1001
customer.firstName = "Fred"
customer.surname = "Flintstone"
customer.street = "1 Rock Road"

assert customer.id == 1001

assert customer.properties == [
id:1001, firstName:'Fred',
surname:'Flintstone', street:'1 Rock Road']
customer.properties.each { println it }

The id field of customer is accessible on the Expando shown in the preceding example even when it does not exist as a property of the bean. Once a property has been set, it can be accessed by using the normal field getter: for example, customer.id. Expandos are a useful extension to normal beans where we need to be able to dump arbitrary properties into a bag and we don't want to write a custom class to do so.

A neat trick with Expandos is what happens when we store a closure in a property. As we would expect, an Expando closure property is accessible in the same way as a normal property. However, because it is a closure we can apply function call syntax to it to invoke the closure. This has the effect of seeming to add a new method on the fly to the Expando.

customer.prettyPrint = {
println "Customer has following properties"
customer.properties.each {
if (it.key != 'prettyPrint')
println " " + it.key + ": " + it.value
}
}

customer.prettyPrint()

Here we appear to be able to add a prettyPrint() method to the customer object, which outputs to the console:

Customer has following properties
surname: Flintstone
street: 1 Rock Road
firstName: Fred
id: 1001

Groovy for Domain-Specific Languages Extend and enhance your Java applications with Domain Specific Languages in Groovy
Published: June 2010
eBook Price: $26.99
Book Price: $44.99
See more
Select your format and quantity:

(For more resources on Groovy DSL see here.)

Categories

Adding a closure to an Expando to give a new method is a useful feature, but what if we need to add methods to an existing class on the fly? Groovy provides another useful feature—Categories—for this purpose. A Category can be added to any class at runtime, by using the use keyword.

We can create Category classes that add methods to an existing class. To create a Category for class, we define a class containing static methods that take an instance of the class that we want to extend as their first parameter. By convention, we name this parameter as self. When the method is invoked, self is set to the object instance that we are extending. The Category can then be applied to any closure by using the use keyword.

class Customer {
int id
String firstName
String surname
String street
String city
}

def fred = new Customer(id:1001,firstName:"Fred", surname:"Flintstone",
street:"1 Rock Road",city:"Bedrock")
def barney = new Customer(id:1002,firstName:"Barney", surname:"Rubble",
street:"2 Rock Road",city:"Bedrock")

def customerList = [ fred, barney]

class CustomerPrinter {
static void prettyPrint(Customer self) {
println "Customer has following properties"
self.properties.each {
if (it.key != 'prettyPrint')
println " " + it.key + ": " + it.value
}
}
}

use (CustomerPrinter) {
for (customer in customerList)
customer.prettyPrint()
}

Java libraries are full of classes that have been declared final. The library designers in their wisdom have decided that the methods they have added are all that we will ever need. Unfortunately, that is almost never the case in practice. Take the Java String class, for example. There are plenty of useful string manipulation features that we might like to have in the String class. Java has added methods progressively to this class over time: for instance, match and split in Java 1.4, with replace and format being added in Java 1.5.

If we needed these style methods before Sun got around to adding them, we could not do it ourselves because of the final modifier. So the only option has been to use classes from add-on libraries such as Commons StringUtils. The Apache Commons Lang component class contains a slew of useful classes that augment the basic capabilities of Java classes, including BooleanUtils, StringUtils, DateUtils, and so on. All of the util class methods are implemented as static, taking String as the first parameter. This is the typical pattern used in Java when we need to mix in extra functionality to an existing class.

import org.apache.commons.lang.StringUtils;

public class StringSplitter {

public static void main(String[] args) {
String [] splits = StringUtils.split(args[0], args[1]);

for (int i = 0; i < splits.length; i++) {
System.out.println("token : " + splits[i]);
}
}
}

Conveniently, this pattern is the same as the one used Groovy categories, which means that the Apache Commons Lang Util classes can all be dropped straight into a use block. So all of these useful utility classes are ready to be used in your Groovy code as Categories.


import org.apache.commons.lang.StringUtils

use (StringUtils) {
"org.apache.commons.lang".split(".").each { println it }
}

Metaclass

In addition to the regular Java Class object that we saw earlier when looking at reflection, each Groovy object also has an associated MetaClass Object. All Groovy classes secretly implement the groovy.lang.GroovyObject interface, which exposes a getMetaClass() method for each object.

public interface GroovyObject {
/**
* Invokes the given method.
*/
Object invokeMethod(String name, Object args);

/**
* Retrieves a property value.
*/
Object getProperty(String propertyName);

/**
* Sets the given property to the new value.
*/
void setProperty(String propertyName, Object newValue);

/**
* Returns the metaclass for a given class.
*/
MetaClass getMetaClass();

/**
* Allows the MetaClass to be replaced with a
* derived implementation.
*/
void setMetaClass(MetaClass metaClass);
}

Pure Java classes used in Groovy do not implement this interface, but they have a MetaClass assigned anyway. This MetaClass is stored in the MetaClass registry. Earlier versions of Groovy required a look-up in the registry to access the MetaClass. Since Groovy 1.5, the MetaClass of any class can be found by accessing its .metaClass property.

class Customer {
int id
String firstName
String surname
String street
String city
}

// Access Groovy meta class
def groovyMeta = Customer.metaClass

// Access Java meta class from 1.5
def javaMeta = String.metaClass

// Access Groovy meta class prior to 1.5
def javaMetaOld = GroovySystem.metaClassRegistry.getMetaClass(String)

Metaclasses are the secret ingredients that make the Groovy language dynamic. The MetaClass maintains all of the metadata about a Groovy class. This includes all of its available methods, fields, and properties. Unlike the Java Class object, the Groovy MetaClass allows fields and methods to be added on the fly. So while the Java class can be considered as describing the compile time behavior of the class, the MetaClass describes its runtime behavior. We cannot change the Class behavior of an object but we can change its MetaClass behavior by adding properties or methods on the fly.

The Groovy runtime maintains a single MetaClass per Groovy class, and these operate in close quarter with the GroovyObject interface. GroovyObject implements a number of methods, which in their default implementations are just facades to the equivalent MetaClass methods. The most important of these to understand is the invokeMethod().

Pretended methods (MetaClass.invokeMethod)

An important distinction between Java and Groovy is that in Groovy a method call never invokes a class method directly. A method invocation on an object is always dispatched in the first place to the GroovyObject.invokeMethod() of the object. In the default case, this is relayed onto the MetaClass.invokeMethod() for the class and the MetaClass is responsible for looking up the actual method. This indirect dispatching is the key to how a lot of Groovy power features work as it allows us to hook ourselves into the dispatching process in interesting ways.

class Customer {
int id
String firstName
String surname
String street
String city
Object invokeMethod(String name, Object args) {
if (name == "prettyPrint") {
println "Customer has following properties"
this.properties.each {
println " " + it.key + ": " + it.value
}
}
}
}

def fred = new Customer(id:1001,firstName:"Fred",
surname:"Flintstone", street:"1 Rock Road",city:"Bedrock")
def barney = new Customer(id:1002,firstName:"Barney",
surname:"Rubble", street:"2 Rock Road",city:"Bedrock")

def customerList = [ fred, barney]


customerList.each { it.prettyPrint() }

Above, we added a Customer.invokeMethod() to the Customer class. This allows us to intercept method invocations and respond to calls to Customer.prettyPrint() even though this method does not exist. Remember how in GroovyMarkup we appeared to be calling methods that did not exist? This is the core of how GroovyMarkup works. The Customer.prettyPrint() method in the previous code snippet is called a pretended method.

Understanding this, delegate, and owner

Like Java, Groovy has a this keyword that refers to the "current" or enclosing Java object. In Java, we don't have any other context that we can execute code in except a class method. In an instance method, this will always refer to the instance itself. In a static method, this has no meaning as the compiler won't allow us to reference this in a static context.

In addition to the instance methods, Groovy has three additional execution contexts to be aware of:

  • Code running directly within a script where the enclosing object is the script.
  • Closure code where the enclosing object is either a script or an instance object.
  • Closure code where the enclosing object is another closure.
  • In addition to the this keyword, Groovy has two other keywords that are referred only in the context of a closure—owner and delegate.

  • The owner keyword refers to the enclosing object, which in the majority of cases is the same as this, the only exception being when a closure is surrounded by another closure.
  • The delegate keyword refers to the enclosing object and is usually the same as owner except that delegate is assignable to another object. Closures relay method invocations that they handle themselves back to their delegate. This is how the methods of an enclosing class become available to be called by the closure as if the closure was also an instance method. We will see later that one of the reasons builders work the way they do is because they are able to assign the delegate of a closure to themselves.

The delegate will initially default to owner, except when we explicitly change the delegate to something else through the Closure.setDelegate method.

The following example illustrates this, owner, and delegate working under various different contexts. This example is necessarily complex, so take the time to read and understand it.

class Clazz {
void method() {
println "Class method this is : " + this.class
}
void methodClosure() {
def methodClosure = {
println "Method Closure this is : " + this.class
assert owner == this
assert delegate == this
}
methodClosure()
}
}

def clazz = new Clazz()

clazz.method()

def closure = { self ->
println "Closure this is : " + this.class
assert this == owner
assert delegate == clazz
def closureClosure = {
println "Closure Closure this is : " + this.class
assert owner == self
assert delegate == self
}
assert closureClosure.delegate == self

closureClosure()
}

closure.delegate = clazz
closure(closure)
clazz.methodClosure()

println this.class

Running the preceding code will output the following text:

Class method this is : class Clazz
Closure this is : class ConsoleScript1
Closure Closure this is : class ConsoleScript1
Method Closure this is : class Clazz
Script this is : class ConsoleScript1

So the rules for resolving this, owner, and delegate in the various contexts are:

  • In a class instance method, this is always the instance object. owner and delegate are not applicable and will be disallowed by the compiler.
  • In a class static method, this, owner, and delegate references will be disallowed by the compiler.
  • In a closure defined within a script, this, owner, and delegate all refer to the Script object unless delegate has been reassigned.
  • In a closure within a method, this and owner refer to the instance object of the enclosing class; as will delegate, unless it has been reassigned to another object.
  • In a script, this is the Script object, and owner and delegate are not applicable.

Summary

In this article, we covered the Meta Object Protocol (MOP) of the Groovy language. We now have an appreciation of what can be achieved by using features in the MOP.


Further resources on this subject:


Groovy for Domain-Specific Languages Extend and enhance your Java applications with Domain Specific Languages in Groovy
Published: June 2010
eBook Price: $26.99
Book Price: $44.99
See more
Select your format and quantity:

About the Author :


Fergal Dearle

Fergal is a seasoned software development professional with 23 years of experience in software product development across a wide variety of technologies. He is currently principal consultant with his own software development consulting company, Dearle Technologies Ltd., engaged in design, development, and architecture for new software products for client companies. In the past Fergal has worked in lead architect and developer roles for Candle Corporation on the OMEGAMON product which is now part of IBMs Tivoli product suite as development manager for the Unix implementations of Lotus 1-2-3. In the early 1990s Fergal lead the team at Glockenspiel that developed CommonView, the first object-oriented UI framework for Microsoft Windows. The team was awarded one of the first ever Jolt Productivity Awards by Dr Dobbs Journal.

Books From Packt

Spring Python 1.1
Spring Python 1.1

NHibernate 2 Beginner's Guide
NHibernate 2 Beginner's Guide

Getting started with Audacity 1.3
Getting started with Audacity 1.3

Pentaho 3.2 Data Integration: Beginner's Guide
Pentaho 3.2 Data Integration: Beginner's Guide

Spring Security 3
Spring Security 3

Apache MyFaces 1.2 Web Application Development
Apache MyFaces 1.2 Web Application Development

GlassFish Security
GlassFish Security

NetBeans Platform 6.9 Developer's Guide
NetBeans Platform 6.9 Developer's Guide

Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software