Working with Value Extractors and Simplifying Queries in Oracle Coherence 3.5

Exclusive offer: get 50% off this eBook here
Oracle Coherence 3.5

Oracle Coherence 3.5 — Save 50%

Create Internet-scale applications using Oracle’s Coherence high-performance data grid with this book and eBook

$35.99    $18.00
by Aleksandar Seovic Mark Falco Patrick Peralta | April 2010 | Oracle

Usually you can access data in the Coherence cache using identity-based operations, such as get and put. In many cases this will be exactly what you need, but there will also be many other situations where you either won't know an object's identity or you will simply need to look up one or more objects based on attributes other than the identity.

In this article by Aleksandar Seovic, Mark Falco and Patrick Peralta, authors of Oracle Coherence 3.5, we will cover:

  • Built-in filters for querying the Data Grid
  • Implementing a custom value extractor
  • Implementing a FilterBuilder class that will allow us to define the queries in a simpler way

Coherence allows you to do look up one or more objects based on attributes other than the identity by specifying a filter for set-based operations defined by the QueryMap interface.

public interface QueryMap extends Map {
Set keySet(Filter filter);
Set entrySet(Filter filter);
Set entrySet(Filter filter, Comparator comparator);
...
}

As you can see from the previous interface definition, all three methods accept a filter as the first argument, which is an instance of a class implementing a very simple com.tangosol.util.Filter interface:

public interface Filter {
boolean evaluate(Object o);
}

Basically, the Filter interface defines a single method, evaluate, which takes an object to evaluate as an argument and returns true if the specified object satisfies the criteria defined by the filter, or false if it doesn't.

This mechanism is very flexible, as it allows you to filter your cached objects any way you want. For example, it would be quite simple to implement a filter that can be used to retrieve all the account transactions in a specific period:

public class TransactionFilter implements Filter {
private Long m_accountId;
private Date m_from;
private Date m_to;
public TransactionFilter(Long accountId, Date from, Date to) {
m_accountId = accountId;
m_from = from;
m_to = to;
}
public boolean evaluate(Object o) {
Transaction tx = (Transaction) o;
return tx.getId().getAccountId().equals(m_accountId)
&& tx.getTime().compareTo(from) >= 0
&& tx.getTime().compareTo(to) <= 0;
}
}

While the previous sample filter implementation is perfectly valid and will return correct results if executed against the transactions cache, it would be very cumbersome if you had to define every single query criterion in the application by implementing a custom filter class as we did previously.

Fortunately, Coherence provides a number of built-in filters that make custom filter implementation unnecessary in the vast majority of cases.

Built-in filters

Most queries can be expressed in terms of object attributes and standard logical and relational operators, such as AND, OR, equals, less than, greater than, and so on. For example, if we wanted to find all the transactions for an account, it would be much easier if we could just execute the query analogous to the select * from Transactions where account_id = 123 SQL statement than to write a custom filter that checks if the accountId attribute is equal to 123.

The good news is that Coherence has a number of built-in filters that allow us to do exactly that. The following table lists all the filters from the com.tangosol.util.filter package that you can use to construct custom queries:

As you can see, pretty much all of the standard Java logical operators and SQL predicates are covered. This will allow us to construct query expressions as complex as the ones we can define in Java code or the SQL where clause.

The bad news is that there is no query language in Coherence that allows you to specify a query as a string. Instead, you need to create the expression tree for the query programmatically, which can make things a bit tedious.

For example, the where clause of the SQL statement we specified earlier, select * from Transactions where account_id = 123, can be represented by the following Coherence filter definition:

Filter filter = new EqualsFilter("getId.getAccountId", 123);

In this case it is not too bad: we simply create an instance of an EqualsFilter that will extract the value of an accountId attribute from a Transaction.Id instance and compare it with 123. However, if we modify the query to filter transactions by date as well, the filter expression that we need to create becomes slightly more complex:

Filter filter = new AndFilter(
new EqualsFilter("getId.getAccountId", accountId),
new BetweenFilter("getTime", from, to));

If you need to combine several logical expressions, this can quickly get out of hand, so we will look for a way to simplify filter creation shortly. But first, let's talk about something we used in the examples without paying much attention to it—value extractors.

Value extractors

As you can see from the previous examples, a query is typically expressed in terms of object attributes, such as accountId or time, while the evaluate method defined by the Filter interface accepts a whole object that the attributes belong to, such as a Transaction instance.

That implies that we need a generic way to extract attribute values from an object instance—otherwise, there would be no way to define reusable filters, such as the ones in the table earlier that ship with Coherence, and we would be forced to implement a custom filter for each query we need to execute. In order to solve this problem and enable extraction of attribute values from an object, Coherence introduces value extractors.

A value extractor is an object that implements a com.tangosol.util.ValueExtractor interface:

public interface ValueExtractor {
Object extract(Object target);
}

The sole purpose of a value extractor is to extract a derived value from the target object that is passed as an argument to the extract method . The result could be a single attribute value, a combination of multiple attributes (concatenation of first and last name, for example), or in general, a result of some transformation of a target object.

Oracle Coherence 3.5 Create Internet-scale applications using Oracle’s Coherence high-performance data grid with this book and eBook
Published: March 2010
eBook Price: $35.99
Book Price: $59.99
See more
Select your format and quantity:

Reflection extractor

In the vast majority of cases, you will want to extract a value of the single attribute of a target object, in which case you can use the built-in ReflectionExtractor class. The ReflectionExtractor accepts a method name as a constructor argument, invokes the specified method on a target object via reflection, and returns the result of that method invocation.

As a matter of fact, the ReflectionExtractor is used so often that you can simply specify a method name as a string in most places where a value extractor is expected and an instance of a ReflectionExtractor will be created automatically for you, which is what we took advantage of in the previous filter definitions. For example, the filter definition:

Filter filter = new BetweenFilter("getTime", from, to);

Is really just a shorter form of:

Filter filter = new BetweenFilter(
new ReflectionExtractor("getTime"),
from, to);

I will have to admit that as useful as ReflectionExtractor is, I have never liked it much. The main reason for this is that it forces you to spell out a full method name for an attribute, when a property name, as defined by the Java Bean specification should've been enough and would've made the code more readable. This is especially bothersome when accessing a Coherence cluster from a .NET client, in which case the 'get' prefix in front of the property name truly feels unnatural.

Fortunately, it is easy to fix the problem by implementing a similar value extractor that uses introspection to obtain an attribute value:

public class PropertyExtractor
implements ValueExtractor, Serializable {
private final String m_propertyName;
private transient volatile Method m_readMethod;
public PropertyExtractor(String propertyName) {
m_propertyName = propertyName;
}
public Object extract(Object o) {
if (o == null) {
return null;
}
Class targetClass = o.getClass();
try {
if (readMethod ==
null || readMethod.getDeclaringClass() != targetClass) {
PropertyDescriptor pd =
new PropertyDescriptor(propertyName, o.getClass());
readMethod = pd.getReadMethod();
}
return readMethod.invoke(o);
}
catch (Exception e) {
throw new RuntimeException(e);
}
}
}

Now we can use PropertyExtractor instead of ReflectionExtractor in our filter definitions:

Filter filter = new BetweenFilter(
new PropertyExtractor("time"),
from, to);

In this example the difference is not significant and it could even be argued that the PropertyExtractor makes the code harder to read as we have to specify it explicitly, instead of using a filter constructor that takes string as an argument and creates ReflectionExtractor for us. However, in the next section we will implement a helper class that makes filter creation much simpler, and the PropertyExtractor will allow us to make things as simple as they can be.

Expression languages and value extractors
If you are familiar with any of the popular expression languages, such as MVEL, OGNL, or SpEL, you will notice that I could've easily implemented the previous value extractor using one of them. Not only would that allow me to do a simple property extraction, but I would be able to use much more sophisticated expressions for extraction.
Considering that I created SpEL (Spring Expression Language) while working on Spring.NET a few years back, you can imagine that I am a big proponent of their usage. To prove that, I have implemented value extractors for MVEL, OGNL, SpEL, Groovy, and even Java 6 Scripting in Coherence Tools, so you can easily use your favorite EL with Coherence.

Other built-in value extractors

While the ReflectionExtractor is definitely the one that is used most often, there are several other value extractors that ship with Coherence.

IdentityExtractor

The simplest extractor is the IdentityExtractor, which doesn't really extract anything from the target object, but returns the target object itself. This extractor can come in handy when you actually want filters to operate on the cache value itself instead of on one of its attributes, which is typically the case only if the value is of a simple type, such as one of the intrinsic numeric types or a string.

ChainedExtractor and MultiExtractor

There are also two composite value extractors, ChainedExtractor and MultiExtractor. Both of them accept an array of value extractors as a constructor argument, but they use them differently.

The ChainedExtractor executes extractors one by one, using the result of the previous extractor as the target object to evaluate the next extractor against. For example, you can use the ChainedExtractor to extract the accountId attribute from a Transaction instance:

ValueExtractor ex =
new ChainedExtractor(new ValueExtractor[] {
new ReflectionExtractor("getId"),
new ReflectionExtractor("getAccountId") });

This is necessary because the Transaction class does not expose the accountId attribute directly—we need to extract the id attribute from a transaction first, and then extract accountId from a Transaction.Id instance.

Avoiding the need for chaining
Of course, we could've easily avoided the need for chaining if we simply exposed the accountId attribute directly on the Transaction class. Doing that is trivial and makes perfect sense in this case.
However, if I had done that, I'd have to come up with an example of ChainedExtractor usage that is outside of our domain.

When creating a ChainedExtractor you can also use a convenience constructor that will parse a dot-separated string and create an array of ReflectionExtractor instances automatically:

ValueExtractor ex =
new ChainedExtractor("getId.getAccountId");

As a matter of fact, you don't even need to go that far—all built-in filters will automatically create a ChainedExtractor containing an array of ReflectionExtractors from a dot-separated string, which is the feature we relied on earlier when we defined a query that returns transactions for a specific account.

On the other hand, MultiExtractor will execute all extractors against the same target object and return the list of extracted values. While you will rarely use this extractor when querying the cache, it can be very convenient when you want to extract only a subset of an object's attributes during aggregation (which we'll discuss shortly), in order to minimize the amount of data that needs to be transferred across the wire.

PofExtractor

One of the features introduced in Coherence 3.5 is PofExtractor—an extractor that can be used to extract values from the POF-serialized binaries without deserialization. This can provide a huge performance boost and reduced memory footprint for queries that would otherwise have to deserialize every single object in the cache in order to evaluate the filter.

However, you will only see those benefits when working with caches containing large objects. For small objects, the overhead of initializing a structure that is used to keep track of the location of serialized attributes within a binary POF value will likely be higher (both from memory and performance perspective) than the full deserialization of an object.

Implementing a custom value extractor

While the built-in value extractors should be sufficient for most usage scenarios, there might be some situations where implementing a custom one makes sense. We have already implemented one custom value extractor, PropertyExtractor, in order to improve on the built-in ReflectionExtractor and allow ourselves to specify JavaBean property names instead of the full method names, but there are other scenarios when this might be appropriate.

One reason why you might want to implement a custom value extractor is to enable transformation of cache values from their native type to some other type. For example, most applications use UI controls such as drop-downs or list boxes to present a list of possible choices to the user. Let's assume that we need to display a list of countries in a drop-down on the registration screen for new customers.

We already have a cache containing all the countries, so we could easily get all the values from it and send them to the client, which would use them to populate the drop-down. However, the Country class we defined has a number of attributes we don't need in order to populate the drop-down list, such as capital, currency symbol, and currency name—the only attributes we do need are the country code and country name, so by sending any other information to the client we would only be wasting network bandwidth.

As a matter of fact, for most, if not all, drop-downs and list boxes in an application we will need only an identifier that will be returned as a selected value, and a description that should be used for display purposes. That means that we can define a class containing only those two attributes:

public class LookupValue implements Serializable {
private Object m_id;
private String m_description;
public LookupValue(Object id, String description) {
m_id = id;
m_description = description;
}
public Object getId() {
return m_id;
}
public String getDescription() {
return m_description;
}
}

Now that we have a holder class that can be used to represent any lookup value, the remaining question is how we can transform instances of the Country class into the instances of the LookupValue class. The answer is simple—we can write a custom value extractor that will do it for us:

public class LookupValueExtractor
extends AbstractExtractor
implements PortableObject, Serializable {
private ValueExtractor m_idExtractor;
private ValueExtractor m_descriptionExtractor;
public LookupValueExtractor(ValueExtractor idExtractor,
ValueExtractor descriptionExtractor) {
m_idExtractor = idExtractor;
m_descriptionExtractor = descriptionExtractor;
}
public Object extractFromEntry(Map.Entry entry) {
Object id = InvocableMapHelper.extractFromEntry(m_idExtractor,
entry);
String description = (String)
InvocableMapHelper.extractFromEntry(m_descriptionExtractor, entry);
return new LookupValue(id, description);
}
// equals and hashCode omitted for brevity
}

The implementation is actually very simple: we allow users to specify two value extractors, an idExtractor and a descriptionExtractor, that we use to extract the values that are used to create a LookupValue instance. However, one thing deserves clarification.

Instead of simply implementing the ValueExtractor interface, we are extending the AbstractExtractor class and implementing the extractFromEntry method. The reason for this is that we want to be able to extract id and description not only from the entry value, but from the entry key as well.

In order to achieve that, we rely on the InvocableMapHelper class, which provides a utility method that can be used to extract a value from any object that implements Map.Entry interface.

Of course, the LookupValueExtractor is only part of the story—we still need a way to execute this extractor against all the objects in the countries cache and get the collection of extracted lookup values back. We will see what the best way to do that is shortly, but for now let's return to Coherence filters and see how we can make complex queries easier to create.

Simplifying Coherence queries

As you have probably realized by now, Coherence queries can become quite cumbersome to create as the number of attributes used within the query grows, especially if non-default value extractors need to be used.

One, and possibly the best, approach would be to implement a real query language. We could define a grammar for Coherence queries that would be used to parse a SQL-like query string into a parse tree representing a Coherence filter. This would actually be fairly straightforward, as grammar elements would map pretty much directly to the built-in filters provided by Coherence.

However, this would distract us from the main topic of the book and lead us into the discussion of topics such as language grammars and parsers, so implementation of a full-blown Coherence query language is out of the scope of this book.

What we will do instead is implement a FilterBuilder class that will allow us to define the queries in a simpler way. While this approach won't allow us to express all possible queries, it will cover a large number of the most common use cases.

Filter builder

The idea behind the FilterBuilder implementation is that many queries are based on simple attribute comparisons, where multiple attribute comparisons are concatenated using the logical AND, or less often OR operator.

If you review the table of the built-in filter types at the beginning of this article, you will see that Coherence already provides all the core facilities we need to implement this: we have all the common comparison operators, as well as some of the less common ones, and there are AllFilter and AnyFilter, which allow us to create logical AND and OR filters for an array of filters respectively. What we don't have is an easy way to create an array of filters, and that's exactly what the FilterBuilder will help us do.

The goal is to be able to create a filter using code similar to this:

Filter filter = new FilterBuilder()
.equals("id.accountId", 123)
.between("time", from, to)
.build();

This will allow us to define complex queries in a much shorter and significantly more readable way. In order to support the previous syntax, we can implement the FilterBuilder class as follows:

public class FilterBuilder {
private Class defaultExtractorType;
private List<Filter> filters = new ArrayList<Filter>();
// constructors
public FilterBuilder() {
this(PropertyExtractor.class);
}
public FilterBuilder(Class defaultExtractorType) {
this.defaultExtractorType = defaultExtractorType;
}
// public members
public FilterBuilder equals(String propertyName,
Object value) {
return equals(createExtractor(propertyName), value);
}
public FilterBuilder equals(ValueExtractor extractor,
Object value) {
filters.add(new EqualsFilter(extractor, value));
return this;
}
public FilterBuilder notEquals(String propertyName,
Object value) {
return notEquals(createExtractor(propertyName), value);
}
public FilterBuilder notEquals(ValueExtractor extractor,
Object value) {
filters.add(new NotEqualsFilter(extractor, value));
return this;
}
public FilterBuilder greater(String propertyName,
Comparable value) {
return greater(createExtractor(propertyName), value);
}
public FilterBuilder greater(ValueExtractor extractor,
Comparable value) {
filters.add(new GreaterFilter(extractor, value));
return this;
}
// and so on...
}

Basically, we are implementing two overloaded methods for each built-in filter: one that accepts a value extractor as the first argument, and one that accepts a string and creates a value extractor for it.

However, unlike the built-in filters, we do not create an instance of a ReflectionExtractor automatically, but delegate the actual creation of an extractor to the createExtractor factory method:

protected ValueExtractor createExtractor(String propertyName) {
if (propertyName.indexOf('.') >= 0) {
return new ChainedExtractor(
createExtractorArray(propertyName.split(".")));
}
if (propertyName.indexOf(',') >= 0) {
return new MultiExtractor(
createExtractorArray(propertyName.split(",")));
}
return createDefaultExtractor(propertyName);
}

As you can see, if the specified property name is a dot-separated string, we will create an instance of a ChainedExtractor. Similarly, we will create an instance of a MultiExtractor for a comma-separated list of property names.

For all other properties, we will delegate extractor creation to the createDefaultExtractor method:

protected ValueExtractor createDefaultExtractor(String propertyName) {
Constructor ctor = getConstructor(defaultExtractorType);
return (ValueExtractor) ctor.newInstance(propertyName);
}

This allows us to control on a case-by-case basis which value extractor should be used within our filter. In most cases, the default PropertyExtractor should work just fine, but you can easily change the behavior by specifying a different extractor class as a constructor argument:

Filter filter = new FilterBuilder(ReflectionExtractor.class)
.equals("getCustomerId", 123)
.greater("getTotal", 1000.0)
.build();

You can even specify your own custom extractor class—the only requirement is that it implements a constructor that accepts a single string argument.

Summary

In this article, you have learned how to query Coherence caches with built-in filters and learned how to build custom ones. In the next article we will cover Obtaining Query Results and Using Indexes.


If you have read this article you may be interested to view :


Oracle Coherence 3.5 Create Internet-scale applications using Oracle’s Coherence high-performance data grid with this book and eBook
Published: March 2010
eBook Price: $35.99
Book Price: $59.99
See more
Select your format and quantity:

About the Author :


Aleksandar Seovic

Aleksandar Seovic is a founder and Managing Director at S4HC, Inc., where he has worked in the architect role on both .NET and Java projects and has led development effort on a number of engagements for Fortune 500 clients, mostly in the pharmaceutical and financial services industries.

Aleksandar led the implementation of Oracle Coherence for .NET, a client library that allows applications written in any .NET language to access data and services provided by Oracle Coherence data grid, and was one of the key people involved in the design and implementation of Portable Object Format (POF), a platform-independent object serialization format that allows seamless interoperability of Coherence-based Java, .NET, and C++ applications.

Aleksandar frequently speaks about and evangelizes Coherence at industry conferences, Java and .NET user group events, and Coherence SIGs. He can be reached at aleks@s4hc.com.

Mark Falco

Mark Falco is a Consulting Member of Technical Staff at Oracle. He has been part of the Coherence development team since 2005 where he has specialized in the areas of clustered communication protocols as well as the Coherence for C++ object model. Mark holds a B.S. in computer science from Stevens Institute of Technology.

Patrick Peralta

Patrick Peralta is a Senior Software Engineer for Oracle (formerly Tangosol) specializing in Coherence and middleware Java. He wears many hats in Coherence engineering, including development, training, documentation, and support. He has extensive experience in supporting and consulting customers in fields such as retail, hospitality, and finance.
As an active member of the Java developer community he has spoken at user groups and conferences across the US including Spring One and Oracle Open World. Prior to joining Oracle, Patrick was a senior developer at Symantec, working on Java/J2EE based services, web applications, system integrations, and Swing desktop clients. Patrick has a B.S. in computer science from Stetson University in Florida.

He currently maintains a blog on Coherence and other software development topics at http://blackbeanbag.net.

Books From Packt


Oracle 11g R1 / R2 Real Application Clusters Handbook
Oracle 11g R1 / R2 Real Application Clusters Handbook

Oracle Application Express 3.2 – The Essentials and More
Oracle Application Express 3.2 – The Essentials and More

Middleware Management with Oracle Enterprise Manager Grid Control 10g R5
Middleware Management with Oracle Enterprise Manager Grid Control 10g R5

Oracle SQL Developer 2.1
Oracle SQL Developer 2.1

Getting Started With Oracle SOA Suite 11g R1 – A Hands-On Tutorial
Getting Started With Oracle SOA Suite 11g R1 – A Hands-On Tutorial

Oracle 11g Streams Implementer's Guide
Oracle 11g Streams Implementer's Guide

The Business Analyst's Guide to Oracle Hyperion Interactive Reporting 11.1
The Business Analyst's Guide to Oracle Hyperion Interactive Reporting 11.1

Oracle Database 11g – Underground Advice for Database Administrators
Oracle Database 11g – Underground Advice for Database Administrators


No votes yet

Post new comment

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Z
2
X
y
4
7
Enter the code without spaces and pay attention to upper/lower case.
Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software