Installing Coherence 3.5 and Accessing the Data Grid: Part 1

When I first started evaluating Coherence, one of my biggest concerns was how easy it would be to set up and use, especially in a development environment. The whole idea of having to set up a cluster scared me quite a bit, as any other solution I had encountered up to that point that had the word "cluster" in it was extremely difficult and time consuming to configure.

My fear was completely unfounded—getting the Coherence cluster up and running is as easy as starting Tomcat. You can start multiple Coherence nodes on a single physical machine, and they will seamlessly form a cluster. Actually, it is easier than starting Tomcat.

Installing Coherence

In order to install Coherence you need to download the latest release from the Oracle Technology Network (OTN) website. The easiest way to do so is by following the link from the main Coherence page on OTN. At the time of this writing, this page was located at, but that might change. If it does, you can find its new location by searching for 'Oracle Coherence' using your favorite search engine.

In order to download Coherence for evaluation, you will need to have an Oracle Technology Network (OTN) account. If you don't have one, registration is easy and completely free.

Once you are logged in, you will be able to access the Coherence download page, where you will find the download links for all available Coherence releases: one for Java, one for .NET, and one for each of the supported C++ platforms.

You can download any of the Coherence releases you are interested in while you are there, but for the remainder of this article you will only need the first one. The latter two (.NET and C++) are client libraries that allow .NET and C++ applications to access the Coherence data grid.

Coherence ships as a single ZIP archive. Once you unpack it you should see the README.txt file containing the full product name and version number, and a single directory named coherence. Copy the contents of the coherence directory to a location of your choice on your hard drive. The common location on Windows is c:\coherence and on Unix/Linux /opt/coherence, but you are free to put it wherever you want.

The last thing you need to do is to configure the environment variable COHERENCE_HOME to point to the top-level Coherence directory created in the previous step, and you are done.

Coherence is a Java application, so you also need to ensure that you have the Java SDK 1.4.2 or later installed and that JAVA_HOME environment variable is properly set to point to the Java SDK installation directory.

If you are using a JVM other than Sun's, you might need to edit the scripts used in the following section. For example, not all JVMs support the -server option that is used while starting the Coherence nodes, so you might need to remove it.

What's in the box?

The first thing you should do after installing Coherence is become familiar with the structure of the Coherence installation directory.

There are four subdirectories within the Coherence home directory:

  • bin: This contains a number of useful batch files for Windows and shell scripts for Unix/Linux that can be used to start Coherence nodes or to perform various network tests
  • doc: This contains the Coherence API documentation, as well as links to online copies of Release Notes, User Guide, and Frequently Asked Questions documents
  • examples: This contains several basic examples of Coherence functionality
  • lib: This contains JAR files that implement Coherence functionality

Shell scripts on Unix
If you are on a Unix-based system, you will need to add execute permission to the shell scripts in the bin directory by executing the following command:

$ chmod u+x *.sh

Starting up the Coherence cluster

In order to get the Coherence cluster up and running, you need to start one or more Coherence nodes. The Coherence nodes can run on a single physical machine, or on many physical machines that are on the same network. The latter will definitely be the case for a production deployment, but for development purposes you will likely want to limit the cluster to a single desktop or laptop.

The easiest way to start a Coherence node is to run cache-server.cmd batch file on Windows or shell script on Unix. The end result in either case should be similar to the following screenshot:

There is quite a bit of information on this screen, and over time you will become familiar with each section. For now, notice two things:

  • At the very top of the screen, you can see the information about the Coherence version that you are using, as well as the specific edition and the mode that the node is running in. Notice that by default you are using the most powerful, Grid Edition, in development mode.
  • The MasterMemberSet section towards the bottom lists all members of the cluster and provides some useful information about the current and the oldest member of the cluster.

Now that we have a single Coherence node running, let's start another one by running the cache-server script in a different terminal window.

For the most part, the output should be very similar to the previous screen, but if everything has gone according to the plan, the MasterMemberSet section should reflect the fact that the second node has joined the cluster:

ThisMember=Member(Id=2, ...)
OldestMember=Member(Id=1, ...)
ActualMemberSet=MemberSet(Size=2, BitSetCount=2
Member(Id=1, ...)
Member(Id=2, ...)
RecycleSet=MemberSet(Size=0, BitSetCount=0)

You should also see several log messages on the first node's console, letting you know that another node has joined the cluster and that some of the distributed cache partitions were transferred to it.

If you can see these log messages on the first node, as well as two members within the ActualMemberSet on the second node, congratulations—you have a working Coherence cluster.

Troubleshooting cluster start-up

In some cases, a Coherence node will not be able to start or to join the cluster. In general, the reason for this could be all kinds of networking-related issues, but in practice a few issues are responsible for the vast majority of problems.

Multicast issues

By far the most common issue is that multicast is disabled on the machine. By default, Coherence uses multicast for its cluster join protocol, and it will not be able to form the cluster unless it is enabled. You can easily check if multicast is enabled and working properly by running the multicast-test shell script within the bin directory.

If you are unable to start the cluster on a single machine, you can execute the following command from your Coherence home directory:

$ . bin/ –ttl 0

This will limit time-to-live of multicast packets to the local machine and allow you to test multicast in isolation. If everything is working properly, you should see a result similar to the following:

Starting test on ip=Aleks-Mac-Pro.home/,
group=/, ttl=0
Configuring multicast socket...
Starting listener...
Fri Aug 07 13:44:44 EDT 2009: Sent packet 1.
Fri Aug 07 13:44:44 EDT 2009: Received test packet 1 from self
Fri Aug 07 13:44:46 EDT 2009: Sent packet 2.
Fri Aug 07 13:44:46 EDT 2009: Received test packet 2 from self
Fri Aug 07 13:44:48 EDT 2009: Sent packet 3.
Fri Aug 07 13:44:48 EDT 2009: Received test packet 3 from self

If the output is different from the above, it is likely that multicast is not working properly or is disabled on your machine.

This is frequently the result of a firewall or VPN software running, so the first troubleshooting step would be to disable such software and retry. If you determine that was indeed the cause of the problem you have two options. The first, and obvious one, is to turn the offending software off while using Coherence.

However, for various reasons that might not be an acceptable solution, in which case you will need to change the default Coherence behavior, and tell it to use the Well-Known Addresses (WKA) feature instead of multicast for the cluster join protocol.

Doing so on a development machine is very simple—all you need to do is add the following argument to the JAVA_OPTS variable within the cache-server shell script:


With that in place, you should be able to start Coherence nodes even if multicastis disabled.

Localhost and loopback address
On some systems, localhost maps to a loopback address, If that's the case, you will have to specify the actual IP address or host name for the tangosol.coherence.wka configuration parameter. The host name should be preferred, as the IP address can change as you move from network to network, or if your machine leases an IP address from a DHCP server.

As a side note, you can tell whether the WKA or multicast is being used for the cluster join protocol by looking at the section above the MasterMemberSet section when the Coherence node starts.

If multicast is used, you will see something similar to the following:

Group{Address=, Port=35461, TTL=4}

The actual multicast group address and port depend on the Coherence version being used. As a matter of fact, you can even tell the exact version and the build number from the preceding information. In this particular case, I am using Coherence 3.5.1 release, build 461.

This is done in order to prevent accidental joins of cluster members into an existing cluster. For example, you wouldn't want a node in the development environment using newer version of Coherence that you are evaluating to join the existing production cluster, which could easily happen if the multicast group address remained the same.

On the other hand, if you are using WKA, you should see output similar to the following instead:

WKA{Address=, Port=8088}

Using the WKA feature completely disables multicast in a Coherence cluster, and is recommended for most production deployments, primarily due to the fact that many production environments prohibit multicast traffic altogether, and that some network switches do not route multicast traffic properly.

That said, configuring WKA for production clusters is out of the scope of this article, and you should refer to Coherence product manuals for details.

Binding issues

Another issue that sometimes comes up is that one of the ports that Coherence attempts to bind to is already in use and you see a bind exception when attempting to start the node.

By default, Coherence starts the first node on port 8088, and increments port number by one for each subsequent node on the same machine. If for some reason that doesn't work for you, you need to identify a range of available ports for as many nodes as you are planning to start (both UDP and TCP ports with the same numbers must be available), and tell Coherence which port to use for the first node by specifying the tangosol.coherence.localport system property. For example, if you want Coherence to use port 9100 for the first node, you will need to add the following argument to the JAVA_OPTS variable in the cache-server shell script:


Accessing the data grid

Now that you have the Coherence data grid up and running, the next step is to put some data into the grid and check if we can retrieve it. After all, a data grid without any data is not a very useful thing.

You will shortly learn how to use the Coherence APIs to store and retrieve your application data from the grid, but for now let's keep things simple and use the Coherence command-line utility to perform basic grid operations.

Coherence console

Coherence ships with a command-line utility that allows you to manipulate caches within a data grid. Keep your grid nodes running and launch Coherence console by executing the script within the bin directory.

For the most part the resulting screen looks exactly the same as the screens for the previously started Coherence nodes. As a matter of fact, the Coherence console is a Coherence node, and you can see that it has joined the cluster as the third member.

The only significant difference is the prompt at the bottom of the screen:

Map (?):

This is where you can enter various commands to create caches, put values into them and get values from them, and perform many other operations. I will show you how to perform some of those tasks in this section, but keep in mind that the full description of the Coherence console functionality is out of the scope of this article. If you would like to learn more, feel free to consult the Coherence manuals for more details.

Warning: Use at your own risk
The Coherence console is a development tool and should not be used in production. Many things can go wrong, and you can easily loose data not only in the Coherence cache, but in the underlying data store as well if you are not careful.

Now that you can't say you haven't been warned, let's see what we can do with it.

Creating caches

The first thing you need to do in order to store some data in the grid is to create a named cache. This can be easily accomplished from the console prompt, using the cache command:

Map (?): cache countries

This will create a cache with the name countries. You can later use that name to obtain a reference to the cache programmatically.

You will notice that the prompt changed to:

Map (countries):

This tells us that the currently active cache is the one we just created. What this means is that all the commands we execute next will be executed against that cache.

Working with the cache

Now that we have an active cache, let's put some data into it. This is also very simple—all we need to do is to execute the put command as many times as we want:

Map (countries): put USA "United States"
Map (countries): put GBR "United Kingdom"
Map (countries): put RUS Russia
Map (countries): put CHN China
Map (countries): put JPN Japan
Map (countries): put DEU Germany
Map (countries): put FRA France
Map (countries): put ITA Italy
Map (countries): put SRB Serbia

As you can see, the put command takes two arguments—key and value. If either of the two arguments contains a space, it has to be enclosed in quotes in order to be parsed properly.

You will also notice that each put command returns null (which was intentionally omitted earlier to reduce clutter). Actually, each cache put returns the old cache value for the specified key. In this case, we started with an empty cache, which is why null was returned for each command. If you modify an existing value, you will see that the original value is returned:

Map (countries): put USA "United States of America"
United States

We can now use the list command to see a list of all cache entries:

Map (countries): list
JPN = Japan
SRB = Serbia
CHN = China
GBR = United Kingdom
ITA = Italy
FRA = France
USA = United States of America
DEU = Germany
RUS = Russia

Notice that the Coherence cache behaves exactly like the standard HashMap and provides no order guarantees for cache items. This is typically not an issue, but it is good to know.

If you only want to find out how many items are in the cache, you can use the size command:

Map (countries): size

Now, that we have some data in the cache, we can retrieve a single item from it, using the get command:

Map (countries): get SRB

Or we can delete an item using the remove command:

Map (countries): remove SRB

Similar to a put command, remove returns the value that was just deleted from the cache.

You can verify that the item was indeed removed by issuing a list or size command again:

Map (countries): list
JPN = Japan
CHN = China
GBR = United Kingdom
ITA = Italy
FRA = France
USA = United States of America
DEU = Germany
RUS = Russia

Map (countries): size

Now, as much fun as all of this is, it won't take you long to realize that the Coherence console has some limitations. For one, the fact that you can only use primitive types, such as numbers and strings, for cache keys and values, will put fairly severe constraints on what you can accomplish.

In the next section, we will see how you can work with Coherence caches programmatically, and how you can store instances of your custom classes within the data grid. However, before we do that, let me show you why Coherence can have such a positive impact on your application's availability and reliability.

Switch to one of the Coherence nodes you started at the beginning of this article and stop it by pressing Ctrl+C. Now go back to the console window and issue a list command. You will notice that all the data is still there. To prove that I'm not playing any Jedi mind tricks on you, start a brand new Coherence node and after it comes up and joins the cluster, kill the other cache server node that you started earlier. Once again, list all the items in the cache from the console and you will see that no data was lost even though both of the original storage nodes have disappeared. Basically, as long as there are enough storage nodes in the grid to hold all the data, the data will be safe.

Now let's see how we can configure the development environment in order to do something slightly more useful.

Configuring the development environment

Before we can write our first Coherence application, we need to configure the development environment. This involves several steps, such as adding the necessary JAR files to the classpath, configuring the IDE to start Coherence nodes, and specifying configuration parameters that will ensure that each developer has a private Coherence cluster for development and testing.

Most of the examples in this section are specific to IntelliJ IDEA, which is the IDE that I am most familiar with. However, adapting them to Eclipse or any other IDE should be trivial—the principles are the same, even if the mechanics are slightly different.

Referencing necessary JAR files

In a day and age when every library seems to come with so many dependencies that we need build tools like Maven and Ivy to help us put everything together, you might be surprised to find out that Coherence has no external dependencies. For the most part, all you need is a reference to a single JAR file, coherence.jar, which can be found in the COHERENCE_HOME/lib directory.

You should also configure the API documentation by pointing your IDE to the COHERENCE_HOME/doc/api directory. This will allow you to access documentation for Coherence classes within your IDE.

For example, a fully configured project-level library referencing Coherence should look similar to the following in IntelliJ IDEA:

This configuration will be sufficient for the article. That said, even though Coherence does not require any dependencies, it does provide a number of integrations with both Oracle and third-party libraries out of the box.

For example, Coherence provides a distributed implementation of CommonJ Work Manager specification. In order to use it you need to include both commonj.jar and coherence-work.jar into the classpath.

Similarly, if you want to use Hibernate, Oracle TopLink, or JPA integration for persistence, you will need to include coherence-hibernate.jar, coherencetoplink. jar, or coherence-jpa.jar respectively, as well as any JAR files required by the persistence provider of your choice.

Enabling IntelliSense for configuration files

Another thing you will likely want to configure in your IDE is the IntelliSense for Coherence configuration files.

Coherence uses a set of XML-based files for configuration. We will cover these files in more detail a bit later, but at the moment it should suffice to say that all of them have a corresponding DTD file within the coherence.jar file.

Most IDEs can take the advantage of that fact and provide assistance when editing XML documents, based on information in a DTD file. However, more likely than not, you will need to tell your IDE where to find the DTD files.

With IDEA, you need to create external resource definitions that point to the files within the coherence.jar file:

With other IDEs the process might be somewhat different, and you might need to extract these three DTD files from coherence.jar and place them in a location that will be accessible to everyone on the team.

Starting Coherence nodes within the IDE

So far we have used the cache-server shell script to start Coherence nodes, but that is not the most convenient way to do so during development.

For one, you will likely start and restart your development cluster many times a day, and switching between your IDE and a number of terminal windows all the time is not the most productive way to work.

Second, you will need to add your application's classes and any third-party libraries they depend on to the classpath of each Coherence node. While you could copy and edit the cache-server shell script to include necessary classes and JAR files into the classpath (and you will need to do so before deploying to production), this tends to be quite cumbersome during development.

Fortunately, as I mentioned earlier, Coherence is a regular Java application, which makes it quite simple to start within the IDE. If you look at the cache-server shell script, you will see that it really doesn't do much—it sets heap size and classpath, and launches using server JVM.

Doing the same within any IDE should be trivial. The following screenshot shows a sample configuration in IntelliJ IDEA:

As you can see, I have configured the cache server to use server JVM and 64 MB of heap. This is significantly lower than the 512 MB allocated by the cache-server script, but it should be more than enough for development, as you will likely not use the complete data set. In any case, you can adjust heap size to fit your needs.

Now that we have DefaultCacheServer configured, we can launch as many nodes as we need within the IDE by simply clicking on a toolbar button. The following screenshot shows two Coherence nodes running within IntelliJ IDEA:

Creating a private cluster

One thing you will definitely want to do while working with Coherence is to ensure that each developer uses a private cluster on their own machine. The very fact that Coherence clusters so seamlessly can lead to some head scratching during development if this is not done.

Just recently a colleague of mine and I were working on two independent tasks, but we both had a few Coherence nodes running on our laptops. The initial implementation of the code he was working on had a trivial bug that loaded objects of a wrong type into one of the caches. He quickly fixed the bug, restarted the nodes on his machine, and was surprised to find out that the invalid objects were still in the cache, although now in addition to the objects of the correct type.

Fortunately, I was sitting right next to him and it didn't take us long to figure out what was going on and fix the problem. Basically, even though he restarted the nodes on his machine before rerunning the tests, the cluster as a whole, and the data within it survived because there were other nodes running on my laptop.

Even though the impact in this case was minimal, the situation might have been different if I was sitting in another office, so it is extremely important to create developer "sandboxes" when working with Coherence. It is also the reason why you need to pay close attention to the information within the MasterMemberSet section when starting Coherence nodes, and to investigate if the result does not match your expectations.

If you were wondering what that tangosol.coherence.ttl system property I specified when configuring DefaultCacheServer within the IDE meant, it is one of the ways to limit cluster reach. By setting it to zero, you can effectively limit multicast packets to a single machine and ensure that your cluster remains private.

If you are using Well-Known Addresses instead of multicast, you can achieve the same goal by setting the tangosol.coherence.wka system property to localhost.

That said, using system properties is only one way to configure Coherence, and in this case probably not the best. The issue is that you need to remember to specify the property in every run configuration you create, and to make sure that each developer does the same for configurations that are not shared. This creates a lot of opportunities for human error, and chances are that sooner or later someone will forget to specify the necessary system property.

The good news is that you can achieve the same goal using configuration files, which can be committed into the source control repository and shared by the whole team. In addition, settings specified within the configuration files will apply automatically to all run/debug configurations that you create in your IDE.

Configuring Coherence

Coherence uses several configuration files. In this section, we will cover one of the core configuration files, the operational descriptor.

Operational configuration

The operational descriptor is used to configure Coherence runtime parameters such as clustering, communication, services, logging, security, license information, and so on.

Coherence ships with a default operational descriptor, tangosol-coherence.xml, packaged within coherence.jar. If you look more carefully at the console printout of a Coherence node, you will see that the very first line printed is similar to the following:

2009-08-12 13:46:01.983/0.259 Oracle Coherence 3.5.1/461 <Info>
(thread=main, member=n/a): Loaded operational configuration from resource

While you are encouraged to review this file to see what the default configuration looks like, you will typically not modify it directly. Instead, you will use one or more override files to modify only those elements of the operational descriptor that need to be changed for your environment.

All you need to do is create an override file and place it in the Coherence node's classpath before coherence.jar.

There are several override files that you can use:

  • If you want to change the settings globally, regardless of the mode Coherence is running in, you can create a file called tangosol-coherence-override.xml and use it to override any of the elements from a default deployment descriptor. The structure of the files is exactly the same, with the exception that all elements in the override file are optional.
  • If you want your changes to apply only to the development environment, you can create an override file called tangosol-coherence-override-dev.xml. Any settings specified in this file will apply only when the Coherence is running in a development mode, which is the default.
  • If you want your changes to apply only to the production environment, you can create a file called tangosol-coherence-override-prod.xml. In order to have settings from this file applied on your production servers, you should set the system property tangosol.coherence.mode to prod in the script that is used to start Coherence nodes.

You can also create all of these files, and they will be applied in order, from most specific to the default one. For example, if you define both tangosol-coherenceoverride-dev.xml and tangosol-coherence-override.xml, settings from the former will be used to override default settings, and settings from the latter will override both the default settings and the settings specified in the tangosolcoherence-override-dev.xml.

I will cover some of the most important operational parameters in the following sections but thorough coverage of all parameters is not in the scope of this article. If you would like to learn more, you should consult the Coherence product manuals, as well as coherence.dtd and tangosol-coherence.xml files within coherence.jar for detailed information on various configuration elements.

Configuring logging

By default, Coherence prints out log statements to stderr using its own logging subsystem. More likely than not, this is not the desired behavior—you will probably want to use the same logging framework for Coherence that you use for the rest of the application. Fortunately, Coherence supports both the JDK logging framework and Log4J.

In order to configure Coherence to use Log4J, you need to create an operational override file and define a logging-config section with a destination element set to log4j:


You will likely want to use the same logging framework both in development and production, so you should place the code above into tangosol-coherenceoverride.xml file, and make sure that the file is in the classpath.

You will also need to configure Log4J by creating a file similar to the following, and placing it in the classpath as well:

log4j.logger.Coherence=DEBUG, CONSOLE, FILE


Obviously, you can set logging thresholds as you like, but my personal preference is to log errors, warnings, and informational messages to the console, and to write all messages into the log file. This allows me to immediately see on the console if anything is out of the ordinary, and to dig into the log file for details.

Writing log messages from all the nodes into a single log file also has tremendous benefits, as it allows you to see the events across the cluster in one place. This is very helpful when debugging Coherence applications.

In any case, while the exact logger configuration is up to you, there are two important things to notice in the previous sample:

  • You need to configure a logger named Coherence, as that's the name that the internal logging subsystem looks for when delegating to Log4J.
  • Also, because Coherence already formats log messages internally to include all necessary information, you should simply print out the message using the %m%n layout pattern in Log4J.

Of course, you could modify the standard Coherence message pattern and use Log4J for formatting, but using the default one will make the process much easier if you need to submit the log file to Coherence support team.

Configuring a private cluster

Now that you know how to use operational descriptors to configure Coherence, let's see how we can use configuration elements to ensure that each developer's cluster is private.

First off, because we want clusters to be private only in the development environment, you will need to create a tangosol-coherence-override-dev.xml file, and ensure that it is in the classpath at runtime. The next step will depend on whether you are using multicast or WKA for cluster join protocol.

In the former case, you will need to use the following configuration:

<coherence xml-override="/tangosol-coherence-override.xml">

While you only need to specify the time-to-live element to ensure that cluster is private, it is recommended that you specify join-timeout-milliseconds as well, and to set it to a relatively small value (one to three seconds should be more than enough for a private cluster). This will override the default production setting of thirty seconds and ensure that your cluster nodes start quickly.

If you are using WKA, configuration will be slightly different:

<coherence xml-override="/tangosol-coherence-override.xml">
<socket-address id="1">

Notice that we still had to specify the join-timeout-milliseconds parameter. Even though multicast is not used, this setting still controls how long the member will wait for evidence of an existing cluster before starting a new cluster and electing itself as a senior member, so you need to specify it in order to avoid a 30-second delay when starting the first member.

Finally, you should notice xml-override attribute within the root configuration element and its value—this is what ensures that the tangosol-coherenceoverride. xml file we created in the previous section is loaded and used to override the settings in this file and the base operational descriptor, so make sure that you include it if you want the logging configuration we defined earlier to be used.

Now that we have both the development environment and Coherence configured properly, we are ready to write some code.

[ 1 | 2 ]

>> Continue Reading: Installing Coherence 3.5 and Accessing the Data Grid - Part 2

If you have read this article you may be interested to view :

You've been reading an excerpt of:

Oracle Coherence 3.5

Explore Title