Chapter 9. Going Global

As we have seen, Hazelcast provides us with a dynamic data and processing backbone to build an application around us. However, recently we have seen the rise of cloud computing. More-and-more applications are designed to work in conjunction with it and have started embracing this new approach to infrastructure. In this chapter, we shall look at:

• Newfound problems with cluster discovery in the cloud world

• Manual specific or nomination node approach

• Cloud provider specific solutions

• Spreading out around the world

Getting setup in the cloud

All of our examples so far have relied on the standard default cluster discovery process. This uses an IP multicast approach, allowing each new node to interrogate the local network to discover any preexisting clusters, and request to join it should it match an expected configuration. One notable feature of public cloud infrastructures is that they tend to share a common network between multiple customers on their virtualized hardware. To avoid any security concerns, distributed networking capabilities such as multicast tend to be blocked; as such we need another mechanism of cluster discovery.

There are two other ways of configuring a cluster:

• Manual seeded unicast configuration

• Discovery supported by Amazon AWS management APIs

Under manual control

In a very similar manner to the way we previously used a thin client to connect to the cluster rather than automatically discovering it, we can nominate a number of nodes that can be used to discover the presence of the wider cluster. This is akin to registering a node's presence with a set of arbiters; then using them to both find existing peers and distributing the appearance of that new node to others in the cluster. There is a higher expectation of availability of the nodes used in this role. Should all of them fail, no new node will be able to join the cluster without adjusting their configuration to address this situation.

So the process works a bit like this. When the 10.0.0.101 node attempts to connect to the cluster that is configured with the knowledge, it should expect peers to exist at 10.0.0.1 and 10.0.0.2. Once it has successfully connected to either of these, it can learn about the rest of the topology of the cluster and establish the required connections...

Discovery – the Amazonian way

Like we mentioned in the previous section, the network fabric supporting the public cloud infrastructure tends to avoid allowing low-level network features that create the possibility of security concerns; this includes broadcast and multicast. However, in the case of Amazon's AWS elastic cloud offering, there are quite a few custom API services that can be used to programmatically control and integrate what has been deployed. To enable non-static but non-multicast discovery, a wrapper to specifically harness EC2's management APIs has been created, and is provided as part of the hazelcast-cloud-2.6.jar file.

Using this, we can configure this capability once this package has been added to the classpath, using the following configuration:

<hazelcast>
  <network>
    <join>
      <aws enabled="true">
        <access-key>ourApiAccessKey</access-key>
        <secret-key>ourApiSecretKey</secret-key>
        <region...

Summary

One of the greatest features of Hazelcast is its distributed scalability. We have now seen that this isn't just limited to our own servers or data center. We can spread out to run our application cluster on public cloud infrastructures, and even on multiple sites around the world, and through the use of an active-active deployment, our application can truly go global.

In the next and final chapter, we shall look at how Hazelcast can collaborate with the existing technology stacks, both at the application and data layers.