Enabling Apache Axis2 Clustering

Exclusive offer: get 50% off this eBook here
Apache Axis2 Web Services, 2nd Edition

Apache Axis2 Web Services, 2nd Edition — Save 50%

Create secure, reliable, and easy-to-use web services using Apache Axis2.

$23.99    $12.00
by Afkham Azeez Deepal Jayasinghe | February 2011 | Web Services Open Source

Axis2 has extensive support for clustering. State replication amongst members in the same group as well as cluster management is supported in Axis2. Cluster management refers to managing a single group or several groups of Axis2 nodes. It is also noteworthy that third-party software such as Apache Synapse, which builds on Apache Axis2, also automatically benefits from Axis2 clustering capabilities.

In this article by Deepal Jayasinghe and Afkham Azeez, authors of Apache Axis2 Web Services, 2nd Edition, we will be covering the following areas:

  • Axis2 cluster configuration language
  • Membership management schemes
  • Cluster management

By the end of this article, you will learn the finer details of configuring an Axis2 cluster in a production deployment.

 

Apache Axis2 Web Services, 2nd Edition

Apache Axis2 Web Services, 2nd Edition

Create secure, reliable, and easy-to-use web services using Apache Axis2.

  • Extensive and detailed coverage of the enterprise ready Apache Axis2 Web Services / SOAP / WSDL engine.
  • Attain a more flexible and extensible framework with the world class Axis2 architecture.
  • Learn all about AXIOM - the complete XML processing framework, which you also can use outside Axis2.
  • Covers advanced topics like security, messaging, REST and asynchronous web services.
  • Written by Deepal Jayasinghe, a key architect and developer of the Apache Axis2 Web Service project; and Afkham Azeez, an elected ASF and PMC member.
        Read more about this book      

(For more resources on Apache Axis2, see here.)

Clustering for high availability and scalability is one of the main requirements of any enterprise deployment. This is also true for Apache Axis2. High availability refers to the ability to serve client requests by tolerating failures. Scalability is the ability to serve a large number of clients sending a large number of requests without any degradation to the performance. Many large scale enterprises are adapting to web services as the de facto middleware standard. These enterprises have to process millions of transactions per day, or even more. A large number of clients, both human and computer, connect simultaneously to these systems and initiate transactions. Therefore, the servers hosting the web services for these enterprises have to support that level of performance and concurrency. In addition, almost all the transactions happening in such enterprise deployments are critical to the business of the organization. This imposes another requirement for production-ready web services servers, namely, to maintain very low downtime. It is impossible to support that level of scalability and high availability from a single server, despite how powerful the server hardware or how efficient the server software is. Web services clustering is needed to solve this. It allows you to deploy and manage several instances of identical web services across multiple web services servers running on different server machines. Then we can distribute client requests among these machines using a suitable load balancing system to achieve the required level of availability and scalability.

Setting up a simple Axis2 cluster

Enabling Axis2 clustering is a simple task. Let us look at setting up a simple two node cluster:

  1. Extract the Axis2 distribution into two different directories and change the HTTP and HTTPS ports in the respective axis2.xml files.
  2. Locate the "Clustering" element in the axis2.xml files and set the enable attribute to true. Start the two Axis2 instances using Simple Axis Server. You should see some messages indicating that clustering has been enabled. That is it! Wasn't that extremely simple?
  3. In order to verify that state replication is working, we can deploy a stateful web service on both instances. This web service should set a value in the ConfigurationContext in one operation and try to retrieve that value in another operation. We can call the set value operation on one node, and next call the retrieve operation on the other node. The value set and the value retrieved should be equal.

Next, we will look at the clustering configuration language in detail.

Writing a highly available clusterable web service

In general, you do not have to do anything extra to make your web service clusterable. Any regular web service is clusterable in general. In the case of stateful web services, you need to store the Java serializable replicable properties in the Axis2 ConfigurationContext, ServiceGroupContext, or ServiceContext. Please note that stateful variables you maintain elsewhere will not be replicated. If you have properly configured the Axis2 clustering for state replication, then the Axis2 infrastructure will replicate these properties for you. In the next section, you will be able to look at the details of configuring a cluster for state replication. Let us look at a simple stateful Axis2 web service deployed in the soapsession scope:

public class ClusterableService {
private static final String VALUE = "value";
public void setValue(String value) {
MessageContext.getCurrentMessageContext().getServiceContext();
serviceContext.setProperty(VALUE, value);
}

public String getValue() {
MessageContext.getCurrentMessageContext().getServiceContext();
return (String) serviceContext.getProperty(VALUE);
}
}

You can deploy this service on two Axis2 nodes in a cluster. You can write a client that will call the setValue operation on the first, and then call the getValue operation on the second node. You will be able to see that the value you set in the first node can be retrieved from the second node. What happens is, when you call the setValue operation on the first node, the value is set in the respective ServiceContext, and replicated to the second node. Therefore, when you call getValue on the second node, the replicated value has been properly set in the respective ServiceContext. As you may have already noticed, you do not have to do anything additional to make a web service clusterable. Axis does the state replication transparently. However, if you require control over state replication, Axis2 provides that option as well. Let us rewrite the same web service, while taking control of the state replication:

public class ClusterableService {
private static final String VALUE = "value";
public void setValue(String value) {
MessageContext.getCurrentMessageContext().getServiceContext();
serviceContext.setProperty(VALUE, value);
Replicator.replicate(serviceContext);
}

public String getValue() {
MessageContext.getCurrentMessageContext().getServiceContext();
return (String) serviceContext.getProperty(VALUE);
}
}

Replicator.replicate() will immediately replicate any property changes in the provided Axis2 context. So, how does this setup increase availability? Say, you sent a setValue request to node 1 and node 1 failed soon after replicating that value to the cluster. Now, node 2 will have the originally set value, hence the web service clients can continue unhindered.

Stateless Axis2 Web Services

Stateless Axis2 Web Services give the best performance, as no state replication is necessary for such services. These services can still be deployed on a load balancer-fronted Axis2 cluster to achieve horizontal scalability. Again, no code change or special coding is necessary to deploy such web services on a cluster. Stateless web services may be deployed in a cluster either to achieve failover behavior or scalability.

Setting up a failover cluster

A failover cluster is generally fronted by a load balancer and one or more nodes that are designated as primary nodes, while some other nodes are designated as backup nodes. Such a cluster can be set up with or without high availability. If all the states are replicated from the primaries to the backups, then when a failure occurs, the clients can continue without a hitch. This will ensure high availability. However, this state replication has its overhead. If you are deploying only stateless web services, you can run a setup without any state replication. In a pure failover cluster (that is, without any state replication), if the primary fails, the load balancer will route all subsequent requests to the backup node, but some state may be lost, so the clients will have to handle some degree of that failure. The load balancer can be configured in such a way that all requests are generally routed to the primary node, and a failover node is provided in case the primary fails, as shown in the following figure:

Enabling Apache Axis2 Clustering

Increasing horizontal scalability

As shown in the figure below, to achieve horizontal scalability, an Axis2 cluster will be fronted by a load balancer (depicted by LB in the following figure). The load balancer will spread the load across the Axis2 cluster according to some load balancing algorithm. The round-robin load balancing algorithm is one such popular and simple algorithm, and works well when all hardware and software on the nodes are identical. Generally, a horizontally scalable cluster will maintain its response time and will not degrade performance under increasing load. Throughput will also increase when the load increases in such a setup. Generally, the number of nodes in the cluster is a function of the expected maximum peak load. In such a cluster, all nodes are active.

Enabling Apache Axis2 Clustering

Apache Axis2 Web Services, 2nd Edition Create secure, reliable, and easy-to-use web services using Apache Axis2.
Published: February 2011
eBook Price: $23.99
Book Price: $39.99
See more
Select your format and quantity:
        Read more about this book      

(For more resources on Apache Axis2, see here.)

Setting up and configuring Axis2 clusters in production

This section will describe in detail how Axis clusters are set up and configured in production systems. The Axis cluster configuration is used for configuring a cluster. Understanding of the Axis2 cluster configuration language is very important when it comes to configuring different clustering setups as well as configuring and fine tuning an Axis2 cluster in production. The clustering configuration section in the axis2.xml file has six main parts. They are as follows:

  • Clustering agent
  • Parameters
  • State management
  • Node management
  • Group management
  • Members

Let us take a look at the semantics and usage of each of these configuration sections in detail.

Clustering agent

The clustering agent is responsible for initializing all clustering-related functionality of an Axis2 member or node. Typically, the initialization of a node in a cluster is handled here. It is also responsible for getting a node to join the cluster. In the default Axis2 clustering implementation, which is based on Apache Tribes, we use org.apache.axis2.clustering.tribes.TribesClusteringAgent.

Clustering agent parameters

In order to initialize the clustering agent, we require some parameters. Here are the parameters that are used in the current implementation.

AvoidInitiation

This parameter indicates whether the cluster has to be automatically initialized when the AxisConfiguration is built. If set to true, the initialization will not be done at that stage, and some other party will have to explicitly initialize the cluster. In the case where Axis2 is embedded by third parties, they may need to take control over the cluster initialization process. Hence this parameter has been provided. For example, we can set this parameter to true so that the cluster can be initialized after the server has completely started and the relevant transports have been initialized.

membershipScheme

The membership scheme is used in this setup. The only values supported at the moment are multicast and WKA.

  • multicast: Multicast-based membership management. Membership is automatically discovered using multicasting. In order for this to work, multicasting should be allowed in the network and all members in the cluster should use the same multicast address (as defined by the mcastAddress parameter) and the exact multicast port (as defined by the mcastPort parameter).
  • Well-known address (WKA): Membership management based on well-known address. Membership is discovered with the help of one or more nodes running at a well-known address. New members joining a cluster will first connect to a well-known node, register with the well-known node, and get the membership list from it. When new members join, one of the well-known nodes will notify the others in the group. When a member leaves the cluster, or is deemed to have left the cluster, it will be detected by the Group Membership Service (GMS) using a TCP ping mechanism. WKA-based membership is necessary when multicast-based membership discovery is not possible. For example, on Amazon EC2, multicasting is not allowed and there is no control over the IP address assigned to EC2 instances. Hence, in such a scenario, WKA-based membership discovery tends to be used.

domain

The clustering domain/group: Note that the words 'domain' and 'group' are used synonymously. There will not be any interference between nodes in different groups. Messages received from members outside the group will generally be ignored. However, special messages, such as cluster management messages or membership messages from members outside the group will be allowed.

synchronizeAll

When a web service request is received, and processed, should we update the states of all members in the cluster prior to the response being sent to the client? If the value of this parameter is set to true, the response to the client will be sent only after all the members have been updated. Obviously, this can be time consuming. In some cases, where this overhead may not be acceptable, the value of this parameter should be set to false. The risk in this case will be that all members will not be in the same state when the response is sent to the client. This condition will have to be handled by the user. In summary, this parameter defines whether or not we should synchronize the state of all members in the group before we send the response to the client.

maxRetries

This defines the maximum number of times we need to retry to send a message to a particular member before giving up and considering that node to be faulty. After reaching the maxRetries number of retries, we give up.

mcastAddress

This is the multicast port to be used. This parameter will only be taken into consideration when the membershipScheme is set to multicast.

mcastPort

This is the multicast address to be used. This parameter will only be taken into consideration when the membershipScheme is set to multicast.

mcastFrequency

This defines the frequency of sending membership multicast messages. The value should be specified in milliseconds. This parameter will only be taken into consideration when the membershipScheme is set to multicast.

memberDropTime

This defines the time interval within which, if a member does not respond, the member is deemed to have left the group. The value should be specified in milliseconds.

mcastBindAddress

This defines the IP address of the network interface to which the multicasting has to be bound to. Multicasting would be done using this interface. This parameter will only be taken into consideration when the membershipScheme is set to multicast. Note that this can be different from the localMemberHost parameter. So we can listen for point-to-point messages on the localMemberHost network interface, while listening for multicast messages on the network interface bound to the mcastBindAddress.

localMemberHost

This defines the hostname or IP address of this member. This is the IP address advertised by this member when it joins the group and sends messages. This should be set to a valid value other than localhost or 127.0.0.1. In most cases, it would suffice to set it to the IP address bound to the network interface, which is used for communicating with members in the group.

localMemberPort

This defines the TCP port used by this member. This is the port through which other members will contact this member. The default value for this port is 4000.

preserveMessageOrder

This parameter indicates that message ordering should be preserved. This will be done according to the sender order. The default value for this parameter is true.

atmostOnceMessageSemantics

This parameter indicates that atmost once message processing semantics need to be maintained. It guarantees that a given clustering message will be delivered at most once to the target process or node. The default value for this parameter is true.

properties

There are properties specific to this member. These properties are simply name-value pairs. When a member joins groups, these properties are bound to this member so that other members in the group can detect these properties. These properties can be used in cases where central cluster management needs to be carried out. For example: Let's think of a situation where an Axis2 cluster can be managed by a separate server. Each Axis2 member will provide two member properties, namely, backendServerURL and mgtConsoleURL. The backendServerURL property is used to connect to the backend management web services of a member. Similarly, if this member wishes to expose its own management console, it can do so by providing the "mgtConsoleURL" property. The backendServerURL property is specified using the <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/> entry. Here, ${hostName} signifes the hostName member property and ${httpsPort} denotes the httpsPort member property. hostName and httpsPort are two implicit member properties. httpPort is another implicit member property. Properties that have been previously declared can be used in subsequent property defnitions. For example, we may define a new property as follows: <property name="foo" value="${backendServerURL}/foo"/>

State management

The stateManager element needs to be enabled if you are required to synchronize state across members in a cluster group. An implementation of the org.apache.axis2.clustering.state.StateManager needs to be provided as the value of the class attribute. In the default Apache Tribes-based implementation, we provide the org.apache.axis2.clustering.state.DefaultStateManager class. The user can choose not to replicate certain properties. This is done by providing a property name pattern. Let us look at the following example:

<stateManager class="org.apache.axis2.clustering.state.
DefaultStateManager" enable="true">
<replication>
<defaults>
<exclude name="local_*"/>
<exclude name="LOCAL_*"/>
</defaults>

<context class="org.apache.axis2.context.ConfigurationContext">
<exclude name="UseAsyncOperations"/>
<exclude name="SequencePropertyBeanMap"/>
</context>

<context class="org.apache.axis2.context.ServiceGroupContext">
<exclude name="my.sandesha.*"/>
</context>
<context class="org.apache.axis2.context.ServiceContext">
<exclude name="my.sandesha.*"/>
</context>
</replication>
</stateManager>

In the preceding example, specifying <exclude name="local_*"/> will exclude all properties having the names prefixed with local_ from replication and <exclude name="*_local"/> will exclude all properties having the name suffixed with _local from replication. It follows that <exclude name="*"/> excludes all properties from replication. Excluding all properties from replication may be useful when one needs to replicate only the properties in a certain contexts. For example, if a user wishes to replicate properties only in the serviceContext, under the exclusion entries of org.apache.axis2.context.ConfigurationContext and org.apache.axis2.context.ServiceGroupContext, the <exclude name="*"/> entry should be added.

The exclusion patterns under the defaults element will be applicable to all contexts. Hence <exclude name="local_*"/>, under the defaults element, means that all the properties prefixed with local_ in all the properties having names prefixed with local_ will be excluded from replication. Shown next is a sample stateManager entry.

Node management

The nodeManager element needs to be enabled in order to have node management functionality. An implementation of the org.apache.axis2.clustering.NodeManager interface needs to be provided as the value of the class attribute, as shown next:

<nodeManager class="org.apache.axis2.clustering.management.
DefaultNodeManager"
enable="true"/>

Group management

When this member is deployed as a cluster manager, the groupManagement element needs to be enabled. A group management agent, which is an instance of the org.apache.axis2.clustering.management.GroupManagementAgent interface, needs to be specified for each group (applicationDomain) that is being managed. The example configuration shows how cluster management has been enabled for two groups—group1 and group2:

<groupManagement enable="true">
<applicationDomain name="group1" description="This is the first
group" agent="org.apache.axis2.clustering.management.
DefaultGroupManagementAgent"/>

<applicationDomain name="group2" description="This is the second
group" agent="org.apache.axis2.clustering.management.
DefaultGroupManagementAgent"/>
</groupManagement>

Static members

The members element is used for specifying static or well-known members. The hostName and the primary port of these members need to be specified. The sample member configuration shows two static members with different IP addresses and ports:

<members>
<member>
<hostName>10.100.1.202</hostName>
<port>4000</port>
</member>
<member>
<hostName>10.100.1.154</hostName>
<port>4001</port>
</member>
</members>

Full configuration

Shown next is a typical clustering configuration in Axis2, which uses the Apache-Tribes-based implementation. In this configuration, we have enabled clustering by setting the value of the enable attribute of the clustering element to true. We are using multicast-based membership management scheme. The value of the synchronizeAll parameter has been set to true; hence the responses to web service requests will only be sent after the state changes have been updated on all the nodes. Even though static members have been defined, they will be ignored as we are using a multicast-based scheme. However, if we were using the well-known address-based scheme (WKA), then the static members will be looked up during cluster initialization. Similarly, the parameters pertaining to multicast-based membership such as mcastAddress, mcastPort, and mcastFrequency are applicable only in the case of multicast-based membership management. Group management is only relevant to a node that acts as a cluster manager, hence is disabled in this configuration. We have enabled state replication by enabling the stateManager element.

<clustering class="org.apache.axis2.clustering.tribes.
TribesClusteringAgent" enable="true">
<parameter name="AvoidInitiation">true</parameter>
<parameter name="membershipScheme">multicast</parameter>
<parameter name="domain">apache.axis2.domain</parameter>
<parameter name="synchronizeAll">true</parameter>
<parameter name="maxRetries">10</parameter>
<parameter name="mcastAddress">228.0.0.4</parameter>
<parameter name="mcastPort">45564</parameter>
<parameter name="mcastFrequency">500</parameter>
<parameter name="memberDropTime">3000</parameter>
<parameter name="mcastBindAddress">10.100.1.20</parameter>
<parameter name="localMemberHost">10.100.1.20</parameter>
<parameter name="localMemberPort">4000</parameter>
<parameter name="preserveMessageOrder">true</parameter>
<parameter name="atmostOnceMessageSemantics">true</parameter>
<members>
<member>
<hostName>10.100.1.21<hostName>
<port>4000</port>
</member>
<member>
<hostName>10.100.1.22</hostName>
<port>4001</port>
</member>
</members>
<groupManagement enable="false">
<applicationDomain name="apache.axis2.application.domain"
description="Axis2 group"
agent="org.apache.axis2.clustering.management.
DefaultGroupManagementAgent"/>
</groupManagement>
<nodeManager class="org.apache.axis2.clustering.management.
DefaultNodeManager"
enable="false"/>
<stateManager class="org.apache.axis2.clustering.state.
DefaultStateManager"
enable="true">
<replication>
<defaults>
<exclude name="local_*"/>
<exclude name="LOCAL_*"/>
</defaults>
<context class="org.apache.axis2.context.ConfigurationContext">
<exclude name="UseAsyncOperations"/>
<exclude name="SequencePropertyBeanMap"/>
</context>
<context class="org.apache.axis2.context.ServiceGroupContext">
<exclude name="my.sandesha.*"/>
</context>
<context class="org.apache.axis2.context.ServiceContext">
<exclude name="my.sandesha.*"/>
</context>
</replication>
</stateManager>
</clustering>

Apache Axis2 Web Services, 2nd Edition Create secure, reliable, and easy-to-use web services using Apache Axis2.
Published: February 2011
eBook Price: $23.99
Book Price: $39.99
See more
Select your format and quantity:
        Read more about this book      

(For more resources on Apache Axis2, see here.)

Membership schemes

The term membership scheme refers to the manner in which group membership is managed in a cluster. A cluster consists of several processes, and each of these processes is known as a member. In the context of Axis2, each member is an Axis2 process. The Axis2 clustering implementations support several membership schemes. Next, we will take a look at the three membership schemes supported by Axis2, and how the clusters are configured to support these membership schemes. We will be looking at:

  • Static membership
  • Dynamic membership
  • Hybrid membership

Static membership

In this scheme, only a defined set of members can be in a group. The Group Membership Service (GMS) will detect members joining or leaving the group. External members cannot join the group. Each node may obtain group member details from a central repository or configuration file.

Enabling Apache Axis2 Clustering

In the figure, S1 up to S5 are static members. These static members can join or leave the group at any time. However, these members need to be specified in the axis2.xml file. A new member, N, which is not specified in the axis2.xml file, cannot join the group in general. Later, we will look at hybrid membership, where the new member, N, will be allowed to join the group, if it knows the details of at least one static member. The members configuration section in the axis2.xml of each member will look like this:

<members>
<member>
<hostName>S1</hostName>
<port>4000</port>
</member>
<member>
<hostName>S2</hostName>
<port>4000</port>
</member>
<member>
<hostName>S3</hostName>
<port>4000</port>
</member>
<member>
<hostName>S4</hostName>
<port>4000</port>
</member>
<member>
<hostName>S5</hostName>
<port>4000</port>
</member>
</members>

All the members should belong to the same group or domain. This will be specified using the domain parameter in the axis2.xml clustering configuration section. For example:

<parameter name="domain">Group S<parameter>

Dynamic membership

In this scheme, membership is not predefined. Members can join a group by specifying the appropriate group name, and also leave the group. The Group Management Service (GMS) will detect new members joining or leaving. Group membership information may be obtained from the GMS.

Enabling Apache Axis2 Clustering

The members in a dynamic group need not be specified in the axis2.xml file. Membership will be discovered using multicasting. However, all members should use the same multicast IP address, multicast port, and domain name, in order to belong to the same group. As shown in the previous figure, a new member, N, can join the group by multicasting its details to the multicast socket of the group and specifying that it belongs to the same group, which is Group D.

The multicast address and multicast port are specified using the following parameters in the axis2.xml clustering configuration section:

<parameter name="mcastAddress">228.0.0.4</parameter>
<parameter name="mcastPort">45564</parameter>

The domain is specified as follows:

<parameter name="domain">Group D</parameter>

Hybrid membership

This scheme is also called well-known-addressed- or WKA-based membership. In this scheme, there are a set of well-known members. We can consider these members as belonging to a static group. External members can join this group by notifying one of the well-known members. These external members can then get the current group membership from this well-known member. When new members join the group, well-known members will notify all other members. When members leave the group, the GMS can detect this event.

Enabling Apache Axis2 Clustering

The static members are specified in the axis2.xml file of each member as follows:

<members>
<member>
<hostName>S1</hostName>
<port>4000</port>
</member>
<member>
<hostName>S2</hostName>
<port>4000</port>
</member>
</members>

As shown in the preceding figure, a new member, N, belonging to the domain Group H and having knowledge about at least one static member in the group, can join the group. This new member will first contact one of the static members and learn about the group membership. The static member will inform N about the group membership {S1, S2, D1, D2, D3} and then notify each member in the group about N joining the group, as shown in the following sequence diagram:

Enabling Apache Axis2 Clustering

An alternative way is for N to inform everyone in the group that it has joined, once it receives the membership details from the static member, as shown in the following sequence diagram:

Enabling Apache Axis2 Clustering

Typically, there should be at least two static members to avoid the single point of failure scenario. If there is only a single static member and that member fails, no new members will be able to join the group. However, a dynamic remapping of the IP address of the failed static member can be done. Suppose there is only a single static member; on failure of this static member, another member in the group can remap the IP address of the failed static member to itself, thereby becoming the new sole static member.

Cluster management

Axis2 can run as a cluster manager for managing several groups in a cluster, as depicted in the following figure. If multicast-based membership discovery is used, the Cluster Manager (CM) needs to be running on the same multicast domain and use the same multicast address and port used by members in all the groups.

Enabling Apache Axis2 Clustering

The cluster manager needs to have the following configuration in order to manage the two groups, Group H and Group D, as shown in the previous figure:

<groupManagement enable="true">
<applicationDomain name="GroupH"
description="This is the first group" agent="org.apache.
axis2.clustering.management.DefaultGroupManagementAgent"/>

<applicationDomain name="GroupD"
description="This is the second group" agent="org.apache
axis2.clustering.management.DefaultGroupManagementAgent"/>
</groupManagement>

Highly available load balancing

In order to avoid single points of failure, the worker Axis2 clusters—Group H and Group D—can be fronted by a load balancer cluster, Group LB, as shown in the following figure. One load balancer in Group LB will be the primary load balancer. The Apache Synapse load balancer can be deployed in this manner, and uses the underlying Axis2 clustering mechanism. The web service clients will talk to the endpoints in the primary load balancer and will be unaware of the backend worker node Axis2 clusters.

Enabling Apache Axis2 Clustering

The Axis2 clustering management API

Axis2 clustering is designed in a way that all functionality is abstracted out by a set of interfaces. By implementing these interfaces and providing the implementation classes in the clustering configuration section, any different clustering implementation can be plugged in. The default clustering implementation in Axis2 is based on Apache Tribes (Apache Tribes n.d.), the popular group management framework used by Apache Tomcat (Apache Tomcat n.d.). One may be required to use a group management framework of one's choice, in which case, he/she can implement these interfaces and then plug them using the clustering configuration section in the axis2.xml file. We will be taking a look at the usages of these interfaces later in this article.

The Axis2 clustering APIs address two concerns—state replication and management. State replication refers to the synchronizing of states related to different members in a cluster group. Management can again be divided into two aspects—group management and node management.

The four interfaces that may be used by an Axis2 clustering implementation are:

  • org.apache.axis2.clustering.ClusteringAgent
  • org.apache.axis2.clustering.state.StateManager
  • org.apache.axis2.clustering.management.NodeManager
  • org.apache.axis2.clustering.management.GroupManagementAgent

Let us take a look at these four interfaces in detail.

org.apache.axis2.clustering.ClusteringAgent

This is the main interface in the Axis2 clustering implementation. In order to plug in a new clustering implementation, this interface has to be implemented. It is mandatory to provide an implementation of this interface.

The ClusteringAgent is responsible for initializing all clustering-related functionalities of an Axis2 member or node. Generally, the initialization of a node in a cluster is handled here. It is also responsible for getting this node to join the cluster. This Axis2 node should not process any web services requests until it successfully joins the cluster. Generally, this node will also need to obtain state information and/or configuration information from a neighboring node. State information needs to be obtained, as prior to joining the group this new member needs to be in sync with the other members with respect to the state. It is also possible that configuration changes have taken place. For example, new service deployments or service undeployments may have taken place after nodes in a group were initialized. Hence, it is essential that all configuration changes are kept in sync across a group. This interface is also responsible for properly initializing the org.apache.axis2.clusterng.state.StateManager, org.apache.axis2.clustering.management.NodeManager, and org.apache.axis2.clustering.management.GroupManagementAgent implementations. In the case of a static membership scheme, members are read from the axis2.xml file and added to the ClusteringAgent. Later in the article, we will take a look at different membership schemes supported by Axis2.

In the axis2.xml file, the instance of this interface is specified using the clustering class attribute.

There can also be several parameter elements, which are children of the clustering element in the axis2.xml file. Generally, these parameters will be specific to the ClusteringAgent implementation. In the default Apache Tribes-based implementation, there are several parameters that are required to initialize the Tribes Channel properly.

org.apache.axis2.clustering.state.StateManager

This interface is responsible for handling state replication. This is an optional interface and if state replication is not required, then this interface can be omitted. The word 'state' here means the serializable values stored in the Axis2 context hierarchy which need to be kept in sync across all the members of a cluster group. Property changes (state changes) in the Axis2 context hierarchy in the node that runs this StateManager will be propagated to all other nodes in its group. In the default Axis2 clustering implementation, we have only enabled replication of serializable objects stored in the Apache Axis2 ConfigurationContext, ServiceGroupContext, and ServiceContext. Hence, if a user requires state replication, it is this user's responsibility to store these values in the proper Axis2 contexts. Generally, web services authors do not have to handle state replication, as it is handled by the clustering implementation just before request completion. This is done at the Axis2 MessageReceivers. However, if the developer writes his own MessageReceiver, he will need to call the org.apache.axis2.clustering.state.Replicator#replicate() method. Also, note that at any point, org.apache.axis2.clustering.state.Replicator#replicate() can be called, if the developer wishes to force state replication.

It is not mandatory to have a StateManager in a node. If we are not interested in high availability, we may disable state replication. In such a scenario, in general, the purpose of a clustered deployment is to achieve scalability. In addition, one may also enable clustering without state replication simply to utilize the group communication capabilities of the underlying Group Communication Framework (GCF). In such a case, the purpose of a clustered deployment may also be management of the cluster using the underlying GCF. The implementation of this interface is set by reading the stateManager element in the axis2.xml clustering section.

org.apache.axis2.clustering.management.NodeManager

This interface is responsible for handling management of a particular member. It is not mandatory to have a NodeManager in a node. Node management is generally used for deploying or undeploying services across a cluster group. The implementation of this interface is set by reading the nodeManager element in the axis2.xml file.

org.apache.axis2.clustering.management.GroupManagementAgent

This is the interface through which group management events are notified and messages are sent to members in a particular group. This will only be used when a member is running in group management mode. In group management mode, a member is special and belongs to all groups that it is managing. Hence, any membership changes in the groups it manages will be notified by the underlying group management framework to the group management agent. A group management agent should be specified for each group that has to be managed by the member. In Axis2 cluster management, the cluster manager node needs to define group management agents for each group.

Apache Synapse is capable of dynamic load balancing. Here, new members can join and leave the application group, and this will be reflected in the membership-aware dynamic load balancer. The clustering configuration for the dynamic load balancer needs to be configured with group management to allow group membership discovery.

Summary

In this article, we looked at how Axis2 clusters can be configured. We also described in detail how the Axis2 cluster configuration language can be used for configuring production clusters. Several membership schemes, which can be used in different scenarios, were also introduced. Towards the end of the chapter, we looked at the Axis2 clustering API, which can be used for writing your own clustering mechanism, which may be based on a different group communications framework.


Further resources on this subject:


About the Author :


Afkham Azeez

 Afkham Azeez  is a Member, Project Management Committee (PMC) member & a committer of the Apache Sotware Foundation.  He works as a Senior Software Architect & Senior Manager, Platform as a Service, at WSO2 Inc. His specializations include distributed computing, Cloud computing, SOA & J2EE technologies. He is a long time Open source contributor and is the author of the clustering implementation for Apache Axis2. Azeez holds a Masters degree in Computer Science specializing in Software Architecture & B.Sc. first class honors degree in Computer Science & Engineering from the University of Moratuwa, Sri Lanka.  He is a regular presenter at many Open Source conferences including the Apache Conferences.

Azeez is a petrolhead and an active member of the AutoLanka forum. He enjoys a game of basketball and also loves spending time with his 3 kids.

Deepal Jayasinghe

Deepal Jayasinghe is a Computer Science PhD student at Georgia Institute of Technology and he is affiliated with Center for Experimental Research in Computer Systems.  Prior to joining Georgia Tech, he was a Technical Lead at WSO2 Inc., an open-source software development company that creates middleware platforms for Web services.  Deepal has over 6 years of experiences with SOA and Web services in addition to being a contributing member of Apache Axis2 project since its inception.  He is a key architect and developer of the Apache Axis2 Web service project and has led a number of releases. In addition to Axis2, he has made major contributions to the Apache Synapse, Apache Axiom and Apache XMLSchema projects. Deepal has written more than 30 technical magazine articles, research papers and has delivered speeches in various SOA and Web Services conferences. He is an Apache Member and PMC members at several Apache projects. His expertise lay mainly in distributed computing, SOA and Web services, Cloud computing, and performance analysis.  Deepal holds a B.Sc. Engineering degree from the University of Moratuwa, Sri Lanka.

He is reachable at deepalk@gmail.com.

Books From Packt


Quickstart Apache Axis2
Quickstart Apache Axis2

RESTful Java Web Services
RESTful Java Web Services

Google App Engine Java and GWT Application Development
Google App Engine Java and GWT Application Development

Spring Web Flow 2 Web Development
Spring Web Flow 2 Web Development

Apache MyFaces 1.2 Web Application Development
Apache MyFaces 1.2 Web Application Development

Magento 1.4 Development Cookbook
Magento 1.4 Development Cookbook

Mastering phpMyAdmin 3.3.x for Effective MySQL Management
Mastering phpMyAdmin 3.3.x for Effective MySQL Management

Amazon Web Services: Migrate your .NET Enterprise Application to the Amazon Cloud: RAW
Amazon Web Services: Migrate your .NET Enterprise Application to the Amazon Cloud: RAW


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software