An SOA Suite cluster can process more composite instances by spreading the load across multiple machines, providing greater capacity. It also provides resiliency by allowing composites to continue to execute on remaining machines in the cluster in the event of a machine failing.
Using a cluster provides the following benefits:
Greater capacity
Greater resiliency
Oracle provides a comprehensive guide to creating an SOA Suite cluster called the Enterprise Deployment Guide (EDG). Rather than duplicating the guide, this chapter will provide recipes that enhance the guide and elaborate on the steps required.
SOA Suite is normally deployed on a WebLogic application server and in this chapter we will use WebLogic nomenclature to describe SOA Suite entities:
The following figure shows the target SOA Suite deployment architecture for a three-machine SOA Suite cluster:
At the heart of the cluster are three physical machines running SOA Suite. They make use of a highly available database and a shared filesystem. HTTP access to the machines is provided through two web server machines which run HTTP servers. Finally, a load balancer is used to distribute the load across the web servers. See the Preparing the network recipe for more details on the load balancer.
This architecture may be scaled by adding additional SOA machines. For most environments, the two web servers are only required for resilience. They can generally handle all but the highest client loads. Each web server machine will distribute requests to all machines in the SOA Suite cluster; there is no affinity between a particular HTTP machine and a particular SOA machine.
Note that each set of machines forms a layer that may be separated by using firewalls to improve security. If this is not required then the web servers may run on the SOA machines, removing the need for the web machines.
The database is required by SOA Suite to store composite instance state and configuration information. The shared filesystem is required by WebLogic to store shared configuration files, transaction logs, and queues. A highly available database, such as Oracle Real Application Clusters (RAC), is recommended.
An SOA Suite cluster is typically made up of several WebLogic clusters; a Web Services Manager cluster, an SOA cluster, and a BAM cluster. These clusters may share hardware, as shown in the following figure:
Note
An SOA Suite Cluster contains not just the core SOA Suite functionality of BPEL, Mediator, Rules, and Human Workflow but also Web Services Manager and BAM. The Web Services Manager and BAM have their own WebLogic clusters which run alongside the core SOA cluster. Hence, the SOA Suite cluster has within it three WebLogic clusters, one of which, the SOA cluster, has the core SOA Suite functionality.
In our three-machine cluster we have chosen to have an SOA Cluster with three managed servers, a BAM cluster with two managed servers, and a WSM cluster with two managed servers. We can adjust the number of managed servers in a cluster to accommodate different numbers of physical machines. Note that in our example each machine hosts at least two servers, but the machines may host more or fewer servers depending on their capacity (CPU, memory, and network).
The Node Manager is responsible for monitoring the state of the managed servers and restarting them in the event of failure, either on the original machine if possible, or in the event of machine failure on another machine in the cluster.
Before starting to build an SOA Suite cluster it is important to ensure that you have all the required configuration information and the environment is prepared correctly. Time spent doing this properly will save a lot of heartache and delay later.
Make sure you know how big a cluster you wish to build in terms of number of managed servers.
Create a drawing of the topology.
Before starting, make sure you understand the topology of the cluster you wish to build and draw a picture of it either on a whiteboard or using a drawing tool such as Visio.
Identify physical machines for web servers.
Get the names of the physical servers that will be running the web servers and fill them in on a list similar to the one shown as follows. Also identify the port number that the web server will be running on and the protocol that it will be using.
Identify physical machines for WebLogic servers.
Get the names of the physical servers that will be running SOA Suite and fill them in on a worksheet similar to the one shown as follows. Use the WebLogic Servers column to identify which servers will normally run on the physical machine, ignore fail over for now.
Identify port numbers for WebLogic servers.
Create a table identifying the port number to be used for each type of server in your cluster similar to the one as follows:
The previous table shows the suggested values from the EDG.
Identify floating IP addresses for WebLogic servers.
Create a table identifying the virtual, or floating, IP addresses to be used for each server that requires whole server migration similar to the one shown as follows:
The topology drawing and the list of physical machines for WebLogic and web servers will help the team provisioning the hardware to understand what physical or virtual machines must be provided and how they are connected.
The list of web server machine names, WebLogic server machine names, and port numbers can be provided to the network team who will use it in conjunction with the topology diagram to configure the firewall. They will also use it to create server pools in the load balancer for each protocol type and then add the web servers to the newly created pools.
The list of WebLogic server floating IP addresses can be used by the network team to allocate suitable IP addresses and, when coupled with the port numbers for WebLogic servers can be used to identify ports that must be opened in any firewalls between the web servers, and the WebLogic servers.
When running the domain wizard, the names of the managed servers and their associated floating IP addresses and port numbers will be required.
The Admin server is treated differently from managed servers because the Admin server does not share its shared filesystem with other WebLogic instances. A failover script can be used to unmount and mount shared storage for the Admin server as part of the failover task.
If the web servers will be running multiple protocols then you can use multiple lines for a single web server machine.
|
Web server port number |
Protocol | |
|---|---|---|
|
WebMachine1 |
|
HTTP |
|
WebMachine1 |
|
HTTPS |
|
WebMachine2 |
|
HTTP |
|
WebMachine2 |
|
HTTPS |
Tip
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www. packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. The code package for the book includes an Excel workbook SOA-Cookbook-Cluster-Workbook.xls with worksheets containing templates for the tables used in this recipe.
This recipe will identify the steps required to prepare the operating system for installation and configuration of an SOA Suite cluster. This recipe uses Linux as the operating system, the actual commands required vary between operating systems. These steps are required because SOA Suite high availability makes use of whole server migration.
Certain tasks mentioned in this recipe must be performed by a system administrator. As the installer of the SOA Suite does not necessarily have system administrator privileges for the operating system, it is a good idea to get all the tasks that require administrator privileges completed before starting the installation and configuration of the SOA Suite.
Grant sudo privileges to the Oracle user.
As root on each machine that will be hosting WebLogic servers, run the
visudocommand and add the following lines to the end of the file:# Node Manager Grants oracle ALL=NOPASSWD: /sbin/ifconfig,/sbin/arping
oracleshould be replaced with the user you will be running SOA Suite under.Set up a shared mount point for use by the domain.
Write a file to the mount point from each machine and ensure that the files are readable and writable from all the other machines.
Set up a shared mount point for use by the Admin server.
Write a file to the Admin server mount point from each machine that will run the Admin server and ensure that the files are readable and writable from all the other machines that can run the Admin server.
Capture the mount points that will be used by the Admin server and by all other managed servers in a worksheet similar to the one as shown next:
Servers
Mount point
Admin server
/share/aserverManaged servers
/share/cluster
The granting of sudo privileges is used to allow the non-root user executing the WebLogic NodeManager to execute a limited subset of commands. These commands are used by the node manager to assign and register the virtual IP addresses used by the SOA managed servers and the BAM server. These commands are also used to unregister and release the virtual IP addresses.
Floating or virtual IP addresses are used by the SOA, Admin, and BAM servers to allow these servers to have the same IP address when they migrate from one machine to another.
When we run the configuration wizard to create our cluster, we will need to have a shared file location in which to create the domain. The Admin server shared file location is used to hold the domain configuration for the Admin server. The domain wide shared mount point is used to hold managed server domain directories as well as configuration plans. The EDG currently recommends using a shared filesystem for both server transaction logs (TLogs) and for JMS Queue storage. When migrating an Admin server, the shared storage for the Admin Server can be unmounted from the original machine and mounted on the new machine hosting the Admin server.
The shared filesystem may use a number of technologies, including SMB, NFS, NAS, and SAN technologies. The key is that the filesystem supports shared access.
It is possible to place the SOA Suite software onto shared storage. When doing this, Oracle recommends having at least two copies of the installed software to make it easier to patch and to reduce the risk of corruption.
Tip
Shared software installations
Certain shared storage configurations can cause very slow startup when the software is placed in shared storage because the classloader reads just a small amount from the disk for each class loaded. When the software disk is shared, it may not be able to cache reads and so classloading can introduce a lot of latency into the startup process. This can more than double the startup time for WebLogic servers. Once started and in the RUNNING state then they will not suffer a performance impact from having shared software storage.
Instead of using shared storage for TLogs and Queue stores, it is possible to place these in the database. This imposes a small performance overhead but simplifies fail over to a DR site because all the transaction logs and queues are replicated as part of the database replication to the DR site.
The Enterprise Deployment Guide, Chapter 14, Configuring Server Migration for an Enterprise Deployment, Section 14.6, Setting Environment and Superuser Privileges for the wlsifconfig.sh script (http://docs.oracle.com/cd/E23943_01/core.1111/e12036/server_migration.htm)
The Enterprise Deployment Guide, Chapter 4, Preparing the File System for an Enterprise Deployment, Section 4.3, About Recommended Locations for the Different Directories (http://docs.oracle.com/cd/E23943_01/core.1111/e12036/file_sys.htm)
An SOA Suite cluster requires specific configuration which is covered in this section.
The database preparation requires SYSDBA privileges. As the installer of the SOA Suite does not necessarily have SYSDBA privileges for the database, it is wise to get all the tasks that require SYSDBA privileges completed before starting the installation and configuration of the SOA Suite.
Check the character set requirements.
Have DBA verify that the database character set is
AL32UTF8.SQL> select value from nls_database_parameters where PARAMETER='NLS_CHARACTERSET';
If it is not
AL32UTF8then the DBA needs to change the character set (easy if the current character set is a strict subset ofAL32UTF8, hard if it is not) or create a new database with the correct character set.Check process requirements are satisfied.
Have the DBA verify that there are sufficient processes, at least 300 for SOA and 400 if using BAM with SOA.
SQL> show parameter PROCESSES
If necessary, increase the number of processes and restart the database.
SQL> Alter system set PROCESSES=400 scope=spfile;
If the database is an RAC database, have the DBA create database services for the SOA components. Additional services can be created for WSM and BAM if desired.
Create database services for SOA and WSM.
SQL> execute DBMS_SERVICE.CREATE_SERVICE(SERVICE_NAME => 'soacluster.cookbook', NETWORK_NAME => 'soacluster.cookbook');
Where
'soacluster.cookbook'is the name of the service you want to create.Register the service with database instances in the RAC cluster.
> srvctl add service –d orcl –s soacluster –r orcl1,orcl2
Where
orclis your database name andorcl1andorcl2are instances in your RAC cluster.Start the service.
> srvctl start service –d orcl –s soacluster
Create the SOA repository.
Have the DBA run the Repository Creation Utility (RCU), it requires SYSDBA privileges. After completion, verify that you can connect to the
<prefix>_soainfraschema.SQL> connect dev_soainfra/welcome1
Configure SOA schema for transaction manager recovery.
With SYSDBA privileges, grant visibility on pending transactions to the
soainfraschema:SQL> grant select on sys.dba_pending_transactions to dev_soainfra;
Grant ability to commit or rollback in doubt transactions to
soainfraschema:SQL> grant force any transaction to dev_soainfra;
With SYSDBA privileges, create a leasing tablespace:
SQL> create tablespace leasing logging datafile '/home/oracle/app/oracle/oradata/orcl/leasing.dbf' size 32m autoextend on next 32m maxsize 2048m extent management local;
Where
/home/oracle/app/oracle/oradata/orclis the location of the database data files (<ORACLE_BASE>/oradata/<DB_NAME>).Create a leasing user with privileges to create tables and connect to the database:
SQL> grant create table, create session to leasing identified by welcome1;
Where
welcome1is a password of your choosing.Set the leasing user to use the leasing tablespace and allow him/her unlimited size in the tablespace:
SQL> alter user leasing default tablespace leasing; SQL> alter user leasing quota unlimited on leasing;
Get a copy of the
leasing.ddlscript found in"<WL_HOME>/server/db/oracle/920"where<WL_HOME>is the location of the WebLogic server directory. This may have to wait until you have installed the WebLogic server software.Run the
leasing.ddlscript as the leasing user:SQL> connect leasing/welcome1 SQL> @leasing.ddl
This assumes that you are running SQL*Plus from the directory where
leasing.ddlis located. Note that if you get errors about unknown commands and an error about table or view does not exist, these can be safely ignored.SQL> @leasing.ddl SP2-0734: unknown command beginning "WebLogic S..." - rest of line ignored. SP2-0734: unknown command beginning "Copyright ..." - rest of line ignored. DROP TABLE ACTIVE * ERROR at line 1: ORA-00942: table or view does not exist
Setting the database to AL32UTF character set is important if you will be processing non-Latin characters through the SOA Suite. Failure to set this character set can result in mis-representation of non-Latin character sets such as Chinese, Arabic, and Korean.
The leasing table is used by WebLogic to track which machines are running which migratable managed servers. A migratable managed server is configured to be able to migrate from one machine to another in the event of machine or other failure. The SOA servers and the BAM servers should be configured to do this.
There are a number of tasks that require network configuration to be completed. As the installer of the SOA Suite does not necessarily have network administrator privileges, it is a good idea to get all the tasks that require administrator privileges completed before starting the installation and configuration of the SOA Suite.
The following figure shows the hostnames associated with our cluster. Note that hostnames associated with floating IP addresses (may migrate between machines) are given in italics and all the names on the load balancer refer to virtual IP addresses.
Collect managed server hostnames and IP addresses.
The Admin server, each SOA managed server, and one of the BAM managed servers will require a unique hostname and IP address that must be routable across the cluster. These IP addresses are separate from the IP addresses of the machines hosting the managed servers. Enter the server type (Admin, SOA, or BAM) and WebLogic server name in a worksheet similar to the one shown next. The server name is the name used within WebLogic to refer to this server. Then have the network administrator complete the table by allocating hostnames and IP addresses for the servers. These hostname/IP address pairs should be put into an internal DNS.
Server type
Server name
Hostname
IP address
Admin
AdminServerAdminHost10.2.0.121SOA
WLS_SOA1SOAHost110.2.0.131SOA
WLS_SOA2SOAHost210.2.0.132SOA
WLS_SOA3SOAHost310.2.0.133BAM
WLS_BAM1BAMHost110.2.0.141Get frontend details for the load balancer.
The SOA Suite cluster will have at least one, and usually two or three, virtual hostnames for use by the load balancer. Create a table listing those requirements and get the network administrator to complete the hostname, port number, and protocol details.
Role
Virtual hostname
Port
Protocol
Admin access
ClusterAdmin.cookbook.com443HTTPS
Internal access
ClusterInternal.cookbook.com80HTTP
External access
Cluster.cookbook.com443HTTPS
Configure the load balancer to listen on all the virtual hostnames and ports identified in step 2 and load balance across all the web server hostnames and ports identified in the Gathering configuration information recipe.
The load balancer is used to distribute requests across the two web servers. The web servers form a routing pool (or multiple routing pools if listening on multiple protocols). The load balancer presents a single address to SOA Suite clients to access the cluster via HTTP and HTTPS.
The web servers will be configured by the EDG to load balance across the WSM cluster using the hostnames of the physical servers running OWSM. They will distribute the load across the BAM cluster using the name of the physical servers running the BAM web interfaces and the virtual hostname of the BAM server itself. The web servers use the virtual hostnames of the SOA servers to distribute the load across the cluster. Finally, the virtual hostname of the Admin server is used to route requests to whichever physical machine is hosting the Admin server at the time of the request.
Using virtual hostnames for the SOA managed servers, the BAM server and the Admin server allows these managed servers to move across physical machines without requiring reconfiguration of the load balancers.
The node managers are dedicated to physical machines and so, like the WSM managed servers, they are able to use the physical hostname of the server on which they run.
Note that although the SOA cluster may not receive SOAP requests, the load balancer may still be required to support access to web-based portions of the SOA Suite such as human workflow, the B2B console, and the SOA composer application. If the only HTTP access to the SOA environment is to the consoles for management purposes, then it may be possible to remove the load balancer and web servers from the installation. In case that no load balancer or web servers are used, EJB clients may access the managed servers directly using a T3 protocol which supports load balancing. Similarly, adapters do not require the load balancer.
The three frontend addresses mentioned are recommended in the EDG, but it is possible to collapse the internal and external access into a single role. It is recommended to keep a separate Admin access role to reduce exposure to hacking.
Although we have shown only a single network interface for both the SOA layer and web layer machines, it is good practice to have two physical network adapters in these layers to provide physical isolation of the networks to increase security. Multiple adapters can also be used to reduce the risk of network outages impacting on the cluster.
The Enterprise Deployment Guide, Chapter 3, Preparing the Network for an Enterprise Deployment, Section 3.4, About IPs and Virtual IPs (http://docs.oracle.com/cd/E23943_01/core.1111/e12036/net.htm)