Performance Tuning – Systems Running BPEL Processes

(For more resources related to this topic, see here.)

One of the main objectives of performance tuning is to achieve quick response time and more throughput. The application architect should work with business to understand the performance goals and set realistic performance expectations. The architect also needs to work with the infrastructure team to design the system's infrastructure and configuration of SOA components based on the performance expectations. The performance tuning step assists us in identifying the optimal settings for infrastructure and application components.

We should know the current performance level and performance goals before tuning the system. The high-level plan for a performance tuning exercise for an application platform is as follows:

  • Document the performance/response time before making any changes also known as the baseline current performance

  • Document the configuration parameters for existing systems also known as the baseline current configurations

  • Document what is the expected end result and backup the entire system once and also at intermediate points during performance tuning

  • Document the performance benefits due to new values of the configuration parameter(s) and update the configuration and performance baselines with suggested values based on the Service Level Agreements (SLAs)

The Oracle SOA Suite is a couple of applications developed by Oracle using J2EE technologies. One of the core applications is SOA-Infra, which is similar to any other J2EE web application. The other major components of the SOA Suite are BPEL Process, Mediator, Business Rules, and Human Task. All the Oracle SOA Suite components are deployed in a WebLogic server platform and also can deploy other platforms such as WebSphere. There are separate tuning parameters available for each of the Oracle SOA Suite applications and sub systems. Depending on your SOA composite application's deployment design one needs to holistically align the configuration parameters of Oracle SOA Suite components in addition to the WebLogic server platform, underlying Java Virtual Machine, Operating Systems, and/or Load Balancers to get the optimal system performance.

Please note that by simply adding more hardware or increasing the hosting system's resources such as CPU and memory you may not improve the performance of an application platform. The application design needs to be capable of utilizing these additional resources for application performance and scalability benefits.

The Java Virtual Machine

The Java Virtual Machine (JVM) allows you to write programs once and run them anywhere. Usually, JVMs are optimized for an operating system in combination with the underlying hardware platform.

The leading JVM vendors are Sun HotSpot JVM and Oracle JRockit for Windows, Unix, and Linux platform. Please note that Oracle is the vendor for both JRockit and Sun HotSpot JVMs. The Sun HotSpot JVM is widely used and is the default choice for Oracle SOA Suite implementations. Therefore, we will discuss the Oracle Sun HotSpot JVM performance tuning in detail.

Garbage collection process

Garbage collection (GC) is a process of JVM that removes the unused Java objects from the JVM heap to recycle the JVM resources. The configuration of the underlying JVM for the WebLogic servers affects the Oracle BPEL Process Manager performance. The details will be discussed in the following sections.

The Java heap memory space is divided into three sections: young, tenured, and permanent generation, as shown in the following diagram. Minor garbage collection occurs at the young generation space and major garbage collection occurs at the tenured space.

Young generation

Young generation memory space is used whenever we create objects in Java. That is why the young generation memory space is also known as new space. The Java objects enter the young generation space and they either get the garbage collected or move into the tenured generation space. Usually the young generation memory space is smaller than the tenured generation space, therefore it has shorter collection times as compared to the full GC in tenured space. The garbage collection process in the young generation is known as Minor Garbage Collection (minorGC).

Tenured generation

The Java objects in the tenured generation space last longer than the young generation space. Usually, the tenured generation memory space is bigger than the young generation space and it takes more time as compared to minor garbage collection occurring in the young generation. The garbage collection process in the tenured generation is known as Major Garbage Collection (majorGC).

Permanent generation

Permanent generation holds the data needed by VM to describe objects (describing classes and methods). It is recommended that we always allocate enough permanent generation space to load Oracle SOA Suite library classes in a JVM. The default permanent generation size is not sufficient for Oracle SOA Suite. There is no garbage collection process in permanent generation space for a JVM.

Garbage collection tuning

The main objective of garbage collection tuning is to achieve less frequent garbage collection with less pause time. Tuning the garbage collection process for a JVM is very important because an application that spends about 10 percent of its time in garbage collection can lose about 75 percent of its throughput when scaled out to multiple processors.

In this section, we will learn the options to tune JVM to run the SOA Suite optimally. The best way to tune the JVM and eliminate memory-related errors is to adjust JVM memory parameters. The JVM parameters are usually part of the WebLogic server start-up scripts. The default install values are not sufficient and may not provide an optimal runtime performance for an Oracle SOA Suite implementation. You need to ensure that the default JVM parameters are updated. The recommended heap size for Oracle SOA Suite is 4 GB. Some of the major JVM parameters for JDK 6 are as follows:

-Xms4096m \ - Minimum Heap
-Xmx4096m \ - Maximum Heap
-Xss512k \ - Set maximum native stack size for any thread
-XX:GCTimeRatio=99 \ - The ratio of GC time to application time
-XX:MaxGCPauseMillis=20 \ - Pause times of 20ms desired.
-XX:PermSize=512m \ - Permanent Size
-XX:MaxPermSize=512m \
-XX:NewSize=1024m \ - Minor GC (Young Generation)
-XX:MaxNewSize=1024m \
-XX:SurvivorRatio=6 \

During performance tuning, it is important to ensure that all endpoint applications are responding properly. When you adjust the minimum and maximum heap parameters, please ensure that you adjust the other parameters as well. The PermSize parameter can be constant but the NewSize and MaxNewSize parameters require tuning.

It is recommended for better performance that we keep the same values for following pairs for JVM parameters:

  • Minimum heap size (-Xms) and maximum heap size (-Xmx)

  • -XX:PermSize and -XX:MaxPermSize

  • -XX:NewSize and -XX:MaxNewSize

Choosing the garbage collection algorithm

One of the key JVM parameters is the choice of the garbage collectors. Two of the major garbage collectors are listed as follows for reference:

  • Mark and sweep (-XX:+UseConcMarkSweepGC)

    Mark and sweep garbage collection algorithm minimizes the garbage collection pause time that enables us to avoid impacting the production transaction during the GC process. This algorithm will run the tenured generation concurrently with the execution of the application.

  • Throughput collector parallel collector (XX:+UseParallelGC)

    That throughput collector distributes the garbage collection load across CPUs; therefore maximizing throughput.

    It is ideal for using with multiprocessor machines usually with four or more processors.

Select NewSize

Set -XX:NewSize to be one-fourth of the size of the maximum heap size. The larger the young generation space means smaller the tenured space. One can't specify the tenured generation size directly; the size of the tenured generation space is determined by the total heap size and NewSize.

Tenured generation space size = Total heap size – NewSize

The higher values of NewSize will result in a lesser minor garbage collection.

To verify the JVM performance, use Oracle SOA Suite Enterprise Manager Console. Select the WebLogic domain and use the JVM performance to view the performance details.

The other available options are to use the WebLogic Console or JVisualVM.

Select heap size

Set the values of -Xms and -Xmx to around 80 percent of the available physical memory on a server. The remaining 20 percent memory is left for the operating system and other processes. Setting the value of -Xmx to a very high value will result in a slower majorGC that occurs less frequently. Setting too high values for a JVM heap size can cause wasted memory and performance impact during a majorGC. Please note that on a 32-bit OS server(s) a JVM can only use a maximum 2.5 GB.

Garbage collection tool – JVisualVM

JVisualVM is a GUI-based JVM monitoring, troubleshooting, and profiling tool that can be used to diagnose JVM performance, JVM garbage collection processes to tune JVM parameters for an application platform such as Oracle SOA Suite. The JVisualVM tool is part of JDK Version 6 that combines several JVM utilities such as JConsole, jstat, jinfo, jstack, and jmap under one roof.

To start the JVisualVM, go to the bin directory of the JDK install and invoke the jvisualvm application. The Java Visual VM tool provides a view of all the Java parameters set for a WebLogic server environment, as shown in the following screenshot:

Using the JVisualVM tool one can see the live CPU usage, GC activity, and Heap activity for an application such as Oracle SOA Suite, as shown in the following screenshot:

You can also view the JVM performance details for the garbage collection activity by adding the following parameters as part of the WebLogic start-up scripts:

  • –XX:+PrintGC: Outputs the basic information at every garbage collection

  • –XX:+PrintGCDetails: Provides the size of live objects before and after garbage collection for the various generations, the total available space for each generation, and the length of time the collection took

  • –XX:+PrintGCTimeStamps: Provides a timestamp at the start of each collection that helps us correlate garbage collection logs with other logged events

  • –XX:+PrintGCTimeStamps: Provides a timestamp at the start of each collection that helps us correlate garbage collection logs with other logged events

You are recommended to create a table with JVM parameters and its impact on the performance during the tuning process, as shown in the following table using excel or word. In the tuning process always analyze GC activities for pause time, frequency, and average memory footprint.

SOA Suite

The SOA Suite's default configuration parameters may not provide optimal performance for your production runtime environment. Tuning the configuration parameters will improve the performance and/or stability of the application platform; however, always document the difference in the performance between before and after changes. Do not change any default configuration parameters if there is no performance or stability benefits for the application platform.

SOA Suite has multiple GUI consoles available to view and update the configuration parameters. The commonly-used consoles are described in the following sections.

SOA infra application

The URL for invoking SOA infra application is http://{soa-host}:{soa- port}/ soa-infra/ that provides the links to various other consoles available for viewing and modifying the configuration parameters, as shown in the following screenshot:

The WebLogic console

The URL for WebLogic console is http://{admin server host}:{admin server port }/console. The WebLogic console can be used for managing WebLogic server domain, configuring datasources, JMS, configuring security, and monitoring the status and health of the servers.

The enterprise manager

The URL for the enterprise manager console is http://{admin server – host}:{admin server-port}/em. The enterprise manager console can be used for deploying and undeploying SOA composite applications, viewing the filesystem and directory structure of SOA fusion middleware, monitoring the performance of SOA composite applications, and monitoring the status of SOA infra applications. The enterprise manager can be used to perform lifecycle operations such as startup/ shutdown of SOA composites. Apart from SOA composites, it is also used to monitor the performance of Oracle's web services and J2EE resources deployed on the WebLogic server.

You can also use the enterprise manager to perform most of the functionalities provided by WebLogic's admin console.

As shown in the following screenshot, you can select the Performance Summary option to monitor the performance of the admin, SOA managed server, and BAM instances:

As shown in the following screenshot, you can monitor the performance of an SOA composite application by selecting the Performance Summary option under SOA Composite:

Dynamic Monitoring Service (DMS)

The DMS metrics tables provide metrics data for fusion middleware. The URL for DMS is http://{adminserver-host}:{adminserver-port}/dms. Click on the left navigation menu items to view the details of configuration parameters. A DMS console looks like the following screenshot:

The B2B console

B2B component facilitates the exchange of documents between trading partners. The B2B console URL is http://{soa-host}:{soa-port}/b2bconsole. A screenshot of a B2B console is as follows:

One can make changes to the B2B properties from the enterprise manager console. The B2B properties can be viewed by selecting the SOA Administration option and then B2B Server Properties, as shown in the following screenshot:

Please make sure to update b2b.inboundThreadCount, b2b.outboundThreadCount, b2b.defaultThreadCount, and MDS cache size to accommodate the messaging load for the production runtime environment. Usually we increase them by 3 to 6 times of the default values to process high volume message loads. The inbound threads responsible for processing the inbound message and outbound threads are responsible for processing the outgoing message from B2B components.

The other available GUIs within Oracle SOA Suite are listed as follows for your reference:

  • Web Services Inspection Language (WSIL):


  • Web Services Manager (WSM): http://{soa-host}:{soa-port}/wsm-pm

  • Composer: composer http://{soa-host}:{soa-port}/soa/composer

  • DefaultToDo:

    http://{soa-host}:{soa-port} /workflow/DefaultToDoTaskFlow

  • Worklist: worklistapp http://{soa-host}:{soa-port}/integration/ worklistapp

  • MessagingService endpoint:


  • MessagingServices preferences:


The System MBeans browser

The SOA MBeans can be viewed from the enterprise manager console (http://{adminserver-host}:{adminserver-port}em). Right-click on the soa-infra and select Administration and then System MBean Browser, as shown in the following screenshot. The search string is used for searching the Mbeans. You can set the performance tuning parameters on MBeans using MBeans browser:

You can view the properties of all the deployed composite applications from MBeans browser including the revision of the application. Click on the composite application name under Application Defined Mbeans for making changes, as shown in the following screenshot:

SOA Suite tuning

Some of the major SOA Suite tuning parameters are listed as follows. To achieve high performance, the default parameters require changes.

  • Transaction timeouts: The default JTA time out of 300 seconds for SOA Suite may not be sufficient for a production runtime. Some of the transactions in the production environment take more than 300 seconds to finish and logfiles show transaction time out exceptions. You will need to increase the transaction time out.

  • Database sessions: Set the process parameter in database > 300. Also set the session parameter > 200.

  • 64-bit JVM: Always ensure that you are using a 64-bit JVM. The 64-bit JVM will allow you to have the recommended heap size of 4 GB. However, for a 32-bit JVM one can get the maximum heap size to 2 GB. On the other hand the 64-bit processor instructions are more efficient than the 32-bit ones.

  • Payload validation and audit level: Payload validation of incoming messages (while configuring the server URL) decreases the performance. Go to the SOA enterprise manager console to disable the payload validation. The URL for EM is http://{adminserver-host}:{adminserver-port}// em/. Right-click on soa-infra from the left navigation menu and select SOA Administration and then BPEL Properties to change the Payload Validation and Audit settings. As shown in the following screenshot, ensure that the checkbox for Payload Validation is unchecked. Usually payload validation is used to intercept the incoming payload data and convert the non schema-complaint data to fault. The payload validation involves additional processing and, therefore, it decreases the overall performance. Setting the Audit Level option higher also decreases the performance. One can change the Audit Level option by selecting Off, Inherit, Minimal, Production, or Development, as shown in the following screenshot.

    • Off: Highest performance

    • Production: Medium performance

    • Development: Impacts performance

    The Audit Trail Threshold parameter sets limits for the audit trial size. The default value is 1 MB. The following screenshot explains how to set the auditing level and disable payload validation:

  • The SOA composite application instance state: Enabling the SOA composite application instance's state decreases the performance of the SOA container and allows you to track the running instance separately. You can use CompositeInstanceStateEnabled property to enable/disable the instance state.

  • Mediator worker threads: Increasing the mediator worker threads results in a performance improvement for asynchronous services. The thread level can be adjusted by changing the Parallel Worker Threads configuration parameter available under SOA Administration | Mediator Properties in Enterprise Manager Console, as shown in the following screenshot. It specifies the number of parallel threads available for message processing. The recommendations for Mediator Properties changes are as follows:

    • Increase Parallel Worker Threads

    • Increase Parallel Maximum Rows Retrieved (x 100)

    • Reduce Parallel Locker Thread Sleep

  • Mediator routing rules: Implementing parallel processing for routing rules of the mediator will improve the performance. The routing rules can be changed using the SOA composite editor. Please note that setting the routing rules impacts the application design.

  • Logging levels: Set all log levels to Error (Severe) in the production environment. You may use the log levels to Notification (Info) during development, troubleshooting, and the tuning process. You can modify the log levels of WebLogic components using WebLogic admin console. Use the enterprise manager console to change the log levels of Oracle's loggers. SOA Suite defines a set of Oracle loggers that can be changed only using the enterprise manager console. Please refer to the following screenshot:

  • BPEL Process Manager Threads: Login to the enterprise manager console (http://{adminserver-host}:{adminserver-port}/em/) and select BPEL Properties in the left navigation menu, as shown in the following screenshot. Update the BPEL Process Manager Dispatcher Threads settings to support the system load as follows:

    • Increase Dispatcher System Threads to at least 10

    • Increase Dispatcher Invoke Threads based on the target load

    • The value of Dispatcher Engine Threads should be equal to the sum of Dispatcher System Threads and Dispatcher Invoke Threads

  • JMS modules: Oracle SOA Suite has many JMS connection factories, queues, and topics for internal use. The changes on these JMS modules are not recommended.

  • Increase the performance of EM Console: As shown in the following screenshot disable the loading of all metric information to the EM console. Once the production systems are up and running for some time, the EM console load time will be very high and won't be acceptable:

Load balancers

Most of the enterprise implementations of web services will demand high availability. In some cases it will be a global high availability that is usually achieved via the system architecture's designs and deployments leveraging multi sites in active-active or active-passive. The passive site is either standby (manual process to fail over and fail back) or hot standby (automatic process to fail over and fail back) to provide the services if the primary site goes down. The load balancers are also used to achieve the horizontal scalability. Load balancers for web services are implemented at either L7 (application Layer) also known as Global Server Load Balancer (GSLB) or Global Traffic Manager (GTM), or at L4 (Transport Layer) also known as Local Server Load Balancer (LSLB) or Local Traffic Manager (LTM) to achieve high availability and scalability. Layer 7 forwards the user to the appropriate local load balancer, any further requests for the user will directly go the local load balancer within a specified time window configured as part of TTL (Time to Live). The Layer 4 load balancer serves all user requests. In other words, the L7 load balancers provides the user forwarding service while L4 load balancers serves the users as middlemen. In a high-availability environment, both L4 and L7 are used. These load balancers come in both software and/or hardware appliances from various vendors. The major vendors are F5, Brocade, NetScaler, and Cisco.

L4 load balancers are usually implemented to achieve high availability within a data center that provides limited horizontal scalability. One needs to implement L7 load balancers to achieve high horizontal scalability. You will mostly find that L4 and L7 load balancers are implemented as a pair, since most implementations don't need to scale beyond a single load balancer capacity. In implementations where L7 load balancers are not used; the L4 load balancers pair is implemented with one of them as hot standby. It is recommended that if you have got L7 load balancers then make all the L4 load balancers active to optimize the performance and capacity during peak loads.

L7 load balancers/GSLB/GTM configurations are as follows:

  • Implement L7 health checks over TCP health checks

  • Use low TTL such as 30 secs

  • Utilize GEO affinity, if available

  • Implement independent endpoints if services are run by different server processes

  • Use sites load factors for weighted round robin load balancing algorithm, if available

L4 load balancers/LSLB/LTM configurations are as follows:

  • Implement L7 health checks over TCP health checks

  • Use least connection load balancing algorithm

  • Use SSL offload unless it is a security mandate

  • Enable compression and caching

  • Enable TCP connection queue

Operating system

Always use a 64-bit operating system instead of a 32-bit one for the Oracle SOA Suite as a 32-bit operating system limits the available JVM memory. Verify the CPU and memory usages under the expected load conditions for the host running the SOA Suite application platform for potential bottlenecks. The CPU usage can be viewed by using the top command for Linux. The memory can be verified by using free –t –m or vmstat. For Windows, use the command systeminfo.

File descriptors

The default values of file descriptors in the Linux operating system will not be enough to handle the required number of concurrent connections by SOA Suite. The operating system parameter ulimit sets the system-wide resource limit for a user. The default value of 1024 for a user is too low for running an Oracle SOA Suite application. Use the ulimit or sysctl Linux commands to change the default value to at least 8192 by executing the following command:

ulimit -n 8192

The following message in the logfile indicates that you need to increase the file descriptors:

Too many open files

Also, make changes to soft and hard file limits from limits.conf located in the /etc/security folder for Linux systems


Please make sure that the operating system file descriptor is set very high if you are using file adaptors.

Tuning JCA adaptors are important for optimum performance. Change the following properties in the inbound and outbound JCA files.

  • Threadcount: By default, the adapter uses the global thread pool. The default value is -1. If you have the threading issues on logfiles and have performance issues, then create separate processor threads by changing the value to zero.

  • MaxRaiseSize: Increase this parameter from default (10,000) if you need to process a large amount of files.

  • ConcurrentThreshold: By default, the translation activities is limited to 20. Increase this value up to 100 for increasing the translation activities allowed to start in parallel for particular outbound scenarios.

  • For database adaptors use indexes, disable delete polling, disable Merge, use connection pooling and synchronous processes.


Tuning of the Oracle database is out of the scope for this book; however, some of the relevant steps for tuning that can be performed as part of SOA Suite tuning are listed in this section. Since SOA Suite is an application running on a J2EE container such as WebLogic, it requires a database for storing meta data. Monitoring and tuning the database associated with SOA Suite provides optimum performance for SOA Suite container and your BPEL composite application.

  • Ensure that the initial connection pool and max connection pool are the same. Use a large number for connection pools to avoid running out of database connections.

  • Use GridLink database source for Oracle RAC connectivity instead of multi pools.

  • Disable database connections verifications, testing, and profiling.

  • Enable statement caching. This improves SOA Suite performance by caching executable statements that are used repeatedly in the database.

Dehydration store

The Oracle SOA Suite saves all the BPEL process statuses in dehydration database tables. The dehydration tables are part of SOA_INFRA schema and created initially during installs as part execution of RCU.

Over a period of time, you need to purge the metadata from the dehydration store to manage the growth of the database. Oracle provides a default purge script for deleting data which is located in {RCU_HOME}/rcu/integration/soainfra/sql/ soa_purge, however, you need to delete additional data to improve the performance and manage the database growth.

The database schema ddl can be found at {Oracle SOA Install directory}\ rcu\integration\soainfra\sql\bpel. You can write SQL statements to delete or update data in dehydration tables.


Most of the database initialization parameters are stored in the Init.ora file for Oracle database 11g. Some of the important configuration parameters related to SOA Suite container performance are as follows:

  • Memory_Target: Manage the RAM resource utilization. Instead of setting SGA_MAX_Size, use SGA and PGA as needed.

  • Audit_Trial: Database auditing can be enabled or disabled using this parameter.

  • Always enable auto extend of Oracle Tablespace to avoid performance and run time issues. Use Automatic Segment Space Management (ASSM) for permanent table spaces.

Automatic Workload Repository

You can use Automatic Workload Repository (AWR) for tuning the SOA_INFRA database. AWR is a part of the Oracle database. The Oracle database automatically collects performance information such as snap shots on periodic bases (every hour). The Oracle database takes care of cleaning the snap shot performance data on a regular basis.

As shown in the following screenshot, please login to the Oracle database to run the report:

As shown in the following screenshot, execute the following command for creating the AWR report:


The sample report will be as follows:

Analyzing the entire AWR reports are out of the scope for this book. You can get a lot of information such as CPU and memory usage, standalone and RAC load profile, operating system statistics, wait events, instance and tablespace IO statistics for your database platform without writing scripts.


In this article, we learned to tune the SOA composite applications designed and deployed on the Oracle SOA Suite platform for optimal performance and scalability. We reviewed the industry leading practices for the Oracle SOA Suite platform components such as Oracle SOA Suite component applications, WebLogic server platform, JVM, operating systems, and load balancers.

Resources for Article :

Further resources on this subject:

You've been reading an excerpt of:

Oracle SOA BPEL Process Manager 11gR1 – A Hands-on Tutorial

Explore Title