Apache Camel Developer's Cookbook

4.3 (4 reviews total)
By Scott Cranton , Jakub Korab
  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. Structuring Routes

About this book

Apache Camel is a de-facto standard for developing integrations in Java, and is based on well-understood Enterprise Integration Patterns. It is used within many commercial and open source integration products. Camel makes common integration tasks easy while still providing the developer with the means to customize the framework when the situation demands it. Tasks such as protocol mediation, message routing and transformation, and auditing are common usages of Camel. Apache Camel Developer's Cookbook provides hundreds of best practice tips for using Apache Camel in a format that helps you build your Camel projects. Each tip or recipe provides you with the most important steps to perform along with a summary of how it works, with references to further reading if you need more information. This book is intended to be a reliable information source that is quicker to use than an Internet search. Apache Camel Developer's Cookbook is a quick lookup guide that can also be read from cover to cover if you want to get a sense of the full power of Apache Camel. This book provides coverage of the full lifecycle of creating Apache Camel-based integration projects, including the structure of your Camel code and using the most common Enterprise Integration patterns. Patterns like Split/Join and Aggregation are covered in depth in this book. Throughout this book, you will be learning steps to transform your data. You will also learn how to perform unit and integration testing of your code using Camel's extensive testing framework, and also strategies for debugging and monitoring your code. Advanced topics like Error Handling, Parallel Processing, Transactions, and Security will also be covered in this book. This book provides you with practical tips on using Apache Camel based on years of hands-on experience from hundreds of integration projects.

Publication date:
December 2013
Publisher
Packt
Pages
424
ISBN
9781782170303

 

Chapter 1. Structuring Routes

In this chapter, we will cover the following recipes:

  • Using Camel in a Java application

  • Embedding Camel in a Spring application

  • Using Camel components

  • Reusing routing logic by connecting routes

  • Asynchronously connecting routes

  • Spanning Camel contexts within a single Java process

  • Using external properties in Camel routes

  • Reusing endpoints

  • Reusing routing logic through template routes

  • Controlling route startup and shutdown

 

Introduction


This chapter will introduce you to the fundamentals of running Apache Camel inside your applications. You will learn how to make use of Camel's rich set of components, and how to structure routes in such a way that common integration logic is able to be reused without duplication. These topics will provide you with the foundation for developing integrations using the framework.

Tip

Downloading the example code

Complete examples for each of the code snippets are located at http://github.com/CamelCookbook/camel-cookbook-examples, and through your account on the Packt Publishing's website at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

To try these examples for yourself, obtain the example code, and build it using Apache Maven 3.0 or newer (http://maven.apache.org). Use the following command from the top-level directory of the project. Complete instructions are also provided in the included README file.

# mvn clean install

The code for this chapter is contained within the camel-cookbook-structuring-routes module of the examples.

 

Using Camel in a Java application


Camel is a framework that is composed of a set of JARs, much as any other library that lives alongside your code. If you wanted to run Camel from the command line, you would define the libraries used within as a list of JARs to be considered by the java and javac command-line tools.

Note

The supporting code for this book uses Camel within the context of Maven projects that build standalone JARs. The JARs are not meant to be executed themselves, rather the Maven project structure is used as a convenient harness for driving JUnit tests that demonstrate the behavior being described.

The Camel libraries are broken up into two categories:

  • Core set of artifacts containing the runtime, test support classes, and build tools.

  • Optional libraries that abstract away the details of dealing with a given technology (for example, messaging via JMS or SOAP services via CXF). At the time of writing, Camel integrates with over 140 technologies (http://camel.apache.org/components.html), and each is encapsulated within its own library with its own dependencies.

This recipe will show you the basic steps to start and stop Camel routes from within your Java application.

Getting ready

The minimal set of libraries typically required to use Camel within a Maven build are:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-core</artifactId>
  <version>${camel-version}</version>
</dependency>
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-test</artifactId>
  <version>${camel-version}</version>
  <scope>test</scope>
</dependency>

The Camel version is usually defined in a properties block within the Maven POM file once, as follows, so as to not need to be repeated over and over:

<properties>
  <camel-version>2.12.2</camel-version>
</properties>

Note

A note on versions

Camel is a very mature framework that can be considered as being "core-complete". It has become the core library for integration in a number of commercial ESBs, and at this stage the underlying architecture is very stable and unlikely to be radically changed.

This book has been written against what will most certainly be an outdated version by the time you read this. Any changes made to Camel since the time of writing are likely to have been additive. There will be more components, and more options around the various integration patterns as further use cases require them. For the very latest detailed documentation, refer to the Camel website.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.simple package.

How to do it...

  1. Define a route using the Camel Java DSL, by extending the org.apache.camel.builder.RouteBuilder abstract class:

    public class LogMessageOnTimerEventRouteBuilder
        extends RouteBuilder {
      @Override
      public void configure() throws Exception {
        from("timer:logMessageTimer?period=1s")
          .log("Event triggered by ${property.CamelTimerName}"
             + " at ${header.CamelTimerFiredTime}");
      }
    };
  2. The following steps occur in your application's main method. See SimpleCamelApplication from this example. Create a CamelContext implementation:

    CamelContext context = new DefaultCamelContext();
  3. Add the route definition to the context; this can be invoked as many times as you have routes:

    context.addRoutes(
        new LogMessageOnTimerEventRouteBuilder());
  4. Start the context. This loads the route definitions that you have added, and processes messages through them in the background:

    context.start();

    Note

    The CamelContext.start method is non-blocking. It will start up associated Components on internal threads, and return to the caller.

  5. When the Camel application is ready to be shut down, call:

    context.stop();

How it works...

The CamelContext interface is the heart of the Camel framework. It is responsible for processing messages along routes.

The from(...) statement at the start of a route defines an endpoint, or a technology-specific location, that the Camel routing engine uses to fetch messages. Endpoints are defined using URIs, such as in the preceding example, timer:logMessageTimer. The first part of the URI specifies the component that is being used to consume the message, and the remaining is a set of instructions for that specific component. See the Using Camel components recipe in this chapter for more details.

The Camel routing engine consumes exchanges from these endpoints and processes them sequentially through each of the steps defined in the route. The engine is responsible for threading, transactions, error handling, copying messages where required, and many other details.

The Camel context is a long-running object; it is intended to live for as long as the application does, and therefore its initialization and shutdown is usually tied to the lifecycle of the application. Typical deployments of Camel define the context within:

  • The main() method of a standalone command-line application; here it waits indefinitely until the user terminates the process

  • As an instance variable within a javax.servlet.ServletContextListener in a web app, starting up and shutting down along with the application

  • An object tied to an OSGi bundle's lifecycle

  • An object within a Spring or OSGi Blueprint context that is itself tied to the application's lifecycle

Routes, which are definitions of the steps that messages should be processed through, are typically added to the newly created context, though they can be added, removed, and modified at runtime. Route definitions can only be added to a context before the context is started, though they can be stopped and restarted while the context is running.

Extending the RouteBuilder abstract class gives access to Camel's Java route definition DSL, or simply the Java DSL. What this means in practice is that within the mandatory configure() method, after typing the first from(...) statement that defines the start of a route, you get context-specific code completion of whichever integration patterns you might be using.

A RouteBuilder implementation may implement one or many routes. That is, within the configure() method, you can specify multiple from(...) statement that Camel will translate into multiple runtime route instances, one per from(...) statement.

There's more...

Camel is a highly configurable framework, in which most behaviors can be customized through service provider interfaces (SPIs). An SPI encapsulates a single behavior, such as a route naming strategy (Camel gives your routes sensible names if you do not do so explicitly). To override the default behavior, you provide your own implementation of the SPI class and set it on the CamelContext object. The context allows you to define the following, amongst others:

  • Listeners that are notified of Camel lifecycle events

  • Naming strategies for routes, route nodes, and JMX

  • Strategies for shutting down the application gracefully

  • Mechanisms for managing thread pools

It is therefore worthwhile getting familiar with the options that this class gives you by going over the Javadocs.

The CamelContext interface makes use of an internal object registry that allows it to look up objects by name. When using a DefaultCamelContext, a JNDI-aware registry is used.

This feature is used extensively throughout the framework for finding components, thread pools, named processor beans, data formats, and the like.

Occasionally, it is necessary to add objects directly to the registry, as in the case of beans that you want to execute, as one of the processing steps in a route. To do this, instantiate an implementation of org.apache.camel.spi.Repository, usually org.apache.camel.impl.SimpleRegistry, and pass it into the constructor of the DefaultCamelContext:

SimpleRegistry registry = new SimpleRegistry();
registry.put("payloadTransformer", new MyPayloadTransformer());
CamelContext context = new DefaultCamelContext(registry);

The CamelContext interface defines type-safe utility methods for setting certain object types, such as components, that allow you to set them without worrying about the registry internals.

Consider the following manual step:

registry.put("mylogger", new LogComponent());

This can be written in a type-safe way as follows:

context.addComponent("mylogger", new LogComponent());

The Registry in Camel can hold any named Java instance, and these instances can be referenced by name from the Camel DSL. The addComponent method of the CamelContext is specifically used for registering Camel components by name. Both approaches do effectively the same thing, though there are some subtle differences, and we would recommend using the addComponent method for components, and adding all your POJOs and custom processors into the registry.

 

Embedding Camel in a Spring application


This recipe will show you how to integrate Camel into a Spring application.

Getting ready

When using Camel within a Spring application, it is necessary to add the following dependencies to the minimal set defined in the Using Camel in a Java application recipe in this chapter:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-spring</artifactId>
  <version>${camel-version}</version>
</dependency>
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-test-spring</artifactId>
  <version>${camel-version}</version>
  <scope>test</scope>
</dependency>

The ${camel-version} property is defined once in the Maven POM.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.simplespring package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with simplespring.

How to do it...

In order to embed Camel into a Spring application, perform the following steps:

  1. In the XML namespace declaration, define the Camel schema alongside any Spring schemas in use:

    <beans
      xmlns="http://www.springframework.org/schema/beans"
      xmlns:camel="http://camel.apache.org/schema/spring"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://camel.apache.org/schema/spring
        http://camel.apache.org/schema/spring/camel-spring.xsd">
  2. The camelContext element should be defined once within the Spring configuration, and it should use the Camel Spring namespace. This signifies that everything within it will be considered Camel configuration as opposed to Spring.

    <camelContext
        xmlns="http://camel.apache.org/schema/spring">
      <!-- routing logic goes here -->
    </camelContext>
  3. Routes can then be defined within the camelContext element using the XML DSL:

    <route>
      <from uri="timer:logMessageTimer?period=1s"/>
      <to uri="mylogger:insideTheRoute?showHeaders=true"/>
      <log
          message="Event triggered by ${property.CamelTimerName} at ${header.CamelTimerFiredTime}"/>
    </route>

How it works...

Camel was designed to be closely integrated with Spring from its inception. The camelContext element results in a SpringCamelContext object being created, initialized with any routes defined within it, and started when the Spring context starts up. The camelContext element is itself a Spring managed object that can optionally be given an ID and treated like any other bean.

The preceding example shows Camel's XML DSL being used. One of the nice things about the DSL is that an XML schema is used to define it. This means that it is possible for your IDE to provide you with code completion.

It is not mandatory to use the XML DSL with Spring. It is possible to use the Java DSL instead, or alongside routes defined through the XML DSL.

To plug in the route defined in the LogMessageOnTimerEventRouteBuilder class that we used in the previous recipe, we first instantiate it as a bean:

<!-- package name has been abbreviated -->
<bean id="logMessageOnTimerEvent"
      class="org.camelcookbook.structuringroutes.simple.LogMessageOnTimerEventRouteBuilder"/>

Then we add it to the camelContext element using the routeBuilder tag:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <routeBuilder ref="logMessageOnTimerEvent"/>
</camelContext>

Multiple routeBuilder elements can be used within a camelContext.

There's more...

If you define a number of RouteBuilders in the same package, it is possible for Camel to scan that package and instantiate all of the routes that it finds:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <packageScan>
    <package>org.camelcookbook.structuringroutes</package>
  </packageScan>
</camelContext>

You can add multiple package elements within the packageScan element, and also use wildcards to include or exclude RouteBuilders by name, using the excludes and includes elements.

Spring provides an alternative feature called component scanning. When enabled, the Spring application context recursively scans a package, and instantiates any class within that is annotated with org.springframework.stereotype.Component. Any properties annotated with @Autowired, or the CDI equivalent @Inject, have their dependencies injected. Camel can be configured to pick up any RouteBuilders wired through this process. The RouteBuilders must first be marked as components:

@Component
public class LogMessageOnTimerEventRouteBuilder
    extends RouteBuilder {
  //...
};

To enable the wiring, turn on component scanning in Spring:

<component-scan
  base-package="org.camelcookbook.structuringroutes"
  xmlns="http://www.springframework.org/schema/context"/>

Then add the appropriate feature to the Camel context to tie it all together:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <component-scan/>
</camelContext>
 

Using Camel components


When writing integration code, you will inevitably have to work directly with libraries that deal with the technology being integrated. The details of invoking web services, consuming or sending files to FTP servers, or messaging over JMS often take up a substantial proportion of development time on a project. Camel abstracts away the repeated details of consuming from or producing to these "transports", by encapsulating this interaction within a Component.

Camel provides a rich set of components, which abstract away this plumbing code from you. These allow you to focus your energies on the business logic of your integration without worrying about the finer details of the transport.

This recipe will show you the basic steps of associating a Camel Component for a given technology with your integration routing logic.

Getting ready

To make use of a component, you will need to make sure that the component's library is included in your project. The camel-core library provides a set of fundamental components to get you started underneath the org.apache.camel.component package.

For integration with any other technologies, your first stop should be the component listing on the Camel website (http://camel.apache.org/components.html). Once you have found the technology that you are looking for, add the JAR dependency to your project, ensuring that the version matches that of the camel-core library you are using. For example, to use the Camel FTP component within your Camel route, you need to add the camel-ftp dependency to your POM:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-ftp</artifactId>
  <version>${camel-version}</version>
</dependency>

The ${camel-version} property is defined once in the Maven POM.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.simple package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with simplespring.

How to do it...

In order to use a Camel component you need to instantiate and register the component, and then reference it from your Camel route as per the following steps:

  1. If working within a Spring application, you use a Component by instantiating it as a bean and give it a meaningful name (id); the Component is automatically visible to Camel:

    <bean id="mylogger"
          class="org.apache.camel.component.LogComponent"/>

    Alternatively, in a non-Spring Java application, you register a Component by instantiating it and then adding it to the CamelContext before starting it:

    CamelContext context = new DefaultCamelContext();
    camelContext.addComponent("mylogger", new LogComponent());
    // add routes here
    context.start();
  2. To use the component as an endpoint in a to(...) or from(...) statement, refer to the name that you assigned it within the scheme part of the endpoint URI:

    .to("mylogger:insideTheRoute?showHeaders=true")

How it works...

To understand exactly what happens when we use an endpoint URI, it is easiest to take a brief look under the covers at the classes that Camel uses within its component framework.

A Component is a factory for creating Endpoints that can act as message Consumers, or Producers, or both. An implementation will typically have bean properties on it that will apply to the transport as a whole. For example, the JMS Component requires that a ConnectionFactory be set on it, in order to make use of the same message broker for all JMS communication:

<bean id="myFavouriteMQ"
      class="org.apache.camel.component.jms.JmsComponent">
  <property name="connectionFactory"
            ref="myConnectionFactory"/>
</bean>

All Components implement a method used to parse the endpoint URI:

Endpoint createEndpoint(String uri) throws Exception;

An Endpoint can be thought of as an address that is specific to the component technology–it is instantiated from the URI by the framework when the route using it is started. The scheme portion of the URI, the part before the first colon (:), identifies the component implementation being used.

Continuing the JMS example, given that the Component already knows where a message broker is, the Endpoint itself describes which queues or topics a message should be sent to or received from, and how:

myFavouriteMQ:someQueue?concurrentConsumers=5

The portion of the URI after the scheme is specific to the endpoint technology. The URI properties themselves, such as concurrentConsumers, correspond to bean properties on the endpoint implementation for that technology, and are set using introspection. If you were to take a look inside the JMS Component library, you would see a JmsEndpoint object with a setConcurrentConsumers(int consumers) method.

You should always refer back to the applicable Camel component page for a full list of properties that can be used.

The Endpoint is also a factory. Camel uses it for creating Producers and Consumers, depending on the context of where the URI is used. The following factory methods are defined on the interface:

Producer createProducer() throws Exception;
Consumer createConsumer(Processor processor) throws Exception;

These classes handle the heavy lifting of talking to the underlying technology.

If the endpoint URI is used in a from(...) statement, Camel will create a Consumer. If it is used in a to(...) block, will create a Producer.

There's more...

The same component can be instantiated multiple times with different IDs. For example, two JMS components might be used when writing routing logic to bridge message brokers from different vendors.

See also

 

Reusing routing logic by connecting routes


It is frequently necessary to execute the same processing steps within multiple routes. Camel provides you with a mechanism to call routing steps in a shared route. You can then reuse that route in a similar fashion to a method in a regular Java program.

This recipe will show you a strategy for developing your routes so that common routing tasks can be defined within a shared route called from other routes.

Getting ready

Determine the routing logic that you would like to reuse, and move it to a route that consumes from a direct: URI.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.direct package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with direct.

How to do it...

Create a route with the from URI using the direct: endpoint, and then in another route, call the shared route using the same direct: endpoint.

  1. The direct: endpoint name, which is an arbitrary alphanumeric string, follows the colon in the URI, must be unique within the containing Camel context:

    In the XML DSL, it is used as follows:

    <route>
      <from uri="direct:prefixBodyWithHello"/>
      <transform>
        <simple>Hello, ${body}</simple>
      </transform>
      <log message="Prefixed message: ${body}"/>
    </route>

    In the Java DSL, the same thing is expressed as:

    from("direct:prefixBodyWithHello")
      .transform(simple("Hello, ${body}"))
      .log("Prefixed message: ${body}");
  2. Invoke the shared route from within the top-level route that needs to make use of this logic by using the same direct: URI:

    In the XML DSL, this is written as follows:

    <route>
      <from uri="..."/>
      <to uri="direct:prefixBodyWithHello"/>
    </route>

    In the Java DSL, this appears as:

    from(...).to("direct:prefixBodyWithHello");

How it works...

Each direct: endpoint may only be consumed (used within the from(...) block) by one route in a Camel context. However, multiple routes may produce, or send, messages to that URI. This allows you to compose your routing logic in easily understandable blocks, much as you would when using methods in a regular Java program.

The shared route can be considered as a part of the top-level route. It operates on the same Exchange object, and participates in any transactions that the top-level route has initiated.

Regardless of whether the message exchange pattern on the exchange is InOnly or InOut, the behavior of invoking a direct: endpoint is the same; that is, the exchange will flow through all of the processing steps defined in the shared route and will be returned to the calling route on completion where it will proceed to the next processing step.

There's more...

It is possible to invoke the shared route, and merge the returned exchange with the exchange containing the original state before the route was invoked. See the Enriching your content with some help from other endpoints recipe in Chapter 4, Transformation, for more details.

See also

 

Asynchronously connecting routes


It is not uncommon to have a portion of route take a long time to process. Rather than tying up a thread that might otherwise be servicing requests from a consumer endpoint, it may be preferable to split out the time consuming step into a separate route, and let that stage of processing be handled by a dedicated pool of threads.

This recipe will show you how to call a route from another, such that the calling route does not block waiting for the response from the called route.

Getting ready

Split out the long running steps into their own shared routes, and assign them with a seda: endpoint with a name that is unique to the Camel context.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.seda package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with seda.

How to do it...

Create a shared route using the seda: endpoint, and then call it from other routes using the same named seda: endpoint.

  1. Create a route consuming (from) using a seda: endpoint. Optionally, define the number of threads that will consume from this endpoint using the concurrentConsumers attribute.

    In the XML DSL, this is written as:

    <route>
      <from
        uri="seda:longRunningPhase?concurrentConsumers=15"/>
      <process ref="longRunningProcessor"/>
      <to uri="..."/>
    </route>

    In the Java DSL, the same thing appears as:

    from("seda:longRunningPhase?concurrentConsumers=15")
      .process(new LongRunningProcessor())
      .to(...); // remainder of route
  2. In the calling route, pass the current exchange to the shared route by invoking the seda: endpoint by name.

    In the XML DSL, this is done as follows:

    <route>
      <from uri="timer:ping?period=200"/>
      <to uri="seda:longRunningPhase"/>
    </route>

    In the Java DSL, this same thing is written as:

    from("timer:ping?period=200")
      .to("seda:longRunningPhase");

How it works...

In the preceding example, a timer: endpoint is used to trigger messages on a regular basis, every 200 ms. The Timer Component uses one thread per timer name (ping). An event can only be raised 200 ms later if the thread is not processing the previous exchange.

As part of our integration, we want to trigger events regularly, and yet have a long-running processor as part of the route. Camel allows us to deal with this scenario by splitting the long-running part into a shared route, and linking the two routes with a seda: endpoint.

Note

SEDA is an acronym that stands for Staged Event-Driven Architecture. It is designed as a mechanism to regulate the flow between different phases of message processing. The idea is to smooth out the frequency of message output from an overall process so that it matches the input.

In practical terms, it allows an endpoint's consumer threads to offload the work of long-running operations into the background, thereby freeing them up to consume messages from the transport.

When an exchange is passed to a seda: endpoint, it is placed into a BlockingQueue. The list exists within the Camel context, which means that only those routes that are within the same context can be joined by this type of endpoint. The queue is unbounded by default, although that can be changed by setting the size attribute on the URI of the consumer.

By default, a single thread assigned to the endpoint reads exchanges off the list and processes them through the route. As seen in the preceding example, it is possible to increase the number of concurrentConsumers to ensure that exchanges are getting processed from that list in a timely fashion.

Assuming our slow processor takes 3,000 ms to complete, we would need to use a number of threads equal to the processing time/the timer frequency to ensure that the triggered events are processed in a timely fashion. Therefore, plugging in the numbers, 3,000 ms / 200 ms, we arrive at 15 threads.

There's more...

It is possible to define multiple routes that consume from the same logical name, unlike a direct: endpoint. To enable this, both routes should set multipleConsumers=true on the seda: URI:

from("seda:timerEvents?multipleConsumers=true")
  // define one set of routing logic here

from("seda:timerEvents?multipleConsumers=true")
  // another here

The effect will be that each route gets its own copy of the exchange, making it a sort of simple in-memory publish-subscribe system. It is often much cleaner to handle this type of requirement using the Multicast pattern. See the Multicast – routing the same message to many endpoints recipe in Chapter 2, Message Routing, for more details.

The SEDA pattern is best suited to processing the InOnly messages, where one route finishes processing and hands off to another to deal with the next phase. It is possible to ask for a response from a seda: endpoint by calling it when the message exchange pattern is InOut.

In this instance, the endpoint will act much like a synchronous direct: endpoint, only with a timeout if the task runs for longer than expected. The default timeout is 30 seconds, but this can be overridden in the producer URI using the timeout attribute.

Note

It is important to note that when using seda:, the shared route does not take part in any transactions started by the top-level route, as transactions are bound to a thread and the seda: route is using its own thread. If the system unexpectedly halts for whatever reason (for example, a power outage), any messages that are being processed within that route will be lost.

If you would like to get the benefits of a SEDA, but have the messages persisted to disk instead of in-memory, and be processed within a transaction, use the JMS or ActiveMQ endpoints with a message broker instead. This also gives you the ability to share the work across Camel contexts and JVMs.

See also

To trigger a background job to run based on the current exchange while your main route continues processing, refer to the Wire Tap EIP. See Wire Tap sending a copy of the message elsewhere recipe in Chapter 2, Message Routing, for more details.

 

Spanning Camel contexts within a single Java process


Sometimes it is convenient to share routing logic between integrations hosted in the same container, for example, as web apps or OSGi bundles. Camel provides two components that allow you to do this, synchronously through a Direct VM Component, or asynchronously through a VM Component.

This recipe will show you how a Camel route can call another route running in a different Camel context.

Getting ready

In order for messages to be passed between routes using the vm: or direct-vm: transports, the exact same instance of the camel-core library must be available within the classloader hierarchy of both applications.

In an application server such as Apache Tomcat, this means placing the camel-core.jar file in the /ext directory of the server. Applications deployed onto the server should not contain camel-core within their WAR files.

When using an OSGi container such as Apache Karaf or Apache ServiceMix, it is simply a case of ensuring that the camel-core bundle is installed and running.

No additional work is necessary to use these transports if you intend to communicate between Camel contexts in the same application.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.vm package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with vm.

How to do it...

Create a route to be shared consuming (from) with the vm: endpoint, and reference it from other routes using the same endpoint name.

  1. Split out the integration logic that you want to share into a route and select a VM Component to consume from that best suits your threading requirements. If your route should handle requests using the same thread as the caller, choose direct-vm:, otherwise if you want the exchange to be processed asynchronously by a different thread, choose vm:. Give the endpoint a name that will be unique across the JVM.

    In the XML DSL, this is written as follows:

    <route>
      <from uri="vm:logMessageToBackendSystem"/>
      <to uri="..."/>
    </route>

    In the Java DSL, you express the same thing as:

    from("vm:logMessageToBackendSystem").to(...);
  2. Invoke the shared route from within the top-level route that needs to make use of this logic by using the same URI prefix.

    In the XML DSL, write the following:

    <route>
      <from uri="..."/>
      <to uri="vm:logMessageToBackendSystem"/>
    </route>

    In the Java DSL, this is written as:

    from(...).to("vm:
    logMessageToBackendSystem");

How it works...

The VM Component is equivalent in functionality to SEDA, but works across applications within the same JVM. All of the same configuration options for a seda: endpoint also apply to the vm: endpoint. As in seda: endpoints, care should be taken that the endpoint name used in the top-level route matches that of the shared route. Otherwise, the exchange is placed onto a named in-memory queue that will never be consumed from.

Aside from working across applications, direct-vm: is functionally equivalent to direct:. The endpoint uses the same thread between the two applications, meaning that it can participate in the same transactions. This should be tested thoroughly to ensure that applications have been correctly configured. For this to work, both applications should make use of the same transactional resources (for example, JDBC DataSource or JMS ConnectionFactory) with the same transaction manager.

Note

When using direct-vm: you should pay particular attention to the order in which the applications or contexts are started up. You can use the block=true option on direct-vm: so that it will block until there is an active consumer associated with it. If the top-level route starts sending messages to the shared route before it is available, an exception similar to the following will be thrown by the Camel runtime:

org.apache.camel.CamelExchangeException : No consumers available on endpoint: Endpoint[direct-vm://someMissingEndpoint]
 

Using external properties in Camel routes


One of the many nice features in Spring is the ability to use property placeholders such as ${database.url} to externalize information outside the application in a properties file. This allows your application's deployable artifacts to be built once and move through environments such as development, system test, UAT, and production, each time changing their behavior based on those external values. Camel provides a corresponding mechanism that you can use to externalize values used within routes.

This recipe will show you an approach for externalizing values, such as host name and port number, such that those values can be changed independent of the routing code.

Getting ready

Define a propertyPlaceholder tag inside the camelContext element:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <propertyPlaceholder
      id="properties"
      location="classpath:placeholder.properties"/>
  <!-- other code here -->
</camelContext>

Properties contained in the file placeholder.properties can then be used directly inside your Camel route using placeholders.

Note

The id value "properties" is mandatory.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.propertyplaceholder package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with propertyPlaceholder.

How to do it...

The placeholder syntax is different from the usual Spring ${..} format, in that properties are delimited by {{ to start and }} to end.

<route>
  <from uri="{{start.endpoint}}"/>
  <transform>
    <simple>{{transform.message}}: ${body}</simple>
  </transform>
  <to uri="{{end.endpoint}}"/>
</route>

Consider the following properties file content:

start.endpoint=direct:in
transform.message=I hear you
end.endpoint=mock:out

Configured with these, the preceding route will consume a message from an in-memory endpoint, prefix the body with I hear you and send the result to a mock endpoint for testing.

How it works...

This bridging functionality is necessary since Spring has some limitations in terms of allowing third-party libraries to use its property placeholder mechanism.

The location URI scheme can take the following forms:

ref:

Uses a named java.util.Properties object defined in the context

file:

Refers to a fixed path on the filesystem

classpath:

Refers to a file within the current application

Location URIs can themselves contain placeholders for JVM system properties and environment variables:

file:${karaf.home}/etc/application.properties
file:${env:CATALINA_HOME}/etc/application.properties

Instead of using a property-placeholder tag, you can also define a PropertiesComponent object in the Spring context with id as "properties", and it will be integrated as expected:

<bean id="properties"
      class="org.apache.camel.component.properties.PropertiesComponent">
  <property name="location"
            value="classpath:placeholder.properties"/>
</bean>

This also works when using Camel directly from a Java application without Spring:

PropertiesComponent properties = new PropertiesComponent();
properties.setLocation("classpath:placeholder.properties");
camelContext.addComponent(properties);

The placeholder mechanism just shown also works when defining routes in Java. The following route works as expected inside a Camel context that has a properties placeholder configured:

from("{{start.endpoint}}")
  .transform().simple("{{transform.message}}: ${body}")
  .log("Set message to ${body}")
  .to("{{end.endpoint}}");

There's more...

Camel provides a drop-in replacement for a Spring PropertyPlaceholderConfigurer, that enables Spring beans to be initialized with ${…} configuration while allowing Camel logic to make use of {{…}} placeholders, with the one piece of configuration.

<bean id="bridgePropertyPlaceholder"
      class="org.apache.camel.spring.spi.BridgePropertyPlaceholderConfigurer">
  <property name="location"
            value="classpath:placeholder.properties"/>
</bean>

See also

 

Reusing endpoints


When an endpoint is going to be used multiple times in a Camel context it is preferable to define it at a single place so that it is used consistently. This recipe will show you a way to do that.

Getting ready

Define your desired routing logic as described in either the Using Camel in a Java application recipe, or the Embedding Camel in a Spring application recipe.

How to do it...

In the XML DSL, define an <endpoint/> element with an id attribute and an uri attribute setting that is set to the URI value you wish to share:

<camelContext xmlns="...">
  <endpoint id="restfulOrdersService"
            uri="jetty:http://localhost:8080/orders"/>
  <route>
    <from ref="restfulOrdersService"/>
    <!-- ... -->
  </route>
</camelContext>

If using the Java DSL, simply define the URI as a String within RouteBuilder configure() method:

String restfulOrdersService =
    "jetty:http://localhost:8080/orders";

from(restfulOrdersService) //...
 

Reusing routing logic through template routes


One of the key advantages of using Java for defining routes is the ability to define the same, or similar, routing logic multiple times in your integrations, while changing key elements.

Consider the case of a route that:

  • Consumes bulk order data from CSV files in an input directory

  • Splits it into individual orders

  • Extracts the date of each order, formatted specific to the country, and converts it to a universal one

  • Places an order confirmation into another directory

Now consider that you may have orders from dozens of different countries, with different order and confirmation directories, and different date formats.

You could write similar routes dozens of times, but that is going to create a maintenance problem. Alternatively, using Camel's Java DSL, you can write the common routing logic once, and then use dependency injection to vary the values that are different when you instantiate the route.

This recipe will show you a strategy for creating Camel routes that can be created with different values at runtime, parameterizing your common routing logic.

Getting ready

Define your route within a RouteBuilder as usual, only this time make the start and end URIs, as well as any beans involved in the processing, properties of the RouteBuilder class:

public class OrderProcessingRouteBuilder extends RouteBuilder {
  String inputUri;
  String outputUri;
  private OrderFileNameProcessor orderFileNameProcessor;

  @Override
  public void configure() throws Exception {
    from(inputUri)
      // split into individual lines
      .split(body(String.class).tokenize("\n"))
        .process(orderFileNameProcessor)
        .log("Writing file: ${header.CamelFileName}")
        .to(outputUri)
      .end();
  }
}

Note

Note that the URI variables are defined as package scoped. This will help us to test the class later.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.templating package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with templating.

How to do it...

Use property setters on your RouteBuilder implementation to instantiate multiple instances of the route with different property values injected.

  1. Add setters for the properties:

    public void setInputDirectory(String inputDirectory) {
      inputUri = "file://" + inputDirectory;
    }
    public void setOutputDirectory(String outputDirectory) {
      outputUri = "file://" + outputDirectory;
    }
    public void setOrderFileNameProcessor(
        OrderFileNameProcessor orderFileNameProcessor) {
      this.orderFileNameProcessor = orderFileNameProcessor;
    }

    Tip

    A useful trick is to construct the endpoint URIs within the setters. This way, when instantiating the RouteBuilder, you only have to worry about which directories to use, not about the various additional attributes that you want to repeat for the file component each time.

  2. Validate that the mandatory bean properties were set. The following code uses the org.apache.commons.lang.Validate class to check for nulls and empty Strings:

    @PostConstruct
    public void checkMandatoryProperties() {
      Validate.notEmpty(inputUri, "inputUri is empty");
      Validate.notEmpty(outputUri, "outputUri is empty");
      Validate.notNull(orderFileNameProcessor,
                       "orderFileNameProcessor is null");
    }

    Tip

    If you are using the RouteBuilder from Spring, add a @PostConstruct method to check that all of the properties have been set. This way, if all of the fields have not been initialized correctly, the application will refuse to start up.

  3. To complete the integration we need to add a Processor that parses dates from a line of CSV text, changes the date to a universal format, and sets a header with the output filename. We encapsulate this logic in a class whose instances vary by a date format that is injected. The source for this class is available in the example code under: org.camelcookbook.structuringroutes.templating.OrderFileNameProcessor.

  4. In your Spring XML file, you can now create multiple instances of this class:

    <bean id="dateFirstOrderFileNameProcessor"
          class="org.camelcookbook.structuringroutes.templating.OrderFileNameProcessor">
      <property name="countryDateFormat" value="dd-MM-yyyy"/>
    </bean>
    
    <bean id="monthFirstOrderFileNameProcessor"
          class="org.camelcookbook.structuringroutes.templating.OrderFileNameProcessor">
      <property name="countryDateFormat" value="MM-dd-yyyy"/>
    </bean>
  5. We now have all of the pieces in place to perform the same integration for a number of countries that use different input and output directories, and date formats, within their order files. We can now go ahead and instantiate the RouteBuilders and inject them into the Camel context:

    <bean id="ukOrdersRouteBuilder"
          class="org.camelcookbook.structuringroutes.templating.OrderProcessingRouteBuilder">
      <property name="inputDirectory"
                value="/orders/in/UK"/>
      <property name="outputDirectory"
                value="/orders/out/UK"/>
      <property name="orderFileNameProcessor"
                ref="dateFirstOrderFileNameProcessor"/>
    </bean>
    
    <bean id="usOrdersRouteBuilder"
          class="org.camelcookbook.structuringroutes.templating.OrderProcessingRouteBuilder">
    <property name="inputDirectory"
              value="/orders/in/US"/>
      <property name="outputDirectory"
                value="/orders/out/US"/>
      <property name="orderFileNameProcessor"
                ref="monthFirstOrderFileNameProcessor"/>
    </bean>
    
    <camelContext 
        xmlns="http://camel.apache.org/schema/spring">
      <routeBuilder ref="ukOrdersRouteBuilder"/>
      <routeBuilder ref="usOrdersRouteBuilder"/>
    </camelContext>

How it works...

By treating our RouteBuilder implementation as just another bean to use in a Spring context, we were able to instantiate it multiple times, introducing varying behavior by changing the injected values. In the future, if we were to change the routing logic, perhaps by adding more logging, it would all be done in one place in the code.

When we defined our URI properties in the RouteBuilder, we set them as package scoped. This is a handy strategy that allows us to inject endpoint types from within the same package that are not file: endpoints, which are set when our public setter methods are used. Since test classes are typically co-located in the same package, this allows us to initialize our RouteBuilder with more easily testable endpoints:

OrderFileNameProcessor processor = new OrderFileNameProcessor();
processor.setCountryDateFormat("dd-MM-yyyy");

OrderProcessingRouteBuilder routeBuilder =
    new OrderProcessingRouteBuilder();
routeBuilder.inputUri = "direct:in";
routeBuilder.outputUri = "mock:out";
routeBuilder.setOrderFileNameProcessor(processor);

See Chapter 9, Testing, for more details on testing.

See also

 

Controlling route startup and shutdown


When integration logic is composed of routes depending on other routes via direct:, it is important that they start up in such a way that dependencies are available before exchanges start flowing. If not, you are likely to see this sort of exception being thrown:

org.apache.camel.CamelExchangeException: No consumers available on endpoint: Endpoint[direct://someMissingEndpoint]

Conversely, on application shutdown, messages should complete processing gracefully rather than fail because a dependent route is no longer available. Camel provides a mechanism to define an order for startup and shutdown that addresses both of these issues at the same time.

This recipe will show you how to control the startup and shutdown order of your routes.

Getting ready

Define your desired routing logic as described in either the Using Camel in a Java application recipe, or the Embedding Camel in a Spring application recipe.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.routecontrol package.

How to do it...

Set the startupOrder property associated with the route.

In the XML DSL, add a startupOrder attribute to the route element:

<route startupOrder="20">
  <from uri="jms:queue:orders"/>
  <to uri="direct:processOrder"/>
</route>

<route startupOrder="10">
  <from uri="direct:processOrder"/>
  <process ref="orderProcessor"/>
</route>

In the Java DSL, call the startupOrder(..) method after the from statement:

from("jms:queue:orders").startupOrder(20)
  .to("direct:processOrder");

from("direct:processOrder").startupOrder(10)
  .process(new OrderProcessor());

How it works...

Routes are started in ascending startupOrder. In the preceding examples, the direct: route will be started before the main entry point to the integration, which consumes from JMS.

You can assign any integer greater than 0 and less than 1,000 to the startupOrder. You cannot assign the same value more than once in a Camel context, otherwise it will refuse to start up. Camel will automatically assign values greater than 1,000 to any routes that do not specifically have one defined.

Tip

Drawing inspiration from BASIC programming (for those old enough to remember it), it is useful to assign startupOrder values in increments of 10. As your integration grows and you find yourself breaking routes down for reuse, it pays to have numbers available so that you do not have to renumber every route.

When you shut down the application, Camel will turn off the routes in the reverse order to that in which it started them. Routes are turned off in descending order.

When a route shuts down, the endpoint consumer is first turned off, and any messages that are flowing through the route ("in-flight") are allowed to complete before the route itself is shut down. Any messages that remain in-flight will be discarded after a timeout of 300 seconds. The timeout is configurable on the Camel context's associated ShutdownStrategy.

There's more...

Routes can be started and shut down programmatically through the CamelContext startRoute() and stopRoute() methods. Since the context is accessible through an Exchange it is possible to perform custom route control logic. Take, as an example, the following class that stops a route whose name is specified by the body of the exchange:

public class RouteStoppingProcessor implements Processor {
  @Override
  public void process(Exchange exchange) throws Exception {
    final String routeName = 
        exchange.getIn().getBody(String.class);
    final CamelContext context = exchange.getContext();
    new Thread(new Runnable() {
      @Override
      public void run() {
        try {
          context.stopRoute(routeName);
        } catch (Exception e) {
          throw new RuntimeException(e);
        }
      }
    }).start();
  }
}

Note

It is best practice to manually shut down routes in Camel in a separate thread than in the one that is processing the exchange. The reason behind this is that Camel waits for all exchanges that are flowing through a route to complete before it stops the route. If a thread processing an exchange attempts to shut down the route through which that exchange is flowing, this results in a deadlock.

See the following for more details: http://camel.apache.org/how-can-i-stop-a-route-from-a-route.html.

Routes can also be stopped and started through the use of a Control Bus endpoint:

from("direct:in").id("mainRoute")
  .log("Stopping route")
  .to("controlbus:route?routeId=mainRoute&action=stop&async=true")
  .log("Signalled to stop route")
  .to("mock:out");

Tip

Note the use of async=true to shut down the route in a background thread and thereby prevent a deadlock.

Manual route control does not take into account the startupOrder, so you must take care when performing it that any routes that you start up or shut down are controlled in an orderly manner.

Routes can be turned off at startup by setting the autoStartup attribute to false.

In the XML DSL, add an autoStartup attribute to the route element:

<route autoStartup="false">
  <from uri="jms:queue:orders"/>
  <!-- ... -->
</route>

In the Java DSL, call the autoStartup(..) method after the from statement:

from("jms:queue:orders").autoStartup(false)...

You would use this feature if you want to control route availability manually, or through a route policy.

A RoutePolicy is an interface that you can build upon to determine whether routes should be running or not. Camel provides route policies out of the box for throttling routes (ThrottlingInflightRoutePolicy), defining uptime through a timer (SimpleScheduledRoutePolicy), and cron expressions (CronScheduledRoutePolicy).

About the Authors

  • Scott Cranton

    Scott Cranton is an open source software contributor and evangelist. He has been working with Apache Camel since the release of version 1.5 almost 5 years ago, and has over 20 years of commercial experience in middleware software as a developer, architect, and consultant. During his time at FuseSource, and now Red Hat, he has worked closely with many core committers for Apache Camel, ActiveMQ, ServiceMix, Karaf, and CXF. He has also helped many companies successfully create and deploy large and complex integration and messaging systems using Camel and other open source projects.

    He divides his professional time between hacking code, delivering webinars on using Camel and open source, and helping companies to learn how to use Camel to solve their integration problems.

    Browse publications by this author
  • Jakub Korab

    Jakub Korab is a consulting software engineer specializing in integration and messaging. With a formal background in software engineering and distributed systems, in the 14 years that he has worked in software across the telecoms, financial services, and banking industries, he naturally gravitated from web development towards systems integration. When he discovered Apache Camel, it became apparent to him how much time and effort it could have saved him in the past compared to writing bespoke integration code, and he has not looked back since.

    Over the years, working as a consultant, he has helped dozens of clients build scalable, fault-tolerant, and performant systems integrations. He currently runs his own specialist consultancy, Ameliant, which focuses on systems integration and messaging using a stack of integration products from the Apache Software Foundation, of which Camel is a corner stone.

    When not gluing systems together, you will find him spending time with his young family, and far too infrequently kitesurfing or skiing—neither of which he gets much chance to do in his adopted home, London.

    Browse publications by this author

Latest Reviews

(4 reviews total)
Goed naslagwerk met veel duidelijke voorbeelden. Geen overbodige tekst. Maar het is geen uitvoerig naslagwerk.
Examples using DSL using diverse techniques and commands. Very recommended for daily basis
Book is a little bit dated. Some methods are deprecated, not so useful.

Recommended For You

Book Title
Access this book, plus 7,500 other titles for FREE
Access now