Apache Camel Developer's Cookbook

The minimal set of libraries typically required to use Camel within a Maven build are:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-core</artifactId>
  <version>${camel-version}</version>
</dependency>
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-test</artifactId>
  <version>${camel-version}</version>
  <scope>test</scope>
</dependency>

The Camel version is usually defined in a properties block within the Maven POM file once, as follows, so as to not need to be repeated over and over:

<properties>
  <camel-version>2.12.2</camel-version>
</properties>

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.simple package.

The CamelContext interface is the heart of the Camel framework. It is responsible for processing messages along routes.

The from(...) statement at the start of a route defines an endpoint, or a technology-specific location, that the Camel routing engine uses to fetch messages. Endpoints are defined using URIs, such as in the preceding example, timer:logMessageTimer. The first part of the URI specifies the component that is being used to consume the message, and the remaining is a set of instructions for that specific component. See the Using Camel components recipe in this chapter for more details.

The Camel routing engine consumes exchanges from these endpoints and processes them sequentially through each of the steps defined in the route. The engine is responsible for threading, transactions, error handling, copying messages where required, and many other details.

The Camel context is a long-running object; it is intended to live for as long as the application does, and therefore its initialization and shutdown is usually tied to the lifecycle of the application. Typical deployments of Camel define the context within:

Routes, which are definitions of the steps that messages should be processed through, are typically added to the newly created context, though they can be added, removed, and modified at runtime. Route definitions can only be added to a context before the context is started, though they can be stopped and restarted while the context is running.

Extending the RouteBuilder abstract class gives access to Camel's Java route definition DSL, or simply the Java DSL. What this means in practice is that within the mandatory configure() method, after typing the first from(...) statement that defines the start of a route, you get context-specific code completion of whichever integration patterns you might be using.

A RouteBuilder implementation may implement one or many routes. That is, within the configure() method, you can specify multiple from(...) statement that Camel will translate into multiple runtime route instances, one per from(...) statement.

Camel is a highly configurable framework, in which most behaviors can be customized through service provider interfaces (SPIs). An SPI encapsulates a single behavior, such as a route naming strategy (Camel gives your routes sensible names if you do not do so explicitly). To override the default behavior, you provide your own implementation of the SPI class and set it on the CamelContext object. The context allows you to define the following, amongst others:

It is therefore worthwhile getting familiar with the options that this class gives you by going over the Javadocs.

The CamelContext interface makes use of an internal object registry that allows it to look up objects by name. When using a DefaultCamelContext, a JNDI-aware registry is used.

This feature is used extensively throughout the framework for finding components, thread pools, named processor beans, data formats, and the like.

Occasionally, it is necessary to add objects directly to the registry, as in the case of beans that you want to execute, as one of the processing steps in a route. To do this, instantiate an implementation of org.apache.camel.spi.Repository, usually org.apache.camel.impl.SimpleRegistry, and pass it into the constructor of the DefaultCamelContext:

SimpleRegistry registry = new SimpleRegistry();
registry.put("payloadTransformer", new MyPayloadTransformer());
CamelContext context = new DefaultCamelContext(registry);

The CamelContext interface defines type-safe utility methods for setting certain object types, such as components, that allow you to set them without worrying about the registry internals.

Consider the following manual step:

registry.put("mylogger", new LogComponent());

This can be written in a type-safe way as follows:

context.addComponent("mylogger", new LogComponent());

The Registry in Camel can hold any named Java instance, and these instances can be referenced by name from the Camel DSL. The addComponent method of the CamelContext is specifically used for registering Camel components by name. Both approaches do effectively the same thing, though there are some subtle differences, and we would recommend using the addComponent method for components, and adding all your POJOs and custom processors into the registry.

When using Camel within a Spring application, it is necessary to add the following dependencies to the minimal set defined in the Using Camel in a Java application recipe in this chapter:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-spring</artifactId>
  <version>${camel-version}</version>
</dependency>
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-test-spring</artifactId>
  <version>${camel-version}</version>
  <scope>test</scope>
</dependency>

The ${camel-version} property is defined once in the Maven POM.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.simplespring package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with simplespring.

In order to embed Camel into a Spring application, perform the following steps:

Camel was designed to be closely integrated with Spring from its inception. The camelContext element results in a SpringCamelContext object being created, initialized with any routes defined within it, and started when the Spring context starts up. The camelContext element is itself a Spring managed object that can optionally be given an ID and treated like any other bean.

The preceding example shows Camel's XML DSL being used. One of the nice things about the DSL is that an XML schema is used to define it. This means that it is possible for your IDE to provide you with code completion.

It is not mandatory to use the XML DSL with Spring. It is possible to use the Java DSL instead, or alongside routes defined through the XML DSL.

To plug in the route defined in the LogMessageOnTimerEventRouteBuilder class that we used in the previous recipe, we first instantiate it as a bean:

<!-- package name has been abbreviated -->
<bean id="logMessageOnTimerEvent"
      class="org.camelcookbook.structuringroutes.simple.LogMessageOnTimerEventRouteBuilder"/>

Then we add it to the camelContext element using the routeBuilder tag:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <routeBuilder ref="logMessageOnTimerEvent"/>
</camelContext>

Multiple routeBuilder elements can be used within a camelContext.

If you define a number of RouteBuilders in the same package, it is possible for Camel to scan that package and instantiate all of the routes that it finds:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <packageScan>
    <package>org.camelcookbook.structuringroutes</package>
  </packageScan>
</camelContext>

You can add multiple package elements within the packageScan element, and also use wildcards to include or exclude RouteBuilders by name, using the excludes and includes elements.

Spring provides an alternative feature called component scanning. When enabled, the Spring application context recursively scans a package, and instantiates any class within that is annotated with org.springframework.stereotype.Component. Any properties annotated with @Autowired, or the CDI equivalent @Inject, have their dependencies injected. Camel can be configured to pick up any RouteBuilders wired through this process. The RouteBuilders must first be marked as components:

@Component
public class LogMessageOnTimerEventRouteBuilder
    extends RouteBuilder {
  //...
};

To enable the wiring, turn on component scanning in Spring:

<component-scan
  base-package="org.camelcookbook.structuringroutes"
  xmlns="http://www.springframework.org/schema/context"/>

Then add the appropriate feature to the Camel context to tie it all together:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <component-scan/>
</camelContext>

To make use of a component, you will need to make sure that the component's library is included in your project. The camel-core library provides a set of fundamental components to get you started underneath the org.apache.camel.component package.

For integration with any other technologies, your first stop should be the component listing on the Camel website (http://camel.apache.org/components.html). Once you have found the technology that you are looking for, add the JAR dependency to your project, ensuring that the version matches that of the camel-core library you are using. For example, to use the Camel FTP component within your Camel route, you need to add the camel-ftp dependency to your POM:

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-ftp</artifactId>
  <version>${camel-version}</version>
</dependency>

The ${camel-version} property is defined once in the Maven POM.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.simple package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with simplespring.

In order to use a Camel component you need to instantiate and register the component, and then reference it from your Camel route as per the following steps:

To understand exactly what happens when we use an endpoint URI, it is easiest to take a brief look under the covers at the classes that Camel uses within its component framework.

A Component is a factory for creating Endpoints that can act as message Consumers, or Producers, or both. An implementation will typically have bean properties on it that will apply to the transport as a whole. For example, the JMS Component requires that a ConnectionFactory be set on it, in order to make use of the same message broker for all JMS communication:

<bean id="myFavouriteMQ"
      class="org.apache.camel.component.jms.JmsComponent">
  <property name="connectionFactory"
            ref="myConnectionFactory"/>
</bean>

All Components implement a method used to parse the endpoint URI:

Endpoint createEndpoint(String uri) throws Exception;

An Endpoint can be thought of as an address that is specific to the component technology–it is instantiated from the URI by the framework when the route using it is started. The scheme portion of the URI, the part before the first colon (:), identifies the component implementation being used.

Continuing the JMS example, given that the Component already knows where a message broker is, the Endpoint itself describes which queues or topics a message should be sent to or received from, and how:

myFavouriteMQ:someQueue?concurrentConsumers=5

The portion of the URI after the scheme is specific to the endpoint technology. The URI properties themselves, such as concurrentConsumers, correspond to bean properties on the endpoint implementation for that technology, and are set using introspection. If you were to take a look inside the JMS Component library, you would see a JmsEndpoint object with a setConcurrentConsumers(int consumers) method.

You should always refer back to the applicable Camel component page for a full list of properties that can be used.

The Endpoint is also a factory. Camel uses it for creating Producers and Consumers, depending on the context of where the URI is used. The following factory methods are defined on the interface:

Producer createProducer() throws Exception;
Consumer createConsumer(Processor processor) throws Exception;

These classes handle the heavy lifting of talking to the underlying technology.

If the endpoint URI is used in a from(...) statement, Camel will create a Consumer. If it is used in a to(...) block, will create a Producer.

The same component can be instantiated multiple times with different IDs. For example, two JMS components might be used when writing routing logic to bridge message brokers from different vendors.

Determine the routing logic that you would like to reuse, and move it to a route that consumes from a direct: URI.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.direct package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with direct.

Create a route with the from URI using the direct: endpoint, and then in another route, call the shared route using the same direct: endpoint.

Each direct: endpoint may only be consumed (used within the from(...) block) by one route in a Camel context. However, multiple routes may produce, or send, messages to that URI. This allows you to compose your routing logic in easily understandable blocks, much as you would when using methods in a regular Java program.

The shared route can be considered as a part of the top-level route. It operates on the same Exchange object, and participates in any transactions that the top-level route has initiated.

Regardless of whether the message exchange pattern on the exchange is InOnly or InOut, the behavior of invoking a direct: endpoint is the same; that is, the exchange will flow through all of the processing steps defined in the shared route and will be returned to the calling route on completion where it will proceed to the next processing step.

It is possible to invoke the shared route, and merge the returned exchange with the exchange containing the original state before the route was invoked. See the Enriching your content with some help from other endpoints recipe in Chapter 4, Transformation, for more details.

Split out the long running steps into their own shared routes, and assign them with a seda: endpoint with a name that is unique to the Camel context.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.seda package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with seda.

Create a shared route using the seda: endpoint, and then call it from other routes using the same named seda: endpoint.

In the preceding example, a timer: endpoint is used to trigger messages on a regular basis, every 200 ms. The Timer Component uses one thread per timer name (ping). An event can only be raised 200 ms later if the thread is not processing the previous exchange.

As part of our integration, we want to trigger events regularly, and yet have a long-running processor as part of the route. Camel allows us to deal with this scenario by splitting the long-running part into a shared route, and linking the two routes with a seda: endpoint.

When an exchange is passed to a seda: endpoint, it is placed into a BlockingQueue. The list exists within the Camel context, which means that only those routes that are within the same context can be joined by this type of endpoint. The queue is unbounded by default, although that can be changed by setting the size attribute on the URI of the consumer.

By default, a single thread assigned to the endpoint reads exchanges off the list and processes them through the route. As seen in the preceding example, it is possible to increase the number of concurrentConsumers to ensure that exchanges are getting processed from that list in a timely fashion.

Assuming our slow processor takes 3,000 ms to complete, we would need to use a number of threads equal to the processing time/the timer frequency to ensure that the triggered events are processed in a timely fashion. Therefore, plugging in the numbers, 3,000 ms / 200 ms, we arrive at 15 threads.

It is possible to define multiple routes that consume from the same logical name, unlike a direct: endpoint. To enable this, both routes should set multipleConsumers=true on the seda: URI:

from("seda:timerEvents?multipleConsumers=true")
  // define one set of routing logic here

from("seda:timerEvents?multipleConsumers=true")
  // another here

The effect will be that each route gets its own copy of the exchange, making it a sort of simple in-memory publish-subscribe system. It is often much cleaner to handle this type of requirement using the Multicast pattern. See the Multicast – routing the same message to many endpoints recipe in Chapter 2, Message Routing, for more details.

The SEDA pattern is best suited to processing the InOnly messages, where one route finishes processing and hands off to another to deal with the next phase. It is possible to ask for a response from a seda: endpoint by calling it when the message exchange pattern is InOut.

In this instance, the endpoint will act much like a synchronous direct: endpoint, only with a timeout if the task runs for longer than expected. The default timeout is 30 seconds, but this can be overridden in the producer URI using the timeout attribute.

To trigger a background job to run based on the current exchange while your main route continues processing, refer to the Wire Tap EIP. See Wire Tap – sending a copy of the message elsewhere recipe in Chapter 2, Message Routing, for more details.

In order for messages to be passed between routes using the vm: or direct-vm: transports, the exact same instance of the camel-core library must be available within the classloader hierarchy of both applications.

In an application server such as Apache Tomcat, this means placing the camel-core.jar file in the /ext directory of the server. Applications deployed onto the server should not contain camel-core within their WAR files.

When using an OSGi container such as Apache Karaf or Apache ServiceMix, it is simply a case of ensuring that the camel-core bundle is installed and running.

No additional work is necessary to use these transports if you intend to communicate between Camel contexts in the same application.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.vm package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with vm.

Create a route to be shared consuming (from) with the vm: endpoint, and reference it from other routes using the same endpoint name.

The VM Component is equivalent in functionality to SEDA, but works across applications within the same JVM. All of the same configuration options for a seda: endpoint also apply to the vm: endpoint. As in seda: endpoints, care should be taken that the endpoint name used in the top-level route matches that of the shared route. Otherwise, the exchange is placed onto a named in-memory queue that will never be consumed from.

Aside from working across applications, direct-vm: is functionally equivalent to direct:. The endpoint uses the same thread between the two applications, meaning that it can participate in the same transactions. This should be tested thoroughly to ensure that applications have been correctly configured. For this to work, both applications should make use of the same transactional resources (for example, JDBC DataSource or JMS ConnectionFactory) with the same transaction manager.

org.apache.camel.CamelExchangeException : No consumers available on endpoint: Endpoint[direct-vm://someMissingEndpoint]

Define a propertyPlaceholder tag inside the camelContext element:

<camelContext xmlns="http://camel.apache.org/schema/spring">
  <propertyPlaceholder
      id="properties"
      location="classpath:placeholder.properties"/>
  <!-- other code here -->
</camelContext>

Properties contained in the file placeholder.properties can then be used directly inside your Camel route using placeholders.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.propertyplaceholder package. Spring XML files are located under src/main/resources/META-INF/spring and prefixed with propertyPlaceholder.

The placeholder syntax is different from the usual Spring ${..} format, in that properties are delimited by {{ to start and }} to end.

<route>
  <from uri="{{start.endpoint}}"/>
  <transform>
    <simple>{{transform.message}}: ${body}</simple>
  </transform>
  <to uri="{{end.endpoint}}"/>
</route>

Consider the following properties file content:

start.endpoint=direct:in
transform.message=I hear you
end.endpoint=mock:out

Configured with these, the preceding route will consume a message from an in-memory endpoint, prefix the body with I hear you and send the result to a mock endpoint for testing.

This bridging functionality is necessary since Spring has some limitations in terms of allowing third-party libraries to use its property placeholder mechanism.

The location URI scheme can take the following forms:

Location URIs can themselves contain placeholders for JVM system properties and environment variables:

file:${karaf.home}/etc/application.properties
file:${env:CATALINA_HOME}/etc/application.properties

Instead of using a property-placeholder tag, you can also define a PropertiesComponent object in the Spring context with id as "properties", and it will be integrated as expected:

<bean id="properties"
      class="org.apache.camel.component.properties.PropertiesComponent">
  <property name="location"
            value="classpath:placeholder.properties"/>
</bean>

This also works when using Camel directly from a Java application without Spring:

PropertiesComponent properties = new PropertiesComponent();
properties.setLocation("classpath:placeholder.properties");
camelContext.addComponent(properties);

The placeholder mechanism just shown also works when defining routes in Java. The following route works as expected inside a Camel context that has a properties placeholder configured:

from("{{start.endpoint}}")
  .transform().simple("{{transform.message}}: ${body}")
  .log("Set message to ${body}")
  .to("{{end.endpoint}}");

Camel provides a drop-in replacement for a Spring PropertyPlaceholderConfigurer, that enables Spring beans to be initialized with ${…} configuration while allowing Camel logic to make use of {{…}} placeholders, with the one piece of configuration.

<bean id="bridgePropertyPlaceholder"
      class="org.apache.camel.spring.spi.BridgePropertyPlaceholderConfigurer">
  <property name="location"
            value="classpath:placeholder.properties"/>
</bean>

Define your desired routing logic as described in either the Using Camel in a Java application recipe, or the Embedding Camel in a Spring application recipe.

In the XML DSL, define an <endpoint/> element with an id attribute and an uri attribute setting that is set to the URI value you wish to share:

<camelContext xmlns="...">
  <endpoint id="restfulOrdersService"
            uri="jetty:http://localhost:8080/orders"/>
  <route>
    <from ref="restfulOrdersService"/>
    <!-- ... -->
  </route>
</camelContext>

If using the Java DSL, simply define the URI as a String within RouteBuilder configure() method:

String restfulOrdersService =
    "jetty:http://localhost:8080/orders";

from(restfulOrdersService) //...

Define your route within a RouteBuilder as usual, only this time make the start and end URIs, as well as any beans involved in the processing, properties of the RouteBuilder class:

public class OrderProcessingRouteBuilder extends RouteBuilder {
  String inputUri;
  String outputUri;
  private OrderFileNameProcessor orderFileNameProcessor;

  @Override
  public void configure() throws Exception {
    from(inputUri)
      // split into individual lines
      .split(body(String.class).tokenize("\n"))
        .process(orderFileNameProcessor)
        .log("Writing file: ${header.CamelFileName}")
        .to(outputUri)
      .end();
  }
}

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.templating package. The Spring XML files are located under src/main/resources/META-INF/spring and prefixed with templating.

Use property setters on your RouteBuilder implementation to instantiate multiple instances of the route with different property values injected.

By treating our RouteBuilder implementation as just another bean to use in a Spring context, we were able to instantiate it multiple times, introducing varying behavior by changing the injected values. In the future, if we were to change the routing logic, perhaps by adding more logging, it would all be done in one place in the code.

When we defined our URI properties in the RouteBuilder, we set them as package scoped. This is a handy strategy that allows us to inject endpoint types from within the same package that are not file: endpoints, which are set when our public setter methods are used. Since test classes are typically co-located in the same package, this allows us to initialize our RouteBuilder with more easily testable endpoints:

OrderFileNameProcessor processor = new OrderFileNameProcessor();
processor.setCountryDateFormat("dd-MM-yyyy");

OrderProcessingRouteBuilder routeBuilder =
    new OrderProcessingRouteBuilder();
routeBuilder.inputUri = "direct:in";
routeBuilder.outputUri = "mock:out";
routeBuilder.setOrderFileNameProcessor(processor);

See Chapter 9, Testing, for more details on testing.

Define your desired routing logic as described in either the Using Camel in a Java application recipe, or the Embedding Camel in a Spring application recipe.

The Java code for this recipe is located in the org.camelcookbook.structuringroutes.routecontrol package.

Set the startupOrder property associated with the route.

In the XML DSL, add a startupOrder attribute to the route element:

<route startupOrder="20">
  <from uri="jms:queue:orders"/>
  <to uri="direct:processOrder"/>
</route>

<route startupOrder="10">
  <from uri="direct:processOrder"/>
  <process ref="orderProcessor"/>
</route>

In the Java DSL, call the startupOrder(..) method after the from statement:

from("jms:queue:orders").startupOrder(20)
  .to("direct:processOrder");

from("direct:processOrder").startupOrder(10)
  .process(new OrderProcessor());

Routes are started in ascending startupOrder. In the preceding examples, the direct: route will be started before the main entry point to the integration, which consumes from JMS.

You can assign any integer greater than 0 and less than 1,000 to the startupOrder. You cannot assign the same value more than once in a Camel context, otherwise it will refuse to start up. Camel will automatically assign values greater than 1,000 to any routes that do not specifically have one defined.

When you shut down the application, Camel will turn off the routes in the reverse order to that in which it started them. Routes are turned off in descending order.

When a route shuts down, the endpoint consumer is first turned off, and any messages that are flowing through the route ("in-flight") are allowed to complete before the route itself is shut down. Any messages that remain in-flight will be discarded after a timeout of 300 seconds. The timeout is configurable on the Camel context's associated ShutdownStrategy.

Routes can be started and shut down programmatically through the CamelContext startRoute() and stopRoute() methods. Since the context is accessible through an Exchange it is possible to perform custom route control logic. Take, as an example, the following class that stops a route whose name is specified by the body of the exchange:

public class RouteStoppingProcessor implements Processor {
  @Override
  public void process(Exchange exchange) throws Exception {
    final String routeName = 
        exchange.getIn().getBody(String.class);
    final CamelContext context = exchange.getContext();
    new Thread(new Runnable() {
      @Override
      public void run() {
        try {
          context.stopRoute(routeName);
        } catch (Exception e) {
          throw new RuntimeException(e);
        }
      }
    }).start();
  }
}

Routes can also be stopped and started through the use of a Control Bus endpoint:

from("direct:in").id("mainRoute")
  .log("Stopping route")
  .to("controlbus:route?routeId=mainRoute&action=stop&async=true")
  .log("Signalled to stop route")
  .to("mock:out");

Manual route control does not take into account the startupOrder, so you must take care when performing it that any routes that you start up or shut down are controlled in an orderly manner.

Routes can be turned off at startup by setting the autoStartup attribute to false.

In the XML DSL, add an autoStartup attribute to the route element:

<route autoStartup="false">
  <from uri="jms:queue:orders"/>
  <!-- ... -->
</route>

In the Java DSL, call the autoStartup(..) method after the from statement:

from("jms:queue:orders").autoStartup(false)...

You would use this feature if you want to control route availability manually, or through a route policy.

A RoutePolicy is an interface that you can build upon to determine whether routes should be running or not. Camel provides route policies out of the box for throttling routes (ThrottlingInflightRoutePolicy), defining uptime through a timer (SimpleScheduledRoutePolicy), and cron expressions (CronScheduledRoutePolicy).