Wednesday, June 8, 2011

File Batch Splitter with Apache Camel and FuseSource Camel IDE

The Splitter from the EIP patterns allows you split a message into a number of pieces and process them individually.

To implement that EIP pattern using Apache Camel and FuseSource Camel IDE is very simple. Here is what you need:

1) Eclipse IDE (Eclipse Helios 3.6.2)
2) Apache Maven (3.0.3)
3) FuseSource Camel IDE (http://fusesource.com/products/fuse-ide-camel/)

After you install and configure the tools mentioned above, start creating a new Fuse IDE Project.


Select the Camel Archetype and type the Group Id, Artifact Id and Package name for your project then click Finish:


At this point you should have a sample Camel project and artifacts available for you. The default Camel project gives you another EIP, the Content-Based Router (http://camel.apache.org/content-based-router.html). You can open the camelContext.xml under the META-INF/spring directory and remove the Camel components available there since we're going to implement a different EIP (right-click and then Delete).

To start creating the File Batch Splitter sample project, right-click in the canvas of the camelContext.xml file and select Add > Route. We're now ready to start building our process. Right-click inside the Route rectangle and select Add > Endpoints > Endpoint.

Then, right-click again and select Add > Routing > Split. Finally, right-click and select Add > Endpoints > Endpoint. Now, click on the first created Endpoint and draw a line to the Split component. Do the same thing between the Split component and the second Endpoint.

You can also click on the Automatic Layout icon available in the Eclipse top menu bar to automatically adjust your route diagram.

Now, you have added all necessary components and should have something similar to the following picture.



You can now configure each component to execute the tasks needed to conclude the File Batch Splitter sample project. Basically, we'll consume files from one directory in the file system (an XML file with multiple records), split each record in the XML file into a new file in the file system.

To do that, click on the first endpoint and go to the Properties tab where you can see the details of each component. In the Uri field, type: file:src/data?noop=true


Highlight the Split component and type the condition you want to generate new records from the input file. In this case, we're going to use the following: //club/person as a XPath expression.



And then, highlight the final endpoint and type the following in the Uri field: file:target/output?fileName=${date:now:yyyyMMddHHmmssSSSSS}.xml
We're using a dynamic file name based on the current date to avoid conflicts and/or overriding the original files. So, every record gets a new file name when Camel creates the file.



Before we execute our Camel project, we need an input file under the src/data directory. Here is the content of a sample file to use:


<?xml version="1.0" encoding="UTF-8"?>
<club>

    <person user="marcelo">
        <firstName>Marcelo</firstName>
        <lastName>Jabali</lastName>
        <city>San Diego</city>
    </person>
    <person user="scott">
        <firstName>Scott</firstName>
        <lastName>Cranton</lastName>
        <city>Boston</city>
    </person>
</club>


The step before we execute the project is to configure how to run it. There are a couple different ways to run it but basically you can configure everything in Eclipse. Just go to Run > Run Configurations and then under Local Camel Context you can create a new configuration just pointing to the camel-Context.xml file. Click Apply and then Run.

If everything was configured correctly you should be able to see two XML files under the src/output directory containing a person record each (don't forget to refresh the view).

You also have the option to execute that project through Maven on the command line. Just navigate to the root directory of the project and type:

>mvn camel:run



That does the same thing that we executed under the Eclipse IDE.

If you want to download the files and play with the samples the project files are available on GitHub (https://github.com/FuseByExample/FileBatchSplitter)

Enjoy the ride...

Setting Up Local Environment for Developing Oracle Intelligent Bots Custom Components

Oh the joy of having a local development environment is priceless. For most cloud based solutions the story repeats itself being hard to tr...