Wednesday, March 17, 2010

Enterprise Application Integration (EAI) with Integration Engine

Every application lives in an ecosystem of different types of software. Some of them are platform software and you use by visibly calling some APIs or invisibly behind the JVM. Application Servers, HTTP Servers, DBMSs are platform software for enterprise applications. EAI is somewhat different. It is in application to application (A2A) or business to business (B2B) style.

Although ERP systems try to meet all business application requirements of an enterprise, IT systems generally have many different applications for some reasons. Sometimes those applications need to share data because business process may span across those applications. Every application manages only a piece of business process. The need for integration of those applications depends on data flow volume. If data volume is high, users wouldn’t want to enter same data to each application again and again (sometimes this maybe impossible). Application integration prevents data errors, diminishes user data entry but it has a cost. Best option for integration is not to do it. Every integration project requires detailed analysis. Every integration decreases application functionality and increase system fragility. For these reasons, ERP systems are indispensable for their highly integrated architecture when imagining islands of many applications.

For integration requirements, ERP systems have some integration capabilities from direct table integration to web service functions. In core modules of ERP, inter-module function calls already requires an integration layer. ERP system exposes those functions to the outside users.

We also recently added this capability to our ERP system. I’m going to share some architectural design tips. At the beginning, when analyzing an integration system, we were considering using messaging system as an integration backbone. However as walking through requirements, I realized that messaging system can only be a part of integration, i.e. for EDI (Electronic Data Interchange). Messaging system usage for file integration would be somewhat cumbersome. Then, “Integration Engine” idea shined. Yes, a new engine is best solution for integration functions. We worked previously to add EDI integration to the messaging system, but we realized that “Integration Engine” is best-suited and neater for the whole integration problem.

Let me summarize, what we thought for function set of “Integration Engine”:
• Integration should be easily developed and plugged to the running system without any compilation which may require IDE. This could be possible by leveraging dynamic programming facility of Rule Engine (see my previous article for that).

• Integration should be possible for any system that Java can talk. Yes, since integration is done through Java programming, we are only limited to Java (By calling web services from Java, this border also expanded). We have supported following type of integrations; Server File, Client File (From Browser), Web Service, EDI Message, Other.

• Integration should work outbound or inbound. That means we can export data to the outer system or import data form outer system.

• Integration functions can be triggered by outer system via web service APIs.

• Integration can work as a batch process. Let’s say you need to continuously monitor a file which is filled by an outer program.

• Integration can be queued. You may need sometimes to approve data then take it to the system. Let’s say you have a web service and want to let data entry to system by controlling validity.

• When running integrations, user may need to give some extra data or edit some data. We designed some integration pages for users in order to run integration, to give parameters and to edit data coming from integration source.

• Field format can be specified. Data format is one of the problems in integration efforts. We provided to give format definition (we used Java format definitions) for numbers, date, time or timestamp.

• Excel integration. Spreadsheet applications are most-wanted applications for internal integration. We make it possible via macro calls to our web services (More portable solution for different spreadsheet applications when compared with platform-specific integration methods).

• Integration usage should be simple. We added an integration link to the module main page to easily navigate between data and integration page.



An integration engine is simply composed of 2 parts; Communicator and Processor. In both export and import situations, these 2 parts are used. These parts are dynamic Java programs and communicate with each other with generic data objects. Communicator is responsible for outer systems. It connects to outer system and pushes or pulls data. Then, it builds a generic data object and passes it to Processor or vice versa. Processor is responsible for dealing with internal system. In export, it access to the internal system via integration APIs and then builds generic data objects and pass it to the Communicator. In import, it receives data objects from Communicator then it calls integration APIs for saving data.

The architecture is as following:


1 comment:

Lydia Justin said...

I would like to say one good thing regarding QA testing
Load & performance testing for ERP application by using our software testing service
Indium software provides Outsourced and In-housing Manual & Regression Test Automation Testing Services across the world. Testing & Enhancing the performance of your ERP mobile or web application under load and stress. We have a strong expertise in cloud based application testing, Game & Mobile testing, Social-Digital media testing and Enterprise apps testing.