Wednesday, October 16, 2013

To integrate or not - thats' NOT the question

The real question is how to get integration done with a minimum effort and maximum future benefit.

Let me start by elaborating a bit on what I mean by integration.

Integration in my opinion the process of making two distinctly different systems work together in a well-defined, consistent and meaningful manner. For instance Dynamics AX and Dynamics CRM - or Dynamics AX and a vertical solution such as the Zynergy solution we are working on in EG Utility. By "different systems" I typically refer to different codebases which can be deployed and used independently in a meaningful way.

As noted before many people have for many years argued for all-in-one solutions like Dynamics AX. Many of them typically in order to avoid the frustration of integrations. The paradigm has been - let Microsoft worry about making it work. Why? Usually because integrations are difficult, cumbersome and provide less "richness" than something which is knitted tightly together.

But there are really just as many good arguments for why two integrated systems provide a better total solution than the all-in-one - even considering the trouble of integration. Take the example of Dynamics AX and Dynamics CRM.

AX is in my opinion (a little biased I admit) the worlds leading ERP solution. At some point some years ago some folks believed that it needed CRM functionality as well. However, that did not work out well - and later someone else invented Dynamics CRM. Which today provides much better CRM functionality.

Now, many mid-sized companies find themself in need of superb ERP functionality as well as superb CRM functionality and suddenly an integration project is born. If you then add a vertical best-of-breed solution to that mix you could end up with a three-way integration challenge.

The three-way integration project

For such project to succeed consider these aspects,
  • How often does the different systems change (think release cycle, patching etc.)
  • To which extent do upgrades impact the integration points
  • Which depth of integration is required
The total complexity of the project typically grows with each of these parameters.

Dynamics Connector
For the Dynamics product suite, Microsoft has released an extensible "Connector" which has some pre-defined integration capabilities as well as an extensible adapter framework (check out MS Partnersource for more details).

For simple integrations (such as synchronizing selected datasets) a pre-defined framework like the connector may be a good idea. It reduces the plumbing code - or rather makes it more a configuration issue - and it probably increases the quality.

For more complex integrations the connector may not be sufficient. This goes especially if the requirement is integrating an end-to-end business process across different systems rather than merely synchronizing data. In this example it is preferred that the two systems already have some notion of business processes. Furthermore that the relevant entry- and exit points in those processes are exposed as web services and preferably supporting some kind of eventing mechanism.

Regarding the Dynamics Connector I have not heard of many projects using this. It would be interesting to get comments or examples from readers of this blog.

Service integration
Integrating two systems through web services is optimal if the two systems have well-defined interfaces with services which are designed with a business scenario in mind. Using a web service method forces the two independant systems to be designed to run both independently as well as connected.

The tricky thing is to design web services with the right granularity. They should be simple enough to be consumed (used) fairly easy, yet provide the right level of autonomous funtionality. If a system ends up with a service catalog of 5000 services it's either a really really large system or the services are too granular. On the other side if a system only has 5 services it's either a really really simple system or the services are way too generic.

Integrating different systems are in no way a simple job. If the end result make people swear and curse every time they are using it you've probably done a bad job. If the users don't know they are working with two systems you've probably done a really good job. In the latter I guess you can even talk about embedded systems. Two separate systems where users sees it as one embedded in the other is rarely found  - good examples can be dropped in the comments section.