Publishing iDocs from SAP to Logic Apps

Recently we were working on an interface building block where we wanted to register Logic Apps to receive iDocs from SAP and then to do stuff with them. We had decided to implement a pattern like below.

The idea was that the receiver Logic App would open a connection and register itself with against a program id in SAP. The SAP team would then be able to configure their model to publish IDocs that they choose to the program id which would mean they are sent to Logic Apps. The Logic App would then publish the messages to a service bus topic and we would be able to use a Publish/Subscribe pattern to be able to process the iDocs with downstream consumers.

In principal this sounds a solid approach, but in practice there were a couple of things we needed to consider in our implementation. I wanted to discuss these to help people who may need to implement a similar approach.

Program ID and How Many Listeners should I have?

I believe that you can only have 1 listener per Program ID. This meant that if we wanted multiple logic apps wanting iDocs from SAP then we would need to either choose the pattern we did or for each Logic App we would have a new bunch of configuration each time relating to both Logic Apps and SAP for the program id, the connection to SAP and the configuration in SAP. Creating 1 listening logic app for all iDocs and then using a pub/sub pattern seems to work quite well.

SAP Action Filter

In the logic app connector you have the option to configure the SAP Actions you are interested in. This will open up an explorer to allow you to navigate various SAP objects to choose the ones you want. In practice we felt that this wasnt that useful. There were scenarios where sometimes we picked a SAP action and the message would appear in a flat file style format and others where we struggled to find the right object. What we ended up doing was to use no SAP Action filter (just leave it blank) and this meant we would receive every iDoc that SAP was configured to pass to our Program ID. We would receive it everytime in an xml format. This kept the Logic App configuration simpler and meant we didnt need to add actions for every new iDoc we wanted to receive.

SAP Action as a Routing Key

When you receive an iDoc from SAP you will get a property for the SAP Action relating to the idoc message. We decided to set this as the label property on the message we published to service bus. This would make it easy to route messages using service bus subscriptions based on the label.

SAP Action in the namespaces

One of the things that was a pain was that the xml message produced by the SAP connector includes the SAP action as namespaces in a number of places within the message body and also some other namespaces which are specific to the connector.

The pain was tha the namespaces/SAP action include a version number relating to the deployment of SAP. This number can potentially change. This would cause a problem.

BizTalk used to have a way to deal with this in the BizTalk SAP adapter so that it would minimize any impact of the version number changing on downstream components. We decided to handle this in the receiver Logic App by using a regular expression to remove the namespaces from the xml. We did this with an inline javascript shape, but there are a number of ways you can do this. Effectively we were pre-processing the message to remove namespaces from the message before publishing it down downstream consumers to protect them from a change in the SAP version number.

Message Size

One problem we did have was that occasionally we would get a message that was big. In most cases they werent too bad and we were using Service Bus Premium which allows us to have a 1mb message size but there were a few cases where we would get a message that was bigger than 1mb. You can design around this limitation in a couple of ways. Unfortunately we didnt seem to have too many options to do much from the SAP side but the options you could consider include:

1. Implement a bunch of maps in the receiver logic app where you would process the idoc and strip out only key information and publish a message to service bus and then downstream logic apps would use either a REST API or BAPI's or other options to look up data from SAP when needed. It seems SAP is publishing a very rich data object and I suspect you could probably spend some time investigating how to extract just the control and data records and get rid of the rest.

2. Option 2 would involve putting the message body somewhere safe like storage and sending a message to service bus which contains a pointer to the message body which the Logic App could get and process

3. Do a router logic app rather than use service bus so you can have a routing table and it would workout which logic apps to send a message to and send the messages over http to the logic apps where there is not the same message size restrictions.

Summary

While there are a number of different things to consider, in practice we had this pattern working pretty well. Once we got over a little initial pain working out how to use the connector properly and working out what we needed the SAP Team to configure, what was needed on the On Premise Data Gateway used by the SAP connector and what the configuration for the Logic App connector and the trigger action was. When that was done it has been a fairly solid pattern for us.

There are some design considerations to make which I have discussed above and the choices are probably specific to what will work best for your organisation.