BAM with Logic Apps

In the final option we will look at how you can use Business Activity Monitoring for the solution. Serverless360 is a Microsoft ISV partner who offer a BAM solution which is commonly used by companies implementing Microsoft based solutions.

Target Users

BAM is aimed at the following users:

• Business Users
• IT Support Operators

Target Use Cases

The common use cases for BAM are:

• Business Users who want to self service their integrations
• Business Users who want visibility into their interfaces which support their key business processes
• IT Support Operators who need a non-expert friendly view of processes to manage the solution
• Testers can see inside the executing interfaces when testing integration solutions

Whats the difference between BAM and Logs

The key thing to understand is that Logging and Business Activity Monitoring is not really the same thing.

Logging is really about telemetry events which happen in your solution. Information about a bit of code being hit and some metadata about the context when that code executed.

Business Activity Monitoring is about creating an expectation of a business transaction which is supposed to be executing and then mapping of telemetry to that business transaction.

With Business Activity Monitoring you now have business telemetry rather than technical telemetry and you can monitor the status and processing of your transactions.

Prerequsites

As mentioned earlier Serverless360 is a 3rd party ISV product. You have a choice if you want to self-host Serverless360 or if you want to use the SaaS solution. There are some articles below to show you how to get started setting it up.

https://docs.serverless360.com/docs/quick-start

https://docs.serverless360.com/docs/bam-step-by-step

The focus for this article is going to be about how to use it in our demo to compare it with App Insights and Log Analytics.

Walk-thru

I will now step through building the business process.

Setting up your BAM Process

These sets will walk you through setting up the business process.

Business Process

In the Business Process / BAM section of Serverless360 you can see I have 2 business processes listed below. The bottom one is called Truck Tracker which I talked about in a previous webinar (https://www.serverless360.com/webinars/business-activity-monitoring-integration-architecture). Today Ill be looking at the business process called Factory Ingredients which is the one for this demo.

Inside the business process you can see I have 2 transactions defined. One is for management of the batch ("Process Ingredient Batch") we will receive from the suppliers and the 2nd one ("Process Ingredient Item") is for the processing of an individual record within the batch,

Business Process Transactions

In the case of this demo I will keep the transactions pretty simple, but with BAM the key part when it comes to designing your business process transactions is to make them represent a process view that would make sense to a business user. If you have a logic app with 100 shapes in it then the business user probably doesnt care about all of these shapes they might only care about a small number of key steps within the transaction. For example they may the following:

• Did the process start
• Was the message valid
• Did the destination system get updated

In BAM you could model a transaction with 3 shapes to represent your complex logic app which shows the key information. In this demo I have 2 transactions as mentioned earlier which will show that the transaction started and if it completed successfully. Below is an example of the process ingredient one.

Properties

When I define the stages within my BAM transaction I can also assign properties in them. This is a way I can take information provided from the BAM event where I can promote info from the context or a message body to be meta data which I can show to the BAM user or allow the BAM user to search on.

Below is an example of the properties I have set for the process ingredient item.

You can see some properties come from the message header and some from the JSON body.

Reprocessing Stage

When I setup my BAM transaction I can also choose to allow message reprocessing from certain stages. If I archive a message body on that stage then I can allow a BAM user to be able to reprocess a transaction if needed. You would configure the reprocessing step on the stage and then at runtime the user can reprocess a message if needed.

You can choose to have messages sent to a Service Bus Queue, Event Hub, Event Grid or HTTP endpoint.

Adding BAM to your Logic App

Once I have setup BAM with the custom connector I can now add the BAM transaction shapes to my Logic App where I need to. Below is an example of configuring the Start Transaction shape.

I then add checkpoints where I need them for key milestones within the transaction that are of interest such as below where you can see the checkpoints for the completion of my transaction.

Below is an image of my full Logic App.

In many real world cases you might just need 2 BAM events from a Logic App. One that it worked and then One that it passed or failed. For many transactions this is all the BAM user is interested in. There are also other cases where your technical implementation is much more complicated and you might have many hundreds of actions across multiple Logic Apps and Functions and other resources. In these cases you can implement a BAM transaction which receives events from different resource types and creates a much simpler view of the key milestones within that very complex subsystem.

Runtime Tracking

Now that our solution has BAM and we have some activity going through our system we can use BAM to give us a business friendly view of what is happening.

Runtime Tracking

At runtime the user can use the tracking feature to see what transactions are being processed. You choose which columns you would like to see based on the global properties you have setup.

Below you can see I have a search which shows the messages being processed. I can choose to show items like the batch id, the customer id and often I display the source and destination system which I capture via meta data sent in the BAM event or by constants in the BAM transaction which help the BAM user to know whats happening.

The user is also able to add search criteria to do things like search for all of the messages from a specific customer.

For the IT Support user I usually track the name of the logic app and the run id as properties in the BAM event so we can easily find the associated logic app if we need to dig deeper into any problem.

Open Transaction

As a BAM user I am able to click on one of the transactions in my search and then I can see a friendly view of the process that executed as shown below.

In this case I can see that my transaction had expereienced an error when uploading data to the dataverse. This simplified view of the key milestones in the transaction (rather than 100 logic app actions) makes it much easier for someone who is not an integration expert to interpret what is happening.

The user can choose to click on a stage within this process view and it will display any properties that were captured and when the stage happened.

It is also possible to have messages archived so you can allow the user to be able to see message bodies if needed like shown below.

Action Required

If we have any failed transactions they will show up in the action required tab. This gives us a to-do list of so we can easily see if any issues need to be addressed. In the image below you can see I have 2 transactions that failed recently.

From the transaction I have a few options as shown below.

I can choose to:

• Add it as a favourite so I can easily find it later
• Ignore the transaction if there is no issue
• Take an action such as reprocessing

The idea of Action Required is when something breaks the user will fix it or ignore it and then the action required list item will be removed so you have a great view on if anything needs to be done.

Reprocessing

If I choose to reprocess a message then when I setup my BAM transaction I needed to have a messasge being archived so it can be used for reprocessing from that stage. I also need to have setup a reprocessing endpoint so I can submit the message back to the system.

It is possible to re-submit the original message or repair it and resubmit if something was wrong.

Dashboards

Using the properties captured by my BAM events I can create BAM user friendly dashboards for my business process. This can display info in a way that makes sense to the business user so they can see how their transactions are processing, when they are happening and what kind of success level they are getting.

Below is an example of a simple dashboard I can create with BAM for this process.

Monitoring

BAM also provides some business process exception management and monitoring. Below you can see the alert history for some alerts I got from BAM.

Some of the common alerts you might configure are:

• Tell me when a transaction fails
• Tell me if there have been any transactions which meet a search criteria

In my alerts I have setup a teams channel which will send a teams notification everytime a transaction fails

I can also setup a query which will tell me if I go 24 hours without receiving a batch for example.

Evaluation

If we reflect on what BAM offers, its really about being able to bubble up key events of interest to the non or semi technical user who wants to know what is happening in your integration solution but normally has no visibility into the integration platform.

Rather than building your own custom solution, Serverless360 aims to provide an easy to use solution which lets you focus on delivering value to the business user.

Strengths

Business User Self Service

The main differentiator for BAM is that it allows business users and non expert IT support users to be able to participate in your integration solution.

Lower IT Costs

Once you can let non integration experts to see whats happening inside your solution then I think there is a big opportunity to reduce the cost of managing your integration solutions. Letting people check whats going on, see performance and deal with problems themselves can reduce the support burden on your IT team in both production and non production scenarios.

In one project I have been involved in I setup testers and business analysts with BAM to manage the integration solution and I believe there has been easily a 50% reduction in the amount of times they have needed to reach out to the integration team to check whats happening with an interface because they didnt see a system get updated as expected. We have been able to let the project team see what happened in the interface and they they could workout that it was bad data being input to the interface, or one of the systems that was being integrated with threw an error. Think about how many times your integration team spend time on these low value tasks troubleshooting a problem that was not really a problem with the interface.

Complex Scenarios
I think one of the areas that BAM has an advantage over the other options we have looked at is around complex solutions. The other options we have looked at are telemetry events which are published and arent necessarily intended to be corrlated with each other in a complex way. We can see that with App Insights there is some ability to implement a correlation across a series of calls but it is intended to be more like a sequence or

Considerations

The following items are some of the things you would need to consider for BAM.

3rd Party Solution & Additional Cost
As an Azure user you need to consider that the Serverless360 BAM offering is not a service from Microsoft Azure. Kovai are part of the Microsoft ISV ecosystem but the BAM product is not part of your Azure and you will have a seperate commercial agreement with them.

BAM is not logging
One of the common questions I have discussed is about BAM versus logging. If you are promoting a BAM event then I think it should have some value to the business user. The Log Analytics approach discussed in the earlier sections is a great way to implement custom logging for technical use cases where you want something like a log file to deep dive into.