LinkedIn

Monday, 28 November 2016

Monitor your Azure API Management Instance with PowerBI

Monitor your Azure API Management Instance with PowerBI

Summary

This document outlines the steps involved to monitor your Azure API management instance with PowerBI.

Note: I will refer to Azure API management as APIM in this document.

Steps will include:


  • Add a logger, using the APIM REST API, to your APIM instance to send events to an event hub
  • Set up a Stream Analytics job - It consists of one or more input data sources, a query expressing the data transformation, and one or more output targets that results are written to. Together these enable the user to perform data analytics processing for streaming data scenarios
  • Build a PowerBI dashboard to see your APIM data in a format that suits your business requirements.

Adding a logger to APIM

First thing you need to do is add a logger to your APIM instance using the APIM REST API. I will use Postman to do this.

Firstly, in your APIM instance, enable the REST API:















Secondly, go to the bottom of the security page where you enabled the REST API and generate a shared access key:



















You will use this later in Postman after we first create an Azure Event hub.

Create an Event Hub

Go into your Azure portal and create an event hub.





Ensure you create to Event Hub shared access policies. 1 for sending to the event hub and 1 for receiving. This is to allow you more granular control over your hub.







































Creating the logger in Postman

Now that you have an Event Hub, the next step is to configure a Logger in your API Management service so that it can log events to the Event Hub.
API Management loggers are configured using the API Management REST API
To create a logger, make an HTTP PUT request using the following URL template.

https://{your service}.management.azure-api.net/loggers/{new logger name}?api-version=2014-02-14-preview
Replace {your service} with the name of your API Management service instance.
Replace {new logger name} with the desired name for your new logger. You will reference this name when you configure the log-to-eventhub policy.


Add the following headers to the request.
Specify the request body using the following template.

{
  "type" : "AzureEventHub",
  "description" : "Sample logger description",
  "credentials" : {
    "name" : "Name of the Event Hub from the Azure Classic Portal",
    "connectionString" : "Endpoint=Event Hub Sender connection string"
    }
}
 Here is mine:












You will see that it returned 201 Created which means we now have a logger.


Now we go into APIM and add a policy on the built in echo API.



This will send event to our event hub when the



ProductUnlimited

APIEcho API

OperationRetrieve resource 

API is hit.


View the Event Hub Events

If you want to view the event hub events for your own sanity check then download Service Bus Explorer and listen to your event hub:





















You will notice that the event hub data shown in the listener is the same data written by our APIM policy.

Create Stream Analytics Job to send data to PowerBI

Next, go to your Azure portal and create a new stream analytics job.

Once it is created then create an input:







































And a PowerBI output: Note that you will be prompted to authorize your PowerBI account.








































Also create a query to transform the data. My query doesnt do anything special:



















Then start your stream analytics job.




















Now go back to your APIM instance and hit the API end point a few times. Make sure it is the API operation 
with the policy on it.

Also, change param1 and param2 on the operation to a few different values so we get somewhat useful data:


















Now look at the trace for that operation in APIM and you will see the log to event hub event has fired:
















Now log into PowerBI on the web and you will see (hopefully) a new streaming dataset:
















Create a dashboard


We will create a PowerBI dashboard to visualise our APIM data using param1 and param2


I will drag a pie chart onto the workspace and set the following: All I did was add param2 as a count. As you can see we get a great visualisation of the number of times param2 was used on the APIM operation.





























So you can see that I set a request with param2=5 a lot more times that I did for other calls.



Obviously you can use your imagination as to what you can use this for.

Here we see a Tree Map, Pie Chart and Funnel displaying data from my APIM. The funnel shows distinct calls from IP address.




thanks
Russ