Application Insights API allows to use the power of Kusto language, “which almost writes itself alone”, to parse completely unstructured data of large datasets in a very easy way and present the result in a clean tabular view.
The aim of this article is to learn in less than 5 minutes, how to exploit the power of Kusto and bring the result within an Azure Databricks notebook exploiting API and python built-in functions to do ETL.
Once we have our query built in Application Insights written in Kusto, as shown below:
It is necessary to go to https://dev.applicationinsights.io/apiexplorer/query and paste the code in the “Query area”
Click on fetch and then select cURL, which will provide the exact query for the API call:
Simply get the URL within the cURL command shown in the dedicated panel shown in the screenshot above and use it for the GET call within within a Databricks notebook. The example will be based on a Python notebook.
Below you can see how to make the GET call within Databricks, and how to use the result:
After you have the data in the dataframe, you can filter the json content as if it is a matrix using the get_json_object() function:
Of course it is now possible to do all sorts of transformation and persist data wherever is needed.