AAS with DevOps

Azure Analysis Services deploy via DevOps

Let’s start from real life! What are the advantages in an Enterprise scenario? We can have multiple teams, with different skills and responsibilities. There are processes to take into account and multiple people have different permissions to access tools, environments and data. Therefore the person in charge of deploying a new tabular model should be able to do it in one single click without even knowing how the model has been developed

And that’s where DevOps comes into play!

Azure DevOps (former VSTS), allows to manage backlog items, work items, task assignments and prioritization, everything via easy to use boards.

boards2x
Azure DevOps board

Besides it’s possible to maintain, test and deploy every piece of code of your software, from a chatbot to whatever application, even a database or a tabular data model

The concept of DevOps is a best practice that could/should be used in every computer science project.

In this article we will see how to be able to deploy a tabular model in one click without knowing anything of the model (how is done, what it contains, what are the credential to access the datasource…) assuming that the reader already knows:
– AzureDevOps environment
– How to manage projects in Visual Studio
– Git concepts

Once you’ve set up you Tabular Model project in visual studio you can sync it on Azure DevOps and you’ll have an organization like what is shown in the picture below:

Repository containing the AAS Project

The .bim file is the core file we need to work with when a new version of the model is developed and needs to be deployed.

If you are a developer you know that you can simply publish a model to the server from Visual Studio, but to make it to work you also need the credentials for the data source and being administrator of the server … probably you won’t have those infos for your customer’s production environment. In general the idea is to modularize the process so that those credentials don’t need to be shared across the organization and the person in charge of doing the deployment won’t need any tools installed on his/her machine.

If you don’t have this package install it in the Azure DevOps environment. Once it’s done let’s set up the Release:

Case in which we already have other pipelines
Case in which we don’t have other pipelines already

We can choose EMPTY JOB here:

Let’s create a simple artifact based on Git Type where to indicate the repository where our project is. This will allow later to find the path of the .bim file within the job’s task configuration

Click on the link just below the “Stage 1” block (1 job, 0 task):

In the new screen that will appear we need to click on the “+” sign:

We’ll be asked to choose to add a task, so we type “analysis services” in the search bar as shown below and we select Azure Analysis Services (if you’ve not installed it previously you’ll be asked to install it from the MarketPlace, so click on the “Get it free” blue button)

Get it free from the market place
Installation step

Otherwise if already installed we’ll see it directly available and it will be possible to add it:

Once added it will show that it needs attention (“Some settings need attention”), don’t worry here, it’s normal. Click on it!

At this point it’s time to add all information of the server and credentials BUT here it’s also possible to put PARAMETERS. In this case we are going to put them and make them secret, but in other cases, with your customer for example, someone else will insert those credentials and we would just use the corresponding parameters.

Let’s first insert them like this:
Analysis Services Admin = $(modelAdmin)
Analysis Services Admin Password = $(modelPassword)
Source User Login = $(dwhadmin)
Source Password = $(dwhpwd)
Data Source Type = Azure SQL
Let’s indicate the path of the famous .bim file:
Model file = $(System.DefaultWorkingDirectory)/……./model.bim

(It is possible to user a service principal as well instead of the “Named User” option: for doing this it’s necessary first to insert the Service Principal as Analysis Services Admin, which cannot be done directly from the portal, but by following this official guide: https://docs.microsoft.com/en-us/azure/analysis-services/analysis-services-addservprinc-admins)

And here is where to set up variables:

Once that the pipeline is set up for the deployment you can go back to the Releases, selecting this pipeline and click on Create Release

It’s possible to choose to have the stage directly run after creation of the release, or you just create the pipeline and manually start it later ( I personally choose the latter)

At this point you can click on the new release created and then deploy it….in one click as promised !

Now for the following deployment it will be enough to upload the new .bim file, create the new release and click on deploy again.

Conclusion
It is possible to deploy Azure Analysis Services model without having admin permission on the server by giving access to the desired user to access DevOps; once the .bim model file is in the folder within DevOps (that is actually the directory containing the AAS project which should contain the solution files) it is now possible to deploy it with a single click.
Advantages:

  • Operations: if the IT group of the project has to maintain all the releases of every part of a solution it doesn’t mean it also owns deep technical knowledge of the technologies deployed. That knowledge is of the development team.
  • Productivity: in this way the deployment is a simple click, and it’s a package prepared for people in charge of this task.
  • Password management: passwords are encrypted and no need to know the data sources’ credentials to be able to do a deployment
  • Versioning: the Analysis Services project is not only a model on the server but it’s a project (.smproj) a solution in visual studio, and the .bim file contains the model structure, all its measures, table, M queries and DAX formulas.

…..with respect to Azure Analysis Services here also: Analysis Services, Incremental Refresh, Dynamic Partition: maintainable, scalable, low code. The simple solution you were looking for!

56 thoughts on “Azure Analysis Services deploy via DevOps”

  1. I loved it. you have described in simple and best way. It was awesome, i hope i will get more knowledge from you about different technologies deployment.

    1. Hello Ashish, thanks a lot for your feedback. Happy you’ve found the article useful. Are you used to Azure DevOps? Are there any specific technologies deployment you are interested in?
      Regards

  2. CARLTON PATTERSON

    Hi,
    Thanks for the blog.
    Following your guide for my Azure DevOps for AAS has been quite straightforward up until:
    $(System.DefaultWorkingDirectory)/……./model.bim
    I’m struggling to understand how to include the path to my model.bim.
    Can you let me know the exact format for the UNC path to model.bim?

    1. Hello Carlton, sure thing: actually you have a 3 dots menu close to the text box. If you click on them you’ll have the directory tree of your project, so you just open the folder within you have your .bim file and the path will be generated automatically. In my precise case within the repo i’ve a folder called ic.data.analysis as you can see in the second screenshot of the article. So the folder is
      _CognitiveData/ic.data.analysis/Model.bim
      more generally:
      _nameOfTheRepo/FolderFater/FolderSon/…/Model.bim
      But go for the 3 dots menu looking for the folder where you have the .bim file
      thanks
      Luca

      1. CARLTON PATTERSON

        Hi Luca, thanks so much for getting back in touch… I didn’t think you would 🙂

        So, I click on the eclipses you mentioned, however I get a blank screen with the message:

        No artifact sources are linked to this release pipeline. Link artifacts from the pipeline tab.

        However, I have I’ve set up my Tabular Model project in visual studio and sync’d it on Azure DevOps and and I have an organization similar to one you have in your demonstration.

        One question, did you create an artifact that wasn’t included in your demo?

        (once again, fantastic blog)

        1. Hello Carlton, thank you very much for your comment. you are totally right and i’ve just updated the article with a new screenshot where to add the artifact. The artifact is of type Git, and it’s for defining exactly what you were missing: the repository definition of the project within the pipeline. Now adding it you’ll find also the path when configuring the job’s task. Hope it will work more smoothly now

          1. CARLTON PATTERSON

            Hi Luca,

            Thanks again for getting in touch.
            I thought there was a missing piece to the jigsaw puzzle. I’ve been scratching my head all week 🙂

            I even engaged Microsoft to assist me ha ha.

            I will take a look at your update now … hopefully all will go smoothly from here.

            Thanks again mate… if you’re ever in London let me know and I’ll take you out for a beer.

            Cheers

  3. Carlton Patterson

    Hi Luca,

    Just wanted to give you a quick shout out to let you know that after checking out your update everything checked out.

    Thanks again.

  4. Hi,

    Can you please specify the powershell requirements this task would need if I am using custom build agent?

    1. Hello Ashvini, actually as far as I imagine, I wouldn’t see particular powershell requirement for this task. Can you share more details on the configuration you would work with ?

  5. Hi Luca,
    Its a great article with screenshots put together.
    I followed it and tried to deploy an aas model, the problem i am facing is the inputs taken as source server and database are not getting reflected to the model.bim. I guess the reason is that there is no build step with the source details taken as we do in Visual studio.
    Could you suggest or provide a fix.

    Thanks,
    Brajesh

    1. Hello bsingh reading the doc of the package it states “Also option to change the connected source datasource during release” https://marketplace.visualstudio.com/items?itemName=liprec.vsts-release-aas. That’s the useful thing actually. Have you tried running the pipeline with system.debug set to true to see more details? you could put the build as I’ve done here for the Database project https://aka.ms/devops1click.sqldatabase but it means that you’ll have set up the info in your local project in advance. Did you find data from another source, or you just check via SSMS the source of the AAS after having refreshed the view?
      Regards

      1. I was able to resolve it with some other plugin – “Replace In Files Between Text”. I replace the connection string to required source server and database before using – “Azure Analysis Service Deployment”.
        The next challenge i am working on is multiple Model deployment with same pipeline.

  6. Shubham Pathak

    Exception setting “name”: “The property ‘name’ cannot be found on this object. Verify that the property exists and can be set.”

    getting above error. followed all the instruction. can you help me with this

  7. I was able to resolve it with some other plugin – “Replace In Files Between Text”. I replace the connection string to required source server and database before using – “Azure Analysis Service Deployment”.
    The next challenge i am working on is multiple Model deployment with same pipeline.

  8. hey Thanks for posting the blog!

    We are having an error though
    2020-01-09T15:51:15.1228805Z ##[command] Select-AzureRMSubscription -SubscriptionId xxxx-xxx-xxx-xxx-xxxx-TenantId ***
    2020-01-09T15:51:17.8623910Z ##[error]Error during adding automatic firewall rule (A connection cannot be made. Ensure that the server is running.)
    2020-01-09T15:51:17.9489812Z ##[error]A connection cannot be made. Ensure that the server is running.

    Not sure which firewall the error is referring to.

    Thanks again and awesome blog

    1. Hello Jacob, thanks for your comment. Are you using variables? Or writing directly the string in the fields? Please double check that variables are well written in the right format $(varName). Once I got the same error and was for a typo. Secondly: how many datasources do you have in your model? Ex.: do you simply have one data source and it a Sql server (db or dwh), or you have also other sources like a CosmosDB or whatever else? Because I noticed this package for now works properly only with one datasource of Sql type.
      Hope this helps
      and do not hesitate to comment back with further infos
      thanks
      Luca

  9. I have Azure Analysis Service tabular model with an Oracle Data Warehouse as the source? Is it possible to use the DevOps pipeline with the Oracle source ? Is there any update planned to include Oracle DB as source.

  10. This is really useful. I am trying to follow the same steps. However while deploying the model using SPN I am facing the below issue :
    Error during adding firewall rule (Could not find server: ‘xxx’ in any resource group in the currently selected subscription: xxxxx. Please ensure this server exists and that the current user has access to it.)
    Could not find server: ‘xxxx’ in any resource group in the currently selected subscription: xxx. Please ensure this server exists and that the current user has access to it.
    Can you please help ?

    1. Hey Monica: did you try running the pipeline with the variable ‘system.debug’ set to true? (https://docs.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch#debug-mode) For having more insights
      Does AAS and the DB have the connection to azure services enabled?

      Also I would try to take a simple AAS model, juse one table in it, and try to publish it in a “safe environment” to isolate the problem, and understand if it comes from the account you are using, the azure config, or actually firewall rules on the server

      Bests
      Luca

  11. Hi,

    It’s an great article with screen-shot to follow.

    I am able to deploy the **.bim file on the AAS server. But when I try to Process DB it gives me following error. Please help me to resolve.
    Failed to save modifications to the server. Error returned: ‘The given credential contains a property with a null value. Data source kind: SQL. Property name: EncryptConnection. The exception was raised by the IDbConnection interface.

    1. Hello Tushar, thanks a lot for the comment.
      Have you tried running the pipeline with system.debug set to true to see more details?
      From your text it seems the problem is on the EncryptConnection property, what did you put in there?

  12. Hello,

    In logs ( system.debug ) I do not see anything relevant. But also observed that even I am giving wrong credentials build is successful.
    Where I can find the EncryptConnection property ?

  13. Hi Luca,
    Great Post. But, during deployment I am getting Authentication error. Any idea for this cause?

    2020-07-06T10:35:38.3643570Z ##[error]Error during adding automatic firewall rule (Unable to obtain authentication token using the credentials provided. If your Active Directory tenant administrator has configured Multi-Factor Authentication or if your account is a Microsoft Account, please remove the user name and password from the connection string, and then retry. You should then be prompted to enter your credentials.)
    2020-07-06T10:35:38.4451427Z ##[error]Unable to obtain authentication token using the credentials provided. If your Active Directory tenant administrator has configured Multi-Factor Authentication or if your account is a Microsoft Account, please remove the user name and password from the connection string, and then retry. You should then be prompted to enter your credentials.
    2020-07-06T10:35:38.6011170Z ##[section]Finishing: Deploy model to asazure://westindia.asazure.windows.net/modeldeploymnetjun30

    1. Hello Manish. Thanks for commenting. What are your answer wrt the error ? Like do you have that authentication? What are you using? ServicePrincipal or your account?

      1. This package is for single datasource @shubham apparently. but it gets updated, so monitor if there are any updates for it.
        what type of accoutn are you using for the authentication? alias@domain.com ? is it admin on the server?

  14. Pushkaraj Pramod Garge

    Hi Luca,
    This is really useful,
    i have one question, this will overwrite the existing model which is already deployed,
    do we have something like BISM tool to deploy selected changes only .
    so that it will retain all the partitions which are already processed.

    1. After the the deployment, if you have made structural changes (new relations, column removal) a full process is needed to be sure that the model is clean and all things that need to be recalculated, are recalculated. This deployment is not including the refresh as well, that could be done also exploiting a logic app.

  15. Thanks for finally talking about > Azure Analysis Services deploy via DevOps –
    Luca Vallarelli < Loved it!

  16. Hi Luca,
    This is really useful,
    But, during deployment I am getting below error. Any idea for this cause?

    ‘D:\a\_tasks\deploy-aas-db_a55f5b79-9828-4b7f-9528-85e9e826c028\1.3.3\ps_modules\VstsAzureHelpers_\openssl\openssl.exe’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

    1. lucavallarelli

      Hello Manish, you are indicating an .exe file tipe whilst it should be the .bim file, the one containing the definition of the model.
      HTH

  17. FELIPE ABBADIA ASSIS

    Great Article,

    I have an environment with AAS with several cubes, the model is the same for everyone, changing only the connection and the name of the cube.

    I did the step by step, it runs well, but when I make any changes to the model (Relationships, Add or remove tables) it does not process these changes. I have to manually open the model and ask to process it.

    Any idea?

  18. FELIPE ABBADIA ASSIS

    Great Article,

    I have an environment with AAS with several cubes, the model is the same for everyone, changing only the connection and the name of the cube.

    I did the step by step, it runs well, but when I make any changes to the model (Relationships, Add or remove tables) it does not process these changes. I have to manually open the model and ask to process it.

    Any idea?

    1. Hello Felipe, thanks for the message. The flow doesn’t include a process of the model
      Here a couple of approaches to do the refresh
      https://www.filepora.com/post/refresh-azure-analysis-service-model-with-azure-data-factory-v2-and-azure-key-vault
      or a logic app
      https://jorgklein.com/2018/02/28/process-azure-analysis-services-objects-using-a-logic-app-part-2/

      and then you can call the logic app with another block within the Devops pipeline (rest API call of the logic app)

  19. Thanks for sharing details for deploying AAS. We are able to deploy the model.dim file using Service Principal admin account. We are facing 2 issues .
    1. The model.bim file which is used is having the dev source configuration , but we are passing another configuration details for SIT in the AAS deployment extension. But somehow, after deployment, we are still finding the same model file got deployed , but the connection source string did not updated from the Dev connection to SIT connection. WE are also using the option to overwrite option during deployment .
    2. We are able to deploy the model.bim to AAS server. But when we tried to process the cube, then gives error as ‘OLE DB or ODBC error: Login failed for user. But we have deployed the same xmla cube and we were able to process cube without any issue.
    We were only able to resolve the issue after updating the password connection details and were able to process it.

  20. Hi,

    Indeed this is a great plugin. While experimenting with this, I got the below error:
    ————————————————————————————-
    Error during adding firewall rule (Firewall rule names should be unique. Duplicate name vsts-release-aas-rule is not allowed.)
    Firewall rule names should be unique. Duplicate name vsts-release-aas-rule is not allowed.
    ————————————————————————————-

    Can you please help me to understand this ? I tried to check firewall rules but cannot find this one. Also if you can provide some behind the scene info as how this connectivity work in detail it will help.

  21. Hi,
    This is a great post and is following the steps mentioned. Also, I would like to thanks the comments and responses and it helped me to better understand a few initial problems.
    One of the problem which is blocking me to deploy is related to firewall. I am getting below error while running this pipeline (Debug)
    —————————————————–
    Error during adding firewall rule (Firewall rule names should be unique. Duplicate name vsts-release-aas-rule is not allowed.)
    Firewall rule names should be unique. Duplicate name vsts-release-aas-rule is not allowed.
    —————————————————–
    I tried to search this firewall rule name but unable to find in azure. Can you please help me to better understand behind the scene connectivity and how to deal with this.

    Thanks in advance

  22. Hi Luca

    How do you use effectively use the Data Source Connection details in your Dev Ops task to successfully apply the correct details? Currently the DSC details are being kept from the Model.bim. I can see BSINGH has mentioned finding and replacing from within the file, but I am keen to use the section you created for this purpose.

    Thanks
    Neal

  23. Hi Luca,
    My pipeline is getting stuck at ‘[debug]Get agent IP address’, doesn’t progress anymore and eventually the agent stops listening and errors out. I am using Microsoft Hosted agents.
    Any idea what i might be missing.

    The last two lines of the log are:
    [debug]Removed firewall rule ‘vsts-release-aas-rule’ on azure database server: serverName
    [debug]Get agent IP address

    Thanks,
    Vishwamit

Leave a Reply to Monika Cancel Reply

Your email address will not be published. Required fields are marked *