Introduction

Some time ago, I wrote about the new preview feature in Azure Automation Accounts to use Managed Identities [1]. I think that it is useful to migrate to this new feature as soon as possible: managed identities are used in many places in Azure and it is better to use them than the older RunAs account. In an ideal world, I’d like to have a policy that prevents people from creating new Automation Accounts that use RunAs accounts as well. Unfortunately it is not possible to do anything with the RunAs account from both ARM Templates and Azure Policies. The solution is to use Powershell.

Check all Automation Accounts in the current tenant

First, I wrote a PowerShell script that checks for all Automation Accounts in all subscriptions in the current tenant [2]. You can upload this script to your CloudShell and then run it from there. When I run this in my own environment, I see something like:

Now we have this, it would be nice to automate this check and send an email when errors are found. In this way, when someone creates a new Automation Account with the built in RunAs account, we will know that instantly. We then can ask the person who deployed the Automation Account to change it.

Check via Runbook

The code in the Runbook that does these checks should be slightly different than the previous PowerShell script: first, we need to logon with “Connect-AzAccount –identity”. Secondly, we should throw an error when automation accounts with RunAs accounts are found.

When the checks fail an alert should be raised and an email should be sent. In the alert you can filter for the name of the run book and also for Failed status. In a test environment it is useful to check for failures every 1 minute and also evaluate the alert rule every minute. By doing so, you can test the runbook and when it throws an error you will get an email very fast. In production however, you might save costs by running these alert checks just once per hour.

I wrote an ARM template to deploy an Automation Account that does these checks as described. You can deploy this template by using the following commands:

az group create --name AMISBlog --location WestEurope
az deployment group create --name CheckRunAsAccountsDeployment --resource-group AMISBlog --template-file '.\AutomationAccount with runbook that checks for RunAs accounts in Automation Accounts.json'

Use parameter testEnvironment = true if you deploy this template for the first time: it will check for RunAs accounts every 15 minutes. The alerts will be checked every minute. When you don’t deploy for Test (in the ARM template it is assumed that you use Production), then the checks will be done every hour and the alerts will be checked every hour as well. testEnvironment = true is the default setting.

After the creation of this Automation Account, go to the management group called “Tenant Root Group” and use Access Control (IAM) to give the Automation Account “reader” access to all the resources in the tenant. When you are not able to see the details for the “Tenant Root Group”, then first go Azure Active Directory and there choose “Properties”. You might need to switch “Access Management for Azure Resources” to Yes:

After giving reader access to the CheckRunAsAccountsInAutomationAccounts-xxxxx, you can then deploy an extra Automation Account which is based on a RunAs account. When you then start the runbook CheckAutomationAccountsInAllSubscriptions manually (in this example: in Edit mode) you will see output like this:

The advantage of throwing an error is that we can use an alert to send a message to an email account. You can see the alert in the Alerts section in the Automation Account (or, of course, as part of all alerts in your tenant in the Monitoring section of Azure). In the alerts section, go to “Manage alert rules” in the top menu. When you open the details, you can see links for both the condition and the action sub pages for the alerts.

You might get a little bit disappointed when you see that the alert configuration is configured to look every minute for the last one minute – and that you then have to wait for five minutes (or more) to get your email. But after five to ten minutes you will receive your email, including the name of the automation account that generated it.

ARM template

In the ARM template, I’m using a conditional configuration for the parameters for test and production:

Parameters:

        "testEnvironment": {
            "type": "bool",
            "metadata": {
                "description": "Use Test = true for faster alerts (PT1M) and scheduled tasks every 15 minutes. For Test = false this is 1 alert per hour and scheduled tasks every hour."
            },
            "defaultValue": true
        },

Variables:

        "evaluationFrequencyTest": "PT1M",
        "evaluationFrequencyProd": "PT1H",

        "windowSizeTest": "PT1M",
        "windowSizeProd": "PT1H",

Resources:

        ...
        "evaluationFrequency":"[if(parameters('testEnvironment'), variables('evaluationFrequencyTest'), variables('evaluationFrequencyProd'))]",
                "windowSize": "[if(parameters('testEnvironment'), variables('windowSizeTest'), variables('windowSizeProd'))]",
        ...

By using variables you can simply change the values in the variables section in the ARM template to change the behavior. In this example I’m using PT1M for both the evaluation frequency and the window size for the Test environment and PT1H for the evaluation frequency and the window size for the Production environment.

In the schedules I like to start the schedules as fast as possible. It turned out, that you need to add a parameter (‘u’) to the upcTime to make this happen [3][4] and that you need to add at least 5 minutes to the system time. Even when I added 6 minutes, this didn’t always work. I therefore added 7 minutes. When you look at the times that the runbook is starting, you see that this is every 15 minutes at exact zero seconds. So when the ARM template is deployed at, let’s say, 12:00:55, the template would deploy other resources first and would then add 6 minutes to the deployment time and then round it to 12:06:00. When it then checks if there are at least 5 minutes between the moment the resource is deployed and the start time, then this is still not the case. This is why we need those (at least) 7 minutes (and not 5 or 6).

Parameter section:

        "deploymentTime": {
            "type": "string",
            ...
            "defaultValue": "[utcNow('u')]"
        }

Resources section:

        {
            "name": "[concat(variables('AutomationAccountName'),'/', variables('ScheduleName'))]",
            "type": "Microsoft.Automation/automationAccounts/schedules",
            ...
            "properties": {
                ...
                "startTime": "[dateTimeAdd(parameters('deploymentTime'), 'PT7M')]",     
                ...
            }
        },

Please also pay attention to the jobSchedule: this is the connection between the schedule and the runbook that should run at the given time. It took me a while to get this done. In the end, it became clear to me that the name of the jobSchedule should be a guid. This means that you are not free to choose your own name here [5]. To create new guids every time my Automation Account is deployed, I add five random characters to every Automation Account name and I added the same random characters to the schedule name. The random characters are based on the deployment date and time to ensure that the guid of the schedule will be different in every deployment.

Variables section:

        "randomString"         : "[substring(uniqueString(parameters('deploymentTime')),0,5)]",
        "AutomationAccountName": "[concat('CheckRunAsAccountsInAutomationAccounts-', variables('randomString'))]",
        "RunBookName"          : "[concat('CheckRunAsAccountsInAllSubscriptions-', variables('randomString'))]",
        "ScheduleName"         : "[concat('CheckRunAsAccountsSchedule-', variables('randomString'))]",
        ...

Resources section:

            "name": "[concat(variables('AutomationAccountName'), '/', guid(variables('ScheduleName')))]",
            "type": "Microsoft.Automation/automationAccounts/jobSchedules",
            ...

Giving IAM access permissions to the Managed Identity

When you deploy the script more often and then try to configure IAM Access Control in the root Management Group, you will see System Managed Identities for Automation Accounts that are long gone. I think this is one of the minor issues that are still there in the current preview for Managed Identities for Automation Accounts. When you also think that this is at least annoying (if not simply: a bug) then please help me to vote this issue up [6].

Cleaning up

You can easily clean up by deleting the resource group AMISBlog.

Don’t forget to remove the Role Assignment in the Access Control (IAM) in the Management Group “Tenant Root Group” as well.

It is also a good practice to disable “Access Management for Azure Resources” (Azure Active Directory > Properties) when you don’t need these permissions.

Links:

[1] See this blog: https://technology.amis.nl/azure/using-a-managed-identity-with-azure-automation-accounts/

[2] See for sources in my GitHub repository https://github.com/FrederiqueRetsema/AMIS-Blog-Azure-2021 , directory “Check for RunAs accounts in Automation Accounts”

[3] The documentation ( https://docs.microsoft.com/en-us/azure/templates/microsoft.automation/automationaccounts/schedules?tabs=json ) is not very clear about this. I found this in the first GitHub example in the Quickstart templates: https://github.com/Azure/azure-quickstart-templates/tree/master/asr-oms-monitoring

[4] More information about the ‘u’ parameter: https://docs.microsoft.com/en-us/dotnet/standard/base-types/standard-date-and-time-format-strings#UniversalSortable

[5] See also: https://feedback.azure.com/forums/246290-automation/suggestions/33065122-redeploying-jobschedule-resource-from-arm-template

[6] Issue that when you delete an Automation Account, the System Assigned Managed Identity still exists. Please vote for it so Microsoft will solve this: https://feedback.azure.com/forums/246290-automation/suggestions/43427058-delete-system-assigned-managed-identity-when-the-a