One common problem we all share when it comes to working with Azure is cost and “garbage”. Those two relate in proportion to each other, meaning the more “garbage” you own in Azure, the more you have to pay. Simple? Yes indeed. What do I mean in terms of “garbage”? Ok, I will ask you another question, do you have a good lifecycle in place for your resources? No? Exactly, this is the problem. The cloud, especially Azure, is an awesome platform to spin up resources, to build services and let them run forever. As easy it is in Azure to create resources, as hard it is to delete them. Although there is a lot of automation services available in Azure, we have not an easy way to automatically cleanup unused resources. Imagine a company starting with Azure, providing sandbox subscriptions so IT folks can try out new services. How do they gain control what will be used and what can be deleted? In the end, the company has to pay for resources which don’t bring any benefit to the business – only cost. Because of that and of course I like to play with Logic Apps I tried to address this problem.
In this post I would like to provide two Logic Apps workflows and some simple Azure Policy magic to show you an easy approach to solve this problem.
Idea
The idea is to delete resource groups automatically, which are living for a certain amount of time. So we are not going to address single resources like VM’s, databases or other things, instead we are going to delete entire resource groups. This makes sense, because resource groups build a boundary for resources which have the same lifecycle. To achieve this goal we need the following components:
- A workflow that runs on a schedule to inventory the resource group and its creation date.
- A storage account to store the collected information. It is basically an inventory table.
- A workflow which determines, based on the storage account table data, if the resource gets deleted or not.
- A policy which puts the creation date and responsible person / team (at creation time) to a resource group tag, so we can store this information in the storage account table.
In summary we have an inventory workflow which runs on a schedule to collect the information from the resource group like name, subscription, creation date, it owner etc. and stores this data in a storage account table. The second workflow runs also periodically and reads the records from the storage account table and determines if this resource group already has expired, if so, the workflow will remind the owner (it owner tag = mail address) that his period has expired and he will be informed 2 more times until the resource group gets deleted. If the owner does not take any action, the resource group will be deleted. In a addition there is a tag “autodelete” if set to “true” the resource group will follow the deletion process, if set to “false” the resource group will not be deleted. Sound easy, right?
Solution
There are three Azure policies in place (assigned) to a subscription. The first policy will automatically add a “creation date” tag with the current date and time to the resource group. The policy rule definition looks like this…
I already blogged about this policy earlier here. The other two policies are the built-in Azure policies “Require specified tag on resource groups” and “Append tag and its default value to resource groups”. All these policies are wrapped together into a an Azure policy set (initiative) which looks like this…
Now we are ready to assign the policy to a subscription or management group. When the policy becomes active the user will be forced to set the “it owner” tag when creating a resource group. Today we are not able to check if the entered value is a valid mail address or not. So we just educate the people to follow this policy.
Finally the tags are applied…
The next step is to deploy the Logic Apps. Therefore I created these as a Visual Studio solution found here. Clone the repo and deploy each workflow into the same resource group and make sure you always point to the same storage account. After a successful deployment there will be the following resource in place…
Next you need to give access to the workflows managed identities.
On the storage account > Access control (IAM) add the managed identities as shown here…
On the target subscription, the subscription where you applied the policies and the resource groups you would like to delete, set the following permissions for the Logic App managed identities…
Having these settings in place, lets the workflows read from the subscription and delete resources on that specific subscription.
Next we need to authorize the provided mail account to send mails. Go to the office365 connection object and authorize the account…
As a last step create a storage account table in the storage account, named “garbagecollector”. You don’t need to add any columns or so, because they will be created by the workflow (I was not able to deploy the table within the ARM template, because it is not supported)…
Now you should be good to go. I also added in the Visual Studio solution a script AfterDeployment.ps1, this will do the manual steps above, if you prefer to do it by script.
The inventory workflow (get-rg-logic) will start on a periodic schedule, let’s say every 5 minutes and works like this…
In summary the workflow iterates through each subscription where it has permission and iterates on each resource group of these subscriptions. During each run it writes the tag information, as well as subscription and resource group information into the storage account table. If the resource group within the subscription already has been stored in the table, it will update the resource group name, “it owner” tag, last checked date and the autodelete setting.
If the resource group does not exists it will just add the following properties:
- Name (string)
- Owner (mail)
- CreationDate (datetime)
- DeclineCount (int)
- Deleted (boolean)
- NotificationDate (datetime)
- DeletionDate(datetime)
- LastCheckedDate (datetime)
- Autodelete (string)
In both workflows the storage account table’s partitionkey is the subscription id and the storage account table row key is set to the resource group name.
The second workflow runs on a periodic schedule as well and reads all entries from the storage account table. Next the results gets filtered by the following AND logic…
- CreationDate is older than 7 days
- NotificationDate is older than 1 days
- Deleted is false
- Autodelete is “true”
If there is a result set left, the workflow iterates through each entry and checks if the count (decline count) is less than 3, if so, it will send an an information mail and increase the count (decline count) by 1. If the count (decline count) is higher than 3 the resource group will be deleted. In case the resource group was deleted BEFORE the workflow could delete it, the table entry in the storage account will be updated.
If everything works correctly, mails will be sent out like this…
…and when the resource group gets deleted, the final mail looks like this…
I am not sure if you will implement this solution 1:1 as I provided, but I hope it gives you some sort of starting point and delivers an idea how you could overcome this problem.
Find the Visual Studio solution here and the workflow drawings here. Have fun!
