How to automate starting new projects in Azure? Cloud project management

Your team needs a tool to plan the work, store code, and automate deployment. You need to request infrastructure resources dedicated to your project. Your infrastructure resources must belong to some subscription. Developers in your team need permissions to these resources. In addition to all this, you have to control the budget…

All these things usually take some time and involve many teams. As a result, you need to wait a long time before the work even starts.

The process can get more complicated if a lot of projects are starting in parallel, or if they are implemented by external vendors. And there are deadlines waiting for you!

In this article, we will go over a solution automating Azure project resources creation and allowing to request them in a self-service mode, therefore making the work of project managers (or Project Owners, as we like to call them at Predica) much easier. At least at the beginning!

Ready? Let’s dive in!

The first step – cloud governance

The goal is to create an automated self-service system for project managers, where they can request all the required assets.

The keyword here is ‘automated’ – to make the entire process last only a couple of minutes, instead of a couple of days.

At the same time, we want to maintain standards, safety, and enforce compliance. 

The self-service system for an automated Azure projects kick-off

So, where do we start?

Governance, of course! It’s an easy topic to ignore, but before even thinking about building the automation process, we should have governance and shared infrastructure already implemented.

The first thing we do is creating an appropriate resource hierarchy in Azure. This way, we enforce standards for multiple aspects (governance) – naming, resource tagging, security, resource configuration…

The set of policies is implemented into management groups. This way, every time a subscription is created, the rules will be enforced.  

This step made it possible to set up a properly configured Landing Zone, which will have all our governance rules guaranteed by default. 

How to provide everything we need to start a cloud project?

Before we cover the specifics, we need to be clear on how the process will work (in other words – specify what the team needs to start working on a project). Let’s map this out.

The automated process will consist of the following steps:

1. Provide required information 
2. Create containers for resources  
3. Create Azure AD Resources – permissions and access
4. Create Azure DevOps Project – a space to develop a project
5. Deploy Landing Zone resources.

About the tools – we decided to automate the process using Azure Pipelines and PowerShell. For pipeline monitoring, we use Log Analytics.  

Deployment process steps

The purpose of this pipeline is to build a self-service landing zone for a specific project, every time it’s launched.

The inputs to the project are:

• name
• environment
• budget, list of Project Owners
• Landing Zone type
• information on whether the external developers are involved in the project.

These parameters will be used to create and configure everything, that the developer and Project Owner needs.  

Now, let’s move on to the next step – a short one – creating containers for resources. 

Create containers for resources

We start with assigning an Azure subscription to this project. Next, we set the subscription name and move it to the proper management group. 

Appropriate roles and Azure policies are assigned there to ensure resources are configured following our standards. We set the appropriate tags at the subscription level and thanks to Azure policies they will be propagated to all the resources within. 

Create Azure AD resources

We don’t want developers to wait forever for access. Actually, it shouldn’t even be something they need to request. Any project member should be granted an accurate set of permissions to be able to work on the project.   

That’s why access to resources must be a self-service feature, managed by the project manager. The project manager should be able to self-manage permissions to all the resources needed. On the other hand, we have to remember about security, so that only basic permissions required are managed directly by the project manager.

To develop this identity self-service system, we use Azure AD group ownership delegation that can be modified in the MyAccess portal. We create a group in Azure AD and set up the owner of this group. This way, a project manager can enter MyAccess and specify people taking part in this project.  

At this point, all the partakers are entitled to the tools needed in the project: infrastructure, external tools (e.g. SonarQube), Azure DevOps (for code modification and deployment automation), but only in this specific project. 

The project manager can also add external users to their Azure AD group. It allows identities invited from other tenants to log into the client’s virtual machines (again – only for the project they work in). Azure AD can be synchronized with Active Directory Domain Services using Microsoft Identity Manager.  

The introduction of this mechanism resulted in: 

• Standardization of permission level across all tools 
• Automation of identity management in Azure AD 
• Creation of Azure AD Domain Services identities for external vendors
• Fast and secure management of basic permissions.

Create the Azure DevOps project

We have the accounts configured, but no place to collaborate. Yet.

We need to create a space where project members will be able to plan tasks, monitor progress, develop, release, and deploy applications.  

At this step, it is essential to decide on the initial configuration (and whether the project members should be able to overwrite it) and automation of Azure DevOps project management at scale. To address these challenges, we use Terraform.  

Terraform allows us to manage projects in Azure DevOps, and Azure AD groups that give developers access to the project. The changes are implemented in the infrastructure-as-a-code approach. 

Terraform creates groups in Azure AD that give access to the Azure DevOps project, the Azure subscription assigned to this project, and, for example, to SonarQube instance. It also allows connecting to the dedicated gateway for that project. So, as a part of this step, we have grant permissions to the Azure DevOps projects and other external resources needed in this project. 

Terraform also creates two app registrations: 

• 1st for service connection in Azure DevOps, which will be needed later to automate the implementation of the application in this project
• 2nd for VPN gateway to provide developers with network access.  

This brings us to the last step.

Landing Zone resource deployment

We want to organize resources inside the Landing Zone, provide connectivity (with the secure network), manage network addressing and permissions granting to the developer group and service connection, created earlier by Terraform. 

Network options 

If the project involves external vendors, they can configure a VPN client connection by themselves. We provide a procedure for it as part of the workflow. They can connect directly but only to the resources in their project. 

Setting up a VPN client connection

Network configuration 

With this configuration, we can connect our project in a dedicated virtual network to the shared services as a separate subscription. There is also a connection from this project to the client’s data center. This way, client’s employees can access project resources, even when an external vendor is involved. There is one VPN connection per project.  

If there are no external vendors involved, the virtual network dedicated to the specific project is simply peered with a Hub VNet. 

Cost management

At the subscription level, we configured the budget and its monitoring. The project manager will be informed by email after every 10% of the budget has been used.

Virtual machines 

There is a self-service to create virtual machines that are added to the domain by default. Cost optimization mechanisms are included, which means that VMs in development environments are turned off when they are not needed. 

RBAC 

We also provide role-based access control in the landing zone. It works the same as before – the project manager assigns the roles and decides on permissions. 

Secrets 

We create a place to store passwords (or other secrets) and monitor whether they are expiring. If that happens, the project manager is informed.

After completing the Landing Zone deployment process, the project manager and developers have everything they need for seamless cooperation on the project. 

And that’s all folks! We’ve just reduced to a minimum the time between the decision to start or migrate the project to the cloud, and the moment when the developers and project manager can start working on it. 

Feel free to contact us if you’d like help with implementing this process at your organization. We’ll be happy to give you a hand.