Moving your business to the cloud comes with many benefits. However, it is also a responsibility and Azure monitoring is a must when using the platform. The data you store needs to be safe from leaks and security breaches. Additionally, there are legal requirements pertaining to data storage which you need to fulfil. Azure monitoring can help you with fulfilling these obligations – here are the key aspects you need to consider.
This is it – you are entering the cloud era! The decision has been made, your business is moving its infrastructure to the cloud, or you are building your first application which will be based on cloud technology.
This is it – your organization’s AHA! moment! The tipping point. From now on, the applications will be faster, deployments easier, and everything will run smoothly.
Your enterprise is truly entering the DIGITAL TRANSFORMATION era.
Isn’t this what the cloud vendor, analytics, and all the consultants told you? Isn’t the cloud the new promised land of effortlessness and easy to handle IT operations?
Well, not exactly. At least not if you don’t prepare yourself, your team and your company for it.
To put it into a more realistic scenario – let’s consider an example of one of the companies we are working with at Predica. It is a manufacturing organization building a new solution for its customers. The solution will gather data, analyze it and manage the customers’ devices. All of it built on top of the Azure platform and managing mission-critical, physical infrastructure.
Isn’t this the perfect case of digital transformation? A device manufacturer entering an area of services and cloud computing?
Indeed, it is. But at the same time, it puts this organization into a completely new area of management and operations of a cloud solution. More specifically, something they need to prepare for in different areas: people skills, tooling, and operations procedures.
This is the keyword here. What we have learned is that for an organization, going to the cloud is the easiest step. However, operating in a cloud environment is a completely different story. A typical situation is that the move to the cloud, especially in applications and using Platform-as-a-Service areas, is something that internal operations teams are not prepared for.
One of the aspects of operations is getting to know what is happening with your solution. You need to gather audit and log operations, process them, get the key facts and metrics, and act on them.
But how can you do it on Azure platform? Is there a ready solution for Azure monitoring?
This article will show you 4 areas of auditing and logging that you should focus on in your solutions. Further, I will show you what mechanisms are available to you. In addition, I will also present a recommendation on how to gather and process this information from the platform.
Plus, there will be a BONUS link to an article that will give you additional advice.
Here is our example of application infrastructure built on top of PaaS Azure services. Its goal is to gather telemetry data and provide access to it for end users.
For simplicity, we will not describe the details on how it gathers this telemetry data, what is the logic behind it and how exactly it is built.
This is a simple application, and a typical example of many modern applications built on top of Azure. However, even this simple architecture is generating lots of information. Gathering it will allow you to effectively manage the platform, and detect problems and security issues.
Read through the whole article and check the above application diagram with the content applied to it. It might be a surprise.
What are the four areas and tools you should focus on in terms of logging and auditing in this architecture, which provide this information?
Azure is a complex service. At every moment the services on Azure generate event logs, performance data, hosts metrics. These are available in two major data sources:
These sources simultaneously provide the vast majority of information on diagnostics logs, metrics and auditing information on Azure operations.
General auditing is good for monitoring events and detecting problems, but what about security monitoring?
From a security point of view, Azure is managed by Azure Active Directory. Azure Active Directory audit and sign-in logs are your first and most important source of information about users and their activity on the platform.
Azure platform security model is based on role-based access control (RBAC). Roles are granted to Azure Active Directory users and groups. If you want to know who was granted which role and when – Azure Activity Log will provide this information for you.
How to get quick insights from Azure Activity Log?
Specific Azure services might have their own security logs and you should always look into it within the documentation. Our example application has three logs already available:
Always check whether the service in your architecture has its audit source – it might be a valuable source of information.
So far we were only showing tools for logging and auditing the Azure platform and services. But what if we want to monitor our components built on the Azure platform? In our case, the web API and application or customer access.
What we’ve found is that not many people know about Azure Application Insights.
Application Insights allow you to monitor your applications at application level – you can deploy an Application Insights agent to Azure App Service and gather important metrics on its performance and application. What is more important is that you can integrate it with your code and:
It is a lot of information to process. And, it comes from multiple sources. Is there one place where you can gather it and process at once? Maybe not for all of it, but for most, right now we have Azure Monitor.
It is a tool providing built-in capabilities for the review and analysis of audit logs. Azure Monitor service gathers all information from Azure Audit and Diagnostics logs and allows to access it from a single location in the Azure portal.
Azure Monitor also allows alert rules to be specified to trigger alerts sent over e-mail and SMS, or to initialize a call to external service through Webhooks in case of a specific event or metric. It is available through a dedicated Azure portal blade – Monitor.
Introduction to Azure Monitor with Tomasz Onyszko
You have to admit: there is a lot of information, logs and metrics generated from Azure services. And there are many ways you can gather them, consume and store them on the platform.
Let’s sum it up for our simple application. Here are all logs and audit sources we have covered, together with information on how they can be accessed and stored:
Not that simple, huh?
Wow, there’s a lot of it.
Which approach to take to gather it all in one place? How to process it? And finally, which tool to use to generate alerts and visualization?
You can take it from here and adjust it to your specifications.
Hint: Use Webhooks to push information from Azure Monitor to your SIEM systems or built-in capabilities for alerts.
Hint: Application map is a quick visual which allows you to spot problems with your app easily.
Sounds like an awful amount of work and integration to do, doesn’t it? Isn’t there something easier which will gather all this information and provide some visuals? In fact, there is.
For a quick visualization and access to data, you can leverage Power BI which provides ready packs for some of the services. Two of such solutions, which you will find immediately ready for your Azure application, are:
If you need something more advanced, Microsoft provides its Operations Management Suite (OMS) as SaaS offering. It gathers information from various data sources and allows you to manage and analyze them. You can find the supported data sources in this article.
And, if OMS is not enough, there are third party services which provide such capabilities. One used by many organizations is Splunk with its add-on for Microsoft Cloud service.
It is a bit more complex, isn’t it?
But YOU’VE MADE IT to this point. You deserve the bonus link, and here it is. This article on Azure Logging and Auditing covers everything I’ve discussed, and more. It is the ultimate guide to logging and auditing on the Azure platform.
This article covered lots of information on how you can gather and process diagnostics and audit information from the Azure platform.
But ultimately, having information is not enough. What’s most important is what you will do with it! Make sure your operations team and procedures are ready for it.
We will gladly help you with this process if necessary – get in touch now!
There are 4 key areas of monitoring your Azure cloud that you need to consider:
With the use of Power BI you can visualize your data and gain additional insights, e.g. on your device or application performance
I covered security in GitHub last time. But some of you likely use Azure DevOps for building your products, so let’s t...
Sometimes it feels like I'm pushing too much with security and software development, but then you prove me wrong. Rec...