Autoscale App Scaling Now You Need It, Now You Don’t – How To Scale Your Application

The cloud has many advantages. As a developer, one of the biggest benefits I’ve found is how easily I can scale applications. Let’s go over some ways we can do this on the Azure platform.

In Azure, many services have autoscaling out-of-the-box, which means they dynamically allocate resources to match performance requirements.  

Your system will be automatically scaled for high throughput if needed. When there is less demand on the service, the system automatically scales down resources. This approach gives your system the ability to be responsive all the time, independent of the load.

Visualization of app scaling

Scale up or scale out?

Scaling up and scaling out are two different ways of scaling or increasing throughput in the system. Both have their advantages and disadvantages.

What is scaling up?   

Scaling up is also referred to as vertical scaling, which means that the system will perform resource upgrades your system to reach the desired performance level (e.g. by adding more RAM or changing the CPU).  

In Azure, when you want to scale up your application, you need to change your app service plan. It does not happen by default. 

Scaling up in a web app

However, scaling up limits your system performance to a single server and becomes increasingly expensive the higher you have to scale.

What is scaling out? 

Scaling out is also called horizontal scaling. It means adding or removing instances of a resource that we are running our system on.  

The application continues running without interruption as the platform provisions new resources. When the provisioning process is complete, it deploys the solution with these additional resources.  

Many Azure services support automatic scaling out. The platform also allows providing custom rules for it. As the owner of the system, you can decide when and how it should be scaled. 

Horizontal scaling is more flexible in the cloud context as it allows you to run potentially thousands of instances to handle the throughput. 

Configuring scaling out using autoscale

Autoscale

[Source] 

In Azure, you can set up autoscale using two modes: 

The first mode is based on analyzing specified metrics from your application. It then compares them to scaling rules you define. For example, you can set up the service to be scaled up when it uses more than 70% of CPU.  

The second mode is fixed. You have to select a period wherein your system will be scaled, so that you can add a rule for it. For instance, you can add a rule that every Saturday between 10-12 am your system will scale to 10 instances. 

Here is a list of services that support the autoscale functionality: 

• Virtual Machine Scale Sets 
• Cloud Services 
• App Service – Web Apps 
• Azure Functions 
• API Management 
• Azure Service Fabric / Azure Service Fabric Mesh 
• Azure Kubernetes Service 

Application insight 

How do you check how many instances the system uses? Easy! Use application insight. It has a user-friendly UI to present the state of our applications, where we can see the components that we are using, and how many running instances we have.  

For example, I created 30,000 events and added them to the event hub. An Azure function was listening on this event hub and stored everything to Azure Table storage. As you can see, my simple application was scaled to 8 instances to handle the load efficiently. 

A demo of how I scaled my application

Tips and tricks for working with autoscale 

Here are a few things worth remembering when working with the autoscale functionality. 

1. Creating a horizontal scale–aware system can be tricky, because you have to keep in mind that your code will be run on many instances at the same time. The best practice is to write a stateless event-driven solution. (Event-driven is a pattern where any change is propagated to other components).  

Scaling is ready to handle a huge volume of small events. So why should we go with stateless? Because the system can be largely multiplied in a short time, so many things won’t work properly.  

If you really want to use the alternative option, I suggest you start using Service Fabric, because it’s one of the best orchestration services that handles stateful services.

2. The system should be asynchronous. If component A calls component B synchronously, A and B are tightly coupled, and that coupled system has a single scalability characteristic: to scale A, you must also scale B.  
3. If you want to use a cache, think about using a distributed cache like Azure Redis. 
4. The best approach is to have many small services without any long-running tasks, because an instance of a process could be prevented from being shut down cleanly when the system scales in, or it could lose data if the process is forcibly terminated.  

Of course, you also need to consider implementing a retry policy and a circuit-breaker. 

Summary 

A benefit of using the cloud to run a system vs. on-premise is an out-of-the-box feature called autoscale for basic services, such as Virtual Machine, App Services, Azure Functions, Kubernetes, Service Fabric.  

You decide which conditions need to be met to scale your system and the size of server you need as a base of the scale.  

Your application can be scaled out or scaled up. The first option reproduces the same environment n times and runs on the same application. The second one upgrades hardware to add more power to it.  

Additionally, you can monitor everything using Application insight, which shows you a graphic of how and for how long your application was scaled. 

The best application to scale is a stateless short-running task that implements an event-driven pattern and asynchronous communication between other applications. 

As always, feel free to get in touch if you’d like to know more on building apps on Azure!