What are the best practices for working with feature flags? Feature flag management

Even though feature flags are a great tool to help deliver software, they require extra care when implemented.

In this post, you will find some advice on how to avoid the majority of feature toggle-related issues.

In my previous articles, I described what feature flags are, how to implement them in .NET applications, and how to manage feature toggles with Azure. Those posts are a good place to start if you need an overview first.

Now, let’s jump right into some feature flag best practices.

Don’t overuse feature flags – the fewer, the better

Every feature flag increases the complexity of your solution. It’s because you need to keep extra code to handle each one. Moreover, you need to set its value for each environment. All this takes time.

The code below illustrates the complexity of managing multiple toggles. Keep in mind that in real life, feature flag management can be even harder.

"FeatureManagement": {
    "UseExtremeScaling": true,
    "UsePriorityShipping": {
        "EnabledFor": [
            {
                "Name": "Microsoft.Targeting",
                "Parameters": {
                    "DefaultRolloutPercentage": 1
                },
            },
        ],
    },
    "UseVipDiscounts": false,
    "UseDiscounts": false,
    "UsePromotionCoupons": {
        "EnabledFor": [
            {
                "Name": "ClaimsCheck",
                "Parameters": {
                    "ClaimType": "ShowBetaFeatures",
                },
            },
        ],
    },
    "BuyNowPayLater": true,
    "PayPalIntegration": false,
    "NewSearchEngine": true,
    "SearchEngineFilters": false,
    "RedisCache": {
        "EnabledFor": [
            {
                "Name": "Microsoft.Targeting",
                "Parameters": {
                    "DefaultRolloutPercentage": 10
                },
            },
        ],
    },
    "NewShoppingBasket": {
        "EnabledFor": [
            {
                "Name": "ClaimsCheck",
                "Parameters": {
                    "ClaimType": "ShowBetaFeatures",
                },
            },
        ],
    },,
  },

The solution can be to limit your feature flags.

You could, for example, set a hard limit to 5. This means that if there are five toggles already, another one cannot be added until one of the active flags is deleted.

This rule might be too strict for some teams. In those cases, monitoring the count might be sufficient, so you don’t use more feature flags than necessary.

You should also consider using a feature flag counter when planning work for your team. If you close some features before starting to develop new ones, the counter should not increase.

If your team starts to develop many features at the same time without finishing any of them, that’s where the problems start to appear. So, the key to success is focus.

Whichever option you choose, you should still stick to the next rule, so the issue of having too many feature flags won’t catch up with you.

Every feature flag should be removed when it is no longer needed

During the course of a project, it is easy to forget about the cleanup.

After the successful production launch of the built feature, a flag might stay in the code.

Without removing it, the solution might start to deteriorate as more and more flags are created.

The development team should always remove the flag when it is not needed anymore. It’s best to follow one of two ways to do this.

Add the cleanup task to your team’s backlog

If your team is using either Jira or Azure DevOps, you can take advantage of the tool and just place a special task for feature flag cleanup.

The most important thing is to actually do the task at the nearest possible opportunity and not leave it forever in the backlog abyss.

Define a feature flag expiration date

Unit tests can be used to force feature flag cleanup.

You can, for example, create timebomb unit tests that will fail when the feature flag expiration date passes. See a sample unit test below (I am using xUnit here).

public class DeprecatedFeatureFlagsTests
{
    [Theory, MemberData(nameof(FeatureFlagsData))]
    public void TimeBomb(string featureFlagName, DateTime addedDateTime, DateTime deadline)
    {
        Assert.True(
            DateTime.UtcNow < deadline,
            $"Feature flag {featureFlagName} was added on {addedDateTime} " +
            $"and deadline has passed {deadline})");
    }
    
    public static IEnumerable<object[]> FeatureFlagsData => new[]
    {
        new object[]
        {
            FeatureFlags.UsePromotionCoupons, 
            new DateTime(2023, 01, 01), 
            new DateTime(2023, 05, 12)
        },
        new object[]
        {
            FeatureFlags.UseExtremeScalingSystem, 
            new DateTime(2023, 03, 18),
            new DateTime(2023, 04, 15)
        },
    };
}

Do not reuse feature flags

If you haven’t put the above rule in place, there is a risk of reusing an old feature flag. This might lead to unexpected behavior when the flag is activated.

This can happen for example, if there is an old feature still linked to the flag that the development team is not aware of.

I think a good illustration is this story which shows that in some cases, reusing feature flags can cause millions of dollars in loss and even bankruptcy.

Avoid nesting feature flags

One feature flag creates two code branches that can be executed. But when you nest feature flags, you are actually creating more branches, which can be hard to follow.

I think the best explanation will be this piece of code:

public async Task<OrderTotalPrice> GetTotalPrice()
{
    var orderPrice = GetOrderPrice();
    var usePromotionCoupons = await _featureManager
        .IsEnabledAsync(FeatureFlags.UsePromotionCoupons);
    
    if (usePromotionCoupons)
    {
        var useTimeLimited = await _featureManager
            .IsEnabledAsync(FeatureFlags.UseTimeLimitedPromotionCoupons);
        if (useTimeLimited)
        {
            if (TimeLimitedOfferApplies())
            {
                var useSingleUsed = await _featureManager
                    .IsEnabledAsync(FeatureFlags.UseSingleUsedPromotionCoupons);
                if (useSingleUsed)
                {
                    if (IsCouponUnused())
                        orderPrice = GetDiscountPrice(orderPrice);
                    else
                        throw new InvalidOperationException("Provided coupon code has been already used.");
                }
                else
                    orderPrice = GetDiscountPrice(orderPrice);
            }
            else
                throw new InvalidOperationException("Provided coupon code has expired.");
        }
        else
            orderPrice = GetDiscountPrice(orderPrice);
    }

    return new OrderTotalPrice(orderPrice);
}

In this case, the UsePromotionCoupons flag creates the first code branch. Either one coupon might be applied, or none at all.

Then, more branching happens for time-limited coupons.

Finally, UseSingleUsedPromotionCoupons controls whether to validate how many times the coupon was used.

As you can see, nested feature flags result in confusion and code complexity. To reduce it, we could try to remove some flags that are not necessary.

For instance, if the promotion coupons feature is live in the production environment, then the toggle UsePromotionCoupons might (and eventually should) be removed.

Another option is to refactor the code to replace nesting with linear code flow. See the refactored version below.

public async Task<OrderTotalPrice> GetTotalPrice()
{
    var orderPrice = GetOrderPrice();

    var usePromotionCoupons = await _featureManager
        .IsEnabledAsync(FeatureFlags.UsePromotionCoupons);
    if (!usePromotionCoupons)
        return new OrderTotalPrice(orderPrice);

    var useTimeLimited = await _featureManager
        .IsEnabledAsync(FeatureFlags.UseTimeLimitedPromotionCoupons);
    if (useTimeLimited && !TimeLimitedOfferApplies())
        throw new InvalidOperationException("Provided coupon code has expired.");

    var useSingleUsed = await _featureManager
        .IsEnabledAsync(FeatureFlags.UseSingleUsedPromotionCoupons);
    if (useSingleUsed && !IsCouponUnused())
        throw new InvalidOperationException("Provided coupon code has been already used.");
        
    orderPrice = GetDiscountPrice(orderPrice);
    return new OrderTotalPrice(orderPrice);
}

Of course, the logic still has some complexity but that inevitably comes with applying feature flags.

If you can minimize the feature flags used in the project, I highly recommend doing it first. Another possibility is to use design patterns that could help you with organizing the code.

Consider replacing an if statement with a strategy pattern

In this instance, the strategy pattern fits best.

Imagine working on a simple operational flag that manages which data store should be used to download basket items.

During high load on the application, a DevOps team can enable Redis cache to improve performance. They can also disable Redis and use the database only in regular app usage periods to save money.

The team created the UseExtremeScalingSystem flag.

How to implement the get basket items endpoint?

Of course, you can just simply use an if statement to decide which data store should be chosen. There is another way though.

Instead of using if in code, you can use a Strategy pattern to deal with feature management.

The benefit is that the client code and service code won’t know about feature flags at all. Only the strategy factory contains feature management logic.

First, let’s create a common interface IBasketProvider and two implementations – DatabaseBasketProvider and RedisBasketProvider.

public interface IBasketProvider
{
    Task<IEnumerable> GetAllItemsAsync();
}


public class DatabaseBasketProvider : IBasketProvider
{
    public async Task<IEnumerable> GetAllItemsAsync() 
    {
        Console.WriteLine("CALLING SQL DATABASE");
        await Task.Delay(2000); // Simulating calling Database

        return new List
        {
            new BasketItem("Printer", 2000),
            new BasketItem("Headphones", 500)
        };
    }
}

public class RedisBasketProvider : IBasketProvider
{
    public async Task<IEnumerable> GetAllItemsAsync() 
    {
        Console.WriteLine("CALLING REDIS");
        await Task.Delay(100); // Simulating calling Redis

        return new List
        {
            new("Printer", 2000),
            new("Headphones", 500)
        };
    }
}

Next, we need to implement a Factory that will return one of the interface implementations based on the UseExtremeScalingSystem feature flag value.

public class BasketProviderFactory
{
    private readonly IFeatureManager _featureManager;

    public BasketProviderFactory(IFeatureManager featureManager)
    {
        _featureManager = featureManager;
    }

    public async Task CreateAsync() 
    {
        var useRedis = await _featureManager.IsEnabledAsync(FeatureFlags.UseExtremeScalingSystem);

        return useRedis
            ? new RedisBasketProvider()
            : new DatabaseBasketProvider();
    }
}

Finally, we can just use it in the target code.

In our example, the factory must create a provider. Then the provider is called for basket items.

[ApiController]
[Route("[controller]")]
public class BasketController : ControllerBase
{
    private readonly BasketProviderFactory _basketProviderFactory;
    
    public BasketController(BasketProviderFactory basketProviderFactory)
    {
        _basketProviderFactory = basketProviderFactory;
    }

    [HttpGet]
    public async Task<IEnumerable> GetBasketItems()
    {
        var provider = await _basketProviderFactory.CreateAsync();
        return await provider.GetAllItemsAsync();
    }
}

Another benefit of changing ifs to a strategy pattern is the ease of removing the flags. You just need to remove the stale implementation and adjust the factory.

In the classic if approach, you need to find and replace the if statement with new code.

Implement a feature flag register

It is really useful for the team to give full visibility of the feature flag situation in the project.

The register can contain as much information as the team needs but in my opinion, the most important are:

• Feature flag name
• Work item reference from Jira or Azure DevOps
• Short description of the feature
• Flag value in different environments
• Expiration date

If you are using Azure App Configuration, then you already have a built-in register which is the Feature Manager panel.

Feature Manager panel view

If you are using another solution that does not provide any built-in register, I highly recommend creating one yourself.

It can be even a simple Excel table like the one below:

A simple feature flag register in Excel

You can generate it on the fly if you have the time to build a tool for it.

The important thing is to have always an updated register because without being up-to-date, it won’t be trusted by the team.

Keep track of versions of released feature flags

Release feature flags describe which new features are working in which environment.

It is good to have a good history of changes of these flags. It might be helpful for debugging purposes when investigating production incidents, so it’s helpful to implement a flag versioning strategy.

One way of achieving this is to use appsettings.json files. App settings can be created for each different environment. Each environment file can contain different feature flag values.

The app settings can be tracked by a version control system (e.g. Git) which will ensure feature flag history is generated.

This approach is achievable using the following code sample:

// appsettings.Production.json
{
  // ...
  "FeatureManagement": {
    "UsePromotionCoupons": false,
    "UseExtremeScaling" : false
  }
  // ...
}

// appsettings.Development.json
{
  // ...
  "FeatureManagement": {
    "UsePromotionCoupons": true,
    "UseExtremeScaling" : false
  }
  // ...
}

Another approach is to use Azure App Configuration which provides feature flag history out of the box. It is visible in Azure Portal.

Feature flag history in Azure App Configuration

Avoid feature flag hell

Feature flags are a great tool to deliver and maintain software. Without proper feature flag management, however, they might become a maintainability burden.

To minimize the cost of maintaining feature toggles, you should follow these rules:

1. The fewer feature toggles, the better
2. Every feature flag should be removed when it is no longer needed
3. Do not reuse feature flags
4. Avoid nesting feature flags
5. Consider replacing an if statement with the strategy pattern
6. Consider having a feature flag register
7. Release feature flags should be versioned

This was the last post of my Feature Flags series. I hope you enjoyed it. And if you need someone to help you with feature flags or anything in your app, don’t forget to get in touch.

Further reading

• An article about feature toggles on Martin Fowler’s page
• Microsoft.FeatureManagement library documentation
• 460 million dollar mistake with feature flags story
• Azure App Configuration documentation

See other articles in the series:

1. What are feature flags, and how to use them?
2. How to implement feature flags and filters in .NET apps?
3. How to manage feature flags in ASP.NET Core apps with Azure?