.NET Core config transforms for collections in different environments

The final solution can be found in Github if you want to check it out.

.NET Core’s configuration system is really powerful and is packed with a lot of features. However it get really complicated when you have a lot of environments to manage and you want to transform the values of array properties per environment. For the purpose of this tutorial we are going to looks at a config item called “Activites” which is an array of different activities you can do. Below is the default configuration we have and we want to transform the mimimumSpeed and minimunDistance for some environments

  "Acivities": [
    {
      "name": "Walking",
      "minimumDistance": "20K",
      "minimumSpeed": "10Km/h"
    },
    {
      "name": "Cycling",
      "minimumDistance": "20K",
      "minimumSpeed": "10Km/h"
    },
    {
      "name": "Kayaking",
      "minimumDistance": "20K",
      "minimumSpeed": "10Km/h"
    }
  ],

Let us also create a C# class representing this config item as below.

// Activity.cs

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace ArrayTransforms
{
    public class Activity
    {
        public string Name { get; set; }
        public string MinimumDistance { get; set; }
        public string MinimumSpeed { get; set; }
    }
}

Let now add the configuration to the “ConfigureServices” in Startup.cs to make in available for the .NET core DI

// This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            services.Configure<List<Activity>>(Configuration.GetSection("Acivities"));
            services.AddControllers();
        }

I have also went ahead and created a new Activities API controller so that we can see the activities returned along with the environment.

namespace ArrayTransforms.Controllers
{
    
    [ApiController]
    public class ActvitiesController : ControllerBase
    {
        private readonly IEnumerable<Activity> _activities;
        private readonly IWebHostEnvironment _hostingEnvironment;

        /// <summary>
        /// Inject the activites config item and hosting environment 
        /// </summary>
        /// <param name="activitiesOptions">Activites options</param>
        /// <param name="hostingEnvironment">To get the current environment name</param>
        public ActvitiesController(IOptions<List<Activity>> activitiesOptions, IWebHostEnvironment hostingEnvironment)
        {
            _activities = activitiesOptions.Value;
            _hostingEnvironment = hostingEnvironment;
        }

        [Route("api/Activities")]
        public dynamic GetAcitivities()
        {
            return new             {
                _hostingEnvironment.EnvironmentName,
                Activities = _activities
            };
        }
    }
}

The Activities Api returns the following data if we run it now. We can see that the current environment is “Development”. The data returned is the default configuration that we currently stored in out config file.

Lets now create a new environment called “Harsh”, were all these activities becomes a lot more difficult and another environment call “Moderate” were minimum distance will increase. We can mimic this by updating the “launchSettings.json” to add to more profiles with different ASPNETCORE_ENVIRONMENT values.

{
  "iisSettings": {
    "windowsAuthentication": false,
    "anonymousAuthentication": true,
    "iisExpress": {
      "applicationUrl": "http://localhost:51056",
      "sslPort": 44357
    }
  },
  "$schema": "http://json.schemastore.org/launchsettings.json",
  "profiles": {
    "IIS Express Development": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "api/activities",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    },
    "IIS Express Moderate": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "api/activities",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Moderate"
      }
    },
    "IIS Express Harsh": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "api/activities",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Harsh"
      }
    }
  }
}

We can now run the different environments from Visual studio using the different profiles and running each of them will return the following different results.

As you can see the only different in the response currently is the environment name. All the activities has the same minimum distance and speed across all the environments.

We can now go and create environment specific configuration for the 2 new environments and for the Moderate environment we will increase the minimum distance, where as for the harsh one we want to increase the minimum speed also. For this , let’s add the environment specific configs for “Moderate” and “Harsh” environments. Lets duplicate “appsettings.json” and create 2 news settings file “appsettings.Moderate.json” and “appsettings.Harsh.json”

If you now check the appsettings.Moderate.json it looks something like below the only values that is different from the default configuration is the “minimumDistance”. However we are duplicating most of the default config to achieve the transform for just “minimumDistance”.

//appsettings.Moderate.json
{
  "Acivities": [
    {
      "name": "Walking",
      "minimumDistance": "50K",
      "minimumSpeed": "10Km/h"
    },
    {
      "name": "Cycling",
      "minimumDistance": "50K",
      "minimumSpeed": "10Km/h"
    },
    {
      "name": "Kayaking",
      "minimumDistance": "60K",
      "minimumSpeed": "10Km/h"
    }
  ]
}

If we check “appsettings.Harsh.json” we can see a similar scenario were we are still duplicating the “name”. Running the API in those 2 environments returns the newly transformed data.

Simplify the transforms

In order to minimise the transform configs , my first attempt was to see if I can access the array elements using the colon (:)

//appsettings.Moderate.json
{
  "Acivities[0]:minimumDistance": "50K",
  "Acivities[1]:minimumDistance": "50K",
  "Acivities[2]:minimumDistance": "60K"
}

However this seems to have no effect and even if it did work, using an array with indexing positions would be a nightmare to manage. If we change the order of the original config, you will have wrong values.

Inorder to fix this and to simply our configuration first we need to modify the original application setting as below. Instead of an array now we have different nested objects under activities. However we can still use List<Activity> to access this configuration item and .NET Cores configuration system is smart enough to map it.

"Acivities": {
    "Walking": {
      "name": "Walking",
      "minimumDistance": "20K",
      "minimumSpeed": "10Km/h"
    },
    "Cycling": {
      "name": "Cycling",
      "minimumDistance": "20K",
      "minimumSpeed": "10Km/h"
    },
    "Kayaking": {
      "name": "Kayaking",
      "minimumDistance": "20K",
      "minimumSpeed": "10Km/h"
    }
  }

Now if you run the Development environment you can see that it is still returning the activities.

Lets now add transform to both the Moderate and Harsh config . As you can see here, we are only listing the config items we are interested in changing.

// these are in 2 different files but for simplicity showed
// here as one
//appsettings.Moderate.json
{
  "Acivities:Walking:minimumDistance": "50K",
  "Acivities:Cycling:minimumDistance": "50K",
  "Acivities:Kayaking:minimumDistance": "60K"
}
//appsettings.Moderate.json
{
  "Acivities:Walking:minimumDistance": "50K",
  "Acivities:Walking:minimumSpeed": "180km/h",
  "Acivities:Cycling:minimumDistance": "50K",
  "Acivities:Cycling:minimumSpeed": "300km/h",
  "Acivities:Kayaking:minimumDistance": "50K",
  "Acivities:Kayaking:minimumSpeed": "200km/h"
}

If you now run the application in Moderate and Harsh environments you can now see the new values returned.

ASPNETCore Health check for DynamoDB

ASP.NET core offer health checks Middlewares for reporting health of an application and its different components. You can expose the health check of an app as HTTP endpoint or you can choose to publish the health of an app at certain intervals to a source such as a queue.

In this blog we will be exploring getting the health stats of an api that uses dynamoDb as it data store. For this tutorial we will be using,

  • Visual studio 2019
  • dotetcore 3
  • Downloadable DynamoDB which can be found here which would also need the latest version of Java SDK

Create ASP.NET Core Web application

Create a new core web application using the Visual studio template and let’s call it DynamoDBHealthCheck and select the API project template.

Add the required dependencies

Lets add the following dependencies to the solution

Add health check to Startup.cs

In Startup call we are adding services.AddHealthChecks(); and endpoints.MapHealthChecks(“/health”); to the UseEndpoints Middleware.

If you run the api now you can see the health status of the application at /health url

Lets now also add in some code to get a bit more detailed health status for the application.

If you run the application now you can see a bit more information that just the text Healthy. Everything up till now is well documented in the Microsoft documentation for health checks.

Add a health check for DynamoDB

Lets add a new class called “DynamoOptions.cs” for holding all the dynamo db configuration

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace DynamoDBHealthCheck
{
    public class DynamoOptions
    {
        public string AWSAcessKey { get; set; }
        public string AWSSecretKey { get; set; }
        public string ConnectionString { get; set; }
        public string AuthenticationRegion { get; set; }
    }
}

and add the following configuration section to the appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "dynamodb": {
    "aWSAcessKey": "fakeKey",
    "aWSSecretKey": "fakeSecret",
    "connectionString": "http://localhost:8000",
    "authenticationRegion": "localhost",
    "tableName": "TestTable"
  },
  "AllowedHosts": "*"
}

Lets now add a class “DynamoHealth.cs” that will implement the IHealthCheck interface from the ” Microsoft.Extensions.Diagnostics.HealthChecks” package.

using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
using Amazon.Runtime;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;

namespace DynamoDBHealthCheck
{
    public class DynamoHealth: IHealthCheck
    {
        private readonly DynamoOptions _options;
        public DynamoHealth(DynamoOptions options)
        {
            _options = options ?? throw new ArgumentNullException(nameof(options));
        }
        public async Task<HealthCheckResult> CheckHealthAsync(HealthCheckContext context, CancellationToken cancellationToken = default)
        {
            try
            {
                var credentials = new BasicAWSCredentials(_options.AWSAcessKey, _options.AWSSecretKey);
                var config = new AmazonDynamoDBConfig();
                config.AuthenticationRegion = _options.AuthenticationRegion;
                config.ServiceURL = _options.ConnectionString;
                var client = new AmazonDynamoDBClient(credentials, config);
                await client.DescribeTableAsync(_options.TableName,cancellationToken);
                return HealthCheckResult.Healthy();
            }
            catch (Exception ex)
            {
                return new HealthCheckResult(context.Registration.FailureStatus, exception: ex);
            }
        }
    }
}

Lets also add an extension methods that can be called on the services.AddHealthChecks() methods from the startup.cs.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using System;
using System.Collections.Generic;

namespace DynamoDBHealthCheck
{
    public static class DynamoDbHealthCheckExtensions
    {
        const string NAME = "dynamodb";
        public static IHealthChecksBuilder AddDynamoDb(this IHealthChecksBuilder builder, DynamoOptions options, string name = default, HealthStatus? failureStatus = default, IEnumerable<string> tags = default, TimeSpan? timeout = default)
        {
            return builder.Add(new HealthCheckRegistration(
                name ?? NAME,
                sp => new DynamoHealth(options),
                failureStatus,
                tags,
                timeout));
        }
    }
}

We now have to update the startup.cs to include the AddDynamoDb extension. If you run the application now you can see that the health check returns an unhealthy status for overall app and also DynamoDb as shown below.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Diagnostics.HealthChecks;
using Microsoft.Extensions.Hosting;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;

namespace DynamoDBHealthCheck
{
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }
        public IConfiguration Configuration { get;}
        
        public void ConfigureServices(IServiceCollection services)
        {
            // Adding the health check services
            services.AddHealthChecks()
                     .AddDynamoDb(Configuration.GetSection("dynamodb")
                                               .Get<DynamoOptions>());
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseRouting();

            app.UseEndpoints(endpoints =>
            {
                // adding the health check route
                endpoints.MapHealthChecks("/health", new HealthCheckOptions()
                {
                    ResultStatusCodes =
                    {
                        [HealthStatus.Healthy] = StatusCodes.Status200OK,
                        [HealthStatus.Degraded] = StatusCodes.Status200OK,
                        [HealthStatus.Unhealthy] = StatusCodes.Status503ServiceUnavailable
                    },
                    ResponseWriter = WriteResponse
                });
                endpoints.MapGet("/", async context =>
                {
                    await context.Response.WriteAsync("Hello World!");
                });
            });
        }
        private static Task WriteResponse(HttpContext httpContext, HealthReport result)
        {
            httpContext.Response.ContentType = "application/json";

            var json = new JObject(
                new JProperty("status", result.Status.ToString()),
                new JProperty("results", new JObject(result.Entries.Select(pair =>
                    new JProperty(pair.Key, new JObject(
                        new JProperty("status", pair.Value.Status.ToString()),
                        new JProperty("description", pair.Value.Description),
                        new JProperty("data", new JObject(pair.Value.Data.Select(
                            p => new JProperty(p.Key, p.Value))))))))));
            return httpContext.Response.WriteAsync(
                json.ToString(Formatting.Indented));
        }
    }
}

{
  "status": "Unhealthy",
  "results": {
    "dynamodb": {
      "status": "Unhealthy",
      "description": null,
      "data": {
        
      }
    }
  }
}

Let’s now make sure a local dynamo db instance is running and we should also create a table called “TestTable” in this local instance. To check whether DyanamoDB’s health we are calling the DescribeTable method which throw an exception when the table is not found.

Instructions on how to run DynamoDB locally can be found here. Once we have started the DynamoDB local server and created the “TestTable” health check will return a health status both for the overall system and also dynamodb.

// https://localhost:44337/health

{
  "status": "Healthy",
  "results": {
    "dynamodb": {
      "status": "Healthy",
      "description": null,
      "data": {
        
      }
    }
  }
}

Additional Information

  • There is actually a collection of health check nuget packages for different types of products including DynamoDB can be found here. The DynamoDB health check in the package actually uses the ListTable method on Dynamo. However I do prefer to check the existence of the table that my app relies on to run.