Where do I even begin?
This HAS to be a common problem that many people deploying code to production have had to grapple with, but I'm finding that the existing documentation and questions on the web are just insufficient to help me land on the ideal solution.
What am I trying to do?
Well I have a React SPA web UI that talks to a backend API that I built using .Net Core (ASP.Net Web API). Data is persisted into a database and so naturally I need to be able to vary the connection string based on the env, and also I should be storing that connection string (or at least the password) in a secret manager.
For hosting I choose to deploy by API to Google Run (why? just because) and to make things extra I've Dockerized the API.
Currently Google Build is triggered based on commits to my 'test' branch. That triggers the dotnet build process and then the API is built into a Docker container, which is then published to Google Run.
So what do I do with the connection string?
In local dev this works ...
// Configure Mongo Database Service
builder.Services.Configure<GameDatabaseSettings>(
builder.Configuration.GetSection("GameDatabase"));
builder.Services.AddSingleton<GamesService>();
(This code is from Program.cs)
I can just load the config from User Settings stored locally. That works perfectly for local development, but once I deploy it to Google I won't have access to that.
So I slapped the same secrets into Google Secret Manager and I can then load them when my GameService loads up, but I just hate this piece of code. It feels like a huge hack...
using Google.Cloud.SecretManager.V1;
using Microsoft.Extensions.Options;
using MongoDB.Driver;
namespace my_api.DataPersistence;
public class GamesService
{
private readonly IMongoCollection<Game> _gamesCollection;
private const string GCP_PROJECT_ID = "*************";
public GamesService(IOptions<GameDatabaseSettings> gameDatabaseSettings)
{
if (gameDatabaseSettings == null || gameDatabaseSettings.Value == null || gameDatabaseSettings.Value.ConnectionString == null)
{
var GetGoogleStoredSecret = (SecretManagerServiceClient client, string secretName, string version) =>
{
var request = new AccessSecretVersionRequest
{
SecretVersionName = SecretVersionName.FromProjectSecretSecretVersion(
GCP_PROJECT_ID, secretName, version),
};
return client.AccessSecretVersion(request);
};
// ...or, load them from Google Secret Manager.
var client = SecretManagerServiceClient.Create();
var connectionString = GetGoogleStoredSecret(client, "GameDatabase-ConnectionString", "latest");
var dbName = GetGoogleStoredSecret(client, "GameDatabase-DatabaseName", "latest");
var colName = GetGoogleStoredSecret(client, "GameDatabase-GamesCollectionName", "latest");
// Get the config from UserSecrets
var mongoClient = new MongoClient(
connectionString.Payload.Data.ToStringUtf8());
var mongoDatabase = mongoClient.GetDatabase(
dbName.Payload.Data.ToStringUtf8());
_gamesCollection = mongoDatabase.GetCollection<Game>(
colName.Payload.Data.ToStringUtf8());
}
else
{
// Get the config from UserSecrets
var mongoClient = new MongoClient(
gameDatabaseSettings.Value.ConnectionString);
var mongoDatabase = mongoClient.GetDatabase(
gameDatabaseSettings.Value.DatabaseName);
_gamesCollection = mongoDatabase.GetCollection<Game>(
gameDatabaseSettings.Value.GamesCollectionName);
}
}
public bool GameCollectionExists { get { return _gamesCollection != null; } }
public async Task<List<Game>> GetAsync() =>
await _gamesCollection.Find(_ => true).ToListAsync();
public async Task<Game?> GetAsync(string id) =>
await _gamesCollection.Find(x => x.Id == id).FirstOrDefaultAsync();
public async Task CreateAsync(Game newGame) =>
await _gamesCollection.InsertOneAsync(newGame);
public async Task UpdateAsync(string id, Game updatedGame) =>
await _gamesCollection.ReplaceOneAsync(x => x.Id == id, updatedGame);
public async Task RemoveAsync(string id) =>
await _gamesCollection.DeleteOneAsync(x => x.Id == id);
}
The Google Cloud Secret Manager API reference suggests that there is an extension method to support a classier way of loading the secrets out of Secret Manager, but I can't find any useful documentation on that.
I feel like the ideal solution would be 3 lines of code in Program.cs that would just load the configuration, the same way that this does from the local system...
builder.Services.Configure<GameDatabaseSettings>(
builder.Configuration.GetSection("GameDatabase"));
I'd welcome a cleaner way to do this.