Browse --- Chat --- Wekan

Skip to content

Better/More configurable lower level state storage

We use snippets to store project ids for active prod environments and ids for projects with remote mirrors.

The first immediate solution for both files is the user passing the project/snippet ids so they are no longer hardcoded.

Whats needed is instance level state storage of non-sensitive, easily accessible and configurable information, i.e. project ids.

This could lead to maintaining a local minio s3 storage solution that gets regularly backed up and imported on system migration. Any info stored (at least to start) would contain non-sensitive information.

Another alternative would be using consuls kv storage, backing up and import only specific folders/keys. Probably not a terrible quick solution.


For project/env state
The next solution for would be better prod env state administration over using snippets, say terraform state.
The only problem with terraform state (in its default usage) is it is per project and iterating over projects is not desirable.
If we used an external s3 storage of a file (not terraform state) the CI would then need the S3 credentials, not desirable.

However, if we can combine both terraform state (CI only uses project credentials) and the instance stores those state files in an external s3 storage (which does have s3 storage access) we can then (with some light work) get all projects that should be currently deployed.
An added bonus to this is each project can supply its own set of options in the state file to use when triggering the production pipeline.


For remote mirrors in test envs
For projects with remote mirrors I dont have a quick solution as the list at this time is manually maintained. The only solution that doesn't scale that well is iterate over all projects every night and populate the list with all projects that have active remote mirrors.
Even then, theres still a snippet file that stores the state and a better solution needed.