In this blog I will be talking about the motivation behind rewriting and the reasons for embracing complexity over simplicity to facilitate maintenance and use of our Helm chart.

How? What? Helm?

Given that in t-matix a Kubernetes-based infrastructure is used, naturally, the question of deployment automation and simplicity came up. At that stage we already had some services deployed to Kubernetes, but the next step was to deploy a couple of dozen software instances with the same code base but different configuration. Helm charts seemed as the perfect solution for this task, so they were accepted by all team members. This is where our story with Helm begins.

The first version of Helm chart that we had was simple and basic – each service that we had to deploy had its own template, which we combined with multiple values files for each instance. This immediately required some refactoring and simplifying to save us some time when preparing a new deployment. If you are experienced in working with Helm charts, you are already familiar with the chart structure and values file, as well as its capability to parse and combine multiple values files. However, if you are a beginner as we were back then, you will probably make the same mistake as we did – ignore global settings defined in root values file, which can hold common values used for all instances using the same chart. For us, that was the first lesson learned. Give yourself enough time to learn to use a new tool. I can’t say that the first chart version was bad; it was a great basis for improvement and the best we could do in the given circumstances. Still, what we eventually needed was something completely different – a chart that could be easily managed and used both internally and externally, also by users that are not so keen on Kubernetes and Helm charts.

Almost a year of improvements

The learning curve for Helm charts and writing templates is pretty short and doesn’t require a lot of time, but to master all the fine tuning and find hidden gems takes some time and can be quite challenging. Just browsing through the official Helm documentation, you will find almost everything you need to create you first template and chart. In fact, with time, I have noticed that our simple solution is becoming too complex for maintenance and it should thus be simplified. The simplification being done by more complex templates, this now looks more like a mash of simplified programming and template engines. Helm is written in Go, so the template is basically a Go template where you can use some implemented Go methods. I love what you can do with loops, flow control and filters, but templates within templates also come in handy.

Considering the situation we were in, the work required some team effort to allow for more freedom with Helm templates. The biggest obstacle to overcome was the legacy code base that was not Docker-oriented, with the complicated configuration written as Python file. Besides that, there was a lack of manpower to do significant and modern upgrades. When it comes to infrastructure, we eventually switched to a modern approach which required some code base improvements, but we are leaning towards the GitOps and DevOps mindset.

After my colleagues from the backend team helped me to simplify our software configuration, it was time to create a template for it. The configuration itself is pretty simple, a bunch of environment variables that you have to pass from the Kubernetes resource to pod. Since we have a Python in backend, I decided to store environment variables in Kubernetes Secrets. Why? It’s simple to create with a Helm template and ensures that complex values are passed correctly.

Examples, please

One of the biggest challenges was passing the database value, which is a Python dict passed to Django. Having in mind the structure that could be changed and the simplicity of modification, the helper template kicked in and I got this:

When it comes to maintenance, a simple and readable code block is one-lined by a not-that-simple but powerful template:

database is defined in the .tpl file in templates folder and the including is automatic, which means you don’t have to include the helper file in the template where you use the defined database template – just use include “database”. I recommend googling Helm templates functions. There are two Go libraries that are implemented in Helm to get familiar with functions used as filters in the example above.

When it comes to the user, defining the database connection in values files, which are in yaml format, is as simple as this:

As you can see, using b64enc filter complex structure with special characters becomes base64 encoded and applied in Kubernetes, which stores it in the same format it has been sent. I opted for this approach because I didn’t want to bother with any bugs in controllers that could fail creating resource because of special characters, or wonder if I should have added quotes to make sure everything worked correctly, at the same time worrying if those quotes would influence the code. Base64 is used to ensure payload is sent without any changes to destination and Kubernetes is storing secrets in cluster base64 encoded, so it seems to have been a natural choice.

After dealing with the obstacle of adding new complex structures, which doesn’t require making too many changes or too much dynamic creation, I was given more freedom to be transparent toward users while maintaining simplicity.

Values saved in files are fetched in templates like this one:

Moreover, the user is given the possibility to set variable name in name and file path in value.

Using maps in this case simplifies template use and writing the values file. You have to pay attention to indentation, but you don’t have to bother with where a dash should or shouldn’t go.

I’m sure you have noticed $file in the template. You can define variables in Helm templates and if you use loops (as I have for these variables), you will have to define them. The reason behind this is the fact that a loop defines a new scope and you can’t get back to root variables. Here’s an example of a loop with a defined variable:

You have to define variable files, because this will override the loop scope.

If you try to fetch .Files without a variable, this will be interpreted as .Values.fileConfig.Files, which doesn’t exist.

I don’t think I can stress the importance of the simplicity of use enough, but that was a good reason to add a bit more complexity to templates in favor of the values file. A single line in a template is represented by a readable values configuration

This also works when you add a string instead of an array of strings, so it provides you with additional flexibility.

If you have noticed $common_config variable and you wonder why it is used, here’s the explanation: it’s easier to write $common_config instead of .Values.commonConfig, or something more complex than that. You would also use a variable, wouldn’t you?

Most of our templates are defined as loops and values are written as maps, avoiding syntax errors and the need to ask ourselves if we should use dashes or just indentation. One of the issues we needed to solve was merging common values with a specific service defined as

With that we had the possibility to define variables that could be overwritten if needed, but you could also define a new variable:

After mergeOverwrite, we get something like this:

Are we done?

You may think that almost a year is too long a period to get from multiple simple templates to a couple of complex templates with some loops, but have in mind that if you can’t make a simple template, you wouldn’t even be able to come to a reusable complex solution. Also, don’t overcomplicate it, because it is like writing code. You might not have to change it today, but you will have to do something with it in a couple of months and then you will ask yourself what you were doing there.

When I look at what has been done and what our Helm charts look like now, I can say that we are almost there – at least with this third or fourth chart version. I didn’t mention any rotten attempts that became obsolete with changes made in our code base, but without these changes it couldn’t be done.

The last thing we can do with the Helm chart is pass it over to our customers efficiently – using Helm repository. If we manage to do that, maybe you will read about it someday too.