This post shows my attempts to manage internal applications deployment as well as Infrastructure in our Kubernetes clusters with different solutions. In the end we find the template-less kustomize is a much better solution than Helm based templating in terms of simplify, clarity, compatibility, flexibility and maintainability.
kubectl
All Kubernetes beginner must have done this. Modify the yaml files and apply directly with kubectl apply . Obviously it’s bad, it’s hard to scale and easy to get wrong. A considerate admin may use diff before the apply. But that’s about all you can improve.
Helm
Moving beyond the basic kubectl apply, I transitioned to Helm, 'the package manager for Kubernetes'. Helm introduces a more structured approach to managing Kubernetes applications. It uses a combination of charts and a values.yaml file, where the charts define the structure of the application and values.yaml allows customization of these applications for different environments.
Initially, Helm seemed like a significant upgrade. The values.yaml track every configuration for an application needed. And I’m provided lovely helm list/install/uninstall etc commands for a more user-friendly interface, improving the visibility and management of resources within the cluster.
However, Helm's limitations soon became apparent. It primarily tracks configurations through the values.yaml file, but this doesn't capture everything. Critical details such as specific chart versions, repository sources, and the intricacies of each release (like the target cluster, namespace, and release names) are not inherently tracked by Helm. Let along the automations.
Apparently Helm alone is not enough.
My journey led me to explore Helmsman and Helmfile. Both tools are similar in functionality, primarily focusing on managing Kubernetes applications through a Desired State File (DSF). It allows for defining the entire state of the Kubernetes cluster in a single, version-controlled document.
I opted for Helmsman to construct my CD pipeline, leveraging its DSF capabilities combined with automation.
DSFs as source of truth:
For each Kubernetes cluster, I crafted specific DSFs that included details like the kube-context, namespaces, helm repositories, and application configurations along with their respective values.
GitHub Action to apply changes automatically
To streamline updates, I set up a GitHub Action. This action triggers whenever there's a change in the DSF, applying the latest configurations to the respective cluster automatically.
Manage a generic-application template
For our internal applications, I maintained a unified Helm chart. This approach enables us to use a shared template across applications, simplifying the configuration process and ensuring consistency in deployment patterns.