If you are into GitOps, you probably already have heard of Argo CD. It is a tremendous tool that gives you a declarative continuous delivery for your Kubernetes Infrastructure if you are using a templating tool such as Helm. And it is fairly easy to set up.
The company developing it is also creating a set of other tools to complement its use and push the GitOps paradigm a step furthAer. This article will focus on one of them, Argo CD Image Updater which will update your container versions for you!
Argo CD provides a way to override the parameters of an Argo CD application. This gives the user a way to dynamically determine the values of those parameters, for instance, an image version, without the need to change the deployment manifests. It can work in two ways:
.argocd-source.yaml in the Argo CD application directory.argocd-source-<argocd-app-name>.yaml directly in the manifest directory, which is watched by your Argo CD applicationNote that Argo CD will first apply the latter than the former. Thus, if you override parameters for the same application in both files, the ones in the .argocd-source-<argocd-app-name>.yaml will be deployed.
One could ask, what does it have to do with Image Updater? It's easy: it watches container registries. When a new version is available, it will commit an override file in your repository. Here is a template example of this file if you are using Helm:
helm:
parameters:
- name: <parameter-to-override>
value: <new-parameter-value>
forcestring: trueNow that we understand what Argo CD Image Updater does, we will dive into its installation and setup.
⚠ N.B: There is currently an issue with the override system. Normally, you can specify Helm parameters inside of the Argo CD application itself. But when you have an .argocd-source-<argocd-app-name>.yaml override file, those parameters are not taken into account.
Use the install manifest to deploy Image Updater with an Argo CD application. My advice is to separate it into smaller manifests to make it more readable and changeable for the configurations to come.
First of all, Argo CD Image Updater needs access to your infrastructure repository. If you are using a private repository, you already have set up accesses for Argo CD.
By default, Image Updater will use those ones. The only thing you have to do is add the write access to the ssh key or the account you are already using.
If you are using a public repository, you probably only have given an HTTPS link in your Argo CD applications, but you did not give any credentials to it. My advice would be to use a private access token. You can create one either for Github, Gitlab, or BitBucket. Your secret should look like this:
apiVersion: v1
kind: Secret
metadata:
name: <your-secret-name>
type: Opaque
stringData:
username: <your-username>
password: <your-private-access-token>We will see later how to give those credentials to Image Updater.
If you are only using images you can access publicly, you can skip this part. If not, what follows could become very handy!
Indeed, if your Kubernetes pod templates use a container image stored in a private registry, Argo CD Image Updater is going to need read access to it in order to be able to list the different tags of your image.
But, at the moment of writing this article, Image Updater does not use any SDK to connect to those registries.
For instance, if you store your images in a public cloud registry, setting up a service account is not enough! Image updater gives us various ways of setting up credentials (I will show you in just a moment) but what you need to remember is that you can only use a username and password authentication.
Here is what your secret should look like:
apiVersion: v1
kind: Secret
metadata:
name: <secret-name>
type: Opaque
stringData:
<your-key>: <username>:<password>Then, in the default configmap of Image Updater, you need to add the registry configuration. Here is what it could look like for a GCP registry:
apiVersion: v1
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/name: argocd-image-updater-config
app.kubernetes.io/part-of: argocd-image-updater
name: argocd-image-updater-config
data:
registries.conf: |
registries:
- name: Google Container Registry
prefix: eu.gcr.io
api_url: https://eu.gcr.io
ping: no
credentials: secret:<secret-namespace>/<secret-name>#<you-key>💡 Pro tip: if you want to use a cloud service account to connect to your registry, but it does not have a username and password, only a JSON configuration file. You can use _json_key as the username and the whole JSON configuration as a password (but not between quotation marks!).
⚠ Warning: in the documentation, you will find the possibility to add a tagsortmode I strongly recommend not to use it because it can make the update bug, depending on the update strategy you will use.
Now that Argo CD Image updater is set up, we can dive into its enabling for any given application. All you have to do is add a few annotations to your Argo CD application. Here are the basic ones:
annotations:
argocd-image-updater.argoproj.io/write-back-method:
git:secret:<secret-namespace>/<your-secret-name>
argocd-image-updater.argoproj.io/image-list: <alias>=<registry-url>The first annotation tells Argo CD to be declarative, i.e. it will commit an override file. By default, the annotation value is Argo CD which is the imperative mode, i.e. it will call the Argo CD API to make an update.
Whether it is for personal or professional use, I prefer the declarative way, so my repository is my only source of truth for the state of my cluster.
The second annotation tells image updater which image in which registry it should watch for updates and gives it an alias (which will become useful soon enough). The secret-namespace and you-secret-name here are references to the secret you created in the Git credentials setup part earlier.
💡 N.B: if you did not set up specific credentials for Image Updater and you are using the ones from ArgoCD, you can just use:
annotations:
argocd-image-updater.argoproj.io/write-back-method: git
argocd-image-updater.argoproj.io/image-list: <alias>=<registry-url>
Image updater gives you the possibility to update tags following different strategies. The three I found the most useful are latest, semver, and digest.
latest can be a little counterintuitive. Indeed, it does not use the common latest tag of a container image; it just updates the tag with the most recent creation date. But you might want to update only when particular tags are pushed.
That is why you have an allow-tags option that lets you set a regexp of authorized tags. To set up this strategy, here are the annotations you need to add to your Argo CD application:
argocd-image-updater.argoproj.io/<alias>.update-strategy: latest
argocd-image-updater.argoproj.io/<alias>.allow-tags: regexp:<expression>N.B. It is this update strategy that is not compatible with the tagsortmode of the registry configuration.
semver update, as its name suggests, allows you to update tags following the convention of the same name. Obviously, you do not need any allow-tags for this one. This is the default update-strategy so it is not mandatory to specify it.
But! 💡 If you want to allow only certain types of updates, for instance, you want to allow patch updates, but no minor or major version changes you can use the annotation this way (if you want to stay in 1.6):
argocd-image-updater.argoproj.io/<alias>.update-strategy: semver:1.6.xThe same goes if you also want to allow minor version changes:
argocd-image-updater.argoproj.io/<alias>.update-strategy: semver:1.x.xdigest update, is the strategy you want to use in order to follow a mutable tag. With this configuration, Image Updater will check if the tag is on a new image by comparing their digests, which is unique. The tag you want to follow must be specified in the image-list annotation.
argocd-image-updater.argoproj.io/image-list: <alias>=<registry-url>:<tag-to-follow>
argocd-image-updater.argoproj.io/<alias>.update-strategy: digest
argocd-image-updater.argoproj.io/<alias>.allow-tags: regexp:<expression>
As I said before, those are not the only existing update-strategies. There also is a name one that allows you to update the tag with the latest entry from a given alphabetically sorted list, but I do not find it very convenient.
The setup I just explained to you will work perfectly fine if your Helm Chart, templates, and values are directly in the same repository as your Argo CD setup.
But if you use Umbrella or OTS Charts, it will not be enough.
Why? Because the parameter you want to override is not image.tag but <chart-name>.image.tag, where chart-name is the name in the dependencies of your chart umbrella. To do so, Image Updater provides us with two other annotations you can configure as follow:
argocd-image-updater.argoproj.io/<alias>.helm.image-name: <chat-name>.image.name
argocd-image-updater.argoproj.io/<alias>.helm.image-tag: <chat-name>.image.tagOnce everything is ready, Argo CD Image updater will be able to do this kind of commit to your repository, and Argo CD will do the rest:
💡 N.B. you can also change the user name and email of Image Updater in its configuration.
If you are using the digest update strategy, you might want to change those annotations to:
argocd-image-updater.argoproj.io/<alias>.helm.image-name: <chat-name>.image.name
argocd-image-updater.argoproj.io/<alias>.helm.image-tag: <chat-name>.image.shasumIndeed, with this strategy, Image Updater does not commit a tag, but the sha 256 of your image, which looks like this :
value: sha256:9884d406f6750a16e70a4cf7275d1e4caa4bf4320dddd17ef0b0e381a1f2bae0To pull an image with its sha 256 sum and not its tag, you need to use <image-url>@sha256:<something-here> and not <image-url>:<something-here>. And you need to handle this difference in your helm chart as follows:
image: "{{ .Values.image.repository }}{{- if .Values.image.shasum }}@{{ .Values.image.shasum }}{{- else }}:{{ .Values.image.tag | default .Chart.AppVersion }}{{- end }}"If you are using Helm to create your own Charts, you might want to read this part. One of the issues with image updater is that you will not be able to manually select a tag for your Pod Templates as long as it is enabled for your Argo CD application. And deleting and adding annotations all the time is not a solution.
That is why I found it useful to add an option of overrideTag in my Charts. To implement this in your pod templates, you can do:
image: "{{ .Values.image.repository }}{{- if .Values.image.overrideTag }}:{{ .Values.image.overrideTag }}{{- else if .Values.image.shasum }}@{{ .Values.image.shasum }}{{- else }}:{{ .Values.image.tag | default .Chart.AppVersion }}{{- end }}"N.B. Here I give the version which also handles the use of the digest strategy. If you do not need it, I let you change this line accordingly 😉
Then in your values, if you do not specify overrideTag, the tag will be used, and Image Updater will do its job. Otherwise, if you specify overrideTag its value will be used, and Image Updater can not do anything about it.
Argo CD image updater is a useful tool that comes with its perks, but also its downsides.
On the one hand:
On the other hand:
overrideTag I showed you, which is convenient but kind of hackydigest strategy makes the Argo CD UI less dev friendly because it will display the sha sum of your image instead of the tagdigest strategy needs to make 3 calls by lookup at your container registry API. You might end up reaching the rate limit quota...While very useful, Image Updater is still a young tool that will evolve quickly, and it is currently not fit for every use. If you are interested in testing it, I hope this post showed you how easy it is to configure Argo CD Image Updater for your infrastructure. 😄
GitOps is being used more and more; while its ecosystem is still not mature, Argo company is doing some great work! And if you want to be updated on their latest tools, I would recommend you to follow our blog.