ArgoCD App of Apps for infra provisioning

You’re finally done bootstrapping your cluster with minimum required components, like ArgoCD, and your ingress or even multiple ingresses. At this point you want to stop deploying anything to your cluster using kubectl apply and similar. You want to use your ArgoCD deployment to deploy using the GitOps approach. This includes using GitOps for infrastructure provisioning, for example to deploy grafana.

ArgoCD has an App of Apps pattern for installing a set of apps to your cluster using various methods like helm charts, kustomize, plain manifests etc. The general idea is to create a single repo which contains declarative application references to other repos (helm charts etc.), which is then added to ArgoCD, and in turn deploys the applications.

Let’s start by creating a repo with the file /apps/Chart.yaml. This serves as the “main” application in ArgoCD, essentially the App of the App of Apps.

apiVersion: v2
name: applications
description: Applications
type: application
version: 0.1.0
appVersion: "1.0"

If the repository is a private repository, create a SSH key pair, and add the public key as a GitHub Deploy Key. Base64 encode the private key, and add it as a GitHub Repository Secret to your bootstrapping repository (the repository where you deploy ArgoCD to your cluster). Finally, add the following to your CD pipeline in your bootstrapping repository. $GH_SSHPRIV is your base64 encoded private key, and $GH_PATH is the path to your repository (e.g. EldarBorge/aks-poc).

# Add Infra GitOps repo for further configuration
echo "$GH_SSHPRIV" | base64 -d | ./argocd repo add git@github.com:$GH_PATH \
--insecure-skip-server-verification --ssh-private-key-path /dev/stdin

./argocd app create infra-gitops \
    --dest-namespace argocd \
    --dest-server https://kubernetes.default.svc \
    --repo git@github.com:$GH_PATH \
    --path apps
./argocd app sync infra-gitops
sleep 30
./argocd app sync -l argocd.argoproj.io/instance=infra-gitops

Now that the App of the App of Apps is created, and automatically added to ArgoCD, we can start defining applications which will be automatically deployed with GitOps. The procedure is to create the namespace, and add a yaml-file to the templates-folder which defines the repo and sets any custom values required.

Let’s deploy Grafana as an example. Create /apps/templates/namespaces.yaml which instructs ArgoCD to create a new namespace for grafana. Sync-wave -1 tells ArgoCD this resource needs to be deployed first.

apiVersion: v1
kind: Namespace
metadata:
  name: grafana
  annotations:
    argocd.argoproj.io/sync-wave: "-1"

Next up is creating /apps/templates/grafana.yaml. This tells ArgoCD we want to deploy a helm chart, where to locate the chart, and which values to define. In this example, I’m using Key Vault CSI Driver to use existing Key Vault secrets as the login for Grafana, and create an IngressRoute to enable public access to Grafana.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: grafana
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    chart: grafana
    repoURL: https://grafana.github.io/helm-charts
    targetRevision: "*"
    helm:
      releaseName: grafana
      values: |
        extraSecretMounts:
          - name: secrets-store-inline
            mountPath: /mnt/secrets-store
            readOnly: true
            csi:
              driver: secrets-store.csi.k8s.io
              readOnly: true
              volumeAttributes:
                secretProviderClass: "grafana-kv"
        admin:
          existingSecret: grafana-admin
          userKey: admin-user
          passwordKey: admin-password
        serviceAccount:
          name: grafana
        extraObjects:
          - apiVersion: traefik.containo.us/v1alpha1
            kind: IngressRoute
            metadata:
              name: grafana
            spec:
              entryPoints:
                - web
              routes:
                - kind: Rule
                  match: Host(`grafana.eldar.cloud`)
                  services:
                    - name: grafana
                      port: 80
          - apiVersion: secrets-store.csi.x-k8s.io/v1
            kind: SecretProviderClass
            metadata:
              name: grafana-kv
            spec:
              provider: azure
              secretObjects:                                 
                - secretName: grafana-admin
                  type: Opaque
                  data:
                    - objectName: <key vault secret name for username>
                      key: admin-user
                    - objectName: <key vault secret name for password>
                      key: admin-password
              parameters:
                clientID: "<azure ad workload identity client id>"
                keyvaultName: "<key vault name>"
                objects: |
                  array:
                    - |
                      objectName: <key vault secret name for username>
                      objectType: secret
                    - |
                      objectName: <key vault secret name for password>
                      objectType: secret
                tenantId: "<tenant guid>"
  destination:
    server: "https://kubernetes.default.svc"
    namespace: grafana

Leave a Reply

Your email address will not be published.