Examples

Examples
- Using Vault Kubernetes Auth Method
- Using Vault AppRole Auth Method

Ready to use sample manifests are provided under samples folder. Just deploy them using kubectl apply -f <sample file>.

Examples hereafter highlight all the features of Open Vault Agent Injector through the supported annotations.

Using Vault Kubernetes Auth Method

Secrets mode - Usage with a K8S Deployment workload

Show example

Only mandatory annotation to ask for Vault Agent injection is ovai.asaintsever.org/inject.

It means that, with the provided manifest below:

Vault authentication done using role test-app-1 (value of com.ovai.application label)
secrets fetched from Vault's path secret/test-app-1/test-app-1-svc using default template
secrets to be stored into /opt/ovai/secrets/secrets.properties

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-1
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test-app-1
      com.ovai.service: test-app-1-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
      labels:
        com.ovai.application: test-app-1
        com.ovai.service: test-app-1-svc
    spec:
      serviceAccountName: ...
      containers:
        - name: ...
          image: ...
          ...

Secrets mode - Static secrets

Show example

With the provided manifest below:

Vault authentication done using role test-app-1 (value of com.ovai.application label)
secrets fetched from Vault's path secret/test-app-1/test-app-1-svc using default template
secrets to be stored into /opt/ovai/secrets/secrets.properties
as we specified static for ovai.asaintsever.org/secrets-type, the secrets will not be refreshed but fetched only once

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-1
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test-app-1
      com.ovai.service: test-app-1-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/secrets-type: "static" # static secrets
      labels:
        com.ovai.application: test-app-1
        com.ovai.service: test-app-1-svc
    spec:
      serviceAccountName: ...
      containers:
        - name: ...
          image: ...
          ...

Secrets mode - Injection into environment variables

Show example

With the provided manifest below:

Vault authentication done using role test-app-1 (value of com.ovai.application label)
secrets fetched from Vault's path secret/test-app-1/test-app-1-svc using default template
secrets to be injected into environment values named after secret keys

Note: you must set an explicit command attribute on containers (even if your image already defines a default ENTRYPOINT/CMD directive) to let the injector determine the process to be run once secrets are mounted as environment variables.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-1
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test-app-1
      com.ovai.service: test-app-1-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/secrets-injection-method: "env"  # Secrets from env vars
        ovai.asaintsever.org/secrets-type: "static" # env vars injection only support static secrets
      labels:
        com.ovai.application: test-app-1
        com.ovai.service: test-app-1-svc
    spec:
      serviceAccountName: ...
      containers:
        - name: ...
          image: ...
          command: ...
          ...

Refer to samples folder for complete examples.

Secrets mode - Usage with a K8S Job workload

When using Kubernetes Jobs, there are specific additional requirements:

Use of serviceAccountName attribute in manifest, with role allowing to perform GET action on pods (needed to poll for job's pod status)
Do not make use of annotation ovai.asaintsever.org/secrets-hook as it will immediately put the job in error state. An error will be raised if this annotation is set on jobs.

Note: This hook is meant to be used with regular workloads only (ie Kubernetes Deployments) as it forces a restart of the application container until secrets are available in application's context. With jobs, as Open Vault Agent Injector looks after status of the job container, the injected signaling mechanism will terminate all the sidecars upon job exit thus preventing use of the hook.

Show example

When submitting a job, use annotation ovai.asaintsever.org/mode and set value to job.

The service account used to run the job should at least have the following permissions:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: job-sa
  namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: job-pod-status
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: job-pod-status
subjects:
  - kind: ServiceAccount
    name: job-sa
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: job-pod-status

Last point is to make sure your job is waiting for availability of secrets file(s) before starting as it can take a couple of seconds before secrets are fetched from the Vault server. A simple polling loop is provided in the sample below.

apiVersion: batch/v1
kind: Job
metadata:
  name: example-2
  namespace: default
spec:
  backoffLimit: 1
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/mode: "job"
      labels:
        com.ovai.application: test
        com.ovai.service: test-app-svc
    spec:
      restartPolicy: Never
      serviceAccountName: job-sa
      containers:
        - name: test-app-1-job-container
          image: busybox:1.28
          command:
            - "sh"
            - "-c"
            - |
              while true; do
                echo "Wait for secrets file before running job..."
                if [ -f "/opt/ovai/secrets/secrets.properties" ]; then
                  echo "Secrets available"
                  break
                fi
                sleep 2
              done
              echo "Job started"
              (...)
              exit_code=$?
              echo "Job stopped"
              exit $exit_code

Secrets and Proxy modes - K8S Job workload

Show example

This sample demonstrates how to enable the proxy mode in addition to the secrets mode used in previous samples. We are here again using a Kubernetes job so we reuse the dedicated service account.

Key annotation to use is ovai.asaintsever.org/mode to let Open Vault Agent Injector knows that proxy mode must be enabled. Optional ovai.asaintsever.org/proxy-port annotation can be handy if default proxy port has to be customized.

Once enabled, your application container can directly interact with the Vault server by sending requests to the injected Vault Agent sidecar that now also acts as a proxy handling authentication with the server. The proxy is available at http://127.0.0.1:<proxy port>.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: job-sa
  namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: job-pod-status
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: job-pod-status
subjects:
  - kind: ServiceAccount
    name: job-sa
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: job-pod-status
---
apiVersion: batch/v1
kind: Job
metadata:
  name: example-3
  namespace: default
spec:
  backoffLimit: 1
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/mode: "secrets,proxy,job"  # Enable 'secrets', 'proxy' and 'job' modes
        ovai.asaintsever.org/proxy-port: "9999"     # Optional: override default proxy port value (8200)
      labels:
        com.ovai.application: test
        com.ovai.service: test-app-svc
    spec:
      restartPolicy: Never
      serviceAccountName: job-sa
      containers:
        - name: ...
          image: ...
          command:
            - "sh"
            - "-c"
            - |
              set -e
              while true; do
                echo "Wait for secrets file before running job..."
                if [ -f "/opt/ovai/secrets/secrets.properties" ]; then
                  echo "Secrets available"
                  break
                fi
                sleep 2
              done
              echo "Job started"
              (...)

              # Vault server can be reached through the local Vault Agent sidecar acting as a proxy
              # You don't need to deal with authentication or the Vault token: Vault Agent proxy is in charge.
              curl [...] http://127.0.0.1:9999/<Vault API endpoint to use>

              (...)
              echo "Job stopped"

Secrets mode - Custom secrets path and notification command

Show example

Several optional annotations to end up with:

Vault authentication using role test-app-4 (value of com.ovai.application label)
secrets fetched from Vault's path secret/test-app-4-svc using default template
secrets to be stored into /opt/app/secrets.properties
secrets changes trigger notification command curl localhost:8888/actuator/refresh -d {} -H 'Content-Type: application/json'

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-4
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test-app-4
      com.ovai.service: test-app-4-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/secrets-path: "secret/test-app-4-svc"
        ovai.asaintsever.org/notify: "curl localhost:8888/actuator/refresh -d {} -H 'Content-Type: application/json'"
      labels:
        com.ovai.application: test-app-4
        com.ovai.service: test-app-4-svc
    spec:
      serviceAccountName: ...
      containers:
        - name: ...
          image: ...
          ...
          volumeMounts:
            - name: secrets
              mountPath: /opt/app
      volumes:
        - name: secrets
          emptyDir:
            medium: Memory

Secrets mode - Ask for secrets hook injection, custom secrets file, location and template

Show example

Several optional annotations to end up with:

Vault authentication using role test-app-6 (value of com.ovai.application label)
hook injected in application's container(s) to wait for secrets file availability
secrets fetched from Vault's path aws/creds/test-app-6 using one custom template
secrets to be stored into /custom-folder/creds.properties

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-5
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test-app-6
      com.ovai.service: test-app-6-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/secrets-hook: "true"
        ovai.asaintsever.org/secrets-destination: "creds.properties"
        ovai.asaintsever.org/secrets-template: |
          {{ with secret "aws/creds/test-app-6" }}
          {{ if .Data.access_key }}
          test-app-6.s3.credentials.accessKey={{ .Data.access_key }}
          {{ end }}
          {{ if .Data.secret_key }}
          test-app-6.s3.credentials.secret={{ .Data.secret_key }}
          {{ end }}
          {{ end }}
      labels:
        com.ovai.application: test-app-6
        com.ovai.service: test-app-6-svc
    spec:
      serviceAccountName: ...
      containers:
        - name: ...
          image: ...
          ...
          volumeMounts:
            - name: secrets
              mountPath: /custom-folder
      volumes:
        - name: secrets
          emptyDir:
            medium: Memory

Secrets mode - Ask for secrets hook injection, several custom secrets files and templates

Show example

Several optional annotations to end up with:

Vault authentication using role test-app-2 (value of com.ovai.application label)
hook injected in application's container(s) to wait for secrets file availability
secrets fetched from Vault's path secret/test-app-2/test-app-2-svc using several custom templates (use --- as separation between them)
secrets to be stored into /opt/ovai/secrets/secrets.properties (using first template) and /opt/ovai/secrets/secrets2.properties (using second template)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-6
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test-app-2
      com.ovai.service: test-app-2-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/secrets-hook: "true"
        ovai.asaintsever.org/secrets-destination: "secrets.properties,secrets2.properties"
        ovai.asaintsever.org/secrets-template: |
          {{ with secret "secret/test-app-2/test-app-2-svc" }}
          {{ if .Data.SECRET1 }}
          my_custom_key_name1={{ .Data.SECRET1 }}
          {{ end }}
          {{ end }}
          ---
          {{ with secret "secret/test-app-2/test-app-2-svc" }}
          {{ if .Data.SECRET2 }}
          my_custom_key_name2={{ .Data.SECRET2 }}
          {{ end }}
          {{ end }}
      labels:
        com.ovai.application: test-app-2
        com.ovai.service: test-app-2-svc
    spec:
      serviceAccountName: ...
      containers:
        - name: ...
          image: ...
          ...

Using Vault AppRole Auth Method

⚠️ AppRole Auth Method can only be used with dynamic secrets.

Show example

Ask Open Vault Agent Injector to use Vault authentication method approle (instead of the default which is kubernetes)
Vault AppRole requires info stored into 2 files (containing role id and secret id) that have to be provided to Open Vault Agent Injector: in the sample below this is done via an init container (replace placeholders <ROLE ID from Vault> and <SECRET ID from Vault> by values generated by Vault server using commands vault read auth/approle/role/test/role-id and vault write -f auth/approle/role/test/secret-id)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-7
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      com.ovai.application: test
      com.ovai.service: test-app-svc
  template:
    metadata:
      annotations:
        ovai.asaintsever.org/inject: "true"
        ovai.asaintsever.org/auth: "approle"
      labels:
        com.ovai.application: test
        com.ovai.service: test-app-svc
    spec:
      serviceAccountName: default
      initContainers:
        - name: test-app-8-container-init
          image: busybox:1.28
          command:
            - "sh"
            - "-c"
            - |
              echo "<ROLE ID from Vault>" > /opt/ovai/secrets/approle_roleid
              echo "<SECRET ID from Vault>" > /opt/ovai/secrets/approle_secretid
          volumeMounts:
            - name: secrets
              mountPath: /opt/ovai/secrets
      containers:
        - name: test-app-8-container
          image: busybox:1.28
          command:
            - "sh"
            - "-c"
            - >
              while true;do echo "My secrets are: $(cat /opt/ovai/secrets/secrets.properties)"; sleep 5; done