In order for Velero to backup volumes in a Pod using an opt-in approach, it requires an annotation on the Pod called `backup.velero.io/backup-volumes` with the value being a comma-separated list of the volumes mounted to that Pod. This policy automatically annotates Pods (and Pod controllers) which refer to a PVC so that all volumes are listed in the aforementioned annotation if a Namespace with the label `velero-backup-pvc=true`.
apiVersion: policies.kyverno.io/v1alpha1kind: MutatingPolicymetadata:name: backup-all-volumesannotations:policies.kyverno.io/title: Backup All Volumespolicies.kyverno.io/category: Veleropolicies.kyverno.io/severity: mediumpolicies.kyverno.io/subject: Pod, Deployment, CronJob, Annotationpolicies.kyverno.io/description: In order for Velero to backup volumes in a Pod using an opt-in approach, it requires an annotation on the Pod called `backup.velero.io/backup-volumes` with the value being a comma-separated list of the volumes mounted to that Pod. This policy automatically annotates Pods (and Pod controllers) which refer to a PVC so that all volumes are listed in the aforementioned annotation if a Namespace with the label `velero-backup-pvc=true`.spec:evaluation:admission:enabled: truematchConstraints:resourceRules:- apiGroups:- ""apiVersions:- v1operations:- CREATE- UPDATEresources:- pods- apiGroups:- appsapiVersions:- v1operations:- CREATE- UPDATEresources:- deployments- daemonsets- statefulsets- apiGroups:- batchapiVersions:- v1operations:- CREATE- UPDATEresources:- jobs- cronjobsnamespaceSelector:matchLabels:velero-backup-pvc: "true"variables:- name: podPvcVolumesexpression: "object.kind == \"Pod\" && has(object.spec.volumes) ? object.spec.volumes.filter(v, has(v.persistentVolumeClaim)).map(v, v.name) : []"- name: podVolumesListexpression: "variables.podPvcVolumes.size() > 0 ? variables.podPvcVolumes.join(',') : ''"- name: deploymentPvcVolumesexpression: "(object.kind == \"Deployment\" || object.kind == \"DaemonSet\" || object.kind == \"StatefulSet\") && has(object.spec.template.spec.volumes) ? object.spec.template.spec.volumes.filter(v, has(v.persistentVolumeClaim)).map(v, v.name) : []"- name: deploymentVolumesListexpression: "variables.deploymentPvcVolumes.size() > 0 ? variables.deploymentPvcVolumes.join(',') : ''"- name: jobPvcVolumesexpression: "object.kind == \"Job\" && has(object.spec.template.spec.volumes) ? object.spec.template.spec.volumes.filter(v, has(v.persistentVolumeClaim)).map(v, v.name) : []"- name: jobVolumesListexpression: "variables.jobPvcVolumes.size() > 0 ? variables.jobPvcVolumes.join(',') : ''"- name: cronjobPvcVolumesexpression: "object.kind == \"CronJob\" && has(object.spec.jobTemplate.spec.template.spec.volumes) ? object.spec.jobTemplate.spec.template.spec.volumes.filter(v, has(v.persistentVolumeClaim)).map(v, v.name) : []"- name: cronjobVolumesListexpression: "variables.cronjobPvcVolumes.size() > 0 ? variables.cronjobPvcVolumes.join(',') : ''"mutations:- patchType: JSONPatchjsonPatch:expression: |object.kind == "Pod" && variables.podVolumesList != '' ?(!has(object.metadata.annotations) ?[JSONPatch{op: "add",path: "/metadata/annotations",value: {"backup.velero.io/backup-volumes": variables.podVolumesList}}] :[JSONPatch{op: "add",path: "/metadata/annotations/" + jsonpatch.escapeKey("backup.velero.io/backup-volumes"),value: variables.podVolumesList}]) : []- patchType: JSONPatchjsonPatch:expression: |(object.kind == "Deployment" || object.kind == "DaemonSet" || object.kind == "StatefulSet") &&variables.deploymentVolumesList != '' ?(!has(object.spec.template.metadata.annotations) ?[JSONPatch{op: "add",path: "/spec/template/metadata/annotations",value: {"backup.velero.io/backup-volumes": variables.deploymentVolumesList}}] :[JSONPatch{op: "add",path: "/spec/template/metadata/annotations/" + jsonpatch.escapeKey("backup.velero.io/backup-volumes"),value: variables.deploymentVolumesList}]) : []- patchType: JSONPatchjsonPatch:expression: |object.kind == "Job" && variables.jobVolumesList != '' ?(!has(object.spec.template.metadata.annotations) ?[JSONPatch{op: "add",path: "/spec/template/metadata/annotations",value: {"backup.velero.io/backup-volumes": variables.jobVolumesList}}] :[JSONPatch{op: "add",path: "/spec/template/metadata/annotations/" + jsonpatch.escapeKey("backup.velero.io/backup-volumes"),value: variables.jobVolumesList}]) : []- patchType: JSONPatchjsonPatch:expression: |object.kind == "CronJob" && variables.cronjobVolumesList != '' ?(!has(object.spec.jobTemplate.spec.template.metadata.annotations) ?[JSONPatch{op: "add",path: "/spec/jobTemplate/spec/template/metadata/annotations",value: {"backup.velero.io/backup-volumes": variables.cronjobVolumesList}}] :[JSONPatch{op: "add",path: "/spec/jobTemplate/spec/template/metadata/annotations/" + jsonpatch.escapeKey("backup.velero.io/backup-volumes"),value: variables.cronjobVolumesList}]) : []
The Kubernetes cluster autoscaler does not evict pods that use hostPath or emptyDir volumes. To allow eviction of these pods, the annotation cluster-autoscaler.kubernetes.io/safe-to-evict=true must be added to the pods.
The Kubernetes cluster autoscaler does not evict pods that use hostPath or emptyDir volumes. To allow eviction of these pods, the annotation cluster-autoscaler.kubernetes.io/safe-to-evict=true must be added to the pods.
CAST AI will not downscale a node that includes a pod with the autoscaling.cast.ai/removal-disabled="true" label on it, this protects sensitive workloads from being evicted and can be attributed to any pod to protect against unwanted downscaling. This policy will mutate jobs and cronjobs to add the removal-disabled label to protect against eviction.