Medusa Purge Job With bitnami/kubectl Image

[vldops]

What happened?

The Medusa purge backup job is failing because it references a non-existent Docker image bitnami/kubectl:1.29.3. The purge job pods fail with ImagePullBackOff as this image does not exist on Docker Hub.

Did you expect to see something different?

The purge backup job should use a valid, existing kubectl image and successfully execute the MedusaTask creation to purge old backups according to the schedule.

How to reproduce it (as minimally and precisely as possible):

1. Deploy a K8ssandraCluster with Medusa backup configuration enabled
2. Configure purgeBackups: true in the Medusa spec
3. Create a MedusaBackupSchedule with operationType: purge
4. Wait for the scheduled purge job to execute
5. Observe the job pod failing with ImagePullBackOff error

Environment

• K8ssandra Operator version:

  1.27.0

• Kubernetes version information:

Server Version: v1.33.5-eks-113cf36

• Manifests:

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: my-cluster
  namespace: cassandra-system
  annotations:
    k8ssandra.io/autoupdate-spec: always
spec:
  auth: true
  cassandra:
    clusterName: my-cluster
    perNodeConfigInitContainerImage: k8ssandra/k8ssandra-client:v0.6.3
    serverType: cassandra
    serverVersion: 4.1.8
    serviceAccount: cassandra-sa
    config:
      cassandraYaml:
        concurrent_compactors: 2
      jvmOptions:
        gc: G1GC
        heap_initial_size: 8G
        heap_max_size: 8G
    datacenters:
    - metadata:
        name: eu-central-1
      size: 1
      storageConfig:
        cassandraDataVolumeClaimSpec:
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 500Gi
          storageClassName: gp3
  medusa:
    cassandraUserSecretRef:
      name: medusa-secret
    purgeBackups: true
    storageProperties:
      backupGracePeriodInDays: 10
      bucketName: my-backups
      concurrentTransfers: 10
      credentialsType: role-based
      maxBackupAge: 10
      maxBackupCount: 10
      prefix: backup-prefix
      region: eu-central-1
      storageProvider: s3
---
apiVersion: medusa.k8ssandra.io/v1alpha1
kind: MedusaBackupSchedule
metadata:
  name: backup-schedule
  namespace: cassandra-system
spec:
  backupSpec:
    backupType: differential
    cassandraDatacenter: eu-central-1
  cronSchedule: 0 3 * * *
  disabled: false
---
apiVersion: medusa.k8ssandra.io/v1alpha1
kind: MedusaBackupSchedule
metadata:
  name: purge-schedule
  namespace: cassandra-system
spec:
  backupSpec:
    cassandraDatacenter: eu-central-1
  cronSchedule: 0 0 * * *
  operationType: purge

• K8ssandra Operator Logs:

Pod manifest showing the non-existent image:

spec:
  containers:
  - command:
    - /bin/bash
    - -c
    - 'printf "apiVersion: medusa.k8ssandra.io/v1alpha1\nkind: MedusaTask\nmetadata:\n  name:
      purge-backups-timestamp\n  namespace: cassandra-system\nspec:\n  cassandraDatacenter:
      eu-central-1\n  operation: purge" | sed "s/timestamp/$(date +%Y%m%d%H%M%S)/g"
      | kubectl apply -f -'
    image: bitnami/kubectl:1.29.3
    name: k8ssandra-purge-backups

status:
  containerStatuses:
  - image: bitnami/kubectl:1.29.3
    ready: false
    restartCount: 0
    started: false
    state:
      waiting:
        message: 'failed to pull and unpack image "docker.io/bitnami/kubectl:1.29.3":
          failed to resolve reference "docker.io/bitnami/kubectl:1.29.3": not found'
        reason: ImagePullBackOff

Anything else we need to know?:

The image bitnami/kubectl:1.29.3 does not exist on Docker Hub. In versions before 1.27.0, this was hardcoded, but in 1.27.0 I cannot find where this image is configured or how to override it.

Requests:

1. Fix the default kubectl image to use a valid existing image
2. Document where/how to configure the kubectl image used for purge jobs
3. Consider allowing this to be configurable via the K8ssandraCluster or MedusaBackupSchedule CRD

This is currently blocking all automated backup purge operations.

[iAlex97]

Hi,
We've struck the same issue, however it could be worked around quite easily by disabling the CronJob and using a MedusaBackupSchedule as described in the docs https://docs.k8ssandra.io/tasks/backup-restore/#scheduling-backup-purges

Edit: I see you're already using a MedusaBackupSchedule for purging, in this case you have 2 mechanisms who accomplish the same thing. In this case you just have to disable the CronJob by doing:

spec:
  medusa:
    # do not create CronJob, we will use MedusaBackupSchedule to do this
    purgeBackups: false