Drop in-built support for bitnami's memcached

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## master / unreleased
 
+* [BREAKING] Drop in-built support for bitnami's memcached #567
+  * memcached dependency and subchart has been removed
+  * Chart users have to take care of the deployment of memcached now (if they require it)
 * [CHANGE] Memcacheds are now enabled by default #551
 * [CHANGE] Change default config: distributors, rulers, queriers, query-frontend, nginx and compactors can be disrupted until 30%
 * [CHANGE] Remove startup probes from compactors from default values. It's not recommended in general #542

Chart.yaml

@@ -15,24 +15,3 @@ maintainers:
 name: cortex
 sources:
   - https://github.com/cortexproject/cortex-helm-chart
-dependencies:
-  - name: memcached
-    alias: memcached-frontend
-    version: 6.14.0
-    repository: https://charts.bitnami.com/bitnami
-    condition: memcached-frontend.enabled
-  - name: memcached
-    alias: memcached-blocks-index
-    version: 6.14.0
-    repository: https://charts.bitnami.com/bitnami
-    condition: memcached-blocks-index.enabled
-  - name: memcached
-    alias: memcached-blocks
-    version: 6.14.0
-    repository: https://charts.bitnami.com/bitnami
-    condition: memcached-blocks.enabled
-  - name: memcached
-    alias: memcached-blocks-metadata
-    version: 6.14.0
-    repository: https://charts.bitnami.com/bitnami
-    condition: memcached-blocks-metadata.enabled

README.md

@@ -84,13 +84,6 @@ Source code can be found [here](https://cortexmetrics.io/)
 
 Kubernetes: `^1.19.0-0`
 
-| Repository | Name | Version |
-|------------|------|---------|
-| https://charts.bitnami.com/bitnami | memcached-frontend(memcached) | 6.14.0 |
-| https://charts.bitnami.com/bitnami | memcached-blocks-index(memcached) | 6.14.0 |
-| https://charts.bitnami.com/bitnami | memcached-blocks(memcached) | 6.14.0 |
-| https://charts.bitnami.com/bitnami | memcached-blocks-metadata(memcached) | 6.14.0 |
-
 ## Values
 
 | Key | Type | Default | Description |
@@ -382,46 +375,10 @@ Kubernetes: `^1.19.0-0`
 | ingress.​tls | list | `[]` |  |
 | memberlist.​service.​annotations | object | `{}` |  |
 | memberlist.​service.​labels | object | `{}` |  |
-| memcached-blocks-index.​architecture | string | `"high-availability"` |  |
-| memcached-blocks-index.​enabled | bool | `true` | Enables support for block index caching |
-| memcached-blocks-index.​extraEnvVars[0] | object | `{"name":"MEMCACHED_CACHE_SIZE","value":"1024"}` | MEMCACHED_CACHE_SIZE is the amount of memory allocated to memcached for object storage |
-| memcached-blocks-index.​extraEnvVars[1] | object | `{"name":"MEMCACHED_MAX_CONNECTIONS","value":"1024"}` | MEMCACHED_MAX_CONNECTIONS is the maximum number of simultaneous connections to the memcached service |
-| memcached-blocks-index.​extraEnvVars[2] | object | `{"name":"MEMCACHED_THREADS","value":"4"}` | MEMCACHED_THREADS is the number of threads to use when processing incoming requests. By default, memcached is configured to use 4 concurrent threads. The threading improves the performance of storing and retrieving data in the cache, using a locking system to prevent different threads overwriting or updating the same values. |
-| memcached-blocks-index.​metrics.​enabled | bool | `true` |  |
-| memcached-blocks-index.​metrics.​serviceMonitor.​enabled | bool | `false` |  |
-| memcached-blocks-index.​replicaCount | int | `2` |  |
-| memcached-blocks-index.​resources | object | `{}` |  |
-| memcached-blocks-index.​service.​clusterIP | string | `"None"` |  |
-| memcached-blocks-metadata.​architecture | string | `"high-availability"` |  |
-| memcached-blocks-metadata.​enabled | bool | `true` | Enables support for block metadata caching |
-| memcached-blocks-metadata.​extraEnvVars[0] | object | `{"name":"MEMCACHED_CACHE_SIZE","value":"1024"}` | MEMCACHED_CACHE_SIZE is the amount of memory allocated to memcached for object storage |
-| memcached-blocks-metadata.​extraEnvVars[1] | object | `{"name":"MEMCACHED_MAX_CONNECTIONS","value":"1024"}` | MEMCACHED_MAX_CONNECTIONS is the maximum number of simultaneous connections to the memcached service |
-| memcached-blocks-metadata.​extraEnvVars[2] | object | `{"name":"MEMCACHED_THREADS","value":"4"}` | MEMCACHED_THREADS is the number of threads to use when processing incoming requests. By default, memcached is configured to use 4 concurrent threads. The threading improves the performance of storing and retrieving data in the cache, using a locking system to prevent different threads overwriting or updating the same values. |
-| memcached-blocks-metadata.​metrics.​enabled | bool | `true` |  |
-| memcached-blocks-metadata.​metrics.​serviceMonitor.​enabled | bool | `false` |  |
-| memcached-blocks-metadata.​replicaCount | int | `2` |  |
-| memcached-blocks-metadata.​resources | object | `{}` |  |
-| memcached-blocks-metadata.​service.​clusterIP | string | `"None"` |  |
-| memcached-blocks.​architecture | string | `"high-availability"` |  |
-| memcached-blocks.​enabled | bool | `true` | Enables support for block caching |
-| memcached-blocks.​extraEnvVars[0] | object | `{"name":"MEMCACHED_CACHE_SIZE","value":"1024"}` | MEMCACHED_CACHE_SIZE is the amount of memory allocated to memcached for object storage |
-| memcached-blocks.​extraEnvVars[1] | object | `{"name":"MEMCACHED_MAX_CONNECTIONS","value":"1024"}` | MEMCACHED_MAX_CONNECTIONS is the maximum number of simultaneous connections to the memcached service |
-| memcached-blocks.​extraEnvVars[2] | object | `{"name":"MEMCACHED_THREADS","value":"4"}` | MEMCACHED_THREADS is the number of threads to use when processing incoming requests. By default, memcached is configured to use 4 concurrent threads. The threading improves the performance of storing and retrieving data in the cache, using a locking system to prevent different threads overwriting or updating the same values. |
-| memcached-blocks.​metrics.​enabled | bool | `true` |  |
-| memcached-blocks.​metrics.​serviceMonitor.​enabled | bool | `false` |  |
-| memcached-blocks.​replicaCount | int | `2` |  |
-| memcached-blocks.​resources | object | `{}` |  |
-| memcached-blocks.​service.​clusterIP | string | `"None"` |  |
-| memcached-frontend.​architecture | string | `"high-availability"` |  |
-| memcached-frontend.​enabled | bool | `true` | Enables support for caching queries in the frontend |
-| memcached-frontend.​extraEnvVars[0] | object | `{"name":"MEMCACHED_CACHE_SIZE","value":"1024"}` | MEMCACHED_CACHE_SIZE is the amount of memory allocated to memcached for object storage |
-| memcached-frontend.​extraEnvVars[1] | object | `{"name":"MEMCACHED_MAX_CONNECTIONS","value":"1024"}` | MEMCACHED_MAX_CONNECTIONS is the maximum number of simultaneous connections to the memcached service |
-| memcached-frontend.​extraEnvVars[2] | object | `{"name":"MEMCACHED_THREADS","value":"4"}` | MEMCACHED_THREADS is the number of threads to use when processing incoming requests. By default, memcached is configured to use 4 concurrent threads. The threading improves the performance of storing and retrieving data in the cache, using a locking system to prevent different threads overwriting or updating the same values. |
-| memcached-frontend.​metrics.​enabled | bool | `true` |  |
-| memcached-frontend.​metrics.​serviceMonitor.​enabled | bool | `false` |  |
-| memcached-frontend.​replicaCount | int | `2` |  |
-| memcached-frontend.​resources | object | `{}` |  |
-| memcached-frontend.​service.​clusterIP | string | `"None"` |  |
+| memcached-blocks | object | `{}` |  |
+| memcached-blocks-index | object | `{}` |  |
+| memcached-blocks-metadata | object | `{}` |  |
+| memcached-frontend | object | `{}` | Memcached properties External installations of memcached can be referenced here |
 | nginx.​affinity | object | `{}` |  |
 | nginx.​annotations | object | `{}` |  |
 | nginx.​autoscaling.​behavior | object | `{}` | Ref: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-configurable-scaling-behavior |

ci/test-configmap-values.yaml

@@ -145,36 +145,3 @@ store_gateway:
   extraVolumeMounts:
     - name: tmp-test
       mountPath: /tmp-test
-
-memcached-frontend:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks-index:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks-metadata:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0

ci/test-deployment-values.yaml

@@ -152,36 +152,3 @@ store_gateway:
   extraVolumeMounts:
     - name: tmp-test
       mountPath: /tmp-test
-
-memcached-frontend:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks-index:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks-metadata:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0

ci/test-sts-values.yaml

@@ -143,36 +143,3 @@ store_gateway:
   extraVolumeMounts:
     - name: tmp-test
       mountPath: /tmp-test
-
-memcached-frontend:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks-index:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0
-memcached-blocks-metadata:
-  enabled: true
-  architecture: standalone
-  replicaCount: 1
-  resources:
-    requests:
-      cpu: 0
-      memory: 0

ct.yaml

@@ -5,5 +5,4 @@ charts:
   - "./"
 chart-repos:
   - base-charts=https://charts.helm.sh/stable
-  - bitnami=https://charts.bitnami.com/bitnami
 helm-extra-args: --timeout 600s

docs/guides/configure_memcached.markdown

@@ -17,42 +17,27 @@ has_toc: false
 ---
 # Configuring memcached as the caching store
 
-Correctly configuring memcached for your needs is a very delicate  task. But courtesy of [@thobianchi](https://github.com/thobianchi) this might be a good start.
+If you want to use [memcached](https://memcached.org/) as a caching store, you first have to deploy it separately (e.g. via separate helm chart). The existing deployments can be referenced in the `values.yaml` so that they are used as caching stores.
+
+Here's how to configure the memcached's via the values:
 
 {% raw %}
 ```yaml
-index_cache:
-  backend: memcached
-  memcached:
-    addresses: 'dns+{{ include "cortex.fullname" $ }}-memcached-blocks-index:11211'
-    timeout: 300ms
-    max_idle_connections: 750
-    max_async_concurrency: 100
-    max_async_buffer_size: 10000000
-    max_get_multi_concurrency: 750
-    max_get_multi_batch_size: 1000
-    max_item_size: 16777216
-chunks_cache:
-  backend: memcached
-  memcached:
-    addresses: 'dns+{{ include "cortex.fullname" $ }}-memcached-blocks:11211'
-    timeout: 300ms
-    max_idle_connections: 750
-    max_async_concurrency: 100
-    max_async_buffer_size: 10000000
-    max_get_multi_concurrency: 750
-    max_get_multi_batch_size: 1000
-    max_item_size: 33554432
-metadata_cache:
-  backend: memcached
-  memcached:
-    addresses: 'dns+{{ include "cortex.fullname" $ }}-blocks-metadata:11211'
-    timeout: 300ms
-    max_idle_connections: 750
-    max_async_concurrency: 100
-    max_async_buffer_size: 10000000
-    max_get_multi_concurrency: 750
-    max_get_multi_batch_size: 1000
-    max_item_size: 16777216
+memcached-frontend:
+  # reference pre-existing service here
+  serviceName: cortex-memcached-frontend
+  containerPort: 11211
+
+memcached-blocks-index:
+  serviceName: cortex-memcached-blocks-index
+  containerPort: 11211
+
+memcached-blocks:
+  serviceName: cortex-memcached-blocks
+  containerPort: 11211
+
+memcached-blocks-metadata:
+  serviceName: cortex-memcached-blocks-metadata
+  containerPort: 11211
 ```
 {% endraw %}

renovate.json

@@ -47,16 +47,6 @@
             ],
             "depNameTemplate": "kiwigrid/k8s-sidecar",
             "datasourceTemplate": "docker"
-        },
-        {
-            "description": "Update memcached image in README.md",
-            "fileMatch": ["^README\\.md$"],
-            "matchStrings": [
-                "\\|[^|]*\\|[^|]*\\(memcached\\)[^|]*\\|\\s+(?<currentValue>.*?)\\s+\\|\\s+"
-            ],
-            "depNameTemplate": "memcached",
-            "registryUrlTemplate": "https://charts.bitnami.com/bitnami",
-            "datasourceTemplate": "helm"
         }
     ]
 }

templates/_helpers.tpl

@@ -85,26 +85,26 @@ app.kubernetes.io/instance: {{ .Release.Name }}
 Create configuration parameters for memcached configuration
 */}}
 {{- define "cortex.memcached" -}}
-{{- if index .Values "memcached-blocks-index" "enabled" }}
+{{- if index .Values "memcached-blocks-index" "serviceName" }}
 - "-blocks-storage.bucket-store.index-cache.backend=memcached"
-- "-blocks-storage.bucket-store.index-cache.memcached.addresses=dns+{{ .Release.Name }}-memcached-blocks-index.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:11211"
+- "-blocks-storage.bucket-store.index-cache.memcached.addresses=dns+{{ .memcached-blocks-index.serviceName }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:{{ memcached-blocks-index.containerPort }}"
 {{- end -}}
-{{- if index .Values "memcached-blocks" "enabled" }}
+{{- if index .Values "memcached-blocks" "serviceName" }}
 - "-blocks-storage.bucket-store.chunks-cache.backend=memcached"
-- "-blocks-storage.bucket-store.chunks-cache.memcached.addresses=dns+{{ .Release.Name }}-memcached-blocks.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:11211"
+- "-blocks-storage.bucket-store.chunks-cache.memcached.addresses=dns+{{ .memcached-blocks.serviceName }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:{{ memcached-blocks.containerPort }}"
 {{- end -}}
-{{- if index .Values "memcached-blocks-metadata" "enabled" }}
+{{- if index .Values "memcached-blocks-metadata" "serviceName" }}
 - "-blocks-storage.bucket-store.metadata-cache.backend=memcached"
-- "-blocks-storage.bucket-store.metadata-cache.memcached.addresses=dns+{{ .Release.Name }}-memcached-blocks-metadata.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:11211"
+- "-blocks-storage.bucket-store.metadata-cache.memcached.addresses=dns+{{ .memcached-blocks-metadata.serviceName }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:{{ memcached-blocks-metadata.containerPort }}"
 {{- end -}}
 {{- end -}}
 
 {{/*
 Create configuration for frontend memcached configuration
 */}}
 {{- define "cortex.frontend-memcached" -}}
-{{- if index .Values "memcached-frontend" "enabled" }}
-- "-frontend.memcached.addresses=dns+{{ .Release.Name }}-memcached-frontend.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:11211"
+{{- if index .Values "memcached-frontend" "serviceName" }}
+- "-frontend.memcached.addresses=dns+{{ .memcached-frontend.serviceName }}.{{ .Release.Namespace }}.svc.{{ .Values.clusterDomain }}:{{ memcached-frontend.containerPort }}"
 {{- end -}}
 {{- end -}}