|
| 1 | ++++ |
| 2 | +title = "Database Configuration" |
| 3 | +description = "" |
| 4 | +weight = 4 |
| 5 | ++++ |
| 6 | + |
| 7 | +{{% kfp-v2-keywords %}} |
| 8 | + |
| 9 | +Kubeflow Pipelines (KFP) uses a relational database to store pipeline definitions, run history, and other metadata. The API server is the main component that interacts with the database. |
| 10 | + |
| 11 | +By default, KFP deploys with MySQL. PostgreSQL is also available as an experimental option. |
| 12 | + |
| 13 | +## MySQL (Default) |
| 14 | + |
| 15 | +MySQL is the default database backend for KFP. It is automatically configured when you deploy KFP using the standard kustomize manifests. No additional database configuration is needed. |
| 16 | + |
| 17 | +To deploy KFP with MySQL using the platform-agnostic overlay: |
| 18 | + |
| 19 | +```bash |
| 20 | +export PIPELINE_VERSION={{% pipelines/latest-version %}} |
| 21 | +kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION" |
| 22 | +kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io |
| 23 | +kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic?ref=$PIPELINE_VERSION" |
| 24 | +``` |
| 25 | + |
| 26 | +For full installation instructions, see the [Installation](/docs/components/pipelines/operator-guides/installation/) guide. |
| 27 | + |
| 28 | +## PostgreSQL (Experimental) |
| 29 | + |
| 30 | +{{% alert title="Warning" color="warning" %}} |
| 31 | +PostgreSQL support is experimental and not yet included in a stable release. It is expected to be available starting from v3.0.0. |
| 32 | +{{% /alert %}} |
| 33 | + |
| 34 | +KFP provides a dedicated kustomize overlay for PostgreSQL-based deployments at `manifests/kustomize/env/platform-agnostic-postgresql`. |
| 35 | + |
| 36 | +### Deploying with PostgreSQL After Official Release |
| 37 | + |
| 38 | +Once PostgreSQL support is included in a stable release (v3.0.0+), the base images will have PostgreSQL support built in. You can deploy KFP with PostgreSQL using the `platform-agnostic-postgresql` overlay: |
| 39 | + |
| 40 | +```bash |
| 41 | +export PIPELINE_VERSION=<version-with-postgresql-support> |
| 42 | +kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION" |
| 43 | +kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io |
| 44 | +kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-postgresql?ref=$PIPELINE_VERSION" |
| 45 | +``` |
| 46 | + |
| 47 | +### Early Access: Building from Source |
| 48 | + |
| 49 | +The current stable release images do not include PostgreSQL support. If you want to try PostgreSQL before the official release, you need to build images from the `master` branch of the [kubeflow/pipelines](https://github.com/kubeflow/pipelines) repository. |
| 50 | + |
| 51 | +#### Build the Required Images |
| 52 | + |
| 53 | +Clone the repository and build the images: |
| 54 | + |
| 55 | +```bash |
| 56 | +cd backend |
| 57 | +make image_api_server |
| 58 | +make image_cache |
| 59 | +``` |
| 60 | + |
| 61 | +#### Push to Your Registry |
| 62 | + |
| 63 | +Tag and push the built images to your own container registry: |
| 64 | + |
| 65 | +```bash |
| 66 | +docker tag gcr.io/ml-pipeline/api-server:latest your-registry/kfp-api-server:your-tag |
| 67 | +docker tag gcr.io/ml-pipeline/cache-server:latest your-registry/kfp-cache-server:your-tag |
| 68 | +docker push your-registry/kfp-api-server:your-tag |
| 69 | +docker push your-registry/kfp-cache-server:your-tag |
| 70 | +``` |
| 71 | + |
| 72 | +#### Create a Custom Overlay |
| 73 | + |
| 74 | +Create a kustomize overlay that references the `platform-agnostic-postgresql` base and overrides the image references with your custom-built images: |
| 75 | + |
| 76 | +```bash |
| 77 | +mkdir -p my-postgresql-overlay |
| 78 | +cd my-postgresql-overlay |
| 79 | +``` |
| 80 | + |
| 81 | +Create a `kustomization.yaml`: |
| 82 | + |
| 83 | +```yaml |
| 84 | +apiVersion: kustomize.config.k8s.io/v1beta1 |
| 85 | +kind: Kustomization |
| 86 | + |
| 87 | +resources: |
| 88 | + - ../manifests/kustomize/env/platform-agnostic-postgresql |
| 89 | + |
| 90 | +images: |
| 91 | + - name: ghcr.io/kubeflow/kfp-api-server |
| 92 | + newName: your-registry/kfp-api-server |
| 93 | + newTag: your-tag |
| 94 | + - name: ghcr.io/kubeflow/kfp-cache-server |
| 95 | + newName: your-registry/kfp-cache-server |
| 96 | + newTag: your-tag |
| 97 | +``` |
| 98 | +
|
| 99 | +#### Deploy |
| 100 | +
|
| 101 | +Apply your custom overlay: |
| 102 | +
|
| 103 | +```bash |
| 104 | +kubectl apply -k my-postgresql-overlay |
| 105 | +``` |
0 commit comments