Skip to content

Commit db30bd6

Browse files
mikeruumikeruu
authored
added vsphere and velero docs (#49)
1 parent 154213b commit db30bd6

File tree

2 files changed

+269
-0
lines changed

2 files changed

+269
-0
lines changed

‎docs/velero-with-swift-vsphere-csi-config-guide.md‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Velero Backup Configuration
2+
3+
## Overview
4+
Velero provides backup and disaster recovery for Kubernetes clusters using OpenStack Swift for object storage and vSphere CSI for volume snapshots.
5+
6+
## Key Configuration Choices
7+
8+
### Storage Backend
9+
```yaml
10+
backupStorageLocation:
11+
- name: iad3-flex-dei7343-a9256
12+
provider: community.openstack.org/openstack
13+
bucket: k8s-dr-velero
14+
```
15+
**Why**: Uses OpenStack Swift via the community plugin for backup metadata storage.
16+
17+
**Note**: Initially attempted to use the AWS S3 plugin with Swift's S3-compatible endpoint, but Swift doesn't support AWS chunked uploads (used for large objects). The native OpenStack plugin provides better compatibility with Swift's API.
18+
19+
### CSI Snapshot Integration
20+
```yaml
21+
configuration:
22+
features: EnableCSI
23+
defaultSnapshotMoveData: false
24+
defaultVolumesToFsBackup: false
25+
volumeSnapshotLoc []
26+
```
27+
**Why**:
28+
- `defaulCSI`: Enables CSI snapshot support for volume backups
29+
- `defaultVolushotMoveData: false`: Uses CSI snapshots instead of file-level backups by default
30+
- `defaultVolumesToFsBackup: false`: Prevents automatic file-level backups (opt-in only)
31+
- `volumeSnapshotLocation: []`: Disables legacy VolumeSnapshotLocation (CSI uses VolumeSnapshotClass instead)
32+
33+
### VolumeSnapshotClass
34+
```yaml
35+
extraObjects:
36+
- apiVersion: snapshot.storage.k8s.io/v1
37+
kind: VolumeSnapshotClass
38+
metadata:
39+
name: velero-vsphere-snapshot-class
40+
labels:
41+
velero.io/csi-volumesnapshot-cl "true"
42+
driver: csi.vspher
43+
deletionPolicy: Delete
44+
```
45+
**Why**: Defines how Velero creates CSI snapshots. The label `velero.io/csi-volumesnapshot-class: "true"` tells Velero to use this class for backups.
46+
47+
### Credentials
48+
```yaml
49+
podEnvFrom:
50+
- secretRef:
51+
name: cloud-credentials
52+
```
53+
**Why**: OpenStack plugin requires environment variables (`OS_AUTH_URL`, `OS_APPLICATION_CREDENTIAL_ID`, etc.) for authentication. The secret contains both individual env vars and a `cloud` key for Velero's credential file mount.
54+
55+
### Node Agent
56+
```yaml
57+
deployNodeAgent: false
58+
```
59+
**Why**: Currently disabled. Kopia (the file-level backup engine) doesn't support OpenStack Swift backend. Requires further research into:
60+
- Using S3-compatible Swift endpoint for Kopia
61+
- Alternative storage backends for file-level backups
62+
- Hybrid approach with separate storage for file-level vs metadata
63+
64+
CSI snapshots provide sufficient backup coverage for current needs.
65+
66+
## Common Pitfalls
67+
68+
### Kopia Backend Incompatibility
69+
**Problem**: "invalid backend type community.openstack.org/openstack" errors during file-level backups.
70+
71+
**Solution**: Kopia (used by node-agent) doesn't support OpenStack. Set `defaultVolumesToFsBackup: false` to use CSI snapshots by default.
72+
73+
### Missing Environment Variables
74+
**Problem**: "Missing input for argument [auth_url]" authentication errors.
75+
76+
**Solution**: The secret must be mounted as environment variables using `podEnvFrom`. Individual `OS_*` keys in the secret are required, not just a clouds.yaml file.
77+
78+
### Swift Temp URL Authentication
79+
**Problem**: "401 Unauthorized: Temp URL invalid" errors.
80+
81+
**Solution**: Both `OS_SWIFT_TEMP_URL_KEY` and `OS_SWIFT_TEMP_URL_DIGEST` are required in the credentials secret. These must match the temp URL key configured on the Swift container.
82+
83+
```bash
84+
# Set temp URL key on Swift container (if not already set)
85+
swift post -m "Temp-URL-Key: <your-key>"
86+
```
87+
88+
The `OS_SWIFT_TEMP_URL_KEY` value must match the key set on the container, and `OS_SWIFT_TEMP_URL_DIGEST` specifies the hash algorithm (typically `sha256`).
89+
90+
### VolumeSnapshotLocation Errors
91+
**Problem**: "spec.provider: Required value" during Helm upgrade.
92+
93+
**Solution**: Set `volumeSnapshotLocation: []` to disable legacy snapshot locations. CSI snapshots use VolumeSnapshotClass instead.
94+
95+
### Pod Security Standards
96+
**Problem**: node-agent DaemonSet fails with "violates PodSecurity" errors.
97+
98+
**Solution**: Velero namespace requires `privileged` Pod Security Standard for hostPath volumes.
99+
100+
## Required Secrets
101+
102+
### cloud-credentials
103+
Contains OpenStack authentication credentials. Must include both environment variables and a `cloud` key.
104+
105+
```yaml
106+
stringData:
107+
# Environment variables for OpenStack plugin
108+
OS_AUTH_URL: https://keystone.api.iad3.rackspacecloud.com/v3
109+
OS_APPLICATION_CREDENTIAL_ID: <credential-id>
110+
OS_APPLICATION_CREDENTIAL_SECRET: <credential-secret>
111+
OS_REGION_NAME: IAD3
112+
OS_SWIFT_TEMP_URL_KEY: <temp-url-key>
113+
OS_SWIFT_TEMP_URL_DIGEST: sha256
114+
```
115+
116+
**Key Fields**:
117+
- `OS_AUTH_URL`: OpenStack Keystone endpoint (required)
118+
- `OS_APPLICATION_CREDENTIAL_ID`: Application credential ID (required)
119+
- `OS_APPLICATION_CREDENTIAL_SECRET`: Application credential secret (required)
120+
- `OS_REGION_NAME`: OpenStack region (required)
121+
- `OS_SWIFT_TEMP_URL_KEY`: Temp URL key for Swift authentication (required, must match container setting)
122+
- `OS_SWIFT_TEMP_URL_DIGEST`: Hash algorithm for temp URLs (required, typically `sha256`)
123+
124+
## Verification
125+
```bash
126+
# Check backup storage location
127+
kubectl get backupstoragelocation -n velero
128+
129+
# Verify CSI snapshot class
130+
kubectl get volumesnapshotclass velero-vsphere-snapshot-class
131+
132+
# Test backup
133+
velero backup create test --include-namespaces=default
134+
135+
# Check backup status
136+
velero backup describe test --details
137+
138+
# View backup logs
139+
velero backup logs test
140+
```
141+
142+
## Backup Usage
143+
144+
### CSI Snapshot Backup (Current Method)
145+
```bash
146+
velero backup create my-backup --include-namespaces=myapp
147+
```
148+
149+
### File-Level Backup (Future)
150+
File-level backups via node-agent are currently disabled and require:
151+
- Compatible storage backend (Kopia doesn't support OpenStack)
152+
- Additional testing and validation
153+
- Possible migration to S3-compatible Swift endpoint or alternative backend
154+
155+
For now, all backups use CSI snapshots exclusively.

‎docs/vsphere-csi-config-guide.md‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# vSphere CSI Driver Configuration
2+
3+
## Overview
4+
vSphere CSI driver provides persistent storage and snapshot capabilities for Kubernetes workloads running on vSphere infrastructure.
5+
6+
## Key Configuration Choices
7+
8+
### Snapshot Support
9+
```yaml
10+
controller:
11+
replicaCount: 3
12+
config:
13+
block-volume-snapshot: true
14+
snapshotter:
15+
image:
16+
registry: registry.k8s.io
17+
repository: sig-storage/csi-snapshotter
18+
tag: v8.2.0
19+
```
20+
**Why**:
21+
- `block-volume-snapshot: true`: Enables block volume snapshot capability in the CSI driver
22+
- `snapshotter` sidecar: Required for the CSI controller to handle VolumeSnapshot requests from Velero
23+
24+
Both settings are required for CSI snapshot functionality.
25+
26+
### Snapshot Controller
27+
```yaml
28+
snapshot:
29+
controller:
30+
enabled: true
31+
```
32+
**Why**: Deploys the snapshot-controller which watches VolumeSnapshot resources and coordinates with the CSI driver to create snapshots.
33+
34+
## Common Pitfalls
35+
36+
### Missing Snapshotter Sidecar
37+
**Problem**: VolumeSnapshots stuck in "Waiting for CSI driver" state.
38+
39+
**Solution**: The `controller.snapshotter` configuration must be present in helm values. The snapshotter sidecar container is NOT enabled by default and must be explicitly configured.
40+
41+
**Verification**:
42+
```bash
43+
kubectl get pod -n vmware-system-csi <controller-pod> -o jsonpath='{.spec.containers[*].name}'
44+
```
45+
Should include `csi-snapshotter` in the output.
46+
47+
### Pod Security Standards
48+
**Problem**: CSI pods fail to start with "violates PodSecurity" errors.
49+
50+
**Solution**: The vmware-system-csi namespace requires `privileged` Pod Security Standard due to hostPath volumes and privileged containers.
51+
52+
```yaml
53+
metadata:
54+
labels:
55+
pod-security.kubernetes.io/enforce: privileged
56+
```
57+
58+
## Required Secrets
59+
60+
### vsphere-config-secret (CSI Driver)
61+
Contains vSphere connection details for the CSI driver. Key: `csi-vsphere.conf`
62+
63+
```ini
64+
[Global]
65+
cluster-id = "k8s-dr"
66+
67+
[VirtualCenter "vcenter.example.com"]
68+
insecure-flag = "true"
69+
user = "[email protected]"
70+
password = "password"
71+
port = "443"
72+
datacenters = "Datacenter1"
73+
```
74+
75+
**Key Fields**:
76+
- `cluster-id`: Unique identifier for this Kubernetes cluster
77+
- `insecure-flag`: Set to "true" for self-signed certificates
78+
- `datacenters`: vSphere datacenter name(s)
79+
80+
### vsphere-cpi-secret (Cloud Provider Interface)
81+
Contains vSphere configuration for the CPI. Key: `vsphere.conf`
82+
83+
```yaml
84+
global:
85+
port: 443
86+
insecureFlag: true
87+
88+
vcenter:
89+
vcenter-name:
90+
server: vcenter.example.com
91+
user: [email protected]
92+
password: "password"
93+
datacenters:
94+
- Datacenter1
95+
```
96+
97+
**Key Fields**:
98+
- `vcenter-name`: Arbitrary name for this vCenter (used as identifier)
99+
- `server`: vCenter hostname or IP
100+
- `datacenters`: List of datacenter names
101+
102+
**Note**: Both secrets use the same vSphere credentials but different formats (INI vs YAML).
103+
104+
## Verification
105+
```bash
106+
# Check CSI driver is registered
107+
kubectl get csidrivers csi.vsphere.vmware.com
108+
109+
# Verify snapshot controller is running
110+
kubectl get pods -n vmware-system-csi | grep snapshot-controller
111+
112+
# Test snapshot capability
113+
kubectl get volumesnapshotclass
114+
```

0 commit comments

Comments
 (0)