Repository navigation
[MTA 8.3.0] Changes to the mta-ops doc - #422
Conversation
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📄 Documentation Preview
|
Minor corrections based on AI analysis
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
I'm not sure if it is preview-only issue, but preview shows old MTA version 8.2, instead of 8.3. I will revisit it later, but overall looks good to me! |
@aufi, don't worry, it's just that the attributes haven't been updated yet. When everything gets published on the release day, the version will be correct :-) Would you please also approve this PR? |
| Direct persistent volume claim migration:: | ||
| Direct migration copies file data directly from the source cluster to the destination cluster. This path uses a secure tunnel. The tunnel is established by using `stunnel` with Transport Layer Security (TLS) version 1.3. | ||
| + | ||
| A file synchronization utility (`rsync`) copies the files across the tunnel. This direct transfer requires a network path between the clusters. You must expose a route or an ingress endpoint on the target cluster. This method preserves file mode bits and ownership. |
There was a problem hiding this comment.
Both modes preserve file mode bits. In both modes, file ownership (UID/GID) is preserved on a best-effort basis and may be normalized when the mover pod runs as a non-root user.
There was a problem hiding this comment.
@nachandr, I updated my suggestion, can you please check?
| + | ||
| [subs="+quotes"] | ||
| ---- | ||
| $ *mta-ops transfer-pvc --source-context=_<source_cluster_context>_ --destination-context=_<target_cluster_context>_ --pvc-name=_<pvc_name>_ --pvc-namespace=_<pvc_namespace>_ --cloud-storage=_<s3_bucket_url>_ --rclone-config-file=_<rclone_config_file>_ --encrypt* |
There was a problem hiding this comment.
- Drop the
--encryptflag. - Replace
--cloud-storage=_<s3_bucket_url>_
with--cloud-storage=remote:my-bucket
There was a problem hiding this comment.
@nachandr, is it ok now?
| $ *mta-ops transfer-pvc --source-context=_<source_cluster_context>_ --destination-context=_<target_cluster_context>_ --pvc-name=_<pvc_name>_ --pvc-namespace=_<pvc_namespace>_ --cloud-storage=_<s3_bucket_url>_ --rclone-config-file=_<rclone_config_file>_ --encrypt* | |
| $ *mta-ops transfer-pvc --source-context=_<source_cluster_context>_ --destination-context=_<target_cluster_context>_ --pvc-name=_<pvc_name>_ --pvc-namespace=_<pvc_namespace>_ --cloud-storage=remote:_<bucket_url>_ --rclone-config-file=_<rclone_config_file>_* |
There was a problem hiding this comment.
I'm not sure how much users care about remote:_<my_bucket>_ when only s3 remote is supported in this release. Both text versions look valid to me. Deffering to Nandini.
There was a problem hiding this comment.
I re-read the section on indirect migration and there are a few gaps.
- We are missing examples of using --rclone-config-file and --rclone-config-secret . Let's drop line 24.
- We are also missing sample rclone.conf files.
- We don't pass an S3 bucket URL to --cloud-storage . So, Replace --cloud-storage=<s3_bucket_url>
with --cloud-storage=remote:my-bucket in both examples . - Refer to the Indirect transfer section on https://github.com/migtools/crane/blob/main/docs/commands/transfer-pvc.md for more information.
- We could add an example on using the optional
--encryptflag in a follow up PR. So, drop--encryptfrom all commands in the indirect transfer section.
I'd suggest adding the following to the documentation.
Indirect migration uses rclone to move data through an intermediate
S3-compatible bucket. You must supply an rclone.conf configuration that
defines the remote named in --cloud-storage. Provide it with one of the
following mutually exclusive options.
Option 1: --rclone-config-file (local file)
Pass the path to a local rclone.conf. The command reads the file locally,
creates a temporary Secret containing it in the source and destination PVC
namespaces, validates both, and deletes the temporary Secrets when the transfer
finishes.
Sample command:
$ mta-ops transfer-pvc --source-context=<source_cluster_context> \
--destination-context=<target_cluster_context> \
--pvc-name=<pvc_name> --pvc-namespace=<pvc_namespace> \
--cloud-storage=remote:my-bucket \
--rclone-config-file=./rclone.conf
Option 2: --rclone-config-secret (pre-created Secret)
Create the Secret yourself, with the same name, in both the source and
destination PVC namespaces before running the transfer. The command reads the
existing Secrets and does not create, copy, or delete them.
The Secret must contain a non-empty data key named rclone.conf:
$ oc create secret generic my-rclone-config \
--from-file=rclone.conf=./rclone.conf \
-n <pvc_namespace> --context <source_cluster_context>
$ oc create secret generic my-rclone-config \
--from-file=rclone.conf=./rclone.conf \
-n <pvc_namespace> --context <target_cluster_context>
Then reference it by name:
$ mta-ops transfer-pvc --source-context=<source_cluster_context> \
--destination-context=<target_cluster_context> \
--pvc-name=<pvc_name> --pvc-namespace=<pvc_namespace> \
--cloud-storage=remote:my-bucket \
--rclone-config-secret=my-rclone-config
Sample rclone.conf file for AWS, Minio, GCS S3 buckets
The section name [remote] must match the prefix in --cloud-storage. For example, --cloud-storage "remote:my-bucket" uses the [remote] section.
.AWS S3
[remote]
type = s3
provider = AWS
access_key_id = <access_key>
secret_access_key = <secret_key>
region = <region>
.S3-compatible storage (for example, MinIO)
[remote]
type = s3
provider = Minio
access_key_id = <access_key>
secret_access_key = <secret_key>
endpoint = <https://minio.example.com>
.Google Cloud Storage (S3-compatible / interoperability mode)
[remote]
type = s3
provider = GCS
access_key_id = <hmac_access_key>
secret_access_key = <hmac_secret>
endpoint = https://storage.googleapis.com
There was a problem hiding this comment.
@nachandr, I updated the section in the suggestion above.
|
@mpershina I peer reviewed all suggested changes and left a couple op suggestions. LGTM overall. |
Co-authored-by: Valentina Ashirova <107574498+vashirova@users.noreply.github.com>
|
Change suggestions and the code itself looks good to me, approving 👍 Given phase of the release process, this needs get in, @nachandr if there are some unresolved things, please create an issue for followup, thank you. |
vashirova
left a comment
vashirova
left a comment
There was a problem hiding this comment.
@mpershina I peer reviewed docs/topics/mta-ops/proc_migrating-persistent-volume-data-indirectly-through-cloud-storage.adoc procedure. Apart from one small detail, LGTM.
Co-authored-by: Valentina Ashirova <107574498+vashirova@users.noreply.github.com>
4 participants
Preview: https://mpershina.github.io/Previews/MTA/mta-ops-8.3.0.html
Tickets: