Skip to content

fix: updates with floating tags#2354

Open
maganaluis wants to merge 9 commits intocopier-org:masterfrom
maganaluis:floating-tag-fix
Open

fix: updates with floating tags#2354
maganaluis wants to merge 9 commits intocopier-org:masterfrom
maganaluis:floating-tag-fix

Conversation

@maganaluis
Copy link

@maganaluis maganaluis commented Oct 15, 2025

Problem

Copier stores git refs (tags/branches) in _commit field of .copier-answers.yml. When using moving/floating tags (e.g., workflows/v1, stable/v2, latest), future copier update commands fail because the tag now points to a different commit.

Revision 1

Solution

Added --resolve-commit-to-sha flag that stores immutable SHA hashes instead of git refs.

Implementation:

  • New CLI flag: --resolve-commit-to-sha
  • Template config option: _resolve_commit_to_sha: true in copier.yml
  • Priority: CLI flag > template config > default (false)

Usage:

# CLI
copier copy --resolve-commit-to-sha template-repo destination

# Template config (copier.yml)
_resolve_commit_to_sha: true

Changes:

  • copier/_main.py: Added resolve_commit_to_sha field to Worker class and logic in _answers_to_remember()
  • copier/_cli.py: Added CLI flag
  • copier/_template.py: Added template config support
  • tests/test_resolve_commit_to_sha.py: Coverage for new functionality

Fixes #987

Revision 2

Solution: Copier now stores BOTH semantic version and SHA hash, then automatically
chooses which to use during updates.

Key Changes:

  1. Dual-Versioning - Always stores both values:

    • _commit: Semantic version (human-readable)
    • _commit_sha: SHA hash (machine-reliable)
  2. Automatic Tag Resolution - Detects and handles floating tags:

    • Floating patterns: latest, stable/*, main, master, develop, feat/*,
      etc.
    • Uses SHA for floating tags, preserves semantic versions for stable tags
  3. Renamed Flag:

    • --resolve-commit-to-sha--ignore-git-tags
    • Flag controls USAGE during updates, not storage

Usage:

# Automatic 
copier update  # Automatic tag resolution handles everything

# Force SHA usage
copier update --ignore-git-tags

# Template config (copier.yml)
_ignore_git_tags: true

Implementation:

Modified Files:

  • copier/_main.py: Automatic tag resolution logic, dual-versioning storage (~40 lines
    reduced)
  • copier/_subproject.py: Use SHA for FROM version in diffs
  • copier/_cli.py: Renamed flag
  • copier/_template.py: Config properties for ignore_git_tags and
    stable_tag_patterns
  • tests/test_ignore_git_tags.py: Complete rewrite with end-to-end floating tag tests

Resolution Logic:

1. CLI flag --ignore-git-tags → Use SHA
2. Stable semantic version → Use semantic version
3. Floating tag pattern → Use SHA (automatic)

@maganaluis maganaluis changed the title fix updates with floating tags fix: updates with floating tags Oct 15, 2025
@maganaluis maganaluis marked this pull request as ready for review October 15, 2025 16:29
@sisp
Copy link
Member

sisp commented Oct 16, 2025

Thanks for submitting this PR, @maganaluis! 🙇

A conceptual question before diving into code details: One of the arguments you're documenting is traceability and reproducibility because the SHA is an immutable and unique reference of the template version while a symbolic reference can be mutated. I wonder whether we'd even want both instead of either one or the other. Wouldn't a user of a template with regular SemVer tags also benefit from the additional immutable reference information?

For example, we could introduce a new metadata field in the answers file:

 _commit: v1.2.3
+_commit_sha: e3b0c44298fc1c149afbf4c8996fb92427ae41e4
 _src_path: ...
 ...

And in your case, you'd also know which floating tag is used to identify updates. In case you use a non-SemVer floating tag like stable, we'd need to add support for it, but I imagine this shouldn't be a huge effort – if a tag can't be parsed with packaging.version.parse(...), then it's treated as a floating tag, i.e. the version sorting and comparison is skipped.

WDYT?

/cc @pawamoy

@pawamoy
Copy link
Contributor

pawamoy commented Oct 16, 2025

Just to make sure I understand: we still need the semver tag to know whether we're upgrading or downgrading when we do an update, right? What happens when the floating tag is v1, and you update to v1? Copier will not only have to compare both versions, but also checkout the template to compare commit SHAs. Then does it know if it's an upgrade or a downgrade? Does it use the commit date to know that?

@sisp
Copy link
Member

sisp commented Oct 16, 2025

I think the scenarios are like this:

  • If _commit can be parsed:
    • If Copier finds a newer version, it performs the update as usual and additionally locks the commit SHA in _commit_sha.
    • If Copier finds no newer version, it treats the tag as floating and switches to the new behavior (see next).
  • If _commit cannot be parsed, then it's definitely a floating tag. In this case, Copier resolves the commit SHA of the tag and compares the new commit SHA with the old commit SHA (we still have it because it's recorded in _commit_sha). If they are different, Copier performs an update from the old commit SHA to the new commit SHA (same process as we have already for template repositories without any tags).

@pawamoy
Copy link
Contributor

pawamoy commented Oct 16, 2025

Thanks! Then yeah I don't see any downsides to recording the commit SHA and using it when we can't parse the version or when there's no new version (except maybe a longer execution time when a project is updated and there's indeed no new version? but that's not very impactful) 👍

@maganaluis
Copy link
Author

@sisp

This is a reasonable ask, I think I would still leave the setting --resolve-commit-to-sha but would rename this to --use-commit-sha so it doesn't attempt to resolve the semantic tags and it uses _commit_sha primarily. I want to do this on the safe side since there might be instances where the _commit could be parsed, and it might still be a floating tag. I will create another revision this week.

I think the scenarios are like this:

* If `_commit` can be parsed:
  
  * If Copier finds a newer version, it performs the update as usual and additionally locks the commit SHA in `_commit_sha`.
  * If Copier finds no newer version, it treats the tag as floating and switches to the new behavior (see next).

* If `_commit` cannot be parsed, then it's definitely a floating tag. In this case, Copier resolves the commit SHA of the tag and compares the new commit SHA with the old commit SHA (we still have it because it's recorded in `_commit_sha`). If they are different, Copier performs an update from the old commit SHA to the new commit SHA (same process as we have already for template repositories without any tags).

@sisp
Copy link
Member

sisp commented Oct 21, 2025

Sounds good. Just some additional ideas for the flag name:

  • --ignore-tags / --ignore-git-tags
  • --no-tags / --no-git-tags – a bit similar to git fetch --no-tags

@maganaluis
Copy link
Author

maganaluis commented Oct 28, 2025

I think this is at a good place now for a second revision.

@maganaluis
Copy link
Author

@sisp are there any additional changes we need to make to get this merged?

@sisp
Copy link
Member

sisp commented Dec 18, 2025

Sorry for my delayed reply, I've been very busy and wanted to dedicate time to give proper feedback.

I've been thinking about the relationship between --ignore-git-tags and --vcs-ref :current:, and I think the UX and semantics aren't quite right yet:

  • copier copy --ignore-git-tags ... seems to be identical to copier copy ... because there is no previously recorded Git tag to reuse.

  • copier update --vcs-ref :current: and copier update --ignore-git-tags imply similar but not identical behavior:

    --vcs-ref :current: --ignore-git-tags
    Search for new ref ✖️ ✖️
    Re-resolve ref's SHA ✔️ ✖️
    • For an immutable SemVer tag/ref, the following commands are identical because the resolved SHA never changes:

      copier update --vcs-ref :current:
      copier update --ignore-git-tags
      copier update --vcs-ref :current: --ignore-git-tags
    • For an automatically detected floating tag/ref, the following commands are identical because the tag/ref name is never updated while the resolved SHA is re-resolved in both cases:

      copier update
      copier update --vcs-ref :current:
    • For a floating tag/ref that is not detected as such (i.e., detected as an immutable SemVer tag/ref), we have the following behavior:

      # Keeps the tag/ref but re-resolves the SHA
      copier update --vcs-ref :current:
      
      # Keeps both the tag/ref and the SHA
      copier update --ignore-git-tags

Please correct me if I'm wrong.

The original intent of the special value :current: passed to the --vcs-ref flag was to ease updating the answers of the questionnaire without updating the template version. This works as expected for immutable SemVer tags but fails for floating tags/refs, as only the ref is reused but the SHA is re-resolved. IIUC, the current behavior of the --ignore-git-tags flag reuses the recorded values of _commit and _commit_sha, which restores the original intent of --vcs-ref :current: for floating tags/refs.

So, perhaps this wall of text boils down to a simple naming problem. How about renaming --ignore-git-tags to, e.g., --frozen (inspired from uv's uv sync --frozen flag)? Then, --frozen means that the template version is fully frozen – both the ref and the SHA.

Consider the following scenarios:

  • For immutable SemVer tags:

    # Update to a new template version
    copier update
    
    # Keep the template version, allow updating only answers
    # (a) legacy
    copier update --vcs-ref :current:
    # (b) preferred because more consistent with floating tags/refs and a full version freeze
    copier update --frozen
    # (c) redundant, perhaps raise a warning?
    copier update --vcs-ref :current: --frozen
  • For floating tags/refs (automatically detected as such):

    # Update to a new template version by re-resolving the SHA
    copier update
    copier update --vcs-ref :current:
    
    # Keep the template version, allow updating only answers
    copier update --frozen
  • For floating tags/refs (detected as immutable SemVer tags/refs):

    # Update to a new template version by re-resolving the SHA
    copier update --vcs-ref :current:
    
    # Keep the template version, allow updating only answers
    copier update --frozen

WDYT?

@maganaluis
Copy link
Author

maganaluis commented Jan 31, 2026

I'm not sure if it's best to use frozen given the intent of this flag is to rely on the SHA rather than the semantic version. See without the flag, I have template with wokrflows/v1 which points to some commit 1234, if a user uses the template it will point to 1234, and it will use workflows/v1. Next iteration of the template will still use workflows/v1 but will point to commit 1235 and the users will not be able to update their projects..

So the intention of the flag is to be leveraged on updates, and to solve the floating tag issue. There might be more use cases, but I think the scope of the PR for now should remain on solving the core issue with floating tags. Unless you think there might be other situation where this could be useful.

I've updated the PR to remove the sem tags pattern, and also fixed a bug where the template doesn't use the flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autoselecting version tags

3 participants