Skip to content

Conversation

@sfc-gh-aling
Copy link
Contributor

@sfc-gh-aling sfc-gh-aling commented Dec 9, 2025

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-NNNNNNN

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
    • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines
  3. Please describe how your code solves the related issue.

two changes:

  1. the current cte optimization would copy the original plan without copying attributes, and it leads to sending extra describe queries when solving conflicting column alias, which can potentially leads to performance regression.

under the parameter protection of CTE optimization and reduce describe query, we copy attributes when copying a snowflakeplan

  1. in CTE optimization, we create a deterministic CTE table name given a query under the parameter protection of reduce desc query and snowpark connect mode, otherwise whenever dataframe.query is called, we will create a new CTE table name for the same dataframe, thus sending extra query to resolve attributes.

this is a pattern identified in snowpark connect that the protobuf of the same spark DF can be parsed multiple times (analyzePlan and execute), leading to dataframe.query is called multiple times, generating different CTE tables

@sfc-gh-aling sfc-gh-aling force-pushed the aling-cte-investigation branch 3 times, most recently from c45541f to b659af7 Compare December 19, 2025 01:13
@sfc-gh-aling sfc-gh-aling changed the title SNOW-2430412: attributes should be copied for selectable SNOW-2430412: attributes copied for selectable and create deterministic CTE name to reduce describe query Dec 23, 2025
@sfc-gh-aling sfc-gh-aling marked this pull request as ready for review December 23, 2025 18:14
@sfc-gh-aling sfc-gh-aling requested review from a team as code owners December 23, 2025 18:14
@sfc-gh-aling sfc-gh-aling force-pushed the aling-cte-investigation branch from c913718 to f243e22 Compare January 20, 2026 19:39
@sfc-gh-aling sfc-gh-aling merged commit 712afe9 into main Jan 20, 2026
27 of 29 checks passed
@sfc-gh-aling sfc-gh-aling deleted the aling-cte-investigation branch January 20, 2026 20:52
@github-actions github-actions bot locked and limited conversation to collaborators Jan 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants