[Python] Fix WriteToBigQuery transform using CopyJob does not work with WRITE_TRUNCATE write disposition (#34247)

[portikCoder]

I was thinking first to use only datasetId+tableId, but i changed my mind, since it makes better sense to use the full reference instead. All the other details can be found in the mentioned bug filing #34247.

Tests

I added integration tests.
Alongside it, I modified the package and used local executor to test the endresult manually (devtest), and worker fine. Didn't test it on master tho, just on the release branch.

NOTE

Originally I aimed for the 2.63.0 branch so it gets fixed at least there and OFC in the next release candidate. If u tell me don't bother this release I'm also fine, but I need to have a working solution ASAP, I don't really care if it's the new version (if it comes alive soon-ish) or the currently latest stable.
I had to figure out the hard way, since nobody was telling me it's not the way you release hotfixes. Anyways, if it'd be still possible to release it as a hotfix, I'd appreciate.

Also, is good to mention probably that I've checked couple releases back and this code didn't change, so it existed for a long time now, exactly since its creation time (13.02.2020 #10668).

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[portikCoder]

retest this please

[github-actions]

Assigning reviewers:

R: @claudevdm for label python.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[portikCoder]

I opened this to master instead, here is why:
#34248 (comment)

[portikCoder]

When u're done with the review, I'll proceed to update the CHANGES.md as well.

[portikCoder]

Hi,
just friendly reminder so the fix can be released next week. Ty!

[portikCoder]

Hi @liferoad, do you see a chance this getting into the current week's release, please?

[liferoad]

@ahmedabu98 can you review the PR?

@portikCoder thanks for fixing this. Let the team review the PR first. It looks good to me.

[ahmedabu98]

Just a nit, but this looks great! Thanks for the thorough testing. LGTM

[portikCoder]

Just a nit, but this looks great! Thanks for the thorough testing. LGTM

Thanks! Updated the CHANGES.md file too. I expect the pipelines to succeed. Do I have rights to merge, or u need to do it?

[ahmedabu98]

I can merge when tests go green.

FYI merging permissions are for committers, see https://beam.apache.org/contribute/become-a-committer/