Skip to content

Conversation

@ysiraichi
Copy link
Collaborator

This PR fixes #9691 by making PyTorch/XLA library visibility property hidden by default.
This is to make sure we don't share data unexpectedly with other libraries.

@qihqi qihqi merged commit d0bcacd into pytorch:master Nov 4, 2025
20 checks passed
@ysiraichi
Copy link
Collaborator Author

Here's an intuition of why I think this fixes a previously undesirable PyTorch/XLA behavior:

  • google::protobuf::internal::ShutdownData is a protobuf C++ class
  • ShutdownData::get() is a singleton function that retrieves an instance of that class
    • It's stored in its function-local variable data
# ====================================================
# Before this PR :
#   - `data` is marked as a 'unique global symbol' (dynamic linker guarantees one copy per process)
#   - `data` is shared with other libraries that also have this symbol
#
$ nm -C _XLAC.cpython-312-x86_64-linux-gnu.so | grep ShutdownData
0000000017351dd0 u guard variable for google::protobuf::internal::ShutdownData::get()::data
0000000017351dd8 u google::protobuf::internal::ShutdownData::get()::data

# ====================================================
# After this PR:
#   - `data` is a local symbol visible to this library only
#   - Other libraries can't get access to `data`
#
$ nm -C  _XLAC.cpython-312-x86_64-linux-gnu.so | grep ShutdownData
000000000d694c58 b guard variable for google::protobuf::internal::ShutdownData::get()::data
000000000d694c60 b google::protobuf::internal::ShutdownData::get()::data

@jeffhataws
Copy link
Collaborator

@ysiraichi @qihqi will you cherry-pick to r2.9 branch? Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[torch-xla 2.9RC1] Random crash in sentencepiece with torch-xla 2.9 when doing vocab loading

3 participants