ENH: Preserve nullable boolean dtype in pivot_table (GH#62244) #62256
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jbrockmendel Please let me know if it needs any other changes. |
|
@jbrockmendel Updated the documentation too! Let me know if this needs any other changes. |
skalwaghe-56
force-pushed
the
fix-62244-unstack-bool-dtype-memory
branch
from
59565e6 to
95b4f5a
Compare
|
Nullable dtypes are opt-in. We don't silently cast to them. This issue will probably go un-resolved until PDEP16 is fully implemented. |
- Convert bool/object columns to BooleanDtype - Skip dtype conversion for margin columns that are DataFrames - Updated test_pivot_table_bool_preserves_boolean_dtype with safe assertions
- Added release note to v3.0.0.rst under Enhancements and Bug fixes, describing that boolean columns are now preserved as nullable BooleanDtype in pivoting and unstacking operations, with missing values as pd.NA. - References issue pandas-dev#62244.
skalwaghe-56
force-pushed
the
fix-62244-unstack-bool-dtype-memory
branch
from
990800f to
98ac57d
Compare
|
@jbrockmendel What do you suggest we do? Close this PR or keep it open? Thank you for your guidance! |
|
Close it, yes. |
|
@jbrockmendel Closing the PR. Thank you! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Convert bool/object columns to BooleanDtype
Skip dtype conversion for margin columns that are DataFrames
Updated test_pivot_table_bool_preserves_boolean_dtype with safe assertions
closes PERF:
DataFrame.unstack()andDataFrame.pivot_table()upcasting take up more memory than needed #62244Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added an entry in the latest
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.ENH: Preserve nullable boolean dtype in pivot_table (GH#62244)
This PR ensures pivot_table preserves nullable boolean dtype instead of upcasting to float
for boolean columns or object columns containing only booleans. It also skips dtype conversion
for margin columns that are returned as DataFrames.
Includes updated test_pivot_table_bool_preserves_boolean_dtype with safe assertions
(assert / assert not) to comply with linting rules.