Repository navigation
Don't scale binary table columns without TSCAL/TZERO - #50
Merged
Merged
Conversation
Without TSCAL/TZERO the scale and zero default to 1.0f0 and 0.0f0, so reading a column computed zero .+ scale.*data and promoted integer data to Float32. Int32 values above 2^24 were silently rounded (16777217 -> 16777216), and fixed-width integer columns came back as Float32. This broke variable-length Int32 columns holding large offsets, e.g. the pointer table of XSTAR's atdb.fits. Skip scaling when the scale is the identity so the data pass through with their stored type. Add a regression test with an astropy-written file. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #50 +/- ##
==========================================
+ Coverage 46.35% 47.53% +1.17%
==========================================
Files 17 17
Lines 2550 2552 +2
==========================================
+ Hits 1182 1213 +31
+ Misses 1368 1339 -29 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a binary table column has no
TSCAL/TZERO, the defaults are1.0f0/0.0f0(Float32), and the readers computefield.zero .+ field.scale.*data. That promotes integer data to Float32, so Int32 values above 2^24 are silently rounded:For a variable-length (
PJ) column the Float32 result is converted back into theVector{Int32}, so the type looks right but the values are wrong. Fixed-width integer columns come back asFloat32. This corrupted the pointer table of XSTAR'satdb.fits(~1M of 1.2M offsets), which astropy reads exactly.Fix
At the top of both
read(::IO, ::BinaryField, ...)methods, setscale = scale && !(field.scale == 1 && field.zero == 0)so columns with no scaling pass through unchanged. Columns with an actualTSCAL/TZERObehave as before.This changes the output type of unscaled integer columns: they are returned as stored (Int32 stays Int32) instead of Float32.
Tests
test/data/int32_large.fits(written with astropy; fixed8Jand variable-lengthPJcolumns around 2^24). It fails without the change and passes with it.atdb.fits: all 1,216,791 real-array offsets match a running sum of the record counts, as astropy reads them.The image, primary, ASCII table and random-groups readers use the same
zero .+ scale.*datapattern and are not changed here.🤖 Generated with Claude Code