With small tile extents on sparse arrays, tile_array_read() aborts, throwing "Assertion Failed: (non_empty > 0)"

[Areustle]

I have a sparse dataset of float dimensions. When I populate a TileDB sparse array with small tile extents I get the following error:

Assertion failed: (non_empty > 0), function sort_fragment_cell_ranges, file /Users/areustle/asr/src/TileDB/core/src/array/array_read_state.cc, line 2158.

Small is a relative an imprecise term. From the context of my problem and a quick look at the source code it seems that my array places no data-points in one or more of the Tiles, resulting in an empty tile and a failure of the assertion. If I grow my Tile extents to encompass the entire domain I avoid this problem but of course this is not ideal for performance.

I have a branch of my own project that reproduces the error here https://github.com/Areustle/fits2tile/tree/small_tile_extent . Currently it depends on cfitsio, which is linked in the readme. If you need me to I can rework the problem to avoid the dependency.

[Areustle]

I am also getting the same error when running small subarray queries along the lines of tiledb_array_read_sparse_2.cc

I have been able to work around the issue by
a) increasing my tile capacity from 4 to 4096
b) importing more data into my array and consolidating.

[stavrospapadopoulos]

Hi Alex,

I will try to recreate the problem and get back to you very soon. I will also provide a couple of tips regarding tile extents and tile capacity for sparse arrays. Apologies for the delay.

Cheers,
Stavros

[Areustle]

Thank you Stavros, I know you're busy (aren't we all). I will try to get a simpler example of the failure that doesn't depend on the idiosyncrasies of my data if I can.

[stavrospapadopoulos]

Quick question: for the particular subarray query that triggers the assertion with the small tile capacity, were you supposed to get any results back, or the query result should be empty for that query? Thanks.

[Areustle]

Should be empty for that query.

[stavrospapadopoulos]

OK, I thought so. I think I know where the problem is, although this has not been caught by some of my tests with empty subarray results on my side. Let's please take this offline, as we may have to exchange a few more messages. Please contact me at "stavrosp@csail.mit.edu". Once the problem is fixed, I will write a summary here upon closing the issue. Thanks a lot!