File tree Expand file tree Collapse file tree 1 file changed +11
-0
lines changed
datafusion/functions-aggregate/src Expand file tree Collapse file tree 1 file changed +11
-0
lines changed Original file line number Diff line number Diff line change @@ -372,6 +372,17 @@ impl Accumulator for ArrayAggAccumulator {
372
372
+ self
373
373
. values
374
374
. iter ( )
375
+ // Each ArrayRef might be just a reference to a bigger array, and many
376
+ // ArrayRefs here might be referencing exactly the same array, so if we
377
+ // were to call `arr.get_array_memory_size()`, we would be double-counting
378
+ // the same underlying data many times.
379
+ //
380
+ // Instead, we do an approximation by estimating how much memory each
381
+ // ArrayRef would occupy if its underlying data was fully owned by this
382
+ // accumulator.
383
+ //
384
+ // Note that this is just an estimation, but the reality is that this
385
+ // accumulator might not own any data.
375
386
. map ( |arr| arr. to_data ( ) . get_slice_memory_size ( ) . unwrap_or_default ( ) )
376
387
. sum :: < usize > ( )
377
388
+ self . datatype . size ( )
You can’t perform that action at this time.
0 commit comments