File tree Expand file tree Collapse file tree 1 file changed +11
-0
lines changed
datafusion/functions-aggregate/src Expand file tree Collapse file tree 1 file changed +11
-0
lines changed Original file line number Diff line number Diff line change @@ -393,6 +393,17 @@ impl Accumulator for ArrayAggAccumulator {
393
393
+ self
394
394
. values
395
395
. iter ( )
396
+ // Each ArrayRef might be just a reference to a bigger array, and many
397
+ // ArrayRefs here might be referencing exactly the same array, so if we
398
+ // were to call `arr.get_array_memory_size()`, we would be double-counting
399
+ // the same underlying data many times.
400
+ //
401
+ // Instead, we do an approximation by estimating how much memory each
402
+ // ArrayRef would occupy if its underlying data was fully owned by this
403
+ // accumulator.
404
+ //
405
+ // Note that this is just an estimation, but the reality is that this
406
+ // accumulator might not own any data.
396
407
. map ( |arr| arr. to_data ( ) . get_slice_memory_size ( ) . unwrap_or_default ( ) )
397
408
. sum :: < usize > ( )
398
409
+ self . datatype . size ( )
You can’t perform that action at this time.
0 commit comments