Skip to content

Commit 190e5f9

Browse files
committed
fix assess in DynamicTimeSeriesBucket when max_total_size is used
1 parent 545ae34 commit 190e5f9

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

lazy_dataset/core.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3351,7 +3351,13 @@ def is_completed(self):
33513351

33523352
def assess(self, example):
33533353
seq_len = self.len_key(example)
3354-
return self.lower_bound <= seq_len <= self.upper_bound
3354+
return (
3355+
(self.lower_bound <= seq_len <= self.upper_bound)
3356+
and (
3357+
(self.max_total_size is None)
3358+
or ((len(self.data) + 1) * max(self.max_len, seq_len) <= self.max_total_size)
3359+
)
3360+
)
33553361

33563362
def _append(self, example):
33573363
super()._append(example)

0 commit comments

Comments
 (0)