in models/mistral.py, function "mistral_completion_fn"
"
batch = {k: v.repeat(batch_size, 1) for k, v in batch.items()}
"
Why do you need to repeat the batch_size items?
And I'm a little confused by "batch" here, it seems batches are generated here, why not depends on "input_strs"?
in models/mistral.py, function "mistral_completion_fn"
"
batch = {k: v.repeat(batch_size, 1) for k, v in batch.items()}
"
Why do you need to repeat the batch_size items?
And I'm a little confused by "batch" here, it seems batches are generated here, why not depends on "input_strs"?