Feat/decoder algorithm - ZipAr #444
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| The index of the token in the last column of the last completed row. | ||
| """ | ||
| # We get its index from the number of completed rows, i.e. the index of the first active row | ||
| return self.active_rows[0] * self.num_cols - 1 |
There was a problem hiding this comment.
Bug: Incorrect proxy token selection for new rows
The get_proxy_token_idx_for_new_row method returns the last token of the last completed row, but when starting a new row N, the proxy token should come from the previous row N-1 (which is still being generated), not from the last completed row. The formula active_rows[0] * num_cols - 1 gives the wrong row when multiple rows are active. For example, when starting row 2 with active_rows = [1, 2], it returns the last token of row 0 instead of row 1, violating the spatial adjacency principle described in the ZipAR paper.
| attentions=decoder_attentions, # type: ignore | ||
| hidden_states=decoder_hidden_states, # type: ignore | ||
| past_key_values=outputs.past_key_values, | ||
| ) |
There was a problem hiding this comment.
Bug: Wrong scores returned in generation output
The GenerateDecoderOnlyOutput returns scores=scores and logits=raw_logits, but the code builds raw_scores at line 651 which should be used instead of scores. When output_scores is enabled, raw_scores is constructed as a tuple containing the final scores, but the return statement uses the unwrapped scores variable instead. This inconsistency means the output format doesn't match the expected tuple format for scores when return_dict_in_generate and output_scores are both enabled.
|
This PR has been inactive for 10 days and is now marked as stale. |
Description
ZipAr algorithm added
Type of Change
Checklist
Additional Notes
There aren't tests implemented for this algorithm yet.