Commit f5896f8

and

authored

Use a small tokenizer.json for unit test (#92) (#94)

* Use a small tokenizer.json for unit test (#92) [ghstack-poisoned] * Update test/test_hf_tokenizer.cpp Co-authored-by: Jack <[email protected]> --------- Co-authored-by: Jack <[email protected]>

1 parent e894ef4 commit f5896f8Copy full SHA for f5896f8

2 files changed

+80

-2379562

lines changed

test
- resources
  - test_hf_tokenizer.json
- test_hf_tokenizer.cpp

2 files changed

+80

-2379562

lines changed

Comments

(0)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Commit f5896f8

2 files changed

2 files changed

File tree

2 files changed

2 files changed

0 commit comments