File tree Expand file tree Collapse file tree 3 files changed +29
-18
lines changed Expand file tree Collapse file tree 3 files changed +29
-18
lines changed Original file line number Diff line number Diff line change 1- # llama-cpp-rs-2
1+ # llama-cpp-rs
22
3- A wrapper around the [ llama-cpp ] ( https://github.com/ggerganov/llama.cpp/ ) library for rust.
3+ This contains two packages used internally at [ Utility AI ] ( https://utilityai.ca/ ) :
44
5- # Goals
5+ - llama-cpp-2
6+ - llama-cpp-sys-2
67
7- - Safe
8- - Up to date (llama-cpp-rs is out of date)
9- - Abort free (llama.cpp will abort if you violate its invariants. This library will attempt to prevent that by ether
10- ensuring the invariants are upheld statically or by checking them ourselves and returning an error)
11- - Performant (no meaningful overhead over using llama-cpp-sys-2)
12- - Well documented
13-
14- # Non-goals
15-
16- - Idiomatic rust (I will prioritize a more direct translation of the C++ API over a more idiomatic rust API due to
17- maintenance burden)
18-
19- # Contributing
20-
21- Contributions are welcome. Please open an issue before starting work on a PR.
8+ See their respective READMEs for more information.
Original file line number Diff line number Diff line change 1+ # llama-cpp-rs-2
2+
3+ A wrapper around the [ llama-cpp] ( https://github.com/ggerganov/llama.cpp/ ) library for rust.
4+
5+ # Goals
6+
7+ - Safe
8+ - Up to date (llama-cpp-rs is out of date)
9+ - Abort free (llama.cpp will abort if you violate its invariants. This library will attempt to prevent that by ether
10+ ensuring the invariants are upheld statically or by checking them ourselves and returning an error)
11+ - Performant (no meaningful overhead over using llama-cpp-sys-2)
12+ - Well documented
13+
14+ # Non-goals
15+
16+ - Idiomatic rust (I will prioritize a more direct translation of the C++ API over a more idiomatic rust API due to
17+ maintenance burden)
18+
19+ # Contributing
20+
21+ Contributions are welcome. Please open an issue before starting work on a PR.
Original file line number Diff line number Diff line change 1+ # llama-cpp-sys
2+
3+ Raw bindings to llama.cpp with cublas support.
You can’t perform that action at this time.
0 commit comments