Examples: (Ollama and) Vector search without an account or an API key - #134
Examples: (Ollama and) Vector search without an account or an API key#134florinutz wants to merge 2 commits into
Conversation
Ollama runs the embedding model in its own process and speaks HTTP, so `vector_ollama.py` demonstrates the vector store with nothing to sign up for and no machine learning stack in the dependency tree. `nomic-embed-text` produces 768 dimensions, well inside what a FLOAT_VECTOR column accepts. `vector_search.py` becomes `vector_openai.py`, now that the backend is the thing that distinguishes the two programs. The suite executes every example, and CI serves no model, so the runner skips the case when it cannot reach Ollama. That mirrors how it already treats a missing OpenAI key, and keeps the reason in the report rather than in a list of files to ignore.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
florinutz
changed the title
The suite executes every example, so an example's coverage records whether the machine could reach the service it talks to. CI serves no model, so this one is measured at the point it skips, and the number says nothing about the code. Its behaviour is still checked: the test runs the file wherever a server is reachable, and a failure there is a failure.
| ollama serve | ||
| ollama pull nomic-embed-text |
There was a problem hiding this comment.
ollama serve waits forever so pull does not work.
| ollama serve | |
| ollama pull nomic-embed-text | |
| ollama pull nomic-embed-text | |
| ollama serve |
| # Optionally set environment variables to configure the Ollama and CrateDB | ||
| # endpoints. | ||
| export OLLAMA_BASE_URL="http://localhost:11434" | ||
| export CRATEDB_SQLALCHEMY_URL="crate://crate@localhost/?schema=doc" |
There was a problem hiding this comment.
Keep the schema separate from the one vector_openai.py uses: both programs share the langchain_embedding table, and its vector width is fixed at 768 or 1536 by whichever of the two creates it first, so other gets vector mismatch error.
| export CRATEDB_SQLALCHEMY_URL="crate://crate@localhost/?schema=doc" | |
| export CRATEDB_SQLALCHEMY_URL="crate://crate@localhost/?schema=doc_ollama" |
| from langchain_cratedb import CrateDBVectorStore | ||
|
|
||
| CRATEDB_SQLALCHEMY_URL = os.environ.get( | ||
| "CRATEDB_SQLALCHEMY_URL", "crate://crate@localhost/?schema=testdrive" |
There was a problem hiding this comment.
langchain_embedding table is shared between programs and its vector width is fixed when the table is first created, vector_openai.py would leave 1536 behind where this program needs 768.
| "CRATEDB_SQLALCHEMY_URL", "crate://crate@localhost/?schema=testdrive" | |
| "CRATEDB_SQLALCHEMY_URL", "crate://crate@localhost/?schema=testdrive_ollama" |
| raise pytest.skip( | ||
| "Skipping test because `OPENAI_API_KEY` is not defined" | ||
| ) from ex | ||
| except ConnectionError as ex: |
There was a problem hiding this comment.
If model didn't pulled with ollama pull it landed here and tests are skipped, I think we should catch model "nomic-embed-text" not found, try pulling it first error and raise it other than skipping so user can understand and pull the model.
2 participants
Closes #131. Supersedes the embeddings half of #17.
What it does
vector_ollama.pyisvector_openai.pywith a different embedding backend and nothing to sign up for. Ollama runs the model in its own process and speaks HTTP, so the dependency tree gains one package rather than torch and seventeen CUDA wheels, which is where #17's sentence-transformers approach ended up.nomic-embed-textproduces 768 dimensions, well inside what aFLOAT_VECTORcolumn accepts.vector_search.pybecomesvector_openai.py, since the backend is now the thing that tells the two programs apart.Skipping, rather than a list of files to ignore
The suite executes every file under
examples/, and CI serves no model. The runner already handles this shape for OpenAI: it catches the missing-key error and skips with a reason. This adds the same branch for Ollama's connection error, keyed on the message so an unrelated network failure still fails the test.That leaves
SKIP_FILESempty and puts the reason in the pytest report.langchain-ollamastill joins the test group, because the skip only works if the import succeeds first.Verification
Ran for real against a
crate/crate:nightlycontainer with a local Ollama servingnomic-embed-text: indexes the document and returns the correct passage for the sample query, in about four seconds, with no key set anywhere.Both branches are exercised. With Ollama reachable the case passes; pointed at a dead port, it reports
SKIPPED ... Ollama is not reachable, which is what CI will show. Full suite is 132 passed with the three pre-existing OpenAI-key failures unchanged, andruff check,ruff format --diffandmypy .are clean.Left alone deliberately
Both programs still read
state_of_the_union.txtfrom a 2023 tag oflangchain-ai/langchain. Moving example data into this repository is a real want —document_loader.pycarries aTODO: Use new URL @ langchain-cratedbfor the same reason, and #17 carried a third — but any URL pointing at this repository 404s until the file is on main, so it wants to be its own change rather than a rider on this one.