Skip to content

Conversation

priyankc
Copy link
Member

Description of changes

The segmentation fault is caused by a zero-capacity cache configuration in the Rust bindings when the system has very limited file handles available.

Fixes chroma-core#4460

Summarize the changes made by this PR.
This PR ensures that we initialize hnsw_cache_size to at least 1, when limited handles are available.

  • Improvements & Bug fixes
    • ...
  • New functionality
    • ...

Test plan

How are these changes tested?

  • Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs section?

@tazarov
Copy link

tazarov commented May 22, 2025

hey @priyankc, sorry for jumping into this. I think it is a great find! Can you tell me how did you come by the conclusion that exhausted FDs is the cause for this? Did you reliably reproduce the bug that this fixes?

@priyankc
Copy link
Member Author

@tazarov I was not able to reproduce the seg fault. It is one plausible hypothesis. I tried a test case by artificially setting FDs to a low number using ulimit -n, but that caused a panic exception with "Too many open files", but did not seg fault.

@Haptein
Copy link

Haptein commented Aug 14, 2025

FWIW this does not fix the issue in my case: chromadb.errors.InternalError: Error executing plan: Internal error: Error finding id.
I get this error very frequently (starts after a couple of days of running the process, then it's every time I search on some collections) using through llama-index in-memory. But on this machine the resource max file handles would be 1024, hence the cache wouldn't be zero.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Segmentation fault on first collection.add() to an empty index

3 participants