[Bug]: Segmentation fault on first collection.add() to an empty index #4460
Description
What happened?
When chromadb 1.0.x (1.0.4, 1.0.8) runs in‑process on Linux, the very first call to collection.add()—after get_or_create_collection() has returned an empty HNSW index—kills the interpreter with SIGSEGV / exit code 139 or hangs long enough for Gunicorn to SIGKILL the worker. Subsequent add() calls succeed, so the crash only manifests on brand‑new clusters. I'm using Chroma as part of a Flask app and I am able to add to existing clusters, but I cannot insert new data. I encountered this error after restarting the service. Subsequent restarts to the service work ~1 out of every 20 times. I am using Persistent Client.
From Gunicorn:
[CRITICAL] WORKER TIMEOUT (pid:1786) ← hangs inside .query()
[ERROR] Worker (pid:1785) was sent code 139! ← seg‑fault in .add()
Versions
chromadb - 1.0.4 (also reproduced on 1.0.8)
Python 3.12
Relevant log output
May 07 01:15:25 cluster gunicorn[20631]: INFO | app:insert:47 - insert: payload parsed, 1 articles
May 07 01:15:25 cluster gunicorn[20631]: INFO | app:insert:59 - insert: chroma collection ready
May 07 01:15:25 cluster gunicorn[20631]: INFO | app:insert:76 - insert: embedding generated (len=1536)
May 07 01:15:26 cluster gunicorn[20631]: ERROR | Worker (pid:20631) was sent code 139!
May 07 01:20:15 cluster gunicorn[20648]: INFO | app:insert:35 - insert: endpoint hit
May 07 01:20:15 cluster gunicorn[20648]: INFO | app:insert:47 - insert: payload parsed, 1 articles
May 07 01:20:15 cluster gunicorn[20648]: INFO | app:insert:59 - insert: chroma collection ready
May 07 01:20:16 cluster gunicorn[20648]: INFO | app:insert:76 - insert: embedding generated (len=1536)
May 07 01:20:16 cluster gunicorn[20648]: INFO | app:insert:86 - insert: query returned 5 distances
May 07 01:20:16 cluster gunicorn[20648]: INFO | app:insert:96 - insert: filtered_results count=1
May 07 01:20:16 cluster gunicorn[20648]: INFO | app:insert:104 - insert: matching cluster found, using cluster_id=6eea6234a823b2a03fff0d0a5044bed3
May 07 01:20:18 cluster gunicorn[20648]: INFO | app:insert:179 - insert: notification sent
May 07 01:20:18 cluster gunicorn[20648]: SUCCESS | app:insert:221 - insert: successfully added cluster of length 1
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:35 - insert: endpoint hit
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:47 - insert: payload parsed, 1 articles
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:59 - insert: chroma collection ready
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:76 - insert: embedding generated (len=1536)
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:96 - insert: filtered_results count=0
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:100 - insert: no existing cluster match, creating new cluster_id=8fcb7470f0872bb8bc1637df96ffb3db
May 07 01:20:28 cluster gunicorn[20648]: INFO | app:insert:194 - insert: new cluster document created
May 07 01:20:28 cluster gunicorn[20648]: SUCCESS | app:insert:221 - insert: successfully added cluster of length 1
Here is an additional log:
May 08 03:43:13 cluster gunicorn[8375]: 2025-05-08 03:43:13.578 | INFO | app:insert:79 - insert: embedding generated (len=1536)
May 08 03:43:13 cluster gunicorn[8375]: 2025-05-08 03:43:13.578 | INFO | app:insert:84 - insert: querying chroma for nearest embeddings
May 08 03:43:13 cluster gunicorn[8375]: 2025-05-08 03:43:13.578 | INFO | app:insert:85 - insert: about to run collection.query() for a potential new cluster
May 08 03:43:13 cluster gunicorn[8375]: Fatal Python error: Segmentation fault
May 08 03:43:13 cluster gunicorn[8375]: Thread 0x000078cc1d1f96c0 (most recent call first):
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/threading.py", line 359 in wait
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/queue.py", line 180 in get
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/logtail/flusher.py", line 50 in step
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/logtail/flusher.py", line 27 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/threading.py", line 1032 in _bootstrap
May 08 03:43:13 cluster gunicorn[8375]: Thread 0x000078cc1d9fa6c0 (most recent call first):
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/threading.py", line 359 in wait
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/queue.py", line 180 in get
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/posthog/consumer.py", line 107 in next
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/posthog/consumer.py", line 76 in upload
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/posthog/consumer.py", line 65 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/threading.py", line 1075 in _bootstrap_inner
May 08 03:43:13 cluster gunicorn[8375]: File "/usr/lib/python3.12/threading.py", line 1032 in _bootstrap
May 08 03:43:13 cluster gunicorn[8375]: Thread 0x000078cc2ebdb080 (most recent call first):
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/chromadb/api/rust.py", line 512 in _query
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/chromadb/api/models/Collection.py", line 221 in query
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/app.py", line 86 in insert
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/flask/app.py", line 902 in dispatch_request
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/flask/app.py", line 917 in full_dispatch_request
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/flask/app.py", line 1511 in wsgi_app
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/flask/app.py", line 1536 in __call__
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 177 in handle_request
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 134 in handle
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 31 in accept
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 69 in run_for_one
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/workers/sync.py", line 125 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/workers/base.py", line 143 in init_process
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 608 in spawn_worker
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 641 in spawn_workers
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 570 in manage_workers
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/arbiter.py", line 201 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/app/base.py", line 71 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/app/base.py", line 235 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/lib/python3.12/site-packages/gunicorn/app/wsgiapp.py", line 66 in run
May 08 03:43:13 cluster gunicorn[8375]: File "/var/www/cluster/.venv/bin/gunicorn", line 10 in <module>
May 08 03:43:13 cluster gunicorn[8375]: Extension modules: markupsafe._speedups, numpy._core._multiarray_umath, numpy.linalg._umath_linalg, yaml._yaml, grpc._cython.cygrpc, google._upb._message, msgpack._cmsgpack, charset_normalizer.md, requests.packages.charset_normalizer.md, requests.packages.chardet.md, _cffi_backend, _time_machine (total: 12)
May 08 03:43:14 cluster gunicorn[8373]: [2025-05-08 03:43:14 +0000] [8373] [ERROR] Worker (pid:8375) was sent code 139!