Python ingest.py error: TypeError: issubclass() arg 1 must be a class

[Argandov]

Hi, I'm receiving the error "TypeError: issubclass() arg 1 must be a class" after running ingest.py for the first time.

For context:

• I'm running privateGPT under WSL 2 (Windows Subsystem for Linux), and python's venv activated.
• Version of Python is 3.9.2
• The exact steps I took: git clone this repository, python3 -m venv venv and proceeded to pip install -r requirements, downloaded the .bin model as suggested, and referenced it in my .env folder.

Thanks in advance 🙏 I have no idea what this error could be about

The full traceback after running ./ingest.py or simply python3 ingest.py:

(venv) user@debian:/mnt/c/Users/USER/Dev_Environment/GPT/privategpt$ python3 ingest.py
Traceback (most recent call last):
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/ingest.py", line 9, in <module>
    from langchain.document_loaders import (
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/__init__.py", line 6, in <module>
    from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/agents/__init__.py", line 2, in <module>
    from langchain.agents.agent import (
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/agents/agent.py", line 16, in <module>
    from langchain.agents.tools import InvalidTool
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/agents/tools.py", line 8, in <module>
    from langchain.tools.base import BaseTool, Tool, tool
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/tools/__init__.py", line 42, in <module>
    from langchain.tools.vectorstore.tool import (
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/tools/vectorstore/tool.py", line 13, in <module>
    from langchain.chains import RetrievalQA, RetrievalQAWithSourcesChain
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/chains/__init__.py", line 2, in <module>
    from langchain.chains.api.base import APIChain
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/chains/api/base.py", line 13, in <module>
    from langchain.chains.api.prompt import API_RESPONSE_PROMPT, API_URL_PROMPT
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/chains/api/prompt.py", line 2, in <module>
    from langchain.prompts.prompt import PromptTemplate
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/prompts/__init__.py", line 3, in <module>
    from langchain.prompts.chat import (
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/prompts/chat.py", line 10, in <module>
    from langchain.memory.buffer import get_buffer_string
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/memory/__init__.py", line 28, in <module>
    from langchain.memory.vectorstore import VectorStoreRetrieverMemory
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/memory/vectorstore.py", line 10, in <module>
    from langchain.vectorstores.base import VectorStoreRetriever
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/vectorstores/__init__.py", line 2, in <module>
    from langchain.vectorstores.analyticdb import AnalyticDB
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/vectorstores/analyticdb.py", line 16, in <module>
    from langchain.embeddings.base import Embeddings
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/embeddings/__init__.py", line 19, in <module>
    from langchain.embeddings.openai import OpenAIEmbeddings
  File "/mnt/c/Users/USER/Dev_Environment/GPT/privategpt/venv/lib/python3.9/site-packages/langchain/embeddings/openai.py", line 67, in <module>
    class OpenAIEmbeddings(BaseModel, Embeddings):
  File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 663, in pydantic.fields.ModelField._type_analysis
  File "pydantic/fields.py", line 808, in pydantic.fields.ModelField._create_sub_type
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 668, in pydantic.fields.ModelField._type_analysis
  File "/usr/lib/python3.9/typing.py", line 835, in __subclasscheck__
    return issubclass(cls, self.__origin__)
TypeError: issubclass() arg 1 must be a class

[ad3sai]

Having Same issue.

[Argandov]

Are you using the exact same setup as me? Venv in WSL?

[dante-is-shipping]

the same issue.
os: mac
python version: 3.8.9

[johnbrisbin]

Your python versions are too old. As the Readme indicates 3.10+ is required.
"To use this software, you must have Python 3.10 or later installed. Earlier versions of Python will not compile."
It usually shows up in a different place, however where 'match' is used. Perhaps you are hitting a different incompatibility here.

[Argandov]

Thank you @johnbrisbin I just installed 3.11 and it works fine! - What are the general RAM requirements? I'm spiking on my RAM as I ingest + prompt my source docs. Is there a benchmark for running privateGPT?

@huiwan-code @ad3sai just upgrade your Python as john suggested!

[ad3sai]

@Argandov It Worked, Thank you. Set the chunk_size = 200 (or lower) in ingest.py to avoid shooting up the RAM. It worked on my machine.

[johnbrisbin]

What are the general RAM requirements? I'm spiking on my RAM as I ingest + prompt my source docs. Is there a benchmark for running privateGPT?

@Argandov, this is a complex subject with many variables.
Because the vector database that is used: Chroma runs with everything in memory, your memory requirements are very dependent on the amount of data you ingest. And the time when you will need the most memory is at the end of the ingest process when Chroma writes the data to disk (persists it) so you can use it later.

You will find that the memory used by the database will have some fixed component plus an amount that is a multiple of the amount of data you have ingested. From what I have seen the fixed base amount is something like 3 GB.
If you are ingesting with the current main branch code, you take the number returned by the python function os.cpu_count() (on an intel processor this is the number of threads = 2 x number of cores) and multiply that number by about 400MB, so 8 threads would mean about 3.2 GB.
I have 2.3GB in my source documents folder and my memory usage (with nothing to do) on ingest will peak at about 18GB. The DB folder size is just less than 8GB.
For the minimal ingest (nothing to do) the memory footprint is about 4 x the source documents size. The bad news is that the foot print very nearly doubles when Chroma writes what is in memory to disk at the end of the operation. In other words, I see over 30GB in memory usage when the data is written back to disk in ingest phase. So, how much data do you need to load into privateGPT?

The other side of the coin is the query side, in the file privateGPT.py. The good news it that it never needs to write the data to disk and it does not need the ingestion processes that took up about 3.2GB during ingest.
The bad news is that it requires in addition the database the LLM you have chosen. On my system, the memory footprint bounces up to 22GB, but then settles back down to a bit over 16GB for the query phase. Where you are waiting enter a query, when I ask a question, the usage slowly increases to just under 22GB. It stays there for subsequent queries.

This is using a Llama model I was experimenting with.

I am using 1000 byte chunks (100 overlap) because 64GB is not enough for the RAM footprint at 500 bytes (sometimes it barely works).

On the subject of chunk_size, this determines how much data is represented by each embedding. Each embedding has 1536 bytes of floating point data and the text data from your source documents tied to it. For the default 500 byte chunks, the data size is actually, on average, just less the 400 bytes (because the splitter rounds down to sentences, paragraphs and pages). Of the 400, 50 bytes is duplicated so that the split points don't breakup the similarity that makes it work.
So if you have 1GB of data, you divide 1GB/350 bytes to get the number of chunks, then multiply that by the 1536 + 400 . This comes down to about 5.5GB just for chunks.

By contrast the similar calculation for 1000 byte chunks with 100 overlap goes like this: 1GB of data divided by the average length ~810 - 100 = 710 then times 1536+810 yields 3.3 GB for chunks. As you can see, this is more efficient, at least for size.

For @ad3sai we can do the same with his recommended 200 bytes chunks with unknown overlap we have to assume he did not change it from 50. So average size might be (generously) 150 bytes less 50 or 100 bytes. So we divide 1GB / 100 (10e7) and multiply it times 1536+150 to get 16.8 GB.

So, to embed with 200 byte chunks takes 3x RAM compared to 500 and 5x RAM compared to 1000.