Logic behind this full project:

[Devangkaruskar]

for every time running a query i required almost 6-7 min per answer. i want to know where time is consuming. as far as my understanding embedding and indexes of text are stored locally right?
can you help me to understand this @imartinez @jiangzhuo @vilaca @sorin @doskoi

[mechanicmuthu]

Time is consumed in Local LLM generating its answer from the contexts. To minimize the time, set the TARGET_SOURCE_CHUNKS to 1, Choose a smaller model (say 7B 4_0) in MODEL_PATH. Set MODEL_N_CTX to 256. Once you start getting fast answers (less than 1 min) you can start optimizing for accuracy. If you don't want complete privacy, use the fork by goskynet where openai API are part of the inference part. https://github.com/go-skynet/privateGPT.

[doskoi]

Embedding is pretty fast, and as same as search in vector db.
You can use online version of GPT to boost your development.