The answers are unfortunately relatively short, how can you increase the max_new_tokens value?

[Crimsonfart]

The answers are unfortunately relatively short, how can you increase the max_new_tokens value?

Bonus question: I am using a Llama.cpp model (Vicuna13b) is it correct that I set the .env value to MODEL_N_CTX=2048 or should I use 1000 because of the other model (ingest model)?

[johnbrisbin]

Ingest does not appear to use MODEL_N_CTX or the main model, so it should not cause a conflict.
Usually, the context length (such as MODEL_N_CTX) is used to estimate how much you dare pack in the prompt to leave enough for the output. You might get more nearest neighbors in the prompt end, if nothing else.
I have not however looked much at all at privateGPT.py, the query module.