Llama-7b performance

[Davido111200]

Hi,

Thanks for sharing your code!

I'm trying to reproduce the results using llama-7b. The results are not good. Attached is the prediction for AirPassengers. Is this expected?

[yyexela]

I am having the same issues.

[mfinzi]

Definitely not how the predictions should be looking. Can you provide more details?

[yyexela]

@mfinzi thanks for reaching out! I am running demo.ipynb for LLAMA2 using the following hyperparameter values from run_monash.py:

llama_hypers = dict(
    temp=1.0,
    alpha=0.99,
    beta=0.3,
    basic=False,
    settings=SerializerSettings(base=10, prec=3, time_sep=',', bit_sep='', plus_sign='', minus_sign='-', signed=True), 
)

model_hypers = {
    'llama-7b': {'model': 'llama-7b', **llama_hypers},
}

model_predict_fns = {
    'llama-7b': get_llmtime_predictions_data,
}

I get the following figure:

[yyexela]

And here's the jupyter notebook of exactly what I'm running. (note: just rename it from "demo.txt" to "demo.ipynb")

demo.txt

EDIT:
I think it might be an environment issue, I'm using newer versions of numpy and pandas, if you get it to run with better results could you send exact package versions?

Thank you!

EDIT 2:
pip freeze attached, see pandas and numpy version

pandas==2.3.1
numpy==2.0.2

pip_freeze.txt

I know the install.sh has pandas<2.0.0 but I wasn't able to get that environment to work with the rest of the packages, it was throwing errors in the darts library, so I implemented the suggestion in this issue.

EDIT 3:
Also, while I have your attention... I'm working on using your model as a comparison against other SciML models in a NeurIPS benchmark paper and I'm noticing that if I want to forecast 1000 time-steps in a 1024 dimension model, that it takes forever, since I need to forecast each dimension separately in an auto-regressive manner.

Any suggestions/ ideas that could be helpful here?

Thanks again:)