Question about conversion Moirai to a Classification model for multivariate time series

[alinangle]

Hello Moirai team,

First of all, thank you for your hard work on Moirai!

I am currently writing my master’s thesis on Foundation Models for custom multivariate time series data at my university. While exploring Moirai, I became interested in converting it into a classification model.

I have already started by removing the param_proj layer and adding a classification head. I also implemented a basic training function.

My current challenge is working with multivariate time series data, because Moirai expects a flattened input and the corresponding variate IDs for the Binary Attention Bias of the Any-Variate Attention.
Specifically, my data currently looks like this (after using prepare_example.ipynb):

{'target': array([[567.44879   , 567.43698   , 567.42517   , ..., 564.  , 564.        , 564.        ],
        [  4.239433  ,   4.25531363,   4.283777  , ...,   4.24945   ,  4.24945   ,   4.24945   ],
        [ 34.90082   ,  34.897     ,  34.89319   , ...,  30.84741   , 93.8436    ,  30.83978   ]], dtype=float32),
 'start': array('1950-12-30T06:00:02.720000', dtype='datetime64[us]'),
 'freq': '16L',
 'item_id': 'item_0'}

I would like to use the converted model for fine-tuning on labeled data and zero-shot classification.
My question is:

• Do you think converting Moirai for classification is feasible?
• How should I preprocess my data to feed it into the modified model?
• Are there any special considerations regarding variate IDs, flattening, or the attention mechanism?

Any guidance or suggestions would be greatly appreciated.

Thank you very much for your time and help!

Best regards,
Alina