This is interesting.

I’m highly sceptical of zero shot style time series models, but I do like that you can mix in effects without having to train, you just run inference or you fine tune. I’m just sceptical whether the models really work, or they just look like they work.

Even the base problem statement that Google TimesFM predicates the whole thing on observed history, so the approach is fundamentally limited. Timestral is fundamentally limited by what it can see, and all plausible quantiles are generated from that.

For a series of sensors this might be good, but what happens when one fails and spikes?

My go to approach is still building probabilistic models but using transformers and embedding outputs as variables in the models or neural net subcomponents. It gives a lot of flexibility like these foundation models do, it allows encoding high level information or sequences into the models, but it leaves you with verifiable systems and can actually be quite useful on small datasets. And if you do it right you can encode this info like “open dates” or “wind forecasts” to improve results.

The real value in traditional models is that you can’t have misspecification errors like negative sales or electricity consumption, as long as you specify the model correctly. (Note: solutions like FB prophet or auto arima have this issue)

You do have to be aware that embedding/encoded features are correlated, you probably want matryoshka embeddings, and the collinearity that all creates for your models, plus the impacts of increased dimensionality have on your model.

But, if treated like hierarchical parameters, you can use information pooling more efficiently to improve model outcomes.

There is no ceiling on the complexity you can include this way, but you have to know what you’re doing.