Transformer Based Models

Transformer Bottleneck

Original Paper (Neurips 2019)

Flow forecast implementation

Stream Flow Forecast Example

Solar Energy Forecasting Example

Succinct summary: A variation of the transformer that replaces standard self attention with convolutions for the query and value vectors. Uses learned positional embedding. Preformed well large temporal datasets.

Model Additions

We added another final layer to enable variable length forecasts. Currently using that to see how the model performs with variable forecast_length. You can choose to exclude this layer if you desire.
We also enabled the model to work with standard loss functions (MSE, RMSE, etc).
Currently working on incorporating meta-data into the model.

Simple Transformer

Original Paper

River Flow Forecasting Example

Succinct summary: This is a vari

Transformer Decoder

Informer

Flow Forecast

Transformer Based Models

Analytics

Transformer Bottleneck

Model Additions

Simple Transformer

Transformer Decoder

Related content