Training Forecasting Models

Getting Flowing

Our repository makes it incredibly easy to take existing CSV file(s) with temporal data and train a model to forecast on them. We have some basic tutorials below with more extensive documentation underneath.

Basic Introduction Video

Tutorial Notebook

Solar Energy Forecasting on Kaggle

Solar Forecasting Video Tutorial

COVID-19 Forecasting Notebook

Data Format

In general your data should be in a simple CSV file with a datetime. For instance data might look like this:

Date	MeanMilkProduction (kg/lactation)	Temperature

Date	MeanMilkProduction (kg/lactation)	Temperature
12/20/20	95	50
12/21/20	190	60
12/22/20	165	55
12/23/20		45
12/24/20	53	55
12/25/20	54	54
12/26/20	92	45
12/27/20	115	44

Setting up experiments

We take inspiration from AllenNLP and use JSON configuration files to effectively track experiments. To get started, define a configuration file in the following format roughly: So here for relevant_columns you

config = {
    "model_name": string,
    "model_type": string, # "PyTorch" or "da_rnn"
    "model_params": {
        # All params from the model's __init__ method.
    },
    "dataset_params": {
        "type": string,
        "relevant_cols": list 
        "training_path": list,
        "validation_path": list,
        "test_path": list,
        "forecast_history":integer,
        "forecast_length":integer 
    },
    "training_params": {
        "criterion":string,
        "batch_size": int
        "optimizer": string,
        "optim_params": {
            "lr": number,
            "momentum": number 
            },
        "epochs": integer
    },
    "GCS": {
        "run_save_path": string,
        "project_id": string,
        "credential_path":string
    },
    "wandb": {
        "project": string,
        "name": string, # or None (optional)
        "tags": List[string] # or None (optional)
    }
}

You can find a list of all available model names and their associated classes in the pytorch_model_dict variable in model_dict_function.py.

Model parameters should include all necessary parameters to initialize your model. For instance, if your model initialization function is def __init__(self, hidden_layer_dim, encoder_layers) then your model_params would be

"model_params": {
    "hidden_layer_dim": integer,
    "encoder_layers": integer
    }

You can find what model parameters are required by looking at the ReadTheDocs page of the model you want to use. You may want to cross-reference with model_dict_function.py to find which directory each model is stored in.

Dataset parameters include things like batch size, file paths, and anything the data loader will need. Data params also allow you to define a train_end and a valid_start parameters if all your data is in the same CSV file. We also support automated methods to interpolate data.

IMPORTANT: your target columns must be the first listed in relevant columns. For instance, if you are forecasting COVID-19 and your targets are new_cases, number_of_deaths would be first in relevant_columns. Also, please remember that at the moment multi-task forecasting is still experimental! There may be errors. See below for more examples

class required: This is the name of the class you plan on using. This will most often be simply default or TemporalLoader (if you are using the Informer or other models that separate the datetime params).

training_path required (str.) The path to the training file (can be a GCS path as well).

validation_path required (str). The path to the validation file (can be same as training path but must use train_end and valid_start in this case).

test_path required (str). The path to test file (can be the same as training path but must use train_end and valid_start in this case).

target_col required (list). The target column or columns for your model to forecast.

num_workers optional. The number of workers to use in the data-loader (from PyTorch)

pin_memory optional. Whether to pin your memory usage to the GPU. See the official data-loader documentation for more information on what this

scaler optional. The scaler you want to use. See the scaler documentation for which scalers are supported. Also if scaler requires additional parameters you will need a scaler_params bloc.

scaler_params optional: These are parameters that match up with the sklearn scaler initialization parameters. If not supplied default parameters are used.

scaler_cols optional. The columns you want to scale. If left blank and scaler is present this will default to all columns

feature_param : optional. An optional block used for creating additional features.

datetime_params use this if you want to automatically date_time features to your model base on the timestamp. IMPORTANT REMEMBER to update n_time_series in model_params (+1 for each additional numerical feauture).

hour : value can be numerical or cyclical

day : value can be numerical or cyclical

weekday : value can be numerical or cyclical

month : value can be numerical or cyclical

year : value can be numerical

"feature_param":
{
   "datetime_params":
      {
      "hour": "cyclical"
      "day": "cyclical"
      
      }
}

interpolate required. Either set to False or takes a parameter block. In the case of a parameter block you will need the following parameters:

train_end optional (int): The row that marks the end of the training data.

valid_end optional (int): The row that marks the end of the validation data.

Additional optional parameters you can include here are scaler, interpolate, and sort_column. WARNING IF YOU ARE CREATING DATETIME FEATURES YOU MUST HAVE A sort_column.

Inference Parameters

Another set of parameters that are required are inference parameters.

datetime_start required. The date you want to start inference evaluation on.
hours_to_forecast required. The number of time steps you want to forecast
num_prediction_samples optional. The total number of prediction samples for your code to make when computing a confidence interval. See the page on confidence intervals.
decoder_params optional (depending on model): These may be required depending on the model that you choose to use. See model-specific documentation for more information.
- unsqueeze_dim : The dimension to add to the tensor to match the size (usually 0 or 1).
test_csv_path required. The path to the file for the model to test on.

These parameters also generally include a copy of datetime_params in the older version of the code. However, in the newer version, you do not need to copy the datetime parameters.

Optional parameters

early_stopping

sweep (either true or false). This must be set to true if you plan to use a Wandb sweep.
wandb (either true or false). Set to false if you are planning on using a sweep. Only set to true if you are using Wandb without a sweep.
sweep_id:
use_decoder: Whether to use a decoder when performing long range forecasting (n>1 timesteps)
n_targets optional: The number of targets should be equal to len(targ_col). Required if doing multi-task forecasting.

Loading Pre-Trained Weights and Removing Layers

Occasionally, you may wish to continue training a model or load a pre-trained model with slightly different layers. In order to do this, you would specify the relevant parameters in your configuration file

weight_path str: This is the path to where the model weights are stored. This could potentially be either a local path or a GCS path (e.g. gs://bucket_name/model.pth)

weight_path_add A block of additional parameters that relate to either deleting layers or freezing layers.

excluded_layers Layers to delete from the loaded model weights.

frozen_layers (Feature to be added) List of layers to be frozen during training.

Meta-Data and Auto-Encoder Models

Defining Weights and Biases Sweep Configuration Sweeps

For end-to-end hyper-parameter sweeps we have to define the following:

Base configuration file (similar to the one above)
Weights and Biases configuration file

Flow Forecast

Training Forecasting Models

Getting Flowing

Data Format

Setting up experiments

Defining Weights and Biases Sweep Configuration Sweeps

Other info