Operational demand and load forecasting

Choose training, model-selection and final-test periods in time order. Learn transformations such as scaling or missing-value treatment on training data only, then apply them to later periods. Do not select a model after repeatedly checking the final holdout and still call that holdout independent.

A rolling-origin exercise repeats the decision: fit using data available up to one origin, predict the intended horizon, score it, then move forward. Match the horizon to the work requirement. A five-minute-ahead result does not establish day-ahead performance; a model benefiting from updated weather cannot fairly be compared with one denied that information.

The lab supplies invented forecasts issued before the observations. You can practice scoring but cannot verify their training history or infer generalisation from six rows. Extend the method to multiple weeks, seasons and stress periods using timestamped input vintages before making an operational recommendation.

• Forecasting: Principles and Practice — time-series cross-validation

Define signed error as actual minus forecast. Positive means under-forecast demand; negative means over-forecast. MAE averages absolute errors and retains MW units. RMSE emphasises larger misses. Signed mean error measures bias, but positive and negative misses can cancel. Use the same valid intervals for every model comparison and report exclusions.

In the lab, baseline errors are 0, 20, 20, 20, −20 and 20 MW: MAE 16.67 MW and mean error 10 MW. Candidate errors are −5, 5, 10, −10, −5 and 5 MW: MAE 6.67 MW and mean error zero. Zero average bias does not mean zero errors. A lower average error also does not ensure better performance at the highest-cost interval.

Convert an error to a commercial exposure only after defining a mechanism. A 10 MW under-forecast for five minutes is 0.8333 MWh; a stated A$120/MWh incremental price difference implies A$100 of incremental cost in that simplified scenario. Multiplying by a spot price without explaining procurement, hedging and settlement does not establish the actual financial loss.

• Forecasting: Principles and Practice — forecast accuracy

Plot actual and forecast MW on the same labelled time axis. Inspect when the misses occur: peaks, ramps, missing measurements or an unusual event. Use a separate error plot or error table to identify patterns concealed by a single average. Report sample size and the operating periods represented.

Define acceptance before comparing models: baseline performance, peak-error tolerance, data freshness, runtime, fallback and monitoring owner. A candidate might lower MAE but be rejected if it fails during the critical peak or depends on unavailable inputs. Prediction intervals require their own coverage checks; a point forecast is not a confidence band.

Your memo should recommend a next test rather than declaring production readiness from the lab. Specify additional periods, input-vintage checks, retraining boundaries and a trigger for reverting to the baseline. Keep a versioned change record so someone can reconstruct which model generated a particular decision.