What is a baseline regression model?

A baseline regression model is a simple prediction method used as a reference for evaluating a more sophisticated regression model.

It answers an important question: Does the trained model predict better than a basic approach?

Common regression baselines

• Mean baseline: predict the average target value from the training set for every new observation. This is a natural reference for squared-error metrics such as MSE or RMSE.
• Median baseline: predict the training-set median for every observation. This is a natural reference for MAE.
• Simple rule-based baseline: use an obvious domain rule, such as the most recent observed value when forecasting a time series.

Simple example

Suppose the training-set average monthly sales is 150 units.

A mean-baseline model predicts:

Predicted sales = 150 units

for every new case, regardless of its input features.

If a trained regression model consistently achieves lower errors than this baseline on the same unseen data, it is providing useful predictive value beyond the simple reference.

How to compare fairly

• Calculate the baseline statistic (such as the mean) using training data only.
• Generate predictions from the baseline and the trained model on the same validation or test observations.
• Compare them using the same evaluation metric.
• Check whether the improvement is meaningful for the actual business problem.

Why is a baseline important?

A complex model can look impressive in isolation while adding little value. If it cannot outperform a basic baseline, investigate the data, features, training setup, or whether the problem is predictable.

A baseline is a minimum reference, not necessarily the performance level required for deployment.

In short, a baseline regression model gives you a simple benchmark—often the training-target mean or median—so you can tell whether a more complex model genuinely improves predictions.