What is polynomial regression?

Polynomial regression is a regression technique that models a curved relationship between an input feature and a continuous target by adding polynomial terms such as x², x³, and higher powers.

It is useful when a straight line is too simple to represent the pattern in the data.

Basic idea

Simple linear regression uses:

ŷ = b0 + b1x

Polynomial regression can extend this to:

ŷ = b0 + b1x + b2x² + b3x³ + ...

The highest power included is called the degree of the polynomial.

Simple example

Suppose sales increase with advertising spend at first, but the gains become smaller at higher spending levels.

A straight line may not capture that curvature well.

A quadratic model such as:

Sales = b0 + b1x + b2x²

can represent a curved relationship.

Why is it still called linear regression?

Even though the relationship with x can be curved, the model is still linear in its coefficients:

b0, b1, b2, ...

The model is fitting a weighted sum of transformed features such as x, x², and x³.

How polynomial features are created

If the original feature is:

x

a degree-3 polynomial representation may create:

x
x²
x³

These transformed features are then used by a linear regression model.

Higher degree means more flexibility

A higher-degree polynomial can represent more complex curves.

But increasing the degree too much can make the model fit noise instead of the underlying pattern, leading to overfitting.

Because of this, the polynomial degree should be chosen using validation data rather than simply making it as large as possible.

Polynomial regression is not the same as every nonlinear model

Polynomial regression is one specific way to introduce nonlinearity by transforming features.

Other models can capture nonlinear relationships without explicitly creating polynomial terms.

In short, polynomial regression extends linear regression with powers of the input features so the model can represent curved relationships while remaining linear in its coefficients.