What are polynomial features?

Polynomial features are new numerical features created by raising existing features to powers or multiplying features together.

They are often used to help a model capture nonlinear relationships.

Simple example

Suppose a dataset contains one feature:

x

A second-degree polynomial transformation can create:

x
x²

A third-degree transformation can create:

x
x²
x³

These additional terms allow a model such as linear regression to fit curved relationships instead of only straight lines.

Polynomial features can also include interactions

With two features, x1 and x2, a second-degree transformation may create terms such as:

x1
x2
x1²
x2²
x1 × x2

The multiplication term captures an interaction between the two features.

Why polynomial features are useful

They can help when the relationship between the inputs and target is not purely linear.

For example, the effect of temperature, speed, age, or distance may change gradually rather than following a straight-line pattern.

Polynomial terms give a linear model more flexibility to represent such curves.

Higher degree is not always better

As the polynomial degree increases, the number of generated features can grow very quickly.

This can:

• increase training time
• make the model harder to interpret
• introduce highly correlated features
• increase the risk of overfitting

For this reason, lower-degree polynomial features such as degree 2 or 3 are often tested before considering more complex transformations.

Scaling may become important

Polynomial terms can have very different numerical ranges.

For example, if x = 100, then:

x² = 10,000
x³ = 1,000,000

For models that are sensitive to feature scale, scaling these features can therefore be important.

Validate whether they actually help

Polynomial features should be evaluated using validation data because adding complexity does not automatically improve predictive performance.

In short, polynomial features add powers and interaction terms so models can represent curved and more complex relationships, but they should be used carefully because the feature space can grow quickly and lead to overfitting.