Machine Learning

Machine learning is a way to build software by fitting models to examples instead of writing every rule by hand. A model sees data, optimizes an objective, and generalizes the learned pattern to new inputs.

What is Mean Squared Error (MSE)?

Mean Squared Error (MSE) measures the average of the squared differences between a regression model's predictions and the actual target values. It gives larger prediction errors more influence because every error is squared before averaging. Formula where: yi is the actual value ŷi is the predicted value n is the number of observations Simple example Suppose the actual values are: and the predictions are: The errors are: After squaring: So: Why square the errors? Squaring has two important effects: positive and negative errors cannot cancel each other out large errors receive a much stronger penalty than small errors For example: So a few large mistakes can increase MSE substantially. When is MSE useful? MSE is useful when large prediction errors are especially undesirable and should influence the evaluation more strongly. It is also widely used as an optimization objective when training regression models because the squared error function is mathematically convenient to optimize. Is lower MSE better? Yes. means every prediction exactly matches the actual value. A lower MSE generally means smaller prediction errors, but comparisons are most meaningful when models are evaluated on the same target and dataset. MSE is not in the original target unit Because the errors are squared, MSE is expressed in squared units . For example, if the target is measured in rupees, MSE is measured in squared rupees. This can make MSE harder to interpret directly than MAE. Root Mean Squared Error (RMSE), covered next, takes the square root of MSE and returns the metric to the target's original unit. In short, MSE measures average squared prediction error. It strongly penalizes large mistakes, making it useful when large errors matter, but its squared units make direct interpretation less intuitive.

What is Root Mean Squared Error (RMSE)?

Root Mean Squared Error (RMSE) measures the typical size of a regression model's prediction errors by taking the square root of the Mean Squared Error. It is useful because it keeps the stronger penalty for large errors from MSE while expressing the final result in the same units as the target variable . Formula where: yi is the actual value ŷi is the predicted value n is the number of observations Simple example Suppose the actual values are: and the predictions are: The squared errors are: So: and: This means the model's prediction error is roughly 3.32 target units according to the RMSE measure. Why take the square root? MSE is measured in squared target units. For example, if the target is measured in rupees, MSE is measured in squared rupees. Taking the square root converts the metric back to the original target unit, which makes it easier to interpret. How does RMSE treat large errors? RMSE is based on squared errors, so large mistakes receive more influence than small ones. For example: This makes RMSE useful when large prediction errors are especially undesirable. RMSE vs MAE MAE treats each error in direct proportion to its size. RMSE gives larger errors more influence because the errors are squared before averaging. Therefore, RMSE is often higher than MAE when a model occasionally makes large mistakes. Is lower RMSE better? Yes. means every prediction exactly matches the actual value. A lower RMSE generally indicates better predictive accuracy, but whether the value is acceptable depends on the scale and context of the target. In short, RMSE is the square root of MSE. It measures regression error in the target's original units while still penalizing large prediction mistakes more strongly than MAE.

What is R-squared (R²)?

R squared (R²) measures how much better a regression model explains the target values compared with a simple baseline that always predicts the mean target value. It is also called the coefficient of determination . Formula where: SS res is the sum of squared residuals from the model SS tot is the total squared deviation of the actual values from their mean Intuition Suppose the baseline predicts the average target for every observation. R² compares the model's squared error with the error from that mean baseline. If the model reduces the error substantially, R² becomes larger. How to interpret R² means the model predicts the evaluated data perfectly. means the model performs no better, in squared error terms, than simply predicting the mean target value. For example: is commonly interpreted as the model accounting for about 80% of the variation in the target relative to the mean baseline. Can R² be negative? Yes. A negative R² means the model performs worse than the mean prediction baseline on the evaluated data. This can happen especially on validation or test data when a model generalizes poorly. Is a higher R² always better? A higher R² generally indicates a better fit on the same dataset, but it should not be used alone. A model can have a high R² and still: overfit the training data make errors that are too large for the business problem show systematic patterns in its residuals rely on unnecessary features R² does not show error size R² is unitless. It does not directly tell you whether predictions are off by ₹100, ₹10,000, or another practical amount. That is why it is often used together with metrics such as MAE or RMSE. Be careful when comparing models R² is most meaningful when models are evaluated on the same target and the same dataset. Adding more predictors to an ordinary regression model can also increase training R² even when the new features are not useful, which is one reason adjusted R² is sometimes used. In short, R² measures how much a regression model improves over predicting the target mean in terms of squared error. Higher values usually indicate a better fit, but R² should be interpreted together with error metrics and validation performance.

What is adjusted R-squared?

Adjusted R squared (adjusted R²) is a regression fit metric that modifies R² to account for the number of predictors in the model. Unlike ordinary R², which cannot decrease when additional predictors are added to an ordinary least squares model on the same training data, adjusted R² can decrease if a new predictor contributes too little improvement . Formula where: R² is the ordinary coefficient of determination n is the number of observations p is the number of predictors, excluding the intercept This standard formula assumes an intercept and requires n p + 1. Why is the adjustment needed? Adding more predictors may slightly reduce training error even when those predictors provide little real value. Adjusted R² imposes a penalty for model complexity, helping compare regression models with different numbers of predictors. Simple example Suppose a model has: R² = 0.80 n = 100 observations p = 5 predictors Then: The adjusted value is slightly lower because the model uses five predictors. How should it be interpreted? Adjusted R² increases: a new predictor improves fit enough to justify the extra complexity under this metric. Adjusted R² decreases: the improvement is too small relative to the complexity penalty. Adjusted R² can be negative: the fitted model performs poorly relative to the mean baseline after adjustment. Important limitation Adjusted R² is mainly useful for comparing linear regression models fitted to the same target and observations . A higher adjusted R² does not guarantee better predictions on unseen data; use validation or test set metrics such as MAE and RMSE to check generalization. In short, adjusted R² modifies R² to discourage unnecessary predictors, making it more informative for comparing regression models of different complexity.

What is Mean Absolute Percentage Error (MAPE)?

Mean Absolute Percentage Error (MAPE) measures the average absolute prediction error as a percentage of the actual value. It is commonly used to evaluate regression and forecasting models. Formula where: yi is the actual value ŷi is the predicted value n is the number of observations The formula requires nonzero actual values. Example Suppose a model predicts monthly sales: Actual: 100, Predicted: 90 → absolute percentage error = 10% Actual: 200, Predicted: 220 → absolute percentage error = 10% So the model's absolute prediction error averages 10% of the actual values . Why use MAPE? Easy to communicate: results are expressed as percentages. Independent of the target's units: unlike MAE, its value is not in rupees, kilograms, or another unit. Useful for forecasting: it can be intuitive when actual values are positive and comfortably away from zero. Important limitation MAPE is undefined when an actual value is zero and can become extremely large when actual values are close to zero. Its percentage interpretation can also be problematic for negative targets. For that reason, check the target values before choosing MAPE. The next article explores when this metric can be misleading. In short, MAPE expresses the average absolute regression error as a percentage of actual values, making results intuitive when actual values are positive and nonzero.

When can MAPE be misleading?

MAPE can be misleading when actual target values are zero or very small, or when percentage errors do not reflect the real cost of prediction mistakes. MAPE divides each absolute error by the magnitude of the actual value: That denominator creates several important limitations. 1. Actual values equal to zero When the actual value is 0, the percentage error requires division by zero and is undefined . This is common in intermittent demand forecasting, such as products with days of zero sales. 2. Actual values close to zero A small absolute mistake can become an enormous percentage error. For example: Actual 1, predicted 3 → absolute error 2, percentage error 200% . Actual 100, predicted 110 → absolute error 10, percentage error 10% . MAPE gives the first error much greater weight even though its absolute difference is smaller. 3. Large and small targets receive uneven emphasis Because each error is divided by its own actual value, observations with smaller actual values can dominate the average. A low MAPE does not necessarily mean the model avoids the most financially costly errors. 4. Negative actual values make interpretation difficult The absolute value formula can be calculated for nonzero negative actuals, but percentage errors relative to negative quantities may be hard to interpret meaningfully in the application. 5. MAPE can favor particular forecasting behavior Percentage error measures are not symmetric when actual values change. Predicting 120 for an actual value of 100 gives 20% , whereas predicting 100 for an actual value of 120 gives approximately 16.7% . This asymmetry can influence which forecasts perform best under MAPE. What should you use instead? MAE: when you want average error in the target's original units. RMSE: when larger absolute errors should receive more penalty. A domain specific metric: when errors have different business costs or the target contains many zeros. No replacement metric is universally best; choose one based on the target distribution and the cost of mistakes. In short, MAPE is easiest to interpret for positive actual values comfortably away from zero. It can be undefined, exaggerated, or misleading when the target contains zeros, near zero values, or highly uneven scales.

How do MAE and RMSE differ?

MAE (Mean Absolute Error) and RMSE (Root Mean Squared Error) both measure prediction errors in regression, but they treat large mistakes differently. How they calculate error MAE: averages the absolute differences between actual and predicted values. RMSE: squares the errors, averages them, then takes the square root. Both are expressed in the same units as the target, and lower is better . Example: one large prediction error Suppose three predictions have absolute errors of 1, 1, and 7. RMSE is higher because squaring gives the error of 7 more influence. Which metric should you use? Choose MAE when you want an easily understood average absolute error and prefer each unit of error to contribute proportionally. Choose RMSE when large prediction mistakes are particularly costly and should weigh more heavily in model evaluation. What do their values tell you? For the same set of predictions, RMSE is always at least as large as MAE . A wide gap between the two can indicate that some errors are much larger than others. Neither metric alone tells you whether the errors are acceptable in your application; compare them against a baseline and the real world cost of mistakes. In short, MAE measures average absolute error, while RMSE gives large errors more influence. Use MAE for straightforward error interpretation and RMSE when large mistakes matter more.

How do you choose the right regression metric?

Choose a regression metric based on which prediction mistakes matter most for your problem—not simply which metric gives the best looking score. Start with the cost of errors MAE: use when you want the typical absolute error in the target's units and every unit of error should count proportionally. RMSE: use when occasional large mistakes are especially costly and should receive more weight. MSE: useful as a training objective or when squared error penalties match your goal, but its squared units are harder to interpret. MAPE: useful when percentage error is meaningful and actual targets are positive and comfortably away from zero. R²: useful as a complementary measure of improvement over a mean prediction baseline; it does not show the size of errors. Consider the target distribution For example, if you forecast sales for products that frequently have zero demand , MAPE is a poor choice because division by zero makes it undefined. MAE or another domain appropriate metric is safer. Ask whether all mistakes cost the same If underpredicting demand is substantially more expensive than overpredicting it, symmetric metrics such as MAE or RMSE may not capture the business objective. Consider an asymmetric or weighted loss aligned with those costs. Evaluate on unseen data Use the same validation or test examples when comparing models, and choose a simple baseline such as predicting the training target mean or median. Report a primary metric that reflects your goal, plus a secondary metric when it reveals an important trade off. For example, MAE alongside RMSE can show whether large errors are a concern. Practical example For house price prediction: Use MAE in rupees if stakeholders need to know the average absolute pricing mistake. Track RMSE too if a few very large price errors could be especially damaging. In short, let the real world consequences of prediction errors determine your primary regression metric, then validate it on unseen data and compare against a baseline.

What does a residual plot tell you?

A residual plot shows how a regression model's prediction errors behave across its predictions. It helps reveal systematic mistakes that a single metric such as MAE or RMSE may hide. How to read the plot In a typical residual versus fitted plot: Horizontal axis: the model's predicted values (ŷ). Vertical axis: residuals (actual − predicted). Zero line: observations the model predicted exactly. A point above zero means the model underpredicted. A point below zero means it overpredicted. Example: If a model predicts 80 but the actual value is 95, the residual is +15. That point appears at predicted value 80, above the zero line. What the pattern tells you Random scatter around zero: no obvious systematic pattern; generally encouraging. Curved or U shaped pattern: the model may be missing a nonlinear relationship. Funnel shaped spread: error variability changes with predicted value, suggesting heteroscedasticity. Mostly positive or negative residuals in one region: consistent underprediction or overprediction there. Isolated extreme points: observations worth checking for data issues, unusual cases, or poor model fit. What a residual plot cannot prove A random looking plot does not guarantee the model is correct or that all regression assumptions hold. Error independence, distributional assumptions, and performance on unseen data may require separate checks. In short, a residual plot helps you see where and how a regression model makes errors. Ideally, residuals fluctuate around zero without a clear pattern or systematic change in spread.

What is heteroscedasticity?

Heteroscedasticity occurs when the variance of regression errors is not constant across different values of the input features or fitted predictions. In simple terms, a model makes errors with different levels of spread in different parts of the data. Simple example Suppose a model predicts house prices: For lower priced homes, predictions are usually within ₹2 lakh of the actual price. For luxury homes, errors may vary by ₹20 lakh or more. If the error spread systematically increases with house price, the model may exhibit heteroscedasticity. How does it appear in a residual plot? Homoscedasticity: residuals have roughly the same vertical spread across fitted values. Heteroscedasticity: residual spread widens, narrows, or otherwise changes systematically. A common sign is a funnel shaped pattern : residuals are tightly clustered at one end of the plot but spread farther apart at the other. Statistical meaning For regression errors ε, heteroscedasticity means: This concerns the variance of errors, not whether their average is zero. Why does it matter? Heteroscedasticity violates the constant error variance assumption of ordinary linear regression. Even when coefficient estimates remain unbiased under other assumptions, conventional standard errors and confidence intervals may become unreliable. In short, heteroscedasticity means regression errors have unequal variance across observations. A changing spread in the residual plot is a useful warning sign, not proof on its own.