Top 10 ML Interview Question & Answers

1. Explain the Linear Regression model and discuss its assumptions.

Answer:

Linear regression is a supervised learning model where the goal is to estimate a function f(x) such that each feature has a linear relationship to the target variable y, expressed as y = X\beta. Here, X is the matrix of predictor variables and $\beta$ is the vector of parameters determining the weight of each variable.

Since it is easily misapplied, you must validate four main assumptions:

Linearity: The relationship between the feature set and the target variable is linear.
Homoscedasticity: The variance of the residuals is constant.
Independence: All observations are independent of one another.
Normality: The distribution of Y is assumed to be normal.

2. Describe the motivation behind Random Forests and mention two reasons why they are better than individual Decision Trees.

Answer:

The motivation behind Random Forests is the "Wisdom of the Crowd." By aggregating the results of many weak learners (decision trees), we reduce dependency on any single set of features. For regression, we take the mean; for classification, the majority vote.

Reasons Random Forests often outperform individual Decision Trees:

Generalization: Decision trees are prone to overfitting. Random Forests generalize better on unseen data because they use randomness in feature selection and data sampling, resulting in lower variance.
Ensemble Strength: As an aggregation of various models, they naturally perform better by smoothing out individual errors.

3. What are the differences and similarities between Gradient Boosting and Random Forest?

Answer:

Similarities:

Both are decision-tree-based algorithms.
Both are ensemble methods that are flexible and require minimal data preprocessing.

Differences:

Bagging vs. Boosting: Random Forest uses Bagging (trees are built in parallel/independently). Gradient Boosting uses Boosting (trees are built sequentially, where each tree minimizes the error of the previous one).
Dependency: In Random Forests, trees are independent. In Gradient Boosting, every tree depends on the previous one.

Trade-offs:

Gradient Boosting Advantages: Can be more accurate (captures complex patterns) and handles unbalanced data better.
Random Forest Advantages: Less prone to overfitting, faster training (parallelization), and easier hyperparameter tuning.

4. Briefly explain K-Means clustering and how to find the best value of K.

Answer:

K-Means is a clustering algorithm that partitions data into K distinct clusters. It iteratively selects centroids, assigns points to the closest cluster, and updates centroids to minimize the total inter-cluster variation (Euclidean distance).

Finding the best K:

The Elbow Method: Plot the explained variation (within-cluster sum of squared errors) against the number of clusters K. Look for a sharp "elbow" or drop-off point where the information gain from adding another cluster diminishes.
The Silhouette Method: Measures how similar a point is to its own cluster compared to others. The coefficient ranges from -1 to 1. A value of 1 implies the point is in the right cluster. You plot the coefficient against different K values to find the maximum.

5. What is dimensionality reduction? Can you discuss one method of it?

Answer:

Dimensionality reduction reduces the complexity of data by decomposing it into a smaller set of variables while retaining important information. It is crucial for summarizing and visualizing large datasets (e.g., projecting to 2D or 3D).

Method: Principal Component Analysis (PCA)

PCA combines highly correlated variables into a smaller set of constructs called principal components. These components are independent linear combinations found sequentially:

The first component captures the maximal variance.
The second component captures the second-highest variance (uncorrelated to the first), and so on.

6. What are L1 and L2 regularizations? What are the differences between the two?

Answer:

Regularization prevents overfitting by penalizing complex models (large weights) in the loss function.

L2 Regularization (Ridge): Adds the sum of the squares of weights to the loss function.
$$Loss = \text{Cost Function} + \lambda \times \text{weights}^2$$
- Effect: Penalizes huge parameters, keeping weights small but rarely exactly zero. Prevents overfitting any single feature.
L1 Regularization (Lasso): Adds the sum of the absolute values of weights to the loss function.
$$Loss = \text{Cost Function} + \lambda \times |\text{weights}|$$
- Effect: can shrink coefficients to exactly zero, effectively performing feature selection by removing insignificant variables.

7. What is the difference between overfitting and underfitting, and how can you avoid them?

Answer:

Image of overfitting vs underfitting in machine learning

Shutterstock

Explore

Overfitting: The model fits the training data too well (capturing noise), leading to low training error but high test error.
- Solutions: Simplify the model, use regularization, collect more data, clean noise, or use early stopping with validation sets.
Underfitting: The model is too simple to capture the underlying pattern, leading to high error in both training and testing.
- Solutions: Use a more complex model, reduce regularization, or perform feature engineering to create better inputs.

8. What are bias and variance in a machine learning model? Explain the bias-variance trade-off.

Answer:

Bias: Assumptions made by the model to simplify the function. High Bias causes underfitting (missing the pattern).
Variance: How much the model's estimate changes with different training data. High Variance causes overfitting (modeling the noise).
Irreducible Error: Error caused by the framing of the problem or unknown variables; cannot be reduced.

The Trade-off:

The goal is low bias and low variance.

Simple models (e.g., Linear Regression) tend to have High Bias / Low Variance.
Complex models (e.g., Neural Networks) tend to have Low Bias / High Variance.
You must tune hyperparameters to find the "sweet spot" balance between the two.

9. Define Precision, Recall, and F1, and discuss the trade-off between them.

Answer:

Precision: The ratio of correct positive predictions to total positive predictions. (How confident are we when we predict "True"?).
Recall: The percentage of actual positives correctly identified. (Did we catch all the "True" cases?).

The Trade-off:

Optimizing for one often hurts the other.

Example: In cancer diagnosis, High Recall is crucial (don't miss a sick patient), even if it lowers precision (some healthy people are misdiagnosed).
Example: In spam filters, High Precision is crucial (don't send a real email to junk), even if recall drops (some spam gets through).

F1 Score:

The harmonic mean of precision and recall, used when both are equally important:

$$F1 = 2 \times \frac{\text{precision} \times \text{recall}}{\text{precision} + \text{recall}}$$

10. Mention three ways to handle missing or corrupted data in a dataset.

Answer:

Handling missing data is critical as many algorithms (like standard scikit-learn implementations) cannot process null values.

Deletion: Deleting rows/columns with null values.
- Pros: Fast and robust.
- Cons: Loss of valuable information; only viable if missing data is minimal.
Algorithms that support missing values: Using models that naturally handle nulls, such as K-NN (can ignore columns in distance measures), Naive Bayes, or certain Random Forest implementations (though scikit-learn's RF does not support this natively).
Imputation: Replacing missing values with estimated ones.
- Simple: Fill with Mean, Median, or Mode. (Fast, but ignores feature covariance).
- Advanced: Use an ML model to predict missing values based on other features. (More accurate, preserves covariance, but computationally expensive).