What is logistic regression?

Logistic regression is a supervised machine learning algorithm commonly used for binary classification, such as predicting whether a customer will churn or a transaction is fraudulent.

Instead of predicting an unrestricted numerical value, it estimates the probability that an observation belongs to a chosen class.

Simple example

Imagine a bank wants to predict whether a transaction is fraudulent.

• Inputs: transaction amount, location, transaction history, and time.
• Possible classes: Fraud and Not Fraud.
• Example output: probability of fraud = 0.82.

If the bank uses a decision threshold of 0.50, it would classify this transaction as Fraud. The threshold can be adjusted according to the cost of false alarms and missed fraud.

Core idea

Logistic regression combines the input features into a linear score and passes that score through a logistic (sigmoid) function to produce a value between 0 and 1.

p = 1 / (1 + e^(-z))
z = w1*x1 + w2*x2 + ... + b

Here, p is the predicted probability of the positive class, and z is the weighted combination of inputs. The formula and training process are explained in the next concept.

Why is it useful?

• Simple and fast: often a strong baseline for classification.
• Interpretable: coefficients describe how features affect the model's log-odds, subject to modeling assumptions.
• Probability output: supports threshold-based decisions.
• Works well with suitable features: especially when the log-odds are approximately linear in the inputs.

Important limitation

Basic logistic regression has a linear decision boundary in its feature space. It may miss complex nonlinear patterns unless features are transformed or interactions are added.

In short, logistic regression is a classification method that converts a linear score into an estimated class probability, making it a practical starting point for binary classification problems.