Introduction to Logistic Regression

In the previous lesson, we established that a key goal of probabilistic models is to predict the target distribution $p(y|\mathbf{x})$, which allows us to quantify uncertainty. We also learned that discriminative models achieve this by directly estimating $p(y|\mathbf{x})$.

Logistic regression is the quintessential example of a discriminative model. It provides the most straightforward way to see the theory of $p(y|\mathbf{x})$ estimation put into practice, especially for binary classification.

Logistic regression

Logistic regression is a discriminative model widely used for classification tasks. The term logistic in logistic regression refers to the utilization of the logistic function. Consider a binary classification problem: the target variable $y_i$ can take on values in $\{0, 1\}$. One way to model the probability distribution of the target label $y_i$ being equal to 1 given the feature vector $\phi(\bold{x}_i)$ is by employing a logistic function defined as:

$$\hat$$ ...