Maximum likelihood

Link: Probability distribution

The goal of maximum likelihood §

In Probability vs likelihood, the likelihood examine the probablity when we shift and move the distribution. The point of maximum likelihood is to find out the best distribution that can maximze the likelihood for the data, aka, find out the best line that fit the data y.

It’s a general approach can be applied to both linear and non-linear.

Why do we need maximum likelihood in logistic regression? §

Because the least square method in linear regression does not work, as the transformed log involves +infinity and -infinity

The intuition of maximum likelihood §

The idea is to find the constants so that the $p(x)$ is the closest to the observed $p(x_i)$

Understand the intuition of maximum likelihood with math

It’s actually calculating a sum of series of independent events, each event is the observed outcome of each feature. The goal is to find the max p(x) so that it maximize the sum probablity of all events

Assume feature 1 is when $x_1,y=1$, feature 2 is $x_2,y=0$, …

$$p(x)=p(y=1|x_1)\cdot (y=0|x_2)\cdot (y=1|x_3)\cdot ..\cdot (y=0\text{or}1|x_i)$$

Maximum likelihood is to get the max of p(x). So, for logistic regression, we substitute the logistic function to $p(x_i)$, and let the computer to find the best constant for us.

See more in details in Likelihood function.

Maximum likelihood in distributions §

• Maximum likelihood in exponential distribution
• Maximum likelihood in binomial distribution
• Maximum likelihood in normal distribution