Conditional Independence and many more!
https://www.youtube.com/watch?v=XtniA_N63z4
It amazes me how I can understand things through this with videos like this.
What is independence?
If X and Y are two events and if X outcome has no relation with Y outcome. They are said to be independent. Suppose X is Liverpool winning and Y is Man Utd and they are not playing against each other. So X has no relation with Y. So these events are independent!
What is Conditional Independence?
Now here comes that thing! Conditional independence of three variables X, Y and Z means that the X and Y are independent when C happens. Remember, we do not care about their dependency before hand of C. They may be independent or dependent at first, but they should be independent after Z comes in!
P(X,Y|Z) = P(X|Z)*P(Y|Z)
Let us illustrate the 3 conditions of Conditional Independece.
First Example:
X: Tossing a coin
Y: Throwing a dice
Z: Drawing a deck of card
X is independent of Y. When Z kicks in X and Y would be independent of each other.
Second Example:
Imagine, Chelsea sitting on top of premier league and all of them are playing their final game. Liverpool and Man United sit 2nd and 3rd on the table. Chelsea host City and tough match. Liverpool play against Man United. So we got 2 events now!
X: Liverpool winning the title after beating Man United
Y: Manchester United winning the title after beating Liverpool
Z: Chelsea winning.
X and Y are dependent on each other. If X wins, Y cannot win and if Y wins, X cannot win or they draw. But if Z comes in, then their result would have no affect on them as they are not winning the title and Chelsea walks away!
At first they are conditionally dependent (winning the title playing against each other), but once Chelsea confirm the win, their result makes no impact as neither of them are crowned Champions!
Third Example:
X: Tossing a coin
Y: Tossing a coin
C: Sum is even number
X and Y are independent at first. But once C comes in, they became conditionally dependent! So if two events no matter their relation what it was before, but after C kicks in and they become dependent on each other is called CONDITIONAL DEPENDENCE!
Formula:
P(x,y,z) = P(x,a) a = (y,z).
From product rule,
P(x,y) = P(x|y)P(y).
P(x,y,z) = P(x,a) = P(x|a).P(a) = P(x|y,z).P(y,z) = P(z). P(y|z).P(x|y,z)
So how do we interpret this?
Probability that x ,y and z occurring is probability of z occurring, y given z occurring and probability that x occurring given y and z.
In General term:
P(x1,x2,x3,x4,x5…xn) = P(x1)P(x2|x1).P(x3|x2,x1).P(x4|x3,x2,x1)…P(X(n)|x(n-1)x(n-2)…x(1))
So, it can be said as Probability that first event occurs times, probability that second event occurs given first event, probability that third event occurs given first and second event. e.t.c.
P(x1,x2,…xn)= P(x1).∏P(Xi)|x1…x(i-1)) i runs from 2 to n