Friday, December 27, 2013

Generative model vs Discriminative model

Generative model (e.g., Gaussian Mixture Model)

Idea:
building model from the observation
P(x|c) - giving class c, guess the data point x.

Pros:
get the underlying idea of what the class is built on

Cons:
1. very expensive, lots of parameter
2. needs lots of data

Discriminative model (e.g. SVM)

Idea:
differentiate different classes
P(c|x) - giving data point x, guess the class c.

Pros:
easy to model

Cons: 
to classify but not to generate the data/observation back

Example:
Let a child go to the zoo to see elephant,
when he is back, giving a set of horse and elephant, if he can differentiate elephant from horse only, what he learnt is the discriminative model.
But if he can draw the elephant, what he learnt is the generative model.

Resource from: http://www.youtube.com/watch?v=OWJ8xVGRyFA&noredirect=1