Questions tagged [generative-models]

A probabilistic or statistical model thought about as describing how the values in a sample is actually generated, and not only as a description or approximation.

Filter by
Sorted by
Tagged with
3
votes
1answer
2k views

Using generative models for classification

I think I never saw a generative model used for a classification task: usually a discriminative model is used; Sometimes (AFAIK, with deep neural networks) a generative model is created as a pre-...
2
votes
0answers
110 views

Beginner working with generative models

I'm getting a set of 20-dimensional feature vectors. I'm getting a 100 vectors per second. These feature vectors are occuring in a sequence, where one feature vector depends on the one that came ...
2
votes
1answer
1k views

Linear discriminant analysis likelihood function

It's a lot of time that I don't do probability stuff, so probably this is a very naive question for most of you. Anyway.. For the linear (fisher) discriminant we have an inpu $X$ and an output $Y$ ...
19
votes
2answers
5k views

Modern Use Cases of Restricted Boltzmann Machines (RBM's)?

Background: A lot of the modern research in the past ~4 years (post alexnet) seems to have moved away from using generative pretraining for neural networks to achieve state of the art classification ...
4
votes
1answer
288 views

Are all generative Models based on Bayes?

Reading about deep learning I encounter various different kinds of hierarchical networks, many of which are generative. 1) Are all of the generative networks based on Bayes? 2) If not, how do they ...
1
vote
1answer
136 views

Question regarding Gaussian Discriminant Analysis, and Generative Learning models

In lecture today, my professor mentioned in the context of GDA and Generative learning, we would like to learn the joint probability $P(x, y)$, where $x \in \mathbb{R}^n$ and $y \in \{+1, -1\}$. ...
99
votes
7answers
66k views

How does the reparameterization trick for VAEs work and why is it important?

How does the reparameterization trick for variational autoencoders (VAE) work? Is there an intuitive and easy explanation without simplifying the underlying math? And why do we need the 'trick'?
0
votes
1answer
89 views

Prediction without labelled data

I am working on a churn prediction model, where I am trying to predict probability of employee churn. For each employee I have the following features 1) Role 2) Total experience 3) Current experience ...
4
votes
1answer
627 views

What is the relationship between generative models and density estimation?

If aren't they synonymous, what distinguishes the one from the other? Is probability density estimation a certain kind of generative model? Can any generative model be regarded as density estimation?
1
vote
1answer
750 views

How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. $$p(\mathbf x|C_k):\mathcal N(\mathbf \mu_k,\...
5
votes
3answers
898 views

What is the correct definition of the Likelihood function?

I am doing the CS229:Machine Learning of Stanford Engineering Everywhere. All trhough the first chapter he uses $$L(\theta) = P(Y | X; \theta)$$ i.e. the likelihood of the parameter $\theta$ is ...
2
votes
0answers
1k views

Difference between a generative MRF and discriminative CRF

I am having trouble developing the intuition behind the difference between a regular generative Markov random field (MRF) and its discriminative counterpart. So, as I think I have understood so far ...
0
votes
1answer
68 views

Latent Dirichlet Allocation - definitions

I am self-studying the article on LDA by Blei, Ng and Jordan (https://www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf). at the start of the second section - the following definitions are given: ...
6
votes
1answer
2k views

How do you evaluate a generative model?

Evaluating a discriminative model is relatively easy: compare the predictions with ground truth, using cross-validation. Unfortunately this strategy can't be used for generative models. Surely this ...
5
votes
0answers
367 views

What enforces features diversity in RBM?

I'm working on an implementation of a Restricted Boltzman Machine (RBM). I made some tests on the MNIST dataset trying to learn a representation of the digit 2. My inputs are binary images. My aim is ...
3
votes
1answer
58 views

Is there some theory of SVMs with infinitely many data?

I am trying to understand what does it means to have a (linear) SVM classifier (with soft margins) given the generative model of the data. And I realize I have not seen any paper on it, nor can I ...
2
votes
1answer
484 views

Discriminative vs. Generative Models

This has be asked before, but I still have not grasped it completely. I know that generative models model the feature distribution and that this includes modelling the P(x|y) and P(y), which are not ...
6
votes
2answers
3k views

Do discriminative models overfit more than generative models?

In an interview, the interviewer said that discriminative models tend to overfit more than generative models because they solve a more complex problem and hence consume more resources (or parameters) ...
0
votes
0answers
57 views

Generative Models: Bayesian statistics hw help

Can anyone shed some light on this problem. Not looking for you to write out the answer for me, just some helpful hints that will hopefully lead me in the right direction. Here is the question: We ...
2
votes
2answers
414 views

Why is the objective different in discriminative and generative learning?

I'm wondering why, in a generative learning algorithm, they try to maximize the probability $$\prod_{i=1}^np(x^{(i)}, y^{(i)})$$ while, in a discriminative learning algorithm, it is $$\prod_{i=1}^...
1
vote
0answers
402 views

Difference between probabilistic generative, discriminative models and "discriminant functions"?

I have been thoroughly confused by a bunch of online resources regarding this issue. I would really like a simple explanation about the differences between generative and discriminative approaches, ...
4
votes
2answers
269 views

Is there a more precise definition of generative models?

Intuitively, a generative model is one that we can generate good data from. What I'm looking for is a formal definition of "good data." For example, for any classification model I could generate ...
2
votes
0answers
90 views

Treating multiple Dichotomies combined?

Let's say I am interested in choosing a new country $c_1, \ldots, c_k$ to live in. For some reason I can only apply to one country and only once. I know for each country a set of 2000 observations (...
1
vote
2answers
788 views

Topic Words Selection in Topic Modeling

I understand how generative model of topic modeling works; for each topic there is a distribution of words, and for each document there is a distribution of topics. Question is how words are ...
1
vote
0answers
107 views

Difference and connection between generative learning, discriminative learning and max-margin learning

I once heard that, generative learning, discriminative learning and max-margin learning can be separated in terms of their respective definition of loss function. I am not sure how to achieve that?
3
votes
1answer
667 views

Generative modeling of a mix of continous and discrete variables

I'm trying to build a generative model to run a Monte Carlo simulation. The existing data consist of a combination of discrete and continuous variables. Suppose I have a number of people... ...
2
votes
0answers
76 views

Statistical model of a website

I know that HMMs can be used to construct statistical models of text. Thus, we can generate text according to this model, and compute the likelihood of a text sample under the model. What tools are ...
11
votes
1answer
224 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
1
vote
0answers
89 views

Generative model that penalizes clumping of data

I'm interested in modeling a generative process that encourages data to be "evenly distributed" over its support, i.e. clumping of data points is penalized. For example, if I have a mixture ...
13
votes
2answers
5k views

Why are Gaussian "discriminant" analysis models called so?

Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate $$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative ...
196
votes
3answers
93k views

Generative vs. discriminative

I know that generative means "based on $P(x,y)$" and discriminative means "based on $P(y|x)$," but I'm confused on several points: Wikipedia (+ many other hits on the web) classify things like SVMs ...
3
votes
0answers
57 views

Generative modelling: what if the generating models have very different "quality of fit"

Say I want to classify my data into two categories. I am pretty sure that my data has been generated by two mixtures of Gaussians -- on has a bimodal and one a trimodal form. I then train the ...
16
votes
2answers
4k views

The connection between Bayesian statistics and generative modeling

Can someone refer me to a good reference that explains the connection between Bayesian statistics and generative modeling techniques? Why do we usually use generative models with Bayesian techniques? ...

1 2 3 4
5