The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
13 views

Gaussian classifier: if two gaussians have equal variance is it possible for them to produce a non-linear decision boundary?

I have been playing with this a bit and I don't believe they can. However, I am very new to machine learning and my maths isn't strong enough to be certain.
0
votes
0answers
15 views

SGD weight learning for mixture of generative models

Let's say that I've learned $k$ generative models $\mathbf{G}=G_i$ in some ways from my data. I'd like to create a mixture from them in order to have something like model averaging in this form: ...
0
votes
0answers
13 views

How alignment between words are generated by IBM model1?

I implemented the translation model IBM1. As a result, I got the translation table P(targetWord|sourceWord) wich is ok. I want also to obtain the alignment of words in the corpus that I used in order ...
2
votes
1answer
106 views

How do you evaluate a generative model?

Evaluating a discriminative model is relatively easy: compare the predictions with ground truth, using cross-validation. Unfortunately this strategy can't be used for generative models. Surely this ...
3
votes
0answers
98 views

What enforces features diversity in RBM?

I'm working on an implementation of a Restricted Boltzman Machine (RBM). I made some tests on the MNIST dataset trying to learn a representation of the digit 2. My inputs are binary images. My aim is ...
2
votes
1answer
38 views

Is there some theory of SVMs with infinitely many data?

I am trying to understand what does it means to have a (linear) SVM classifier (with soft margins) given the generative model of the data. And I realize I have not seen any paper on it, nor can I ...
0
votes
0answers
61 views

GB-RBM unable to learn and generate simple 2D-Motions?

I am trying to apply a GB-RBM to a variation of the LASA Handwriting Dataset to be able to generate new examples. My dataset contains motion trajectories of simple shapes like 'S' shapes, spirals, 'C' ...
1
vote
1answer
401 views

Example of how the log-sum-exp trick works in Naive Bayes

I have read about the log-sum-exp trick in many places (e.g. here, and here) but have never seen an example of how it is applied specifically to the Naive Bayes classifier (e.g. with discrete features ...
0
votes
0answers
343 views

How to implement a Gaussian-Rectified RBM and generate data?

I have a bit trouble implementing a gaussian-rectified RBM. I would like to show you how I would implement it. It would be nice if you point out errors or comment the implementation. These are the ...
2
votes
1answer
170 views

Discriminative vs. Generative Models

This has be asked before, but I still have not grasped it completely. I know that generative models model the feature distribution and that this includes modelling the P(x|y) and P(y), which are not ...
5
votes
2answers
283 views

Do discriminative models overfit more than generative models?

In an interview, the interviewer said that discriminative models tend to overfit more than generative models because they solve a more complex problem and hence consume more resources (or parameters) ...
0
votes
0answers
45 views

Generative Models: Bayesian statistics hw help

Can anyone shed some light on this problem. Not looking for you to write out the answer for me, just some helpful hints that will hopefully lead me in the right direction. Here is the question: ...
2
votes
1answer
65 views

Discriminative learning and generative learning

I'm wondering why in Generative learning algorithm, they try to maximize the probability $\prod_{i=1}^np(x^{(i)}, y^{(i)})$ While in Discriminative learning algorithm, it is $\prod_{i=1}^np(y^{(i)} ...
1
vote
0answers
116 views

Difference between probabilistic generative, discriminative models and “discriminant functions”?

I have been thoroughly confused by a bunch of online resources regarding this issue. I would really like a simple explanation about the differences between generative and discriminative approaches, ...
2
votes
2answers
52 views

Is there a more precise definition of generative models?

Intuitively, a generative model is one that we can generate good data from. What I'm looking for is a formal definition of "good data." For example, for any classification model I could generate ...
2
votes
0answers
80 views

Treating multiple Dichotomies combined?

Let's say I am interested in choosing a new country $c_1, \ldots, c_k$ to live in. For some reason I can only apply to one country and only once. I know for each country a set of 2000 observations ...
1
vote
2answers
223 views

Topic Words Selection in Topic Modeling

I understand how generative model of topic modeling works; for each topic there is a distribution of words, and for each document there is a distribution of topics. Question is how words are ...
1
vote
0answers
71 views

Difference and connection between generative learning, discriminative learning and max-margin learning

I once heard that, generative learning, discriminative learning and max-margin learning can be separated in terms of their respective definition of loss function. I am not sure how to achieve that?
2
votes
1answer
205 views

Generative modeling of a mix of continous and discrete variables

I'm trying to build a generative model to run a Monte Carlo simulation. The existing data consist of a combination of discrete and continuous variables. Suppose I have a number of people... ...
2
votes
0answers
66 views

Statistical model of a website

I know that HMMs can be used to construct statistical models of text. Thus, we can generate text according to this model, and compute the likelihood of a text sample under the model. What tools are ...
8
votes
1answer
136 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
0
votes
0answers
61 views

Generative model that penalizes clumping of data

I'm interested in modeling a generative process that encourages data to be "evenly distributed" over its support, i.e. clumping of data points is penalized. For example, if I have a mixture ...
6
votes
2answers
2k views

Why are Gaussian “discriminant” analysis models called so?

Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate $$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative ...
27
votes
2answers
4k views

Generative vs. discriminative

I know that generative means "based on $P(x,y)$" and discriminative means "based on $P(y|x)$," but I'm confused on several points: Wikipedia (+ many other hits on the web) classify things like SVMs ...
2
votes
0answers
46 views

Generative modelling: what if the generating models have very different “quality of fit”

Say I want to classify my data into two categories. I am pretty sure that my data has been generated by two mixtures of Gaussians -- on has a bimodal and one a trimodal form. I then train the ...
10
votes
2answers
463 views

The connection between Bayesian statistics and generative modeling

Can someone refer me to a good reference that explains the connection between Bayesian statistics and generative modeling techniques? Why do we usually use generative models with Bayesian techniques? ...