alto
  • Member for 10 years, 4 months
  • Last seen more than 3 years ago
Performance metrics to evaluate unsupervised learning
Accepted answer
60 votes

In some sense I think this question is unanswerable. I say this because how well a particular unsupervised method performs will largely depend on why one is doing unsupervised learning in the first ...

View answer
Using deep learning for time series prediction
29 votes

There has been some work on adapting deep learning methods for sequential data. A lot of this work has focused on developing "modules" which can be stacked in a way analogous to stacking restricted ...

View answer
Deep belief networks or Deep Boltzmann Machines?
20 votes

Although Deep Belief Networks (DBNs) and Deep Boltzmann Machines (DBMs) diagrammatically look very similar, they are actually qualitatively very different. This is because DBNs are directed and DBMs ...

View answer
Restricted Boltzmann Machines for regression?
Accepted answer
18 votes

You are right about unlabeled data. RBMs are generative models and most commonly used as unsupervised learners. When used for constructing a Deep Belief Network the most typical procedure is to ...

View answer
Training a convolution neural network
Accepted answer
13 votes

You need to first calculate all your updates as if the wieghts weren't shared, but just store them, don't actually do any updating yet. Let $w_k$ be some weight that appears at locations $I_k = \{(i,...

View answer
How do I train HMM's for classification?
Accepted answer
9 votes

The approach you describe for using HMMs for classification is really only applicable to settings where you have independent sequences you want to classify. For example, if I was classifying the ...

View answer
Why does my naive Bayes classifier only give me probabilities near 0?
9 votes

Naive Bayes generally uses a decision rule like $$ \text{argmax}_{C_i} P(C_i)P(D|C_i), $$ which comes from the fact we can write $$ P(C_i|D) = \frac{P(C_i)P(D|C_i)}{P(D)}. $$ and drop the denominator ...

View answer
What is the interpretation of log-loss value?
Accepted answer
8 votes

Let $D = \{(x_1, y_1), \ldots, (x_n, y_n)\}$ be a set of i.i.d. observations where $x_i$ is some $n$ dimensional vector of independent variables and $y_i$ is a binary dependent variable. A common ...

View answer
Do discriminative models overfit more than generative models?
7 votes

This is a fun question as it provides good context for why the often used heuristic that more parameters $\implies$ more risk of overfitting is just that, a heuristic. To ground the discussion let's ...

View answer
How to deal with low frequency examples in classification?
Accepted answer
7 votes

For sake of simplicity let's assume you're doing binary classification, everything I'll say generalizes straightforwardly to the multiclass case, with $D = X \times Y$ your dataset and $P(Y=1) < P(...

View answer
Why is the state-action value function required when the model is unknown?
6 votes

The $V(s)$'s are sufficient to determine a policy precisely because you have access to the model. In particular you have access to information about the transition structure of the MDP (you know how ...

View answer
Dynamically adjusting NN architecture: inventing the unnecessary?
Accepted answer
6 votes

Cascade-Correlation Neural Networks adjust their structure by adding hidden nodes during the training process, so this may be a place to start. Most of the other work I've seen that automatically ...

View answer
How is a Sparse RBM different from a Gaussian-Bernoulli RBM
Accepted answer
5 votes

The models in both papers are Gaussian-Bernoulli RBMs. The difference is the sparse variant includes a term in the object which penalizes hidden units whose conditional expectation deviates from a ...

View answer
How is Hyndman's explanation of proper Time Series Cross Validation different from Leave-One-Out?
Accepted answer
5 votes

The short answer is if you used leave-one-out CV for time series, you would be fitting model parameters based on data from the future. The easiest way to see how this is to just write out what both ...

View answer
Sharing a model trained on confidential data
Accepted answer
5 votes

You could use the hashing trick. That way rather than providing a table which maps words to indices, which would reveal information about the words in your training data, you could just provide a hash ...

View answer
Algorithm convergence with logistic classifier
Accepted answer
5 votes

First, multiclass logistic regression is a linear classifier, are your inputs linearly separable? If they're handwritten digits I highly doubt it. Next, multiclass logistic regression uses the ...

View answer
Supervised approaches vs. topic models in sentiment analysis
Accepted answer
5 votes

One disadvantage of an unsupervised method like LDA is it will generally take considerably longer to train compared to supervised methods. I'm also confused about the 2% increase you mention, based on ...

View answer
How to handle high dimensional feature vector in probability graph model?
Accepted answer
4 votes

A Hidden Markov Model is defined by two different probability distributions, namely $$ \begin{align*} p(s_t \mid s_{t-1}),&\;\;\;\text{the transition probabilities, and}\\ p(x_t \mid s_t),&\;\...

View answer
Machine learning task with feedback loop
4 votes

It sounds like you want reinforcement learning. I'm having a little trouble parsing the exact details of your specific problem, but perhaps it could be cast in the framework of a Multi-Armed Bandit ...

View answer
conditional mutual information and how to deal with zero probabilities
Accepted answer
3 votes

Ignoring the terms where this happens is the correct thing to do. You can justify this by noting that in each case you've outlined, no matter what happens inside the $\log$ you will have $P(x,y,z) = 0$...

View answer
Hidden Markov model for event prediction
Accepted answer
3 votes

One problem with the approach you've described is you will need to define what kind of increase in $P(O)$ is meaningful, which may be difficult as $P(O)$ will always be very small in general. It may ...

View answer
Training set as donor for test set in binary classification problem
Accepted answer
3 votes

What you describe is probably one of the oldest and most well known classifiers, the $k$-nearest neighbors ($k$-nn) classifier with $k=1$. It has some interesting thoerectical properties. For ...

View answer
Machine learning to catch fraud
Accepted answer
3 votes

In my opinion a machine learning approach is going to be overkill for your problem. The first thing I would try is a system that looks something like Given a new address compute the Levenshtein ...

View answer
Equal number of training instances of each classification label?
3 votes

It really depends on what your ultimate goal is. If you just care about overall accuracy and the class priors you observe in your training set are a good estimate of what you are likely to see in the ...

View answer
What is the name of this perceptron-like classifier?
3 votes

What you describe is essentially just logistic regression with a scaled output using squared loss rather than the usual log loss. Notice that $\tanh(x) = 2\sigma(x) - 1$ where $$ \sigma(x) = \frac{1}{...

View answer
What machine learning algorithm solves this problem?
3 votes

A HMM can give you probabilities of a sequence, so you could just learn a HMM for each class. Classification of a new sequence then comes down to 1.) calculating the probability of the new sequence ...

View answer
Denormalizing Data
Accepted answer
2 votes

Most techniques for "normalization" are invertible. If you think of your normalization procedure as a function $f$, this means we can find a function $f^{-1}$ such that $y = f^{-1}(f(y))$. This means ...

View answer
Leave One Out Cross Validation
Accepted answer
2 votes

After thinking a bit about your problem it is essentially coreset selection, i.e., finding a small subset (the coreset) of the training data such that the model trained on the subset is as close as ...

View answer
How to compute minimum required VC dimension for a classifer to classify a specific data
2 votes

In general I don't think this is going to be possible (let alone useful). Recall that VC-dimension is an existence property, i.e., for some hypothesis class $H$, $\text{VCdim}(H) = d$ if there exists ...

View answer
Random walk under changing conditions
2 votes

It sounds like a Hidden Markov model will do exactly what you want. A HMM assumes there is a discrete set of latent (unobservable) states which evolve according to a discrete time Markov process and ...

View answer