alto
• Member for 10 years, 4 months
• Last seen more than 3 years ago

In some sense I think this question is unanswerable. I say this because how well a particular unsupervised method performs will largely depend on why one is doing unsupervised learning in the first ...

There has been some work on adapting deep learning methods for sequential data. A lot of this work has focused on developing "modules" which can be stacked in a way analogous to stacking restricted ...

Although Deep Belief Networks (DBNs) and Deep Boltzmann Machines (DBMs) diagrammatically look very similar, they are actually qualitatively very different. This is because DBNs are directed and DBMs ...

You are right about unlabeled data. RBMs are generative models and most commonly used as unsupervised learners. When used for constructing a Deep Belief Network the most typical procedure is to ...

You need to first calculate all your updates as if the wieghts weren't shared, but just store them, don't actually do any updating yet. Let $w_k$ be some weight that appears at locations $I_k = \{(i,... View answer Accepted answer 9 votes The approach you describe for using HMMs for classification is really only applicable to settings where you have independent sequences you want to classify. For example, if I was classifying the ... View answer 9 votes Naive Bayes generally uses a decision rule like $$\text{argmax}_{C_i} P(C_i)P(D|C_i),$$ which comes from the fact we can write $$P(C_i|D) = \frac{P(C_i)P(D|C_i)}{P(D)}.$$ and drop the denominator ... View answer Accepted answer 8 votes Let$D = \{(x_1, y_1), \ldots, (x_n, y_n)\}$be a set of i.i.d. observations where$x_i$is some$n$dimensional vector of independent variables and$y_i$is a binary dependent variable. A common ... View answer 7 votes This is a fun question as it provides good context for why the often used heuristic that more parameters$\implies$more risk of overfitting is just that, a heuristic. To ground the discussion let's ... View answer Accepted answer 7 votes For sake of simplicity let's assume you're doing binary classification, everything I'll say generalizes straightforwardly to the multiclass case, with$D = X \times Y$your dataset and$P(Y=1) < P(...

The $V(s)$'s are sufficient to determine a policy precisely because you have access to the model. In particular you have access to information about the transition structure of the MDP (you know how ...

Cascade-Correlation Neural Networks adjust their structure by adding hidden nodes during the training process, so this may be a place to start. Most of the other work I've seen that automatically ...

The models in both papers are Gaussian-Bernoulli RBMs. The difference is the sparse variant includes a term in the object which penalizes hidden units whose conditional expectation deviates from a ...

The short answer is if you used leave-one-out CV for time series, you would be fitting model parameters based on data from the future. The easiest way to see how this is to just write out what both ...

You could use the hashing trick. That way rather than providing a table which maps words to indices, which would reveal information about the words in your training data, you could just provide a hash ...

First, multiclass logistic regression is a linear classifier, are your inputs linearly separable? If they're handwritten digits I highly doubt it. Next, multiclass logistic regression uses the ...

One disadvantage of an unsupervised method like LDA is it will generally take considerably longer to train compared to supervised methods. I'm also confused about the 2% increase you mention, based on ...

A Hidden Markov Model is defined by two different probability distributions, namely \begin{align*} p(s_t \mid s_{t-1}),&\;\;\;\text{the transition probabilities, and}\\ p(x_t \mid s_t),&\;\... View answer 4 votes It sounds like you want reinforcement learning. I'm having a little trouble parsing the exact details of your specific problem, but perhaps it could be cast in the framework of a Multi-Armed Bandit ... View answer Accepted answer 3 votes Ignoring the terms where this happens is the correct thing to do. You can justify this by noting that in each case you've outlined, no matter what happens inside the \log you will have P(x,y,z) = 0... View answer Accepted answer 3 votes One problem with the approach you've described is you will need to define what kind of increase in P(O) is meaningful, which may be difficult as P(O) will always be very small in general. It may ... View answer Accepted answer 3 votes What you describe is probably one of the oldest and most well known classifiers, the k-nearest neighbors (k-nn) classifier with k=1. It has some interesting thoerectical properties. For ... View answer Accepted answer 3 votes In my opinion a machine learning approach is going to be overkill for your problem. The first thing I would try is a system that looks something like Given a new address compute the Levenshtein ... View answer 3 votes It really depends on what your ultimate goal is. If you just care about overall accuracy and the class priors you observe in your training set are a good estimate of what you are likely to see in the ... View answer 3 votes What you describe is essentially just logistic regression with a scaled output using squared loss rather than the usual log loss. Notice that \tanh(x) = 2\sigma(x) - 1 where \sigma(x) = \frac{1}{...

A HMM can give you probabilities of a sequence, so you could just learn a HMM for each class. Classification of a new sequence then comes down to 1.) calculating the probability of the new sequence ...

Most techniques for "normalization" are invertible. If you think of your normalization procedure as a function $f$, this means we can find a function $f^{-1}$ such that $y = f^{-1}(f(y))$. This means ...

After thinking a bit about your problem it is essentially coreset selection, i.e., finding a small subset (the coreset) of the training data such that the model trained on the subset is as close as ...

In general I don't think this is going to be possible (let alone useful). Recall that VC-dimension is an existence property, i.e., for some hypothesis class $H$, $\text{VCdim}(H) = d$ if there exists ...