Tag Info

1 vote

R squared of subgroups

How you perform and calculate such a statistic depends on what you want to learn by from it. My belief about $R^2$ is that it is a comparison of how your model performs (in terms of square loss) vs ...
• 46.6k
1 vote

Why can LASSO MAE be worse than individual feature linear regression MAE?

why did LASSO chose multiple features that gives higher error rather than only keeping a single or fewer features that gives a lower error? You told the regression to minimize the LASSO loss and then ...
• 46.6k

Neural network parameters dependency vs gradient descent

From my experience, there is not a simple solution to this issue. A reasonable approach is to use the state-of-the-art parameters in your field, like computer vision and so on. For example, the choice ...
• 19

prior & posterior probability in Bayesian Decision Theory

Let us give an example that makes this as simple as possible. Suppose your samples are all taken from a Bernoulli distribution, i.e. these are binary samples'', these are just $1$'s and $0$'s. Let ...
• 1,725

prior & posterior probability in Bayesian Decision Theory

In a very simplified way: Posterior probability = $P(\gamma | D)$ = probability that your parameter (or vector of parameters) $\gamma$ is equal to the value you've sampled given your dataset $D$. ...
• 145

prior & posterior probability in Bayesian Decision Theory

$x$ and $c$ in Bayes theorem are random variables. Any random variables. Bayes's theorem is about being able to flip sides of the conditional distribution from $P(x|c)$ to $P(c|x)$ or the other way ...
• 128k

Combining image and scalar inputs into a neural network

The features vector can be combined to an image by - Adjusting the features shape by using tf.reshape and tf.tile Combining the features and image by performing concatenation, add (as described in ...
• 101

I need to get 100% accuracy on my training data

What you are describing is that you need to losslessly compress and retrieve the data. Did you consider any off-the-shelf caching solution? Since you care about “predicting” the seen data, it's about ...
• 128k
1 vote

Why do the error derivatives become small if we start with a large learning rate?

So in the case of sigmoid neurons, having large weights means the hidden unit output saturates, so then changes in the weights have minimal effect on the hidden unit output, and therefore the error ...
• 5,384
Accepted

Is the iid assumption in Linear Regression necessary?

If you are asking about the i.i.d. assumption in machine learning in general, we already have that question answered in the On the importance of the i.i.d. assumption in statistical learning question. ...
• 128k

What are the methods to increase the dimension of a feature space?

There are lots of different ways. A classic example is polynomial expansions - you take all powers of your input variables (https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theorem from 1885) ...
1 vote
Accepted

Calculating KL divergence with entropy and cross entropy for VAEs

The Kullback–Leibler divergence is also called relative entropy. It is easy to see that: \begin{align} D_{KL}(P\|Q) &=\int p(x)\log\left(\frac{p(x)}{q(x)}\right)dx\\ &=\underbrace{\int p(x)\...
• 17k

What are the methods to increase the dimension of a feature space?

A feature transformation can be "learned" by fitting a neural network. For example, if the original features $x \in \mathbb{R}^d$ are mapped to one of $k$ classes, the mapping may be modeled ...

Practical usefulness of PCA

I use three examples in my lectures to illustrate what PCA can do (click the links for pointers to the slides). They're chosen to show how useful it is in general data science practice and how ...

What are the methods to increase the dimension of a feature space?

Square them. Log them. Multiply them. Multiply their logs. Log their products. Take Fourier or wavelet transforms of time series data. Any function of your set of features is a possible additional ...
1 vote

Practical usefulness of PCA

Lots of what we're trying to get at with dimensionality reduction, whether linear or nonlinear, is abstraction of the data away from a raw space and towards a manifold or embedding in which complex ...

How to handle machine learning inputs that should be considered as a set of vectors, but whose interpretation is order invariant (order agnostic set)

This is "multiple instance learning"; the wikipedia page has a good introduction. Your examples sound like they don't conform to the standard assumption, so you'll need some "metadata-...
• 3,715

Practical usefulness of PCA

It is absolutely not clear, obvious, or agreed-upon what "intelligence" is in humans (or, for that matter, in nonhuman animals). What is clear is that people's performance on a variety of ...

Can I use K-Means to group customers based on a single variable?

If you want a data driven clustering, k-means looks promising in the sense that it will produce clusters with similar within-cluster variance, which may make sense in your application. The problem of ...
• 16.7k

Practical usefulness of PCA

PCA just comes down to using the eigendecomposition of the (empirical) covariance matrix of the data. The full eigendecomposition of the covariance matrix results in a set of eigenvectors and ...

Practical usefulness of PCA

I've used PCA in facial motion capture for real time animatronic control of the 'lots of dots on a face' variety. I was able to find out which dots - which is to say regions of the face - encoded the ...

Practical usefulness of PCA

First, from the perspective of education, PCA is a good entryway to the world of dimension-reduction techniques and associated methods. Whether we're talking ICA, non-negative matrix factorization, ...
1 vote

Can I use K-Means to group customers based on a single variable?

Given your description, I would just assign cutoff points at percentiles of the distribution of total spend. With five categories, equal intervals (same number of observations in each category) would ...
• 21
1 vote

Does feature selection and model testing have to be coupled in each fold of the cross-validation?

Yes, you should run the entire pipeline for each fold You are right to say that using feature selection on all samples would mean you have data leakage in the following classification cross-validation ...
• 405

Practical usefulness of PCA

One important use of PCA is in analysis of electroencephalography (EEG) data. To measure an EEG, dozens of electrodes are attached to your scalp and measure electric currents in your brain, either at ...

Practical usefulness of PCA

One application of PCA that I have used a few times is the construction of social indicators. We use the projection of each observation (usually households) over a component axis (usually the first), ...

• 128k
1 vote

What does the error in artificial neural network stand for, is the same with mean square error (MSE)

It depends on what loss or error function you use when you code the network. If your loss is the sum of squared errors (SSE), then divide this value by the number of predictions being made. If your ...
• 46.6k

Machine learning method(s) to compare probability of success of two groups

You are looking for a probabilistic classifier. You can feed in your predictor data for the two groups (so the predictors will presumably only differ in the group membership) and get the success ...
• 109k

How to predict both category and sub category in machine learning classification?

With neural networks, you would do this by having multiple outputs: the (logit)-probability of being in each category from a to d (that's the easy bit with not too many choices) AND one of the ...
• 27.2k