David Marx
  • Member for 10 years
  • Last seen more than a month ago
Comparing SVM and logistic regression
42 votes

Linear SVMs and logistic regression generally perform comparably in practice. Use SVM with a nonlinear kernel if you have reason to believe your data won't be linearly separable (or you need to be ...

View answer
Error "system is computationally singular" when running a glm
35 votes

It means your design matrix is not invertible and therefore can't be used to develop a regression model. This results from linearly dependent columns, i.e. strongly correlated variables. Examine the ...

View answer
Gentler approach to Bayesian statistics
Accepted answer
31 votes

The use of probability density functions in calculations. In other words how to evaluate such equations. I think you're still thinking of this from a frequentist perspective: if you're looking for a ...

View answer
Difference between Bayes network, neural network, decision tree and Petri nets
Accepted answer
30 votes

Wow, what a big question! The short version of the answer is that just because you can represent two models using diagrammatically similar visual representations, doesn't mean they are even remotely ...

View answer
Interpretting LASSO variable trace plots
Accepted answer
24 votes

In both plots, each colored line represents the value taken by a different coefficient in your model. Lambda is the weight given to the regularization term (the L1 norm), so as lambda approaches zero, ...

View answer
Understanding Metropolis-Hastings with asymmetric proposal distribution
Accepted answer
22 votes

The bibliography states that if q is a symmetric distribution the ratio q(x|y)/q(y|x) becomes 1 and the algorithm is called Metropolis. Is that correct? Yes, this is correct. The Metropolis ...

View answer
Is variation the same as variance?
Accepted answer
22 votes

Here's a full wikipedia article discussing this topic: http://en.wikipedia.org/wiki/Statistical_dispersion As described by others in the comments here, the short answer is: no, variation $\ne$ ...

View answer
Stats is not maths?
16 votes

Mathematics deals with idealized abstractions that (almost always) have absolute solutions, or the fact that no such solution exists can generally be described fully. It is the science of discovering ...

View answer
How can I interpret what I get out of PCA?
Accepted answer
15 votes

Pages 13-20 of the tutorial you posted provide a very intuitive geometric explanation of how PCA is used for dimensionality reduction. The 13x13 matrix you mention is probably the "loading" or "...

View answer
Q: what book on Bayesian statistics, preferably with R?
Accepted answer
13 votes

Peter D. Huff. A First Course in Bayesian Statistical Methods. Springer (2010) Also Andrew Gelman et. al. Bayesian Data Analysis (3rd ed.). CRC (2013) The Gelman book isn't constrained to R but ...

View answer
What is the difference between a hierarchical linear regression and an ordinary least squares (OLS) regression?
11 votes

Building hierarchical models is all about comparing groups. The power of the model is that you can treat the information about a particular group as evidence relating how that group compares to the ...

View answer
How do I choose parameters for my beta prior?
10 votes

If your prior belief is that 9 of the 10 coin flips will come up heads, then you want the expectation of your prior to be 0.9. Given $X \sim \mathrm{Beta}(\alpha,\beta)$ (for conjugacy in the beta-...

View answer
Comparing histograms
8 votes

Your histograms need to not just have the same number of bins, they need to have the same bins. Set your histograms relaive to the max and min of your whole dataset, not just relative to each gender. ...

View answer
Prediction of a binary variable
Accepted answer
8 votes

A binary logistic regression is generally used for fitting a model to a binary output, but formally the results of logistic regression are not themselves binary, they are continuous probability values ...

View answer
Constructing confidence intervals for predictive model
Accepted answer
8 votes

Bootstrap sample your training data many times (let's say, N times) and train a model from each bootstrapped sample (giving N models). Calculate a prediction on your test set using each model (giving ...

View answer
Does more variables mean tighter confidence intervals?
7 votes

do I ALWAYS get a tighter confidence interval if I include more variables in my model? Yes, you do (EDIT: ...basically. Subject to some caveats. See comments below). Here's why: adding more variables ...

View answer
Metropolis Algorithm
Accepted answer
6 votes

If you're just using a metropolis algorithm, you need a symmetric proposal distribution, where "symmetric" in this context means $p(a|b) = p(b|a)$ (i.e. the probability of moving from $a$ to $b$ is ...

View answer
symmetry of linear regression
6 votes

The reason regression is not symmetrical is because it specifically is minimizing the error between the regression line and the response variable, i.e. in the direction of the y-axis for y=ax and in ...

View answer
Why do we use conditional expectation vs regular expectation in regression?
6 votes

$E(Y)$ is just the mean of your responses: it's the same thing as a regression where all you have is an intercept. For all values of $X$, you are predicting the same value for the response. We use ...

View answer
Visualizing data - design idea
6 votes

You could use sparklines or something like that, but instead I'd recommend presenting your data overlaid on top of a table of people's names, or perhaps a seating chart to add the additional ...

View answer
Named entity recognition and class imbalance
Accepted answer
6 votes

Some things you can try: Oversample your target classes. Insert duplicate records of your other three classes to augment your training dataset Undersample the negative responses. Instead of including ...

View answer
Interpreting plot of PCA results (from 3 to 2 dimensions)
5 votes

Your initial data was rotated in the existing three dimensions such that the bulk of the variance was along the X axis, then rotated again such that the remaining variance was predominantly along the ...

View answer
Is random forest a boosting algorithm?
5 votes

I believe you are confusing boosting in particular with ensemble methods in general, of which there are many. Your "definition" of boosting is not the full definition, which is elaborated on in Pat's ...

View answer
Explaining a boxplot and providing a reference in a technical paper
Accepted answer
5 votes

I recommend that you be explicit about all elements of the plot. Explain how the boxplot indicates the median (mean?), quartiles (quantiles?), and extreme values (distant quantiles?)... assuming that'...

View answer
Why do we estimate the mean response in Confidence interval but predict individual outcome in prediction?
Accepted answer
5 votes

The difference between the confidence interval for the mean response and the prediction interval is subtle but important. I'll explain it first and then provide you with a graphical intuition which ...

View answer
Can the t-distribution be defined as the distribution on the true mean of a sampled normal?
5 votes

You don't have a distribution on the true mean, you have a distribution on the difference between the true mean and the sampled mean, and this difference is scaled by the sampled standard deviation (...

View answer
How to standardize data in Excel?
Accepted answer
5 votes

Like this: $$\frac{X - \bar{X}}{SD(X)}$$ This is often referred to as a Z-Score. I believe NORM.S.DIST gives you access to the CDF and PDF for $Z \sim N(0,1)$. This is not the same as Z-...

View answer
Is there "unsupervised regression"?
5 votes

The closest thing I can think of is a little black magic that stirred people up when it was announced a few years ago, but I don't believe it gained any real traction in the community. The authors ...

View answer
Differentiate Ordinal and Nominal variables using python
4 votes

There's no practical way to do this automatically. For example: if some data uses integers for IDs, how is your algorithm supposed to know that these are nominal and not ordinal? This distinction ...

View answer
Conditional distribution of a discrete random variable given a continuous one
Accepted answer
4 votes

(hint hint) Try starting out by writing out the Bayes formula with the full PDFs of $p(X|Y)$ and $p(Y)$ (we can ignore $p(X)$: can you tell me why?). Multiple $p(X|Y)$ and $p(Y)$ together and see ...

View answer
1
2 3 4 5