Skip to main content
19 votes
Accepted

QQ Plot Reference Line not 45°

Should it be a 45 degree line? It depends! A QQ plot is the parametric curve defined by: \begin{align*} x &= F^{-1}(p)\\ y &= G^{-1}(p) \end{align*} for $p \in [0, 1]$. Where $F^{-1}$ and $...
Matthew Gunn's user avatar
  • 22.5k
19 votes
Accepted

How can I draw a value randomly from a kernel density estimate?

A kernel density estimator (KDE) produces a distribution that is a location mixture of the kernel distribution, so to draw a value from the kernel density estimate all you need do is (1) draw a value ...
whuber's user avatar
  • 325k
18 votes

How do I test if regression slopes are statistically different?

Assuming you have the original data and not just the summary of the fits, the general solution to this problem is to fit a model with an interaction, i.e. to go back to the data and fit the model $$ Y ...
Ben Bolker's user avatar
13 votes
Accepted

How is $\theta$, the polar coordinate, distributed when $(x,y) \sim U(-1,1) \times U(-1,1)$ vs. when $(x,y) \sim N(0,1)\times N(0,1)$?

You're referring to a transformation from a pair of independent variates $(X,Y)$ to the polar representation $(R,\theta)$ (radius and angle), and then looking at the marginal distribution of $\theta$. ...
Glen_b's user avatar
  • 284k
12 votes
Accepted

Relation between Covariance matrix and Jacobian in Nonlinear Least Squares

This is based on the standard approximation to the Hessian of a nonlinear least squares problem used by Gauss-Newton and Levenberg-Marquardt algorithms. Consider the nonlinear least squares problem: ...
Mark L. Stone's user avatar
12 votes
Accepted

Forward or backward sequential feature selection?

The facts that you are getting different answers from forward and backward selection, and that you get different answers when you change the seed, should give you pause. Clearly, these can't all be ...
gung - Reinstate Monica's user avatar
11 votes

Contrasting covariance calculation using R, Matlab, Pandas, NumPy cov, NumPy linalg.svd

Note that numpy.cov() considers its input data matrix to have observations in each column, and variables in each row, so to get ...
grand_chat's user avatar
  • 2,842
9 votes
Accepted

Logistic regression with {-1,+1} labels

Expanding Frank Harrells answer, to derive likelihood function you first need to define the probabilistic model of the problem. In the case of logistic regression, we are talking about a model for ...
Tim's user avatar
  • 139k
9 votes

In Convolutional Neural Networks (CNN), how we can decide number of kernels between input and hidden layer?

It's important not to confuse kernels with feature maps. The kernels are the masks used to perform convolution on your input image. The feature maps are the result of the convolution, your new ...
Julep's user avatar
  • 507
9 votes
Accepted

Mean squared error of OLS smaller than Ridge?

That is correct because $b_{OLS}$ is the minimizer of MSE by definition. The problem ($X^TX$ is invertible here) has only one minimum and any value other than $b_{OLS}$ will have higher MSE on the ...
gunes's user avatar
  • 57.5k
9 votes

How to propagate measurement uncertainty in predictors *and* responses for multidimensional, non-parametric regression (and software to do it)?

One of the more interesting choices in R is rstan, where you could code this up yourself in the Stan modeling language (which tends to be amazing in that it can ...
Björn's user avatar
  • 32.9k
8 votes
Accepted

Completely different results after each cross validation

I think, you need to provide more information, as there are a lot of possible causes: you have a small dataset, for example (extreme one) only 10 points your classifier depends strong on randomness, ...
Mayou36's user avatar
  • 1,165
8 votes

How is $\theta$, the polar coordinate, distributed when $(x,y) \sim U(-1,1) \times U(-1,1)$ vs. when $(x,y) \sim N(0,1)\times N(0,1)$?

To complete the fairly good answers given by Glen and Michael, I'll just compute the density of $\theta$ when the distribution of $(X,Y)$ is uniform on the square $[-1,1]\times[-1,1]$. This uniform ...
Elvis's user avatar
  • 12.8k
7 votes

Cluster Sequences of data with different length

One way to do it (among many other ways) is to treat the element of your sequence as a word. In other words, if your assume your list is a sentence, then you can extract ngrams. ...
Areza's user avatar
  • 1,148
7 votes
Accepted

Is a very high cost function value a problem by itself?

Directly examining the cost function can be useful, but be aware of some basic issues: Units: Eg. if you measured house price in a less valuable currency (eg. Yen) all the numbers would be higher. ...
Matthew Gunn's user avatar
  • 22.5k
7 votes
Accepted

How is the spherical elevation angle distributed when $(x,y,z)$ are uniformly and normally chosen?

In my discussion here I am assuming your $\theta$ is effectively a longitude and $\phi$ is effectively a latitude. Perhaps more typical spherical co-ordinates use an angle down from the north pole ...
Glen_b's user avatar
  • 284k
7 votes
Accepted

SVM: How to get predicted output from SVM with Gaussian / RBF Kernel? Andrew Ng's course svmPredict

In kernel SVM you map your data points into a possibly infinite dimension Hilbert space $\mathcal H$. It turns out that $w$, the normal vector to the separating hyperplane (which completely ...
jld's user avatar
  • 20.4k
7 votes
Accepted

How to fit the SIR and SEIR models to the epidemiological data?

I am going to confine my comments to the SEIR model - the issues for the SIR model are similar and it can be treated as a special limiting case of the SEIR model anyway (for large $\delta$). What you'...
S. Catterall's user avatar
  • 3,937
7 votes
Accepted

Gaussian process with time series

In reverse order there are many decent GP libraries such as SKLearn, GPy, pyGPs, GPflow and so on. Secondly your input is clearly the time and you can preprocess this as you see fit but you should ...
j__'s user avatar
  • 2,362
6 votes
Accepted

Why would I use any MC technique other than basic sampling

I'll tackle your second question first. Your method doesn't sample from the beta distribution, but you're sort of right in that there are simple methods that work well when you know the pdf. Say that ...
Elizabeth Santorella's user avatar
6 votes
Accepted

Matlab difference between normalized histogram and pdf

If you look carefully, plots 1 and 2 are essentially the same. You've plotted them on different axes, which obfuscates things, but the probability densities at the peaks are essentially identical (...
vbox's user avatar
  • 616
6 votes

How is $\theta$, the polar coordinate, distributed when $(x,y) \sim U(-1,1) \times U(-1,1)$ vs. when $(x,y) \sim N(0,1)\times N(0,1)$?

I will answer the question about the normal case leading to the uniform distribution. It is well known that if $X$ and $Y$ are independent and normally distributed the contours of constant ...
Michael R. Chernick's user avatar
6 votes

How is the spherical elevation angle distributed when $(x,y,z)$ are uniformly and normally chosen?

The complementary cumulative distribution for the spherical latitude $\phi$ gives the chance that a random point in the cube $[-1,1]^3$ will lie above the cone that graphs the function $z = \cot(\phi)\...
whuber's user avatar
  • 325k
6 votes
Accepted

K-Fold Cross validation ambiguity?

I think that you may be misunderstanding how cross-validation works, as well as what it is used to do. Additional detail on your research question and methodology would help. But TL;DR: the average ...
Upper_Case's user avatar
6 votes
Accepted

Wilcoxon signed rank test fails for small sample size

The Wilcoxon sign-rank test is a nonparametric test for difference in ranked differences between two groups. Let's take that apart: Wilcoxon: He gets the credit for developing this test. sign-rank: ...
Alexis's user avatar
  • 30.1k
6 votes
Accepted

Identifying a distribution

I will show this for $U^2,$ instead of $U^5.$ See @olooney's comment below my Answer. Short answer: If $U \sim \mathsf{Unif}(0,1),$ then $X = U^2 \sim \mathsf{Beta}(\frac 1 2, 1).$ Proof: For $x \in ...
BruceET's user avatar
  • 56.5k
6 votes

Reconstruction error PCA using all eigenvectors not zero

It's rounding errors. Floating point arithmetic (IEEE 754) is not exact. In double-precision, errors on the order of $10^{-16}$ are effectively zero. Some related discussion, in the context of p-...
Sycorax's user avatar
  • 91.6k
6 votes

Testing equivalence for consistency validation

To answer the first part of your question, i.e. what you are doing wrong. In your example, you are testing the following hypotheses: $H_0: \mu_s \le \mu$ and $H_1: \mu_s > \mu$ where $\mu = 2.366$ ...
Pitouille's user avatar
  • 1,492
6 votes
Accepted

Overconfidence of Bayesian linear regression models

You should remember that in Bayesian statistics, the outcome (posterior probability) always depends on your prior assumptions (expressed as a prior distribution). In your case, your assumption is is ...
J. Delaney's user avatar
  • 5,390
5 votes

Speed of prediction: neural network vs. random forest?

If tweaking the software and model architecture doesn't do the trick, there's another interesting approach. Say you have a large ensemble model (like a random forest) that has good prediction ...
user20160's user avatar
  • 32.7k

Only top scored, non community-wiki answers of a minimum length are eligible