charles.y.zheng
  • Member for 10 years, 10 months
  • Last seen more than 1 year ago
Can someone explain Gibbs sampling in very simple words?
Accepted answer
184 votes

You are a dungeonmaster hosting Dungeons & Dragons and a player casts 'Spell of Eldritch Chaotic Weather (SECW). You've never heard of this spell before, but it turns out it is quite involved. ...

View answer
What's the difference between probability and statistics?
66 votes

It's misleading to simply say that statistics is simply the inverse of probability. Yes, statistical questions are questions of inverse probability, but they are ill-posed inverse problems, and this ...

View answer
What is the purpose of characteristic functions?
Accepted answer
60 votes

Back in the day, people used logarithm tables to multiply numbers faster. Why is this? Logarithms convert multiplication to addition, since $\log(ab) = \log(a) + \log(b)$. So in order to multiply ...

View answer
How to choose between Pearson and Spearman correlation?
40 votes

This happens often in statistics: there are a variety of methods which could be applied in your situation, and you don't know which one to choose. You should base your decision the pros and cons of ...

View answer
Intuitive explanation of Fisher Information and Cramer-Rao bound
35 votes

Here I explain why the asymptotic variance of the maximum likelihood estimator is the Cramer-Rao lower bound. Hopefully this will provide some insight as to the relevance of the Fisher information. ...

View answer
Why are random variables defined as functions?
Accepted answer
24 votes

If you are wondering why all this machinery is used when something much simpler could suffice--you are right, for most common situations. However, the measure-theoretic version of probability was ...

View answer
Do we have a problem of "pity upvotes"?
13 votes

Conduct an experiment. Randomly downvote half of the new posts at a particular time every day.

View answer
Is there more to probability than Bayesianism?
12 votes

The Bayesian interpretation of probability suffices for practical purposes. But even given a Bayesian interpretation of probability, there is more to statistics than probability, because the ...

View answer
Is it worthwhile to publish at the refereed wiki StatProb.com?
Accepted answer
8 votes

As long as the sponsors of the site are committed to keeping the site running, it would be premature to declare it 'dead.' It is not out of the question that StatProb.com may experience a revival in ...

View answer
Stein's loss for multivariate normal covariance estimator
7 votes

I worked out the details omitted from JMS's answer. Let $P_0 = \Sigma_0^{-1}$ and $P_1 = \Sigma_1^{-1}$ We have $$KL(N_0||N_1) = \int N_0(x) \frac{1}{2}[\ln |P_0| - \ln|P_1| + (x-\mu_0)^T P_0 (x-\...

View answer
How to interpret the divergence of Fisher information expectation?
Accepted answer
6 votes

A rough answer is that the MLE of a location parameter of distribution with non-translation-invariant support can converge faster than $O(1/\sqrt{n})$. In this case, the divergent entry in the ...

View answer
How to measure the "well-roundedness" of SE contributors?
6 votes

EXAMPLE: say there are three sites, and we want to compare the well-roundedness of the Users A, B, C. We write the reputations of the users across the three sites in vector form: User A: [23, 23, ...

View answer
How to understand moments for a random variable?
Accepted answer
6 votes

If you have a linear rod, the center of gravity is the first moment (the expected value), and the moment of rotational inertia about the center of gravity is the variance. (A rod with centrally ...

View answer
Estimate the nearest of N random points in a box in E^d?
6 votes

First, here are some commonly known facts which will be useful. Suppose i.i.d $X_1, \cdots, X_n$ have the cumulative distribution function $F(X) = P[X \leq x]$, then cumulative distribution function ...

View answer
How to use rejection sampling to generate draws from Unit Exponential
Accepted answer
5 votes

Try a location shift on the Gamma(2,1) EDIT: Illustration

View answer
Given two responses for two groups, how to decide what to test on response or profit?
5 votes

The reason why you are conducting this test is to determine which policy is more valuable, and if value is measured in profitability, then it makes no sense to do statistical testing on any other ...

View answer
Fitting 4-moment distribution with mixture gaussian
5 votes

I am not exactly sure what your code is trying to do, but it seems like you should be using $rnorm()$ (with a large $n$) instead of $dnorm()$, since the functions $mean()$, $var()$, etc. are designed ...

View answer
Hot topics in mathematical statistics
Accepted answer
4 votes

Persi Diaconis wrote an article on "Mathematical Statistics" in the Princeton Companion of Mathematics. He discusses several ongoing research areas in mathematical statistics, including the search ...

View answer
Property of entropy
Accepted answer
4 votes

There is a simple interpretation of the above grouping property. Suppose your alphabet is $A, B, C,...$ where the letters have frequency $p_1, p_2, p_3, ..$ Now let $S$ be a random sequence of large ...

View answer
How to predict and optimize a queue?
4 votes

Queue models depend on a few key distributions: the distribution of the time gaps between incoming jobs, the distribution of service times (how long it takes to process a job). Some commonly used ...

View answer
Estimating variability of unseen factor
Accepted answer
4 votes

Supposing for independent $X$, $Y$, we know $E[X]$, $E[XY]$, $var(X)$, $var(XY)$. Thus we know $E[Y] = E[XY]/E[X]$, $E[X^2]=var(X) - E[X]^2$ and $E[(XY)^2]=var[XY]-E[XY]^2$. Since $E[(XY)^2] = E[X^2 ...

View answer
How to ensure properties of covariance matrix when fitting multivariate normal model using maximum likelihood?
3 votes

An alternative parameterization for the covariance matrix is in terms of eigenvalues $\lambda_1,...,\lambda_p$ and $p(p-1)/2$ "Givens" angles $\theta_ij$. That is, we can write $$\Sigma = G^T \...

View answer
Learning from unordered tuples?
3 votes

To add to onestop's response, it was confirmed on math.SE that the polynomials $$w_1 = x_1 + \cdots + x_n$$ $$w_2 = x_1^2 + \cdots + x_n^2$$ $$\cdots$$ $$w_n = x_1^n + \cdots + x_n ^n$$ give you all ...

View answer
Choosing sample size to achieve pre-specified margin-of-error
3 votes

If you weight your measurements (proportion of subpopulation/proportion of subpopulation in sample), your estimates will be unbiased. I assume this is what you meant by "poll results being ...

View answer
Establishing relationship between 2 diseases
Accepted answer
3 votes

There is no simple answer for how to "establish a relationship between two variables;" indeed, your question is one of the central issues in statistics and research is still going on on how to do this....

View answer
What is the expected dot product of two evolving vectors?
Accepted answer
3 votes

EDIT 2: We have to simplify the problem by removing the restriction that $x \ne y$ always in case two. Otherwise correlation issues vastly complicate the answer. Let $|v|$, the 1-norm, denote the ...

View answer
2x2 chi-square test vs. binomial proportion statistic
Accepted answer
3 votes

Suppose you have Case 1: A=200, B=100 C=100, D=200 versus Case 2: A=200, B=0 C=200, D=200 The B=0 in case 2 means that case 2 provides much stronger evidence than case 1 of a relationship between ...

View answer
What is a good way of estimating the dependence of an output variable on the input parameters?
Accepted answer
3 votes

EDIT: After some reflection, I modified my answer substantially. The best thing to do would be to try to find a reasonable model for your data (for example, by using multiple linear regression). If ...

View answer
Is there a way to continue a R/JAGS MCMC chain that did not converge?
2 votes

If there is an option to choose a starting point--yes. You can "continue" an MCMC chain simply by using the last point of the chain as a new starting point.

View answer
Feature selection for low probability event prediction
2 votes

The problem of estimating probabilities falls under the category of "regression," since the probability is a conditional mean. Classical methods for feature selection (AKA "subset selection" or "...

View answer