Questions tagged [inference]

Drawing conclusions about population parameters from sample data. See https://en.wikipedia.org/wiki/Inference and https://en.wikipedia.org/wiki/Statistical_inference

Filter by
Sorted by
Tagged with
169
votes
4answers
249k views

How to interpret a QQ plot

I am working with a small dataset (21 observations) and have the following normal QQ plot in R: Seeing that the plot does not support normality, what could I infer about the underlying distribution? ...
92
votes
12answers
10k views

Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
80
votes
10answers
57k views

Understanding “variance” intuitively

What is the cleanest, easiest way to explain someone the concept of variance? What does it intuitively mean? If one is to explain this to their child how would one go about it? It's a concept that I ...
64
votes
8answers
6k views

What is a good, convincing example in which p-values are useful?

My question in the title is self explanatory, but I would like to give it some context. The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
58
votes
11answers
9k views

Two-tailed tests… I'm just not convinced. What's the point?

The following excerpt is from the entry, What are the differences between one-tailed and two-tailed tests?, on UCLA's statistics help site. ... consider the consequences of missing an effect in the ...
44
votes
7answers
5k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
41
votes
3answers
57k views

Testing equality of coefficients from two different regressions

This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this? More formally, ...
37
votes
6answers
28k views

Rule of thumb for number of bootstrap samples

I wonder if someone knows any general rules of thumb regarding the number of bootstrap samples one should use, based on characteristics of the data (number of observations, etc.) and/or the variables ...
33
votes
4answers
3k views

What is the fiducial argument and why has it not been accepted?

One of the late contributions of R.A. Fisher was fiducial intervals and fiducial principled arguments. This approach however is nowhere near as popular as frequentist or Bayesian principled arguments. ...
31
votes
3answers
2k views

Why does basic hypothesis testing focus on the mean and not on the median?

In basic under-grad statistics courses, students are (usually?) taught hypothesis testing for the mean of a population. Why is it that the focus is on the mean and not on the median? My guess is that ...
31
votes
2answers
1k views

Performing a statistical test after visualizing data - data dredging?

I'll propose this question by means of an example. Suppose I have a data set, such as the boston housing price data set, in which I have continuous and categorical variables. Here, we have a "quality"...
30
votes
7answers
18k views

Inference vs. estimation?

What are the differences between "inference" and "estimation" under the context of machine learning? As a newbie, I feel that we infer random variables and estimate the model parameters. Is my this ...
30
votes
3answers
865 views

Accommodating entrenched views of p-values

Sometimes in reports I include a disclaimer about the p-values and other inferential statistics I've provided. I say that since the sample wasn't random, then such statistics would not strictly apply....
27
votes
6answers
10k views

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

I just browsed through this wonderful book: Applied multivariate statistical analysis by Johnson and Wichern. The irony is, I am still not able to understand the motivation for using multivariate (...
27
votes
3answers
7k views

What if your random sample is clearly not representative?

What if you take a random sample and you can see it is clearly not representative, as in a recent question. For example, what if the population distribution is supposed to be symmetric around 0 and ...
26
votes
2answers
4k views

Should we address multiple comparisons adjustments when using confidence intervals?

Suppose we have a multiple comparisons scenario such as post hoc inference on pairwise statistics, or like a multiple regression, where we are making a total of $m$ comparisons. Suppose also, that we ...
24
votes
4answers
8k views

Intractable posterior distributions

In Bayesian statistics, it is often mentioned that the posterior distribution is intractable and thus approximate inference must be applied. What are the factors that cause this intractability?
23
votes
3answers
1k views

Kullback-Leibler divergence WITHOUT information theory

After much trawling of Cross Validated, I still don't feel like I'm any closer to understanding KL divergence outside of the realm of information theory. It's rather odd as somebody with a Math ...
22
votes
2answers
921 views

What does “fiducial” mean (in the context of statistics)?

When I Google for "fisher" "fiducial" ...I sure get a lot of hits, but all the ones I've followed are utterly beyond my comprehension. All these hits do seem to ...
22
votes
1answer
4k views

How many times must I roll a die to confidently assess its fairness?

(Apologies in advance for use of lay language rather than statistical language.) If I want to measure the odds of rolling each side of a specific physical six-sided die to within about +/- 2% with a ...
21
votes
3answers
4k views

Comparison between MaxEnt, ML, Bayes and other kind of statistical inference methods

I'm in no way a statistician (I've had a course in mathematical statistics but nothing more than that), and recently, while studying information theory and statistical mechanics, I met this thing ...
21
votes
6answers
163k views

What is the difference between descriptive and inferential statistics?

My understanding was that descriptive statistics quantitatively described features of a data sample, while inferential statistics made inferences about the populations from which samples were drawn. ...
21
votes
2answers
2k views

What non-Bayesian methods are there for predictive inference?

In Bayesian inference a predictive distribution for future data is derived by integrating out unknown parameters; integrating over the posterior distribution of those parameters gives a posterior ...
21
votes
3answers
6k views

Neyman-Pearson lemma

I have read the Neyman–Pearson lemma from the book Introduction to the Theory of Statistics by Mood, Graybill and Boes. But I have not understood the lemma. Can anyone please explain the lemma to ...
20
votes
2answers
31k views

How does the inverse transform method work?

How does the inversion method work? Say I have a random sample $X_1,X_2,...,X_n$ with density $f(x;\theta)={1\over \theta} x^{(1-\theta)\over \theta}$ over $0<x<1$ and therefore with cdf $F_X(x)=...
19
votes
2answers
2k views

Elastic/ridge/lasso analysis, what then?

I'm getting really interested in the elastic net procedure for predictor shrinkage/selection. It seems very powerful. But from the scientific point of view I don't know well what to do once I got the ...
19
votes
2answers
2k views

If the likelihood principle clashes with frequentist probability then do we discard one of them?

In a comment recently posted here one commenter pointed to a blog by Larry Wasserman who points out (without any sources) that frequentist inference clashes with the likelihood principle. The ...
19
votes
2answers
83k views

How to derive the standard error of linear regression coefficient

For this univariate linear regression model $$y_i = \beta_0 + \beta_1x_i+\epsilon_i$$ given data set $D=\{(x_1,y_1),...,(x_n,y_n)\}$, the coefficient estimates are $$\hat\beta_1=\frac{\sum_ix_iy_i-n\...
18
votes
2answers
5k views

Why is the Fisher Information matrix positive semidefinite?

Let $\theta \in R^{n}$. The Fisher Information Matrix is defined as: $$I(\theta)_{i,j} = -E\left[\frac{\partial^{2} \log(f(X|\theta))}{\partial \theta_{i} \partial \theta_{j}}\bigg|\theta\right]$$ ...
17
votes
3answers
20k views

Do the pdf and the pmf and the cdf contain the same information?

Do the pdf and the pmf and the cdf contain the same information? For me the pdf gives the whole probability to a certain point(basically the area under the probability). The pmf give the probability ...
17
votes
3answers
1k views

What is the difference between a “statistical experiment” and a “statistical model”?

I am following A.W. van der Vaart, asymptotic statistics (1998). He talks of statistical experiments, claiming that they are different from a statistical model, but he defines neither. My question: ...
17
votes
3answers
11k views

Do descriptive statistics have p-values?

I'm being asked to find the p-values for descriptive statistics. However, it's my understanding that p-values are for test statistics. If I'm not mistaken, a p-value is the probability of observing a ...
16
votes
3answers
1k views

Using regularization when doing statistical inference

I know about the benefits of regularization when building predictive models (bias vs. variance, preventing overfitting). But, I'm wondering if it is a good idea to also do regularization (lasso, ...
16
votes
4answers
641 views

War stories where wrong decisions were made based on statistical information?

I think it is fair to say statistics is an applied science so when averages and standard deviations are calculated it is because someone is looking to make some decisions based on those numbers. ...
15
votes
2answers
4k views

Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

My understanding is that when using a Bayesian approach to estimate parameter values: The posterior distribution is the combination of the prior distribution and the likelihood distribution. We ...
15
votes
2answers
677 views

When to stop refining a model?

I have been studying statistics from many books for the last 3 years, and thanks to this site I learned a lot. Nevertheless one fundamental question still remains unanswered for me. It may have a very ...
15
votes
2answers
3k views

Why is the posterior distribution in Bayesian Inference often intractable?

I have a problem understanding why Bayesian Inference leads to intractable problems. The problem is often explained like this: What I don't understand is why this integral has to be evaluated in the ...
15
votes
2answers
653 views

Are we frequentists really just implicit/unwitting Bayesians?

For a given inference problem, we know that a Bayesian approach usually differ in both form and results from a fequentist approach. Frequentists (usually includes me) often point out that their ...
15
votes
1answer
19k views

What are “coefficients of linear discriminants” in LDA?

In R, I use lda function from library MASS to do classification. As I understand LDA, input $...
14
votes
5answers
2k views

What does it mean for a linear regression to be statistically significant but has very low r squared?

I understand it to mean that the model is bad at predicting individual data points but has established a firm trend (e.g. y goes up when x goes up).
14
votes
2answers
455 views

Are sampling distributions legitimate for inference?

Some Bayesians attack frequentist inference stating that "there is no unique sampling distribution" because it depends on the intentions of the researcher (Kruschke, Aguinis, & Joo, 2012, p. 733). ...
14
votes
2answers
2k views

Optimal software package for bayesian analysis

I was wondering which software statistical package do you guys recommend for performing Bayesian Inference. For example, I know that you can run openBUGS or winBUGS as standalones or you can also ...
14
votes
3answers
12k views

How would you do Bayesian ANOVA and regression in R? [closed]

I have a fairly simple dataset consisting of one independent variable, one dependent variable, and a categorical variable. I have plenty of experience running frequentist tests like ...
14
votes
1answer
192 views

Minimizing bias in explanatory modeling, why? (Galit Shmueli's “To Explain or to Predict”)

This question references Galit Shmueli's paper "To Explain or to Predict". Specifically, in section 1.5, "Explaining and Prediction are Different", Professor Shmueli writes: In explanatory ...
13
votes
3answers
1k views

Why trace of $I−X(X′X)^{-1}X′$ is $n-p$ in least square regression when the parameter vector $\beta$ is of p dimensions?

In the model ${y} = X \beta + \epsilon$, we could estimate $\beta$ using the normal equation: $$\hat{\beta} = (X'X)^{-1}X'y,$$ and we could get $$\hat{y} = X \hat{\beta}.$$ The vector of residuals ...
13
votes
4answers
2k views

Inference for the skeptical (but not math-averse) reader

I just watched a lecture on statistical inference ("comparing proportions and means"), part of an intro to stats online course. The material made as little sense to me as it always does (by now I ...
13
votes
2answers
2k views

Example of an inconsistent Maximum likelihood estimator

I'm reading a comment to a paper, and the author states that sometimes, even though the estimators (found by ML or maximum quasilikelihood) may not be consistent, the power of a likelihood ratio or ...
13
votes
2answers
1k views

Does “Inference” include estimation or only testing?

Does the term "statistical inference" include only hypothesis testing or does it also include point estimation, interval estimation etc. Authoritative references will be greatly appreciated.
13
votes
3answers
2k views

Good summaries (reviews, books) on various applications of Markov chain Monte Carlo (MCMC)?

Are there any good summaries (reviews, books) on various applications of Markov chain Monte Carlo (MCMC)? I've seen Markov Chain Monte Carlo in Practice, but this books seems a bit old. Are there ...
13
votes
2answers
3k views

Online resources for learning statistics, exercises (with solutions)?

I'm currently working as a teaching assistant at my university, in an introductory statistics course (for medical students). Offline, there are many books available with information to aid the ...