Questions tagged [inference]

Drawing conclusions about population parameters from sample data. See https://en.wikipedia.org/wiki/Inference and https://en.wikipedia.org/wiki/Statistical_inference

Filter by
Sorted by
Tagged with
171
votes
4answers
252k views

How to interpret a QQ plot

I am working with a small dataset (21 observations) and have the following normal QQ plot in R: Seeing that the plot does not support normality, what could I infer about the underlying distribution? ...
20
votes
2answers
32k views

How does the inverse transform method work?

How does the inversion method work? Say I have a random sample $X_1,X_2,...,X_n$ with density $f(x;\theta)={1\over \theta} x^{(1-\theta)\over \theta}$ over $0<x<1$ and therefore with cdf $F_X(x)=...
42
votes
3answers
58k views

Testing equality of coefficients from two different regressions

This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this? More formally, ...
9
votes
1answer
1k views

What is the difference between conditioning on regressors vs. treating them as fixed?

Sometimes we assume that regressors are fixed, i.e. they are non-stochastic. I think that means all our predictors, parameter estimates etc. are unconditional then, right? Might I even go so far that ...
5
votes
1answer
3k views

How to infer correlations from correlations

I have a question regarding correlation inference. Consider, I have two sets of variables X and Y. For an x element of X I know the correlation to an unknown variable z. I also have the covariance ...
44
votes
7answers
5k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
7
votes
7answers
2k views

Is there a GLM bible?

Is there consensus in the field of statistics that one book is the absolute best source and completely covering every aspect of GLM - detailing everything from estimation to inference?
92
votes
12answers
10k views

Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
39
votes
6answers
29k views

Rule of thumb for number of bootstrap samples

I wonder if someone knows any general rules of thumb regarding the number of bootstrap samples one should use, based on characteristics of the data (number of observations, etc.) and/or the variables ...
30
votes
3answers
865 views

Accommodating entrenched views of p-values

Sometimes in reports I include a disclaimer about the p-values and other inferential statistics I've provided. I say that since the sample wasn't random, then such statistics would not strictly apply....
27
votes
6answers
10k views

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

I just browsed through this wonderful book: Applied multivariate statistical analysis by Johnson and Wichern. The irony is, I am still not able to understand the motivation for using multivariate (...
7
votes
1answer
5k views

What is probabilistic inference?

I am reading Chris Bishop's Pattern Recognition and Machine Learning textbook. I came across the term probabilistic inference several times. I have a couple of questions. Is probabilistic inference ...
4
votes
1answer
697 views

Sufficient statistics for Uniform $(-\theta,\theta)$

So, I know that $\max(-X_{(1)},X_{(n)})$ is a sufficient statistic for the parameter $\theta$. But can I also say that $(X_{(1)},X_{(n)})$ are jointly sufficient for the parameter $\theta$ ? In other ...
19
votes
2answers
84k views

How to derive the standard error of linear regression coefficient

For this univariate linear regression model $$y_i = \beta_0 + \beta_1x_i+\epsilon_i$$ given data set $D=\{(x_1,y_1),...,(x_n,y_n)\}$, the coefficient estimates are $$\hat\beta_1=\frac{\sum_ix_iy_i-n\...
7
votes
1answer
436 views

When will a less true model predict better than a truer model?

In "To Explain or to Predict?", Pr. Galit Shmueli said that sometimes a less true model can predict better than a truer model. Why is it so? When will it happen? How does it happen? Is explanation a ...
9
votes
2answers
267 views

Statistical inference under model misspecification

I have a general methodological question. It might have been answered before, but I am not able to locate the relevant thread. I will appreciate pointers to possible duplicates. (Here is an excellent ...
80
votes
10answers
58k views

Understanding “variance” intuitively

What is the cleanest, easiest way to explain someone the concept of variance? What does it intuitively mean? If one is to explain this to their child how would one go about it? It's a concept that I ...
12
votes
4answers
8k views

Can a trend stationary series be modeled with ARIMA?

I have a question / confusion about stationary series required for modeling with ARIMA(X). I am thinking of this more in terms of inference (effect of an intervention), but would like to know if ...
11
votes
1answer
177 views

Should degrees of freedom corrections be used for inference on GLM parameters?

This question is inspired by Martijn's answer here. Suppose we fit a GLM for a one parameter family like a binomial or Poisson model and that it is a full likelihood procedure (as opposed to say, ...
22
votes
2answers
2k views

What non-Bayesian methods are there for predictive inference?

In Bayesian inference a predictive distribution for future data is derived by integrating out unknown parameters; integrating over the posterior distribution of those parameters gives a posterior ...
16
votes
3answers
1k views

Using regularization when doing statistical inference

I know about the benefits of regularization when building predictive models (bias vs. variance, preventing overfitting). But, I'm wondering if it is a good idea to also do regularization (lasso, ...
17
votes
2answers
5k views

Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

My understanding is that when using a Bayesian approach to estimate parameter values: The posterior distribution is the combination of the prior distribution and the likelihood distribution. We ...
8
votes
2answers
7k views

Interpret Regression Coefficients After various Differencing

There are few explanations I can find that describe how to interpret linear regression coefficients after differencing a time series (to eliminate a unit root). Is it just so simple that there is no ...
15
votes
1answer
19k views

What are “coefficients of linear discriminants” in LDA?

In R, I use lda function from library MASS to do classification. As I understand LDA, input $...
9
votes
2answers
1k views

Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
5
votes
0answers
190 views

Effects of model selection and misspecification testing on inference: Probabilistic Reduction approach (Aris Spanos)

This question is about pre-test bias, inference after model selection and data snooping within the Probabilistic Reduction (PR) methodology by Aris Spanos (which is related to the Error Statistics ...
12
votes
2answers
7k views

Formula for 95% confidence interval for $R^2$

I googled and searched on stats.stackexchange but I cannot find the formula to calculate a 95% confidence interval for an $R^2$ value for a linear regression. Can anyone provide it? Even better, let'...
6
votes
3answers
419 views

If we disbelieve $H_0$, why quote a p value calculated assuming $H_0$ was true?

Hypothesis testing seeks to reject a null hypothesis ($H_0$) on the basis of an assumption made about the sample following a certain distribution. This assumption is conditional on $H_0$ being true. ...
2
votes
1answer
125 views

Resources for when population data is available

There are hundreds if not thousands of textbooks that detail how to make population inferences from sample. However for almost all my applications at work I have the entire population of data for ...
23
votes
3answers
1k views

Kullback-Leibler divergence WITHOUT information theory

After much trawling of Cross Validated, I still don't feel like I'm any closer to understanding KL divergence outside of the realm of information theory. It's rather odd as somebody with a Math ...
32
votes
3answers
2k views

Why does basic hypothesis testing focus on the mean and not on the median?

In basic under-grad statistics courses, students are (usually?) taught hypothesis testing for the mean of a population. Why is it that the focus is on the mean and not on the median? My guess is that ...
33
votes
4answers
3k views

What is the fiducial argument and why has it not been accepted?

One of the late contributions of R.A. Fisher was fiducial intervals and fiducial principled arguments. This approach however is nowhere near as popular as frequentist or Bayesian principled arguments. ...
19
votes
2answers
2k views

If the likelihood principle clashes with frequentist probability then do we discard one of them?

In a comment recently posted here one commenter pointed to a blog by Larry Wasserman who points out (without any sources) that frequentist inference clashes with the likelihood principle. The ...
18
votes
2answers
5k views

Why is the Fisher Information matrix positive semidefinite?

Let $\theta \in R^{n}$. The Fisher Information Matrix is defined as: $$I(\theta)_{i,j} = -E\left[\frac{\partial^{2} \log(f(X|\theta))}{\partial \theta_{i} \partial \theta_{j}}\bigg|\theta\right]$$ ...
14
votes
2answers
2k views

Optimal software package for bayesian analysis

I was wondering which software statistical package do you guys recommend for performing Bayesian Inference. For example, I know that you can run openBUGS or winBUGS as standalones or you can also ...
28
votes
3answers
7k views

What if your random sample is clearly not representative?

What if you take a random sample and you can see it is clearly not representative, as in a recent question. For example, what if the population distribution is supposed to be symmetric around 0 and ...
12
votes
2answers
601 views

How to define a Rejection Region when there's no UMP?

Consider the linear regression model $\mathbf{y}=\mathbf{X\beta}+\mathbf{u}$, $\mathbf{u}\sim N(\mathbf{0},\sigma^2\mathbf{I})$, $E(\mathbf{u}\mid\mathbf{X})=\mathbf{0}$. Let $H_0: \sigma_0^2=\...
8
votes
2answers
2k views

Is there a test/technique/method for comparing principal components decompositions between samples?

Is there any methodical way to compare the directions, magnitudes, etc of PCA results for different samples drawn from the same population? I'm leaving the nature of the test deliberately vague ...
11
votes
1answer
3k views

Neg Binomial and the Jeffreys' Prior

I'm trying to obtain the Jeffreys' prior for a negative binomial distribution. I can't see where I go wrong, so if someone could help point that out that would be appreciated. Okay, so the situation ...
7
votes
2answers
3k views

AR(q) model with F-test

I am wondering that if we have an AR($q$) model for time series: $$X_i=\beta_1X_{i-1}+..+\beta_{p}X_{i-p} + \beta_{p+1} X_{i-p-1} +...+\beta_{q} X_{i-q}+\epsilon_i,\epsilon_i \;\text{iid}\; N(0,\...
7
votes
1answer
454 views

Estimating Size of a Set based on two Overlapping Subsets

I've searched everywhere for a similar question and many things come close but are not the same. I'm looking for a way to estimate the size of a set if two partially overlapping subsets are known (...
7
votes
2answers
1k views

Comparing estimators of location of the Cauchy distribution

I'm comparing the following 4 estimators of location of the Cauchy distribution: Let $x_{1},..x_{n}$ be observations and $l$ be the log likelihood function. $x=median(x_{1},..x_{n})$, $y=x+\frac{l'(x)...
5
votes
2answers
128 views

Explanatory variables may bias predictions

I' m asking this question out of sheer curiosity, my teacher was not able to explain it. If I'm using logistic regression with categorical variables they are coded like {1,2,3}. I guess it wouldn't ...
1
vote
1answer
7k views

Probability and Sampling distribution

Would you please explain me the difference between Probability distribution and Sampling distribution easily ? Is that the difference : in probability distribution we have probability for every ...
64
votes
8answers
6k views

What is a good, convincing example in which p-values are useful?

My question in the title is self explanatory, but I would like to give it some context. The ASA released a statement earlier this week “on p-values: context, process, and purpose”, outlining various ...
15
votes
2answers
657 views

Are we frequentists really just implicit/unwitting Bayesians?

For a given inference problem, we know that a Bayesian approach usually differ in both form and results from a fequentist approach. Frequentists (usually includes me) often point out that their ...
17
votes
3answers
20k views

Do the pdf and the pmf and the cdf contain the same information?

Do the pdf and the pmf and the cdf contain the same information? For me the pdf gives the whole probability to a certain point(basically the area under the probability). The pmf give the probability ...
13
votes
3answers
1k views

Why trace of $I−X(X′X)^{-1}X′$ is $n-p$ in least square regression when the parameter vector $\beta$ is of p dimensions?

In the model ${y} = X \beta + \epsilon$, we could estimate $\beta$ using the normal equation: $$\hat{\beta} = (X'X)^{-1}X'y,$$ and we could get $$\hat{y} = X \hat{\beta}.$$ The vector of residuals ...
15
votes
2answers
3k views

Why is the posterior distribution in Bayesian Inference often intractable?

I have a problem understanding why Bayesian Inference leads to intractable problems. The problem is often explained like this: What I don't understand is why this integral has to be evaluated in the ...
13
votes
3answers
2k views

Good summaries (reviews, books) on various applications of Markov chain Monte Carlo (MCMC)?

Are there any good summaries (reviews, books) on various applications of Markov chain Monte Carlo (MCMC)? I've seen Markov Chain Monte Carlo in Practice, but this books seems a bit old. Are there ...