Questions tagged [inference]

Drawing conclusions about population parameters from sample data. See https://en.wikipedia.org/wiki/Inference and https://en.wikipedia.org/wiki/Statistical_inference

Filter by
Sorted by
Tagged with
224 votes
4 answers
418k views

How to interpret a QQ plot

I am working with a small dataset (21 observations) and have the following normal QQ plot in R: Seeing that the plot does not support normality, what could I infer about the underlying distribution? ...
user avatar
  • 18.7k
25 votes
2 answers
58k views

How does the inverse transform method work?

How does the inversion method work? Say I have a random sample $X_1,X_2,...,X_n$ with density $f(x;\theta)={1\over \theta} x^{(1-\theta)\over \theta}$ over $0<x<1$ and therefore with cdf $F_X(x)=...
user avatar
  • 1,045
42 votes
3 answers
5k views

How do DAGs help to reduce bias in causal inference?

I have read in several places that the use of DAGs can help to reduce bias due to Confounding Differential Selection Mediation Conditioning on a collider I also see the term “backdoor path” a lot. ...
user avatar
  • 1,750
110 votes
7 answers
176k views

T-test for non normal when N>50?

Long ago I learnt that normal distribution was necessary to use a two sample T-test. Today a colleague told me that she learnt that for N>50 normal distribution was not necessary. Is that true? If ...
user avatar
  • 2,197
16 votes
2 answers
5k views

What is the difference between conditioning on regressors vs. treating them as fixed?

Sometimes we assume that regressors are fixed, i.e. they are non-stochastic. I think that means all our predictors, parameter estimates etc. are unconditional then, right? Might I even go so far that ...
user avatar
  • 917
27 votes
2 answers
13k views

Why is it necessary to sample from the posterior distribution if we already KNOW the posterior distribution?

My understanding is that when using a Bayesian approach to estimate parameter values: The posterior distribution is the combination of the prior distribution and the likelihood distribution. We ...
user avatar
  • 1,661
64 votes
3 answers
101k views

Testing equality of coefficients from two different regressions

This seems to be a basic issue, but I just realized that I actually don't know how to test equality of coefficients from two different regressions. Can anyone shed some light on this? More formally, ...
user avatar
  • 1,746
10 votes
2 answers
2k views

Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
user avatar
  • 183
46 votes
6 answers
25k views

Why do we need multivariate regression (as opposed to a bunch of univariate regressions)?

I just browsed through this wonderful book: Applied multivariate statistical analysis by Johnson and Wichern. The irony is, I am still not able to understand the motivation for using multivariate (...
user avatar
  • 1,096
50 votes
7 answers
7k views

Why would someone use a Bayesian approach with a 'noninformative' improper prior instead of the classical approach?

If the interest is merely estimating the parameters of a model (pointwise and/or interval estimation) and the prior information is not reliable, weak, (I know this is a bit vague but I am trying to ...
user avatar
7 votes
1 answer
5k views

How to infer correlations from correlations

I have a question regarding correlation inference. Consider, I have two sets of variables X and Y. For an x element of X I know the correlation to an unknown variable z. I also have the covariance ...
user avatar
  • 71
67 votes
32 answers
4k views

What are the worst (commonly adopted) ideas/principles in statistics?

In my statistical teaching, I encounter some stubborn ideas/principles relating to statistics that have become popularised, yet seem to me to be misleading, or in some cases utterly without merit. I ...
7 votes
7 answers
3k views

Is there a GLM bible?

Is there consensus in the field of statistics that one book is the absolute best source and completely covering every aspect of GLM - detailing everything from estimation to inference?
58 votes
6 answers
61k views

Rule of thumb for number of bootstrap samples

I wonder if someone knows any general rules of thumb regarding the number of bootstrap samples one should use, based on characteristics of the data (number of observations, etc.) and/or the variables ...
user avatar
  • 971
34 votes
2 answers
177k views

How to derive the standard error of linear regression coefficient

For this univariate linear regression model $$y_i = \beta_0 + \beta_1x_i+\epsilon_i$$ given data set $D=\{(x_1,y_1),...,(x_n,y_n)\}$, the coefficient estimates are $$\hat\beta_1=\frac{\sum_ix_iy_i-n\...
user avatar
  • 3,145
95 votes
12 answers
11k views

Who Are The Bayesians?

As one becomes interested in statistics, the dichotomy "Frequentist" vs. "Bayesian" soon becomes commonplace (and who hasn't read Nate Silver's The Signal and the Noise, anyway?). In talks and ...
user avatar
32 votes
3 answers
979 views

Accommodating entrenched views of p-values

Sometimes in reports I include a disclaimer about the p-values and other inferential statistics I've provided. I say that since the sample wasn't random, then such statistics would not strictly apply....
user avatar
  • 11.7k
14 votes
2 answers
10k views

What is probabilistic inference?

I am reading Chris Bishop's Pattern Recognition and Machine Learning textbook. I came across the term probabilistic inference several times. I have a couple of questions. Is probabilistic inference ...
user avatar
100 votes
9 answers
69k views

Understanding "variance" intuitively

What is the cleanest, easiest way to explain someone the concept of variance? What does it intuitively mean? If one is to explain this to their child how would one go about it? It's a concept that I ...
user avatar
  • 13.7k
26 votes
4 answers
3k views

Kullback-Leibler divergence WITHOUT information theory

After much trawling of Cross Validated, I still don't feel like I'm any closer to understanding KL divergence outside of the realm of information theory. It's rather odd as somebody with a Math ...
user avatar
  • 1,944
8 votes
2 answers
384 views

Effects of model selection and misspecification testing on inference: Probabilistic Reduction approach (Aris Spanos)

This question is about pre-test bias, inference after model selection and data snooping within the Probabilistic Reduction (PR) methodology by Aris Spanos (which is related to the Error Statistics ...
user avatar
6 votes
1 answer
5k views

Sufficient statistics for Uniform $(-\theta,\theta)$

So, I know that $\max(-X_{(1)},X_{(n)})$ is a sufficient statistic for the parameter $\theta$. But can I also say that $(X_{(1)},X_{(n)})$ are jointly sufficient for the parameter $\theta$ ? In other ...
user avatar
  • 187
25 votes
3 answers
10k views

Inference after using Lasso for variable selection

I'm using Lasso for feature selection in a relatively low dimensional setting (n >> p). After fitting a Lasso model, I want to use the covariates with nonzero coefficients to fit a model with no ...
user avatar
  • 2,089
10 votes
1 answer
610 views

When will a less true model predict better than a truer model?

In "To Explain or to Predict?", Pr. Galit Shmueli said that sometimes a less true model can predict better than a truer model. Why is it so? When will it happen? How does it happen? Is ...
user avatar
  • 1,366
9 votes
2 answers
3k views

Should I use a machine learning model to calculate propensity score?

In my study, running a simple linear model to calculate de propensity score for each example seemed to not be able to model my treatment choosing process correctly. My question is, does it make sense ...
user avatar
  • 331
20 votes
2 answers
533 views

Minimizing bias in explanatory modeling, why? (Galit Shmueli's "To Explain or to Predict")

This question references Galit Shmueli's paper "To Explain or to Predict". Specifically, in section 1.5, "Explaining and Prediction are Different", Professor Shmueli writes: In explanatory ...
user avatar
9 votes
2 answers
446 views

Statistical inference under model misspecification

I have a general methodological question. It might have been answered before, but I am not able to locate the relevant thread. I will appreciate pointers to possible duplicates. (Here is an excellent ...
user avatar
7 votes
3 answers
4k views

Finding minimum/maximum peaks in a n-modal distribution

I have distributions that show n-modal behavior. I need to find the values of the largest and smallest modes. For example, in the histogram below I need to find the values representing the yellow ...
user avatar
  • 173
7 votes
2 answers
2k views

Comparing estimators of location of the Cauchy distribution

I'm comparing the following 4 estimators of location of the Cauchy distribution: Let $x_{1},..x_{n}$ be observations and $l$ be the log likelihood function. $x=median(x_{1},..x_{n})$, $y=x+\frac{l'(x)...
user avatar
23 votes
3 answers
4k views

Using regularization when doing statistical inference

I know about the benefits of regularization when building predictive models (bias vs. variance, preventing overfitting). But, I'm wondering if it is a good idea to also do regularization (lasso, ...
user avatar
23 votes
2 answers
3k views

What non-Bayesian methods are there for predictive inference?

In Bayesian inference a predictive distribution for future data is derived by integrating out unknown parameters; integrating over the posterior distribution of those parameters gives a posterior ...
user avatar
13 votes
4 answers
11k views

Can a trend stationary series be modeled with ARIMA?

I have a question / confusion about stationary series required for modeling with ARIMA(X). I am thinking of this more in terms of inference (effect of an intervention), but would like to know if ...
user avatar
  • 7,770
5 votes
1 answer
205 views

Motivations for experiment design in statistical learning?

My interests in statistics centre around statistical learning, including Bayesian inference, inference in combinatorial spaces, Monte Carlo methods, Markov decision processes, modeling stochastic ...
user avatar
  • 2,101
16 votes
2 answers
17k views

Formula for 95% confidence interval for $R^2$

I googled and searched on stats.stackexchange but I cannot find the formula to calculate a 95% confidence interval for an $R^2$ value for a linear regression. Can anyone provide it? Even better, let'...
user avatar
  • 12.7k
12 votes
1 answer
479 views

Should degrees of freedom corrections be used for inference on GLM parameters?

This question is inspired by Martijn's answer here. Suppose we fit a GLM for a one parameter family like a binomial or Poisson model and that it is a full likelihood procedure (as opposed to say, ...
user avatar
  • 53.6k
1 vote
2 answers
552 views

Regression's population parameters

Suppose I've specified a linear regression model: $$ Y = \beta_0 + \beta_1 X + \epsilon $$ where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters ...
user avatar
37 votes
2 answers
7k views

Should we address multiple comparisons adjustments when using confidence intervals?

Suppose we have a multiple comparisons scenario such as post hoc inference on pairwise statistics, or like a multiple regression, where we are making a total of $m$ comparisons. Suppose also, that we ...
user avatar
  • 26.7k
27 votes
2 answers
8k views

Why is the Fisher Information matrix positive semidefinite?

Let $\theta \in R^{n}$. The Fisher Information Matrix is defined as: $$I(\theta)_{i,j} = -E\left[\frac{\partial^{2} \log(f(X|\theta))}{\partial \theta_{i} \partial \theta_{j}}\bigg|\theta\right]$$ ...
user avatar
  • 373
20 votes
3 answers
27k views

Do the pdf and the pmf and the cdf contain the same information?

Do the pdf and the pmf and the cdf contain the same information? For me the pdf gives the whole probability to a certain point(basically the area under the probability). The pmf give the probability ...
user avatar
  • 675
19 votes
3 answers
33k views

What are "coefficients of linear discriminants" in LDA?

In R, I use lda function from library MASS to do classification. As I understand LDA, input $...
user avatar
  • 3,145
13 votes
2 answers
11k views

Interpret Regression Coefficients After various Differencing

There are few explanations I can find that describe how to interpret linear regression coefficients after differencing a time series (to eliminate a unit root). Is it just so simple that there is no ...
user avatar
  • 7,770
14 votes
1 answer
2k views

Checking if a coin is fair based on how often a subsequence occurs

Results of 100 coin toss experiments are recorded as 0 for "Tails" and 1 for "Heads". The output $x$ is a string of zeros and ones of length 100. The number of times the sequence 1-...
user avatar
  • 205
5 votes
1 answer
346 views

Definition of Statistic

I keep seeing conflicting definitions of a statistic. Is a statistic a random variable such that it is a function of the random variables of a random sample? Or is it the value of the function of the ...
user avatar
5 votes
1 answer
8k views

Confidence interval for mean of a uniform distribution

I've been trying to compute a 95% confidence interval for the mean of a height sample, which is uniformly distributed. I have calculated the following sample statistics: $$n=10 \quad \quad \bar{x} = ...
user avatar
6 votes
3 answers
453 views

If we disbelieve $H_0$, why quote a p value calculated assuming $H_0$ was true?

Hypothesis testing seeks to reject a null hypothesis ($H_0$) on the basis of an assumption made about the sample following a certain distribution. This assumption is conditional on $H_0$ being true. ...
user avatar
  • 1,197
4 votes
1 answer
204 views

Resources for when population data is available

There are hundreds if not thousands of textbooks that detail how to make population inferences from sample. However for almost all my applications at work I have the entire population of data for ...
user avatar
  • 409
3 votes
2 answers
274 views

Independence of events in real-life data

Most of statistical methods (if not all) rely on independence of events. How do we know that this assumption is valid in real-life problems like clinical trials or web crawling? What might be the ...
user avatar
  • 161
3 votes
2 answers
870 views

inferring heavy-tail distribution from finite sample of histogram data

I have some data in the form of bins and counts. Here is one complete non-truncated example: ...
user avatar
  • 41
32 votes
3 answers
3k views

Why does basic hypothesis testing focus on the mean and not on the median?

In basic under-grad statistics courses, students are (usually?) taught hypothesis testing for the mean of a population. Why is it that the focus is on the mean and not on the median? My guess is that ...
user avatar
  • 665
33 votes
4 answers
4k views

What is the fiducial argument and why has it not been accepted?

One of the late contributions of R.A. Fisher was fiducial intervals and fiducial principled arguments. This approach however is nowhere near as popular as frequentist or Bayesian principled arguments. ...
user avatar
  • 5,371

1
2 3 4 5
7