Questions tagged [inference]

Drawing conclusions about population parameters from sample data. See https://en.wikipedia.org/wiki/Inference and https://en.wikipedia.org/wiki/Statistical_inference

Filter by
Sorted by
Tagged with
9
votes
1answer
539 views

Is it acceptable to run two linear models on the same data set?

For a linear regression with multiple groups (natural groups defined a priori) is it acceptable to run two different models on the same data set to answer the following two questions? Does each group ...
9
votes
2answers
212 views

Reverse birthday problem with multiple collisions

Assume you had an alien year with an unknown length N. If you have a random sample of said aliens and some of them share birthdays, can you use this data to estimate the length of the year? For ...
9
votes
3answers
168 views

Measuring some of the patients more than once

I'm conducting a clinical study where I determine an anthropometrical measure of the patients. I know how to handle the situation where I have one measure per patient: I make a model, where I have a ...
8
votes
2answers
500 views

How is prior knowledge possible under a purely Bayesian framework?

This is more of a philosophical question, but from a purely Bayesian standpoint how does one actually form prior knowledge? If we need prior information to carry out valid inferences then there seems ...
8
votes
3answers
3k views

Mixture Models and Dirichlet Process Mixtures (beginner lectures or papers)

In the context of online clustering, I often find many papers talking about: "dirichlet process" and "finite/infinite mixture models". Given that I've never used or read about dirichlet process or ...
8
votes
1answer
960 views

Understanding the Behrens–Fisher problem

This section of this article says: Ronald Fisher in 1935 introduced fiducial inference in order to apply it to this problem. He referred to an earlier paper by W. V. Behrens from 1929. Behrens and ...
8
votes
2answers
7k views

Interpret Regression Coefficients After various Differencing

There are few explanations I can find that describe how to interpret linear regression coefficients after differencing a time series (to eliminate a unit root). Is it just so simple that there is no ...
8
votes
2answers
2k views

Is there a test/technique/method for comparing principal components decompositions between samples?

Is there any methodical way to compare the directions, magnitudes, etc of PCA results for different samples drawn from the same population? I'm leaving the nature of the test deliberately vague ...
8
votes
1answer
2k views

Complete sufficient statistic

I've recently started studying statistical inference. I've been working through various problems and this one has me completely stumped. Let $X_1,\dots,X_n$ be a random sample from a discrete ...
8
votes
2answers
120 views

What are the recent works and research scope in asymptotic inference (large sample theory)?

What are some current significant theoretical work that has been done in the field of asymptotic inference / large sample theory? What is the research scope in this field right now? Is there any open ...
8
votes
2answers
4k views

Complete statistic for $\sigma^2$ in a $N(\mu,\sigma^2)$

I would like to know if the statistic $$T(X_1,\ldots,X_n)=\frac{\sum_{i=1}^n (X_i-\bar{X}_n)^2}{n-1}$$ is complete for $\sigma^2$ in a $N(\mu,\sigma^2)$ setting. Does this depend on whether $\mu$ is ...
8
votes
1answer
598 views

Choosing non-informative priors

I am working on a model relying on an ugly parametrized function acting as a calibration function on a part of the model. Using a Bayesian setting, I need to get non-informative priors for the ...
8
votes
3answers
993 views

Bayesian inference on a sum of iid real-valued random variables

Let $X_1$, $X_2$, ..., $X_n$ be iid RV's with range $[0,1]$ but unknown distribution. (I'm OK with assuming that the distribution is continuous, etc., if necessary.) Define $S_n = X_1 + \cdots + X_n$...
8
votes
1answer
923 views

Why is the marginal likelihood difficult/intractable to estimate?

I have a generally basic question to ask here that has been troubling to me for a while. Through most of my reading of bayesian statistics, it stated matter-of-factedly that the marginal likelihood is ...
8
votes
1answer
826 views

What is probabilistic programming?

For the past year, I've been hearing a lot about Probabilistic Programming (PP) frameworks like PyMC3 and Stan, and how great PP is. And today, someone shared this link with me: Pyro: a Deep ...
8
votes
1answer
953 views

What is the difference using a Fisher's Exact Test vs. a Logistic Regression for $2 \times 2$ tables?

For a $2 \times 2$ table, two ways to do inference on the table is through Fisher's Exact Test and also a Logistic Regression. I was told that using a Fisher's Exact Test, we are only interested in ...
7
votes
2answers
539 views

Does the posterior necessarily follow the same conditional dependence structure as the prior?

One of the assumptions in a model is the conditional dependence between random variables in the joint prior distribution. Consider the following model, $$p(a,b|X) \propto p(X|a,b)p(a,b)$$ Now suppose ...
7
votes
2answers
1k views

Is confirmatory vs exploratory statistics “induction vs deduction”?

This webpage says: Inferential Statistics - Deductive Approach Descriptive Statistics - Inductive Approach But I doubt it. If I understand correctly, Inferential Statistics is "given some ...
7
votes
7answers
2k views

Is there a GLM bible?

Is there consensus in the field of statistics that one book is the absolute best source and completely covering every aspect of GLM - detailing everything from estimation to inference?
7
votes
2answers
1k views

If I prove the estimator of $\theta^2$ is unbiased, does that prove that the estimator of parameter $\theta$ is unbiased?

Let $X_i$ be an iid random variable having pdf $f(\mathbf{x}|\theta)$, where $E(X_i) = 6\theta^2$, and $\theta > 0$. I have calculated an estimator for the parameter ($\theta$) of $f(\mathbf{x}|\...
7
votes
2answers
3k views

AR(q) model with F-test

I am wondering that if we have an AR($q$) model for time series: $$X_i=\beta_1X_{i-1}+..+\beta_{p}X_{i-p} + \beta_{p+1} X_{i-p-1} +...+\beta_{q} X_{i-q}+\epsilon_i,\epsilon_i \;\text{iid}\; N(0,\...
7
votes
1answer
438 views

When will a less true model predict better than a truer model?

In "To Explain or to Predict?", Pr. Galit Shmueli said that sometimes a less true model can predict better than a truer model. Why is it so? When will it happen? How does it happen? Is explanation a ...
7
votes
3answers
3k views

What does a 'tractable' distribution mean?

For example, in generative adversarial network, we often hear that inference is easy because the conditional distribution of x given latent variable z is 'tractable'. Also, I read somewhere that ...
7
votes
5answers
3k views

Likelihood Ratio for the Bivariate Normal distribution

For a random sample from a Bivariate Normal distribution with $\rho=\frac{1}{2}$ and equal variances, i.e. $\sigma^2_x=\sigma^2_y=\sigma^2$, I would like to derive the Likelihood Ratio Test for the ...
7
votes
2answers
1k views

Comparing estimators of location of the Cauchy distribution

I'm comparing the following 4 estimators of location of the Cauchy distribution: Let $x_{1},..x_{n}$ be observations and $l$ be the log likelihood function. $x=median(x_{1},..x_{n})$, $y=x+\frac{l'(x)...
7
votes
2answers
242 views

On the hardness of data to learn

Almost in all texts which are discussing theorems of statistical learning, they assume analyzing arbitrary unknown distribution (the worst case). But in practice different problems (different data) ...
7
votes
1answer
5k views

What is probabilistic inference?

I am reading Chris Bishop's Pattern Recognition and Machine Learning textbook. I came across the term probabilistic inference several times. I have a couple of questions. Is probabilistic inference ...
7
votes
1answer
461 views

Estimating Size of a Set based on two Overlapping Subsets

I've searched everywhere for a similar question and many things come close but are not the same. I'm looking for a way to estimate the size of a set if two partially overlapping subsets are known (...
7
votes
2answers
1k views

Maximum likelihood estimator for $\theta$ and $E[X]$

Let $X_1,..., X_n $ be a random sample of a variable with PDF: $$f(x|\theta)=\frac{\theta}{x^2} I_{(\theta, \infty)}(x), \theta >0$$ Find the maximum likelihood estimator for $\theta$ and $ E[X]$ ...
7
votes
1answer
2k views

What is a predictive distribution? [duplicate]

I understand that we calculate a posterior belief by updating our prior belief with the information from given data, like $$ p(\theta|y_1,...,y_n)\propto p(y_1,...,y_n|\theta)p(\theta)$$ But I dont ...
7
votes
4answers
946 views

$\chi^2$ tabulated value

I noticed that the critical $\chi^2$ value increases as the degrees of freedom increase in a $\chi^2$ table. Why is that?
7
votes
1answer
1k views

Fisher information for $\rho$ in a bivariate normal distribution

I have seen many times people using the Delta method in order to find the asymptotic distribution of $r$, the sample correlation coefficient, for bivariate normal data. This distribution is given by $...
7
votes
2answers
175 views

If a random sample all came out positive, what can be inferred about the population? [duplicate]

Specifically, let's say I take a random sample of 20 products from a manufacturing batch of 1000 and they all tested good, what assumptions and conclusions can I make about the whole batch? Is it ...
7
votes
1answer
81 views

Is inference based on full (global) regression model appropriate?

Is inference based on a full model appropriate, and if so, in which circumstances? Suppose you are interested in the potential relationship between a response variable and several candidate predictor ...
7
votes
1answer
926 views

How much can the “pyramid of evidence” be applied to economics and political sciences?

When trying to assess a validity of a claim relying on statistics, I was taught (in the school of epidemiology) that the scale to use is “the pyramid of evidence“ However, when conducting a ...
7
votes
1answer
392 views

Sufficient Statistic for non-exponential family distribution

Question: Let $X_1,X_2,\ldots,X_n$ be an iid sample from $N(\theta , 4 \theta^2 )$. I want to show that this model is not a member of the exponential family and to find a sufficient statistic for $\...
7
votes
2answers
2k views

Bayesian inferencing: how iterative parameter updates work?

I have been struggling with this for a while. A typical optimisation problem can be viewed as optimising some cost function which is a combination of a data term and a penalty term which encourages ...
7
votes
1answer
660 views

Conditioning on independent random variables

I am in a situation where I have to compute: $$E(u(x_1)|\bar{X},S^2)$$ where $X_1$ is a normally distributed random variable and $u(.)$ some function. I know that by the student's theorem the sample ...
7
votes
1answer
154 views

Does taking the logs of the dependent and/or independent variable affect the model errors and thus the validity of inference?

I often see people (statisticians and practitioners) transforming variables without a second thought. I've always been scared of transformations, because I worry they could modify the error ...
7
votes
3answers
492 views

What is the role of feature engineering in statistical inference?

This may be a dumb question. I'm a recent college grad who is working in the area of predictive modeling and finding that there is a heavy emphasis on performing feature engineering. However, in most ...
7
votes
1answer
208 views

Bayesian vs. Frequentist calculation steps

This article contains an example of Bayesian vs. Frequentist philosophies. An old drug successfully treats 70% of patients. To test a new drug, researchers give it to 100 patients, 83 of whom ...
7
votes
1answer
1k views

What can be concluded from small sample size?

I had just conducted a test where the customer wants to see a 20% improvement in perforation length from a baseline perforator. The baseline perforator has not been tested in the customer's specific ...
7
votes
1answer
2k views

What's the problem with model identifiability?

I understand that in a decision perspective, identifiability of a model is needed to ensure the convergence (with increasing number of observations) of the parameters to estimate through a single ...
7
votes
0answers
270 views

Dealing with dependent data in a Bayesian model

Background: Consider a series of dependent data points, $$ y_1,y_2,y_3,\cdots,y_N. $$ In cases where the dependence is well described by an exponentially decaying auto-correlation function, it is ...
6
votes
3answers
420 views

If we disbelieve $H_0$, why quote a p value calculated assuming $H_0$ was true?

Hypothesis testing seeks to reject a null hypothesis ($H_0$) on the basis of an assumption made about the sample following a certain distribution. This assumption is conditional on $H_0$ being true. ...
6
votes
3answers
907 views

What to conclude when most results are statistically significant to fail to reject null hypothesis but not all?

I have sampled 8 bags of a certain brand of candy to compare the color distributions of the candies. I have 4 bags for each size of bag, 8 oz and 1.9 lb. The bags were paired randomly. Here are my ...
6
votes
2answers
214 views

Inference with Gaussian Random Variable

Let $X = N(0,\frac{1}{\alpha})$, $Y = 2X + 8 + N_{y}$, and $N_{y}$ be a noise $N_{y} = N(0,1)$. Then, $P(y|x) = \frac{1}{\sqrt{2\pi}}exp\{ -\frac{1}{2}(y - 2x - 8)^{2} \}$ and $P(x) = \sqrt{\frac{\...
6
votes
2answers
817 views

Differences between prior distribution and prior predictive distribution?

While studying Bayesian statistics, somehow I am facing a problem to understand the differences between prior distribution and prior predictive distribution. Prior distribution is sort of fine to ...
6
votes
1answer
2k views

Expectation of an estimator?

When evaluating an estimator in a frequentist setting, using MSE and let say to compute the Bias of the estimator we compute the expectation of this estimator, are we supposing that the estimator has ...
6
votes
1answer
132 views

Is the sample quantile unbiased for the true quantile?

I would like to find a way to show whether the sample quantile is an unbiased estimator of the true quantiles. Let $F$ be strictly increasing with density function $f$. I will define the $p$-th ...