Questions tagged [inference]
Drawing conclusions about population parameters from sample data. See https://en.wikipedia.org/wiki/Inference and https://en.wikipedia.org/wiki/Statistical_inference
3,010
questions
0
votes
0
answers
12
views
which statistical test should be used for testing the significance of the mean differences in time series data?
I was looking to investigate if the monthly total return of US Treasury bond index depends on the magnitude of the US inflation readings.
So i downloaded the US CPI YOY monthly data for the last 50 ...
- 547
6
votes
3
answers
464
views
Inference vs prediction terminology
The terminology "inference" and "prediction" seems to have different usage across numerous sources and at my current work.
In some places, it seems "inference" refers to ...
- 685
1
vote
1
answer
62
views
Overlapping confidence intervals of two proportions
According to to G. Cumming and S. Finch in their paper "Inference by Eye" the 95% confidence intervals of two independent means can overlap and the difference between the two means can still ...
- 77
-1
votes
0
answers
10
views
so the question is about calculating MAP and PPD. I know the formulas for both, but find it confusing, so can someone explain step by step? [closed]
Now suppose that you run each model, and they make the following predictions:
p(yt+1 | yt, θ1) = .4 p(yt+1 | yt, θ2) = .75 p(yt+1 | yt, θ3) = .6.
What is the maximum a posterior estimate p(yt+1 | yt,...
0
votes
0
answers
13
views
Testing change in ratios
I do not know how to perform this statistical test. I have a proportion of a bodily liquid substance, and I am interested in the ratio with respect to another. For instance, ratio of water (20l) and ...
3
votes
2
answers
269
views
Why do we need a consistency assumption in causal inference?
Why do we need a consistency assumption in causal inference? I think the consistency assumption is quite obvious and it is more like a definition for the observed outcome.
- 31
1
vote
1
answer
65
views
Causal inference on time-series data: is intervention needed?
I'm working on the topic of causal inference, I use time-series data. I have two scenarios in front of me and I don't understand the difference:
Given X and Y "time" features. I would like ...
- 23
0
votes
0
answers
8
views
Statistical Integration of Bayesian and Frequentist Approaches: Weighing Methodology
I'm uncertain about where to post this question. I'm currently working with geotechnical data (soil parameters) and aiming to obtain realistic and safer parameter values.
To achieve this goal, I've ...
- 123
0
votes
1
answer
36
views
Identify outliers in testing data based on trained Gaussian mixture model
I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as:
$p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
- 11
0
votes
0
answers
4
views
What type of rotational invariance information can I get from interest points (coordinates) of corners from an image?
Assume that you are using the FAST algoritm for corner/feature detection. You pick an image and run the FAST algorithm.
Question:
All these green dots are actully coordinates in x and y direction. ...
- 381
0
votes
0
answers
39
views
Good training/test results, but very poor performance in inference as the stream data is coming
I am interested in the audio classification problem. After labeling the audio recordings I have in Praat software environment, I extract the MFCC features from each labeled frame and create an SVM ...
- 23
2
votes
2
answers
179
views
Regression coefficient before vs. after demeaning X
It is well known that $\beta = (X^{T}X)^{-1}X^{T}Y$ for linear regression. While experimenting with this, I found that $\beta$ (excluding the intercept term) remained the same before and after ...
- 31
0
votes
0
answers
11
views
How do I determine sample size when the conversion rate for the null hypothesis is not deterministic?
I use a ML model to rank prospects in terms of purchase probability to send marketing emails on a monthly basis. Most of the people in the population (for example, the entire U.S population) would not ...
- 1
1
vote
0
answers
28
views
Unsure whether to use bootstrap or not
I've only recently came across the bootstrap procedure and I'm not entirely well versed in statistics so apologies if I'm completely wrong here. I'm looking at the average temperatures across a month ...
- 111
2
votes
2
answers
63
views
Are regression CIs valid when the outcome is an ML prediction?
If we use classification labels from a machine learning model in a linear regression, and treat those labels as the ground truth, will the resulting confidence intervals be valid?
Imagine we have $n$ ...
- 21
0
votes
0
answers
31
views
Determine "best" winrate between data with different sample sizes
Setup
I've a collection of winrates where each winrate represents a particular combination of "power ups" a player can select for a character they play in a game. The numerator represents ...
- 28
0
votes
1
answer
74
views
Is Power Function the same as Probability of Type I error?
I am currently reading Casella & Berger and at Paragraph 8.3.1 Error Probabilities and the Power Function It says the following:
Suppose $R$ denotes the rejection region for a test. Then for
$\...
1
vote
1
answer
59
views
Expectation propagation: $g_i(\theta)\sim\mathcal{N}(\mu_i,\Sigma_i)$ but $\Sigma_i^{-1}$ is non-invertible
I paste here the section on Expectation Propagation from Gelman et al.'s Bayesian Data Analysis (pg 340). I have two questions:
In Step 5 (with the red box), where we infer the hyperparameters of the ...
2
votes
0
answers
18
views
How to determine impact of a Visiting Team on Attendance at a Sporting Event? [Personal Thought Exercise]
I am trying to figure out the impact of a visiting team on the attendance draw of a sporting event.
This is a personal inquiry exercise to see if the team that I root for has any impact on the other ...
- 21
1
vote
0
answers
59
views
Can evidence of absence be obtained only in one-sided hypothesis tests?
It seems that if a one-sided Z-test returns a p-value $>1 - \alpha$, you have enough evidence of absence to reject the alternative hypothesis in favor of the null hypothesis, because if you were to ...
- 111
1
vote
1
answer
210
views
Is this a correct usage of confidence intervals?
I need to quantify the relative error of a process that computes the volume of some objects. I also have a sample ($n=400$) of objects that are known to be made of a homogeneous material. Since the ...
- 13
0
votes
0
answers
20
views
How can I apply McNemar’s test in matched case control study on SPSS?
When conducting a case-control study with 120 cases 1:1 to 120 controls and examining whether they had a binary exposure prior to developing the disease. I am using conditional logistic regression for ...
3
votes
1
answer
265
views
In multiple linear regression does heteroskedasticity need to be evaluated for each predictor or just for the overall model?
If we want to make valid inferences about a simple linear regression estimator with one term, then homogeneity of variance of the residuals is a common assumption for making valid inferences.
However, ...
2
votes
0
answers
117
views
finding the UMPU test
When trying to find the uniformly most powerful unbiased (UMPU) test for the exponential family Lehmann and Romano give a "simple" approach.
Suppose that we want to test $H_0: \theta = \...
- 1,323
2
votes
1
answer
25
views
Coefficients of log-transformed outcome
I would like to know what is the appropriate word for regression coefficients of log-transformed outcome(y) after taking the anti-log of regression coefficients.
It is not an odds ratio or risk ratio. ...
2
votes
1
answer
57
views
Checking for association between two multi level categorical variables
As part of a personal journey to learn inferential statistics, I am trying to understand if and how income levels influence political views, using data from the general Social Survey (GSS) found here
...
- 23
4
votes
1
answer
396
views
Independent t-test formula
I am trying to understand the independent t-test formula. It is to test if A and B have the same mean, assuming A and B have the same variance. The t is calculated by the following formula:
$A = A_1, ...
- 155
1
vote
1
answer
36
views
Validity of the comorbidity polypharmacy score (CPS), or any other composite independent variable
I'm looking at carrying out inference on the effect of comorboditity, polypharmacy and personal characteristics (age, ethnicitiy etc) on health outcomes such as time on trial and emergency scans/...
- 571
1
vote
0
answers
33
views
Providing biased data to analyst -- how could they use information about the bias? [closed]
I'm working on a project that provides an anonymized dataset to a service. Clients of this service will often use our data to make inferences to the population it's drawn from.
Our data is known to be ...
- 133
0
votes
0
answers
13
views
Applying Bayesian Statistics to Determine Probability of Improved Process
I am new to Bayesian Statistics but from my basic knowledge, I feel there may be real practical benefit in using it at work.
My role is to find process improvements during fabrication to improve batch ...
- 155
2
votes
0
answers
39
views
Understanding the Intersection Between Causal and Statistical Inference
Assume a simple example motivating a causal research design. Say that I collect a data set on rural counties in Texas and I wish to understand if rainfall causes a change in crop sales. Working with ...
- 623
0
votes
0
answers
8
views
Dealing with Dominant Past Sales Predictor in Linear Regression for Store Sales Inference
I am using linear regression to do inference and know how much each predictor affects sales.
In have data for several stores with features and sales during a certain time frame.
There is not much ...
- 101
0
votes
1
answer
99
views
Detecting and Forecasting Intermittent Time Series
I am building a model to forecast some metrics. Those metrics are quite seasonal giving me good forecasts as shown below:
However, some new requirements dictate that I target those forecasts per ...
- 21
1
vote
0
answers
35
views
How to construct confidence intervals for multiple choice survey responses where the measurement is the percentage of respondents?
Say I have $k$ categories of responses to a survey question from $n$ respondents. Assume n of each category is about 50, so we can't just work off asymptotic behavior necessarily.
How do I compute ...
0
votes
0
answers
19
views
Z test for proportion of sums
I want to perform a Z-test for two variables: $ X $ (Buyer proportion) and $ Y $ (Spent proportion \$ ). Below the table you can see a dummy example of the data I have:
Let's assume the conditions ...
1
vote
1
answer
52
views
Two sample proportions: How to test the hypothesis that the difference in proportions is less than a specified threshold?
Say I have treatment and control, perfectly randomized.
I have 70000 conversions for treatment in 80000 users.
I have 70100 converions for control in 800100 users.
My null hypothesis is that there is ...
1
vote
1
answer
24
views
How can I calculate response rate when I don't know how many people saw the item in the first place?
I have a count of how many people responded to an ad, but I don't have an official count of how many people saw the ad in the first place. I do have an estimate of how many people saw the ad though ...
- 113
1
vote
1
answer
33
views
The role of variance of the distribution plays in Bayesian inference
Given prior $ \mu \sim \mathcal{N}(\mu_0, \tau^2) $, likelihood $ X_i | \mu \sim \mathcal{N}(\mu, \sigma^2) $, we know the closed-form solution of posterior is $ \mu | X_1, X_2, \ldots, X_n \sim \...
- 363
2
votes
0
answers
32
views
Convergence of variational posterior
Let $q_\lambda(z)\in\mathcal Q$ be the variational posterior approximation of $p(z|y)$.
Note that the optimal $\lambda^*$ is approximated by the following recursive sequence
$$
\lambda^{(k+1)}=\lambda^...
- 153
0
votes
1
answer
37
views
Is bayesian updating framework a valid concept?
When I google search for the term, only 6 pages showed up. There is no authoritative paper on this, except https://arxiv.org/abs/1306.6430 which argues for using informatics concepts to generalize a ...
- 1
0
votes
2
answers
31
views
How to obtain the local unbiased condition for an estimator from global unbiased condition?
A standard problem in classical statistics is to find a good estimator that minimizes a given cost function under certain conditions. Normally we want to require the estimator $\hat\theta$ to be ...
- 117
6
votes
4
answers
2k
views
How to calculate standard deviation when only mean of the data and sample size is available?
Many a times it happens that you get comprehensive data at a group level and you have to compare one group against another but you don't have the data points separated. In these cases, how do we get ...
- 129
0
votes
0
answers
20
views
Which statistical test to apply to calculate if a difference is significant between annotations?
I have the following problem: I have a text which needs to be annotated with named entities. I have two annotators who have annotated the text twice. How can I calculate if the change in the number of ...
- 101
6
votes
1
answer
95
views
Puzzle: Detecting Duplicate Solutions
This question has been on my mind for a while, so I am asking it here.
Suppose there are n students who are tasked with an assignment. Students can either produce an original solution, or they can ...
- 237
0
votes
0
answers
17
views
Reporting policy effect for specific subpopulation - how to interract?
I am seeking guidance on model selection when reporting a causal policy effect for a specific subgroup of the panel. I am open to any form of help: paper, maths, intuitions... Below my setup, please ...
- 21
1
vote
0
answers
26
views
How do I know which statistic is for which parameter when calculating joint sufficient statistics using factorization criteria?
For the normal distribution for example, after factorization we get
$\mathcal{L} = (2 \pi \sigma^2)^{-\frac{n}{2}}\exp\left(-\frac{n\mu^2}{2\sigma^2}\right) \exp\left(-\frac{1}{2\sigma^2}\left(\sum_{i=...
- 11
0
votes
0
answers
44
views
Calculate posterior distribution and full conditional of a HMM
Set up a Bayesian analysis of an hidden Markov model and calculate the posterior distribution and the full conditionals, given this assumptions:
The state space of the hidden process has size m
$Z_t|...
1
vote
0
answers
51
views
If we know the exact value of the population parameter , is there any point in doing the hypothesis testing? [duplicate]
Question - In a year there are 956 births in a town A of which 52.5% were males, while in town A and B combined, this proportion in a total of 1406 births was 0.496. Is there any significant ...
- 31
1
vote
0
answers
19
views
Unifying multiple sample data (meta-analysis)
I would like to unify multiple sample data (of different sizes) into some "unified sample" to evaluate its collective variance. Is this something statisticians do? I thought of unifying ...
- 11
1
vote
1
answer
69
views
Extract systematic and random variation
I am working with the data from some manufacturing line. The parts are processed in batches of say 20. In order to control process, we do measurements on manufactured parts. We know that the ...
- 55