Questions tagged [mathematical-statistics]
Mathematical theory of statistics, concerned with formal definitions and general results.
7,286
questions
-1
votes
0
answers
16
views
Your colleagues in a comp-bio lab have sequenced DNA from a large population in order to understand how a gene [closed]
Your colleagues in a comp-bio lab have sequenced DNA from a large population in order to understand how a gene (G) influences two particular traits (T1 and T2). They find that P(G) = 0.6, P(T1|G) = 0....
- 99
0
votes
1
answer
28
views
Is it better to have more degrees of freedom or less?
The concept of degrees of freedom appears in many places in statistics. Yet I still don't understand what is the role of degrees of freedom. Why can we just use sample size (and we need to decrease it ...
- 3,689
1
vote
0
answers
11
views
Computing a Weighted AUC Metric
Consider the standard ROC-AUC metric, which has been shown to be equivalent to the normalized Wilcoxon-Mann-Whitney:
$$
AUC = \frac{1}{N^{(+)}N^{(-)}} \sum_{i}\sum_{j}1[\hat{p}_i^{(+)} > \hat{p}_j^{...
- 281
9
votes
2
answers
693
views
When mathematical statistics outsmarts probability theory
This is not a question, but it is too good to pass. I read it is originally due to Enis, Peter. "On the relation $E (X) = E [E (X∣ Y)]$." Biometrika 60, no. 2 (1973): 432-433.
Assume $Y$ has ...
- 53.8k
0
votes
0
answers
15
views
Expected magnitude of cosine similarity among $B$ Gaussian vectors?
Take an IID sample of $B$ vectors $x$ drawn from a Gaussian with mean $\mu$ and covariance $\Sigma$. Define the following:
$$S=\frac{1}{B^2}\left\|\sum_i^B x_i x_i^T\right\|^2_F$$
$S$ gives the ...
- 5,403
1
vote
1
answer
33
views
why not using sample variance (instead of MSE) to estimate the error variance in linear regression?
Assuming the true equation for Y is linear as below:
$$Y_i =\beta_1X_i +\beta_0 + \epsilon_i$$
Assuming X is fixed, then the variance of each Y is:
$$var(Y_i )=var(\epsilon_i)=\sigma^2$$
In order to ...
- 179
0
votes
1
answer
34
views
Is it valid to move a logarithm inside of an expectation?
I have the following derivation for a latent variable model.
$$
\newcommand{\d}[1]{\mathrm{d}#1}
$$
$$
\begin{align*}
\mathbb{E}_{q(x_0)}\left[-\log p_\theta(x_0)\right]\\
&=^{1.} \mathbb{E}_{q(...
- 227
1
vote
0
answers
14
views
Does the number of samples, as opposed to the sample size in each sample, matter for the Central Limit Theorem? [closed]
(1)So here is a formula that describes CLT I found at https://en.wikipedia.org/wiki/Central_limit_theorem.
According to the first part of the explanation, n as in Xn describes the number of samples(i....
- 11
2
votes
0
answers
72
views
Unbiased estimator for $\mu_1/\mu_2$
Let $X_1,X_2,\ldots,X_n$ and $Y_1,Y_2,\ldots,Y_n$ be independent random samples from $N(\mu_1,1)$ and $N(\mu_2,1)$ populations respectively with $\mu_2\neq0$.
I need to find an unbiased estimator for $...
- 21
0
votes
0
answers
13
views
Deriving slope of linear regression [duplicate]
Hi Stack Exchangers,
Reaching out for clarification on two different ways to derive B0 for a best fit line using OLS. Are they related?
From MIT Notes, B1 is defined as the minimum sum of squared ...
0
votes
0
answers
16
views
calculate minimum number of games a player needs for metric to be statistically significant
I have a dataset of players playing a variety of games of various team sizes (individual, pairs, and teams of 3-10) across multiple "seasons". Some players have played dozens of these games, ...
- 1
0
votes
0
answers
22
views
Contradiction in Stationarity of AR(1) under the condition that |Φ| < 1 [duplicate]
Something must have gone wrong with my reasoning.
UPDATE: This is the answer to my question provided by Dr. Ben.
Where is $|\theta|<1$ used in recursive method derivation of invertibility of MA(1)?
...
- 139
0
votes
0
answers
12
views
Method of moments - what determines the number of moment conditions?
Given the following definition:
What is the function $f$ here?
For me, the method that is presented in the wikipedia article is very clear:
https://en.wikipedia.org/wiki/Method_of_moments_(statistics)...
- 1
0
votes
0
answers
16
views
for loop in R to compute the yearly evolution of a variable [closed]
here is the structure of my dataset for reproducibility :
...
0
votes
0
answers
13
views
Determining what lift can be attributed to tiered rewards program
Statistically, is there a way to attribute the sales increase from tiered rewards program?
First off, I feel like I know this answer to this (I need more data), but I’m new to stats and thought you ...
0
votes
0
answers
15
views
Invertibility of first-order lag polynomial
Assume we have a first order lag polynomial $\phi(L) = 1 - \phi L$. It is commonly taught that if $|\phi| < 1$, $\phi(L)$ is invertible. It means that $\exists \phi(L)^{-1}$ such that $\phi(L)\...
- 139
1
vote
1
answer
104
views
Can we apply hypothesis testing to not-actively sampled groups?
For argument's sake, in the below please assume that the hypothesis test we'd be considering would be a simple z-test to check whether an observed difference between two groups' means or proportions ...
- 11
1
vote
2
answers
39
views
Estimation vs Prediction Vs Forecast
I already read the q/a's from here and here. These answers are quite a bit complicated for me. I need something easy to understand and explain to any interviewer. I read lots of articles like;
What is ...
- 635
2
votes
1
answer
28
views
Average Sample Size For Curtailed Double Sampling Plan
I am currently implementing double sampling. The goal is to accept or reject a lot by only sampling a few instances from it. In a two-stage testing procedure (double sampling) one first draws a ...
- 103
0
votes
0
answers
29
views
mathematical model of well variability
I would like some help in how I can analyse or approach an experiment in a statistically/mathematically way.
I have two plates where I applied a different technique that kills cells. I would like to ...
- 111
1
vote
0
answers
15
views
Combined Probability of 5 Events with Different Probabilities and 25 Trials
I have 5 events, A through E, with individual probabilities:
P(A) = 0.12
P(B) = 0.05
P(C) = 0.11
P(D) = 0.12
P(E) = 0.60
After 25 trials here were the results:
A = 1 success
B = 3 successes
C = 4 ...
- 11
-1
votes
0
answers
23
views
Reason behind Pareto principle [closed]
Can some one explain the scientific reason behind Pareto principle/Matthew principle.
- 101
0
votes
0
answers
8
views
How to correctly compare 2 correlation values using the Fisher r-to-z transform?
In https://www.statisticshowto.com/fisher-z/, it is stated that “But it’s probably most commonly be used to test the significance of the difference between two correlation coefficients, r1 and r2 from ...
- 90
1
vote
0
answers
36
views
Ratio of first two moments of sample covariance eigenvalues
Suppose we form $X$ by stacking $b$ examples drawn from some distribution in $n$ dimensions as rows and look at $\{\lambda_i\}$, the eigenvalues of $\Sigma=X^TX$. Consider the following random ...
- 5,403
0
votes
0
answers
5
views
Multi entry raffle one win per person [duplicate]
I have a statistical question. I wanted to know the probability of winning a multi entry raffle with 20 wins and only 1 win per person. Each Person can have multiple tickets.
Let’s assume I have 2 ...
0
votes
0
answers
14
views
Statistical test to use to compare subgroups with each other
I don't have a stats background, so I apologize in advance if my question is dummy. I sent a quiz to several people and I would like to run some statistics' tests on it (I'm using R).
Essentialy I ...
- 1
0
votes
0
answers
27
views
Best approximation for the size of a test
Let $X \sim \mathrm{Bernoulli}(\vartheta)$ for some unknown $\vartheta \in (0,1)$, and let $(X_1, …, X_n)$ be a moderately large IID sample for $X$.
Let $\vartheta_0 \in (0,1)$. I want to test $H_0 \...
- 101
0
votes
0
answers
12
views
Can this time series be described using an additive model?
I figure these fluctuations are roughly constant but just want to confirm.
-2
votes
0
answers
28
views
What is the significance of the random variable $\dfrac{n+m-2}{n+m-2+T^2}$ and how is it derived from a likelihood-ratio test statistic?
Why does this likelihood-ratio test statistic $\Lambda^{\dfrac{2}{n+m}}$ define a random variable that is $\dfrac{n+m-2}{n+m-2+T^2}$?
$n+m-2$ are degrees of freedom of the t-distribution.
I did not ...
- 503
0
votes
0
answers
12
views
Scatter Plot of the Ranks of spearman vs Hoeffding [closed]
Could anyone tell me why the answer is AccountBal
0
votes
0
answers
15
views
How to use auxiliary factors as weights in a Portfolio
(1) I am looking to form a Market Neutral Portfolio based off possibly 2 factors.
Factor1 is the Primary factor, I rank all the stocks (say 1000 stocks) based off the Factor1 value, and pick top 100 ...
- 163
0
votes
0
answers
26
views
What is the probability that the first number chosen is greater than the sum of the next three numbers chosen? [duplicate]
Let $ E_i\sim \mathcal N(20,3), i \in \{1,2,3,4\}.$
What is the chance that $E_1$ is greater than $E_2+E_3+E_4? $
I started by $\mathbb P(E_1>(E_2+E_3+E_4))=\mathbb P(E_1-E_2-E_3-E_4>0).$
What ...
- 1
3
votes
4
answers
465
views
Why do we need importance sampling?
Let's say we want to calculate the following expectation:
$$
\mathbb{E}_{z\sim p_z(z)}[f(z)]
$$
One issue, is that the samples from $p_z(z)$ could be not very informative:
We see here that $f(z)$ ...
- 227
0
votes
0
answers
31
views
How does one prove each of these formulas from Stata for means and proportions power analysis?
http://rforpublichealth.blogspot.com/2013/06/sample-size-calculations-equivalent-to.html says that Stata and R give different formulas for calculating power for proportion-hypothesis tests and means ...
- 503
0
votes
0
answers
21
views
Request a geometric proof that difference of sample-means has additive variance larger than the components?
Request a geometric proof that difference of sample-means has additive variance larger than the components?
You are not allowed to use the formula quoted.
In general, the variance of the sum of $\...
- 503
0
votes
0
answers
17
views
Is the Karlin-Rubin theorem test the same as a one-sided monotone likelihood-ratio test?
Is the test described in the Karlin-Rubin theorem test the same as a one-sided monotone likelihood-ratio test?
- 503
-1
votes
0
answers
22
views
Why can't the random-varibale $Q_{k-1}$ be computed once we observed $X_1,...,X_k$? and why its distribution becomes $\chi^2_{k-3}$?
We encounter hypotheses $H_{0}$ in which the multinomial probabilities $p_{1}, p_{2}, \ldots, p_{k}$ are not completely specified by the hypothesis $H_{0}$. That is,
under $H_{0}$, these probabilities ...
- 503
1
vote
0
answers
49
views
Are there any (exponential) families without a minimal sufficient statistic?
Bahadur's theorem says that if a minimal sufficient statistic exists, then a complete sufficient statistic is also minimal sufficient.
Are there any (homogenous, identifiable) families with a complete ...
0
votes
1
answer
23
views
Is it OK to re-sample the same patients at multiple points in time and then use the data for classification?
I'm trying to predict which patients will have a bad outcome (BO) within the next 2 weeks based on medical readings (plus some behavioral data) from the last 6 weeks. Total data timespan is 8 weeks, i....
- 1
0
votes
0
answers
23
views
computing the "mix effect" in the evolution of a variable on R
Background:
I am currently working on a dataset representing the evolution of the income of a hospital which hosts several medical specialties.
The ratio income/medical act increases between year $N$ ...
- 1
0
votes
0
answers
12
views
VECM and ECM relationship?
I know that VECM can be decomposed into single equations.
In my VECM with 4 variables, there are only one cointegration relationship.
Then if I decompose VECM into single equations,
only one equation ...
- 9
0
votes
0
answers
29
views
Minimize cost given working days/hours/required hours per job/start & end dates
I am trying to solve an optimization problem on Excel or Python. My constraints are the available number of employees (=20), the number of working days/month (=20), the number of working hours/day (=8)...
1
vote
0
answers
24
views
Proper way to write a regression equation with multiple interaction terms
As I am about to submit my master thesis, I am having doubts about the mathematical correctness of the regression equation I have used to represent my model. A bit of context: I am predicting review ...
- 11
0
votes
0
answers
8
views
How to propagate errors in Model Building?
I have developed two maps using a predictive linear regression where each pixel represents a predicted value of biomass or volume.
Next I want to use those predicted values of biomass or volume to ...
1
vote
1
answer
43
views
Confidence interval for C-index
After asking this question earlier today, I figured the approach I should use is not very clear to me so I turn to the community hoping for some answer or good reading material because I kind of get ...
- 11
0
votes
0
answers
11
views
estimating variance components of random effects model without fitting the model
I have the following scenario: we are interested in the modelling a response $y_{ij} = u_i + \beta x_{ij} + \epsilon_{ij}$, where $u_i \sim N(0,\sigma^2_u)$ is a random intercept, and $\epsilon_{ij} \...
- 1,055
0
votes
0
answers
15
views
Control differences with dunnett adjustment chart [closed]
Why the answer is A? but store A is not appear in the chart
0
votes
1
answer
18
views
How to calculate the accuracy and error rate in confusion matrix?
(0 - negative outcome, 1 - positive outcome)
Am I correct? Because I found a different answer.
My answer:
Accuracy rate: TP+TN/ALL(58+25/58+44+23+25)=83/150
Error rate: FN+FP(23+44)= 67/150
Someone ...
0
votes
1
answer
31
views
How can we incorporate an offset into a software library which performs least squares without offset
Disclaimer: this is not homework, just preparation for an exam I'll be taking
How do I incorporate an offset term into a software library that performs least squares (without offset)?
Let me give a ...
- 67
1
vote
1
answer
28
views
Categorising the PCA Method
PCA (Principle Component Analysis) is often used in statistical settings, yet it has also been described as an unsupervised machine learning method.
It's surprisingly difficult to find clarity online ...
