New answers tagged mathematical-statistics
0
votes
Can I use K-Means to group customers based on a single variable?
If you want a data driven clustering, k-means looks promising in the sense that it will produce clusters with similar within-cluster variance, which may make sense in your application. The problem of ...
- 16.7k
0
votes
minimizer weighted linear regression
$\require{cancel}$
$$\operatorname{Cov}(\theta^\ast)=E[{\theta^\ast}{\theta^\ast}']-E[{\theta^\ast}]E[{\theta^\ast}]'$$
$$E[{\theta^\ast}]E[{\theta^\ast}]'=\theta \theta'$$
$$y = X\theta+u$$
$$E[{\...
- 17k
1
vote
Can I use K-Means to group customers based on a single variable?
Given your description, I would just assign cutoff points at percentiles of the distribution of total spend. With five categories, equal intervals (same number of observations in each category) would ...
- 21
0
votes
Normal Distribution/Probability problem
For the first question, the weight of the standard egg (SE) shouldn't be compared with a fixed value ($52=65\times 4/5$), since the weight of picked large egg (LE) each time is not fixed but is a ...
- 733
0
votes
Find most unique image from pair scores of a set of Images
There isn't one "correct" metric. Reasonable approaches might be to find the image with the smallest mean similarity, or smallest median similarity, or smallest maximum similarity, or ...
- 6,756
4
votes
Formal definition of p-value
But unfortunately I couldn't find the definition of $S_{\alpha}$
The p-value is not defined in an unambiguous way
"The probability to get, given the null hypothesis, an effect-size equal to or ...
- 61k
5
votes
Formal definition of p-value
Lehmann is talking about a nested sequence of critical regions $\langle S_\alpha\rangle$ with the index being the size of the corresponding test. This is due to the fact that he needs to find the ...
- 5,137
0
votes
Use the delta method to find confidence intervals
Are you finding $\mathrm{SE}(\widehat{\lambda})$ and $\mathrm{SE}(\widehat{\lambda}\log{\widehat{\lambda}})$ instead? Let $g(\widehat{\lambda})=\widehat{\lambda}\log\widehat{\lambda}$. By delta method,...
- 66
0
votes
What is likelihood actually?
After studying Tim's answer and realising that code can give me the best short sentences. I realised that I could understand best via code.
The following code outputs the likelihood of a particular ...
- 639
2
votes
Accepted
How to find asymptotically normal estimator if I know probability density function
You can use the maximum likelihood estimator, which, in regular cases as this one has limiting normal distribution. This is defined as
$$
\hat\theta = \text{arg max}_{\theta\in\Theta} L(\theta),
$$
...
- 8,101
5
votes
What is the roadmap to self-taught probability and statistics for artificial intelligence?
If you were an academic, one must assume you already have a good reference for multivariable calculus, linear algebra, and differential equations – these are not optional. I personally heard from ...
2
votes
Accepted
Find a function of $\theta$ so that there exists an unbiased estimator and the variance coincides with Cramér-Rao lower bound
When $X_i\overset{\textrm{iid}}{\sim}f(x\mid\theta), $ and $\hat\tau(\mathbf x) $ is an unbiased estimator of $\tau(\theta), $ then the unbiased estimator attains the CRLB if and only if there exists ...
- 5,137
0
votes
Covariance of $\frac{x}{\|x\|}$ for Gaussian x?
By applying dherera's formula to trace-normalized zero-centered Normal with diagonal $d\times d$ covariance matrix $H$, we get the following estimate for the corresponding covariance matrix $H_p$ of ...
- 5,859
1
vote
Covariance of $\frac{x}{\|x\|}$ for Gaussian x?
I think that the distribution of $y=\frac{X}{||X||}$ where $X\sim \mathcal{N}(0, \Psi)$ is the projected normal distribution.
I came across a similar problem and used an approximation that works well ...
- 168
2
votes
What is likelihood actually?
As stated by many others: the likelihood function $\mathcal L_y$ is the probability density function $f_\theta$ of the observed data $y$, but viewed as a function of the (unknown) parameter $\theta$, ...
- 4,135
5
votes
Is more data really always better in machine learning?
You are right, it is not only about the size of the dataset. As two other answers pointed out, having more data (vs very little) is desired, as even in a noiseless scenario it may help you to get a ...
Tim♦
- 127k
5
votes
Relationship between z-score and the normal distribution
I think there is some confusion here due to the word "normalization". In this context, normalization means that the data are transformed to have zero mean and unit standard deviation. The ...
- 698
5
votes
Is more data really always better in machine learning?
My intuition is that, given $(x_{i},y_{i})_{i=1}^n$ and $(x_{i},y_{i})_{i=1}^N$ have the same "information" (I know this is a fuzzy term), using $(x_{i},y_{i})_{i=1}^N$ should not better the ...
- 61k
3
votes
What is likelihood actually?
Likelihood is a slippery concept. The likelihood function, $L(t | w)$, expresses how probable the data $t_n$ are in relation to the model function $y(x,w)$.
The uncertainty in the empirical ...
- 131
5
votes
Accepted
Relationship between z-score and the normal distribution
The ``normal distribution'' is an entire family of different distributions. We use the notation $\textbf{Normal}(\mu,\sigma^2)$ to indicate what type of normal we get. If you pick a certain choice for ...
- 1,705
7
votes
Relationship between z-score and the normal distribution
There is no relationship. The (sample) z-score is defined as
$$
z_i = \frac{ x_i - \bar x } {s}
$$
where $i$ indexes observations $\{x\}$, $\bar x$ is the sample mean, and $s$ is the sample standard ...
- 511
5
votes
Is more data really always better in machine learning?
Intuitively, having more data will tell the neural network where to turn, by how much, and in what direction (up/down, left/right, combinations, extensions in high-dimension spaces, etc).
Imagine your ...
- 46.5k
2
votes
Can we recover joint distribution from a continuous range of convolutions?
I am not sure I understand the question, because I don't understand the relation with convolutions. If we assume that $X$ and $Y$ are independent, then $pX+(1-p)Y$ is indeed a convolution, but what if ...
- 173
9
votes
What is likelihood actually?
There have been numerous responses including some to your very posts earlier and the present one too.
It should be reiterated that $\mathcal L(\theta\mid \mathbf x) $ or $\ell_\mathbf x(\theta)$ (to ...
- 5,137
10
votes
Accepted
What is likelihood actually?
The likelihood function parametrized by a parameter $\theta$ in statistics is defined as
$$
\mathcal{L}(\theta \mid x) = f_{\theta}(x)
$$
where $f_{\theta}$ is the probability density or mass function ...
Tim♦
- 127k
0
votes
What is likelihood actually?
I am not exactly sure I fully understand the question but I suspect it might come down to understanding what the likelihood function is measuring exactly.
Let $X$ be a (absolutely) continuous random ...
- 1,705
1
vote
Sufficiency of $|X|$ when $X\sim N(0,\sigma^2)$ without using Factorization theorem
$[\rm I]$ notes for $T(\mathbf X) $ to be sufficient for $\theta, $
$$ \mathbb P_\theta(\mathbf X=\mathbf x\mid T(\mathbf X) =T(\mathbf x))=\frac{p(\mathbf x\mid\theta)}{q(T(\mathbf x)\mid \theta)} \...
- 5,137
2
votes
Sufficiency of $|X|$ when $X\sim N(0,\sigma^2)$ without using Factorization theorem
The problem of your derivation is that you misunderstood the concept of conditional distribution. It is not $P_\sigma(X \leq x |T(X) \leq t)$ -- it should be $P_\sigma(X \leq x | T(X) = t)$. For a ...
- 10.4k
2
votes
Understanding different approaches to construct confidence intervals with bootstrap
As @dipetkov says in a comment, a full answer requires chapters if not a book. If you want to understand the bootstrap, originally developed by Efron, then you could do worse than to consult the ...
- 77.4k
4
votes
Using Jeffreys prior for Bernoulli distribution to find the prior of a transformation on p
You know the Jeffreys' prior density for $p$ is proportional to $\sqrt{\frac{1}{p(1-p)}}$ and that the corresponding odds are $\eta=\frac{p}{1-p}$.
This looks as if it is just a change of variables ...
- 34.6k
0
votes
Within and Between Variation in fixed and random effects models
RE models rely on within- and between-group variation, while FE models only rely on within-group estimation.
However, this is only true if the explanatory variable is independent from group specific ...
- 13
1
vote
A question about the exchangeability assumption in conformal predictions
There are conformal prediction approaches that explicitly consider time series analysis. Here is an article that discusses this: Conformal prediction for time series. The R caretForecast and the ...
- 1,838
3
votes
Regularity conditions hint
Written in that way, it seems complicated. But what the hint really says is just the following simple inequality:
\begin{align}
\sup_{\theta \in \Theta}(f(\theta) + g(\theta)) \leq
\sup_{\theta \in \...
- 10.4k
1
vote
How is bandwidth and roughness penalty in nonparametric regression connected?
The relation between spline estimators and spline estimators is far from trivial. The "best starting reference" is likely: Lin et al. (2004) Equivalent kernels of smoothing splines in ...
- 39.3k
2
votes
Which Fisher information should I use for Cramer-Rao lower bound?
Cramér-Rao Lower Bound would be of the form $\operatorname{Var}_\theta(T(\mathbf X) ) \geq \mathscr I(\theta)^{-1}. $ For exponential family, $\mathscr I(\theta) =\mathbb E_\theta\left[-\partial^2_\...
- 5,137
5
votes
Question about sample size indicated from power analysis for chi-square analysis in Python
As you don't provide a lot of details about the goal of your study, from the outside it looks a bit like your null hypothesis may be ill-defined:
why using a chi-squared test, when the variable ...
- 1,044
4
votes
Question about sample size indicated from power analysis for chi-square analysis in Python
The boundary for what is "just statistically significant" (i.e. the p-value is just below some "significance threshold" such as 0.05) is, if everything you observe is the true ...
- 27.1k
4
votes
Help with rigorous derivation of multinomial distribution
Your present attempted proof is question-begging
Firstly, well done on your initial attempt. You appear to have a basic idea of how you would like to proceed, and you are making an attempt to set ...
- 109k
6
votes
Help with rigorous derivation of multinomial distribution
Even under the rigorous measure-theoretic framework, your proof is overly verbose, probably due to that you confused the underlying probability space $(\Omega, \mathscr{F}, P)$, where $X_1, X_2, \...
- 10.4k
1
vote
Accepted
How would you describe the distribution of this data?
First, you should use a more descriptive title, if possible. You have ordinal data (scores between 1 and 10?), where the data is mostly saturating the scale (getting the top value).
Your data seems ...
- 168
0
votes
Finding the expected value for the mean squared
Here is a more expanded version of the derivation:
\begin{align}
E\left[\bar X_n^2\right] &= E\left[\left(\frac{1}{n}\sum^n_{i=1} X_i\right)^2\right] \\
&= E\left[\left(\frac{1}{n}\sum^n_{i=1} ...
- 3,658
1
vote
Are the method-of-moments-based normal confidence intervals asymptotically valid and optimal?
For a normal random variable, the moment-matching estimator (MME) for the mean is the maximum likelihood estimate (MLE). For the variance, the MME and the MLE differ just by the bias adjustment ( n/(n-...
- 11
1
vote
Which hypothesis test should I use in this scenario?
You probably want to treat your variable Age as an ordinal variable (ordered category). Since Employed is dichotomous, it could be treated as either nominal (nominal category) or ordinal.
Since you ...
- 10.4k
3
votes
How can I compute the joint distribution function of normal distribution and beta distribution?
I presume your intent is that the marginal distributions are beta and normal respectively, rather than say one conditional and one marginal or both conditional.
Specifying the marginal distributions ...
- 270k
1
vote
Samples from a multivariate t distribution
Because you get multivariate Student t as the fraction of the normal vector and a chi-squared random variable. So, of course, the histogram of entries of the vector will be normal, but for every ...
- 11
2
votes
Accepted
How to find variance of multivariable expression
This is a partial answer and partially a request for clarification.
If I understand correctly,
$a$ is a constant.
$X_i$ is Bernoulli with $\Pr(X_i=1)=p_i.$
$Y_i$ is Bernoulli with $\Pr(Y_i=1)=q_i.$ ...
- 306k
8
votes
Statistical test for vector of expected values
The problem can be solved using a slightly modified paired $t$-test in which, the null is $H_0:\mu_1 -2\mu_2 = 0$ and the alternative is $H_1:\mu_1 -2\mu_2 \neq 0$. The procedure is thus to apply a $t$...
- 8,101
6
votes
Accepted
UMVUE for $g(\theta)=\theta^2$ of Poisson random variables
Yes, that's correct. To make it rigorous, you can apply Lehmann–Scheffé Theorem with $Y = T(X)$ and $\varphi(Y) = \frac{1}{n^2}(Y^2 - Y)$.
After a closer look at the Wikipedia link above, it seems ...
- 10.4k
0
votes
How combine two statistics from different tests?
The simplest approach, but also most conservative, is to use a Bonferroni correction. In your case, this would be simply using $2 \times \text{min}(p_1, p_2)$, where $p_1$ and $p_2$ are the p-values ...
- 19k
7
votes
Accepted
Showing that $\frac{\Vert Au\Vert_2^2}{\Vert u\Vert_2^2}\sim \chi_d^2$
If you denote by $a_i$ the rows of $A$, then each element of $a_i$ has a standard normal distribution. Therefore,
$$
a_i^\top u = a_{i1}u_1 + \cdots +a_{ip}u_p \sim N(0, u^\top u)
$$
and thus
$$
\frac{...
- 8,101
Top 50 recent answers are included