Questions tagged [mathematical-statistics]
Mathematical theory of statistics, concerned with formal definitions and general results.
6,846
questions
0
votes
0answers
56 views
Probability Mass Functions [closed]
I have asked a related question Probability Mass Function making the Truncated Normal Discrete
I am trying to understand the difference between 2 answers (the difference between 2 PMFs given).
What is ...
0
votes
1answer
48 views
Bayes error and nearest neighbor classification
Upon studying for my midterm using Pattern Classification by Richard O. Duda, David G. Stork, Peter E.Hart (2001), I stumbled upon the following exercise:
Using the solutions manual written by David ...
0
votes
0answers
16 views
What kind of regression to use in predictive model for a positive-only value
I'm trying to use regression in Excel to predict the % of bin contamination (dependant variable), based on my independent variables that are generally relating to demographics such as age/household ...
-1
votes
0answers
21 views
Relationship among exponential family, natural exponential family and exponential dispersion model
I have read the wikipedia of these three notions:
(1)exponential family
(2)natural exponential family
(3)exponential dispersion model
I have known the following relationship among them:
(1)The natural ...
0
votes
2answers
80 views
Linear regression residuals and OLS method
Assuming a simple linear regression framework, $y_i=\beta_{0}+\beta_{1}.x_{i} + \epsilon_{i}$, why can't I estimate $\beta_{0}$ and $\beta_{1}$ by optimizing the sum of residuals such that the sum is ...
1
vote
2answers
112 views
Why does regression to the mean work?
According to Wikipedia, regression towards the mean is "the phenomenon that arises if a sample point of a random variable is extreme (nearly an outlier), in which case a future point is likely to ...
1
vote
0answers
38 views
Comparing performance of Quasi-binomial model and Beta-binomial model
I read some books in biostatistics about fitting binary date with Beta-Binomial regression model and Quasi-Binomial regression model. It proposes a setting:
Setting:
Assuming we have a sequence of ...
1
vote
0answers
11 views
Why duality of confidence interval and hypothesis testing fails in random effect test for presence random effect?
I am reading Faraway's Extending the linear model with R chapter 10 section 2.
"In this case, the lower bound is zero. This is not surprising given our earlier
uncertainty over whether there ...
0
votes
0answers
14 views
Confidence interval of a sample of a survey
I am having trouble framing my question and identifying variables because of sub-populations.
If I have a survey of 10,000 people from a population of 100,000 people and within that survey 2,500 leave ...
0
votes
0answers
33 views
model output without correlation to input variable
let's assume that a,b,c,d,t,u,v,w,x,y,z and o are all correlated to one another. I want to make a prediction for outcome variable "o". However, I don't want my prediction to be to ...
0
votes
0answers
33 views
Global minimum of prediction intervals for multiple linear regression
I'm trying to solve the following problem:
Prove that the width of the prediction interval for $\hat{y}$,
given the vector $x_0 = (1, x_1,...,x_{p-1})$ of explanatory variables,
reaches a global ...
0
votes
0answers
20 views
High value for SD for data in range -100 to +250?
I Have a 1D data . I am trying to approximate this data to a gaussian distribution. I want to use MLE to estimate the parameters.
...
0
votes
1answer
27 views
Are the position indices (quartiles, median, mean) of distribution in classes always different from those of the unit distribution?
Are the position indices (quartiles, median, mean) of distribution in classes always different from those of the unit distribution? If so, for what reason?
In theory, since the class distribution is a ...
0
votes
0answers
32 views
Asymptotic distribution of a method of moment estimator via the delta method
Let $X_1,...,X_n$ be a random sample from X with a Beta distribution $B(\theta; 2)$, with $\theta > 0$, i.e. :
\begin{align*}
f(x,\theta) = \theta(\theta+1)x^{\theta-1}(1-x) \mathbb{I}_{0<x&...
0
votes
1answer
17 views
Confusion surrounding matrix multiplication rules used in proof
Let $\mathbf{Z}$ be the design matrix of a linear regression.
Where $\mathbf{Z}^{T}\mathbf{Z}$ is symmetric.
Set $\mathbf{V} = (\mathbf{Z'Z})^{\frac{1}{2}}(E(\hat{\beta}) - \beta)$
Where $\hat{\beta} ,...
0
votes
1answer
30 views
Interpreting the total-variance when the model lies over our data
I know that:
$Variance_{Total}=Variance_{Explained} + Variance_{Unexplained}$,
but I am wondering how the $Variance_{Total}$ relates to the $Variance_{Unexplained}$ and the $Variance_{Explained}$ if ...
0
votes
0answers
22 views
Understanding the notion of best estimators between 2 randoms variables with a weighting
A colleague put the mess in my head following a discussion about what he calls the "best weighting" from a statistic point of view when we treat 2 random variables.
He told me that if we ...
0
votes
1answer
30 views
Confusion surrounding differentiation of parameter vector ridge regression
Let $\mathbf{\beta}$ be the parameter vector of a ridge regression.
Now we can say that:
\begin{equation}
\frac{\partial \lambda \beta^T \beta}{\partial \beta}=2\lambda\beta.
\end{equation}
Why is ...
0
votes
0answers
19 views
What effect would clusters get when adding more variables to clustering task?
I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
1
vote
1answer
52 views
ANOVA model assumptions
Suppose the following:
$X_{lj}$ ~ $N(\mu_{l} , \sigma^{2})$ Where $l = 1, ...g$ and $j = 1,..n_{g}$
Furthermore assume $\sigma^{2}_{1} = \sigma^{2}_{2}...\sigma^{2}_{g} = \sigma$
Then we can define ...
1
vote
0answers
14 views
How do I write sum of squares and degrees of freedom for multiple Latin squares put together?
I have notes from my professor in which a Latin square model ($s$ treatments in an $s \times s$ square) is described as follows:
\begin{align*}
Y_{ijk} &= \mu + \alpha_i + \beta_j + \gamma_k + \...
1
vote
1answer
22 views
law of total variance derivation without using variance short-cut formula
In the derivation of the law of total variance from the original variance definition (not using the variance short-cut formula), you add and subtract the term $E(y|x)$, group them to the two terms in ...
1
vote
2answers
34 views
Statistical test to compare means when I have two observations per group?
I am trying to interpret the results of my experiment but am not sure what statistical method I should choose. For my experiment, I added 3 types of different fertilizers to soil, with a control, and ...
0
votes
0answers
19 views
Spearman's correlation coefficient
How calculate manually rs and pvalue for each of the one sided and double sided tests if I have:
N=25
And the D values-
D=6085, D=1001.2 , D=0 , D=2500.3?
I try to use this formula-
and I got these ...
0
votes
0answers
23 views
Different way to do PCA: overall comparison
Given a dataset PCA can be performed via 3 ways:
Eigenvalue decomposition
Singular value decomposition
Non-linear iterative partial least-squares algorithm
Can anyone shed light on comparative study ...
0
votes
0answers
11 views
How to use the mean value theorem [migrated]
I have the following question:
I answered the first part of the question (i) by saying
I am unsure if i have answered correctly and I dont know how to show that for g''(b)
0
votes
0answers
11 views
Proving the Population Mean Squared Error formula from Greene (7th Ed)
On page 93 of Greene 7th edition, there is a statement:
MSE $= E_yE_x[y - x'\gamma]^2$ can be written as $E_{y,x} \{ y-E[y|x] \}^2 + E_{y,x} \{ E[y|x]-x'\gamma] \}^2$.
In order for this to be true, ...
2
votes
0answers
17 views
How to get the training error of Leave-one-out cross-validation?
When I do Leave-one-out cross-validation, I choose an expression that does not require iteration, which is given by
$$CV_{(n)}=\frac{1}{n}\sum_{i=1}^n(\frac{y_i-\hat{y}_i}{1-H_{ii}})^2, (*)$$
where &...
1
vote
0answers
43 views
What is the meaning of these subscripts in iid random variables? [duplicate]
I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
1
vote
1answer
22 views
Show condition so $\beta_{m+1}$ is the solution of weighted least squares problem
In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
0
votes
0answers
12 views
Prove that innovation (residual) of Kalman filter is orthogonal to all future state prediction
The question is to prove that $$v(k) \perp \hat{x}(j|k-1) \space \forall j>k-1$$
where v(k) is innovation (residual) at time k and $\hat{x}(j|k-1)$ is the estimate state at time j given ...
1
vote
1answer
23 views
Understanding numerical example of expectation maximization
I was trying to understand Expectation maximization algorithm. This is how it is defined in Andrew Ng's Stanford CS229 course:
$$
\text{Repeat until convergence \{}\quad\quad\quad\quad\quad\quad\...
0
votes
1answer
25 views
Show that solving this equation is the same as m'th Fisher Scoring step
In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
0
votes
0answers
39 views
Score statistic and Fisher information
In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
0
votes
0answers
20 views
Maximizing the mutual information related to minimizing KL divergence?
Let's say we are drawing two random variables from two distributions, like x~p and z~q,
then maximizing mutual information I(x;z) leads to decreasing KL divergence D_kl(p|q) ?
This sounds correct to ...
0
votes
0answers
11 views
Pareto distribution with adjustable inflection point
How do you generate a Pareto distribution without a given dataset? For example I have 1,000 Widgets and 10 groups. I would like to distribute them among the groups according to the Pareto principle ...
2
votes
1answer
53 views
What does correlation formula really tell you?
The formula for correlation coefficient is as follows:
$$\begin{align}\mathrm{corr} \left(\vec x, \vec y\right) = \frac{1}{n} \sum_{i=1}^n \frac{\left(x_i-\bar x\right)}{\sigma_x} \cdot \frac{\left(...
2
votes
0answers
69 views
What is the space that a class of probability distributions spans when T is a complete sufficient statistic?
There are a few good posts/notes (see here, and here) giving high level geometric intuition of a complete statistic ($E_{T}[g(T); \theta] = 0 \Rightarrow P(g(T)=0; \theta) = 1 \text{ almost everywhere}...
0
votes
0answers
11 views
How to implement the contribution analysis using PCA?
I have been looking into implementing the Q-Residual and Hotelling's T statistics calculation to the PCA components which is similar to the following article and website:
Structural Health Monitoring ...
0
votes
0answers
25 views
Order statistics is minimal sufficient statistics for unknow density function
I'm trying to prove the problem, but there is a problem on definition of term.
The theorem that I use to prove it is
However, what exact meaning of "family of densities ~ all have common support&...
0
votes
0answers
17 views
Simplifying the expected loss integral
I'm currently taking a class on machine learning and we discussed loss functions and minimizing the expected loss to get decision boundaries.
The Profressor gave this integral for calculating the ...
0
votes
0answers
18 views
Estimate assignment probabilities of a multi-class classification problem
I a dataset of $N$ observations $x_1, \ldots, x_N$. We know that to each observation $x_i\in\mathbb{R}^k$ one of $m$ possible class labels has been assigned to it $y_i\in\{1, \ldots, m\}$.
For each ...
0
votes
0answers
5 views
Estimate the square of the response variable for the in-sample observations
experts,
I already have an ML/DL model to predict E[Y|X] and now I want to estimate E[Y^2|X]. I am not sure if I need to run a new model whose feature vector is X and response is Y^2.
Let E[Y|X] = f(X)...
1
vote
1answer
71 views
Statistical modeling vs data mining [duplicate]
What is the difference between "statistical modeling" and "data mining"? I have searched the internet, but I can't see it clearly. Is there any overlap? Can they be considered ...
2
votes
1answer
61 views
Steps in CLT proof unclear
In john rice Mathematical Statistics and Data Analysis we find a proof about the central limit theorem.
Let $X_1, X_2,\ldots$ be a sequence of independent random variables having variance $\sigma^2$ ...
0
votes
1answer
40 views
How can I simply the following Bayes probability?
I know $p(x_2|y)$, and I can calculate $p(x_1|y)$
How can I simplify $p(y|x)=\frac{p(x|y)}{p(x)}\cdot p(y)$ in terms of the above?
Where $x \in \{x_1,x_2\}$
2
votes
0answers
51 views
Does the margin of a classifier scale with the dimension of the problem?
Consider a binary classification problem on $\mathcal Z=\mathcal X \times \mathcal Y $, where $\mathcal X$ is some subset of $\mathbb R^d$ and $\mathcal Y = \{-1;1\}$. You have a dataset of $n$ ...
1
vote
1answer
26 views
Confirmatory Factor Analysis identifying
I have been explained this technique in class. However, I did not understand some stuff.
The professor said that a model to be estimated needs to be at least identified. Identified meaning to be when ...
0
votes
0answers
12 views
Understanding Binomial Classifier
In trying to predict the binary class label $y∈\{0,1\}$ given the input $x$, how can I avoid making assumptions about the data, what are the forms of the distributions and how many parameters do I ...
1
vote
1answer
37 views
Why is the p-value given by p-value${}=P(\chi^2>\Delta D)$ rather than p-value${}=2P(\chi^2>\Delta D)$ as $(*)$?
In the Binomial Regression Models for Binary Data, we have the general Wald statistics result for the Hypothesis $H_0: \beta_j=0, \, H_a: \beta_j \neq 0$ that the p-value is given by
$$\text{p-value}=...
