Questions tagged [mathematical-statistics]

Mathematical theory of statistics, concerned with formal definitions and general results.

Filter by
Sorted by
Tagged with
0
votes
0answers
56 views

Probability Mass Functions [closed]

I have asked a related question Probability Mass Function making the Truncated Normal Discrete I am trying to understand the difference between 2 answers (the difference between 2 PMFs given). What is ...
0
votes
1answer
48 views

Bayes error and nearest neighbor classification

Upon studying for my midterm using Pattern Classification by Richard O. Duda, David G. Stork, Peter E.Hart (2001), I stumbled upon the following exercise: Using the solutions manual written by David ...
0
votes
0answers
16 views

What kind of regression to use in predictive model for a positive-only value

I'm trying to use regression in Excel to predict the % of bin contamination (dependant variable), based on my independent variables that are generally relating to demographics such as age/household ...
-1
votes
0answers
21 views

Relationship among exponential family, natural exponential family and exponential dispersion model

I have read the wikipedia of these three notions: (1)exponential family (2)natural exponential family (3)exponential dispersion model I have known the following relationship among them: (1)The natural ...
0
votes
2answers
80 views

Linear regression residuals and OLS method

Assuming a simple linear regression framework, $y_i=\beta_{0}+\beta_{1}.x_{i} + \epsilon_{i}$, why can't I estimate $\beta_{0}$ and $\beta_{1}$ by optimizing the sum of residuals such that the sum is ...
1
vote
2answers
112 views

Why does regression to the mean work?

According to Wikipedia, regression towards the mean is "the phenomenon that arises if a sample point of a random variable is extreme (nearly an outlier), in which case a future point is likely to ...
1
vote
0answers
38 views

Comparing performance of Quasi-binomial model and Beta-binomial model

I read some books in biostatistics about fitting binary date with Beta-Binomial regression model and Quasi-Binomial regression model. It proposes a setting: Setting: Assuming we have a sequence of ...
1
vote
0answers
11 views

Why duality of confidence interval and hypothesis testing fails in random effect test for presence random effect?

I am reading Faraway's Extending the linear model with R chapter 10 section 2. "In this case, the lower bound is zero. This is not surprising given our earlier uncertainty over whether there ...
0
votes
0answers
14 views

Confidence interval of a sample of a survey

I am having trouble framing my question and identifying variables because of sub-populations. If I have a survey of 10,000 people from a population of 100,000 people and within that survey 2,500 leave ...
0
votes
0answers
33 views

model output without correlation to input variable

let's assume that a,b,c,d,t,u,v,w,x,y,z and o are all correlated to one another. I want to make a prediction for outcome variable "o". However, I don't want my prediction to be to ...
0
votes
0answers
33 views

Global minimum of prediction intervals for multiple linear regression

I'm trying to solve the following problem: Prove that the width of the prediction interval for $\hat{y}$, given the vector $x_0 = (1, x_1,...,x_{p-1})$ of explanatory variables, reaches a global ...
0
votes
0answers
20 views

High value for SD for data in range -100 to +250?

I Have a 1D data . I am trying to approximate this data to a gaussian distribution. I want to use MLE to estimate the parameters. ...
0
votes
1answer
27 views

Are the position indices (quartiles, median, mean) of distribution in classes always different from those of the unit distribution?

Are the position indices (quartiles, median, mean) of distribution in classes always different from those of the unit distribution? If so, for what reason? In theory, since the class distribution is a ...
0
votes
0answers
32 views

Asymptotic distribution of a method of moment estimator via the delta method

Let $X_1,...,X_n$ be a random sample from X with a Beta distribution $B(\theta; 2)$, with $\theta > 0$, i.e. : \begin{align*} f(x,\theta) = \theta(\theta+1)x^{\theta-1}(1-x) \mathbb{I}_{0<x&...
0
votes
1answer
17 views

Confusion surrounding matrix multiplication rules used in proof

Let $\mathbf{Z}$ be the design matrix of a linear regression. Where $\mathbf{Z}^{T}\mathbf{Z}$ is symmetric. Set $\mathbf{V} = (\mathbf{Z'Z})^{\frac{1}{2}}(E(\hat{\beta}) - \beta)$ Where $\hat{\beta} ,...
0
votes
1answer
30 views

Interpreting the total-variance when the model lies over our data

I know that: $Variance_{Total}=Variance_{Explained} + Variance_{Unexplained}$, but I am wondering how the $Variance_{Total}$ relates to the $Variance_{Unexplained}$ and the $Variance_{Explained}$ if ...
0
votes
0answers
22 views

Understanding the notion of best estimators between 2 randoms variables with a weighting

A colleague put the mess in my head following a discussion about what he calls the "best weighting" from a statistic point of view when we treat 2 random variables. He told me that if we ...
0
votes
1answer
30 views

Confusion surrounding differentiation of parameter vector ridge regression

Let $\mathbf{\beta}$ be the parameter vector of a ridge regression. Now we can say that: \begin{equation} \frac{\partial \lambda \beta^T \beta}{\partial \beta}=2\lambda\beta. \end{equation} Why is ...
0
votes
0answers
19 views

What effect would clusters get when adding more variables to clustering task?

I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
1
vote
1answer
52 views

ANOVA model assumptions

Suppose the following: $X_{lj}$ ~ $N(\mu_{l} , \sigma^{2})$ Where $l = 1, ...g$ and $j = 1,..n_{g}$ Furthermore assume $\sigma^{2}_{1} = \sigma^{2}_{2}...\sigma^{2}_{g} = \sigma$ Then we can define ...
1
vote
0answers
14 views

How do I write sum of squares and degrees of freedom for multiple Latin squares put together?

I have notes from my professor in which a Latin square model ($s$ treatments in an $s \times s$ square) is described as follows: \begin{align*} Y_{ijk} &= \mu + \alpha_i + \beta_j + \gamma_k + \...
1
vote
1answer
22 views

law of total variance derivation without using variance short-cut formula

In the derivation of the law of total variance from the original variance definition (not using the variance short-cut formula), you add and subtract the term $E(y|x)$, group them to the two terms in ...
1
vote
2answers
34 views

Statistical test to compare means when I have two observations per group?

I am trying to interpret the results of my experiment but am not sure what statistical method I should choose. For my experiment, I added 3 types of different fertilizers to soil, with a control, and ...
0
votes
0answers
19 views

Spearman's correlation coefficient

How calculate manually rs and pvalue for each of the one sided and double sided tests if I have: N=25 And the D values- D=6085, D=1001.2 , D=0 , D=2500.3? I try to use this formula- and I got these ...
0
votes
0answers
23 views

Different way to do PCA: overall comparison

Given a dataset PCA can be performed via 3 ways: Eigenvalue decomposition Singular value decomposition Non-linear iterative partial least-squares algorithm Can anyone shed light on comparative study ...
0
votes
0answers
11 views

How to use the mean value theorem [migrated]

I have the following question: I answered the first part of the question (i) by saying I am unsure if i have answered correctly and I dont know how to show that for g''(b)
0
votes
0answers
11 views

Proving the Population Mean Squared Error formula from Greene (7th Ed)

On page 93 of Greene 7th edition, there is a statement: MSE $= E_yE_x[y - x'\gamma]^2$ can be written as $E_{y,x} \{ y-E[y|x] \}^2 + E_{y,x} \{ E[y|x]-x'\gamma] \}^2$. In order for this to be true, ...
2
votes
0answers
17 views

How to get the training error of Leave-one-out cross-validation?

When I do Leave-one-out cross-validation, I choose an expression that does not require iteration, which is given by $$CV_{(n)}=\frac{1}{n}\sum_{i=1}^n(\frac{y_i-\hat{y}_i}{1-H_{ii}})^2, (*)$$ where &...
1
vote
0answers
43 views

What is the meaning of these subscripts in iid random variables? [duplicate]

I've been studying statistics on my own and I'm having a hard time understanding some notations. On this page: http://scipp.ucsc.edu/~haber/ph116C/iid.pdf, specifically on the second paragraph, the ...
1
vote
1answer
22 views

Show condition so $\beta_{m+1}$ is the solution of weighted least squares problem

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
0
votes
0answers
12 views

Prove that innovation (residual) of Kalman filter is orthogonal to all future state prediction

The question is to prove that $$v(k) \perp \hat{x}(j|k-1) \space \forall j>k-1$$ where v(k) is innovation (residual) at time k and $\hat{x}(j|k-1)$ is the estimate state at time j given ...
1
vote
1answer
23 views

Understanding numerical example of expectation maximization

I was trying to understand Expectation maximization algorithm. This is how it is defined in Andrew Ng's Stanford CS229 course: $$ \text{Repeat until convergence \{}\quad\quad\quad\quad\quad\quad\...
0
votes
1answer
25 views

Show that solving this equation is the same as m'th Fisher Scoring step

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
0
votes
0answers
39 views

Score statistic and Fisher information

In my exercise we assume that $Y_i|X_i$ has distribution with density $f_i(y_i,\eta_i) $ for $i=1,...,n$ where $\eta_i=X_i^T$ is the linear predictor. The generalized linear model with an exponential ...
0
votes
0answers
20 views

Maximizing the mutual information related to minimizing KL divergence?

Let's say we are drawing two random variables from two distributions, like x~p and z~q, then maximizing mutual information I(x;z) leads to decreasing KL divergence D_kl(p|q) ? This sounds correct to ...
0
votes
0answers
11 views

Pareto distribution with adjustable inflection point

How do you generate a Pareto distribution without a given dataset? For example I have 1,000 Widgets and 10 groups. I would like to distribute them among the groups according to the Pareto principle ...
2
votes
1answer
53 views

What does correlation formula really tell you?

The formula for correlation coefficient is as follows: $$\begin{align}\mathrm{corr} \left(\vec x, \vec y\right) = \frac{1}{n} \sum_{i=1}^n \frac{\left(x_i-\bar x\right)}{\sigma_x} \cdot \frac{\left(...
2
votes
0answers
69 views

What is the space that a class of probability distributions spans when T is a complete sufficient statistic?

There are a few good posts/notes (see here, and here) giving high level geometric intuition of a complete statistic ($E_{T}[g(T); \theta] = 0 \Rightarrow P(g(T)=0; \theta) = 1 \text{ almost everywhere}...
0
votes
0answers
11 views

How to implement the contribution analysis using PCA?

I have been looking into implementing the Q-Residual and Hotelling's T statistics calculation to the PCA components which is similar to the following article and website: Structural Health Monitoring ...
0
votes
0answers
25 views

Order statistics is minimal sufficient statistics for unknow density function

I'm trying to prove the problem, but there is a problem on definition of term. The theorem that I use to prove it is However, what exact meaning of "family of densities ~ all have common support&...
0
votes
0answers
17 views

Simplifying the expected loss integral

I'm currently taking a class on machine learning and we discussed loss functions and minimizing the expected loss to get decision boundaries. The Profressor gave this integral for calculating the ...
0
votes
0answers
18 views

Estimate assignment probabilities of a multi-class classification problem

I a dataset of $N$ observations $x_1, \ldots, x_N$. We know that to each observation $x_i\in\mathbb{R}^k$ one of $m$ possible class labels has been assigned to it $y_i\in\{1, \ldots, m\}$. For each ...
0
votes
0answers
5 views

Estimate the square of the response variable for the in-sample observations

experts, I already have an ML/DL model to predict E[Y|X] and now I want to estimate E[Y^2|X]. I am not sure if I need to run a new model whose feature vector is X and response is Y^2. Let E[Y|X] = f(X)...
1
vote
1answer
71 views

Statistical modeling vs data mining [duplicate]

What is the difference between "statistical modeling" and "data mining"? I have searched the internet, but I can't see it clearly. Is there any overlap? Can they be considered ...
2
votes
1answer
61 views

Steps in CLT proof unclear

In john rice Mathematical Statistics and Data Analysis we find a proof about the central limit theorem. Let $X_1, X_2,\ldots$ be a sequence of independent random variables having variance $\sigma^2$ ...
0
votes
1answer
40 views

How can I simply the following Bayes probability?

I know $p(x_2|y)$, and I can calculate $p(x_1|y)$ How can I simplify $p(y|x)=\frac{p(x|y)}{p(x)}\cdot p(y)$ in terms of the above? Where $x \in \{x_1,x_2\}$
2
votes
0answers
51 views

Does the margin of a classifier scale with the dimension of the problem?

Consider a binary classification problem on $\mathcal Z=\mathcal X \times \mathcal Y $, where $\mathcal X$ is some subset of $\mathbb R^d$ and $\mathcal Y = \{-1;1\}$. You have a dataset of $n$ ...
1
vote
1answer
26 views

Confirmatory Factor Analysis identifying

I have been explained this technique in class. However, I did not understand some stuff. The professor said that a model to be estimated needs to be at least identified. Identified meaning to be when ...
0
votes
0answers
12 views

Understanding Binomial Classifier

In trying to predict the binary class label $y∈\{0,1\}$ given the input $x$, how can I avoid making assumptions about the data, what are the forms of the distributions and how many parameters do I ...
1
vote
1answer
37 views

Why is the p-value given by p-value${}=P(\chi^2>\Delta D)$ rather than p-value${}=2P(\chi^2>\Delta D)$ as $(*)$?

In the Binomial Regression Models for Binary Data, we have the general Wald statistics result for the Hypothesis $H_0: \beta_j=0, \, H_a: \beta_j \neq 0$ that the p-value is given by $$\text{p-value}=...