Questions tagged [estimation]
Any statistical process which seeks to approximate an unknown value, such as a distribution, a point estimate (e.g. mean), or confidence interval.
2,269 questions
0
votes
1answer
20 views
arbitrariness in bootstrap bias estimation
The bootstrap estimates bias by applying the "plug-in" principle to $$E(\hat{\theta}_n) - \theta$$
I got this knowledge from p.124 of Efron, Tibshirani, 1994.
equation(10.1) $\text{bias}_F=E_F[s(\...
0
votes
0answers
34 views
Question regarding MLE method
Let $Y$ be a linear function of $X \sim G[p]$, a geometric random variable, such that $Y=100-10X$. Given $\{Y_i\}^n_{i=1}$ observations find $\hat E(Y)_{MLE}$. I tried the following method:
First, ...
1
vote
0answers
23 views
How does one estimate parameters in a GARCH-M(1,1) model?
Say you have a GARCH-M(1,1) model as follows:
$y_t = \beta y_{t-1} + \delta h_t + \epsilon_t, \quad \epsilon_t \sim N(0, h_t) $
$h_t = a_0 + a_1 \epsilon^2_{t-1} + b_1 h_{t-1}.$
How exactly does ...
1
vote
0answers
17 views
How to optimize the values of parameters of a distribution? [on hold]
Let $X$ and $Y$ be two sequences of random variables, where
$X$ is the sequence of real, observed values that is assumed to follow a gamma distribution as in $Gamma(\alpha^*, \beta^*)$. Thus, $X = {...
0
votes
0answers
32 views
Consistency of estimators
For $1\leq i\leq K$, I have an estimator of $\mu_{i}$ given by $\hat{\mu}_{i}=\frac{1}{K}\sum_{j\neq i=1}^{K}\frac{Y_{ij}}{n_{ij}}$, where $Y_{ij}\sim N(n_{ij}(\mu_{i}-\mu_{j}),\sigma^{2}n_{ij})$. ...
1
vote
0answers
16 views
Understanding Rao-Blackwell [duplicate]
From Casella and Berger:
Let $W$ be an unbiased estimator of $\tau(\theta)$ and let $T$ be a sufficient statistics for $\theta$. Define $\phi(T) = E[W|T]$. Then $E_{\theta}[ \phi(T)] = \tau(\theta)$ ...
0
votes
0answers
34 views
Estimating sample variance
Suppose that we have 2 independent samples $X_{11}, X_{12},.., X_{1n_1}$ and $X_{21}, X_{22},.., X_{2n_2}$ from a normally distributed population with $n_1<n_2$. Does that mean that the sample ...
0
votes
0answers
10 views
Effect of estimate of one variable on estimate of other variable
I have a linear equation: as Y = -4 + 5X1 + 0.9X2 +0.5X3;
Suppose correlation between X1 and X2 is 0.7. Since X1 has more predictive power than X2, regression picked X1 with more weightage. X2 was ...
0
votes
0answers
14 views
How can I not show the initialization of the estimation in the Extended Kalman Filter?
I'm making estimates through the Extended Kalman Filter and I have a problem related to the vertical axis of my figure, it's too big, so I can not see population dynamics. However, I wish it did not ...
0
votes
0answers
11 views
Identification and estimation of my structural model with a latent variable
I am having some trouble trying to identify the parameters in the following structural model that I am trying to estimate.
$$
y = a'x_1 + \beta\eta + \epsilon_1
$$
$$
\eta = b'x_2 + \delta T+\...
0
votes
1answer
59 views
+50
Iterated estimation of Taylor series
Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
1
vote
0answers
35 views
Overfitting decreases over time
During my training I have got the following results after 10th epoch:
training accuracy amounts to 97% and is stable;
training loss is decreasing from 0.2530 and is decreasing 0.02-0.05 per epoch, ...
0
votes
0answers
15 views
Estimate probability from sample frequency in a binomial distribution [duplicate]
If I get $s$ successes out of $n$ trials in a binomial distribution, what is the probability $p$ of getting a success in each individual trial?
Presumably $p = s/n$, but what if $s = 0$ or $s = n$? ...
0
votes
0answers
9 views
Fisher information under different noise models
Directly lifted from Wikipedia: Fisher information (sometimes simply called information) is a way of measuring the amount of information that an observable random variable $X$ carries about an unknown ...
5
votes
1answer
92 views
+100
Estimating the mean with previous knowledge
I have an unknown (discrete) probability distribution $p=\{p_s\}$, where $p_s$ is the probability of observing configuration $s$. To each configuration is associated an energy that I can compute $E_s$....
1
vote
1answer
33 views
Estimator of ratio of central moments
In the context of Control Variates one has to estimate, for example, the following ratios of central moments:
$$\frac{\mu_{1,1}}{\mu_{0,2}} \quad \text{and} \quad \frac{\mu_{1,1}^2}{\mu_{0,2}}$$
...
0
votes
1answer
41 views
How to find an unbiased estimator of $\mathsf{Uniform}(-\theta/2,\theta/2)$
How to find an unbiased estimator of $\mathsf{Uniform}(-\theta/2,\theta/2)$.
Is it a function of the order statistics?
0
votes
0answers
12 views
On the estimation of target function
I am beginner to machine learning and study it from Mitchels book. In page 10 of it they try to estimate target function $V$. So they suppose that $\hat{V}$ is the last estimation of $V$. They ...
3
votes
1answer
79 views
How to calculate standardized coefficients from an estimated multiple regression model
I have a multiple regression model(original model) that has been estimated already, and the details on the mean and standard deviations of the regressors, and the standard errors of coefficients from ...
1
vote
1answer
27 views
Is there a collinearity issue when using: x, dummy indicating extreme negative value of x and their interaction?
I was wondering whether I can build my baseline model using the following variables without incurring in any multicollinearity issue:
$X_1$= Net capital flows over GDP (which may be positive and ...
0
votes
0answers
21 views
Overcoming the problem with matrices in estimating model parameters for disaggregation model
Loucks et al. (1981) defines the basic disaggregation models as
${\bf X_y = AZ_y + BV_y } $
where
${\bf Z_y} = (Z^1_y, ...,Z^N_y)^T $ is the vector of $N$ transformed normally distributed annual ...
0
votes
0answers
16 views
Choose between ratio of estimators or estimator from the ratio of data
I have to estimate a function which is, say, the proportion of people under 20 in each point of a given territory. Let's call that $h(x,y)$ for $(x,y)$ in my territory.
To get that, I have, for every ...
3
votes
0answers
47 views
Posterior mean estimator with MCMC (Metropolis Hastings Algorithm) - Concrete example
I have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case).
I have the 6 parameters to estimate : $p=(\...
0
votes
0answers
15 views
Estimating hidden parameter controlling bias of coin
I have a strange coin whose bias $p$ (probability of heads) depends on 2 environmental factors $a, b\in\mathbb{R}$ which I can control at will, and an innate hidden parameter $\theta$:
$p(a,b,\theta) ...
1
vote
1answer
31 views
Finding a UMVU estimator for the mean
I am struggling with the following problem.
We are given an i.i.d sample of size $n,$ with the form $X_{i}=\mu+n_{i}$, where $\mu$ is a deterministic unknown constant, and $n_{i}$ is a noise with a ...
3
votes
2answers
114 views
Optimizing $\chi^2$ using MCMC
I have measurements of an object.
Let's say I have its length $L$, mass $M$, and age $t$: $$\mathbf y = (10~\text{m},\ 0.01~\text{g},\ 5~\text{s}).$$ I also have the uncertainties on my measurements ...
1
vote
1answer
46 views
Proof that posterior median is the Bayes estimate of absolute loss?
It is always argued that the posterior median is the Bayes estimate associated to the absolute loss function. The proofs I have come across rely on differentiating the conditional Bayesian risk and ...
0
votes
1answer
29 views
Diff in diff model with multiple treatments in multiple perdiods?
Can I estimate a diff in diff model to compare the effects of two different treatments that apply in different time periods in different countries?
I have 30 countries for an average time span of 34 ...
1
vote
2answers
54 views
Method of moments for linear regression?
I have been reading about the method of moments, and now I understand how to obtain the method of moments estimator for a random sample $x_1,...,x_n$ from a distribution $f(x;\theta)$, in the ...
3
votes
1answer
46 views
Finding MLE of $p$ where $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$
Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$
be independent Bernoulli random variables where $p\in[0,1/3]$. Derive
the MLE of $p$.
We have that
$$L(p\mid \vec{x})=p^{x_1}(1-...
5
votes
0answers
74 views
Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$
Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf
$$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 &
x\lt\theta \end{cases}$$
where $\theta \...
0
votes
1answer
60 views
Confidence limits for constrained penalized log likelihood model
I am estimating parameter $\beta$ as:
\begin{align}
\hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
0
votes
0answers
12 views
Restricting Properties of Point Estimators in Discrete, Unordered Parameter Spaces
A common principle used to deal with the fact that there are typically no uniformly best estimators (in the sense that they uniformly have least frequentist risk) seems to be to restrict the space of ...
0
votes
0answers
16 views
Standard Errors for estimates obtained from custom objective function
I want to find standard errors of estimates obtained by optimizing a custom objective function, which I will explain further below:
The objective function is: $L(a,b|y,X_1) - \lambda \times RSS(f(y),...
3
votes
1answer
69 views
Bias in estimators of population variance
Let $\{ X_i | i = 1, 2, . . . ,n \}$ be a sequence of independent and identically distributed (IID) random variables from a population and define $\mu \equiv \mathbb{E}(X)$ and $\sigma^2 \equiv \...
1
vote
0answers
62 views
Finding the pdf of the posterior distribution and the corresponding Bayes Estimate of $\theta$
Suppose that we observe i.i.d random variables $X_1, X_2, \ldots , X_n$ having pmf
$$f_{X}(x\mid\theta) =\theta(1−\theta)^{x−1}I_{\{1,2,3,\ldots\}}(x)$$
where $\theta\in(0,1)$. Consider the ...
0
votes
0answers
23 views
Cross tab with missing values
I have a following kind of problem.
I have several countries. For every country I know the cumulative value of the variable of interest after four periods. I also know the cross-country sum for every ...
0
votes
0answers
13 views
Estimability of 2way ANOVA
Consider the model,
yijk = µ + Ai + Bij + eijk ; i, j, k= 1,...,5
I tried testing for estimability by trying to reducing the coefficient matrix. But the matrix is really big that it's getting ...
2
votes
1answer
58 views
Admissibility does not imply minimax
The answer to minimax estimator explains why minimax does not imply admissibility. The relevant statement is from https://www.stat.berkeley.edu/~yuekai/201b/lec6.pdf which says,
minimaxity does not ...
1
vote
1answer
35 views
GARCH specification - why are $\sigma_t^2$ and $\epsilon_t^2$ not the same?
Often times people specify the GARCH model as follows:
$$ \sigma _{t}^{2}=\omega +\alpha _{1}\epsilon _{t-1}^{2}+\cdots +\alpha _{q}\epsilon _{t-q}^{2}+\beta _{1}\sigma _{t-1}^{2}+\cdots +\beta _{p}\...
1
vote
1answer
49 views
Maximum likelihood estimator for a mixture of 2 distributions
Let $X_1, ..., X_n$ be iid with one of two PDFs.
If $\theta = 0$, then
$f(x; \theta) = 1, \ 0 < x < 1$.
if $\theta = 1$, then
$f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$.
What ...
0
votes
1answer
32 views
Test for same coefficients with different estimation approaches
I'm estimating coefficients $w_1,\dots, w_d$ with different approaches (for example with ML and with leave one out CV) a few times. Now I'd like to test if the coefficients are the same, but I dont ...
2
votes
0answers
14 views
Theory of convergence for this case?
I hope you can guide me to the right place.
I am estimating two equations, where each equation produces an output which is needed in the other equation as input. Something like this:
$Y_t = AX_t + f(...
0
votes
0answers
19 views
Estimability in Design model
consider the design model $y=\theta+e$
I know we can obtain the normal equations from observation to estimate the parameters. my question is- is the estimation BLUE?
Given normal equations are:
$...
1
vote
1answer
29 views
Estimation the evolution of error variance with time?
I have data tuples $(x_i,\varepsilon_i,t_i)$ generated from some observations and I suspect that $\varepsilon \sim \mathcal{N}\left(0,\sigma(t)^2\right)$, where $\sigma(t)$ is an increasing function ...
1
vote
1answer
21 views
Probability for mean of a subpopulation
Suppose I have drawn 21 samples from a population which I assume to be normal, where the sample has mean 3.8 and sample standard deviation 0.7.
What is the probability that the mean of the next 210 ...
4
votes
2answers
299 views
Estimating the MLE where the parameter is also the constraint
Independent random variables $X_1,X_2,\ldots,X_n \sim f_X$ are modeled with a common density
$$f_X(x) = \frac{\alpha(x/\beta)^{\alpha-1}}{\beta} \quad \quad \quad \text{for all } 0 \le x \le \beta.$$
...
0
votes
0answers
14 views
Bias of the eigenvalues of sample covariance matrix
Consider and i.i.d sample $X_1, \ldots X_n$ in $\mathbb{R}^p$ with covariance matrix $\Sigma \in \mathbb R^{p\times p}$
What is the expectation of the eigenvalues of the sample covariance matrix?. ...
0
votes
1answer
24 views
Estimating the variance of the GLS estimator
Consider the linear regression model where $y = XB + u$. Assume that $\mathrm{E}[u \mid X] = 0$. Assume that $\mathrm{V}[u \mid X] = \sigma^2I$. This is the simple linear model. Now relax the ...
0
votes
1answer
90 views
Maximum likelihood estimator for Bernoulli parameter based on standard normal
$X_i \sim Normal(\psi,1), \ \ i = 1, ..., n$
$Y_i = 1$ if $X_i \ge 0.$
$Y_i = 0$ if $X_i < 0.$
Let $\theta = P(Y_i = 1)$.
What is the MLE of $\theta$?
I know how to find the MLE of a Bernoulli ...
