Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [estimation]

Any statistical process which seeks to approximate an unknown value, such as a distribution, a point estimate (e.g. mean), or confidence interval.

0
votes
1answer
20 views

arbitrariness in bootstrap bias estimation

The bootstrap estimates bias by applying the "plug-in" principle to $$E(\hat{\theta}_n) - \theta$$ I got this knowledge from p.124 of Efron, Tibshirani, 1994. equation(10.1) $\text{bias}_F=E_F[s(\...
0
votes
0answers
34 views

Question regarding MLE method

Let $Y$ be a linear function of $X \sim G[p]$, a geometric random variable, such that $Y=100-10X$. Given $\{Y_i\}^n_{i=1}$ observations find $\hat E(Y)_{MLE}$. I tried the following method: First, ...
1
vote
0answers
23 views

How does one estimate parameters in a GARCH-M(1,1) model?

Say you have a GARCH-M(1,1) model as follows: $y_t = \beta y_{t-1} + \delta h_t + \epsilon_t, \quad \epsilon_t \sim N(0, h_t) $ $h_t = a_0 + a_1 \epsilon^2_{t-1} + b_1 h_{t-1}.$ How exactly does ...
1
vote
0answers
17 views

How to optimize the values of parameters of a distribution? [on hold]

Let $X$ and $Y$ be two sequences of random variables, where $X$ is the sequence of real, observed values that is assumed to follow a gamma distribution as in $Gamma(\alpha^*, \beta^*)$. Thus, $X = {...
0
votes
0answers
32 views

Consistency of estimators

For $1\leq i\leq K$, I have an estimator of $\mu_{i}$ given by $\hat{\mu}_{i}=\frac{1}{K}\sum_{j\neq i=1}^{K}\frac{Y_{ij}}{n_{ij}}$, where $Y_{ij}\sim N(n_{ij}(\mu_{i}-\mu_{j}),\sigma^{2}n_{ij})$. ...
1
vote
0answers
16 views

Understanding Rao-Blackwell [duplicate]

From Casella and Berger: Let $W$ be an unbiased estimator of $\tau(\theta)$ and let $T$ be a sufficient statistics for $\theta$. Define $\phi(T) = E[W|T]$. Then $E_{\theta}[ \phi(T)] = \tau(\theta)$ ...
0
votes
0answers
34 views

Estimating sample variance

Suppose that we have 2 independent samples $X_{11}, X_{12},.., X_{1n_1}$ and $X_{21}, X_{22},.., X_{2n_2}$ from a normally distributed population with $n_1<n_2$. Does that mean that the sample ...
0
votes
0answers
10 views

Effect of estimate of one variable on estimate of other variable

I have a linear equation: as Y = -4 + 5X1 + 0.9X2 +0.5X3; Suppose correlation between X1 and X2 is 0.7. Since X1 has more predictive power than X2, regression picked X1 with more weightage. X2 was ...
0
votes
0answers
14 views

How can I not show the initialization of the estimation in the Extended Kalman Filter?

I'm making estimates through the Extended Kalman Filter and I have a problem related to the vertical axis of my figure, it's too big, so I can not see population dynamics. However, I wish it did not ...
0
votes
0answers
11 views

Identification and estimation of my structural model with a latent variable

I am having some trouble trying to identify the parameters in the following structural model that I am trying to estimate. $$ y = a'x_1 + \beta\eta + \epsilon_1 $$ $$ \eta = b'x_2 + \delta T+\...
0
votes
1answer
59 views
+50

Iterated estimation of Taylor series

Say your data generating process is given by the function $y=f(x|\theta)$, where $y$ and $x$ represent variables (data) and $\theta$ represent parameter(s). For convergence reasons (e.g. $f(\cdot)$ is ...
1
vote
0answers
35 views

Overfitting decreases over time

During my training I have got the following results after 10th epoch: training accuracy amounts to 97% and is stable; training loss is decreasing from 0.2530 and is decreasing 0.02-0.05 per epoch, ...
0
votes
0answers
15 views

Estimate probability from sample frequency in a binomial distribution [duplicate]

If I get $s$ successes out of $n$ trials in a binomial distribution, what is the probability $p$ of getting a success in each individual trial? Presumably $p = s/n$, but what if $s = 0$ or $s = n$? ...
0
votes
0answers
9 views

Fisher information under different noise models

Directly lifted from Wikipedia: Fisher information (sometimes simply called information) is a way of measuring the amount of information that an observable random variable $X$ carries about an unknown ...
5
votes
1answer
92 views
+100

Estimating the mean with previous knowledge

I have an unknown (discrete) probability distribution $p=\{p_s\}$, where $p_s$ is the probability of observing configuration $s$. To each configuration is associated an energy that I can compute $E_s$....
1
vote
1answer
33 views

Estimator of ratio of central moments

In the context of Control Variates one has to estimate, for example, the following ratios of central moments: $$\frac{\mu_{1,1}}{\mu_{0,2}} \quad \text{and} \quad \frac{\mu_{1,1}^2}{\mu_{0,2}}$$ ...
0
votes
1answer
41 views

How to find an unbiased estimator of $\mathsf{Uniform}(-\theta/2,\theta/2)$

How to find an unbiased estimator of $\mathsf{Uniform}(-\theta/2,\theta/2)$. Is it a function of the order statistics?
0
votes
0answers
12 views

On the estimation of target function

I am beginner to machine learning and study it from Mitchels book. In page 10 of it they try to estimate target function $V$. So they suppose that $\hat{V}$ is the last estimation of $V$. They ...
3
votes
1answer
79 views

How to calculate standardized coefficients from an estimated multiple regression model

I have a multiple regression model(original model) that has been estimated already, and the details on the mean and standard deviations of the regressors, and the standard errors of coefficients from ...
1
vote
1answer
27 views

Is there a collinearity issue when using: x, dummy indicating extreme negative value of x and their interaction?

I was wondering whether I can build my baseline model using the following variables without incurring in any multicollinearity issue: $X_1$= Net capital flows over GDP (which may be positive and ...
0
votes
0answers
21 views

Overcoming the problem with matrices in estimating model parameters for disaggregation model

Loucks et al. (1981) defines the basic disaggregation models as ${\bf X_y = AZ_y + BV_y } $ where ${\bf Z_y} = (Z^1_y, ...,Z^N_y)^T $ is the vector of $N$ transformed normally distributed annual ...
0
votes
0answers
16 views

Choose between ratio of estimators or estimator from the ratio of data

I have to estimate a function which is, say, the proportion of people under 20 in each point of a given territory. Let's call that $h(x,y)$ for $(x,y)$ in my territory. To get that, I have, for every ...
3
votes
0answers
47 views

Posterior mean estimator with MCMC (Metropolis Hastings Algorithm) - Concrete example

I have a little project for which I have to estimate parameters on a PSF (Point Spread Function = response of the system to a dirac, i.e a star in my case). I have the 6 parameters to estimate : $p=(\...
0
votes
0answers
15 views

Estimating hidden parameter controlling bias of coin

I have a strange coin whose bias $p$ (probability of heads) depends on 2 environmental factors $a, b\in\mathbb{R}$ which I can control at will, and an innate hidden parameter $\theta$: $p(a,b,\theta) ...
1
vote
1answer
31 views

Finding a UMVU estimator for the mean

I am struggling with the following problem. We are given an i.i.d sample of size $n,$ with the form $X_{i}=\mu+n_{i}$, where $\mu$ is a deterministic unknown constant, and $n_{i}$ is a noise with a ...
3
votes
2answers
114 views

Optimizing $\chi^2$ using MCMC

I have measurements of an object. Let's say I have its length $L$, mass $M$, and age $t$: $$\mathbf y = (10~\text{m},\ 0.01~\text{g},\ 5~\text{s}).$$ I also have the uncertainties on my measurements ...
1
vote
1answer
46 views

Proof that posterior median is the Bayes estimate of absolute loss?

It is always argued that the posterior median is the Bayes estimate associated to the absolute loss function. The proofs I have come across rely on differentiating the conditional Bayesian risk and ...
0
votes
1answer
29 views

Diff in diff model with multiple treatments in multiple perdiods?

Can I estimate a diff in diff model to compare the effects of two different treatments that apply in different time periods in different countries? I have 30 countries for an average time span of 34 ...
1
vote
2answers
54 views

Method of moments for linear regression?

I have been reading about the method of moments, and now I understand how to obtain the method of moments estimator for a random sample $x_1,...,x_n$ from a distribution $f(x;\theta)$, in the ...
3
votes
1answer
46 views

Finding MLE of $p$ where $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$

Let $X_1\sim\text{Bernoulli}(p)$ and $X_2\sim\text{Bernoulli}(3p)$ be independent Bernoulli random variables where $p\in[0,1/3]$. Derive the MLE of $p$. We have that $$L(p\mid \vec{x})=p^{x_1}(1-...
5
votes
0answers
74 views

Finding MLE and MSE of $\theta$ where $f_X(x\mid\theta)=\theta x^{−2} I_{x\geq\theta}(x)$

Consider i.i.d random variables $X_1$, $X_2$, . . . , $X_n$ having pdf $$f_X(x\mid\theta) = \begin{cases} \theta x^{−2} & x\geq\theta \\ 0 & x\lt\theta \end{cases}$$ where $\theta \...
0
votes
1answer
60 views

Confidence limits for constrained penalized log likelihood model

I am estimating parameter $\beta$ as: \begin{align} \hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
0
votes
0answers
12 views

Restricting Properties of Point Estimators in Discrete, Unordered Parameter Spaces

A common principle used to deal with the fact that there are typically no uniformly best estimators (in the sense that they uniformly have least frequentist risk) seems to be to restrict the space of ...
0
votes
0answers
16 views

Standard Errors for estimates obtained from custom objective function

I want to find standard errors of estimates obtained by optimizing a custom objective function, which I will explain further below: The objective function is: $L(a,b|y,X_1) - \lambda \times RSS(f(y),...
3
votes
1answer
69 views

Bias in estimators of population variance

Let $\{ X_i | i = 1, 2, . . . ,n \}$ be a sequence of independent and identically distributed (IID) random variables from a population and define $\mu \equiv \mathbb{E}(X)$ and $\sigma^2 \equiv \...
1
vote
0answers
62 views

Finding the pdf of the posterior distribution and the corresponding Bayes Estimate of $\theta$

Suppose that we observe i.i.d random variables $X_1, X_2, \ldots , X_n$ having pmf $$f_{X}(x\mid\theta) =\theta(1−\theta)^{x−1}I_{\{1,2,3,\ldots\}}(x)$$ where $\theta\in(0,1)$. Consider the ...
0
votes
0answers
23 views

Cross tab with missing values

I have a following kind of problem. I have several countries. For every country I know the cumulative value of the variable of interest after four periods. I also know the cross-country sum for every ...
0
votes
0answers
13 views

Estimability of 2way ANOVA

Consider the model, yijk = µ + Ai + Bij + eijk ; i, j, k= 1,...,5 I tried testing for estimability by trying to reducing the coefficient matrix. But the matrix is really big that it's getting ...
2
votes
1answer
58 views

Admissibility does not imply minimax

The answer to minimax estimator explains why minimax does not imply admissibility. The relevant statement is from https://www.stat.berkeley.edu/~yuekai/201b/lec6.pdf which says, minimaxity does not ...
1
vote
1answer
35 views

GARCH specification - why are $\sigma_t^2$ and $\epsilon_t^2$ not the same?

Often times people specify the GARCH model as follows: $$ \sigma _{t}^{2}=\omega +\alpha _{1}\epsilon _{t-1}^{2}+\cdots +\alpha _{q}\epsilon _{t-q}^{2}+\beta _{1}\sigma _{t-1}^{2}+\cdots +\beta _{p}\...
1
vote
1answer
49 views

Maximum likelihood estimator for a mixture of 2 distributions

Let $X_1, ..., X_n$ be iid with one of two PDFs. If $\theta = 0$, then $f(x; \theta) = 1, \ 0 < x < 1$. if $\theta = 1$, then $f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$. What ...
0
votes
1answer
32 views

Test for same coefficients with different estimation approaches

I'm estimating coefficients $w_1,\dots, w_d$ with different approaches (for example with ML and with leave one out CV) a few times. Now I'd like to test if the coefficients are the same, but I dont ...
2
votes
0answers
14 views

Theory of convergence for this case?

I hope you can guide me to the right place. I am estimating two equations, where each equation produces an output which is needed in the other equation as input. Something like this: $Y_t = AX_t + f(...
0
votes
0answers
19 views

Estimability in Design model

consider the design model $y=\theta+e$ I know we can obtain the normal equations from observation to estimate the parameters. my question is- is the estimation BLUE? Given normal equations are: $...
1
vote
1answer
29 views

Estimation the evolution of error variance with time?

I have data tuples $(x_i,\varepsilon_i,t_i)$ generated from some observations and I suspect that $\varepsilon \sim \mathcal{N}\left(0,\sigma(t)^2\right)$, where $\sigma(t)$ is an increasing function ...
1
vote
1answer
21 views

Probability for mean of a subpopulation

Suppose I have drawn 21 samples from a population which I assume to be normal, where the sample has mean 3.8 and sample standard deviation 0.7. What is the probability that the mean of the next 210 ...
4
votes
2answers
299 views

Estimating the MLE where the parameter is also the constraint

Independent random variables $X_1,X_2,\ldots,X_n \sim f_X$ are modeled with a common density $$f_X(x) = \frac{\alpha(x/\beta)^{\alpha-1}}{\beta} \quad \quad \quad \text{for all } 0 \le x \le \beta.$$ ...
0
votes
0answers
14 views

Bias of the eigenvalues of sample covariance matrix

Consider and i.i.d sample $X_1, \ldots X_n$ in $\mathbb{R}^p$ with covariance matrix $\Sigma \in \mathbb R^{p\times p}$ What is the expectation of the eigenvalues of the sample covariance matrix?. ...
0
votes
1answer
24 views

Estimating the variance of the GLS estimator

Consider the linear regression model where $y = XB + u$. Assume that $\mathrm{E}[u \mid X] = 0$. Assume that $\mathrm{V}[u \mid X] = \sigma^2I$. This is the simple linear model. Now relax the ...
0
votes
1answer
90 views

Maximum likelihood estimator for Bernoulli parameter based on standard normal

$X_i \sim Normal(\psi,1), \ \ i = 1, ..., n$ $Y_i = 1$ if $X_i \ge 0.$ $Y_i = 0$ if $X_i < 0.$ Let $\theta = P(Y_i = 1)$. What is the MLE of $\theta$? I know how to find the MLE of a Bernoulli ...