Questions tagged [estimation]

This tag is too general; please provide a more specific tag. For questions about the properties of specific estimators, use [estimators] tag instead.

1,033 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
13
votes
0answers
828 views

Estimation of ARMA: state space vs. alternatives

I am interested in estimation of ARMA models. I understand that a popular approach is to write the model down in the state space form and then maximize the likelihood of the model using some ...
10
votes
0answers
315 views

Validity of confidence interval for $\rho$ when $X\sim N_3(0,\Sigma)$ with $\Sigma_{ij}=\rho^{|i-j|}$

Suppose $X\sim N_3(0,\Sigma)$, where $\Sigma=\begin{pmatrix}1&\rho&\rho^2\\\rho&1&\rho\\\rho^2&\rho&1\end{pmatrix}$. On the basis of one observation $x=(x_1,x_2,x_3)'$, I have ...
9
votes
0answers
103 views

Rationale behind Good–Turing frequency estimation?

Good–Turing frequency estimation is a smoothing estimator for estimating a multinomial distribution. It seems very convoluted. From mathematical statistics point of view, what is the rationale ...
7
votes
0answers
241 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
7
votes
0answers
866 views

Gauss-Newton method for MA parameter estimation

Please check my solution below for estimating Moving Average parameter using the Gauss-Newton (Linearization) method. I consider MA(1). MA(1) model: $$z_t=a_t-\theta_1a_{t-1}.$$ Solution: The ...
7
votes
0answers
1k views

Sampling distribution of average of some covariance matrices

I have $K$ datasets, each with $N$ variables and $M$ samples (they are in fact EEG time series, but I discard time and treat them as $K$ iid multivariate samples) and assume they are coming from the ...
6
votes
0answers
913 views

Why would I use ratio estimation instead of regression estimation to estimate means?

I am taking a graduate course on survey data analysis. I was recently introduced to ratio estimation and regression estimation. I understand that using ratio estimator may be easier if we are ...
6
votes
0answers
1k views

Methods of Proving that a UMVUE does not exist?

Are there efficient methods of showing when a UMVUE does not exist? I can think of the trivial case when no unbiased estimators exist at all. But that's not really interesting. I feel like this ...
6
votes
0answers
180 views

bias of an estimator when using stopping rules

Consider the setting where $X_1,X_2,...$ are i.i.d. real-valued random variables with $\mathbb{E}[X_i] = \theta$ and let the random variable $\tau$ be an associated stopping time. I'm wondering what ...
6
votes
0answers
172 views

Estimating population size, minimum variance estimators

I am trying to understand what can be proved about minimum variance estimators. I am a little confused by Cramér–Rao and how to apply it even to really simple examples or if it's even the right tool ...
6
votes
0answers
181 views

Time series modeling the number of users of a mobile app

I want to model the number of users of an mobile app. This app has two kinds of users: free and paid. I thought of this autoregressive model: $x_t = Ax_{t-1}$ with $x_t$ being a 4-dimensional vector,...
6
votes
1answer
331 views

Expression for the median of a sum of half-Cauchy random variables

This is a technical problem I encountered in research. Sorry if it reads simple for professional statisticians. Let $X_1, \cdots, X_n$ be iid standard Cauchy random variables. What estimate or exact ...
6
votes
1answer
640 views

Error in estimation with continuous data

Is there a way to correlate error in a fit (MSD - mean squared displacement) to the error of the a calculation performed with the parameters associated with the fit? My specific problem is dealing ...
5
votes
0answers
196 views

Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$

Question: Let $X_1,X_2,\dots,X_n$ be random sample from Poisson($\theta$). Find MVUE of $e^{-2\theta}$ My attempt has been by modifying the answer from this question: The Poisson distribution is a one-...
5
votes
0answers
384 views

Probability with an unbalanced coin where consecutive flips are not independent

Thanks in advance for the help. Suppose someone has an unbalanced coin that they flip 100,000 or so times in a row. This person then gives you the results. You do not know the probability of ...
5
votes
0answers
664 views

Implementing ARMA Log Likelihood with the Kalman Filter Algorithm

A popular algorithm to determine the (complex) log likelihood function of an ARMA(p,q) process involves generating it through the use of a state-space model and the Kalman Filter. I started reading ...
5
votes
1answer
277 views

Are there better estimators of misclassification error than the fraction of misclassified test points?

Assume we train a binary classification model using the training set. Also assume that the model returns an estimate of the probability of success $\hat f(x)$ for every feature vector $x$ and was ...
5
votes
1answer
316 views

Combining unbiased estimators with unknown variance

Say we are given a sequence of independently (but not identically) distributed random variables $X_1,...,X_n$ which are known to be bounded, $X_t \in (a,b)$, and to have the same mean, $\mathbb{E}X_t =...
5
votes
0answers
286 views

Hierarchical model: question on frequentist estimation

I am interested in understanding the differences between Bayesian and Frequentist estimation in the context of hierarchical models. Consider $n$ subjects, where for subject $i$ there are $k_i$ ...
5
votes
0answers
1k views

The distribution of STD/MAD for a Student-t

Where $X \sim a$ symmetric Student-t Distribution $t_\alpha$, with power law tail $\alpha>2$, looking for the distribution of $$ \frac{\sqrt{ \sum_{i=1}^n x_i^2 }}{\sum_{i=1}^n |x_i|}, $$ in ...
5
votes
0answers
123 views

Estimation After Selection on Non-central F Random Variables

Suppose that you observe $F_1,F_2,\ldots,F_k$ each independently. drawn from non-central F distributions with common, known, d.f. $\nu_1, \nu_2$, and with (unknown) non-centrality parameters $\...
5
votes
0answers
2k views

Why would concatenating feature vectors lead to better estimates?

I wish to estimate the state of a system from two separate and disparate observations. A simple approach that I have seen in some literature is to combine the feature vectors (observations) by simply ...
4
votes
0answers
59 views

Is it possible to show that this estimator has minimum variance?

Doing some exercises I stumbled upon this tricky one: Suppose we have an independent random sample $(X_1, ... , X_n)$ with $X_i \sim Poisson(\lambda)$. Define $\theta = e^{-\lambda}$. Let $$ \...
4
votes
0answers
90 views

Dynamic panel data model with AR(2) process in the errors

I set up the following dynamic panel data model: $$y_{it}=\alpha y_{it-1}+x_{it}^T\beta+v_{it}$$ Additionally, I have the process in the errors: $$v_{it}=\rho_1u_{it-1}+\rho_2u_{it-2}+\epsilon_{it}$$ ...
4
votes
0answers
173 views

The difference of normal means is also minimax?

Let $X_i \sim N(\xi, \sigma^2)$ and $Y_i \sim N(\eta, \tau^2)$ for known $\sigma^2$ and $\tau^2$. I know that $\bar{X}$ and $\bar{Y}$ are minimax under squared error loss since their variance is ...
4
votes
0answers
2k views

Manual estimation of a GARCH(1,1) parameters using MLE vs rugarch package in R

I want to estimate parameters of a GARCH(1,1) model using rugarch package in R and manually(using maximum likelihood). Firstly, I import and transfrom the data as below(Amazon return data) ...
4
votes
0answers
341 views

Existence of UMVUE of $\theta$ for sample from $\small{\frac{\ln\theta}{\theta-1}\theta^x\,\mathbf1_{0<x<1}}$?

Suppose $X_1,X_2,\ldots,X_n$ is a random sample drawn from a distribution with pdf $$f_{\theta}(x)=\small\frac{\ln\theta}{\theta-1}\theta^x\,\mathbf1_{0<x<1}\quad,\,\theta>1$$ Does there ...
4
votes
0answers
1k views

Understanding the Invariance Property Proof of MLE

There are multiple versions of this question on CV and many proofs are given but I did not fully understand the proof (technique) yet. Theorem. If $$\hat\theta = \operatorname*{argsup}_{\theta\in\...
4
votes
0answers
367 views

Why are the prediction error estimate and the true prediction error negatively correlated?

Section 12.2 of Computer Age Statistical Inference reports simulation results that indicate that the cross-validated estimate of the prediction error for a given prediction rule is negatively ...
4
votes
1answer
41 views

What is Generative Training?

I know that the difference between generative and discriminative classifiers is that the generative ones directly model the distribution of the observed data while the discriminative ones do not. ...
4
votes
0answers
90 views

What is a reasonable alternative when the mean does not exist?

Consider a continuous random variable $F''$ following a doubly noncentral $F$ distribution with $8$ and $2$ degrees of freedom, and noncentrality parameters $0$ and $10$; which is to say: $$ F'' \sim ...
4
votes
0answers
344 views

Mixed effects model with an ARIMA correlation structure

I'm fitting a mixed effects model where each cluster is a time series. For this purpose, I'm using the R package nlme, whose main module (also ...
4
votes
0answers
218 views

German tank problem: comparing two estimators

The following estimators can be used for the german-tank-like problems. If we collect a sample of size $k$ with sample mean $\bar{x}$ and highest number $m$, then we can estimate the population size ...
4
votes
0answers
93 views

Interesting application of E-M algorithm

Suppose the following dataset: [3 4 3 4 6 12 12 7 8 9] [2 5 3 4 12 2 2 10 7 6] [3 4 3 4 5 11 10 7 8 9] These numbers are totally random. So this dataset, depicts ...
4
votes
2answers
628 views

What is the proper way to measure error for an estimation algorithm?

Our algorithm is about estimating the true statistic values from a data set. The data set is a table in relational database, we are going to estimate the statistic value for filtered records, like <...
4
votes
0answers
79 views

How many observations to estimate a parameter of an Archimedean copula?

Let's consider for example the bivariate Gumbel copula. $$C(u_1, u_2)=\exp \left[-\left(\left(-\ln\left(u_1\right)\right)^{\theta}+\left(-\ln\left(u_2\right)\right)^{\theta}\right)^{\frac{1}{\theta}}\...
4
votes
0answers
221 views

Comparing the performance of two classifiers using cross-validation

Consider the following excerpt (paraphrased, see sec. 4.6.3 for original wording) from Introduction to Data Mining (free chapter) by Pang-Ning Tan, Michael Steinbach, and Vipin Kumar. Suppose we ...
4
votes
0answers
512 views

Delta Method vs. Lognormal

I have a single parameter $\theta > 0$ of a probability model I estimate with MLE on i.i.d. data. To get rid of the positivity constraint I instead estimate $\log \theta$ for which MLE gives me an ...
4
votes
0answers
164 views

Estimation of sample mean for hypergeometric distribution under a constraint

Assume that we have a dataset $D$ that contains the age of N people: $X_1, \dots , X_N$. We draw a sample $S$ of size $n$ using sampling without replacement ($Y_1, ... , Y_n$). In order to estimate ...
4
votes
0answers
289 views

Unbiased Estimator of the truncation points in a truncated normal distribution?

Consider the variables $x_i \text{~} \mathcal{N}(\mu, \sigma^2,a,b)$ iid with truncation points $a$ and $b$, i.e. $a < x_i < b$. Suppose all 4 parameters, namely $\mu, \sigma, a, b$ are all ...
4
votes
0answers
444 views

Estimating Fisher Information Matrix by Covariances

A method I've seen suggested (e.g. p 446 of this text) for estimating the Fisher information matrix (FIM) is by computing the sampling covariance matrix of the scores. That is, $$ \hat{\mathcal{I}}_n ...
4
votes
0answers
99 views

U statistics for product kernels

Background Let $X$ be a real, univariate random variable with probability distribution function $p(x)$. Let $x_1, \ldots, x_n$ be a sample of size $n$ drawn from $X$. U statistics provide an unbiased ...
4
votes
0answers
266 views

Comparison of estimation techniques for ARIMA model

I'm a math graduate student with not much knowledge in statistics. I could note that we have different techniques to estimate ARIMA parameters for a time series: using Bayes's Theorem, maximizing the ...
4
votes
0answers
65 views

How do I find the center and radius of my circularly distributed data?

I'm analysing data that was collected in an optics experiment. The measurements are roughly in the form of a ring. In an ideal world the center of this ring would coincide with the origin, but due to ...
4
votes
0answers
79 views

How to derive an estimator for the parameter of a continuous uniform distribution

$X_1, X_2,\dots.,X_n$ are i.i.d. random variates drawn from a continuous uniform distribution over $[0,\theta].$ The sufficient statistic is denoted $\max$. I want an estimator $e$ of $\theta$ that ...
4
votes
0answers
39 views

German tank variant: estimate resolution of camera given cropped photo sizes

Make whatever assumptions you like, but I like the flavor of nonparametric techniques. I have a list of the $x_i$ by $y_i$ resolutions of a number of photos, all cropped from photos taken at the same ...
4
votes
0answers
89 views

Show that MLE of $\lambda = \frac{n-T_n}{S_n+cT_n}$

$X_i$ are i.i.d exponential, mean $\lambda^{-1}$ for $1 \leq i \leq n$ and, the values are measured such that $X_i = c$ if $X_i \geq c$ and $X_i$ otherwise. Show that MLE of $\lambda = \frac{n-T_n}{...
4
votes
0answers
250 views

Finding $Var(S^2), E(S^4),$ and unbiased estimator for $\sigma^4$ from random, normal samp

Let $X_1,...,X_n$ be a random sample of size $n$ from the normal distribution $N(\mu,\sigma^2)$ and let $S^2$ be the sample variance. (a) Find $V(S^2)$ and derive $E(S^4)$. (b) find an unbiased ...
4
votes
0answers
229 views

Maximum likelihood estimation and density estimation

Let's consider a general signal processing estimation problem where the measurements are modeled as $${\bf x}[n]={\bf s(\theta)}+{\bf w}[n],$$ where ${\bf w}$ is a non-Gaussian r.v. (noise term) and ${...
4
votes
0answers
45 views

Calculating a Constrained Mean

Assume that I have ten measurements of a physical quantity that must be non-negative (e.g., mass). Because of measurement error, some of the results are nevertheless negative. Can someone suggest a ...

1
2 3 4 5
21