Questions tagged [estimation]

This tag is too general; please provide a more specific tag. For questions about the properties of specific estimators, use [estimators] tag instead.

Filter by
Sorted by
Tagged with
1
vote
1answer
25 views

Linear relations between statistics in exponential family of distributions

I am reading about point estimation from Theory of Point Estimation by Lehmann and Casella (1999). I couldn't understand the following point mentioned in p.24, under the exponential family of ...
4
votes
2answers
286 views

Trying to find estimators for 3 parameters in a simple equation

I came across the following equation (source):๏ฟผ $$ y=ax^be^{-cx} $$ where a, b and c are parameters, and x, y are variables. I was curious to estimate the parameters a, b, c for a dataset of x,y pairs,...
1
vote
2answers
72 views

Small sample size fails to approach inverse CDF

When sample size $n$ gets large, we know that a sorted set of the $n$ samples approaches the inverse cumulative distribution function (CDF) sampled at $\frac{1}{n}, \frac{2}{n}, \dots, \frac{n}{n}$. ...
0
votes
0answers
13 views

Optimal sampling rate for forward algorithm

I have a system with a binary-state. The system state is estimated by an HMM forward-algorithm. Also, the system allows a varying sampling rate. Considering that the system state transition takes a ...
3
votes
0answers
34 views

Burg's method for estimation of AR-models

I am trying to get an intuitive understanding of Burg's method for estimation the coefficients of an AR-model. Say we have an AR(1)-process with \begin{equation*} X_t = a X_{t-1} + \varepsilon_t \...
2
votes
0answers
75 views

Finding credible interval for $\mu_1-\mu_2$ when $X_i\sim N(\mu_1,\sigma^2)$ and $Y_i\sim N(\mu_2,\sigma^2)$

Let $X_1,X_2,\ldots,X_m$ and $Y_1,Y_2,\ldots,Y_n$ be independent samples from $N(\mu_1,\sigma^2)$ and $ N(\mu_2,\sigma^2)$ populations respectively where $\sigma^2$ is known. I need to find a $100(1-\...
2
votes
1answer
40 views

Estimating mean in the presence of serial correlation

Consider the following generating equation: \begin{equation} X_{d+1} = a X_d + b + {\cal E}_d \end{equation} where $a$ and $b$ are constants with $0 <a < 1$ and $b > 0$. Further let ${\...
0
votes
2answers
108 views

Can a Bayesian estimator perform better than an MVUE?

According to wikipedia: In statistics a minimum-variance unbiased estimator (MVUE) or uniformly minimum-variance unbiased estimator (UMVUE) is an unbiased estimator that has lower variance than any ...
9
votes
3answers
582 views

Why is $R_t$ (or $R_0$) and not doubling rate or time the go-to metric for measuring Covid-19 expansion?

In my head, the natural way to measure the expansion speed of an epidemic across populations of different sizes is simply fitting an exponential over recent infection numbers (with any strategy), ...
0
votes
0answers
12 views

Control for common factors

Hi I have a question about how to control for common factors. I used people of a state as the sample to investigate a causal relation between two variables, y and x. Although there are a lot of ...
9
votes
3answers
2k views

Revisiting the Rule of Three

The rule of three is a method for calculating a 95% confidence interval when estimating $p$ from a set of $n$ IID Bernoulli trials with no successes. My understanding from its derivation is that the ...
2
votes
1answer
53 views

Making sense of MLE

Since we were taught MLE (Maximum Likelihood Estimation), a number of questions often bothered me. Why does Maximum Likelihood Estimation work ? Why does it always produce almost accurate results ...
2
votes
1answer
38 views

How to find the time complexity of an MLE based algorithm

How to calculate or what is the time complexity (big-Oh) for this method? Based on my understanding, MLE depends on number of datapoints, $N$ so time complexity for MLE is O(N). However, there are ...
0
votes
2answers
27 views

OLS estimators are independent of the scale of the predictor data

The title says it all: How can I see that OLS estimators (linear regression) do not depend on the scale of the predictor data? I am reading my lecture notes about lasso estimators, where it is ...
1
vote
1answer
87 views

How to combine confidence intervals of a regression when calculating a mean of estimated value

I am facing the following problem: I have a regression similar to the following: fit <- lm(I(hp > 100) ~ cyl + wt + disp + am,data = mtcars) The idea is more ...
1
vote
0answers
46 views

Cramer-Rao bound in case of non-invertible Fisher Information matrix

I am learning about the Cramer-Rao lower bound (CRLB) and Fisher Information matrix (FIM), and started trying to apply it to some simple toy models from physics. However, even for a simple example I ...
1
vote
1answer
68 views

A problem about confidence interval

How can I solve this? Two new drugs were given to patients with hypertension. The first drug lowered the blood pressure of $16$ patients an average of $11$ points, with a standard deviation of $6$ ...
0
votes
0answers
15 views

Bootstrapping the time until an event happens

Assume I have some measurements $\{(y_i, t_i)\}^n_{i=1}$, where $y_i$ is a real number (measured quantity) from some unknown random function $y_i = f(x_i)$, $x_i$ are inputs to that function and $t_i$ ...
0
votes
1answer
12 views

Test whether the theoretical model estimated from the sample predicts the same mean

I have a sample of experimentally observed data and a parametric distribution model which is expected to explain the data. I estimated the model parameters by the maximum likelihood. Now I need to ...
2
votes
2answers
39 views

How to estimate mean and variance for rate of change when I only have state data at different ages

I'll give you the intuition behind my problem first. I have data on whether children ($n \approx 200$) can read and their age in integers from 0 to 14. For each age, it is straightforward to ...
10
votes
3answers
363 views

Randomly Sample M samples from N numbers with replacement, how to estimate N?

Can you estimate $N$ with MLE or method of moment or whatever strategy? $N$ numbered balls are in a bag. $N$ is unknown. Pick a ball uniformly at random, record its number, replace it, shuffle. After ...
1
vote
0answers
35 views

How to obtain standard error of parameters in maximum likelihood when reparametrized/transformed?

Sometimes parameters in the maximum likelihood estimation process are reparametrized for numerical convenience. As an example if I'm fitting maximum likelihood estimation (MLE) to a data that is ...
1
vote
0answers
14 views

Risk of loss function Bernoulli random variable

Can anyone help me with the part b of the question? I understand the risk is expected value of the loss function, but how to incorporate expected value in the piece-wise function. I will be grateful. ...
0
votes
0answers
18 views

standard techniques for picking a rolling window to estimate the mean

I have a time-series of noisy estimates of an observable. To get a better estimate (let's ignore stationarity issues), I can take an average of the past N values. But it seems, that instead of ...
2
votes
1answer
38 views

Complete and Sufficient Statistic for Discrete Distribution

I have a single observation X from the following distribution: $$๐‘ƒ(๐‘‹=โˆ’1)=\dfrac{๐‘}{3},๐‘ƒ(๐‘‹=0)=(1โˆ’๐‘),๐‘ƒ(๐‘‹=1)=\dfrac{2๐‘}{3}$$ I'm trying to find a complete and sufficient statistic for p based on ...
0
votes
0answers
25 views

Recursive estimation

I'm trying to assess the stability of the parameters of an MA(2) model on unemployment data using recursive estimation. However, I'm in doubt how to interpret the outcome of the code I'm using: ...
3
votes
3answers
136 views

Standard error of estimated covariance

Let $X_1,...,X_n$ and $Y_1,...,Y_n$ be two independent random samples from $\mathcal{N}(\mu, \sigma^2)$ where both $\mu$ and $\sigma$ are unknown parameters. I estimate their covariance using: $$\hat{\...
1
vote
0answers
19 views

Confidence interval for a log-log equation

I have the following equation: $\mathrm{ln}(y) = \beta_1 + \beta_2 \mathrm{ln}(x)$. Assume I have an estimate of $\beta_2$ and its standard error. How do I calculate the confidence interval? Is it ...
0
votes
0answers
14 views

$MSE(\hat{\theta}_3) = 6$?

Let $\hat{\theta}_1$, $\hat{\theta}_2$ and $\hat{\theta}_3$ be the estimators of $\theta$. We know that $E(\hat{\theta}_1) = E(\hat{\theta}_2) = \theta$, $E(\hat{\theta}_3) \not= \theta$, $V(\hat{\...
5
votes
1answer
115 views

Estimation precision of lower- vs. higher-order moments

I have a vague feeling that for a fixed sample size, lower-order moments of a distribution would typically be estimated more precisely than higher-order moments. E.g. mean would be estimated more ...
0
votes
0answers
8 views

Can we use the Graduated Non-convexity for parameters estimation of GMM?

To estimate the parameters of the Gaussian mixture model (GMM), the popular solution is to use the Expectation maximization (EM) algorithm. However, the estimated parameters for GMM using the EM can ...
2
votes
0answers
21 views

OLS but accidentally swapping $X$ and $Y$ and calculating new parameters [duplicate]

Just pondered over the following thought experiment for lunch. Suppose I am meant to perform an OLS for $Y=\beta X+\epsilon$, but when estimating $\hat\beta$ I accidentally use $X$ for $Y$ and $Y$ for ...
3
votes
0answers
41 views

What is the probability distribution and variance of the OLS estimate $s^2$ of the error variance $\sigma^2$ in linear regression?

Consider the standard linear regression model $$ y = X \beta + \varepsilon, $$ where the error $\varepsilon$ has fixed variance $\sigma^2$. We can make an unbiased estimate of the error variance in a ...
1
vote
1answer
83 views

Rubin's rule, applied to absolute effect size or relative effect size (Cohen's d)?

Cohen's d is a way to describe the effect size relative to the standard deviation of the data. For instance in the case of the difference between the means of two populations $$\begin{array}{} \text{...
7
votes
1answer
145 views

Why is the observed Fisher information defined as the Hessian of the log-likelihood?

In an MLE setting with probability density function $f(X, \theta)$, the (expected) Fisher information is usually defined as the covariance matrix of the fisher score, i.e. $$ I(\theta) = E_\theta \...
3
votes
1answer
230 views

Prove that ML estimate of a function of a parameter, $g(\theta)$ is the function of its ML estimate $g(\hat{\theta})$ [duplicate]

As titles say, given $\hat{\theta}$ is the Maximum Likelihood estimate of a parameter $\theta$, how to prove that Maximum Likelihood estimate of $g(\theta)$ is $g(\hat{\theta})$. Additionally, is this ...
1
vote
0answers
25 views

Bias in estimation of a latent / hidden variable drawn from a skewed distribution: what is it called?

I observe a bias effect in my measurement system that I can explain and correct using a simple latent or hidden variable model. I am sure this kind of effect has been described earlier in other fields ...
6
votes
3answers
132 views

Sample standard deviation is a biased estimator: Details in calculating the bias of $s$

In this post Why is sample standard deviation a biased estimator of $\sigma$? the last step is shown as: $$\sigma\left(1-\sqrt\frac{2}{n-1}\frac{\Gamma\frac{n}{2}}{\Gamma\frac{n-1}{2}}\right) = \sigma\...
1
vote
1answer
25 views

Why might the functional form of a distribution be “inappropriate” for a particular application?

Working through Bishop's Pattern Recognition and Machine Learning(a great read so far!) and on page 67 he says: "One limitation of the parametric approach is that it assumes a specific ...
0
votes
0answers
13 views

Bernoulli sampling method estimator

Let $t = N\bar x$ be an estimator of the total of population in a bernoulli sampling method where $$\bar x = \frac{\sum_{k = 1}^Nx_kI_S(x_k)}{\# S}$$ $\# S$ is the size of the sample. Prove that $E[t] ...
2
votes
1answer
86 views

Ridge fit is not orthogonal to ridge residuals

So I'm reading https://arxiv.org/pdf/1509.09169.pdf on ridge regression. On page 8 under Example 1.3 it says From the figure it is obvious that for any $\lambda >0$ the โ€˜ridge fitโ€™ $\widehat{Y}(\...
3
votes
1answer
76 views

How to model errors around the estimation of proportions - with measurement error?

I have a situation I'm trying to model. I would appreciate any ideas on how to model this, or if there are known names for such a situation. Background: Let's assume we have a large number of movies (...
2
votes
0answers
26 views

MLE for the number of samples given $k$ largest values

I have the views on the top 100 videos using a tag in TikTok and want to estimate the total number of videos in that tag. I know the distribution for other tags so I can make a guess as to what it is ...
0
votes
1answer
58 views

How to estimate SEIR model parameter in R via RSS?

I tried to estimate the SEIR MODEL parameter in R by using the following code .Unfortunately I couldn't get it because I get the following error : "Error in names(parameters) <- c("...
1
vote
1answer
24 views

Calculating the trimmed mean as an estimator

My book provides the following steps to calculate the $ 100 \alpha$ percent trimmed mean for a sample data of $n$ measurments: 1-Order the measurments. 2-Discard the smallest $100\alpha$ percent and ...
1
vote
1answer
111 views

Why can't we have a $ 100 \% $ confidence level?

As the title implies, why can't $ 1 - \alpha = 1$ , so that we have a $100 \%$ confidence level? I saw these two answers, but want a more mathematical proof: How to estimate 100% confidence interval ...
0
votes
1answer
777 views

What's the advantage of a point estimate over an interval estimate?

A point estimate is : A single numerical value that is used to estimate the corresponding population parameter. Whereas an interval estimate is : An estimate that consists of two numerical values ...
1
vote
0answers
94 views

Meaning of “Uniformly” in UMP test and in UMVUE

What does the term "Uniformly" mean in UMP(Uniformly Most Powerful) test and in UMVUE(Uniformly Minimum Variance Unbiased Estimator)? So far I have learnt, it means for all $\theta$(...
1
vote
1answer
565 views

How to calculate the expected value of an estimator?

According to my book : An estimator, say, T, of the parameter $ \theta $ is said to be an unbiased estimator of $\theta$ if $ E\left( T\right) = \theta$. It then explains how to calculate $E\left( T\...
2
votes
0answers
34 views

Maximum Likelihood Estimator for a given density function

I have the following problem: Assume you observe $Y_1,...,Y_N$ independently from the distribution $f_y$: $$ f_{Y}(y)=\frac{12}{12-\theta}\left\{\begin{array}{ll}-\theta(y-0.5)^{2}+1 & \text { if ...

1 2
3
4 5
โ€ฆ
57