Questions tagged [estimation]
This tag is too general; please provide a more specific tag. For questions about the properties of specific estimators, use [estimators] tag instead.
2,843
questions
4
votes
1answer
62 views
Why do we multiply log likelihood times -2 when conducting MLE?
When we are performing maximum likelihood estimation (MLE) to estimate parameters, the fit function is often to -2 * LL, rather than just LL. I also see this "-2LL" term expressed as "...
0
votes
0answers
7 views
Should there be a transpose in calculating the parameters of SVM
I am trying to code SVM from scratch using a small toy problem that involves five support vector values. In the code below, there are 5 support vectors arbitrary chosen and denoted by the variables <...
1
vote
0answers
15 views
Geometric distribution and entropy
According to wikipedia, among all discrete probability distributions supported on $\{1, 2, 3, ... \}$ with given expected value $\mu$, the geometric distribution X with parameter $p = \frac{1}{
\mu} $ ...
0
votes
1answer
21 views
Link between Maximum Likelihood and Maximum Probability [duplicate]
How can I see that the maximum likelihood approach finds the parameter values of the probability distribution that maximize the probability of the observed sample?
Maximum likelihood is not the ...
0
votes
0answers
12 views
Confusion about Maximum likelihood estimation [duplicate]
How can I see that the maximum likelihood approach finds the parameter values of the probability distribution that maximize the probability of the observed sample?
Maximum likelihood is not the ...
0
votes
0answers
19 views
Point estimation of parameters
Let $\theta$ be a parameter with values in $\Theta$ that should be estimated by some given data $X$.
The corresponding estimate is denoted as $\hat\theta = \hat\theta(X)$.
Several times I read that ...
1
vote
0answers
20 views
What-If Scenario Regression Modelling
I'm pondering a scenario involving some insurance data but this could be relevant in many fields. The idea is that I have a total count of some event. Let's imagine this count is the # of attorney ...
0
votes
1answer
12 views
Non-adversarial robustness
One measure of an estimator's robustness is the breakdown point, which tells us how many adversarial observations are necessary to make the estimator useless.
However, is there a notion of non-...
0
votes
0answers
8 views
How to build an estimator from missing data?
Given a data set with company sizes (number of employees) $x_i$ and revenues $y_i$ for three consecutive years. A snippet is shown below: ("*" denotes a given value and "NaN" a ...
0
votes
0answers
4 views
Creating Proxy variable to minimise estimation error
As part of my econometrics project, I am investigating wage structures for female workers. The econometric model I am using follows the Mincer earnings function. The dataset available includes ...
1
vote
1answer
23 views
Non linear regression on 2D features space using SVR failed
I try to estimate the execution time of some application based on the mCPUs (fraction of physical CPU) used for the run and the overall size of input data that the application is processing.
The data ...
1
vote
1answer
34 views
Did I correctly apply the factorisation theorem in this example?
Suppose that we have a density $f(x,\theta)=c(\theta)\psi(x)\unicode{x1D7D9}_{x \in]\theta,\theta+1[}(x)$.
Find a sufficient statistic for this density.
My attempt : Since $\Bbb L(X,\theta)=c(\theta)^...
7
votes
2answers
107 views
variance estimation using order statistics
I have four largest samples drawn from a distribution of N i.i.d Gaussian R.V. with standard deviation (Sigma) where sigma is unknown. N is known to be between 50-200. Mean is given to be 0.
How do ...
2
votes
1answer
24 views
Fitting ARMA-GARCH sequentially vs simultaneously
I am interested in fitting an ARMA-GARCH model to my data. After reading a few pages online I did so sequentially by first applying ARMA and then feeding the residuals into GARCH. I then took the ...
2
votes
0answers
21 views
Nonparameteric Empirical Estimator For Stochastic Process
Motivation: If $X$ is a random-variable defined on some probability space $(\Omega,\Sigma,\mathbb{P})$ then Glivenko-Cantelli lemma guarantees that the empirical distribution $\frac1{N}\sum_{n=1}^N \...
0
votes
0answers
21 views
What's the maximum likelihood estimation of $\theta$ in this density?
Suppose we have a n-sample $X=(X_1,..,X_n)$ with a distribution $f(x,\theta)=exp(\theta - x)\unicode{x1D7D9}_{x \geq \theta}(x)$.
Find the maximum likelihood estimator $T$ of $\theta$ and prove that $...
-1
votes
0answers
31 views
Finding the UMVUE of $P(X_1\ge k)$ [duplicate]
Let $X_1,X_1,...,X_n$ be a random sample of size $n$ from $N(\mu, \sigma^2)$ where both $\mu$ and $\sigma$ are unknown. It is required to find the UMVUE of $P(X_1\ge k)$.
In this problem I am able to ...
0
votes
0answers
25 views
Distribution of hat sigma in simple linear regression model [duplicate]
How can I prove that the distribution of estimated σ^2 (the distribution of the estimated variance of errors εi, in case of homoschedasticity) is equal to a chi-squared with n-2 df * σ^2/(n-2) ?
0
votes
0answers
26 views
How to prove that the t distribution doesn't belong to the exponential family?
Or in other words, is there anyway prove that the t distribution doesn't belong to the exponential family without going through all that calculation? Since the density has the gamma function in it ...
1
vote
0answers
18 views
Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance
I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
1
vote
0answers
37 views
MLE and MOM estimates coincide in Normal distribution
Let $(X_1,Y_1),...,(X_n,Y_n)$ be a sample form $N(\mu_x,\mu_y,\sigma^2_x,\sigma^2_y,\rho)$ population. If $n\geq 5$ and $\mu_x$ and $\mu_y$ are unknown, I want to conclude that the estimates of all ...
0
votes
2answers
21 views
Can you use a given prediction interval to find the predicted value of Y on the estimated regression line?
Suppose you don't know the estimates of a linear regression model (b0, b1 are unknown) and you are given a 95% prediction interval at x = 6 to be [5,15].
Then, since prediction intervals are ...
3
votes
1answer
45 views
Geometric Waiting Time MLE
If the time is measured in discrete periods, a model that is often used for the time $X$ failure of an item is:$$P_{\theta}[X=k]=\theta^{k-1}(1-\theta), k=1,2,...$$
where $0<\theta<1$. Suppose ...
3
votes
1answer
73 views
MLE exists and is unique for iid series
Let $X_1,...,X_n\in R^p$ be i.i.d. with density,
$$f_{\mathbf{\theta}}(\mathbf{x})=c(a)\exp(-|\mathbf{x-\theta}|^a), \mathbf{\theta}\in \mathbb R^p, a\geq 1$$
where $$c^{-1}(a)=\int_{R^p}\exp(-|\...
0
votes
0answers
17 views
unbiased estimation of the variance of $p$ (proportion) of a random sample without replacement
Given a random sample without replacement of size $n$ from population of size $N$ and $p$ is the estimator of the proportion $P$. How could one show that:
\begin{equation*}
\frac{N-n}{N(N-1)}pq
\end{...
0
votes
0answers
37 views
Finding MLE of this confusing setup of distributions
I have been given two random variables $X$ and $Y$ following exponential with means $\lambda_1$ and $\lambda_2$. Let $Z_1 = min(X,Y)$ and $Z_2$ will be 0 if $Z_1 = X_1$ and it will be 1 if $Z_1 = X_2$...
1
vote
0answers
18 views
What is the covariance between function and its argument?
For example, we have log-likelihood function $\ell(\theta)$ and maximum likelihood estimator $\hat \theta_{ML}$ obtained as $\ell(\theta)' = 0$. What would be
$$Cov(\ell(\theta), \hat \theta_{ML}) ?$$
...
1
vote
0answers
14 views
Bayes estimate of upper limit of uniform distribution with exponential prior
Let $X_{1}, . . . , X_{n} > 0$ be a random sample from $U(0, \theta)$. Suppose $\theta$ has the prior $\pi(θ) = e^{-\theta} ; \theta > 0$. Find the Bayes estimate of $\frac{1}{\theta}$ with ...
0
votes
1answer
23 views
What Cramer-Rao bound should I use?
I have been researching about the Cramer-Rao bound and I have found two inequalities:
$$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\...
1
vote
1answer
50 views
Confidence interval for $\mu_i$ (multivariate normal): some $\mu_j$ are known, unknown $\sigma^2$ and an estimation of the correlation structure?
Say we have a multivariate normal with m dimensions (let's say that m=1,000), the mean vector $\mu$ is known only for the first 100 elements, but unknown for the remaining 900.
We have a single ...
1
vote
1answer
40 views
Find covariance of estimator and derivative of the log-likelihood function
Problem:
Given and estimator $\hat k$. The estimation method is unknown (so, it can be max. likelihood, method of moments or another method), however, we know that $bias(\hat k) = 0$.
Let $L$ be the ...
6
votes
1answer
160 views
What is the score function of two parameters?
According to this wikipedia article, score is the derivative of the log-likelihood function. However, I don't understand what if we have two parameters? For example, the logarithm of pdf has the ...
0
votes
0answers
12 views
What are good resources/packages for implementing copulas?
Say I have bivariate data $[X_i,Y_i]$ with $i=1...n$ and suppose I know the marginal distributions of $X$ and $Y$. I wish to estimate their joint distribution. I read that copulas can be used for this....
1
vote
1answer
20 views
How to check the consistency of OLS estimator in macroeconomic models
Problem:
We have a model $$C_t = a + b Y_t + e_t$$ and $$ Y_t = C_t + I_t$$ It's known that $Cov(I, e)$ is zero.
A student estimates the following model: $$C_t = a + b Y_t + e_t$$
Are the estimators $\...
2
votes
1answer
65 views
How to show that $d(X)$ is unique Bayes with respect to any prior whose mean is $\frac12$ and variance is $\frac18$?
Consider estimation of $\theta$ where $X\sim \text{Bernoulli}(\theta)$. Under squared error loss, I am asked to show that $d(X)=\frac{2X+1}{4}$ is unique Bayes with respect to any prior for which $\...
0
votes
0answers
7 views
Intra-run and inter-run variability formula
I have 1 machine that can perform 15 runs with 10 measures per run (so I have 150 results in total). I would like to estimate the variability (in terms of CV) of the results in order to evaluate the ...
1
vote
1answer
65 views
Confusion about prior used in Recursive Bayes Filter
I'm currently using this thesis to understand key concepts about probabilistic inference in computer vision which is being a great source.
The frame of the question is the following:
Let us assume we ...
0
votes
1answer
37 views
MLE with uneasy density function
I want to find the MLE of $\theta$ with the following density.
$ (1- \lvert x - \theta \rvert) \mathbb{1}_{[\theta-1, \theta+1]} (x)$, to confirm : $\widehat{\theta_{n}} = \overline{X_{n}}$ found ...
3
votes
0answers
37 views
Derive the closed form of a probability expression
Consider the following probability
$$
A\equiv Pr\Big (\delta_1+v+\lambda \epsilon_1\geq \delta_2+v+\lambda \epsilon_2 \text{, } \delta_1+v+\lambda \epsilon_1\geq \epsilon_0 \Big )
$$
where
$\lambda\...
3
votes
1answer
39 views
Determine probability of an event using maximum likelihood estimation
Problem
I have a bag of many red and green balls. To find out the ratio between the two, I randomly picked balls with replacement. Out of the 100 outcomes, 60 were red balls and 40 were green balls. ...
1
vote
0answers
6 views
Involving data length as a decision variable in a optimization for parameter estimation
Consider I have $N$ historical data as outputs from a unknown time-varying system. Assuming within any small time interval $T$, the time-varying system can be approximated by a linear time invariant (...
0
votes
0answers
15 views
How do I choose between regression methods for inference?
Suppose my goal is to understand the relationship between variable $y$ and covariates $X$. Let's say $y$ is a rate, the number of success in $n$ trials, therefore bounded between $0$ and $1$.
Now I ...
0
votes
2answers
81 views
Why do OLS and logistic regression coefficients have opposite sign?
I have data $y$ which is the rate of success in $n$ trials. I also have covariates $X$ that I want to regress against $y$ to understand the relationship between them. I tried 2 different approaches. ...
1
vote
1answer
37 views
Where plus 1 came from in variance estimation [duplicate]
While
$$
\mathrm{E}(\tilde{\mathrm{y}})=\alpha+\beta \tilde{\mathrm{x}}
$$
Subject is Regression Analysis and this formula is from the "Features of Estimation ".
and y is a neutral variable.
...
0
votes
0answers
49 views
How are the number of actual rape cases estimated?
I see in news articles of number of rapes reported and an estimate for actual number of rape cases. I am interested in knowing about statistical techniques used for making such estimations. Also, are ...
4
votes
1answer
50 views
Practical method to do MLE for natural parameters in exponential family
I encountered the following question in my research and I hope this is the correct place to post it. I'm following the notation in this lecture note by Michael I. Jordan.
Assume random vector $X$ ...
2
votes
0answers
9 views
estimating the total number of balls (n) after 2 draws of k balls
The problem is predicting the total number of balls (n) after 2 draws of k balls.
After the first draw of k balls, we have marked them so we can see if we had them before.
The balls are returns and we ...
0
votes
0answers
20 views
Is the mundlak estimator equal to the within estimator?
Mundlak has proposed to estimate the following correlated random effects model:
$$
y_{i t}=\boldsymbol{x}_{i t}^{\prime} \boldsymbol{\beta}+\overline{\boldsymbol{x}}_{i}^{\prime} \gamma+\omega_{i}+u_{...
0
votes
0answers
18 views
Giving a job lecture on “minimax statistical estimation”. What kind of questions to expect?
As part of a job application process, I am giving a short "mock lecture" for students on minimax estimation in statistics. I will introduce the basic concept and give an example.
I am able ...
0
votes
0answers
12 views
Estimating parameters of correlated random variables
Let's say there are two correlated random variables $X$ and $Y$ with coefficient of correlation as $r$.
Assume during some hypothesis testing we found that due to some change in the environment mean $\...
