Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [consistency]

Refers generally to a property of a statistical procedure to go to the "right" place as the sample size tends to infinity, primarily referring to estimators converging to the true parameter value as the sample sizes diverges. Use also for Fisher consistency, the property that an estimator when applied to the complete population gives the right answer.

0
votes
0answers
24 views

Consistency of variance estimator in OLS [duplicate]

Given the model, $$ y_i = x_i'\beta + \epsilon_i \quad \epsilon_i \sim N(0, \sigma^2) \quad iid \quad \forall i = 1, ..,n $$ how can I prove that the estimator of the variance $\hat{\sigma}^2 = \...
0
votes
0answers
8 views

Low internal consistency

I am currently analysing the data for a survey I have completed. In doing so, one question is quite simple (nominal) and require a typical “I do/I do not” and “I do not/I do” response from the ...
2
votes
1answer
45 views

How to measure the consistency of improvement on different conditions?

I want to measure whether the speed improvement of method 1 over method 2 is consistent on different conditions. Below are two examples of the speedup values of method 1 over method 2 on 5 conditions. ...
1
vote
1answer
47 views

Effect of adding more sample data on Maximum Likelihood estimator [closed]

I have samples $\{x_1, x_2, x_3, \dots , x_n\}$ of a random variable $X$. I compute Maximum Likelihood Estimator $\hat{\theta}_n$ using the sample data. Now, if I collect one more sample $x_{n+1}$ ...
1
vote
0answers
12 views

Given a simple logit panel data model, the MLE estimators are inconsistent

Consider a binary choice model, $P \left( y _ { i t } = 1 | x _ { i t } , \alpha _ { i } \right) = F \left( x _ { i t } \beta _ { 0 } + \alpha _ { i } \right)$, $$F ( z ) = \frac { e ^ { z } } { 1 + e ...
0
votes
0answers
11 views

Is LinUCB Hannan consistent?

I'm going through the textbook Bandit Algorithms, by Lattimore and Szepesvari (http://downloads.tor-lattimore.com/banditbook/book.pdf). It describes regret bounds for the LinUCB algorithm of the form:...
5
votes
1answer
52 views

Is MLE intrinsically connected to logs?

My mathematical exploration led me the following claim: Claim: MLE is fundamentally connected to logs (and KL divergence, which also uses logs). It’s not correct to say log shows up simply to make ...
3
votes
1answer
50 views

Consistency in uniform distribution

I know what consistency is but in options C and D both U and V are given whose covariance is quite difficult to find.
0
votes
0answers
41 views

OLS assumptions: prediction vs inference

Taking an OLS model (actually, is it a "model" or an "estimator"?) as an example, there are several assumptions (such as strict exogeneity and spherical errors) which are important for the consistency ...
0
votes
0answers
18 views

Consistency metric explanation

I am trying to understand a bit more about the consistency metric (to understand how consistency-based subset evaluation works). I find on this paper the following equation : $$\text{Consistency}_s =...
2
votes
1answer
62 views

Invariance property

I am a bit confused regarding what exactly is the invariance property of sufficient estimators, consistent estimators and maximum likelihood estimators. As far as I know, Invariance property of ...
2
votes
1answer
79 views

OLS estimator for regression without intercept [duplicate]

Consider a linear regression model: $Y_i = \beta_1 A_i + \beta_2 B_i + u_i$ where all variables are assumed to have mean 0, and $A_{i}$ is distributed independently of both $B_{i}$ and $u_{i}$, but $...
1
vote
0answers
20 views

Are Poisson Regressions with Serial Correlation Biased or Inconsistent? (No Fixed Effects)

Let's say I've got panel data where a count outcome $y$ and continuous independent variable $x$ observed each time period $t=(1,2,...T)$ for each individual $i$. I am interested in how $x_{it}$ ...
0
votes
0answers
31 views

Bayesian analysis of multilevel model with lagged dependent variable

Currently, I am constructed a bayesian multilevel model to analyze a panel data set which now basically looks like the following: $y_{ijt} = \beta_{0ij} + X\beta + \epsilon_{ijt}$. So, now only a ...
0
votes
0answers
43 views

Are vanishing bias and variance enough for pointwise consistency for KDE-based estimation?

Question: Is the condition that asymptotic bias and asymptotic variance goes to zero for infinite samples sufficient to guarantee the pointwise consistency of an estimator based on plug-in kernel ...
1
vote
1answer
50 views

Bayesian consistency in compact uncountable parameter space

Let $p(y_i \mid \theta)$ be the likelihood we are using of a single data point, $p(\theta)$ be the prior, and $f(y_i)$ the true distribution of the data. Also, let $\theta_0$ be the parameter that ...
0
votes
0answers
25 views

Gaussian process for machine learning consistent property explanation

I am currently reading Gaussian process for machine learning book from Christopher Williams, and I encounter a note on function-space view where consistency property is explained, what I am having ...
0
votes
0answers
32 views

consistency of an estimator not based on total sample size

How do I show the consistency of an estimator of a parameter, say $\mu$, that is not based on the sample size $n$ but a function of $n_{i}$'s where $\sum_{i=1}^{K}n_{i}=n$ ? Consider for example the ...
0
votes
0answers
35 views

Consistency of estimators

For $1\leq i\leq K$, I have an estimator of $\mu_{i}$ given by $\hat{\mu}_{i}=\frac{1}{K}\sum_{j\neq i=1}^{K}\frac{Y_{ij}}{n_{ij}}$, where $Y_{ij}\sim N(n_{ij}(\mu_{i}-\mu_{j}),\sigma^{2}n_{ij})$. ...
0
votes
0answers
14 views

Fisher information under different noise models

Directly lifted from Wikipedia: Fisher information (sometimes simply called information) is a way of measuring the amount of information that an observable random variable $X$ carries about an unknown ...
1
vote
0answers
25 views

Unbiasedness and consistency

Assume the simple regression model satisfying all Gauss-Markov assumptions. Somebody suggests the estimator Why may someone consider such an estimator? Why will this estimator be consistent? Why ...
3
votes
1answer
155 views

Why is it important that estimators are unbiased and consistent?

I am clear on the definition of unbiasedness and consistency. But why are these the criteria we use to judge whether an estimator is a good one? There are other criteria, of course, like the variance ...
1
vote
0answers
26 views

How does the concept of consistency apply to the full bayesian posterior as opposed to a single estimate?

Towards the goal of making a bayesian statistical inference, I start by collecting $M$ independent and identically distributed data observations $D_i$. Then I take a Bayesian approach to learning the ...
2
votes
0answers
30 views

Asymptotic consistency and normality

I need help getting the following problem Let $X_1,..,X_n$ be independent $N(\mu,1)$-distributed random variables. Define $\hat{\theta_n}$ as the point of minimum of $\sum_{i=1}^n(X_i-\theta)^4$...
4
votes
3answers
257 views

How is it that an ML estimator might not be unique or consistent?

Christian H Weiss says that: In general, it is not clear if the ML estimators (uniquely) exist and if they are consistent. Can someone explain what he means? Do we not generally know the shape of ...
0
votes
1answer
65 views

Consistency of the estimator of the variance of the error

In the classical linear regression model, the estimator of the variance of the regression error is $s^2 = \frac{e'e}{n-k} = \frac{u'Mu}{n-k}$ where u is the error vector, e is the residual vector, and ...
1
vote
1answer
39 views

instrumental variables property

How can I show that the instrumental variables (IV) estimator is consistent from this equation using the two stage least squares method? Where does this equation come from?
1
vote
1answer
57 views

Role of random sample assumption in consistency of OLS estimator

I guess in part what this all amounts to is what does the assumption {(x_i,y_i) : i=1,2,...,n} being i.i.d. imply about the i.i.d-ness of functions of it? I am confused because for example I have ...
0
votes
0answers
48 views

MLE Asymptotics of Two independent samples

Given two independent samples, parameterized by the same parameter yet with different distributions (for example, Exponential(lambda) and Gamma(lambda, 2)), under what conditions is the parametric MLE ...
0
votes
1answer
16 views

Cronbach alpha and item selection

I am a relative novice when it comes to statistical analysis, so forgive me if my question is unclear, or simply stupid. I am also not a matemathician, as I work in social science research, but I have ...
0
votes
1answer
56 views

Does “random effect” really exist in real data when we use random/mixed effect model? [closed]

If I understand correctly, here is a standard case when we need the mixed effect model: We are interested in studying the how drugs influence human health conditions, so we collected information ...
2
votes
2answers
109 views

$\sqrt{n}$-consistency of M-estimator based on plug-in estimator

Note: This is a follow-up on a previous question that was concerned about consistency, but this time seeking $\sqrt{n}$-consistency. Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = ...
6
votes
1answer
123 views

Consistency of M-estimator based on plug-in estimator?

Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = \hat{\theta}(\eta)$ that solves the estimating equation $$S_n(\tilde{\theta}, \eta_0) = 0$$ where $\eta_0$ is a nuisance ...
0
votes
0answers
23 views

Bayesian asymptotics

I'm reading about Bayesian asymptotics, but I'm getting confused by the different definitions and notations used here and there. Could anybody explain to me what is the difference between ...
3
votes
1answer
71 views

Consistency and rates of convergence

Suppose that I have two statistics that are known to be consistent , e.g : $ S_{n} ^2 $ (biased sample variance about sample mean) and $ S_{n-1}^2$ (bessel-corrected sample variance, that is unbiased)....
0
votes
0answers
15 views

Seeking a measure of multiple rater consistency for cateorical ratings with sparse data

I have multiple raters and sparse data. The raters can decide how many attributes apply to a person. I have many empty cells due to many attributes and few attributes actually chosen. What tools ...
2
votes
1answer
67 views

Most relaxed assumptions to get consistency of linear regression?

What are the most relaxed assumptions to get consistency of the linear regression estimates with $p$ variables? The most basic assumptions that I know are in White (1984): 1) The model is correct 2)...
7
votes
2answers
1k views

why does unbiasedness not imply consistency

I'm reading deep learning by Ian Goodfellow et al. It introduces bias as $$Bias(\theta)=E(\hat\theta)-\theta$$ where $\hat\theta$ and $\theta$ are the estimated parameter and the underlying real ...
3
votes
1answer
128 views

Parameter estimation and model selection consistencies

In a highly cited paper by Zhao (2006) it is stated that (Section 2) An estimate which is consistent in terms of parameter estimation does not necessarily consistently selects the correct model (or ...
3
votes
1answer
258 views

Consistency of lasso

I would appreciate help in understanding the following theorem from Knight and Fu (2002) paper: Consider linear regression model of the form $$Y_i = \beta_0 + x_i'\beta + \varepsilon_i,$$ where $\...
0
votes
0answers
62 views

Big Oh Pee / Little Oh Pee & Matrix Calculus

It's clear from Big Oh Pee / Little Oh Pee calculus that for $A_n$, $B_n$ scalars, we have $A_n = Op(a_n), B_n = op(b_n) \implies A_n B_n = op(a_n b_n)$ However, does the same applies to matrices? ...
1
vote
0answers
49 views

Are the Inverses of two asymptotically equivalent matrices themselves asymptotically equivalent

Suppose $M_n = P_n + op(1)$. Is it the case that $M_n^{-1} = P_n^{-1} + op(1)$, if both $M_n^{-1}$ and $P_n^{-1}$ exist with probability going to 1 as $n$ increases? Can the Continuous Mapping ...
1
vote
0answers
37 views

Consistency of the risk estimator in the L1 regularized logistic regression

Prof. Ryan Tibshirani in CMU explained in his class notes how to prove the consistency of the risk estimator in the L1 regularized linear regression, but how to prove the consistency of the risk ...
6
votes
1answer
52 views

Is $R^2_{adjusted}$ both unbiased and consistent under the alternative in simple regression?

Consider a simple regression model $$ y_i = \beta_0 + \beta_1 x_i + \varepsilon_i. $$ and suppose it is the correct model for the data. As far as I know, $R^2_{adjusted}$ is an unbiased estimator of ...
1
vote
1answer
191 views

Showing Bayes Estimator is Consistent

$X_1, X_2, ... , X_n$ are iid $N(0, \theta)$ random variables with $\theta$ in $(0, \infty)$. With the prior distribution $\pi(\theta)$=$\frac{4e^\frac{-2}{\theta}}{\theta^3}$, I calculated the ...
0
votes
0answers
28 views

Model selection consistency of Dantzig selector

Is it known that Dantzig selector of Candes and Tao: https://arxiv.org/abs/math/0506081 has model selection consistency, i.e, with high probability aporoaching 1 the model will select true features,(...
5
votes
0answers
48 views

Do shrinkage estimators solve the Neyman-Scott paradox?

I read the following SE question: What problem do shrinkage methods solve? And I wondered if shrinkage estimators provide a consistent estimator of the sample variance in a "mixed-effects" model using ...
0
votes
2answers
376 views

Showing that estimator is consistent

Let $\hat{\theta}_n= -\frac{n}{\sum_{i=1}^n \log(X_i)}$, where $X_i$ are i.i.d. samples from distribution with pdf $\theta x^{\theta-1}$ for $x \in (0,1)$. How to prove that $\hat{\theta}_n$ is ...
0
votes
0answers
96 views

Asymptotic distribution of the first order statistics and consistency

$ X_{i} $'s are i.i.d and $ X_{1},...,X_{n} $ ~ $ f(x;a,\theta )=\dfrac{1}{\theta}e^{-\dfrac{(x-a)}{\theta}}I_{[a,\infty)}(x)$ I found that MLE of $ a $ is $\widehat{a}=X_{(1)} $ and MLE of $\theta$ ...
3
votes
2answers
112 views

Which is the relation between population/probability space/sampling?

I am trying to understand the relation between population/probability space/sampling. My arguments are divided in 3 sub-questions which trace my attempt to link in a logical way the three concepts. I ...