Skip to main content
16 votes
Accepted

Is the invariance property of the ML estimator nonsensical from a Bayesian perspective?

As Xi'an says, the question is moot, but I think that many people are nevertheless led to consider the maximum-likelihood estimate from a Bayesian perspective because of a statement that appears in ...
pglpm's user avatar
  • 1,306
8 votes
Accepted

Doubt in the Invariance Property of Consistent Estimators

$X_{n+1}$ converges not to a constant, but to a distribution with variance 1. Therefore, Slutsky's theorem (with the two "estimators" $\dfrac{1}{2}\overline{X}_n$ and $\dfrac{1}{2}X_{n+1}$ ...
Julian Schuessler's user avatar
8 votes

Is the invariance property of the ML estimator nonsensical from a Bayesian perspective?

From a non-Bayesian view point, there is no definition of quantities like $$p(x|\theta = -\sqrt \eta \lor \theta = \sqrt \eta)$$ because $\theta$ is then a fixed parameter and the conditioning ...
Xi'an's user avatar
  • 106k
8 votes

Neural network to read short strings - translational invariance in CNNs

If you really want to use deep learning for this, then I'd consider a character-level recurrent neural network (such as a bidirectional LSTM) or if you want a transformer, which would take as the ...
Björn's user avatar
  • 33k
7 votes

Neural network to read short strings - translational invariance in CNNs

You don't need deep learning for that. You have a list of keywords and need to match them. The problem is that they may be misspelled. Another problem is that sometimes you need to match a keyword ...
Tim's user avatar
  • 139k
6 votes

Is there any difference between estimating $\sigma^2$ and $\sigma$ in a simulation study?

I find this question of interest because it highlights the artificial nature of seeking unbiasedness above everything else. A few points: the variance $\sigma^2$ allows for an unbiased estimator, ...
Xi'an's user avatar
  • 106k
5 votes
Accepted

Are HPD intervals invariant to reparameterization?

In general, they are not, even under the hypothesis that the transformation is monotonic. This is due to the fact that, when we transform variables, the density of the transformed variable is the ...
Lucas Prates's user avatar
  • 1,223
5 votes
Accepted

Find the MLE of $\mathbb{P}(X<Y)$ for $X\sim N(\mu_X,\sigma^2)$, $Y\sim N(\mu_Y,\sigma^2)$

We know $X-Y$ is normally distributed so let $Z=X-Y$ then, \begin{equation} Z\sim N(\mu_X - \mu_Y,2\sigma^2) \end{equation} MLE of $\mu_X$ and $\mu_Y$ are $\hat{\mu}_X = \overline{x}$ and $\hat{\mu}_Y ...
homelessmathaddict's user avatar
4 votes
Accepted

Significance of parameterisation invariance of Jeffreys prior

I think I see your point. If you define a prior probability density by $\pi_{\theta}(x) = \frac{d}{d\theta}F(\theta)$ then under (nice) reparametrization $\lambda(\theta)$, you get that that $\pi_{\...
Pohoua's user avatar
  • 2,618
4 votes
Accepted

Jeffreys' prior invariance under reparametrization

The "invariance" of Jeffreys' priors should have been called "equivariance" to avoid confusion. It does not mean that it does not change under reparameterisation but rather that ...
Xi'an's user avatar
  • 106k
4 votes
Accepted

Ways of implementing Translation invariance

This answer by Matt Krause on What is translation invariance in computer vision and convolutional netral network? contain some pointers: One can show that the convolution operator commutes with ...
Franck Dernoncourt's user avatar
4 votes
Accepted

Fisher Information invariant by a specific reparameterization of the Exponential Distribution

This is the impact of the Jacobian term for this specific transform (and only for this specific transform): denoting $I_1$ the information on $\beta$ and $I_2$ the information on $\lambda$ $$I_1(\beta)...
Xi'an's user avatar
  • 106k
4 votes
Accepted

Meaning of Invariance of Maximum Likelihood Estimator

Casella and Berger (2002) explain this by saying that, "In many cases, this simple version of the invariance of MLEs is not useful because many of the functions we are interested in are not one-...
Ben's user avatar
  • 127k
3 votes

Induced Likelihood Function for Max Likelihood Estimators

i) Yes the apostrophe is here to emphasize that these are two differents functions (that may return differents values). Take for example $X \sim N(\theta, 1)$ and $g(\theta) = \theta-1$ then $$ L_X(1,...
periwinkle's user avatar
  • 3,523
3 votes
Accepted

Why is invariance (in relation to neural networks) called invariance?

The term "invariance" or "invariant" in this context is not directly related to the statistical meaning of the term "variance" - it is using the basic English meaning of the words variant/varying/etc: ...
Bryan Krause's user avatar
  • 1,506
3 votes
Accepted

Is it alright to place equality constraints on the items loadings for assessing configural invariance using effect coding approach?

Is it alright to place equality constraints on the items loadings for assessing configural invariance using effect coding approach? Yes, invariance/equality of measurement parameters can be tested ...
Terrence's user avatar
  • 2,138
2 votes

Invariance of mutual information (two-dimensional Gaussian)

This is just a calculation error. Your first D should be $D=log(\sigma_1/\sigma_2)$. So they are the same.
siyisoy's user avatar
  • 21
2 votes

What is Exact definition of Invariance principle

The basic intuition behind invariance is that statistical conclusions should not depend on choice of measurement scale. Some examples: Measurement of distance in meters or parsecs. Angle measurement ...
kjetil b halvorsen's user avatar
2 votes
Accepted

Intuitive explanation of "invariance"

Invariance is a relative property not an absolute one. Something is described as invariant under some manipulation. For a non-mathematical (and hence non-statistical) example we consider the egg. The ...
mdewey's user avatar
  • 17.9k
2 votes
Accepted

Finding a distribution with a particular invariance property: F(x/b) - F(x/a) independent of x

Suppose $F$ has a derivative $f$ and the support is all real numbers. Then, $f(\frac{x}b) \frac{1}b=f(\frac{x}a) \frac{1}a$ and with $x=a y$, $f(\frac{a y}b) \frac{a}b=f(y)$. Let $r=\frac{a}b$. Then, $...
John L's user avatar
  • 2,483
2 votes
Accepted

On the definition of maximal invariant

Suppose $a(y) = a(y')$ but $y'$ cannot be expressed as a transformation of $y$ for any $g \in \mathcal{G}$. In other words, the equivalence class $[y']$ that $y'$ belongs (aka, the "orbit") ...
Zhanxiong's user avatar
  • 20.1k
2 votes
Accepted

Multi-group SEM - constraints and regression paths

The regression slope coefficients in your structural (latent variable) model only involve the covariance structure (latent variances and covariances). Therefore, loading (metric/weak) invariance is ...
Christian Geiser's user avatar
2 votes
Accepted

Why the Pitman estimator is given by the sample mean of X and Y?

The general formula for the best equivariant estimator is $$\delta(x)=\frac{\int_{-\infty}^{\infty} \theta f(x_1-\theta,\dots,x_n-\theta)\,\text d\theta}{\int_{-\infty}f(x_1-\theta,\dots,x_n-\theta)\,\...
Xi'an's user avatar
  • 106k
2 votes

Measurement invariance: Model fit increases with increasing constraints

lavaan gives me this warning: "some restricted models fit better than less restricted models That's not what shows in your output. The threshold-equivalence ...
Terrence's user avatar
  • 2,138
2 votes

Does measurement invariance analysis assume equal latent scores across groups?

In the metric invariance model (invariant loadings but non-invariant intercepts) latent means are typically not estimated (i.e., they are fixed at zero in all groups and therefore "equal" ...
Christian Geiser's user avatar
1 vote

Compare different loadings of a SEM model across different datasets

You can pass a list() of covariance matrices (and a vector of corresponding sample sizes) to lavaan() instead of raw ...
Terrence's user avatar
  • 2,138
1 vote
Accepted

Data simulation in R for Measurement invariance

Simulating data with measurement invariance is easy with the lavaan package. See the R code below. To simulate measurement ...
Preston Botter's user avatar
1 vote
Accepted

If $X_1 \sim \text{binom}(p_1,n_1)$ and $X_2 \sim \text{binom}(p_2,n_2)$, how to prove that the MLE of $p = p_1 - p_2$ is $\hat{p}_1 - \hat{p}_2$?

I'll show you how the invariance property of the MLE applies to this case. Consider the joint distribution of the two random variables, which depends on the parameter vector $\mathbf{p} = (p_1,p_2)$. ...
Ben's user avatar
  • 127k
1 vote

Which is the measurement invariance level necessary for a multiple regression analysis?

It depends on what the substantive question you are trying to answer is, but in general scalar invariance is required for group comparisons. Let's say you have the following regression model: $y = α + ...
peter1002's user avatar
1 vote
Accepted

Bayesian estimator under transformation of the parameters

Let's say you want to estimate the posterior on $\phi$, you can do the following; $$ \begin{aligned} p(\phi \vert x) &= \int d\mu \int d\sigma^2 \ p(\phi, \mu , \sigma^2\vert x)\\ &= \int d\mu ...
Peter Pang's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible