New answers tagged

0 votes

why are errors normal in OLS?

As was mentioned by several commentators, ordinary least-squares does not require normal errors. Only when you want to say something about the sampling distribution of the OLS estimator in finite ...
Durden's user avatar
  • 859
1 vote

why are errors normal in OLS?

In OLS, the errors do not have to have a normal distribution or even any particular distribution at all. All OLS does is solve the correspondence: $$ \hat\beta_{OLS}\in\\ \underset{\beta=\left( \...
Dave's user avatar
  • 55.1k
0 votes

maximum likelihood estimator of regression coefficient

you can view this as repeated measurements at $x_\text{odd}=-1$ and $x_\text{even}=1$. I would rewrite your expression splitting sums over even and odd and using $n_\text{even}$, $n_\text{odd}$ and $\...
seanv507's user avatar
  • 5,798
7 votes

Why use a scoring rule different from the loss function?

In general, I absolutely agree with your point, and usually recommend that the same loss function should be used both in training and in evaluation. However, there are situations where you might want ...
Stephan Kolassa's user avatar
9 votes

Why use a scoring rule different from the loss function?

One place where this is done all the time is when the loss function includes a penalty term. We train the model according to some function that has the penalty term, but then we evaluate without that ...
Dave's user avatar
  • 55.1k
2 votes

Sufficient Statistic and Maximum likelihood

If there is a sufficient statistic, then it contains all the information that a sample has about the distribution. Other statistics that describe the sample are superfluous. To expand, I might be ...
Sextus Empiricus's user avatar
2 votes

How are robust standard errors applied in logistic regression

As Friedman once said, robust standard errors provide the right estimates for the wrong quantities if the model is misspecified. This is not very appealing. And Gould has found that for binary ...
Frank Harrell's user avatar
9 votes
Accepted

How to do maximum likelihood estimation when numerical derivatives cannot be calculated

There are optimisation algorithms that don't require derivatives. You can divide them into algorithms that assume derivatives exist but don't require them algorithms that don't assume smoothness A ...
Thomas Lumley's user avatar
2 votes

Likelihood determination for a step-like pdf

As $x_1$ and $x_2$ are independent, the PDF can be written as a mixture of two uniform distributions, one with probability $(1-f)$ and support $(0,1)$, and the other with probability $f$ and support $(...
jbowman's user avatar
  • 37.1k
2 votes
Accepted

Why does higher dimensional data has higher likelihood?

Actually this isn't always the case, it's just often the case when the data is sufficiently spread out or discrete. Let's start with the discrete case because it's the easiest. Suppose in one ...
Cliff AB's user avatar
  • 19.6k
0 votes

Relation between sample standard deviation from data and maximum likelihood estimates

The maximum likelihood estimator to which you refer (for a Gaussian likelihood) can be derived as $\hat\sigma =\sqrt{ \dfrac{ \sum_{i=1}^N\left( X_i-\bar X\right)^2 }{ N } }$. The function you’ve ...
Dave's user avatar
  • 55.1k

Top 50 recent answers are included