New answers tagged

2

I suspect that there has been a misunderstanding or a typo. In GLMMs the fixed effects estimate are conditional on the random effects. That is, the estimates apply to units in the same subject/cluster/group. They cannot be interpreted as an average over all subjects/clusters/groups. The latter would be marginal estimates, while the former are conditional. In ...


0

MAE, RMSE are in the same unit, but MSE is in unit-squares. The rules are: If you add/subtract two variables of the same unit, the result will be in the same unit, i.e. $u\pm u\rightarrow u$ If you multiply them, it will be unit-squares, i.e. $u\times u \rightarrow u^2$ If you divide them, the result is unit-less, i.e. $u/u\rightarrow 1$ Additionally, $\...


2

Accuracy is, always, the number of correct guesses out of the total number of guesses. If you guess the right category when there are two categories, you had an accurate prediction. If you guess the wrong category when there are ten categories, you had an inaccurate prediction. Think of it like your score on a multiple choice test. However, Cross Validated ...


2

First, lets make some assumptions: (Assumption 1) Suppose each measurement that you make with the ruler is of the form $\theta+E$ where $\theta$ is the "true" measure and $E \sim N(0,\sigma^2)$; (Assumption 2) Suppose each measurement you make is independent of the other; (Assumption 3) You can freely move the sticks $A$ and $B$; (Assumption 4) ...


1

I think the key part for these two loss functions is that they are unbounded due to their using logs. That is, with $x \in [0,1]$, as $x \rightarrow 0$, $\log{x} \rightarrow -\infty$. Here, "brittleness" seems to be implying that certain records can unduly influence the loss, so that a (small) number of these in the training set (and not the test ...


3

Mean absolute error is defined as an average of the absolute differences between observed values $y_i$ and the predictions for them $\hat y_i$ $$ \frac{\sum_{i=1}^n |y_i - \hat y_i|}{n} $$ So it is not a percentage of anything. My guess is that they mean that the error is measured in percentage point units, because their target is the relative runtime (p. 3),...


4

I haven't read the paper in detail (though their equation (2), where they claim that the MAE against the mean is the standard deviation, casts some doubt on the statistical expertise involved). If you divide the MAE by the mean of the time series, you can interpret the result as a weighted Mean Absolute Percentage Error (wMAPE; see Kolassa & Sch├╝tz, 2007)...


0

It usually makes little sense to average over different series' RMSEs, since these may be on very different scales. What does make sense is averaging some kind of relative RMSE. For instance, for each series, you could run a simple benchmark forecast, like the overall mean. This gives you an $\text{RMSE}_{\text{benchmark}}$ in the holdout sample. Now you run ...


0

A common error calculation is MAPE (Mean Absolute Percent Error) which is often used in demand forecasting. $$\frac{\sum|(actuals-forecasts)|}{\sum(actuals)}$$ In your case it seems that the simulations are the forecasts and the empirical data are the actuals. It is similar to what you are doing except all the numerators are summed up and all the ...


3

Does this usually imply that the dependent variable is normally distributed itself? If by "dependent variable" you mean the marginal distribution, then the answer is no. the easiest counter example is a t test where the data are truly normal. The data could be bimodal and OLS/Gaussian GLM could still be applied. Here is an example of that. ...


2

Since I want to show the validity of my prediction model ( by comparing the predicted values and actual ones), I need to show the errors in percentage There is no logical connection between showing the validity of a predicton model and using percentage errors. Percentage errors are one very specific loss function. There are others. A point (!) prediction ...


Top 50 recent answers are included