The error of an estimate or prediction is its deviation from the true value, which may be unobservable (e.g., regression parameters), or observable (e.g., future realizations). Use the [error-message] tag to ask about software errors.

learn more… | top users | synonyms

1
vote
1answer
20 views

Cross Validation Train Test Gap Question

Question: is minimizing test set mean validation error more important than the gap between train and test errors? Let's say I can tweak parameters in my model to give me mean validation error of 4500 ...
0
votes
0answers
14 views

R , code execution no to stop even after getting error [on hold]

Need some clarifications for my R code execution stops / terminates after getting error inside while loop but i want my code to run next line which is 2nd while loop and subsequently 3rd loop , The ...
0
votes
0answers
5 views

How to translate theorem into R code properly? (ruin probability of a discrete-time bi-risk model) [migrated]

We're having trouble with the realization of this theorem (recurrence relation for estimating ruin probability of a discrete-time bi-risk model). Basically, here you can see the theorem itself with ...
1
vote
1answer
14 views

Loss function/error measurement for allocation problem

I'm working on a prediction rule for an allocation problem. So, it's data like this: ...
0
votes
0answers
25 views

Does the unconditional distribution of $y_i$ only depend on the distribution of the errors?

In linear regression, does the unconditional distribution of $y_i$ only depend on the distribution of the errors? For example, is it not the case that if $$y_i = \beta_0 +\beta_1 x_i + u_i $$ and ...
0
votes
0answers
12 views

Tweaking SVM error rates?

I currently have a one-vs-all SVM setup. Each SVM outputs a score. If I take the maximum score as the correct corresponding class, this gives me FARs of 0.008%. However it also gives me FRRs of about ...
2
votes
1answer
41 views

Is the margin of error for a survey a “fake” statistic?

When I conduct a survey for a client, they are often very concerned about the margin of error for the survey, and that’s totally reasonable. Oftentimes however, when they say, “We want a margin of ...
0
votes
0answers
9 views

An appropriate measure of predictive accuracy on non-uniform data?

When you have a model making a prediction about non-uniform data, how do you approach deciding on an appropriate measure of accuracy? For example, if the the data tends to be a normal distribution ...
0
votes
1answer
13 views

Measuring forecasting risk of linear regression

I want to measure how much risk I take by forecasting something. I know I can measure the error and things like MAD, MSE, ...
1
vote
1answer
23 views

Other types of mean error

Let $\tilde f$ be an approximation of the function $f(x) = \arccos(x)$. I'm using MATLAB to figure out how good this approximation is by calculating a mean error. My first idea was to use this formula ...
3
votes
0answers
22 views

Why is the standard deviation the error on the singular measurement?

I'm a beginner with the study in data analysis in Physics. I'm trying to understand the meaning, in the field of experimental Physics, of the standard deviation $\sigma$ of a series of data. There ...
1
vote
0answers
31 views

Alternatives to predictions' MSE that are 'proportional' to $Y$

Assume that all values are real. let $Y$ be a vector of observations and $\hat{Y}$ be a vector of predictions. Then the MSE of the predictions is $$1/n\sum_{i=1}^n(\hat{Y}_i-Y_i)^2$$ Let $$S = ...
0
votes
0answers
10 views

Planned Orthogonal Comparison Type I Error Rate and the Number of Independent Contrasts

I am confused about Type I error rate and the number of planned independent contrasts in Planned Orthogonal Comparisons in one-way ANOVA. If we have a factor with 5 groups and we would like to test ...
0
votes
0answers
22 views

Using a change detection error matrix approach for raster aggregation

I am assessing annual land use change along a 10 year period and have rasterized a vector land use dataset to base resolution 'n' metres to do so (vector data is taken as ground truth, errors in it ...
3
votes
1answer
39 views

Error term in multiple regression model

I am trying to run a multiple regression model to see the effect of field characteristics such as soil texture, slope and hydraulic conductivity on drainage density. My samples are agricultural ...
0
votes
0answers
19 views

Modeling error in regression

A few weeks ago I posted in this forum about a regression analysis I wanted to run. My outcome was number of organs and they values went from 1-7. Well, as someone pointed out, I could have some bias. ...
2
votes
0answers
26 views

Estimator of true probability — understanding margin of error for very small probability

I have a coin whose probability of landing on heads when flipped is unknown, but could be anywhere between 0 and 100%. I want to flip the coin some number of times and estimate the true probability ...
0
votes
0answers
7 views

Gini impurity and generalization error

Has anyone seen papers on relationship between information-based criterions (such as Gini impurity, information gain etc.) and generalization error? Is there theoretical justification of using such ...
1
vote
0answers
33 views

Forecast accuracy metric for forecasts over different time horizons

I a dataset of 81 oil price forecasts from more than 30 different forecasters. Those forecasts consist of a forecast made on various days in 2014 for the average oil spot price of 2015. For instance, ...
1
vote
1answer
19 views

How can I deal with numerical errors in a large-scale linear regression?

I am currently conducting a linear regression on a large-scale data set which has many sparse features ($\simeq 10^5$) and many observations ($\simeq 10^6$) by using scikit-learn. (Most of the ...
0
votes
1answer
31 views

Prediction interval, forecast error for a function of a forecast

I have two variables $X$ and $Y$. For each variable I created a forecasting model (using time series) and estimated $X_{t+1}$ and $Y_{t+1}$ and the prediction interval and the error for each. I have ...
0
votes
0answers
10 views

Error propagation taylor second-order (multivariate)

I'm trying to understand how the second-order approximation mentioned in this posting http://stats.stackexchange.com/a/13005 looks like for a mulitvariate function. I have only found the ...
0
votes
0answers
15 views

k.means relation between sum squared error and variance

I work with k-means algorithm and I don't understand the relation between sum squared error and variance. Is there a relation between these values?. I work with k=1. And the values are Sum squared ...
1
vote
1answer
22 views

calculate the internally studentized residual

it says that ...an ordinary residual divided by an estimate of its standard deviation $s(e_{i})$ As we can see from the example that mean for four residuals is 0, so ...
0
votes
0answers
27 views

Log transformed MAE to original value

In order to evaluate the forecast accuracy of a model I'm using a step wise cross validation to get a MAE value and use that again to calculate the MASE. As part of the model specification the data ...
0
votes
1answer
23 views

Compare RMSE between original and logged time series data

I have some time series data, both in the original form y and in logged form log(y). I should compare the models generated by applying neural networks and find the best one. How to decide which one is ...
0
votes
0answers
4 views

Interpolating data with errors (limited model knowledge)

I have data which I know follows a function $y = f(x)$ such that it is quadratic i.e. $y =\alpha x^2$ for some $\alpha$ when $x\rightarrow 0$ and $y = \beta x$ for large $x$. The data itself has ...
0
votes
1answer
40 views

Derivation of Equation of Reducible and Irreducible Error [duplicate]

I am currently reading An Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani, and I am stuck on one of the leaps they take when defining reducible and irreducible error. ...
1
vote
0answers
29 views

Alternating Least Squares Test Error better than Train

I have been running some trials for recommendations using Collaborative Filtering, specifically Alternating Least Squares (ALS). I am using two versions of ALS, one with fixed lambda regularisation ...
0
votes
1answer
23 views

How may I convert Perplexity to F Measure

In the practice of Machine Learning accuracy of some models are determined by perplexity, (like LDA), while many of them (Naive Bayes, HMM,etc..) by F Measure. I like to evaluate all the models with ...
0
votes
0answers
12 views

Can I propagate standard deviation and standard error together?

Can I mix standard deviation and standard error when propagating error? For example, if I multiply two values and one has error in terms of standard deviation and the other in terms of standard ...
0
votes
0answers
8 views

Modelling errors of linear & logistic regression

How can the errors of linear regression models be modelled to make the results even more accurate? Also, how are errors in logistic regression measured? Is it possible to model the errors of logistic ...
1
vote
0answers
31 views

How to compare posterior distributions for different observed data? KL-divergence?

So I'm solving an inverse problem with the Bayesian approach $p(u | y) \propto p(y| u )p(u)$. Assuming I have two datasets $y_1$ and $y_2$, what can be said about the difference in the posteriors ...
2
votes
1answer
18 views

What method for forecasting error measuring in a poisson process?

There's so much different measures for the forecast error that I kind of lose the sight on which one to use. Out of the following: MAPE, ...
0
votes
1answer
10 views

Size of Mean Squared Error (MSE)

Is there a rule or a scale for judging if a certain MSE is small, very small, big or very big? Please give also a source to underpin your recommendation.
0
votes
0answers
22 views

Are linear regression errors independent? Mean independent? Uncorrelated?

All I know is that we assume zero conditional mean (and hence zero mean) and conditional homoscedasticity (and hence homoscedasticity). When trying to prove that $E[(\hat{\beta_1} - \beta_1)\bar{u}] ...
0
votes
1answer
42 views

What to check in cross-validation - MAE or MSE?

When using cross-validation for model selection, should one look at MSE or MAE. I know that MSE and MAE are related but which is the more appropriate measure?
2
votes
1answer
22 views

ANOVA / t-test to compare the errors of different models

I have two forecasting models, moving average and single exponential smoothing. The values of Mean Absolute Percentage Error (MAPE) is 5.2%, 5.8%. Since the difference of MAPE between the models are ...
0
votes
0answers
22 views

How to use the off-diagonal terms of the covariance matrix when calculating confidence intervals?

The nonlinear fitting routine I use is MATLAB's fitnlm and it gives the covariance matrix. How can I take into account the off-diagonal values of this matrix to ...
4
votes
1answer
32 views

Difference between errors and residuals. What is the mean and variance of each of them?

Suppose there is a simple linear model $y=\beta_0+\beta_1x+u$. Can we state that $\bar y=\beta_0+\beta_1 \bar x + \bar u$? I have this question because I come up with $Var(\bar u)$ when doing some ...
0
votes
0answers
13 views

Effect of number of samples on error in classification/regression

Let's say for a classifier, $E_{out}$ is the test error and $E_{in}$ is the training error, $N_{out}$ is the no. of test samples and $N_{in}$ is the number of training samples. I want to know the ...
0
votes
0answers
31 views

Understanding Mean Squared Error

When determining the quality of an estimator, I understand a simple metric such as, if the expected and predicted are close, then we consider this instance to be correct. Then sum up all correct and ...
0
votes
0answers
19 views

Prediciton with a help of regression that minimizes the RMSE

I have two data sets containing the same variables (43 variables). The first set contains 107 observations and the second - 30. The dependent variable is "Revenue" (the values of this variable are ...
0
votes
1answer
28 views

T-test between groups while removing within group variability

I'm comparing two conditions based on an outcome variable of satisfaction with group learning (scale 1-9). The first condition consists of telling 3 groups of 3 people that their group project WILL be ...
0
votes
1answer
26 views

which model with count data and 3 factors, problem with glm

This is for a study for the abundance of counted individuals of the taxon Nematocera (gnats) in dependence of the three factors "Betrieb"(two levels), "Standort" (8 levels) and "Variante" (3 levels). ...
0
votes
0answers
10 views

Nonindependent Errors

I'm trying to better understand the way in which error can be nonindependent, specifically in terms of distinguishing between crossed/within vs. nested/between. That is, determining whether error ...
0
votes
0answers
17 views

Confidence interval for the median - continued

I have a set of values $x_i\quad i=1,…,N$ of which I can calculate the median M. Each $x_i$ has an error $\delta x_i$. The $x_i$ values are the result of a maximum likelyhood estimation and their ...
0
votes
0answers
24 views

How to calculate errors/confidence intervals of parameters from least squares fitting which have high correlation

I'm no statistician but I want to be more honest about my error bars. Let's say I have a dataset which is described by some model $f(x,{params})$ where ${params}$ is a vector of many parameters for ...
1
vote
0answers
12 views

How to sample for success/failure from small populations and probabilities

I am interested in sampling data that comes in a yes/no form. A couple things I would like help in understanding. Why does the reasonable sample size change when my population shrinks? We might be ...
2
votes
0answers
113 views

Regression models with comparable MAE but differing R²

I have trained two regression models on the same dataset. They perform with comparable mean absolut errors $MAE_{1,2} \approx 0.45$, but the coefficient of determination differs significantly with ...