RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

learn more… | top users | synonyms (1)

1
vote
0answers
25 views

Determine best ARIMA model with AICc and RMSE

I have done a training set to fit different ARIMA models and then a test set to assess their performance (with R). From what I understood, I can use the AICc to determine the best model by choosing ...
1
vote
0answers
22 views

What measure should I use to compare two sets of calculations?

Assume I have two sets of calculations produced by two different simulators. There is no way to precisely measure actual values for these calculations, so I'm defaulting to the assumption that one of ...
0
votes
1answer
30 views

Comparing RMSE to model

I'm assessing the accuracy of the prediction of my model using the RMSE on a new data set. Now the RMSE in itself doesn't give any indication of whether it is a good model since there is no threshold ...
3
votes
3answers
115 views

Correct Grammatical Form for RMSE? [closed]

Which is preferred for use in journal publication: Root-Mean-Square Error or Root Mean Squared error? Root-mean-square sounds a little casual, like "ice tea" instead of "iced." On the other hand, ...
0
votes
0answers
40 views

Why does RMSE underestimate model variance?

I have read that RMSE of calibration/validation/cross validation is frequently used for model selection (e.g., for ANN), but can lead to over-fitting because the prediction error represents the ...
1
vote
1answer
29 views

Cross Validation Train Test Gap Question

Question: is minimizing test set mean validation error more important than the gap between train and test errors? Let's say I can tweak parameters in my model to give me mean validation error of 4500 ...
1
vote
1answer
36 views

Using a combined RMSE?

I have 12 soil water sensors with a few years of actual soil water samples that have been retrieved from near each of the sensors. We have found that individually regressing the data from each sensor ...
1
vote
0answers
21 views

How to identify the prediction equation from a regression model using splines

I find it difficult to connect the coefficients of a regression model that includes splines to the actual prediction equation. For example, how could that be done with the following model? ...
1
vote
0answers
33 views

What are the bias and variance of a model returning the observed mean for a training set?

It seems to me that bias = variance = 0 but MSE > 0, possibly very high, so clearly my intuition, and math, are wrong. For a training set $T$ and a regression problem let $M(T) = \text{Ave}(y(T))$. ...
0
votes
0answers
30 views

Interpreting RMSE and MAE values

Returns of a index, where daily returns ($R_t$) are defined as $R_t:=\log(P_t/P_{t-1})$. And daily volatility ($\sigma_t^2$) is defined as $R_t^2 = \sigma_t^2$. After an evaluation of a naive ...
0
votes
0answers
11 views

How to see if my calculated values match the actual value?

I wanted to confirm that I was using the right statistics to measure the whether or not our experimental model of calculating the speed of sound was accurate. We experimentally calculated the speed ...
1
vote
1answer
33 views

Notation for computing MSE confuses me?

I wish to compute MSE of my models. Say my data was generated from the following model: $y_i=f(x_i)+e_i$ where $e_i$ is some noise around the true relationship $...
0
votes
1answer
116 views

Evaluate forecasting ability of GARCH models with RMSE and MAE

I am evaluating different forecasting models and their ability to forecast index volatility during period of market turmoil, using two measurements, Root Mean Square Error and Mean Absolute Error. For ...
1
vote
1answer
107 views

cross validation and validation( to find RMS error and correlation ) in matlab

Dear Experts; i have text data (sample points are 324) of different climatic parameters. 3rd column of each text file was contained some missing or NaN data. Using Scatter data interpolation in ...
1
vote
0answers
94 views

Very Large RMSE with Linear Regression

I am working on a regression problem that has about 180 binary features and approximately 280,000 data samples. For certain train-test splits of my data, the resulting linear-regression model (...
0
votes
0answers
27 views

RMSE vs R Squared literature

I have found many related answers and explanations, but not one that involves literature. I have a model and I have used cross validation. Some models have really high R and adjusted R squared values, ...
0
votes
0answers
42 views

How to use RMSE when having data normalization?

I am new in machine learning and I am studying time series prediction using neural networks. Pseudocode 1: ...
1
vote
1answer
83 views

How does Stata calculate RMSE in regression with weights?

This problem came up because I was trying to replicate some results I was getting in Stata with R, and I was able to replicate everything except for the root mean squared error. When I run a ...
3
votes
1answer
142 views

RMSE is scale-dependent; is RMSE%?

I've got a graph of RMSE% vs. unit size and it declines nicely. Is this scale-dependence or does the "%" compensate for that? $$ \text{RMSE%} = 100\% \cdot \frac{\sqrt{\frac{1}{n}\Sigma_{i=1}^n (y_i ...
0
votes
0answers
50 views

compare different Imputation method by RMSE

My original dataset : ...
0
votes
0answers
49 views

Standard errors and confidence interval in cox regression model validation using RMS package

I am using RMS package of R to validate cox regression model with bootstrap. Please see the sample code below. I have three questions: (1) How do I request the standard errors and/or confidence ...
1
vote
1answer
54 views

Which denominator to use for cross-validated RMSE: $n$ or $n-k$?

I'm having a hard time understanding when I should take out the number of parameters from the denominator of the root mean squared error. From what I understood (other question, and wiki article), if ...
0
votes
0answers
24 views

Custom metric in Caret that puts more weight on bias

I am fitting a machine learning model that needs to have a low bias; variance is not as important. As such I would like to fit a model that places more weight on bias, than using the custom metric for ...
0
votes
0answers
13 views

Assessing a vector of errors in modeling

The quality of a model is often assessed based on a figure of merit such as RMSE. This reduces the individual errors in the model to a single number without assessing the errors as a population of ...
0
votes
0answers
104 views

How to interpret poor $R^2$ score but good RMSE value?

I split my data into training set and test set and am running linear regression on it. I am using Python's "scikit" library and I am getting an $R^2$ score of 0.31 and an RMSE value of 0.037. The ...
0
votes
2answers
34 views

How to apply a model from one data set to another

I'm still new to SAS, and I need to take a basic OLS regression from one data set, and find the RMSE when applied to a different set. This feels simple, but I haven't found anything. Help would be ...
1
vote
0answers
58 views

Shouldn't the root mean square error (RMSE) be called root mean square residual?

As far as I understand, estimating the error of a model, say an artificial neural network, requires to know the "true" model. Wikipedia says in its article "Errors and residuals": "The error (or ...
0
votes
1answer
37 views

How to request predicted points for each patient based on nomogram in RMS?

I am generating nomogram for a survival analysis project using RMS package of R. A PI of the project would like to have predicted points for each patient in the sample predicted by the nomogram. Is ...
0
votes
1answer
23 views

RMS and probability of steady values

Given an RMS value of a signal/waveform/list of numbers, I would like to construct an equation that calculates the probability that $n$ consecutive points are the same (or close). Suppose I have a ...
0
votes
0answers
130 views

How can I calculate the root mean square error (RMSE) for two covariance matrices?

I want to compare different methods of estimating the covariance matrices on the basis of RMSE and will recommend having the minimum RMSE. I have a sample of, say, 356 weekly observations of 10 ...
0
votes
0answers
95 views

Minimizing RMSE while using a log-transformation

I am trying to fit a linear model to some data, but the dependent variable y is clearly not normally distributed. It has a heavy tail on the right. A log-transformation helps to make the ...
0
votes
1answer
84 views

Is it possible to compare the RMSE of different models with different datasets?

I have 5 datasets and for each one of them I created 2 prediction models. For such task I divided each dataset in training and testing set (70/30%). Then I assessed the RMSE for each one and ...
0
votes
2answers
108 views

How to predict using ordered probit regression and calculate prediction accuracy?

I want to do an ordered probit regression, then cross-validate model prediction accuracy with 80% data for training and 20% for validation, and calculate RMSE for predictions. Consider this dataset: ...
0
votes
0answers
63 views

How to compare forecast accuracy of ordered probit and the equivalent Bayesian heierarchical model in R?

I have a dataset of a metric predictor variable $X$, and an ordered categorical predicted value $Y$ for several individuals. The dataset are from two groups $G_1$ and $G_2$. I want to estimate $Y$ ...
0
votes
0answers
233 views

How to discuss RMSE, MAE, $R^2$ results. What values are good/bad?

I have regression results for a study where I'm trying to develop a predictive model to estimate the value of a continuous output value for a physical process. My database has around 400 records and ...
3
votes
2answers
1k views

RMSE (Root Mean Squared Error) for logistic models

I have a question regarding the validity of using RMSE (Root Mean Squared Error) to compare different logistic models. The response is either 0 or ...
1
vote
0answers
29 views

Standard performance measure for regression?

I will perform time-series prediction and I will report the accuracy of my system with a measure like RMSE or MAE. However, the variables I will predict are in different ranges. So let's say one is ...
0
votes
2answers
165 views

How to change baseline patient in Predict function in rms package in R?

I am doing a time dependent Cox model using cph function in rms package. I use Predict and plot.Predict to plot the hazard ratio on y axis and a continuous covariate (e.g. LDL cholesterol) on X axis ...
1
vote
0answers
80 views

Root-Mean Squared Error for Bayesian Regression Models

I'm trying to get a sense of my prediction errors for a Bayesian regression model and I was using the Root-Mean-Squared Error. My question is, since are predictions are stochastic, would it make ...
0
votes
1answer
66 views

Using {rms} package for multinomial logit

Is it possible to use the rms package to model multinomials logits, or elsewise to model several binary logits to achieve the same effect? I am aware that there are many other packages specifically ...
0
votes
1answer
69 views

Visualizing nonlinear regression

I have following model using mtcars dataset: ...
0
votes
0answers
208 views

KS, RMSE tests in R

I have a set of yearly peak river flows and I am trying to use the KS test to conclude if the data is coming from a generalized extreme value (GEV) distribution. I am also trying to calculate a root ...
2
votes
1answer
67 views

Can there be a situation where one regression model gives lower RMSE than the other but also lower R-squared?

Consider the following scenario where you use the same data X (the same number of predictors p, same number of observations <...
0
votes
1answer
580 views

what is the meaning of RMSE in caret::train [duplicate]

I'm confused by the exercise solutions of the book Applied Predictive Modeling. In https://github.com/topepo/APM_Exercises/blob/master/Ch_06.pdf at the beginning of page 4 ...
1
vote
0answers
81 views

Recommendation system and baseline predictors

I'm participating in programming contest, where I have a data, and where the first number is a user, second number is a movie, and the third is a number in then-points rating. ...
0
votes
1answer
53 views

RMSE normalisation, what method prefer?

According to this article on wikipedia http://en.wikipedia.org/wiki/Root-mean-square_deviation, two methods are widely used to normalise the RMSE. The first is dividing by the range: $$NRMSE = \frac{...
2
votes
1answer
84 views

Comparing linear and nonlinear models

Is it possible to compare between these two types of model? I have a set of data that involves 6 independent variables and 1 dependent variable. It is based of a questionnaire for social science ...
3
votes
3answers
261 views

Smaller standard errors *after* multiple imputation?

I have 1771 observations, with 30% missing data for x1 (Yes:No), and no other missing values from 26 other variables (mix of continuous and factor). I am using quantile regression in R, with and ...
6
votes
1answer
9k views

How to interpret error measures in Weka output?

I am running the classify in Weka for a certain dataset and I've noticed that if I'm trying to predict a nominal value the output specifically shows the correctly and incorrectly predicted values. ...
0
votes
1answer
216 views

MSEP and R2pred for Linear Model

I have two set of data 1-Training (Calibrating) 2-Test. With these datasets, I Fit the model using first dataset. predict using the second dataset x-variables I have to test the closeness of the ...