RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

learn more… | top users | synonyms (1)

0
votes
0answers
7 views

RMSE for shifted datasets

I am facing the problem to compare a series of simulations to a set of observations. Normally I would employ the Root Mean Square Error (RMSE) as goodness-of-fit measure to understand which is the ...
0
votes
2answers
29 views

Is Root Mean Squared Error computed according to model with all variables or only significant ones?

I made VAR model where there are some statistically insignificant variables. I ran forecast and I wanted to compute Root Mean Square Error. I tought that RMSE is computed according to model with only ...
0
votes
1answer
20 views

Comparison of 3 different data sets

I have two time varying datasets that are on different scales (one typically of the order of magnitude of 1e-3 and the other of order of magnitude 100). I've been working on a process that generates a ...
1
vote
0answers
14 views

statistical model goodness of fit

I have an article under revision where I created a regression model (Random Forests) to estimate a certain X variable. For that model I have reported the coefficient of determination (R2) and the root ...
0
votes
0answers
7 views

Evaluation metrics for kernel density estimation

Given a true (benchmark) density $p(x)$, and several density estimation algorithms, I want to empirically compare which one works better than the other. In this case, what kind of evaluation metric is ...
5
votes
1answer
131 views

Is it possible to compute RMSE iteratively?

I am working on continuous evaluation of a regression model on streaming data from sensors. I think that Mean Absolute Error (MAE) can be found out iteratively similar to this link for averaging. $$ ...
1
vote
0answers
30 views

Determine best ARIMA model with AICc and RMSE

I have done a training set to fit different ARIMA models and then a test set to assess their performance (with R). From what I understood, I can use the AICc to determine the best model by choosing ...
1
vote
1answer
29 views

What measure should I use to compare two sets of calculations?

Assume I have two sets of calculations produced by two different simulators. There is no way to precisely measure actual values for these calculations, so I'm defaulting to the assumption that one of ...
0
votes
1answer
32 views

Comparing RMSE to model

I'm assessing the accuracy of the prediction of my model using the RMSE on a new data set. Now the RMSE in itself doesn't give any indication of whether it is a good model since there is no threshold ...
3
votes
3answers
118 views

Correct Grammatical Form for RMSE? [closed]

Which is preferred for use in journal publication: Root-Mean-Square Error or Root Mean Squared error? Root-mean-square sounds a little casual, like "ice tea" instead of "iced." On the other hand, ...
0
votes
0answers
42 views

Why does RMSE underestimate model variance?

I have read that RMSE of calibration/validation/cross validation is frequently used for model selection (e.g., for ANN), but can lead to over-fitting because the prediction error represents the ...
1
vote
1answer
32 views

Cross Validation Train Test Gap Question

Question: is minimizing test set mean validation error more important than the gap between train and test errors? Let's say I can tweak parameters in my model to give me mean validation error of 4500 ...
1
vote
1answer
39 views

Using a combined RMSE?

I have 12 soil water sensors with a few years of actual soil water samples that have been retrieved from near each of the sensors. We have found that individually regressing the data from each sensor ...
1
vote
0answers
22 views

How to identify the prediction equation from a regression model using splines

I find it difficult to connect the coefficients of a regression model that includes splines to the actual prediction equation. For example, how could that be done with the following model? ...
1
vote
0answers
33 views

What are the bias and variance of a model returning the observed mean for a training set?

It seems to me that bias = variance = 0 but MSE > 0, possibly very high, so clearly my intuition, and math, are wrong. For a training set $T$ and a regression problem let $M(T) = \text{Ave}(y(T))$. ...
0
votes
0answers
33 views

Interpreting RMSE and MAE values

Returns of a index, where daily returns ($R_t$) are defined as $R_t:=\log(P_t/P_{t-1})$. And daily volatility ($\sigma_t^2$) is defined as $R_t^2 = \sigma_t^2$. After an evaluation of a naive ...
0
votes
0answers
11 views

How to see if my calculated values match the actual value?

I wanted to confirm that I was using the right statistics to measure the whether or not our experimental model of calculating the speed of sound was accurate. We experimentally calculated the speed ...
1
vote
1answer
33 views

Notation for computing MSE confuses me?

I wish to compute MSE of my models. Say my data was generated from the following model: $y_i=f(x_i)+e_i$ where $e_i$ is some noise around the true relationship $...
0
votes
1answer
133 views

Evaluate forecasting ability of GARCH models with RMSE and MAE

I am evaluating different forecasting models and their ability to forecast index volatility during period of market turmoil, using two measurements, Root Mean Square Error and Mean Absolute Error. For ...
1
vote
1answer
118 views

cross validation and validation( to find RMS error and correlation ) in matlab

Dear Experts; i have text data (sample points are 324) of different climatic parameters. 3rd column of each text file was contained some missing or NaN data. Using Scatter data interpolation in ...
1
vote
0answers
107 views

Very Large RMSE with Linear Regression

I am working on a regression problem that has about 180 binary features and approximately 280,000 data samples. For certain train-test splits of my data, the resulting linear-regression model (...
0
votes
0answers
28 views

RMSE vs R Squared literature

I have found many related answers and explanations, but not one that involves literature. I have a model and I have used cross validation. Some models have really high R and adjusted R squared values, ...
0
votes
0answers
44 views

How to use RMSE when having data normalization?

I am new in machine learning and I am studying time series prediction using neural networks. Pseudocode 1: ...
1
vote
1answer
92 views

How does Stata calculate RMSE in regression with weights?

This problem came up because I was trying to replicate some results I was getting in Stata with R, and I was able to replicate everything except for the root mean squared error. When I run a ...
3
votes
1answer
154 views

RMSE is scale-dependent; is RMSE%?

I've got a graph of RMSE% vs. unit size and it declines nicely. Is this scale-dependence or does the "%" compensate for that? $$ \text{RMSE%} = 100\% \cdot \frac{\sqrt{\frac{1}{n}\Sigma_{i=1}^n (y_i ...
0
votes
0answers
52 views

compare different Imputation method by RMSE

My original dataset : ...
0
votes
0answers
57 views

Standard errors and confidence interval in cox regression model validation using RMS package

I am using RMS package of R to validate cox regression model with bootstrap. Please see the sample code below. I have three questions: (1) How do I request the standard errors and/or confidence ...
1
vote
1answer
59 views

Which denominator to use for cross-validated RMSE: $n$ or $n-k$?

I'm having a hard time understanding when I should take out the number of parameters from the denominator of the root mean squared error. From what I understood (other question, and wiki article), if ...
0
votes
0answers
27 views

Custom metric in Caret that puts more weight on bias

I am fitting a machine learning model that needs to have a low bias; variance is not as important. As such I would like to fit a model that places more weight on bias, than using the custom metric for ...
0
votes
0answers
14 views

Assessing a vector of errors in modeling

The quality of a model is often assessed based on a figure of merit such as RMSE. This reduces the individual errors in the model to a single number without assessing the errors as a population of ...
0
votes
0answers
111 views

How to interpret poor $R^2$ score but good RMSE value?

I split my data into training set and test set and am running linear regression on it. I am using Python's "scikit" library and I am getting an $R^2$ score of 0.31 and an RMSE value of 0.037. The ...
0
votes
2answers
36 views

How to apply a model from one data set to another

I'm still new to SAS, and I need to take a basic OLS regression from one data set, and find the RMSE when applied to a different set. This feels simple, but I haven't found anything. Help would be ...
1
vote
0answers
65 views

Shouldn't the root mean square error (RMSE) be called root mean square residual?

As far as I understand, estimating the error of a model, say an artificial neural network, requires to know the "true" model. Wikipedia says in its article "Errors and residuals": "The error (or ...
0
votes
1answer
38 views

How to request predicted points for each patient based on nomogram in RMS?

I am generating nomogram for a survival analysis project using RMS package of R. A PI of the project would like to have predicted points for each patient in the sample predicted by the nomogram. Is ...
0
votes
1answer
23 views

RMS and probability of steady values

Given an RMS value of a signal/waveform/list of numbers, I would like to construct an equation that calculates the probability that $n$ consecutive points are the same (or close). Suppose I have a ...
0
votes
0answers
142 views

How can I calculate the root mean square error (RMSE) for two covariance matrices?

I want to compare different methods of estimating the covariance matrices on the basis of RMSE and will recommend having the minimum RMSE. I have a sample of, say, 356 weekly observations of 10 ...
0
votes
0answers
107 views

Minimizing RMSE while using a log-transformation

I am trying to fit a linear model to some data, but the dependent variable y is clearly not normally distributed. It has a heavy tail on the right. A log-transformation helps to make the ...
0
votes
1answer
88 views

Is it possible to compare the RMSE of different models with different datasets?

I have 5 datasets and for each one of them I created 2 prediction models. For such task I divided each dataset in training and testing set (70/30%). Then I assessed the RMSE for each one and ...
0
votes
2answers
114 views

How to predict using ordered probit regression and calculate prediction accuracy?

I want to do an ordered probit regression, then cross-validate model prediction accuracy with 80% data for training and 20% for validation, and calculate RMSE for predictions. Consider this dataset: ...
0
votes
0answers
64 views

How to compare forecast accuracy of ordered probit and the equivalent Bayesian heierarchical model in R?

I have a dataset of a metric predictor variable $X$, and an ordered categorical predicted value $Y$ for several individuals. The dataset are from two groups $G_1$ and $G_2$. I want to estimate $Y$ ...
0
votes
0answers
250 views

How to discuss RMSE, MAE, $R^2$ results. What values are good/bad?

I have regression results for a study where I'm trying to develop a predictive model to estimate the value of a continuous output value for a physical process. My database has around 400 records and ...
3
votes
2answers
2k views

RMSE (Root Mean Squared Error) for logistic models

I have a question regarding the validity of using RMSE (Root Mean Squared Error) to compare different logistic models. The response is either 0 or ...
1
vote
0answers
30 views

Standard performance measure for regression?

I will perform time-series prediction and I will report the accuracy of my system with a measure like RMSE or MAE. However, the variables I will predict are in different ranges. So let's say one is ...
0
votes
2answers
178 views

How to change baseline patient in Predict function in rms package in R?

I am doing a time dependent Cox model using cph function in rms package. I use Predict and plot.Predict to plot the hazard ratio on y axis and a continuous covariate (e.g. LDL cholesterol) on X axis ...
1
vote
0answers
82 views

Root-Mean Squared Error for Bayesian Regression Models

I'm trying to get a sense of my prediction errors for a Bayesian regression model and I was using the Root-Mean-Squared Error. My question is, since are predictions are stochastic, would it make ...
0
votes
1answer
69 views

Using {rms} package for multinomial logit

Is it possible to use the rms package to model multinomials logits, or elsewise to model several binary logits to achieve the same effect? I am aware that there are many other packages specifically ...
0
votes
1answer
70 views

Visualizing nonlinear regression

I have following model using mtcars dataset: ...
0
votes
0answers
224 views

KS, RMSE tests in R

I have a set of yearly peak river flows and I am trying to use the KS test to conclude if the data is coming from a generalized extreme value (GEV) distribution. I am also trying to calculate a root ...
2
votes
1answer
67 views

Can there be a situation where one regression model gives lower RMSE than the other but also lower R-squared?

Consider the following scenario where you use the same data X (the same number of predictors p, same number of observations <...
0
votes
1answer
621 views

what is the meaning of RMSE in caret::train [duplicate]

I'm confused by the exercise solutions of the book Applied Predictive Modeling. In https://github.com/topepo/APM_Exercises/blob/master/Ch_06.pdf at the beginning of page 4 ...