RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

learn more… | top users | synonyms (1)

0
votes
0answers
21 views

Constant RMSE margin between training and teseting set

I have a large number of independent datasets of varying size but same feature meaning. Features and outcome are both binary. I am trying to fit logistic regression to the data. I estimate ...
1
vote
1answer
60 views

Individual and overall RMSE for multivariate data

I have a dataset which contains missing values, and I'm using imputation packages (Rs mi and ...
0
votes
0answers
13 views

RMSE for Model Validation and Calibration

Based on UK's Framework for Marine and Esuarine Model Specification,the % RMSE for current speed is calculated by dividing RMS by maximum peak current. $$\text{RMSE(%)} = ...
0
votes
0answers
13 views

Cross validation to test the performance of two spatial interpolation methods

I have 14 weather stations' temperature data for the period between 2010–2013. I need to evaluate the performance of two spatial interpolation methods. I suggest to select 10 days from this period of ...
0
votes
1answer
24 views

Inconsistency between RMSE and 95% CI Coverage

I am running simulations to compare different weighting methods to estimate the mean of y (with missing values). I use bias, RMSE and 95% CI Coverage as my performance metrics. However, looking at ...
1
vote
0answers
30 views

Average of root mean square error

Is taking the average of different rmse valid? for example average rmse = (rmse1+rmse2+rmse3)/3 Thank you for your help!
0
votes
1answer
38 views

Looking for help with ITSM software (or other comparable software)

I'm looking for someone who is familiar with the ITSM software. I have some data that needs to be fit with an ARIMA/SARIMA model and then forecast using Holt-Winters/Seasonal method. I then need to ...
1
vote
0answers
30 views

Forecast accuracy – can we use correlation and $t$-tests?

Does it make sense to compare actual vs. forecast using correlation analysis / see how close $R^2$ is to 1? Does it make sense to use a paired t-test to test actual vs. forecast to get accuracy of ...
0
votes
1answer
27 views

Confusion related to variance and mse

I was reading this wikipedia article and it states that MSE of a predictor is equivalent to variance of the error. To test it I did something like this ...
0
votes
0answers
29 views

Difference between RMSE and Spearman Correlation

I am trying to evaluate model performance (regression problem). In literature, some use RMSe and others use correlation. Is there any difference between both the approaches? Here: What are good ...
1
vote
0answers
39 views

rms validate on models with a predict function such as coxph and glmnet

I would like to use bootstrapping to evaluate models generated by coxph and glmnet. Would that be somehow possible with rms validate ? From the documentation it seems limited to rms functions (cph, ...
0
votes
1answer
85 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
0answers
41 views

RMSE for negative binomial hurdle models?

I am working in Program R. I am conducting analysis on zero inflated over dispersed data. I am leaning towards hurdle regression with the count aspect modeled in a negative binomial framework. I am ...
0
votes
1answer
75 views

What are RMSE SD and Rsquared SD metrics in resampling results using R package:caret?

I've been doing predictive modelling with R package caret. When resampling regression models, I get the traditional RMSE and Rsquared metrics, but also RMSE SD and ...
1
vote
0answers
29 views

Calculate RMS for regression line

The RMS for the regression line is: $\sqrt{(1-r^2)}\cdot sd(y)$ where $sd(y)$ is the standard deviation of $y$ -- let's call this eq 1. Another way of calculating it is: ...
1
vote
2answers
130 views

What is the RMSE of k-Fold Cross Validation?

I am testing a neural net to predict numeric values. For that i am using a Training,Validation and Test split. I made a manual 4-Fold CV, this means i am getting 4 RMSE error, each one is the error ...
0
votes
0answers
37 views

Comparison of two different RMSEs

I performed interpolation on two elevation datasets. Is it possible to compare the two RMSE beyond just which one is larger and which one is smaller, to see if they are statistically different? ...
0
votes
0answers
169 views

Interpretation of regression data, RMSE, and model predictions

I am doing an analysis where I am using one data set of 12 rows (Mold), and running a linear regression analysis on this data set to generate two different linear regression equations. From there I ...
4
votes
1answer
122 views

Is there a computationally less expensive way to calculate RMS error between two signals?

Root-mean-square error (RMS error) between two signals can be calculated as given: ${\text{RMS}(x_\textrm{ actual}(t)-x_\textrm{ reference}(t))}$ When you want to calculate within a sliding window, ...
4
votes
2answers
658 views

Calibration of Cox regression survival analysis

To perform calibration of a Cox regression model (i.e. assessing for the agreement between the predicted and the observed outcome), what is the best method to present the accuracy of the model in ...
1
vote
0answers
82 views

RMSE, MAD of vectors

I have a set of $N$ high-dimensional vectors. I use some approximation routine to make my output faster. Now I would like to evaluate the error of the approximation. Typically I use the RMSE to ...
1
vote
0answers
134 views

SE of RMSE in R

I have crossvalidated my models and measured RMSE between the modelled values and reality: RMSE <- function(err) sqrt(mean(err^2)) RMSE(predicted - reality) I ...
0
votes
0answers
131 views

Error in validation of a Cox PH model using rms package in R

I am trying to do validation of extended cox model with time-varying covariates in R using rms package. Here is a toy data that looks similar to my data. It has 385 ...
0
votes
0answers
75 views

RMSE when parameter is observed with uncertainty

Here is my situation: I want to know the value of some parameter $\theta$ and I have a bunch of different estimates of it. Because each estimate is drawn from a different sample, I observe the ...
0
votes
1answer
534 views

fastbw with rule=“p” in R's rms package: why do results depend on number of covariates?

I've been trying to use the fastbw function from the rms package in R to perform logistic regression with backward selection, with p-values as exclusion criterion (I am well aware of the arguments ...
2
votes
2answers
1k views

Computing c-index for an external validation of a Cox PH model with R

First off, I'll state that I'm aware many questions get asked about the c-index. I've searched this site and others, and I haven't found an answer for my situation. I can successfully use ...
4
votes
1answer
348 views

Best method to validate a multiply imputed Cox model with R?

This question is with regards to using a test data set to validate an imputed Cox model using R. With a non-imputed data set I would use val.surv() from ...
8
votes
1answer
603 views

Using multiple imputation for Cox proportional hazards, then validating with rms package?

I've been researching the mice package, and I haven't yet discovered a way to use the multiple imputations to make a Cox model, then validate that model with the rms package's ...
2
votes
1answer
152 views

Does good calibration surpass roughly met assumptions and mediocre discrimination?

My question arises from my current task to develop a clinical prediction model using ordinal logistic regression (with rms), but applies to any kind of regression analysis. The proportional odds ...
9
votes
2answers
1k views

Cross validation and ordinal logistic regression

I am trying to understand cross-validation for ordinal logistic regression. The aim of the game is to validate the model used in an analysis... I first construct a toy data set: ...