RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

learn more… | top users | synonyms (1)

4
votes
0answers
22 views

What is the “pdm” stat in the “rms” R package?

I am new to the world of Regression in statistics and I have been doing a research in which I am building an ordinal logistic regression model (ORM). In order to fit my ORM model, I am using the 'orm' ...
0
votes
0answers
26 views

Coefficient of determination contradicts the normalized RMSE

I have a model that allows me to estimate the values of a signal and I’m testing it under different environmental conditions. To compare my estimated signal to the real (reference) signal I’m using ...
1
vote
0answers
13 views

How to separate model errors from measurement errors?

I've got data measured with errors that have known measured_RMS. I am testing a model that has model_RMS differences between model prediction and measured data. What would be a reasonable estimate of ...
0
votes
0answers
8 views

Using confidence level for error evaluation

I want to evaluate performances of a forecasting system. I have some samples of the whole population. But what i actually have is the mean value of the sample and the forecast for the mean of the ...
0
votes
0answers
8 views

Accuracy Assessment: Do I still have to use any other test statistics?

I have a training set and a separate test set. In both sets, I have extracted two different parameters and I have compared the predicted values of these parameters to the actual values. So, I have ...
0
votes
0answers
6 views

multimodel inference when using rms package

I would be glad to have some advise about how to proceed with multimodel inference to obtain weighted estimates based on AICc after running ordinal logistic analyzes with "rms' package. I used the ...
0
votes
1answer
62 views

Evaluating predicted vs observed - RMSE vs. Pearson's R interpretation

I'm evaluating the error in three cross-validated models plotting observations against predictions. To do so, I'm comparing the RMSE (root-mean-squared-error) and the Pearson's R between predictions ...
2
votes
2answers
34 views

Is it possible for a predictor to have low correlation but low rmse as well

I have this strange condition. I have two predictors. One of the predictors has low correlation with the target but less rmse. On the other hand another predictor has high correlation but high rmse as ...
2
votes
0answers
24 views

Normalized RMSE

I have several time-series in a VAR(1) and, due to some of them haven't the same unit of measure, I'd like to estimate the RMSE in percentage. I know that it could be done in several ways (see below) ...
1
vote
0answers
31 views

Equivalence in model performance metric like RMSE

I calculated the root mean squared error (RMSE) to compare the simulated values of a hydrologic model with the corresponding observations for three observational datasets.The number of data points in ...
0
votes
0answers
153 views

how to calculate Root Mean Square Error (RMSE) for predicted Probability Density Function (PDF) in Matlab

I have used Mixture Density Networks for probability density function prediction. I am wondering how I can calculate Root Mean Square Error (RMSE) of predicted pdf in MATLAB. Thanks.
1
vote
0answers
11 views

error on truncated rms

I am computing the RMS of a sample to estimate the standar error $\sigma$ of the underlying distribution (for simplicity let say a normal distribution $N[\mu$, $\sigma$]). $ \text{RMS} = \sum_{i=1}^N ...
0
votes
0answers
28 views

Constant RMSE margin between training and teseting set

I have a large number of independent datasets of varying size but same feature meaning. Features and outcome are both binary. I am trying to fit logistic regression to the data. I estimate ...
1
vote
1answer
92 views

Individual and overall RMSE for multivariate data

I have a dataset which contains missing values, and I'm using imputation packages (Rs mi and ...
0
votes
0answers
37 views

RMSE for Model Validation and Calibration

Based on UK's Framework for Marine and Esuarine Model Specification,the % RMSE for current speed is calculated by dividing RMS by maximum peak current. $$\text{RMSE(%)} = ...
0
votes
0answers
19 views

Cross validation to test the performance of two spatial interpolation methods

I have 14 weather stations' temperature data for the period between 2010–2013. I need to evaluate the performance of two spatial interpolation methods. I suggest to select 10 days from this period of ...
0
votes
1answer
29 views

Inconsistency between RMSE and 95% CI Coverage

I am running simulations to compare different weighting methods to estimate the mean of y (with missing values). I use bias, RMSE and 95% CI Coverage as my performance metrics. However, looking at ...
1
vote
0answers
47 views

Average of root mean square error

Is taking the average of different rmse valid? for example average rmse = (rmse1+rmse2+rmse3)/3 Thank you for your help!
0
votes
1answer
81 views

Looking for help with ITSM software (or other comparable software)

I'm looking for someone who is familiar with the ITSM software. I have some data that needs to be fit with an ARIMA/SARIMA model and then forecast using Holt-Winters/Seasonal method. I then need to ...
1
vote
0answers
55 views

Forecast accuracy – can we use correlation and $t$-tests?

Does it make sense to compare actual vs. forecast using correlation analysis / see how close $R^2$ is to 1? Does it make sense to use a paired t-test to test actual vs. forecast to get accuracy of ...
0
votes
1answer
34 views

Confusion related to variance and mse

I was reading this wikipedia article and it states that MSE of a predictor is equivalent to variance of the error. To test it I did something like this ...
0
votes
0answers
46 views

Difference between RMSE and Spearman Correlation

I am trying to evaluate model performance (regression problem). In literature, some use RMSe and others use correlation. Is there any difference between both the approaches? Here: What are good ...
1
vote
1answer
67 views

rms validate on models with a predict function such as coxph and glmnet

I would like to use bootstrapping to evaluate models generated by coxph and glmnet. Would that be somehow possible with rms validate ? From the documentation it seems limited to rms functions (cph, ...
0
votes
1answer
188 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
0answers
83 views

RMSE for negative binomial hurdle models?

I am working in Program R. I am conducting analysis on zero inflated over dispersed data. I am leaning towards hurdle regression with the count aspect modeled in a negative binomial framework. I am ...
0
votes
1answer
156 views

What are RMSE SD and Rsquared SD metrics in resampling results using R package:caret?

I've been doing predictive modelling with R package caret. When resampling regression models, I get the traditional RMSE and Rsquared metrics, but also RMSE SD and ...
1
vote
0answers
30 views

Calculate RMS for regression line

The RMS for the regression line is: $\sqrt{(1-r^2)}\cdot sd(y)$ where $sd(y)$ is the standard deviation of $y$ -- let's call this eq 1. Another way of calculating it is: ...
1
vote
2answers
200 views

What is the RMSE of k-Fold Cross Validation?

I am testing a neural net to predict numeric values. For that i am using a Training,Validation and Test split. I made a manual 4-Fold CV, this means i am getting 4 RMSE error, each one is the error ...
0
votes
0answers
48 views

Comparison of two different RMSEs

I performed interpolation on two elevation datasets. Is it possible to compare the two RMSE values beyond just which one is larger and which one is smaller, to see if they are statistically different? ...
0
votes
0answers
204 views

Interpretation of regression data, RMSE, and model predictions

I am doing an analysis where I am using one data set of 12 rows (Mold), and running a linear regression analysis on this data set to generate two different linear regression equations. From there I ...
4
votes
1answer
180 views

Is there a computationally less expensive way to calculate RMS error between two signals?

Root-mean-square error (RMS error) between two signals can be calculated as given: ${\text{RMS}(x_\textrm{ actual}(t)-x_\textrm{ reference}(t))}$ When you want to calculate within a sliding window, ...
5
votes
2answers
1k views

Calibration of Cox regression survival analysis

To perform calibration of a Cox regression model (i.e. assessing for the agreement between the predicted and the observed outcome), what is the best method to present the accuracy of the model in ...
1
vote
0answers
97 views

RMSE, MAD of vectors

I have a set of $N$ high-dimensional vectors. I use some approximation routine to make my output faster. Now I would like to evaluate the error of the approximation. Typically I use the RMSE to ...
1
vote
0answers
175 views

SE of RMSE in R

I have crossvalidated my models and measured RMSE between the modelled values and reality: RMSE <- function(err) sqrt(mean(err^2)) RMSE(predicted - reality) I ...
0
votes
0answers
160 views

Error in validation of a Cox PH model using rms package in R

I am trying to do validation of extended cox model with time-varying covariates in R using rms package. Here is a toy data that looks similar to my data. It has 385 ...
0
votes
1answer
724 views

fastbw with rule=“p” in R's rms package: why do results depend on number of covariates?

I've been trying to use the fastbw function from the rms package in R to perform logistic regression with backward selection, with p-values as exclusion criterion (I am well aware of the arguments ...
2
votes
2answers
1k views

Computing c-index for an external validation of a Cox PH model with R

First off, I'll state that I'm aware many questions get asked about the c-index. I've searched this site and others, and I haven't found an answer for my situation. I can successfully use ...
4
votes
1answer
409 views

Best method to validate a multiply imputed Cox model with R?

This question is with regards to using a test data set to validate an imputed Cox model using R. With a non-imputed data set I would use val.surv() from ...
8
votes
1answer
751 views

Using multiple imputation for Cox proportional hazards, then validating with rms package?

I've been researching the mice package, and I haven't yet discovered a way to use the multiple imputations to make a Cox model, then validate that model with the rms package's ...
2
votes
1answer
159 views

Does good calibration surpass roughly met assumptions and mediocre discrimination?

My question arises from my current task to develop a clinical prediction model using ordinal logistic regression (with rms), but applies to any kind of regression analysis. The proportional odds ...
9
votes
2answers
1k views

Cross validation and ordinal logistic regression

I am trying to understand cross-validation for ordinal logistic regression. The aim of the game is to validate the model used in an analysis... I first construct a toy data set: ...