RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

learn more… | top users | synonyms (1)

0
votes
0answers
6 views

How to keep a simulation from crashing when one application of the lrm function in rms cannot be fit? [migrated]

I am running a Monte Carlo simulation with 1000 iterations. Within each iteration, I am fitting a weighted logistic regression model using the lrm function from the Harrell's rms package. The model is ...
0
votes
1answer
18 views

MSEP and R2pred for Linear Model

I have two set of data 1-Training (Calibrating) 2-Test. With these datasets, I Fit the model using first dataset. predict using the second dataset x-variables I have to test the closeness of the ...
1
vote
0answers
80 views

Multiple Imputation - Help Needed

These multiple imputation results relate to data I have previously described and shown here - Skewed Distributions for Logistic Regression Three variables I am using have missing data. Their names, ...
0
votes
0answers
23 views

Absolute Loss in R

I have been asked to compared between Robustness of absolute lost regression and its variants compared to least squares. I have done the least squares should I use Lasso now? ...
0
votes
0answers
39 views

Calculation of RMSE for Weibull distribution model

I'm currently studying Weibull distribution and I have questions regarding 'statistical tests' (RMSE for now) for Weibull distribution. First, I want to be clear on one thing here. What does 'actual ...
4
votes
0answers
31 views

What is the “pdm” stat in the “rms” R package?

I am new to the world of Regression in statistics and I have been doing a research in which I am building an ordinal logistic regression model (ORM). In order to fit my ORM model, I am using the 'orm' ...
0
votes
0answers
34 views

Coefficient of determination contradicts the normalized RMSE

I have a model that allows me to estimate the values of a signal and I’m testing it under different environmental conditions. To compare my estimated signal to the real (reference) signal I’m using ...
1
vote
0answers
15 views

How to separate model errors from measurement errors?

I've got data measured with errors that have known measured_RMS. I am testing a model that has model_RMS differences between model prediction and measured data. What would be a reasonable estimate of ...
0
votes
0answers
11 views

Using confidence level for error evaluation

I want to evaluate performances of a forecasting system. I have some samples of the whole population. But what i actually have is the mean value of the sample and the forecast for the mean of the ...
0
votes
0answers
9 views

Accuracy Assessment: Do I still have to use any other test statistics?

I have a training set and a separate test set. In both sets, I have extracted two different parameters and I have compared the predicted values of these parameters to the actual values. So, I have ...
0
votes
0answers
9 views

multimodel inference when using rms package

I would be glad to have some advise about how to proceed with multimodel inference to obtain weighted estimates based on AICc after running ordinal logistic analyzes with "rms' package. I used the ...
0
votes
2answers
197 views

Evaluating predicted vs observed - RMSE vs. Pearson's R interpretation

I'm evaluating the error in three cross-validated models plotting observations against predictions. To do so, I'm comparing the RMSE (root-mean-squared-error) and the Pearson's R between predictions ...
2
votes
2answers
40 views

Is it possible for a predictor to have low correlation but low rmse as well

I have this strange condition. I have two predictors. One of the predictors has low correlation with the target but less rmse. On the other hand another predictor has high correlation but high rmse as ...
2
votes
0answers
36 views

Normalized RMSE

I have several time-series in a VAR(1) and, due to some of them haven't the same unit of measure, I'd like to estimate the RMSE in percentage. I know that it could be done in several ways (see below) ...
1
vote
0answers
34 views

Equivalence in model performance metric like RMSE

I calculated the root mean squared error (RMSE) to compare the simulated values of a hydrologic model with the corresponding observations for three observational datasets.The number of data points in ...
0
votes
0answers
184 views

how to calculate Root Mean Square Error (RMSE) for predicted Probability Density Function (PDF) in Matlab

I have used Mixture Density Networks for probability density function prediction. I am wondering how I can calculate Root Mean Square Error (RMSE) of predicted pdf in MATLAB. Thanks.
1
vote
0answers
11 views

error on truncated rms

I am computing the RMS of a sample to estimate the standar error $\sigma$ of the underlying distribution (for simplicity let say a normal distribution $N[\mu$, $\sigma$]). $ \text{RMS} = \sum_{i=1}^N ...
0
votes
0answers
34 views

Constant RMSE margin between training and teseting set

I have a large number of independent datasets of varying size but same feature meaning. Features and outcome are both binary. I am trying to fit logistic regression to the data. I estimate ...
1
vote
1answer
112 views

Individual and overall RMSE for multivariate data

I have a dataset which contains missing values, and I'm using imputation packages (Rs mi and ...
0
votes
0answers
44 views

RMSE for Model Validation and Calibration

Based on UK's Framework for Marine and Esuarine Model Specification,the % RMSE for current speed is calculated by dividing RMS by maximum peak current. $$\text{RMSE(%)} = ...
0
votes
0answers
21 views

Cross validation to test the performance of two spatial interpolation methods

I have 14 weather stations' temperature data for the period between 2010–2013. I need to evaluate the performance of two spatial interpolation methods. I suggest to select 10 days from this period of ...
0
votes
1answer
30 views

Inconsistency between RMSE and 95% CI Coverage

I am running simulations to compare different weighting methods to estimate the mean of y (with missing values). I use bias, RMSE and 95% CI Coverage as my performance metrics. However, looking at ...
1
vote
0answers
55 views

Average of root mean square error

Is taking the average of different rmse valid? for example average rmse = (rmse1+rmse2+rmse3)/3 Thank you for your help!
0
votes
1answer
124 views

Looking for help with ITSM software (or other comparable software)

I'm looking for someone who is familiar with the ITSM software. I have some data that needs to be fit with an ARIMA/SARIMA model and then forecast using Holt-Winters/Seasonal method. I then need to ...
1
vote
0answers
63 views

Forecast accuracy – can we use correlation and $t$-tests?

Does it make sense to compare actual vs. forecast using correlation analysis / see how close $R^2$ is to 1? Does it make sense to use a paired t-test to test actual vs. forecast to get accuracy of ...
0
votes
1answer
35 views

Confusion related to variance and mse

I was reading this wikipedia article and it states that MSE of a predictor is equivalent to variance of the error. To test it I did something like this ...
0
votes
0answers
58 views

Difference between RMSE and Spearman Correlation

I am trying to evaluate model performance (regression problem). In literature, some use RMSe and others use correlation. Is there any difference between both the approaches? Here: What are good ...
2
votes
2answers
93 views

rms validate on models with a predict function such as coxph and glmnet

I would like to use bootstrapping to evaluate models generated by coxph and glmnet. Would that be somehow possible with rms validate ? From the documentation it seems limited to rms functions (cph, ...
0
votes
1answer
253 views

How to use rfe object with function pickSizeTolerance in R package caret

I run caret's recursive feature selection with randomForest. While running rfe function with method repeatedcv, I had parameter ...
0
votes
0answers
105 views

RMSE for negative binomial hurdle models?

I am working in Program R. I am conducting analysis on zero inflated over dispersed data. I am leaning towards hurdle regression with the count aspect modeled in a negative binomial framework. I am ...
0
votes
1answer
203 views

What are RMSE SD and Rsquared SD metrics in resampling results using R package:caret?

I've been doing predictive modelling with R package caret. When resampling regression models, I get the traditional RMSE and Rsquared metrics, but also RMSE SD and ...
1
vote
0answers
30 views

Calculate RMS for regression line

The RMS for the regression line is: $\sqrt{(1-r^2)}\cdot sd(y)$ where $sd(y)$ is the standard deviation of $y$ -- let's call this eq 1. Another way of calculating it is: ...
1
vote
2answers
265 views

What is the RMSE of k-Fold Cross Validation?

I am testing a neural net to predict numeric values. For that i am using a Training,Validation and Test split. I made a manual 4-Fold CV, this means i am getting 4 RMSE error, each one is the error ...
4
votes
1answer
206 views

Is there a computationally less expensive way to calculate RMS error between two signals?

Root-mean-square error (RMS error) between two signals can be calculated as given: ${\text{RMS}(x_\textrm{ actual}(t)-x_\textrm{ reference}(t))}$ When you want to calculate within a sliding window, ...
5
votes
2answers
1k views

Calibration of Cox regression survival analysis

To perform calibration of a Cox regression model (i.e. assessing for the agreement between the predicted and the observed outcome), what is the best method to present the accuracy of the model in ...
1
vote
0answers
102 views

RMSE, MAD of vectors

I have a set of $N$ high-dimensional vectors. I use some approximation routine to make my output faster. Now I would like to evaluate the error of the approximation. Typically I use the RMSE to ...
1
vote
0answers
190 views

SE of RMSE in R

I have crossvalidated my models and measured RMSE between the modelled values and reality: RMSE <- function(err) sqrt(mean(err^2)) RMSE(predicted - reality) I ...
0
votes
0answers
182 views

Error in validation of a Cox PH model using rms package in R

I am trying to do validation of extended cox model with time-varying covariates in R using rms package. Here is a toy data that looks similar to my data. It has 385 ...
1
vote
1answer
810 views

fastbw with rule=“p” in R's rms package: why do results depend on number of covariates?

I've been trying to use the fastbw function from the rms package in R to perform logistic regression with backward selection, with p-values as exclusion criterion (I am well aware of the arguments ...
2
votes
2answers
2k views

Computing c-index for an external validation of a Cox PH model with R

First off, I'll state that I'm aware many questions get asked about the c-index. I've searched this site and others, and I haven't found an answer for my situation. I can successfully use ...
4
votes
1answer
442 views

Best method to validate a multiply imputed Cox model with R?

This question is with regards to using a test data set to validate an imputed Cox model using R. With a non-imputed data set I would use val.surv() from ...
8
votes
1answer
838 views

Using multiple imputation for Cox proportional hazards, then validating with rms package?

I've been researching the mice package, and I haven't yet discovered a way to use the multiple imputations to make a Cox model, then validate that model with the rms package's ...
2
votes
1answer
159 views

Does good calibration surpass roughly met assumptions and mediocre discrimination?

My question arises from my current task to develop a clinical prediction model using ordinal logistic regression (with rms), but applies to any kind of regression analysis. The proportional odds ...
9
votes
2answers
2k views

Cross validation and ordinal logistic regression

I am trying to understand cross-validation for ordinal logistic regression. The aim of the game is to validate the model used in an analysis... I first construct a toy data set: ...