Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rms]

RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

1
vote
0answers
39 views

Why using RMSE as loss function in logistic regression takes non convex form but doesn't in linear regression? [duplicate]

I am taking this deep learning course from Andrew NG. In the 3rd lecture of 2nd week of the first course, he mentions that we can use RMSE for logistic regression as well but it will take a nonconvex ...
0
votes
0answers
9 views

Help with saving stats in rms bootcov [migrated]

I'm trying to save the distribution of R2 values as I bootstrap a model, using the ols and bootcov functions in the rms package. ...
0
votes
0answers
11 views

Interpreting RMSE across a range of values

Imagine I have predicted values from a model. They have a gaussian distribution and range from 1-100. Comparing the predicted to the original variable, I calculated the root-mean-square-error (rmse) =...
0
votes
1answer
26 views

Explain Root Mean Square Error to non-technical audience

My company is in the process of switching equipment from one vendor to another. We measured several metrics from the existing and new equipment and compared the time series. The ideal is to have no ...
1
vote
3answers
61 views

Does the RMSE formula have a $k$ in the denominator?

In what circumstances does the RMSE formula have a $k$ in the denominator? StackOverflow's What does RMS stand for? shows this formula for RMSE: $$RMSE=\sqrt{\frac1{n-k}\sum_i(y_i-\hat{y}_i)^2}$$ ...
0
votes
0answers
21 views

Normalised Root Mean square error

I have $10$ people in a group and they undergone a surgery. I have the root mean square of each subject before and after the surgery. ...
0
votes
1answer
6 views

rmse of differenced vs undifferenced data

if I created a forecast on a differenced data set and compared that to the differenced holdout set, would this rmse be the same as if I reverted back to the undifferenced data and then performed the ...
0
votes
1answer
48 views

Calculating RMSEC and RMSECV of PCA in R

I have been trying to calculate the root mean squares error of calibration (RMSEC) and the root mean squares error of cross validation (RMSECV) for a PCA model made in R using the mdatools package. ...
1
vote
0answers
31 views

Cross-validation of (Cox) survival model with very high censoring rate

I am currently working on survival analysis of data with very high censoring rate (~99%), and the number of events is only about 500, using R. I would like to ask in such case, whether the validate() ...
0
votes
0answers
28 views

aregImpute or mice for imputation of survival data

I would like to use multiple imputation to analyse associations between an exposure variable (exp) and different disease risks in a dataset with some missing data (...
0
votes
0answers
129 views

How can RMSE be compared between a regression model and a neural network model?

In the calculation of RMSE, linear regression uses degrees of freedom(n-p) as divisor and neural network(feed-forward in my case) uses the total data number(does it have degrees of freedom as well?). ...
0
votes
1answer
153 views

Is it possible to find Normalized Root Mean Square Error (NRMSE) of Root Mean Square Error (RMSE) in R?

I have code in R that calculates the RMSE from a Linear Regression model: ...
1
vote
1answer
176 views

calculating overall error in k-fold cross validation

when using k-fold cross validation i thought the overall error was equal to the mean of errors of each fold. the error being anything from MAE and RMSE to NDCG,F-measure, precision and recall. however ...
0
votes
0answers
54 views

caret chooses non-optimal RMSE?

I run a linear regression via caret / glmnet method with "RMSE" as metric. In the final model, caret tells me which values of the tuning parameters alpha and lambda were selected to minimize RMSE. If ...
0
votes
1answer
41 views

How is probability y = j|X calculated from an ordinal logistic regression model?

I have an ordinal logistic regression model fitted with lrm from the rms library in R, and am presenting results as prob y = j|X ...
0
votes
0answers
29 views

How to best evaluate a cross validation of a logistic regression using cbind

I ran a logistic GLMM using cbind for the response: ...
0
votes
1answer
29 views

What statistical test do I need for comparing forecasted data with actual data?

I’m currently completing my dissertation and need to compare forecasted wave height to the actual wave height. However I am unsure what statistical test to use. Thanks, Jess
0
votes
0answers
90 views

Trend in residual plot

I have a regression problem: I have to predict for how long (in days) a house will stay in the market before that someone will buy it. To predict how many days the house stays in the market I am ...
1
vote
0answers
42 views

What is the point of using PRESS instead of RMSECV?

What is the point of using predicted residual sum of squares (PRESS) instead of root-mean-squared-error-of-cross-validation(RMSECV)? In many books, especially in the area of chemometrics, the authors ...
0
votes
1answer
17 views

Why compare measures of dispersion with ratios?

I'm writing a paper on a novel statistical model estimated using MCMC and am currently evaluating it using simulated data. We are comparing the performance of our model to an established model as ...
1
vote
1answer
33 views

Comparing the RMSE produced by two models

Suppose I have two models, each of which have the purpose of estimating a sequence of magnitudes. Suppose further that there are $N$ such magnitudes to be estimated, and that each magnitude is know to ...
0
votes
0answers
12 views

Comparing the RMSE produced by two models [duplicate]

Suppose I have two models, each of which have the purpose of estimating a sequence of magnitudes. Suppose further that there are $N$ such magnitudes to be estimated, and that each magnitude is know to ...
1
vote
1answer
460 views

Interpreting RMSE of log-values

I am modelling a regression with a GBM and evaluate by RMSE. My model input & target is log-transformed which results in an RMSE that is also on log-scale. How can i interpret this in an ...
0
votes
1answer
48 views

What is the best way to test and validate a multivariate regression using OLS?

I am implementing a multivariate regression from scratch using Ordinary Least Squares to get the weights. I noticed that this method does not have any hyperparameters to tweak, so I am not sure what I ...
1
vote
2answers
668 views

Random Forest “out of bag” RMSE

Normally, I used a test set to calculate the RMSE of my RandomForest model. But currently I am using the whole data set in the Random Forest. I want to validate (RMSE) my model with the "out of bag ...
0
votes
2answers
42 views

Performance evaluation

I'd like to test the performance of a penalized regression. I did three separate regressions for each response variable (one numerical, one binomial and one multinomial). I was checking this link, and ...
1
vote
0answers
21 views

Can i calculate the RMSE of two sets of mean values

I measured the tree heights for 13 forest plots in 2018 and want to compare it with the extrapolated reference data from 2013 to see how accurate my method is. I cannot calculate the RMSE of each ...
2
votes
1answer
361 views

How to test if two RMSE are significantly different?

Say I have two models for a regression task and from each model I get a RMSE. One RMSE is smaller than the other, however I wish to test if the difference is statistically significant in order to be ...
1
vote
0answers
17 views

Missing measurements in nonlinear chemical processes

I am using an imputation method to handle missing measurements;TSR.The prediction model used is LW-PLS. Based on the my results, the RMSE increases when the percentage of missing measurements ...
4
votes
1answer
198 views

Ridge regression to minimize RMSE instead of MSE

Cross-posted from my identical question on math.stackexchange: Given a metrix $X$ and a vector $\vec{y}$, ordinary least squares (OLS) regression tries to find $\vec{c}$ such that $\left\| X \vec{c} ...
0
votes
0answers
23 views

Comparing overall and conditional models

Let's say I have two regression models with 5 variables, one that is for all observations, and one that only includes values when the temperature is higher than 75 degrees. As an example I have this: ...
1
vote
1answer
134 views

RMSE significance

I am running linear regression, I have over 80,000 observations and 5 predictor variables. I am trying to pick the best model with RMSE and R-squared and I understand how all that works but... What ...
13
votes
2answers
2k views

Why does minimizing the MAE lead to forecasting the median and not the mean?

From the Forecasting: Principles and Practice textbook by Rob J Hyndman and George Athanasopoulos, specifically the section on accuracy measurement: A forecast method that minimizes the MAE will ...
3
votes
1answer
2k views

Difference Between Rho and Decay Arguments in Keras RMSprop

I am working to tune a RNN for the purposes of predictive analytics on time series data. I am testing different optimizers and am currently working with RMSprop. I have reviewed Hinton's lecture ...
0
votes
1answer
54 views

Quantiles in ols/cph

I need to conduct regression analyses with continuous and categorised data (as the latter is common in my field). So far, I used to do this as follows: ...
1
vote
1answer
1k views

K-fold cross validation results interpretation

My linear model has a 0,08642 RMSE and after I perform 10-fold cross validation I get a 0,091276 RMSE. I have read on similar questions like mine, that RMSE of fit and RMSE of prediction should be ...
3
votes
0answers
25 views

Can I use F test to compare linear and exponential model?

I have a data set consisting of crop biomass measurements ("response" variable) and corresponding spectral vegetation indices measurements (predictor variables). The measurements have been made on two ...
1
vote
1answer
864 views

Normalized Root Mean Square (NRMS) vs Root Mean Square (RMS)?

I am trying to find the best-fit model from my observation and model predicated data. I came across these two different approach which have been used in the literature: Normalized Root Mean Square and ...
0
votes
1answer
1k views

RMSE range value [closed]

Do RMSE value has range? For ex: If I have RMSE value 2,25 so what does it means? I did study case for An automatic Correcting essay system. I used RMSE to calculate accurate system value.
1
vote
0answers
269 views

Cross-validation for RMSE

The formula I have for cross-validation error is the following: $$ \hat{R}_{CV}(f) = \frac{1}{N}\sum_{i=1}^N L\left( y_i, \hat{f}^{-k(i)}(x_i) \right) $$ where $\hat{f}^{-k(i)}$ is the model trained ...
0
votes
1answer
294 views

How to plot the calibration curve for an ordinal logistic regression model applied to a test sample?

I'm doing a validation study of an ordinal logistic regression model that was made with the lrm function of the rms package in R. How can I plot the calibration curve for the model when applied to new ...
0
votes
1answer
138 views

Root-mean Square Error reference

Does anyone know the first author (a reference) using RMSE as a criterion to evaluate the performance of a model?
2
votes
0answers
65 views

Unable to fit model using “orm.fit” from package “rms”

I want to compare species richness from different sample types collected from different stations. Below is a subset of my data: ...
5
votes
3answers
148 views

Why does the rank order of models differ for R squared and RMSE?

I am comparing $R^2$ and RMSE of different models. Interestingly, the rank ordering of the models with respect to $-R^2$ and RMSE is different and I do not understand why. Here is an example in R: <...
1
vote
0answers
59 views

Is Spearman's rank and RMS error an appropriate measure of similarity between two signals?

I am working on a project comparing the accuracy of two imaging techniques to measure displacement. I have attached a graph comparing the displacement measured by both techniques over time. I am ...
1
vote
0answers
139 views

Why is the penalty in the logistic regression likelihood ratio test different from the penalty I specified when fitting the model?

I'm fitting a penalized logistic regression model using the rms package in R. When I print the result, the penalty in the model likelihood ratio test is different from the penalty I used to fit the ...
1
vote
0answers
259 views

Measuring accuracy for forecasting in R [closed]

I have a dataset of restaurant orders. In that data set I need to predict the outcome of the next 12 months, i.e. how much order will be given. using the following test ...
0
votes
0answers
24 views

2nd degree polynome instead of restricted cubic splines

I am looking for an alternative to restricted cubic splines, which can provide a numerical result that is easier to interpret and compare (as far as I understand, this is not easily possible with ...
0
votes
1answer
132 views

same cross-validation set for parameter tuning and RMSE calculations

I miss some very basic distinction between cross-validations used for parameter tuning and cross-validation used for calculating the performance of my algorithms (RMSE). I have two functions: one ...
0
votes
1answer
74 views

How to determine whether difference in RMSE is meaningful

I often run into the situation where I have several regression models, each of which gives a RMSECV, and I need to choose which one is "best". Of course, I can choose the one with the minimum RMSECV, ...