We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [rms]

RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

Filter by
Sorted by
Tagged with
0
votes
1answer
30 views

Assessing loss of different parsimony levels (Cox Model)

I have a Cox Proportional Hazard model with 6 covariates to determine OS. I am now trying to simplify this model by taking some of this covariates down. This is intended for a wide audience so I'm ...
0
votes
0answers
12 views

The network converges to standard deviation of my output and would not further improve [duplicate]

I am training a fully convolutional neural network. My network predicts close values to mean value of the target output and the final RMSE is very close to standard deviation of my output. If I go on ...
1
vote
0answers
17 views

Higher accuracies for larger k at cross validations?

I am fitting an artificial neural network with Python's scitkit-learn. The data source is experimental data from my study. Objective is identifying an optimal ...
1
vote
0answers
44 views

when should I normalize with $\log(1+x)$ instead of with $\log$?

I've seen people log-normalize data by using the $\log(1+x)$ (np.log1p) method for instance normalizing the price of diamonds in the diamonds dataset using ...
0
votes
2answers
42 views

AUC or $R^2$/RMSE for binary classification

I am using doing a binary classification to classify things 0 or 1 using a set of features with LightGBM and XGBoost. Both models give AUC scores roughly in the <...
0
votes
0answers
14 views

Comparing differences in RMSE

I tried to search on the web as well as here. Seems like no answer. There was a question previously asked in stack exchange, but it was discussed instead of being answered. I have RMSE from 6 groups. ...
0
votes
0answers
26 views

What does the root mean square difference tell me that the mean difference doesn't?

If I have the below data and my difference measure is "actual/extrapolated-1" for the same observation. What does the RMSD tell me that the mean difference doesn't? If I understand the 'mean ...
0
votes
1answer
40 views

correlation coefficient in pandas (pearson) [duplicate]

I have divided my data into training and testing, and I am outputting the error metrics on the testing data. This is what I get: ...
1
vote
1answer
45 views

$R^2$ of 1 but RMSE > 0

I am running k-fold cross validation on my training data, and then choosing the best set of hyper parameters, re-training on the training data and testing on a new (unseen) testing data. I am getting ...
0
votes
0answers
11 views

Training of SVM with polynomial kernel with RMSE superior to 1

I am training a SVM with polynomial kernel to do mineral potential modelling. I am using the caret library in R. I created a search grid for the three parameters for the polynomial kernel and used a ...
2
votes
0answers
74 views

Custom RMSE loss not the same as taking the root of built-in Keras MSE loss [closed]

I have defined a custom RMSE loss function: def rmse(y_pred, y_true): return K.sqrt(K.mean(K.square(y_pred - y_true))) I was evaluating it against the mean ...
2
votes
0answers
41 views

'pspline ' or 'rms' in a Cox model?

I am quite new in the spline subject and I have a question! I am using a Cox model and I was afraid that some of the variables included in the model have a non-linear effect on survival. So I tested ...
1
vote
2answers
98 views

outer folds errors in nested cross-validation

I have a time series data that I wish to be able to obtain the general performance of it. For that, I use nested cross-validation with time series flavor as described in this amazing blog. As you ...
1
vote
0answers
30 views

Is it correct to make a conclusion as to whether a model is best for weekly or daily forecasting by comparing the root mean square errors?

I am performing daily and weekly forecasts for 28 days and 4 weeks respectively. Once I have used the same model to obtain the respective forecasts an root mean square errors (RMSE), I will like to ...
1
vote
1answer
32 views

RMSE with and without standardizing the output variable

I have a time series data that I would like to be able to forecast. I was trying to standardize the data as my columns are all of different ranges. I have standardized the input variables, but was ...
1
vote
0answers
36 views

Why Massive Random Spikes of Validation Loss?

My problem is to estimate the length of a straight line in an image, in pixel. My training size is 6000 images, validation is 1000 images. Each image has 200 x 200 pixels. My data is generated using ...
0
votes
1answer
22 views

rmse value meaning [duplicate]

rmse = sqrt(mean_squared_error(y_val,y_pre)) print('Val RMSE: %.3f' % rmse) Val RMSE: 1.825 I got this value for RMSE value but not sure what this number ...
9
votes
2answers
1k views

Can RMSE and MAE have the same value?

I am implementing cross validation and calculating error metrics such as RMSE, $R^2$, MAE, MSE, etc. Can RMSE and MAE have the same value?
0
votes
1answer
56 views

RSME, MAE and prediction interval [closed]

Could someone please clarify, whether it is appropriate to define a prediction interval or an equivalent for an RMSE and MAE measure. If so, could you please suggest how such an interval is defined.
1
vote
0answers
22 views

Straightforward explanation of Theil inequality statistic for curve fitting

I’ve come across a paper (here) that uses a “Thiel inequality statistic” (pages 9 and 10 of the supplementary information) to determine goodness of fit for a model with its training data. I was ...
1
vote
1answer
30 views

Parameter not estimating due to singular information matrix and mutually exclusive categories in R

I have some data that has two categorical variables that are somewhat correlated (there is a row and a column of zeros where the levels are mutually exclusive), similar to the tabulation below. ...
0
votes
2answers
95 views

“Percentage” alternatives to MAPE

I'm aware of the problems of MAPE as a measurement, and particularly it's uselessness in the event of a time series where 0 is one of the many values of y. The downside to ditching MAPE in favour of ...
0
votes
1answer
20 views

How to quantify the fluctuation of an error?

First of all sorry for the bad title, but unfortunately, I can't think of a better one at the moment. Hopefully, that will change when my question is answered. Let's say I have two sets of values: ...
0
votes
1answer
26 views

Which metric to use to report regression results: RMSE, R2 or Pearsons Corrrelation?

I'm a bit confused about when to use RMSE, R2 or Pearsons Correlation Coefficient (Rp). I've read some papers that reported RMSE and Rp and didn't even mention R2, but I also found papers reporting ...
2
votes
1answer
128 views

What is the correct definition of the root mean square percentage error (RMSPE)?

Göçken et al. define the root mean square percentage error (RMSPE) as \begin{equation} \text{RMSPE} = \sqrt{\frac{100\%}{n} \cdot \sum_{i=1}^n \Delta X^2_{\text{rel},i}} \end{equation} with \begin{...
1
vote
0answers
15 views

Is there something like a Root Mean Square Percentage Error (RMSPE)? Or: What is the name of this error? [duplicate]

\begin{equation} \text{RMSPE} = \sqrt{\frac{1}{n} \cdot \sum_{i=1}^n \Delta X^2_{\text{rel},i}}\cdot 100\% \end{equation} with \begin{equation} \Delta X_{\text{rel},i}=\frac{X_i}{T_i}-1, \end{...
0
votes
1answer
150 views

Is there something like a Root Mean Square Relative Error (RMSRE)? Or: What is the name of this error?

\begin{equation} \text{RMSRE} = \sqrt{\frac{1}{n}\cdot\sum_{i=1}^n \Delta X^2_{\text{rel},i}} \end{equation} with \begin{equation} \Delta X_{\text{rel},i} = \frac{X_i}{T_i}-1, \end{equation} where $...
1
vote
0answers
153 views

Why using RMSE as loss function in logistic regression takes non convex form but doesn't in linear regression? [duplicate]

I am taking this deep learning course from Andrew NG. In the 3rd lecture of 2nd week of the first course, he mentions that we can use RMSE for logistic regression as well but it will take a nonconvex ...
0
votes
1answer
82 views

Explain Root Mean Square Error to non-technical audience

My company is in the process of switching equipment from one vendor to another. We measured several metrics from the existing and new equipment and compared the time series. The ideal is to have no ...
1
vote
3answers
74 views

Does the RMSE formula have a $k$ in the denominator?

In what circumstances does the RMSE formula have a $k$ in the denominator? StackOverflow's What does RMS stand for? shows this formula for RMSE: $$RMSE=\sqrt{\frac1{n-k}\sum_i(y_i-\hat{y}_i)^2}$$ ...
0
votes
0answers
48 views

Normalised Root Mean square error

I have $10$ people in a group and they undergone a surgery. I have the root mean square of each subject before and after the surgery. ...
0
votes
1answer
7 views

rmse of differenced vs undifferenced data

if I created a forecast on a differenced data set and compared that to the differenced holdout set, would this rmse be the same as if I reverted back to the undifferenced data and then performed the ...
1
vote
1answer
120 views

Calculating RMSEC and RMSECV of PCA in R

I have been trying to calculate the root mean squares error of calibration (RMSEC) and the root mean squares error of cross validation (RMSECV) for a PCA model made in R using the mdatools package. ...
1
vote
0answers
50 views

Cross-validation of (Cox) survival model with very high censoring rate

I am currently working on survival analysis of data with very high censoring rate (~99%), and the number of events is only about 500, using R. I would like to ask in such case, whether the validate() ...
0
votes
0answers
68 views

aregImpute or mice for imputation of survival data

I would like to use multiple imputation to analyse associations between an exposure variable (exp) and different disease risks in a dataset with some missing data (...
0
votes
0answers
244 views

How can RMSE be compared between a regression model and a neural network model?

In the calculation of RMSE, linear regression uses degrees of freedom(n-p) as divisor and neural network(feed-forward in my case) uses the total data number(does it have degrees of freedom as well?). ...
1
vote
1answer
313 views

Is it possible to find Normalized Root Mean Square Error (NRMSE) of Root Mean Square Error (RMSE) in R?

I have code in R that calculates the RMSE from a Linear Regression model: ...
1
vote
1answer
374 views

calculating overall error in k-fold cross validation

when using k-fold cross validation i thought the overall error was equal to the mean of errors of each fold. the error being anything from MAE and RMSE to NDCG,F-measure, precision and recall. however ...
0
votes
0answers
95 views

caret chooses non-optimal RMSE?

I run a linear regression via caret / glmnet method with "RMSE" as metric. In the final model, caret tells me which values of the tuning parameters alpha and lambda were selected to minimize RMSE. If ...
0
votes
1answer
57 views

How is probability y = j|X calculated from an ordinal logistic regression model?

I have an ordinal logistic regression model fitted with lrm from the rms library in R, and am presenting results as prob y = j|X ...
0
votes
0answers
32 views

How to best evaluate a cross validation of a logistic regression using cbind

I ran a logistic GLMM using cbind for the response: ...
0
votes
1answer
33 views

What statistical test do I need for comparing forecasted data with actual data?

I’m currently completing my dissertation and need to compare forecasted wave height to the actual wave height. However I am unsure what statistical test to use. Thanks, Jess
0
votes
0answers
219 views

Trend in residual plot

I have a regression problem: I have to predict for how long (in days) a house will stay in the market before that someone will buy it. To predict how many days the house stays in the market I am ...
1
vote
0answers
55 views

What is the point of using PRESS instead of RMSECV?

What is the point of using predicted residual sum of squares (PRESS) instead of root-mean-squared-error-of-cross-validation(RMSECV)? In many books, especially in the area of chemometrics, the authors ...
0
votes
1answer
33 views

Why compare measures of dispersion with ratios?

I'm writing a paper on a novel statistical model estimated using MCMC and am currently evaluating it using simulated data. We are comparing the performance of our model to an established model as ...
1
vote
1answer
52 views

Comparing the RMSE produced by two models

Suppose I have two models, each of which have the purpose of estimating a sequence of magnitudes. Suppose further that there are $N$ such magnitudes to be estimated, and that each magnitude is know to ...
2
votes
2answers
801 views

Interpreting RMSE of log-values

I am modelling a regression with a GBM and evaluate by RMSE. My model input & target is log-transformed which results in an RMSE that is also on log-scale. How can i interpret this in an ...
0
votes
1answer
49 views

What is the best way to test and validate a multivariate regression using OLS?

I am implementing a multivariate regression from scratch using Ordinary Least Squares to get the weights. I noticed that this method does not have any hyperparameters to tweak, so I am not sure what I ...
1
vote
2answers
2k views

Random Forest “out of bag” RMSE

Normally, I used a test set to calculate the RMSE of my RandomForest model. But currently I am using the whole data set in the Random Forest. I want to validate (RMSE) my model with the "out of bag ...
0
votes
2answers
44 views

Performance evaluation

I'd like to test the performance of a penalized regression. I did three separate regressions for each response variable (one numerical, one binomial and one multinomial). I was checking this link, and ...