Questions tagged [rms]

RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Internal validation of BART using bootstrap resampling

I developed a BART model for binary outcome predictions given two numeric independent variables using the pbart R package (v2.9). The code looks like this (sub-setting for 23 observations, complete ...
1
vote
0answers
24 views

BIC and RMSE are Contradict each other for ARIMA Model Selection using R: Do I Err in Theory or in Practice?

I was surprised to see that RMSE and BIC have contradictory trends for the same time-series data. EDITED The procedures in my code are: simulate a 15 AR series of ...
2
votes
2answers
93 views

Root-mean-square error when having multiple prediction horizons

I have a basic question about the root-mean-square error (RMSE). I have a prediction using an ARIMA model. I predicted a time series and use a rolling-horizon approach with overlapping or non-...
0
votes
1answer
50 views

How to calculate the RMSPE when the data contains zeros?

I made some predictions with my model and measured its performance using the RMSE. I cannot compare the errors between predictions because some time series have different scales. Some in the 1000's ...
0
votes
0answers
43 views

Is RMSE a good non-parametric substitute for Pearson's r?

I am examining how well two non-stationary time series, which are supposed to be representing the same thing, match each other. I believe I should, therefore, be specifically looking for a linear ...
3
votes
1answer
55 views

Will a model with a low R² value have higher RMS error rates?

I'm testing different models and my general expectation is that models that have a high coefficient of determination should roughly also have a lower error rate (RMSE in that case) than those with a ...
0
votes
1answer
28 views

What is the effect of PCA on the Error?

I am fitting an ElasticNet model using an array of values for alpha and l1_ratio. I then plot the result of the negative root mean squared error from cross validation in a heatmap, which gives me the ...
0
votes
0answers
21 views

What does it mean for the RMSE to be the same as the mean of the range?

I made some predictions on some time series data. The plots look good, the predictions line up with the original values quite well. But the error values don't make much sense to me. I calculated the ...
0
votes
0answers
10 views

Study on Human Behavior: what is a good value of RMSE when a linear regression is performed on a dataset that represents human behavior? [duplicate]

I am completely new to machine learning. I am working on my undergrad thesis that tries to predict how satisfied a person will be in a specific area by using linear regression. Since this study ...
1
vote
1answer
23 views

Presentation of a 3x3 categorical interaction for Cox proportional hazards regression using RMS package

I wanted to verify my presentation of a 3x3 interaction of two categorical variables var1 (exposure of interest) and var2 (effect modifier of interest) using the rms...
0
votes
0answers
154 views

How big a difference for test/train RMSE is considered as overfit?

I read that when: RMSE of test > RMSE of train => OVER FITTING of the data. RMSE of test < RMSE of train => UNDER FITTING of the data. Is there a actually delta threshold that determine if ...
1
vote
0answers
217 views

How to interpret NRMSE (Normalised Root Mean Square Error) without comparing models?

I have fitted some robust mixed effects linear regression models (using robustlmm::rlmer in R). I have calculated the normalised root mean square error (NRMSE) for these models but I want to make sure ...
1
vote
0answers
25 views

How is possible to obtain similar RMSEs but very different Bias between two predictor models?

I'm comparing the results of two predictors models, CNN convolutional neural network and WRF weather numerical model, against a ground truth (Merra). WRF-Merra is the weather numerical model vs Merra ...
0
votes
0answers
22 views

How to evaluate a regression model with multiple results?

I created a neural network for time series forecasting. My experiments involve comparing the effects the different regularizers have on the model. I used cross-validation and measured my model's ...
0
votes
0answers
58 views

(LSTM with R) How Backtested cross validation RMSE can used for the actual LSTM forecasting model

This blog was super helpful to build LSTM code in R https://www.r-bloggers.com/2018/04/time-series-deep-learning-forecasting-sunspots-with-keras-stateful-lstm-in-r/ So, I tried to use RMSE that is ...
0
votes
0answers
3 views

Recommendation Model Performance Analysis

Assume I have to recommendation models A and B. I used two evaluation metrics RMSE and F-score. RMSE values of A is lower than B. However when I was evaluating the F-score I noticed the F-score value ...
2
votes
1answer
106 views

Aggregating error metrics like RMSE for multiple time series

With metrics like MAE, taking a mean is the last operation in calculating error values. So given a data set of values and forecasts for multiple series for many steps into the future, taking the mean ...
0
votes
0answers
70 views

How to explain RMSE to business folks and in a simple and easy way [duplicate]

If a model predicting 'days to payment etc.' has an RMSE of 25 days and MAE of 20 days. How to explain RMSE of 25 days easily to them, just like the MAE of 20 days (which is average prediction error) ...
0
votes
0answers
13 views

Is Root Mean Square Error (RMSE) affected by Simpsons's paradox?

I wonder if RMSE will be different in intra- vs. interindividual calculation. For example I have multilevel data: well-beeing measured at several time points (level 1) nested in people (level 2). I ...
0
votes
0answers
26 views

What does “RMSE wrt the 1:1 line” mean

I am reading an article where the authors state root-mean-squared error (RMSE) with respect to the 1:1 line But I'm not sure what they mean? RMSE can be calculated as $$ RMSE = \sqrt{ \frac{1}{N}\...
2
votes
1answer
351 views

What is MCRMSE (mean columnwise root mean squared error)?

The MCRMSE evaluation metric was used in the Kaggle Competitions Africa Soil Property Prediction Challenge(6 years ago) and OpenVaccine: COVID-19 mRNA Vaccine Degradation Prediction(On-going) ...
1
vote
1answer
80 views

RMSE to accuracy

I have seen multiple questions and answers about this, but I haven't been able to understand, so, I'm gonna try to ask as simple as possible. I have built several models to forecast future value of a ...
0
votes
1answer
88 views

How to apply the uniform shrinkage factor to the logistic regression to get the updated coefficients and intecept in R?

Hope to ask a bit about uniform shrinkage factor in updating the coefficients and intercept of prediction model: I have built up a prediction model with "rms" and got the uniform (global) ...
0
votes
0answers
63 views

Using RMSE and AIC to compare three separate “final” models (one with double observations)?

I'm looking at three models (linear mixed effect) looking at crime. The first looks at total crime so there are ~96000 observations. In the second model, I look at crime as a function of crime type (...
2
votes
1answer
94 views

What is the difference between RMSE and SEP

I would like to understand the difference between Root Mean Squared Error and the Standard Prediction Error. The SEP formula is simillar to the RMSE, but with an aditional term called bias inside the ...
1
vote
3answers
565 views

Compare RMSE for the same model but varying sample size

My empirical research is based on a variable $a_{i,t} \sim f(\mathrm{RMSE})$, i.e. it is based on the root mean squared error (RMSE) of a certain regression model $Y_{i,t} = f(X_{i,t}, \beta) + \...
1
vote
0answers
32 views

What is the name of this data denoising method

I've been working on extracting data from an extremely noisy signal. The signal itself is the 1st derivative of raw mean squared (RMS) of an audio that may contain segments with some single low ...
0
votes
0answers
23 views

Rms::orm - weighting data?

I am interested in using the rms::orm function in R to model a continuous ordinal outcome in a complex survey design (see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5675816/). Since the data is from ...
0
votes
0answers
7 views

Is it acceptable to use the standard error of the residuals to determine a confidence interval?

I have modeled data using a dynamic linear model and forecast future observations. Across the modeled data the RMSE is just 3.03. However, using standard code copied from relevant literature [see ...
0
votes
0answers
76 views

Can RMSE be greater than standard deviation of noise

I am working on model selection problem for noisy data sets. I am having non-parametric models like SVR, regression splines etc. which have can overfit if the hyperparameters are not tuned properly. I ...
0
votes
1answer
24 views

Which is a better determinant of linear regression performance: the RMSE, R-squared or significance of coefficients?

I have created 2 linear regression models on a data set and its extract (I removed some features from teh first data set for this second model as the number of samples was too small). None of them ...
0
votes
1answer
30 views

How to interpret the RMSE value generated by a regression model?

What exactly does an RMSE(root mean squared error) value imply? How to interpret it with regards to any regression problem?
0
votes
0answers
32 views

What would be a good method to compare the results from my model to actual measurement data?

I have a complex physical model of an engine and I get certain outputs for a given set of inputs. However, these outputs are of course, not exact and deviate from the physically observed values for ...
0
votes
0answers
8 views

error of the regression line - confidence interval of a regression line

I have two measurement methods for the same analyte (CK-MB) One of the methods gives false reactions with macro-CK izoenzyimes, which are rare cases, but with significant pathology. I have no method ...
1
vote
0answers
43 views

huge difference between RMSE and MAE in non-linear regressors

I am building non-linear regression models such as a Random Forest regressor , a KNN regressor or a SVM using a RBF kernel and I decided to use both RMSE and MAE as evaluation metrics. I know that a ...
0
votes
0answers
57 views

Bootstrapping a RMSE

I am trying to compute a bootstrapped distribution of the root mean squared errors to 1 of a distribution "A". However, I am not sure how to do this. My approach was to resample the distribution "A" ...
0
votes
1answer
40 views

How to evaluate multiple time series forcasting model?

Hi I have multiple time series forcasting model and I want to evaluate the predictive power of this model. Let's say, we are predicting $A_T$ and $B_T$ by using $A_t,t\in[0,...,T-1]$ and $B_t,t\in[0,.....
0
votes
0answers
44 views

RMSE normalization. Number of bins

I am using RMSE (Root mean squared error) as a measure of goodness of fit. I am fitting a formula to binned data. The number of bins is not fixed: if there are less than 5 data values in a certain bin,...
0
votes
0answers
34 views

How do I interpret RMSE in layman words? [duplicate]

For example, I am predicting a score that can have value from 0 to 100. The RMSE = 10. How ...
2
votes
1answer
55 views

How does the RMSE work?

I am basing my understanding of the Root Mean Squared Error on this answer. From what I understand it averages the error between the target and the prediction. The root and square parts are for ...
0
votes
1answer
112 views

Positive model bias but negative percent bias; why?

I'm comparing my training data with some predicted values using the Metrics package in R. When running the bias function, R ...
1
vote
0answers
15 views

What is the distribution(root mean square gamma distribution)

What is the distribution when you sample from the gamma distribution and take the root mean square? Please tell me how to prove
1
vote
0answers
18 views

Curve Fitting Metrics: Mean Percent Difference

I recently discovered my colleague (not a mathematician) was evaluating their experimental regression analyses by reporting the mean percent difference of each estimated output (from their fitted ...
1
vote
0answers
157 views

Hypothesis testing for RMSE

I want to compare two distributions of values. While the mean value is approximatively the same in both distributions, the RMSE in both with respect to 1 is quite different. With RMSE I mean $\sqrt{(�...
0
votes
0answers
75 views

Why does the RMSE value goes high? [duplicate]

I have been trying to predict the glucose values of patients by using regression algorithms. I used Support Vector Regression (RMSE: 65), Logistic Regression (RMSE: 86), Linear Regression(RMSE: 64) ...
0
votes
0answers
324 views

Calculation of Vector RMSE

I have 2 sets of 2-dimensional vectors, one from observations and one produced by a model. I would like to calculate a statistic similar to the RMSE for these. I believe the correct way of doing this ...
0
votes
0answers
18 views

Correct application of RMSE and MAE

I have ~ 120 different datasets (different scales, sample size etc) and for each dataset, I predict ONE statistical parameter (doesn't matter what for my question) with different methods. To compare ...
2
votes
1answer
56 views

What does it mean when MAE improved while RMSE worsened?

I am comparing two models: One is a black box that I cannot understand, the other is a GLM. How can I describe why the differences are like this? Is the GLM performing worse than the blackbox?
4
votes
2answers
642 views

What is the difference between an RMSE and RMSLE (logarithmic error)?

RMSE vs RMSLE Root Mean Squared Error (RMSE) and Root Mean Squared Logarithmic Error (RMSLE) both are the techniques to find out the difference between the values predicted by the machine learning ...
0
votes
1answer
68 views

Assessing loss of different parsimony levels (Cox Model)

I have a Cox Proportional Hazard model with 6 covariates to determine OS. I am now trying to simplify this model by taking some of this covariates down. This is intended for a wide audience so I'm ...

1
2 3 4 5