Questions tagged [rms]

RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

Filter by
Sorted by
Tagged with
1 vote
1 answer
41 views

Confirming cubic spline was done on imputed datasets (imputed by mice Package) and the estimate is the pooled based on Rubin's rule

I am performing restricted cubic spline (Cox proportional hazard ratio) after imputing 10 datasets using mice package. My variables as follow: Outcome: DM Exposure: BMI time to events: time Covariates:...
user avatar
  • 25
0 votes
0 answers
12 views

Comparing RMSE/MSE of original data and log10-values of the same data

I want to see if my models work better on the original data or on log10 transformed data. But how can I do this? Normally I train the model, calculate the RMSE/MSE on the test data and compare these ...
user avatar
  • 21
1 vote
1 answer
23 views

Showing RMSE and MAE results as percentage error

I have the results of RMSE and MAE from different spatial interpolation methods as a monthly averages (See the figure below). As ...
user avatar
  • 11
0 votes
0 answers
16 views

How to implement Rubin's Rules to assess model fit on imputed test data with continuous outcome? (e.g. RMSE and 95% CI)

I'm working on a project now which involves the use of multiple imputation while developing machine learning models (using a training/test split, ~7000 observations total) for a continuous outcome. I ...
user avatar
  • 5
3 votes
2 answers
103 views

Does it make any statistical sense to subtract / add RMSE from the predicted results to make the model more accurate?

If RMSE represents the standard deviation of the residuals (prediction errors), while residuals are a measure of how far from the regression line data points are, does it make any sense to subtract / ...
user avatar
  • 133
0 votes
0 answers
25 views

Can we compare whether different groupings of data improve model accuracy?

I have data from 100 different lab incubations of manure samples. For each sample, a 3-day incubation was done, measuring values (y-axis) against time (x-axis). I want to perform a linear regression ...
user avatar
0 votes
0 answers
16 views

RMS and PSD of Sinusoid + Noise

This is a basic question on computing the statistics of a combined signal: the sum a stochastic signal (eg noise), and a deterministic signal (eg sinusoid). How to use the RMS equation, to find ...
user avatar
  • 1
1 vote
1 answer
52 views

What is the purpose of the model AIC if you can iteratively check the forecast RMSE at each lag?

If the purpose of the time series modelling is to build one that gives the most accurate forecast, may I ask if it necessary to check the model AIC to determine the optimal lag when you can ...
user avatar
0 votes
1 answer
42 views

Root Mean Square value, but with reverse order of operation

I've got a question about name of some specific value. Root Mean Square value is defined as such: $$ RMS = \sqrt{\frac{1}{N}\sum_{i}^{N}{x_i^2}} $$ I came across a value, that is similar, but is ...
user avatar
0 votes
0 answers
14 views

RMSE on Training set - Linear regression

I need to get the Mean squared error for training set using Linear Regression. Yet, after checking numerous places I was unable to find how to get that from the training set. Checking the ...
user avatar
  • 1
0 votes
0 answers
79 views

How to interpret RMSE of 0 with a lot of features

I have 122 features for a regression problem. Here are some stats on a random forest model using RMSE: With no scaling or dimensionality reduction: Train RMSE: 0.0 Test RMSE: 0.0 CV RMSE: 0.0 With ...
user avatar
0 votes
0 answers
19 views

Can I use RMSE to diagnose overfitting in a Bayesian Calibration?

I am fitting a simulation model using Bayesian Calibration (DREAMZS MCMC using the BayesianTools packages in R). I have several time series I am calibrating to (e.g. log stream flow and nitrate ...
user avatar
2 votes
0 answers
58 views

Lower RMSE but worse model prediction

I am using a KNN model to predict quantity sold for a highly seasonal business. I chose KNN because I thought that using nearest neighbors would inform my model about said seasonality better than a ...
user avatar
1 vote
1 answer
25 views

When would you need scaled error between different time series evaluations?

Let's say we have 3 time series for three different fruits sales over one year. Although they are all fruits, their daily sold volume is very different. For example, imagine ...
user avatar
  • 377
0 votes
0 answers
40 views

Is it better to use the prediction interval or the RMSE?

I use a measurement technique $X$ to assess if a good is within production specifications during its manufacturing. $X$ is slow but precise. I wish to find out if I can use a measurement technique $Y$ ...
user avatar
  • 1
0 votes
0 answers
15 views

what is the difference between RMSE and Mc error (in Bayesian estimation)?

Please, I want to know if there is a difference between Root mean square error (RMSE) and Monte carlo error (Mc error). I need to compare my models but in Bayesian modelling I found a value like Mc ...
user avatar
1 vote
1 answer
157 views

Why would my R Squared increase AND my RMSE increase when I add more variables to a model?

I ran a regression with tidymodels following this following along with the random forest example here but using different data. When I ran it with four variables or so, I got an R Squared of 0.94 but ...
user avatar
0 votes
0 answers
48 views

How can I transform a rmse of a log scaled target?

I am struggling to interpret the rmse of a model with the log scaled target. When the rmse is 0.6, how can I know the real rmse value in order to know if the model is acceptable or not ?
user avatar
  • 1
0 votes
0 answers
38 views

About optimization intuition of RMSProp - oscillations (From Video of Andrew Ng)

In RMSProp video, it is stated that in the initial case where we have only gradient descent, we will have lots of oscillation to arrive at an optimal point for the loss. We assume that for ex. we move ...
user avatar
  • 189
1 vote
0 answers
133 views

how to know if the difference between test and train RMSE is significant to say its overfitted/under-fitted

I am trying to figure out if the model is overfitted/under-fitted based on RMSE values on test and train errors. On the training data, the RMSE is 0.283 On the test data, the RMSE is 0.758 since ...
user avatar
  • 31
5 votes
2 answers
2k views

RMSE or MAPE? which one to choose for accuracy?

I have a weekly times series for which I would like to find the best fit model. So far I've tried arima, Harmonic regression with arima error, neural network and in the end I would like to decide ...
user avatar
0 votes
0 answers
80 views

Computing square error for BIAS, RMSE and centre-pattern RMSE (or Unbiased RMSE) and its 95% confidence interval

I hope you are all well and can help me with my question. I'm comparing in situ data (X in log10) with model-estimated data (Y in log10) using linear regression. For this I calculate the BIAS, RMSE ...
user avatar
0 votes
1 answer
82 views

How do I report results of an internal validation in Caret?

I have the following question. In a machine learning project I have to solve a regression and a classification task. See also: Hold-Out VS Cross-Validation - R caret For this I have about ~650 cases ...
user avatar
  • 11
1 vote
2 answers
210 views

Hold-Out VS Cross-Validation - R caret

I have a question regarding hold-out vs. cross-validation. I have a dataset with ~650 cases which I am analyzing in R using the caret package. There I have a regression problem and a classification ...
user avatar
  • 11
1 vote
2 answers
58 views

What would be a reason to use the Root Mean Square Error (RMSE) to combine data?

In this machine learning tutorial by Google they use the Root Mean Square to create a similarity measure between two shoes. They first calculate the difference between the size of two shoes and then ...
user avatar
1 vote
1 answer
87 views

Random state value changes the results of rmse and R2

I want to know why everytime I run my algorithm (XGBoost regressor) with a different random state (applied to train/test split part) I get different values for R2 and RMSE. For example : Random state ...
user avatar
0 votes
0 answers
16 views

How can it be that two models differ significantly with repsect to R2 but not RMSE?

I have two models. Wilcoxon rank sum test says that the RMSE of these models (10-fold cross-validation) is not significant, but it is when using R2 instead of RMSE. How can this be? Could it be that ...
user avatar
  • 2,482
0 votes
0 answers
9 views

Repeated Cross-Validation: High Variance between Repeats (different splits) in RMSE?

I would like to compare how different linear models (different predictors) perform in predicting one outcome. Therefore, I use repeated cross-validation with RMSE as loss function. My n is around 160, ...
user avatar
  • 1
0 votes
0 answers
15 views

RMSE averaging NWP observation impact

I have a statistical question. I'm estimating the impact of wind observations in a global weather prediction model. Therefore, I calculate the difference between the 72hr forecast of a variable for a ...
user avatar
0 votes
1 answer
98 views

Is there a good replacement of sum of squared deviations that do not tend to split on edges?

I build a predictive model (regression) on a dataset that has just one real-valued feature and one real-valued target. To make it even simpler I want to find just a step function (decision tree with ...
user avatar
  • 973
1 vote
0 answers
55 views

High RMSE and High MAE in Autoencoder Regression

I have been developing a simple autoencoder model using PyTorch by which I am training the reconstructed output to be the same and input and also do regression on the hidden layer to predict a single ...
user avatar
0 votes
0 answers
126 views

Use fit.mult.impute results to generate ROC

I am new to the rms package but finding it has a lot to offer over base model functions. I have used MICE for MI on one variable with some missingness. I want to check the way that I am using the '...
user avatar
  • 225
0 votes
0 answers
14 views

How can I get standard error using filt.mult.impute?

Now I am struggling with obtaining standard error of hazard ratio. There are some missing values, so that I am trying to use Hmisc::fit.mult.impute to perform multiple imputations. Here you can see ...
user avatar
0 votes
0 answers
95 views

Cox Proportional Hazards Model for panel data

I want to get the same results of implementing cox-box by R for this SAS code, ...
user avatar
  • 31
0 votes
1 answer
95 views

Interpretation of RMSE

How is RMSE value interpreted? What makes it a good value? I used the tidymodels collect_metrics() function and am getting an rmse value of 182 for one of my models....
user avatar
  • 133
0 votes
1 answer
127 views

Difference between RMSE Mean vs RMSE Std?

I had two datasets that I wanted to cross validate using xgb.cv. I used a pandas dataframe, split each it into 2 where X = what I wanted to predict, y = features. ...
user avatar
3 votes
2 answers
173 views

How to reduce RMS error value in regression analysis & predictions - feature engineering, model selection

There's this dataset containing the metadata of Twitch's top 1,000 streamers of 2020. You can have the details here. I am currently participating in a challenge to predict the values for Followers ...
user avatar
0 votes
1 answer
126 views

How to compare two models with and without restricted cubic spline by likelihood test? #mice #survival # rms

I’m new to using multiple imputations and I would like an opinion on using it with survival analysis in R. I would like to perform a multiple imputation on data with missing values (mice package) and ...
user avatar
1 vote
0 answers
34 views

Is the following methodology to find the error metrics between the two curves technically sound?

I am comparing two curves, one of which is derived from physical experiment, while the latter is obtained from a simulation. The experimental curve was used to calibrate the simulation results. The ...
user avatar
  • 111
0 votes
1 answer
130 views

Meaning of interaction with %ia% in rms? Three-way interaction?

In this very illustrative post on evaluating added value of predictors by Frank Harrell, he codes a logistic regression model as such: ...
user avatar
4 votes
1 answer
90 views

Model performance in time-series forecasting with some outliers

I'm creating forecasts for products where some of them have large seasonal spike during times like Christmas and/or Easter but relatively low sales volume on other times. For this particular product ...
user avatar
1 vote
1 answer
25 views

Evaluating Supervised Model Performance Against a Baseline

My question is regarding how I can interpret the performance of a supervised ML task relative to a baseline estimator. I have run a supervised ML as a regression, and used K-fold CV to evaluate ...
user avatar
  • 213
1 vote
0 answers
133 views

How to do negative binomial regression with the rms package in R? [closed]

How can I use the rms package in R to execute a negative binomial regression? With the MASS package, I use the ...
user avatar
  • 1,557
0 votes
1 answer
139 views

Using standardized values (z-score) for MAE (Mean absolute error)

I have two models/indices that try to predict observed values. I've compared them using correlation and regression, but I'd like to use MAE (Mean absolute error) to asses which of them is more closer ...
user avatar
  • 1
1 vote
0 answers
78 views

Unrealistically High R squared value of 1 (and low RMSE of zero) [closed]

I am working on a regression problem of load prediction (where I try to estimate next hour consumption using previuous consumption values). At first I had relatively poor perfromance, however when I ...
user avatar
1 vote
1 answer
60 views

Low MAE, RMSE, RMSLE and MAPE, but also a low R^2

I have a dataframe containing the IDs of 2000 questions, a list of scores representing difficulty, and the following features: how often the question was answered, how often the answer has been ...
user avatar
1 vote
0 answers
20 views

Could a nomogram be built from meta-regression coefficients?

I'm trying to build a nomogram from meta-regression co-efficients. The meta-regression model is done using the metafor package. However, I would like to represent the results into a nomogram. The RMS ...
user avatar
  • 11
9 votes
2 answers
2k views

RMSE vs MSE loss function - the optimization solutions are equivalent?

If we optimize a function $f$ with respect to loss $L$, which is defined as RMSE; Are we going to get the same solution as optimizing MSE ? Even, if the function $f$ is non-linear (e.g. a neural ...
user avatar
0 votes
0 answers
37 views

How to combine multiple RMSE values for same study group?

So I am measuring accuracy between dentures manufactured by two different techniques. I do have RMSE value for each different, but I want to combine for each group and compare if there is a ...
user avatar
0 votes
1 answer
21 views

Why is a squared residual being called an RMSD?

I'm hoping that someone can help me understand why someone would call something an RMSD that is really just a squared residual. I'm using someone else's R script which performs simulations according ...
user avatar
  • 3

1
2 3 4 5 6