Skip to main content

Questions tagged [rms]

RMS stands for 'root-mean-square' is a measure of the typical size of a varying quantity. It occurs in the n-denominator form of standard deviation (the RMS deviation from the mean)

Filter by
Sorted by
Tagged with
0 votes
0 answers
13 views

Is there any value in models that have a larger out of sample RMSE than a standard deviation?

I am predicting y values from x values using various regression models, elastic net and partial least squares regression (PLSR). To quantify performance of models we utilize root mean squared error (...
Sir Veza's user avatar
0 votes
0 answers
14 views

rms::val.surv function estimated the same survival probability for all cases

I have fit a cox regression model, and used val.surv function to plot calibration plot to compare observed survival probability with predicted survival probability. ...
Xixuan Zhu's user avatar
1 vote
0 answers
20 views

Testing the difference between two Root Mean Square Error values for statistical significance [duplicate]

I would like to compare the predictive power of 2 models. The models are meant to model count data and respective probabilities. I am using two metrics as means of comparison: Root Mean Square Error ...
Astral's user avatar
  • 133
2 votes
1 answer
21 views

Is Mean Square Prediction Error acceptable to use if predicted values are continuous but actual observed values are discrete?

I would like to compare the predictive power of 2 models. The models are meant to model count data, so the actual observed values are discrete. However both models are designed such that they output ...
Astral's user avatar
  • 133
0 votes
0 answers
10 views

How to determine the ideal number of components for PLSR using RMSE?

I would like to determine a non-visual, numeric based approach to determine the ideal number of components for my PLSR model. There are 10 components in the model for 1 target variable. If I simply ...
tds's user avatar
  • 1
1 vote
1 answer
142 views

Regression spline for time to allow for slope changes

Suppose we have a regression / survival model where we would like to model follow-up time using a regression spline. Follow-up time has two phases (first treatment active, and second treatment ...
user167591's user avatar
0 votes
0 answers
16 views

Standard error of RMSE and differences in RMSE

I have a set of models $M = \{1, ..., m, ..., K\}$, and for each I am calculating RMSE on out-of-sample data as standard: \begin{equation} \mathrm{RMSE_{m}} = \sqrt{\frac{1}{N} \sum_{i=1}^{N} (\...
user_15's user avatar
  • 185
4 votes
1 answer
228 views

Approximation function for MLP and LSTM

I have total of 6300 samples, 5800 of which are training data, and 500 of which are testing data. We compare the performance of LSTM and multilayer perceptron (MLP) with one hidden layer in terms of ...
D. S.'s user avatar
  • 69
0 votes
0 answers
24 views

How to compute the confidence interval of root mean square (e.g., ci of the RMS of the alternating current signal)

If $\bf x$ are $N$ measurement of alternating current signal (sinusoidal), its root mean square is computed using $RMS=\sqrt{\frac{\sum{x_i^2}}{N}}$. My question is: Does the confidence interval exist ...
Yuanyi Wu's user avatar
  • 101
6 votes
1 answer
114 views

Valid confidence intervals in GAM’s using shrinkage estimation

In this blog article: https://www.fharrell.com/post/improve-research/ it states: “The frequentist paradigm does not provide confidence intervals or p-values when parameters are penalized”. I was ...
user167591's user avatar
1 vote
0 answers
26 views

Observational pre post design and confounder adjustment

I have observational data with baseline, and two follow-up measures with a binary treatment. Dependant variable is questionnaire scale score ***. I am planning on fitting a linear mixed effects model ...
user167591's user avatar
5 votes
3 answers
167 views

Cox model - unsure of time unit of analysis

I am running a survival analysis (Cox model) on time to event in cancer patients. The start of followup is end of treatment. Tests for the event (recurrence) are performed every 6 months from cancer ...
user167591's user avatar
0 votes
0 answers
42 views

Question about LASSO, RMSE, and Standardization

I have a question about doing LASSO in R using glmnet. It's kind of a conceptual question; I learned that we should interpret RMSE after performing ...
noone's user avatar
  • 43
0 votes
0 answers
14 views

in ann model statistical calculation can mae be greater than rmse? [duplicate]

in ann modelling creation can mae be greater than rmse? i am using the LMNN algorithm for creting the ann model.i am having the values as said above in the performance evaluation of the ann model.can ...
Tapaswini Mohanty's user avatar
4 votes
1 answer
92 views

Shrinkage of covariates in the Cox model

In a regression model (e.g Cox model) when there are too few events to support modeling all desired covariates / confounders, a possible solution is to apply shrinkage / penalise all but the exposure(...
user167591's user avatar
4 votes
2 answers
48 views

Machine learning benchmarks: MAE, RMSE, and R-squared

I'm working on a machine learning problem, and I'm having trouble interpreting different measures of model performance. I have a single dependent variable (proportion change between two treatments, ...
S. Robinson's user avatar
3 votes
1 answer
71 views

Restricted cubic splines for time-to-event data

I'm kind of new in fitting rcs for cont. variables as a clinican, so I have a few questions: ...
sjg's user avatar
  • 33
1 vote
0 answers
43 views

How to compute relative error of multi-dimensional time-series?

I have written a python script that uses a variety of different integrators to simulate the gravitational N-body problem. I would like to compare the positions obtained from my simulation to the ...
user23358153's user avatar
0 votes
0 answers
22 views

Why is taking the mean RMSE sometimes so far off overall RMSE?

I'm working with a multi-threaded program, which splits a dataset into N chunks, and evaluates some regression model's performance, predicting a score for each item in each chunk. I'm using RMSD as ...
Seán Healy's user avatar
1 vote
0 answers
17 views

Poor RMSEA/Fit for Simple Poisson Regression

I am running a simple Poisson regression. $X$ = time, $Y$ = count data. This is a huge dataset with many years. There is significance between $X$ and $Y$. But model shows poor fit via high RMSEA value....
mmt1026's user avatar
  • 11
0 votes
0 answers
26 views

how to calculate overall RMSE accumulated during several processing steps

I have a digital terrain model (DTM) downloaded from NASA's SRTM dataset at a resolution of 1 arc second covering Spain and France. This has a stated RMSE of 9.73m [output 1] I projected this to ...
richjh's user avatar
  • 1
0 votes
1 answer
40 views

How to construct an optimal spline model when two continuous independent variables are included

I am interested in evaluating the relationship between age, BMI and lipid level. The lipid level is an outcome in my study. I think that the relationship between lipid level and age and the ...
Totti's user avatar
  • 35
1 vote
1 answer
28 views

Comparing families of classifiers on large datasets using mixed effect logistic regression models on individual questions

I have a testing dataset of about 6000 images which I am going to try about 25 different neural networks on in a multi-class classification problem. Each network will belong to around 5 families (e.g. ...
James's user avatar
  • 473
1 vote
1 answer
75 views

Can you compare regression models using RMSE when samples have different proportions of zeros?

I am using the ranger package (which implements random forests) in R to build regression models of tree species' basal area, a continuous measure of abundance and ...
Jim Worrall's user avatar
0 votes
0 answers
30 views

Compare Root Mean Square Values

I'm trying to compare a regression neural network to a commonly used equation. I have an 80:20 split for my training:test, and I get the root mean square error on the test set from the neural network ...
Jack789's user avatar
4 votes
1 answer
85 views

Why isn't there a square root version of the Brier score similar to how RMSE complements MSE?

When computing the mean squared error of a regression model, we get a metric in square units. For ease of interpretation, we can therefore instead compute the root mean squared error, which are in ...
another_student's user avatar
0 votes
0 answers
22 views

RMS residual normalized by standard-deviation

Is there a proper name for the following misfit quantification? misfit=√{∑[(xi−xi')^2/(n*𝜎i^2)]} where n is the number of data points xi−xi' is the ith residual, ...
geoweaser's user avatar
1 vote
1 answer
669 views

How to interpret interaction effect in ordinal (logistic) regression?

I ran an ordinal regression in R with the polr function from the MASS package as described in this tutorial, which is very good. However, the tutorial does not ...
Simone's user avatar
  • 252
0 votes
0 answers
136 views

Normalization of log-returns or normalization of cumulative log-returns

This questions seeks for discussion to find theoretical support for normalizing cumulative log-returns vs normalizing log-returns By "normalizing" (also known as standarizing) I mean it in ...
Alfonso's user avatar
  • 21
1 vote
0 answers
125 views

Cox model with ridge term: how to choose value of theta?

The coxph() function in R package "survival" is used to fit the Cox proportional hazards model. This function allows a ridge() term in the formula to penalise selected terms, which requires ...
user167591's user avatar
0 votes
1 answer
50 views

Root Mean Square Error of the addition of two measurements whose RMS Error is known

I am working on a measurement system which tries to measure the distance between two values i.e $\Delta F=F_1-F_2$. Where $F_1, F_2$ are the values I actually measure. I have set up a Monte Carlo ...
bad_at_stats's user avatar
1 vote
1 answer
323 views

Can I say that the relative root mean squared error is the averaged percentage error?

RMSE is an error metric in which the mean of the data minimizes its loss function: $\text{RMSE} = \sqrt{\frac{\sum_{t=1}^{n}(y_t - \hat{y_t})^2}{n}}$ But it gives ...
Guilherme Parreira's user avatar
0 votes
0 answers
116 views

Cramer-Rao bound (CRB) and Root-Mean-Square-Error / Mean-Square-Error (RMSE / MSE)

My question is regarding the comparison between the CRB of a given vector parameter and RMSE/MSE obtained from Monte-Carlo (MC) simulation. The approach I used is this: For $\boldsymbol{\theta} \in \...
Zero's user avatar
  • 121
3 votes
0 answers
132 views

Longitudinal RCT modeling of continuous time

I have data from an intervention study (10 clinics, 5 control, 5 treatment). The outcome is counts, and we have monthly data at baseline, treatment active phase, and post treatment phase. The number ...
user167591's user avatar
1 vote
1 answer
40 views

Should the RMSE of an unrestricted VAR model decrease as compared to a restricted Autoregression model when there is Granger Causality

I have 2 time series, say for instance, T1 and T2. T1 granger causes T2 at lag 2. Should this mean that if I make a VAR model with these two time series, and an autoregression model with just T2, the ...
Ritik P. Nayak's user avatar
1 vote
1 answer
46 views

Should the RMSE of the unrestricted (VAR) model for a time series that is being Granger caused by another be lesser than its restricted counterpart?

I have a couple of time series, say, T1 and T2. I have established (using the grangercausalitytest library of Statsmodels in ...
Ritik P. Nayak's user avatar
0 votes
1 answer
117 views

Comparing RMSE values across different datasets

I am working on a PV energy production forecasting problem. With various ML models (ANN, RNN, LSTM) I am trying to predict the energy for the following day, based on the historical data. The ...
GCMeccariello's user avatar
0 votes
1 answer
166 views

Is there an error metric that decreases the weight when the target is near zero?

As precipitation prediction models can only predict positive values, they won't be able to undershoot small values by much. When it comes to overshooting, there is no boundary. High precipitation ...
schefflaa's user avatar
2 votes
1 answer
105 views

RMSE model interpretation

Let's say I train a model and it has an RMSE of 2.5. Does this mean, that on average, my prediction will be 2.5 away from the true value? Or does some scaling need to be done in oreder to get this ...
the man's user avatar
  • 285
0 votes
1 answer
899 views

Why the results of Cox regression are different between coxph() and cph() in rms package

I found the predicted hazard (the h(t) of Cox regression) through Predict() and cph() in rms package was different from common coxph(). ...
tumidou's user avatar
  • 75
0 votes
0 answers
24 views

Obtaining 12 month ahead in sample RMSEs

I am attempting to recreate the results of the paper written by King, Stock and Watson in 1995: Temporal instability in the unemployment inflation relationship. The paper estimates a VAR model with 12 ...
Varun Sinha's user avatar
1 vote
1 answer
178 views

Cox model: how to model treatment variable when timing is unknown

We have cancer medical registry data, including information on date of diagnosis, treatment, and followup e.g. date of death etc. However, we only know type of treatment received for each person. We ...
user167591's user avatar
2 votes
1 answer
840 views

Restricted cubic spline looks like a linear curve, but p for nonlinear < 0.001

I am analyzing a association between a frailty index and care needs using the cox model. I use R and use rms package to fit restricted cubic spline. This is my R code. ...
li jiaqi's user avatar
0 votes
0 answers
100 views

ROC with bootstraping

I have a data with 2 variables: diagnosis- yes/no Score- numeric variable from 0-10. I need to do ROC analysis for this data and to find the best cut off values. The problem is the data is too small ...
Inbar Lavie's user avatar
1 vote
2 answers
115 views

Get the R2/RMSE for each category of a dataset

This might be a dumb question ! I built a model and I'm satisfied enough with the model, given that I have a dataset with categorical variables I wanted to see the R2/RMSE for each of those categories,...
Omar Sow's user avatar
2 votes
1 answer
68 views

How do you interpret the value of RMSE/MSE in English to stakeholders?

For example, if you have a R^2 of 0.95, you can explain this number to stakeholders in a presentation as: Our model explains 95% of the total variance within the data. However, if we have a RMSE of 11,...
Katsu's user avatar
  • 1,011
1 vote
1 answer
141 views

Why RMSE and MAPE changes with the change of axis?

I need your help regarding the information inside the picture. As you know all the information will change with the change of NDVI axis from y-axis to X-axis, except R2 and p-value remain the same? ...
Hushiar Raheem's user avatar
1 vote
1 answer
440 views

How do we relate RMS and standard deviation for continuous signals?

Because the discrete formula for RMS, $\displaystyle X_{RMS}=\sqrt{{1 \over N}(x[1]^2+x[2]^2+...+x[N]^2)}$, is almost the same as the formula for standard deviation (assuming mean zero), except for a ...
Homero Esmeraldo's user avatar
1 vote
1 answer
26 views

What values of the independent variable do I use to compute the predicted data when calculating RMSE?

I have a data set with x = [10 16.25 16.25 16.25 16.25 20 22.5 22.6875 24.57 24.57 41.86 47 47 53.8 66.43 77.9 91.201 96.2 97.2] and y = [1.28 4.15 3.42 1.53 3.44 4.89 2.91 8.51 9.03 14.91 9.73 8.07 ...
fttoinches's user avatar
1 vote
1 answer
80 views

Summarizing a set of root-mean-squared error values

From simulation studies, I have repeated (1000) measures of the root-mean-squared-error (deviation) between a sample of observed values and the predicted value. The obtained RMSE values are, naturally ...
CrimsonDark's user avatar

1
2 3 4 5
7