1
$\begingroup$

I have code in R that calculates the RMSE from a Linear Regression model:

library(caTools)
library(hydroGOF)

Time <- c(406, 472, 4462, 172770, 172430, 176570) 
V1 <- c(-2.312226542, -3.043540624, -2.303349568, 2.007418028, -0.446950896, -0.515512845) 
Amount <- c(0, 529, 239.93, 3.99, 60.5, 9.81) 
ClassABC <- c(1, 1, 1, 0, 0, 0)

df <- data.frame(Time, V1, Amount, ClassABC)

df

set.seed(2)
split <- sample.split(df, SplitRatio=0.7)

train <- subset(df, split=TRUE)
Actual <- subset(df, split=FALSE)

# Create the model
Model <- lm(ClassABC ~.,data=train)

#Prediction
Prediction <- predict(Model, Actual)

#Comparing predicted vs actual model
plot(Actual$ClassABC,type = "l",lty= 1.8,col = "red")
lines(Prediction, type = "l", col = "blue")
plot(Prediction,type = "l",lty= 1.8,col = "blue")

#Finding Accuracy
rmse <- sqrt(mean(Prediction-df2$ClassABC)^2)/diff(range(df2$ClassABC))==1
rmse

The output for the RMSE is 3.805235e-15, is it possible to normalize this using NRMSE in R? If so, how?

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ What do you mean by "NRMSE"? How do you want to normalize your RMSE? Incidentally, a RMSE that is essentially zero strongly suggests to me that something is not quite right. $\endgroup$
    – Stephan Kolassa
    Commented Jan 20, 2019 at 16:35
  • $\begingroup$ NRMSE is the normalized root mean square function in R. I want to use it to transform the current RMSE value into a percentage (without hard code). I too have noticed the low RMSE value, my datum ranges from 0-1000 so an RMSE of 3.8 means the model is highly inaccurate. @StephanKolassa $\endgroup$
    – Nick
    Commented Jan 20, 2019 at 16:46
  • $\begingroup$ However, the appearence of the test and predicted models itself implys that the model is accurate, so it's contradicting @StephanKolassa $\endgroup$
    – Nick
    Commented Jan 20, 2019 at 16:48
  • 1
    $\begingroup$ There is no function NRMSE() in base R, nor nrmse(), so I assume you are using a package. Please tell us which one. If you want to transform your RMSE into a percentage, you will need to specify a meaningful baseline (a percentage of what?). And your RMSE is not $3.8$, it is $3.8e-15=3.8\times 10^{-15} =0.0000000000000038$, which still looks essentially like zero. $\endgroup$
    – Stephan Kolassa
    Commented Jan 20, 2019 at 16:50
  • $\begingroup$ The package I am using is hydroGOF, I am trying to find the percentage of the matching data points found in Class, with the total number of datums (1000). I can probably do this in R code when hard coding but I want it to be dynamic if a user uploads different data, hence the use of the function. @StephanKolassa $\endgroup$
    – Nick
    Commented Jan 20, 2019 at 17:02

1 Answer 1

Reset to default
2
$\begingroup$

Per the help page ?nrmse, it normalizes the RMSE either by the standard deviation of the actuals, or by their range, depending on whether norm="sd" or norm="maxmin". So either

sqrt(mean(Prediction-thedata2$Class)^2)/sd(thedata2$Class)

or 

sqrt(mean(Prediction-thedata2$Class)^2)/diff(range(thedata2$Class))

should do what you want.

(Since your target variable is called Class, I wonder whether this is a categorical variable, in which case all of the RMSE, the SD and the range would not be defined.)

Improve this answer
$\endgroup$
12
  • $\begingroup$ Changed the column in the file to 'ClassABC', and matched with the variables in the code. After running both functions the result is still: > sqrt(mean(Prediction-thedata2$ClassABC)^2)/diff(range(thedata2$ClassABC)) [1] 3.805235e-15 $\endgroup$
    – Nick
    Commented Jan 20, 2019 at 17:19
  • $\begingroup$ If diff(range(thedata2$ClassABC))==1, then that is to be expected. I suggest you edit your question to include a minimum working example, otherwise we can't help you very well. $\endgroup$
    – Stephan Kolassa
    Commented Jan 20, 2019 at 17:21
  • $\begingroup$ The output was FALSE, I have edited the question so it is a minimum working example. $\endgroup$
    – Nick
    Commented Jan 20, 2019 at 17:37
  • $\begingroup$ Your question is not a working example. We do not have access to data on your hard drive. Please simulate data within the code so your code runs on our machines. $\endgroup$
    – Stephan Kolassa
    Commented Jan 20, 2019 at 20:31
  • $\begingroup$ Thanks for being so patient with me, I have added a dataframe so it should be good to go @Stephan $\endgroup$
    – Nick
    Commented Jan 20, 2019 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.