1
$\begingroup$

Hi my question is a bit long please bare and read it till the end.

I am working on a project with 30 participants. We have two type of data set (first data set has 30 rows and 160 columns , and second data set has the same 30 rows and 200 columns as outputs=y and these outputs are independent), what i want to do is to use the first data set and predict the second data set outputs.As first data set was rectangular type and had high dimension i have used factor analysis and now have 19 factors that cover up to 98% of the variance. Now i want to use these 19 factors for predicting the outputs of the second data set.

I am using neuralnet and backpropogation and everything goes well and my results are really close to outputs.

My questions :

1- as my inputs are the factors ( they are between -1 and 1 ) and my outputs scale are between 4 to 10000 and integer , should i still scaled them before running neural network ?

2-I scaled the data ( both input and outputs ) and then predicted with neuralnet , then i check the MSE error it was so high like 6000 while my prediction and real output are so close to each other. But if i rescale the prediction and outputs then check The MSE its near zero. Is it unbiased to rescale and then check the MSE ?

3- I read that it is better to not scale the output from the beginning but if i just scale the inputs all my prediction are 1. Is it correct to not to scale the outputs ?

4- If i want to plot the ROC curve how can i do it. Because my results are never equal to real outputs ?

Thank you for reading my question

Improve this question
$\endgroup$
5
  • $\begingroup$ What you mean with scale input or output? $\endgroup$
    – Alvaro Silvino
    Commented Feb 21, 2016 at 9:56
  • 1
    $\begingroup$ I mean bringing the data between range 0 and 1 because my inputs are already in range (-1 and 1 ) and my output (y) are in thousands. maxs <- apply(data, 2, max) mins <- apply(data, 2, min) scaled <- as.data.frame(scale(data, center = mins, scale = maxs - mins)) $\endgroup$
    – ramin
    Commented Feb 21, 2016 at 13:59
  • $\begingroup$ you used normalization from 0 to 1. You should do that to the output. So your MSE will be from 0 to 1. $\endgroup$
    – Alvaro Silvino
    Commented Feb 21, 2016 at 14:17
  • $\begingroup$ I have done that before but i do not understand why my MSE error will get so high (MSE= 5234), but if i rescale the output and the predicted value back to original scale and calculate the MSE its near zero. is that a reliable approach ? $\endgroup$
    – ramin
    Commented Feb 21, 2016 at 14:57
  • $\begingroup$ You have to use normalization in the output. Or use MAPE measure ( perceptual error). Yes that's reliable. $\endgroup$
    – Alvaro Silvino
    Commented Feb 21, 2016 at 15:02

1 Answer 1

Reset to default
1
$\begingroup$

I will answer about MSE and ROC curve.

2- if you want to use MSE, you usually need to normalize the input data. So the squared error can be minimized to zero. Another approach is using MAPE measurement.

4- ROC curve is used to classification problem. You can't use ROC curve to regression output.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.