2
$\begingroup$

My question is about the typical feed-forward single-hidden-layer backprop neural network, as implemented in package nnet, and trained with 'train()' in package caret. This is related to this question [Multi-layer neural network wont predict negative values ] but in the context of the nnet and caret packages in R.

I demonstrate the problem with a simple regression example where Y = sin(X) + small error:

(a) raw Y ~ raw X: predicted outputs are uniformly zero where raw Y < 0.

(b) scaled Y (to 0-1) ~ raw X: solution looks great; see code below.

library(nnet)
X <- t(t(runif(200, -pi, pi)))
Y <- t(t(sin(X)))           # Y ~ sin(X)
Y <- Y + rnorm(200, 0, .05) # Add a little noise
Y_01 <- (Y - min(Y))/diff(range(Y)) # Y linearly transformed to have range 0-1.
plot(X,Y)
plot(X, Y_01)
dat <- data.frame(cbind(X, Y, Y_01)); names(dat) <- c("X", "Y", "Y_01")
head(dat)
plot(dat)

nnfit1 <- nnet(formula = Y ~ X, data = dat, maxit = 2000, size = 8, decay = 1e-4)
nnpred1 <- predict(nnfit1, dat)
plot(X, nnpred1)

nnfit2 <- nnet(formula = Y_01 ~ X, data = dat, maxit = 2000, size = 8, decay = 1e-4)
nnpred2 <- predict(nnfit2, dat)
plot(X, nnpred2)

When using train() in caret, there is a preProcess option but it only scales the inputs. train(..., method = "nnet", ...) appears to be using the raw Y values; see code below.

library(caret)
ctrl <- trainControl(method = "cv", number = 10) 
nnet_grid <- expand.grid(.decay = 10^seq(-4, -1, 1), .size = c(8))
nnfit3 <- train(Y ~ X, dat, method = "nnet", maxit = 2000, 
        trControl = ctrl, tuneGrid = nnet_grid, preProcess = "range")
nnfit3
nnpred3 <- predict(nnfit3, dat)
plot(X, nnpred3)

Of course, I could linearly transform the Y variable(s) to have a positive range, but then my predictions will be on the wrong scale. Though this is only a minor headache, I'm wondering if there is a better (built-in?) solution for training nnet or avNNet models with caret when the output has negative values.

Thanks for any suggestions or insight you can provide!

$\endgroup$
1
$\begingroup$

Is the question related to the scale in which nnet predicts? Since Y is roughly between -1 and 1 you should also use linout = FALSE in your nnet and train calls.

$\endgroup$
  • $\begingroup$ linout was the key, though the default is FALSE so it is actually linout = TRUE that solves the problem. Thanks. $\endgroup$ – bsbk Sep 22 '14 at 13:03
0
$\begingroup$

The pre-process options apply to the X variables, NOT the Y variable. There's currently no way to preProcess the Y variable (that I know of). Perhaps there needs to be? Please post an issue to the tracker!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.