0
$\begingroup$

I would like to analyze the prostate gene expression data which has a link named 12859_2005_967_MOESM4_ESM.tgz in the site here.

In a paper I read, the author scaled the predictors in the training set before fitting a logistic regression model. Hence, I need to scale the predictors in the training set to unit variance before the analysis.

However, my program code (in R) generated NA values after scaling the data.

What can be the reason for this?

The code is below:

library(R.matlab)
library(dplyr)

data <- readMat("prostate_data_label.mat")

X <- data$prostate.data
colnames(X) <- paste0("Gene",1:ncol(X))
y <- data$prostate.label
y <- y-1
colnames(y) <- "y"

datab <- cbind(y,X)
datab <- as.data.frame(datab)

ind1 <- which(y==1)
ind0 <- which(y==0)

set.seed(1)
train.ind1 <- sample(ind1, size=floor(length(ind1)*0.7))
train.ind0 <- sample(ind0, size=floor(length(ind0)*0.7))
train.ind <- c(train.ind1, train.ind0)

train <- datab %>% slice(train.ind)
test <- datab %>% slice(-train.ind)

trainx <- train %>% select(-"y")
trainy <- train %>% select("y")

trainx <- scale(trainx)

any(is.na(trainx))
# [1] TRUE
$\endgroup$
0
$\begingroup$

It seems that you have problems with the genes that do not show any variation. For example, I had a look at Gene783 where you get NA values after the scaling: the column corresponding to this gene in your data consists only of ones. It is impossible to scale the degenerate random variable in a way that will result in it having a unit variance.

I do not have much experience with the scale function myself, but I guess it tries to do that, fails, but the result is NA.

You have quite a few of such predictors, you can find their names by which(is.na(colSums(trainx))).

I would remove them from the sample before performing logistic regression.

New contributor
Misius is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$
7
  • $\begingroup$ I'm aware that is the result of small/no variation. However, this data set was used in many papers, and they reported the results. If the data can not be scaled, how did they obtain them? $\endgroup$ – mert 19 hours ago
  • $\begingroup$ @mert I think that you are talking about some other paper, because in the one you are citing the authors do not mention neither scaling, nor logistic regression. Unfrotunately, I can not comment on how people obtained some results without knowing the context. $\endgroup$ – Misius 18 hours ago
  • $\begingroup$ There is also a possibility that their training set included some variation of the data so they could scale the results. For example, talking about Gene783, there is one entry that is not equal to 1.0, and if this entry is in the training set, then the data can be scaled. $\endgroup$ – Misius 18 hours ago
  • $\begingroup$ I think that is not the case. Because they claim that they used random splitting and repeated the process 50 times. This is one of the papers: Penalized logistic regression with the adaptive LASSO for gene selection in high-dimensional cancer classification. Anyway, I will accept your answer. $\endgroup$ – mert 18 hours ago
  • $\begingroup$ Yeah, I see that they assume the genes are standardized but they never actually say how they standardize them in practice. If I would be trying to replicate their study, I think I would just leave those genes as they are, maybe only scaling them so that they lie in [-1, 1]. With couple of genes, it does not matter that much. I guess they wouldn't be selected anyway. $\endgroup$ – Misius 18 hours ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.