I'm creating a regular linear regression model to establish a baseline before moving on to more advanced techniques. I scaled my data as below:
from sklearn.preprocessing import StandardScaler sc = StandardScaler() X_train_std=pd.DataFrame(sc.fit_transform(X_train), columns=data.columns) X_test_std=pd.DataFrame(sc.transform(X_test), columns=data.columns)
However, the variables mostly have an extreme skew (right tail), but I can't figure out how to apply a log transform on them.
Would I apply the log transform to variables in both the X_train and X_test datasets? Do I need to do this before applying the scaling? I just can't think through the right way to go about this in terms of applying predictions to the X_test set. Any ideas?