0
$\begingroup$

I am very new to machine learning and I am trying to understand the whole process of preparing the data for the machine learning part. I am making use of pipeline from sklearn.

Lets say I have the following dataset:

df.head()
Out[16]: 
  school sex  age address famsize Pstatus  Medu  Fedu     Mjob      Fjob  \
0     GP   F   18       U     GT3       A     4     4  at_home   teacher   
1     GP   F   17       U     GT3       T     1     1  at_home     other   
2     GP   F   15       U     LE3       T     1     1  at_home     other   
3     GP   F   15       U     GT3       T     4     2   health  services   
4     GP   F   16       U     GT3       T     3     3    other     other   

   reason guardian  traveltime  studytime  failures schoolsup famsup paid  \
0  course   mother           2          2         0       yes     no   no   
1  course   father           1          2         0        no    yes   no   
2   other   mother           1          2         3       yes     no  yes   
3    home   mother           1          3         0        no    yes  yes   
4    home   father           1          2         0        no    yes  yes   

  activities nursery higher internet romantic  famrel  freetime  goout  Dalc  \
0         no     yes    yes       no       no       4         3      4     1   
1         no      no    yes      yes       no       5         3      3     1   
2         no     yes    yes      yes       no       4         3      2     2   
3        yes     yes    yes      yes      yes       3         2      2     1   
4         no     yes    yes       no       no       4         3      2     1   

   Walc  health  absences  G1  G2  G3  
0     1       3         6   5   6   6  
1     1       3         4   5   5   6  
2     3       3        10   7   8  10  
3     1       5         2  15  14  15  
4     2       5         4   6  10  10  

As you guys can tell, there are a lot of categorical variables. I do understand that you can use pd.get_dummies to do the OneHot encoding but I want to use pipeline to achieve the same thing. Here is what I have tried so far:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import OneHotEncoder
from sklearn.preprocessing import OrdinalEncoder
from sklearn.compose import ColumnTransformer

cat = ['school', 'sex','address', 'famsize', 'Pstatus','Mjob', 'Fjob', 'reason', 'guardian','schoolsup', 'famsup', 'paid', 'activities', 'nursery',
       'higher', 'internet', 'romantic']
nom = ['schoolsup', 'famsup', 'paid', 'activities', 'nursery','higher', 'internet', 'romantic']
ord = ['school', 'sex', 'address', 'famsize', 'Pstatus','Mjob', 'Fjob', 'reason', 'guardian']

num = ['age','Medu', 'Fedu','traveltime', 'studytime','failures','famrel', 'freetime', 'goout', 'Dalc',
       'Walc', 'health', 'absences']

### Pipeline for categorical variables which are Yes/No
ord_pipeline = Pipeline([
    ("onehot",OneHotEncoder()),
])

### Pipeline for categorical variables which have multiple categories
nom_pipeline = Pipeline([
    ("ordinal",OrdinalEncoder()),
])

num_pipeline = Pipeline([
                        ("scaler", StandardScaler()),
                        ])

ct = ColumnTransformer(transformers = [

    ("ord",ord_pipeline,ord),

    ("nom",nom_pipeline,nom),

    ("num",num_pipeline,num)])

Xprep = ct.fit_transform(df)

Now the result from the above is an array.... with no column information. So I want to run a correlation between the input and the target variables, I am not sure how to do so...

Does anyone have any ideas? Or am I understanding it wrong?

Thank you

$\endgroup$

closed as off-topic by Tim Aug 14 at 14:44

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Tim
If this question can be reworded to fit the rules in the help center, please edit the question.