I am new to machine learning and this community too. So please pardon me if i make any mistake while putting up this question.

I am trying this https://www.kaggle.com/doaaalsenani/usa-cers-dataset problem from kaggle where i am trying to predict price of cars based on various parameters.

And I am not sure which algorithm to apply for this type of problem as it is having both categorical and numeric data but then also i tried to apply linear regression to it on price feature as price is dependent on all other features but after converting all categorical features such as color , model , brand to one hot encoding and applying feature scaling to all it gave me 2.4 mean squared error which is terribly bad , may be i am using irrelevant or too many features but i don't feel straight forward linear regression is good choice.

I completed a course on machine learning where when i applied linear regression all data was in numeric form but not in this case and i know linear regression is not a good choice for this problem or may be i need to make some modifications to it but i don't know what to do else.

Can anybody suggest me what should i begin with problems like this ? What algorithms are used or what kind of modifications can be used. I am open to any suggestions as i really don't have a path for solving this type of problem.

I dont know how to include dataset in question so i had given a link to problem and including a snapshot of dataset too. Sceenshot of dataset

And in my computation i dropped vin , lot , Unnamed:0 columns.

Please help me with my issues.


Why did linear regression behave so poorly here ? Is there any way I can determine before applying linear aggression if it should be applied or not , other than covariance/correlation matrix as that cannot be much useful with this case because it is having categorical data too.


Random forests for Regression is a suitable algorithm when there are mixed features (both numeric and categorical). I am assuming you are using Python for coding. Python has sklearn library which supports this algorithm. But right now, if I am right, there is no support for categorical variables for Random forests in sklearn. So you need to encode the variables. Below code you may try

from sklearn.ensemble import RandomForestRegressor
df = pd.read_csv('data.csv') # load the data here
X_cat = pd.get_dummies(df[['brand', 'model', 'year', 'title_status','color','state',' country', 'condition'])]
X = np.c_[X_cat.values ,df[['mileage']].values]
y = df[['price']].values
regr.fit(X, y)

Now you can know the feature importance's using


Before applying any machine learning make sure you preprocess the text, do standardization, convert text to numerical vectors. Below are the steps you may follow if you are just starting out(beginner) to implement some ML models (I have excluded the feature engineering steps)

1. EDA/Preprocessing Step

1.1 Perform the exploratory data analysis of different features, combination of features, see correlation between the features. With this step you will have a fell about the data and will get to know what features are actually important.

1.2 If you have sentences then consider removing the stopwords, perform stemming and remove any special characters.

1.3 Check for null values in the column. For numerical features you can impute them with mean imputation and similar techniques. For text features you can fill them as accordance with the problem. If you want, you can remove these null rows also.

1.4 If you are tackling the classification problem, then check the number of different class labels($Y_i$). If the dataset comes out to be imbalanced, then consider performing the upsampling/downsampling.

2. Data preparation step

2.1 Perform train-cv-test split

2.2 Perform the scaling of the numerical features. Use the mean and variance of the train dataset to scale the cv and test dataset

2.2 Convert the text into numerical vectors. For this you can use the word-embeddings like bag-of-words, tf-idf, word2vec etc. Learn the embedding/vocabulary from the train dataset only and then using these learned features transform your test ans cv datset

2.3 Let suppose you have $N$ datapoints and have columns as A(type = text), B(type = categorical) and C(type = numerical). Let us take the first row of the dataset. Let suppose after converting text into numerical vector and performing the one hot encoding, you get the first row as $[0.33\;1.978\;0.68]$ (value vector for column A), $[0\;1\;0]$(Value vector of column B) and $[9]$ (value vector for column C, this was already a numerical feature)
Now perform the concatenation step to get a single vector that you can feed the model. We will define vector $D$ as the concatenation of A, B and C. So $D\;=\;[0.33\;1.978\;0.68\;0\;1\;0\;9]$

2.4 Now you are ready to think about which model to implement

3.Training the model and prediction

3.1 Pick up a model, let say logistic regression/linear regression. Then Pick up a metric to check the performance of the model. For the above problem you are predicting the price so you may consider choosing Root mean square error(RMSE) as a metric.

3.2 Train the model and perform the hyperparameter tuning using the cross-validation score.

3.3 Use those hyperparameter and predict the results

You can google on how to implement the above steps. scikit-learn offers an excellent documentation, you may refer them.

Hope this helps!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.