I'm brand new to machine learning and am currently taking a course on it. In the course, we have been using things such as gradient boosting, and linear regression to take data that has been collected and use it to predict the amount of rainfall at a particular location.
X has been given to us to use, and consists of 60,000 rows and 91 columns. Each of these 91 columns has something to do with the amount of rainfall (at least from my understanding). These columns have to do with the date, time of year, cloud patches, pressure at the time, velocity of wind, infrared data containing water vapor amounts, humidity, and brightness, and much more satellite and other types of data. This data gets mapped to a corresponding
Y value which is the amount of rainfall given that specific row of data.
Each time we use this data (we use kaggle), we have to take the model we trained on and predict rainfall given a bunch of
Xtest data. We make predictions and end up getting a mean squared error with a "score" back. Since I'm trying to learn more then just linear regression and gradient boosting (and have been encouraged to do so by my professor), I've been looking through some different models for machine learning. My question is, are any of these valid, and which model sounds right for the job?
kNN regression - Since I have a lot of fields of data, I figured running a kNN regression model would work well, using something like 50 nearest neighbors. The only problem I'm thinking of is how to translate 91 columns of data into a kNN regression problem. Would I have to essentially create one kNN model for each column and somehow reduce the dimensions until I have a 2D model that I can predict rainfall on? If so how would I accomplish something like this? Or am I completely wrong about kNN?
Deep learning/neural networks. I'll be honest I don't really understand a lot about the hidden layers, and looking through PyLearn2 (which my professor suggested) almost seems like a black box in a way. I can't really understand what is happening, and am not really learning much from it. From what I've read online, it also seems like I do not have enough training data for a good neural network. Despite that, would this be a recommended way to go about solving my problem?
Gradient boosting. So far, out of the two things I've tried (gradient boosting and linear regression), gradient boosting has yielded the best results, but not by much. Is there something I should be looking for to optimize an algorithm with this much data? Or is this not a good way to go?
I'm assuming linear regression is probably not the right job for this problem as I've had pretty bad results from using it.
However, I'm wondering, is there some model that could fit this problem? Or something I can learn about that might help me better understand a way to predict the rainfall?