How to work a data set with a huge number of features? I'm new to machine learning so I apologize in advance if the answer is obvious.
I have a dataset that includes an address feature.  I was hoping to use one hot encoding to create a feature for each street that appears in the dataset. So if my dataset looked like:
Miles| Address
4    | 111 First St
2    | 456 Grenada Ave
9    | 789 Grenada Ave

It would become:
Miles| Address          | First_St | Grenada_Ave
4    | 111 First St     | 1        | 0
2    | 456 Grenada Ave  | 0        | 1
9    | 789 Grenada Ave  | 0        | 1

Doing this blows up the number of features out to over 1500 and puts both the train and test files from around 90 mb to over 5gb.
I was hoping to apply a RandomForest to these new datasets, but when I call forest.fit (using Python/sklearn) the memory usage balloons out to 60gb and then the Python process dies.
Couple of questions:
Is it considered bad practice to create this many features? If not - how does one go about modeling with data this size?
 A: One hot encoding is probably the most generic approach in such a situation. However, note that a lot of features will be zero. If you have say, 1000 different streets, 999 features will be 0. 
To handle this kind of situations, you should probably use a sparse representation of your matrix, if you don't do it already.
Using:
class sklearn.preprocessing.OneHotEncoder(n_values='auto',
categorical_features='all', dtype=<type 'float'>, 
sparse=True, handle_unknown='error')

You can have a sparse representation of your data, after one hot encoding. And making sure that:
sklearn.ensemble.RandomForestClassifier

Handles sparse representations (which is the case, more informations here http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)
Note that, when doing one hot encoding, it may be the case that a lot of columns are 0 most of the time (if a street appears only once or twice). It is usually better (in terms of performance of prediction of the model) to drop these scarce observations.
One hot encoding may not be the only solution in your situation.
Are you trying to predict something (say, the prices of houses for the sake of the example) according to the street, or according to the location? 
You could also turn the adresses into their GPS coordinates (might require some efforts though) and use them as continuous predictors.
A: Agree with @RUser4512 - is the street really the best representation of that feature? Would something else like zip code, census tract, etc. be better for the problem you're trying to solve?
If street is the appropriate representation, then SVM could be a good approach. It works well with sparse data, and is the approach Google takes with their Sofia package.
