I have a long list of access logs, associated with some persons, lets say web access-log. I want to build prediction model for some target varaible, associated with persons, and i have a training set for this.
What troubles me, is that to prepare training set, containing some extraction from web logs, like domain name + number of pages visited, i need to build some flat table with enormous number of columns, representing unique domains (10k or maybe 100k depending on part of the log i will extract).
What is a common approach for such problem? Should i try to reduce dimentionality first, trying co group up domain names? But that will be a loss of data, because i believe some combination of domains can influence target variable. Or should i learn some algorithms, able to work with "longitudal" data, because most techniques i familiar with require data to be flat?
Thanks for any advice and direction, my primary instruments are R and SQL