1
$\begingroup$

I have a large body of records pertaining to user activity on a website. What I want to do is some sort of classification on each user as they navigate my website. Every algorithm I found so far uses one row at a time, but I want to take into account the sequence in which rows appear, as well. That is, there is a time component to the data and dependencies between rows. I want to be able to tell the algorithm/software that it should take each set of rows with the same SessionIdentifier to be the same "record"

Example of training data:

Bob   | Page1  | SessionIdentifier1 | timestamp
Bob   | Page2  | SessionIdentifier1 | timestamp
Bob   | Page3  | SessionIdentifier1 | timestamp
Alice | Page1  | SessionIdentifier2 | timestamp
Alice | Page4  | SessionIdentifier2 | timestamp
Alice | Page1  | SessionIdentifier2 | timestamp
Alice | Page3  | SessionIdentifier2 | timestamp
Alice | Page2  | SessionIdentifier3 | timestamp
Alice | Page24 | SessionIdentifier3 | timestamp
Alice | Page34 | SessionIdentifier3 | timestamp
etc...

Can someone point me in the right direction of a place to get started on this project?

$\endgroup$
2
  • $\begingroup$ This is not a specific programming question. If you need help choosing statistical methods or machine learning techniques to analyze data, try posting to Cross Validated or Data Science instead. Otherwise, make your question much more specific about the programming challenge you are struggling with. $\endgroup$
    – MrFlick
    Commented Dec 11, 2014 at 16:53
  • $\begingroup$ This is a well known problem in web analytics where you want to classify a sequence of requests by a given user over a given timespan as a single "visit". It's easy to do in R but your question is too vague. Post some real data - enough to demonstrate the problem. $\endgroup$
    – jlhoward
    Commented Dec 11, 2014 at 17:00

1 Answer 1

1
$\begingroup$

This should be a good start for ya, but you need work on your understanding of the statistical modeling and pre-processing of data.

FYI: This will error out at the end because I did not handle the index so you can go ahead and correct that.

Also, SequenceVar this will become your factor that is inputted into the model, also I do not know what your data looks like but if it has too many levels then it will not be able to be handled by most models.

library(sqldf)
library(reshape)
Test = structure(list(UserName = structure(c(2L, 2L, 2L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L), .Label = c("Alice", "Bob", 
"Chris"), class = "factor"), Page = structure(c(1L, 2L, 4L, 1L, 
7L, 1L, 4L, 2L, 3L, 5L, 1L, 2L, 6L, 8L, 9L), .Label = c("Page1", 
"Page2", "Page24", "Page3", "Page34", "Page35", "Page4", "Page5", 
"Page6"), class = "factor"), SessionID = c(1965520.44, 1965520.44, 
1965520.44, 2995108.043, 2995108.043, 2995108.043, 2995108.043, 
2995108.043, 2995108.043, 2995108.043, 1842002.603, 1842002.603, 
1842002.603, 1842002.603, 1842002.603), CT = c(1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 1, 1, 1, 1)), .Names = c("UserName", "Page", 
"SessionID", "CT"), row.names = c(NA, -15L), class = "data.frame")

Test$RowID = 1:nrow(Test)
Test$Page = as.character(Test$Page )

# First I would switch it to binary variables for classification
Pivot = cast(Test, UserName +  SessionID  ~ Page )
Pivot[is.na(Pivot)] <- 0
Pivot

# Second I would create the sequence and compare the results to the previous model
Test$SequenceVar = ''

for(i in 1:nrow(Test)){

Sequence = Test[2][i,]

for(Seek in 1:999){

if(Test[1][i,] == Test[1][i+Seek,])
    {

    Sequence = paste(Sequence,"-",Test[2][i+Seek,],sep="")
    Test$SequenceVar[i] = Sequence

        }
        else if(Test[1][i,] != Test[1][i+Seek,]){break}
    }
}

library(sqldf)

sqldf("Select UserName,Page,SessionID,CT,RowID,SequenceVar
        from Test b
        where RowID  in (select min(RowID )
                  from Test b1
                  where b.UserName = b1.UserName
                  group by UserName)")
$\endgroup$
2
  • $\begingroup$ Thanks for the starting point! Is there a resource in particular you would recommend for me to learn more? $\endgroup$
    – towelyey
    Commented Dec 11, 2014 at 20:20
  • $\begingroup$ @towelyey I would start out with a strong understanding of SQL, then move on to R & Python. Some of the best resources are free. Check out udemy, coursera, edx, and YouTube. Once you have a stable coding background look into data mining which focuses on the following: identifying sources, cleaning, variable selection, variable transformation, and variable creation. Best of luck cheers! $\endgroup$ Commented Dec 11, 2014 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.