1
$\begingroup$

Data:

I have about 1,000 Longitudinal Datasets from 1000 schools, similar to the sample table shown below, which I am trying to classify. Instead of yearly scores, I have detailed monthly scores.

Problem

Each School (Entire Dataset) has a corresponding label representing whether or not it was effected by a natural disaster during this period. There is no information pertaining to when (which year or month) exactly the disaster occurred, but rather the objective is to seek trends that distinguish the two classes.

Question

What algorithms can I leverage to solve such a problem? I have used a Convolutional Neural Network but the results are very disappointing.

enter image description here

$\endgroup$

2 Answers 2

1
$\begingroup$

Given that the ordering of students is arbitrary, I don't see why a (2D) convolutional neural network makes sense. However, a 1D convolutional layer would be quite a good idea--based on the question it sounds like you're looking for a feature that is a shift-invariant convolutional feature in the time domain, after all. However, neural network training is finicky and it's hard to interpret results, so I wouldn't really suggest it.

Logistic regression is much more reasonable, though I'm assuming there are quite a lot of students per dataset and so I'd suggest you average over students but keep the years intact. It sounds like you're not really interested in any individual student (because each student only went to one school) but rather average trends -- this also helps because now you should have the same number of features per data point (each data point here corresponding to what you refer to as a data set).

$\endgroup$
1
  • $\begingroup$ I will try 1D convolution. Thank you for the perspective, I really doubt averaging would work however because subtle fluctuations may dampen the effect of the event significantly. $\endgroup$
    – mamafoku
    Commented Sep 8, 2017 at 13:09
0
$\begingroup$

Why not just try a logistic regression? Just think what the presence of a disaster might affect. An obvious guess would be change in raw scores between before-disaster and after-disaster years. As you don't know the exact date of the disaster you will probably just have to take the absolute change from grade 1 to grade 6 as this new factor and treat it as your independent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.