1
$\begingroup$

I've recently conducted a survey of students in a class who had recently taken a difficult examination. Part of the output data is like so:

  • x_Jan=Hours spent studying weekly in Jan [0 ... n hrs/wk]
  • x_Feb=Hours spent studying weekly in Feb [0 ... n hrs/wk]
  • (and so on to x_Dec)
  • y=Exam score

Here's an example month: January

How can I analyze this data to know what amount of time spent studying weekly per month is best correlated with examination outcomes? For the sake of simplicity, I think the question & answer can be simplified to a single month (x), and the solution to multiple months can be easily extrapolated.

On the surface, one might guess those who spend the most time studying perform the best, but, looking at the data qualitatively, it is evident that it isn't quite the case--for example, perhaps some students spend too much time studying too early, or too much time studying too late. I hope to use the data to help students prepare an effective study plan and schedule.

I considered doing linear regression comparing 0 hours to each of X hours vs test score, but sample size quickly becomes a limiting factor. I'd have to bin the data fairly wide to get any significance. (i.e. 0 to 20, 21 to 40, etc.).

Any suggestions? I appreciate your insight!

$\endgroup$
6
  • 1
    $\begingroup$ Are you able to add any predictor of academic performance to your study? I ask as the results would be affected by both intelligence (somewhat measurable) and academic laziness (not so measurable), with a proxy somewhat combining both factors being the individual's past academic performance. $\endgroup$ – James Phillips Nov 3 '18 at 10:00
  • $\begingroup$ Yes. I plan to stratify the results by other factors, such as pre-test GPA, as well. Might dig out some effect modifiers / confounders. $\endgroup$ – aliigleed Nov 3 '18 at 11:06
  • $\begingroup$ It seems that exam score is a continue variable, so fit a linear model, and try to select useful covariates and build a final model. $\endgroup$ – user158565 Nov 3 '18 at 19:20
  • $\begingroup$ @a_statistician I can't say for certain whether there's a linear relationship, the plot in the OP is just a piece of a larger dataset. I'd like to leave room for other possibilities, for example, that those who study most may perform poorer than those who study a moderate amount. Also, which component of the linear model would answer the question? It could help determine whether there is a linear relationship between hours studied and exam outcomes, but I'm not sure whether it could say which number of hours studied is most correlated with exam outcome unless you apply wide bins to the hours. $\endgroup$ – aliigleed Nov 13 '18 at 1:30
  • 1
    $\begingroup$ Linear model does not mean linear relationship. $Y=\beta_0+\beta_1 X_1 +beta_2 X_2...\beta_kX_k +\epsilon$, $X$s can be anything you can derived from the variables (excerpt Y), such as hours spending on the study. They can be square of the hours, spending less than 4 hours, spending 4 or more house, average of hours 3 months before last exam ... $\endgroup$ – user158565 Nov 13 '18 at 2:04
1
$\begingroup$

You are right that the data are best combined into a single covariate, namely hours. There is no reason to believe that studying for 2 hours in January is any different than studying for 2 hours in February, provided the hours studied are for the same test. I'm assuming here that this is the only test the students are taking (e.g. no tests in other courses) which is obviously false, but without that data it is something we have to live with.

Linear regression is not a good idea. Your model will tell you that studying sufficiently long will guarantee you a perfect score or higher, which is ridiculous.

One method may be to transform the scores to be between 0 and 1 and then do a logistic or beta regression. That way, your predicted mean would be constrained to be within the unit interval and you will avoid problems like the one I have mentioned earlier.

I would caution you from relying too heavily on the inferences made from whatever model you construct. There is a great deal of between subject variability in this data, so just because your model predicts that studying 4 hours will result in an A on average does not mean that you, or anyone else, are guaranteed an A because you sit starting at your books mindlessly for 4 hours.

$\endgroup$
3
  • $\begingroup$ Thank you for your thoughtful reply. The case here is somewhat exceptional in that the exam is cumulative over all subjects over multiple years for students enrolled in identical courses/schedules, so variability in studying for other courses should play less of a role. I acknowledge making any defensible inferences will be terribly difficult (and probably inappropriate) given the number of other confounding factors, but the statistical challenge is appealing and I'd like to see if anything interesting can be gleaned from this portion of the survey. I'll give logistic/beta regression a shot. $\endgroup$ – aliigleed Nov 14 '18 at 4:01
  • $\begingroup$ Also, there is some reasoning behind keeping hours separated by month. Studying 40 hrs/wk in January vs. February may not be too different, but could have significantly different weight on exam outcomes compared to 40 hrs/wk in May for a June exam. If all I get in the end is some data to support even broad suggestions, like "Excess early prep isn't associated with exam success," "Cramming 1-2 months before the exam is associated with poorer outcomes compared to regular study over 3-4 months," etc. I'll be content. $\endgroup$ – aliigleed Nov 14 '18 at 4:16
  • $\begingroup$ @aliigleed Sounds like you are well informed about your limitations and are prepared to begin modelling. $\endgroup$ – Demetri Pananos Nov 14 '18 at 4:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.