0
$\begingroup$

I have data where for a specific threshold, I calculate what percentage of values meet a certain criteria. I want to calculate whether there is a statistically significant trend when I increase the threshold and there is a corresponding increase in the response variable. To give an example, I have the following data for one instance

x_threshold = [1,2,5,10,15,20,25,30,35,40]
y_percentage = [47.90046656,53.894081,58.47457627,57.14285714,57.77777778, 
               57.89473684,59.375,63.15789474,68.75,69.23076923,72.72727273, 
               80]

At first, I tried the simple Linear Regression which did give a significant slope for this instance. However I am not convinced whether this is the right model to apply since the variables are not exactly continuous. Also note that at each threshold, the sample size n is different and % of n which is meeting a certain criteria is the response variable.

Can an appropriate model be suggested where this kind of data can be fit?

Improve this question
$\endgroup$
0
$\begingroup$

For cases where the response variable is ordinal, you can explore the following options.

  1. Generalized Additive Model : represent the ordinal variable using polynomial splines.

Reference : https://statistics.stanford.edu/research/estimating-functions-mixed-ordinal-and-categorical-variables-using-adaptive-splines

  1. Penalized Regression : Fit a dummy variable regression along with the original vector of predictors. Then, penalize the difference between the coefficients of adjacent categories to ensure smoothness in transition between two adjacent ordinal values.

Reference: https://epub.ub.uni-muenchen.de/2100/1/tr015.pdf

  1. Polynomial contrasts : using a combination of dummy variables and orthogonal polynomial functions of your predictor variable. e.g. $X$,$X^2$ ... $X^n$. This will capture the overall effect that each ordinal value has in your regression.

There are some great SE threads that discuss regression with ordinal predictors and other great research articles available online.

(More links in comments)

Improve this answer
$\endgroup$
  • $\begingroup$ 1. Reference: springer.com/us/book/9780387951874 2. cran.r-project.org/web/packages/ordPens 3. stats.stackexchange.com/questions/33413/… 4. stats.stackexchange.com/questions/195246/… $\endgroup$ – SidV Sep 20 '17 at 2:37
  • $\begingroup$ Thanks Sid for the references. My predictor variable is ordinal and I think smoothing of ordinal independent variables is a good idea. However my response variable is percentage and hence bounded between 0 and 100. Based on reading other threads, I am using beta regression since it models the response variable as continuous proportions which I can convert my percentages into. Let me know if you think this is not appropriate $\endgroup$ – Nabeel Ahmed Sep 20 '17 at 13:31
  • $\begingroup$ I agree. I was focused on addressing the ordinality of your covariates. Beta regression is very flexible and works fine. If you know a priori that these percentages are concentrated in certain subspaces of [0,1], then you can choose other link functions e.g. logit, zero-inflated-poisson, etc. $\endgroup$ – SidV Sep 22 '17 at 22:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.