# Half-normal, discrete data - What statistical test should I use?

I have difficulties identifying the correct statistical test for the following. My data show a half-normal distribution, so the histogram would look something like this:

X=0: 200 Y data points
X=1: 150 Y data points
X=2: 80 Y data points
X=3: 40 Y data points
X=4: 30 Y data points
X=5: 20 Y data points
X=6: 10 Y data points
X=7: 5 Y data points
X=8: 3 Y data points
X=9: 2 Y data points
X=10: 0 Y data point
X=11: 1 Y data point
X=12: 0 Y data point
X=13: 0 Y data point
X=14: 0 Y data point
X=15: 1 Y data point

Note 1: X is a positive integer, so discrete/not continuous.
Note 2: Y data points are normally distributed for each X and continuous.


I have multiple datasets for Y, so you could say I have a dataset Y1, Y2, … Yi. For each Y1, Y2, …Yi, X is distributed exactly as above and only all data points in Y1, Y2, … Yi are different.

Now, I am interested in knowing whether which Y (Y1, Y2, or … Yi) best correlates with the data. I can create a bar graph for each Yi and then do a Spearman or Pearson linear regression analysis to calculate Spearman R and p-value or Pearson R and p-value. Then the Yi that has the largest R and the smallest p-value would best fit the data.

Question: Is this correct at all, since this involves discrete and continuous data, which are half-normally and normally distributed, respectively? This is not a scatter plot type of data, so I have doubts about whether Spearman/Pearson correlation is the best here.

I would appreciate any tips for the best test here.

Pearson correlation is equivalent to $R^2$ in classical OLS linear regression. OLS, in turn, is based solely on two assumptions (see Gauss-Markov theorem):

• distribution of $Y$ conditional on any $X$ is $\mathcal{N}(\mu(X), \sigma^2)$, where all observations are conditionally independent given $X$, and have the same $\sigma$.
• $\mu(X)$ is a linear function of $X$.

Please note that these assumptions do not involve distribution of $X$: it doesn't have to be a random variable at all!

The first assumption seems to be satisfied in your case: data points are normaly distributed for each $X$. You can test whether their variance is also equal for each $X$, and if it is not the case, look for heteroschedasticity-compatible methods.

And if it is the case, then OLS makes sense for your problem, and you can use $R^2$ as a goodness-of-fit statistic. If you suspect that relation between $Y$ and $X$ may be nonlinear, you can e.g. include polynomial terms.

• David, thank you so much for your help! My background is not in statistics, so I had to look up a few of your comments, but it is overall clear what you mean. I would like to ask you a follow-up question, please. You suggested that if the variance for each X is not equal (which is the case for my data), then "look for heteroschedasticity-compatible methods". Would you have a suggestion for which method would be best in this case, please? Thanks again - Peter. Feb 24 '18 at 11:45
• Peter, in fact, OLS and R^2 can still work in case of heteroschedasticity (although OLS estimates lose some of their nice properties). But you can also find a way either to rescale your data to make it homoscedastic, or assing different weights to observations (see WLS). Feb 24 '18 at 11:53
• Thanks again, David. That's very helpful. I will probably go for Spearman correlation tests, because I have thousands of Ys to test, so it will not be practical to, for each Y: (1) check how to best rescale the data and (2) check for the presence of outliers. So, I will prefer Spearman correlations over Pearson correlations. Many thanks! Feb 24 '18 at 13:24