3
$\begingroup$

I'm doing research into the relationships between physical features of audio signals and the perceptual responses to these signals. For example, the data below contains the "Naturalness of Timbre" ratings reported by three subjects in response to four different audio signals.

data <- data.frame(
  Signal=c('S1', 'S1', 'S1', 'S1', 'S1', 'S1', 'S1', 'S1', 'S1', 'S2','S2', 'S2', 'S2', 'S2', 'S2', 'S2', 'S2', 'S2', 'S3', 'S3', 'S3', 'S3', 'S3', 'S3', 'S3', 'S3', 'S3', 'S4', 'S4', 'S4','S4', 'S4', 'S4', 'S4', 'S4', 'S4'),
  NaturalnessOfTimbre=c(0.78338906,0.88641009,1.06669688,0.95402992,0.90072169,0.99965679,1.04912434,0.95402992,0.95402992,0.52583649,0.80914432,0.90072169,0.85125413,0.98943111,0.68608960,0.71044781,0.75231903,0.73480602,-0.24682121,-0.50910356,-0.32408698,-0.11804493,0.02841791,-0.68224000,-0.33596713,-0.28823883,-0.28823883,-1.48307353,-1.43156302,-1.35005196,-1.22638307,-1.35005196,-1.45731828,-1.48179115,-1.48179115,-1.48179115),
  SpectralSlope=c(11.35967,11.35967,11.35967,11.35967,11.35967,11.35967,11.35967,11.35967,11.35967,11.38028,11.38028,11.38028,11.38028,11.38028,11.38028,11.38028,11.38028,11.38028,11.32053,11.32053,11.32053,11.32053,11.32053,11.32053,11.32053,11.32053,11.32053,11.08847,11.08847,11.08847,11.08847,11.08847,11.08847,11.08847,11.08847,11.08847),
  Repetition=c(1,2,3,1,1,2,3,2,3,1,2,1,2,3,1,2,3,3,1,1,2,3,1,2,3,2,3,1,2,1,2,3,3,1,2,3),
  Subject=c(1,1,1,3,2,2,2,3,3,1,1,2,2,1,3,3,2,3,1,2,1,1,3,2,2,3,3,1,1,2,2,2,1,3,3,3)
)

ggplot(data,aes_string(y='NaturalnessOfTimbre',x='SpectralSlope')) + geom_point(aes(shape=Signal,colour=Signal)) + geom_smooth(method=lm) 

enter image description here

I'd like to report on the relationship between the feature "SpectralSlope" and the reported "Naturalness" values. I'm considering doing this using a simple linear regression model. This gives me a $\beta_1$ estimate of $7.5563$ and a very low p value ($p < 0.001$).

summary(lm("NaturalnessOfTimbre~SpectralSlope",data))

My question: in this case, is a simple linear regression, as above, the appropriate way to investigate the relationship? Is the analysis I've done reasonable? I'm concerned because most textbook examples that I see of linear regression have evenly spaced explanatory variables (e.g. SpecSlope of [1,2,3,4]), rather than unevenly spaced ones. Is this method still applicable when the variables aren't evenly spaced, as in my data?

$\endgroup$
8
  • 2
    $\begingroup$ My thought: The variables don't need to be evenly spaced. But having few values on the x-axis combined with the gap between them does make the linear relationship less certain. For example, the y-values of data corresponding to SpecSlope = 11.25 could suggest a different kind of curve. $\endgroup$ Nov 27 '17 at 20:01
  • 2
    $\begingroup$ My understanding is that the spacing itself is not a problem when fitting data to a straight line, however if the number of data points in each category is not equal you will be effectively weighting the regression towards the group with the most data points - in that case using the average values per category rather than directly fitting the the raw data is worth considering. $\endgroup$ Nov 27 '17 at 20:02
  • 2
    $\begingroup$ You should look at diagnostic plots, such as the "Cook's distance" and "Residuals vs Leverage" plots provided by plot.lm. The easiest solution to the uneven spacing is a simple transformation of your predictor, but of course this changes the model and should be based on science. You could also do weighted regression. There are other issues to consider too: You are not including the subjects in your model (I suggest a mixed effects model). Also, how are the "NaturalnessOfTimbre" values created? $\endgroup$
    – Roland
    Nov 28 '17 at 9:14
  • 1
    $\begingroup$ The main issue is not the unequal spacing in itself but the fact that S4 lies well separated from the others and so your line is almost forced to go close to the S4 points. If you follow @Roland suggestion of examining the diagnostic plots you may get some more insight into this. $\endgroup$
    – mdewey
    Nov 28 '17 at 17:09
  • $\begingroup$ The main question is whether linear regression is appropriate for something like "naturalness"? Spectral slope variable is an objective measurable quantity. Naturalness is some phsycho-physio metric, subjective too, that doesn't necessarily have to be linear to anything. $\endgroup$
    – Aksakal
    Nov 29 '17 at 20:00
2
+50
$\begingroup$

Ultimately, the best approach will depend on whether the OP has more interest in the slope of the line or a difference in group means of NaturalnessOfTimbre.

Regarding analysis of SpecSlopes as groups: I'm uncomfortable transforming a measurement derived from a continuous signal into unordered groups for the purpose of conducting an ANOVA analysis. Doing so obliterates your ability to comment on correlation between SpecSlope and NaturalnessOfTimbre- you will produce only a F-statistic- and lose the ability to comment on how much a 1 unit change in SpecSlope affects NaturalnessOfTimbre. Creating four ranks for SpectralSlopes preserves the ranks of the data better than unordered groups, but ignores the distance between groups. I vote to keep SpecSlope in its natural form and use a linear model, unless the SpecSlope value is immaterial to values of NaturalnessOfTimbre.

The OP has already demonstrated the effects of a linear model and estimated a 7.5563unit increase in NaturalnessOfTimbre for each 1 unit increase in SpecSlope. However, this ignores that NaturalnessOfTimbre values are not independently generated. These values are generated by subjects and each subject generates 3 values per exposure to a given SpecSlope. While the number of measurements may not be enough to create a difference in the final standard errors, ignoring the correlation seems untenable.

The naive model:

summary(lm(NaturalnessOfTimbre ~ SpectralSlope, data = df)) 

Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) -85.2902 6.1689 -13.83 1.62e-15 *** SpectralSlope 7.5563 0.5465 13.83 1.62e-15 ***

The Subject-level repeated measures model:

summary(lmer(NaturalnessOfTimbre ~ SpectralSlope + (1|Subject), data=df))

Fixed effects: Estimate Std. Error t value (Intercept) -85.2902 6.1689 -13.83 SpectralSlope 7.5563 0.5465 13.83

Lastly, a model of repeated measures for each Subject for each Repetition:

summary(lmer(NaturalnessOfTimbre ~ SpectralSlope + (Repetition|Subject), data=df))

Fixed effects: Estimate Std. Error t value (Intercept) -85.2902 6.1689 -13.83 SpectralSlope 7.5563 0.5465 13.83

While not the best object lesson in repeated measures, the last approach incorporates all relevant aspects of the data in its native form. If I were reviewing this as a publication, I would ask for more experiments and a repeated measures approach at least at the level of Subject.

$\endgroup$
2
$\begingroup$

Based on your description, I would recommend using ANOVA instead of Simple Linear Regression to investigate this relationship. It looks like the SpectralSlope is more-or-less a proxy for Signal. You can use Signal as your 'group' variable, and NaturalnessOfTimbre as the 'continuous outcome' measure. The null hypothesis is that the mean values of NaturalnessOfTimbre is same across all groups of Signal.

For the Simple Linear Regression approach, since the independent variable is unevenly spaces, the regression slope is misleading in that it would seem to suggest that one point increase in SpectralSlope would result in 7.5563 increase in NaturalnessOfTimbre. This is problematic, because (1) this goes beyond the modeling space (i.e., range of values available for training the model), and (2) there's no way to know whether that relationship holds when SpectralSlope has values in the 11.2 neighborhood.

You'd be better off studying how the outcome differs (or does not differ) across the four levels of Signal. In doing so, you'd be ignoring the fluctuations in SpectralSlope within each Signal group -- but looking at the actual values, these differences are either non-existent or in the fourth decimal point. Those small differences may not even be meaningful in practice. (Could these be measurement errors?)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.