Significance test for longer pronounciation

Question

I have a set of words spoken by a group of native speakers and a group of non-native speakers. In each record I determined the length of the burst (a phonetic term). I believe non-native speakers tend to have longer burst-times. However, the burst-times vary with word, no matter the native language, so I cannot just perform a t-test on the absolute length in ms. Instead, for each word I calculated the ratio of the (mean)length $A_i$ of the native speakers and the (mean)length $B_i$ of the non-native speakers and I want to perform a one-sided one-sample t-test on the nullhypothesis that this ratio is 1.

Is this statistically adequate?

I'm afraid there is something wrong with it, because with one word my native-speakers have 0 burst-time so the quotient becomes infinity. Also I suspetct the outcome of the test can depend (not in my case though) on how I take the quotient ($A_i/B_i\leftrightarrow B_i/A_i$), which it shouldn't.

Answer 1

Is this statistically adequate?

Well, it leaves a lot on the table and may be limiting your options. I think to get a better analysis you might want to consider a richer framework than the $t$-test.

How I am interpreting your study is that you have a sample of native and of non-native speakers. Probably, you would like to make general statements about some population of native and non-native speakers, so hopefully these samples are at least reasonably representative.

Also, you have a sample of words from the language. Again, you probably want to generalize to the population of words in the language. Again, in that case, hopefully these words could be considered at least to be representative of such words.

My recommendation would be to consider the linear mixed effect model framework. A very minimal model for this situation would be to consider fluency status (native, non-native) as a fixed effect, speakers as random effects, and words as random effects. Speakers are nested in fluency status.

Alternatively, you may have a set of particular words of interest or a complete set of words, in which case you would include words as a fixed effect in the model.

More complex models would consider potential interactions between fluency status and word or speakers and words. You may also have more than one measurement of burst for each word for each speaker.

To clarify what I mean, here is a simulated example with a very simplistic analysis using R:

#-------------------------------------------------
# Make something interesting to analyze.

Data <- data.frame(
  Status  = rep(c("Native", "Non-native"), each=50),
  Speaker = rep(1:20, each=5),
  Word    = rep(c("W1", "W2", "W3", "W4", "W5"), 20),
  Burst   = rep(0, 100)
)

#   Overall average burst around 20
Data$Burst <- 20

#   Speakers have variation in burst with SD=2
#   Native speakers have lower burst by 3
Data$Burst[ 1:50] <-  Data$Burst[ 1:50]  + rep(rnorm(10, 0, 2), each=5)
Data$Burst[51:100] <- Data$Burst[51:100] + rep(rnorm(10, 3, 2), each=5)

#   Words have variation in burst with SD=1
Data$Burst <- Data$Burst + rep(rnorm(5, 0, 1), 20)

#   Measurement error has SD=0.5
Data$Burst <- Data$Burst + rnorm(100, 0, 0.5)

The data look like this:

> head(Data)
  Status Speaker Word    Burst
1 Native       1   W1 22.05368
2 Native       1   W2 23.02794
3 Native       1   W3 22.59183
4 Native       1   W4 21.65342
5 Native       1   W5 23.77227
6 Native       2   W1 18.30863

You can conduct a simplistic analysis like this:

#-------------------------------------------------
# Set up the analysis.

# Plot the data.
library(ggplot2)
p <- ggplot(Data, aes(x=Word, Burst, col=Status)) + geom_point()
print(p)

# Analyze the data.
library(lme4)
library(car)

fit <- lmer(Burst ~ Status + (1|Speaker) + (1|Word), data=Data)
plot(fit)
summary(fit)
Anova(fit)

Here are extracts of some results from the above:

Random effects:
 Groups   Name        Variance Std.Dev.
 Speaker  (Intercept) 6.7674   2.6014  
 Word     (Intercept) 0.5543   0.7445  
 Residual             0.2986   0.5464  
Number of obs: 100, groups:  Speaker, 20; Word, 5

Fixed effects:
                 Estimate Std. Error t value
(Intercept)       20.2469     0.8908  22.728
StatusNon-native   3.5044     1.1685   2.999

Here is an analysis of variance table:

Analysis of Deviance Table (Type II Wald chisquare tests)

Response: Burst
        Chisq Df Pr(>Chisq)   
Status 8.9941  1   0.002709 **
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

So, with this you would conclude that there is a difference in burst between native and non-native speakers across words of about 3.5.

Note that this is just an example to show the general idea. Your specific study is bound to have more interesting details and you probably have other questions that you are interested in answering. The distribution of your data may create issues --- for example, if there are many zeroes then you may need some other approach, such as using a generalized linear mixed effect model.

Also, depending on the situation, you may want to use the nlme package rather than the lme4 package. You could implement this analysis in SAS using PROC MIXED, in SPSS, or in other statistical packages. With the richer framework you do get some trade-off in complexity of implementation and interpretation!

The upside of this modeling framework is that it allows you to draw more interesting conclusions. For example, you can discuss the relative contribution to variation of speakers, words, and measurement error. You can provide quantitative summaries of interesting effects such as difference between native and non-native burst rates in the population. You can incorporate as covariates other linguistic or socio-linguistic variables such as preceding syllables, phrase length, speed, sex, age, or whatever --- of course, depending on how much data you have!