# Interpreting t-value as a measure of evidence

1. Came across an article on the web on "evaluation of training effectiveness". The author suggests that the "t-value" obtained from a "Paired t-test" conducted using pre-test and post test scores can be used to quantify the effectiveness of training.
2. The t-value was termed as the "Index of Learning" and the author claimed the following:
a. An "Index of learning" (t-value) between 0 to 1.5 indicates no evidence of learning.
b. A t-value between 1.5 to 2.0 indicates some evidence of learning.
c. A t-value between 2 to 3 indicates strong evidence of learning.
d. A t-value above 3 indicates very strong strong evidence of learning.
3. In my limited understanding of the subject, the t-value, critical t-value and p-value of a t-test would depend on the significance level and degrees of freedom of the associated data. I have thus not understood the rationale behind the generalization made by the author (Para 2 (a) to 2 (c) above). I would be extremely grateful for any explanations. Thanks.
-

The difficulty here appears to stem from terminology and convention.

First let's just define terms in terms of statistical theory. We observe a data $X_1 ... X_n$, each a random variable. We define a statistic as a function of that data, $S(X_1...X_n)$. As a function of random variables, the statistic is also a random variable. So far this constitutes the estimation part of statistics.

Certain statistic functions are better suited for detecting certain characteristics. In the case of the paired t-value, a statistic, it is designed to detect differences in the mean between 2 populations. But how do we know when $S$ is "abnormal", or "substantially different"? Here like most other things in like we go to some well-defined set of reference values, and compare our obtained statistic with the critical statistic. This part constitutes the inference part of statistics.

In this case, if we make the assumption that the data is independent, normally distributed, then we can go to the t-value table to look up the corresponding critical t-value. If the t-value is more extreme than the critical value then we can call it statistically significant, and vice versa.

This division between estimation and inference is the difference.

The paper you referred to appeared more interested in the estimation of differences in learning. It may be of interest to make the determination--the inference--of whether of not the learning was statistically significant. But that means you end up with a binary determination of "yes, significant" or "no, not significant". Letting the estimation--the t-values--be the measurement allows the user to quantitatively compare paired units.

The same thing is done for estimating, for example, bone density. The measured bone density (DEXA), a number, is converted into a t-value based on age- and ethnicity-controlled reference population. Wikipedia article. The t-value is then used to make the diagnosis of osteoporosis (t<-2.5) versus osteopenia (t between -2.5 and -1)

-
Dear Mr Ming, I am grateful for the trouble taken by you to post a detailed explanation. Am a little confused though. Checked up on some websites regarding Bone Density. The t-values specified in that context are in relation to standard, predefined bone density figures. Comparative t-values in terms of Std Deviations would thus make sense. In my opinion, it may not be proper to adopt a similar analogy in case of paired t-test used to evaluate pre/ post test scores simply because there are no standard/predefined figures to compare with. What do you think? Thanks anyway. –  user13226 Aug 11 '12 at 18:22
Got it. I'm of the opinion that the author is trying to quantify the degree of learning (hence the nomenclature of "Index of Learning"), and not so much the weight of evidence. But I may be wrong, in which case Dr. Chernick's answers will be more relevant. My argument is that whether 1-sample t, 1-sample z, 2-sample t, or whatever statistic one comes up with, it is a standardized score that may be helpful in quantifying size of effect versus strength of evidence (which appears to be the gist here). Hope this helps. –  Ming-Chih Kao Aug 12 '12 at 3:33