0
$\begingroup$

I've got children's questionnaire scores (mean of 6 items) regarding a particular variable (here: how well the child likes to interact with technology), and a score of the same child by a parent (they used a different questionnaire, also mean of a couple of items regarding how well their -- this -- child likes to interact with technology).

I want to find out whether their judgments differ, i.e., whether (H0) children's and parent's judgments are the same vs (H1) whether they do differ (and if so, whether the parents over- or underestimate their children's desire to interact with technology).

Normally (if the assumptions hold) I would use a t-Test for dependent samples, giving that parent and children are linked and judge the same thing (how well these particular child interacts with technology, self-rating vs. parent-rating). But the children's questionnaire has an answer scale of 1 to 4 (more or less: strongly disagree, disagree, agree, strongly agree), while the parent's questionnaire has an answer scale of 1 to 6 (strongly disagree, disagree, slightly disagree, slightly agree, agree, strongly agree).

So just calculating a t-Test for dependent samples would be a bad idea.

My more or less intuitive first idea would be to calculate a t-Test for dependent samples but with z-standardized scores. Just use a z-Transformation on the child scale and parent scale and then calculate the t-Test. Alternatively perhaps to stretch or shorten the answer scale.

But I'm confused here, so that's why I ask the following questions:

  1. Can I calculate a t-Test for dependent samples with z-standardized scores this way or am I missing something?

  2. Is there a better way to see whether the judgments differ?

  3. Anything else I have missed (like I said, currently a bit confused)?

$\endgroup$
1
$\begingroup$

You ask students if they like apples and grapes in school lunches. You ask parents if they think students like bananas and oranges in school lunches. Can the answers help you decide whether parents and the student children are on the same page about the desirability of fruit for lunch?

Substantive (non-numerical) difficulties abound for this project.

  • First, you will have to look at the questions globally to make sure the same issues are addressed with about equal emphasis on the student and parent questionnaires. (Are the kinds of technology the same? Are envisioned degrees/kinds of interaction the same?)

  • Second, you will have to look at the questions individually to make sure that "Agreement" always indicates enthusiasm for interaction with technology. (Sometimes questionnaires try to test whether respondents are equally likely to say they Agree and to say they Disagree by reversing the sense of a few questions.)

  • Third, if the first two issues can be favorably resolved, you will have to find a way to make the 4-point and 6-point Likert scales agree. Issues about the comparability of 6-point and 4-point Likert scales are discussed below.

You ask only about the third part. I don't know exactly what you mean by "z-standardized scores," but if you mean to subtract the sample mean and divide the difference by the sample standard deviation, then that would clearly be counterproductive because both sets of scores will be centered at $0,$ with no hope that a test will find a difference.

  • The simplest and perhaps least controversial method of making responses similar for the two groups would be to decide if each answer is favorable 1 (or not 0) as to the child's interest (or guessed interest) in interaction with technology.

In my experience it is difficult to convert from one Likert scale to another, while keeping essential information intact. Here are some things you might try:

  • If the 6-point scale has lots of 1s and 6s and the 4-point scale has lots of 1s and 4s, then it may be OK to reduce the 6-point scale to a 4-point scale by mapping 6s into 4s and collapsing 2-3 to 2 and 4-5 to 3.

  • If there are relatively few extreme scores on both questionnaires, then it may be best to map 1-2 onto 1 and 5-6 onto 4.

  • You might try make the 6-point scale "match" a 4-point scale by using values something like 1 1.5 2 3 3.5 4 instead of 1 2 3 4 5 6. But once differences are found for a paired test, this scheme still might give the (modified) 6-point scale an 'edge' in determining ranks in a Wilcoxon (signed-rank) test. (There would be no corresponding difficulty for t tests, if used.) Also, there is controversy whether Likert scores should be taken as interval numerical data instead of the ordinal categories that they actually are. This scheme goes far in the direction of deciding in favor of numerical data.

Experiments with simulated data. You don't say how many parent-student pairs of subjects you have. I will assume 100 pairs here. Simulated scores might be as summarized and plotted below. [Summaries, plots, and tests were done in R.]

st = rbinom(100, 3, .6)+1
table(st)
st
 1  2  3  4 
 6 30 48 16 
table(pr)
pr
 1  2  3  4  5  6 
 3  4 21 18 23 31 

cor(st,pr)
[1] 0.7339378

set.seed(617)
stj = st+runif(100, -.2,.2)
prj = pr+runif(100, -.2,.2)
plot(prj, stj, ylab="Student", xlab="Parent", pch=20)
 abline(v=1:6, col="green2");  abline(h=1:4, col="green2")

enter image description here

In the scatterplot, integer values have been slightly 'jittered' (randomly displaced) to avoid overplotting at a few grid points.

Students and Parents both have 'Successes' and 'Failures'. Reducing the data to 1 = Favorable, 0 = Not, and then looking at a test of binomial proportions:

st.01 = st
st.01[st <= 2] = 0;  st.01[st >= 3] = 1
table(st.01)
st.01
  0  1 
 36 64 

pr.01 = pr
pr.01[pr <= 3] = 0;  pr.01[pr >= 4] = 1
table(pr.01)
pr.01
 0  1 
28 72  

sum(st.01==0 & pr.01==0)
[1] 24

A contingency table with rows counting Students Less (0) and More (1) inclined toward interacting with technology and columns counting Parents' assessments of their children as being Less (0) and More (1) inclined is as follows.

TAB = rbind(c(24,12), c(4,60));  TAB
     [,1] [,2]
[1,]   24   12
[2,]    4   60
rowSums(TAB);  colSums(TAB)
[1] 36 64
[1] 28 72

A chi-squared test for independence of responses between students and parents strongly rejects the null hypothesis of independence.

chisq.test(TAB)

        Pearson's Chi-squared test 
     with Yates' continuity correction

data:  TAB
X-squared = 38.773, df = 1, p-value = 4.76e-10

For counts this large, a continuity correction may not be appropriate, but the null hypothesis is rejected either way.

chisq.test(TAB, cor=F)$p.val  # without continuity correction
[1] 1.055272e-10

Fisher's exact test for independence gives a similar result.

fisher.test(TAB)$p.val
[1] 1.642062e-10

Parents do suspect slightly more enthusiasm than students report, but of the $4 + 12 = 16$ instances of 'disagreement', it would not be significantly rare (5% level) to have a 4:12 or a 12:4 split or a stronger disagreement by chance alone.

2*pbinom(4, 16, .5)
[1] 0.07681274

Parent Scores between 1 and 4. It is possible that reducing the data to 0s and 1s has lost too much information about differences between parents and their children. Let's try the second scheme for changing the 6-point scale to a scale that runs from 1 to 4:

pr.4 = pr
pr.4[pr==2]=1.5;  pr.4[pr==3]=2;  pr.4[pr==4]=3
pr.4[pr==5]=3.5;  pr.4[pr==6]=4
table(pr.4)
pr.4
  1 1.5   2   3 3.5   4 
  3   4  21  18  23  31 

Here is a summary of the differences when student 4-point scores are subtracted for parent scores converted to values between 1 and 4.

d = pr.4 - st
summary(d)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 -1.000   0.000   0.000   0.355   1.000   2.000 
t.test(d)

For my simulated data, a paired t test overwhelmingly rejects the null hypothesis that Parents have a realistic view of the their children's enthusiasm for interacting with technology. They tend to overestimate it. (Maybe the enthusiasm for interacting with technology they perceive is sometimes just an excuse to get out of helping with the dishes.)

        One Sample t-test

data:  d
t = 5.7189, df = 99, p-value = 1.14e-07
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
 0.2318301 0.4781699
sample estimates:
mean of x 
    0.355 

Note: In case it is of interest, the method of simulating my artificial data is shown below.

set.seed(616)
st = rbinom(100, 3, .6)+1
table(st)
st
 1  2  3  4 
 6 30 48 16 
pr = rbinom(100, 5, st/4) + 1
table(pr)
pr
 1  2  3  4  5  6 
 3  4 21 18 23 31 
cor(st,pr)
[1] 0.7339378
$\endgroup$
2
  • $\begingroup$ Note: The original version of this answer posted yesterday had a mistake in a chi-squared test for data reduced to 0's and 1's; I have just now revised my answer to correct this mistake. $\endgroup$
    – BruceET
    Jun 17 '20 at 19:35
  • $\begingroup$ Kudos. And thank you :-) $\endgroup$ Jul 2 '20 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.