0
$\begingroup$

I'm making a research and doing a final data analysis. The research was undertaken using a questionnaire and then structured interviews. Questions were basically the same. In the questionnaire I surveyed a theme and, using a Likert scale, I got a mean result of 4 (using a scale from 0 to 10). The same topic was studied using interviews with quantitative analysis. Here the result is 7.8. The questionnaire had 193 samples, interview only 4. The interview were done with the aim to acquire data to confirm findings. As the difference between both values is too high, results cannot be confirmed.

Can I use the standard deviation (calculated in 2.68) to affirm that, accepting a maximum deviation of 1, this theme cannot be established and needs more analysis?

Generalizing, the statistical question is if the standard deviation of samples can be used on two mean values acquired with different instruments on a different population.

$\endgroup$
1
$\begingroup$

Your data are drawn from two different distributions. You'd need to first establish some kind of relationship or correlation between the two distributions, before being able to transfer knowledge found in one distribution to the other, I think.

Intuitively, imagine you look at the price of doughnuts in country X, this might be only weakly correlated with the price of doughnuts in country Y, eg because cost of living etc is different. But if you looked at the ratio of doughtnut price to milk price between the two countries, you might find ok correlation. Then, knowing the ratio of eg hamburger price to milk price in one country might tell you something about the ratio of hamburger price to milk price in another. But you'd probably want to correlate a bunch of such ratios first, using training/validation sets of ratios, before predicting the ratio for your test set.

$\endgroup$
0
$\begingroup$

Comparing the sample size in both the methods, you can put a lot of confidence in the first test as compared to the second. Therefore you don't need to confirm your finding after first test and you certainly cannot deny the first test based on second.

That applies perfectly if the tests are supposed to bring out same responses. And statistically, both the tests would converge to same distribution if sample size is large for both. And hence you can compare their SDs.

But in a real life scenario, you have to make the decision whether you're confident about the correctness of response in either of the tests, that is, the results of the test represent actual results or not. It seems that you aren't with the questionnaire, and 7 is a very small number if you haven't explicitly tried to best approximate the true population in those 7. So I guess you either need to trust the questionnaire or do more interviews to conclude on results

$\endgroup$
  • $\begingroup$ Maybe people are more truthful on questionnaires and less on personal interviews in order to avoid conflicts. Other way round could be that people are more serious in interviews rather than in a questionnaire where you can literally answer randomly. Its a tricky thing $\endgroup$ – Ujjwal Kumar Jan 14 '16 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.