0
$\begingroup$

I have the following data:

Class       Branch  LA_type Method_type     Method_call     Branch_type     Branch_condition    Tested_parameter
Goal        12      Smooth  public static   never called    IFNE            TRUE                 String
TreeApp     20      Rugged  constructor     none            IF_ICMPGE  FALSE                     int
Password    4       Smooth  private         never called    IFEQ    FALSE                        int
XMLParser   9       Rugged  constructor     none            IFNONNULL   TRUE                     String
MapClass    33      Smooth  public          never called    IFGT    FALSE                        double

I would like to know whether there is a relationship between the LA_type variable and each of the Method_type, Method_call, Branch_type, Branch_condition, or Tested_parameter variables. I applied the Chi Square test with LA_type and each of the five variables and I got the following results:

Variable            p-value     X-squared
Method_type         2.03E-15    125.86
Method_call         0.0009524   34.667
Branch_type         2.15E-08    96.535
Branch_condition    4.65E-08    33.767
Tested_parameter    1.83E-08    56.263

Besides I got a message in R that says In chisq.test(t) : Chi-squared approximation may be incorrect which indicates that the sample is small (i.e., total of 140 records). For that, I used the Fisher's exact test that gave me the following results:

Variable            p-value     X-squared
Method_type         0.0004998   125.86
Method_call         0.0004998   34.667
Branch_type         0.0004998   96.535
Branch_condition    0.0004998   33.767
Tested_parameter    0.0004998   56.263

I am not really confident about the results of Fisher's exact test's p-values. By looking at the p-values, can I say there is a relationship between the LA_type and each of the five variables? In other words, is the LA_type and each of the five variables dependent? In addition, does the higher X-squared value indicate the the two variables are related? For example, the higher value of X-squared (125.86) indicates the Method_type is the most related variable with LA_type. Is that correct?

The observed and expected values are shown in this for LA_type and Method_type

$\endgroup$
13
  • $\begingroup$ What do your observed and expected values look like? $\endgroup$ Oct 17 '19 at 12:05
  • 1
    $\begingroup$ @william3031 I updated the question and added a link to the CSV file that shown the observed and expected values. $\endgroup$
    – Adam Amin
    Oct 17 '19 at 12:17
  • $\begingroup$ Fisher's exact test is usually used for 2x2 tables. Your csv file looks a bit messy. Are the column headings correct? $\endgroup$ Oct 17 '19 at 12:25
  • $\begingroup$ @william3031 The column headers are the values of the Method_type $\endgroup$
    – Adam Amin
    Oct 17 '19 at 12:26
  • 1
    $\begingroup$ Note that 0.0004998 merely represents $1/(2000+1).$ Type ?chisq.test for information on where the $2000$ comes from and how to change it. $\endgroup$
    – whuber
    Oct 17 '19 at 13:45
1
$\begingroup$

In statistics, hypothesis testing takes one null hypothesis, considered to be true until proven otherwise, and you see if data show evidence of that hypothesis being false. The p-values you got show that exact kind of evidence. Yes, it is quite clear that those features are related, all of them, apparently. Is it a surprise?

About how to tell what variables are most related, chi-square value is not only a test statistic, but also a widely used measure of association between categoric variables. Another option is Cramer's phi, which is basically a normalized version of chi-square statistics, if you have variables with different numbers of levels, you should use Cramer's phi to compare associations (and to see which are the strongest). I don't know any function for computing phi on base r, but there are many in other packages.

$\endgroup$
2
  • $\begingroup$ Thanks for the explanation. When you mentioned The p-values you got show that exact kind of evidence, is this based on chi-square p-values or the fisher exact test? I'm still not sure whether I can rely on the chi-square test since I only have 140 observations. $\endgroup$
    – Adam Amin
    Oct 17 '19 at 14:43
  • $\begingroup$ We statisticians are used to make the strongest assertions with even more little samples. Related variables are not uncommon at all, they are the norm instead, so why you bother so much? If your sample is representative of your population of interest, even if it is little, it provides a strong evidence for those associations to exist. Those two sets of p-values are each very little, so, even if they are not precise, evidence is still very strong. $\endgroup$
    – carlo
    Oct 17 '19 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.