1
$\begingroup$

I got the following columns:

Level 1        240    12     7
Level 2        98     5      5
Level 3        46     4      6
Level 4        21     0      1

I try to prove that there is a correlation between the "Level" and the number of people (represented by each column) I used the following forumla (Table is the table..) :

RS2 = r2_score(Table.iloc[1:5,0], Table.iloc[1:5,1], Table.iloc[1:5,2])

print(RS2 )

The result is negative (-4....) which is wrong.

I am undecided, but maybe I should calculate the columns together as one group, finding mean and then the estimated values? if not, what should I do?

** note my data is small, concatenating the columns will probably give a large confidence interval..

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ I've taken the liberty of editing your title to be more informative. Take a look and see if that captures your main question, and feel free to improve on it by adding more specificity about your particular problem. $\endgroup$
    – Sycorax ♦
    Commented Dec 9, 2021 at 17:27
  • $\begingroup$ It does not make sense for there to be a negative correlation involving a nominal variable. Is there any order to the levels, or are they like "dog", "cat", "horse", "kangaroo"? // If you do want to find a negative correlation for variables where such a notion makes sense, $R^2$ is not a tool that will help you. That would be a correlation coefficient, which has a relationship to $R^2$ under certain circumstances. // Why do you want to show a negative correlation? $\endgroup$
    – Dave
    Commented Dec 9, 2021 at 17:33
  • $\begingroup$ I edit my question. I want to show that as long as the level increases - > the number of people declining $\endgroup$
    – TheUndecided
    Commented Dec 9, 2021 at 17:37
  • $\begingroup$ So there is an order to the levels? Do you know the difference between each of the levels? Is it constant in the sense that $L4 - L2 = L3 - L1$, etc? $\endgroup$
    – Dave
    Commented Dec 9, 2021 at 17:39
  • $\begingroup$ The relations expressed here through "rating" from 1 -> 4 , if certain people "pass" L1 they go to L2 and so on. My theory: L1 is easiest stage, there are more people, as long as you keep increasing the levels, it's becoming harder so then less people capable to be on an advanced level. $\endgroup$
    – TheUndecided
    Commented Dec 9, 2021 at 17:51

1 Answer 1

Reset to default
2
$\begingroup$

$R^2$ has nothing to do with the sign of a correlation. While there are ways of getting $R^2<0$ in a regression model (an indication of a poor fit), the notation comes from the fact that $R^2 = r^2$, where $r$ is the sample correlation between two variables, when you fit a regression model $\hat y_i = \hat\beta_0 + \hat\beta_1x_i$ with the extremely common method of least squares.

However, it looks like you have one variable with the levels in it and one variable with the numerical observations>

$$ X = (1,2,3,4,1,2,3,4,1,2,3,4)\\ Y = (240, 98, 46, 21, 12, 5, 4, 0, 7, 5, 6, 1) $$

Since your levels appear to be ordinal---that is, ordered but with unclear differences between them---Spearman's rank correlation is appropriate here. In R, the line is cor(x, y, method = "spearman").

This gives me a result of about $- 0.5$. However, the plot is not so convincing. When I do a test of the Spearman correlation being nonzero via cor.test(x, y, method = "spearman"), I get a p-value that tends to be considered inconclusive, $p = 0.099$, along with a warning that the exact p-value cannot be computed, due to tied values. I am not sure how serious this is, but, combined with the graph, I am skeptical increasing the level decreases the $Y$ variable.

Improve this answer
$\endgroup$
13
  • $\begingroup$ Thanks, Probably due to low number of population (n)? $\endgroup$
    – TheUndecided
    Commented Dec 9, 2021 at 18:01
  • $\begingroup$ "While there are ways of getting $R^2<0$"… Would you care to share what these ways are for a correlation between exactly two variables? $\endgroup$
    – Alexis
    Commented Dec 9, 2021 at 18:21
  • $\begingroup$ @Alexis set.seed(2021); N <- 1000; x <- runif(1000); y <- 10*x + rnorm(N); preds <- 5 - y; plot(x, y); points(x, preds, col = 'red'); r2 <- 1 - (sum((y - preds)^2))/sum((y - mean(y))^2); r2 $\endgroup$
    – Dave
    Commented Dec 9, 2021 at 18:33
  • $\begingroup$ Well, that's three variables, not 2. The correlation between $y$ and $preds$ is $-1$, and the $R^2$ is therefore 1: summary(lm(y~preds)) (look at R-squared) and cor(y,preds), so no: I do not think I am (yet) persuaded. (Alternately: summary(lm(preds~y)) gives the same thing: $R^2 = 1$.) $\endgroup$
    – Alexis
    Commented Dec 9, 2021 at 20:00
  • $\begingroup$ @Alexis Those predictions are generated by $\hat y_i = 5 - 10x_i$, a model that gives $R^2<0$. $\endgroup$
    – Dave
    Commented Dec 9, 2021 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.