Hot answers tagged correlation
5
Yes, they are the same. The Matthews correlation coefficient is just a particular application of the Pearson correlation coefficient to a confusion table.
A contingency table is just a summary of underlying data. You can convert it back from the counts shown in the contingency table to one row per observations.
Consider the example confusion matrix used ...
4
What the correlation says is that there isn't much correlation between these two variables; that is not exactly the same as independence. It is possible to have completely dependent variables with a 0 correlation:
set.seed(201139)
x <- rnorm(100)
y <- x^2
cor(x,y) # -.18, with another seed, could be closer to 0
plot(x,y) #perfect dependence
I am not ...
4
This relatively short question raises numerous different issues. Here are some:
In statistics, independence is an absolute condition, not something that exists in degrees.
Correlation measures correlation, which is at most linear dependence. A zero correlation could reflect some nonlinear relationship.
A check on your data shows that you are using ...
3
Demidenko, Williams, and Swartz (2009) is one example where log dose and log time are used instead dose and time. The authors provide a good explanation as to why they log transformed the data in their study.
In other words, this is a hint to do a little bit of reading.
Reference: Radiation Dose Prediction Using Data on Time to Emesis in the Case of ...
3
OK. Well, if you are looking for alternative correlation measures, maybe we should start with some basic time series stuff - which the author of this blog seems unaware of.
Of course, getting a quantitative measure of “highly correlated” to
match with my intuitive understanding has never worked out so well.
Correlated, for me, tends to mean: ...
3
Two options come to mind. What I would prefer is a scatter plot of X against Y with a non-parametric smoother, such as LOESS, to show the general trend between them. It doesn't directly correspond to a rank correlation, but since the Spearman correlation is a measure of monotonic trend, you can eyeball a non-parametric smooth curve to determine just how ...
3
Warning: use this solution only if you can safely assume that you have no errors in $x$. For a solution when both variables contain errors, look @Nick Cox' answer or at the bottom of mine.
The 1:1 line is a line with slope 1. You could use a Wald test to test whether your slope differs significantly from a line with slope 1. The Wald-statistic is:
$$
...
3
Heart rates vary in a cyclic pattern that is driven by the respiratory rate. Inspiration causes decreased filling of the left atrium and the heart rate increase to maintain cardiac output. You need to detrend the respiratory influence.
Because the instantaneous heart rate is just the inverse of the RR interval, you do not need to wait for 15 or 20 seconds ...
3
It is the sample size. For a simple correlation between two variables, the coefficient and sample size are the only pieces of information that you need in order to compute the standard error, and hence assess statistical significance. So if the coefficient does not differ significantly from zero despite being a canonical "medium" sized effect, then the only ...
2
The correlation coefficient is what it is - basically an effect size measure - & any rules of thumb about what's 'small' or 'weak' ignore the context of what real things the variables are measuring. You can test for its statistical significance but its practical/theoretical significance is for a subject-matter expert to determine.
(Spearman's is a ...
2
Weka is open sourced software, so you can find it out from its source code comment:
"Evaluates the worth of an attribute by measuring the correlation (Pearson's) between it and the class.
Nominal attributes are considered on a value by value basis by treating each value as an indicator. An overall correlation for a nominal attribute is arrived at via a ...
2
The "Class Correlation" is Pearson Correlation Coefficient between target variable and the other variables. i.e. corr(Species, sepal length) = 0.7826
Here is the R output (i do not have Weka handy):
> iris2 <- data.frame(iris$Sepal.Length, iris$Sepal.Width, iris$Petal.Length, iris$Petal.Width, as.numeric(iris$Species))
> head(iris2)
...
2
The size of the correlation coefficient doesn't tell you something about the significance of the effect. Look at the Wiki page about the Fisher transformation. As Jake Westfall pointed out, only the correlation coefficient $r$ and the sample size $n$ are used to calculate the standard error $SE=1/\sqrt{n-3}$ and the $z$-value which is then used to calculate ...
2
Concordance correlation is a measure of agreement between variables. See http://en.wikipedia.org/wiki/Concordance_correlation_coefficient for discussion and references. It was named by Lin but was earlier suggested by Krippendorff.
Unlike regression, concordance correlation treats variables symmetrically. That may or may not be closer to what you want.
1
Clive Granger's Nobel Prize Lecture contains an excellent explanation and it's not very technical either (no equations). It's one of the best lecture's I've ever watched. Notice how Clive does not even need to use lecture slides! A lesson in teaching time-series and in teaching in general. Perhaps an idea would be to suggest that they watch this lecture.
1
In the machine learning world, this is called a supervised learning problem. Each of the 100 columns are called "features" which may or may not have any predictive value for the outcome. Our goal in supervised learning can be to select the smallest combination of features to create the highest amount of discrimination in the binary outcome as possible, so ...
1
After doing some serious googling I found that this test is referred to as “Oldham’s test”. Apparently it can be shown that the correlation between the change and the mean of two measurements is:
$\text{Corr}[x-y, (x+y)/2] = \frac{s_{x}^{2}- s_{y}^{2}}{\sqrt{ (s_{x}^{2}- s_{y}^{2})^2 -4r_{xy}^2 s_{x}^{2} s_{y}^{2}}}$
This becomes a test of the difference ...
1
@Glen_b examples showed that a strong correlation is not strictly impossible in this situation but I think your intuition is right. Limited variation (for example range restriction) can bias sample correlations toward 0. The problem is not merely one of “significance”, the correlation also appears systematically smaller than it would be if you would consider ...
1
It's possible to get high point biserial correlation even with 27 $1$'s and a $0$. Indeed, you can get as high as 1, so it's not that:
y <- c(0,rep(1,27))
x <- y
cor(x,y)
[1] 1
-- and making x continuously distributed doesn't substantively alter that conclusion:
y <- c(0,rep(1,27))
x=c(rnorm(1,0),rnorm(27,100))
cor(x,y)
[1] 0.9987537
...
1
A quick Google search led me to this article: Estimating the Correlation in Bivariate Normal Data With Known Variances and Small Sample Sizes. In this article, they discussed several possible priors: "uniform", "Jeffreys", and "arc-sine". Specifically, take the "uniform" prior for example, it assumes that $\rho$ follows a uniform distribution on $[-1,1]$.
...
1
You could code yes-no as 1, 0 (or any other integers) and then use standard formulas and software.
Whether that's any use will depend on your data and your problem.
I'd calculate fraction yes for each age and then plot against age. Some kind of smoother may help if patterns are not obvious.
Two wild guesses are that
Spearman correlation will make ...
1
This truncate-transform method is the first of two ad hoc methods I've contrived. I'd appreciate constructive criticism, questions, or other feedback about it to help improve it or avoid inappropriate use, and I'm certainly interested in better, more principled strategies -- preferably posted as other answers.
Before using this method, we specify $\eta$ ...
1
The following is a rigorous exposition of a simple intuitive idea: you can take any single point out of a point cloud and, by moving it sufficiently far away, make the correlation coefficient equal to any value you please in the interval $(-1, 1)$. Think about how you would draw the scatterplot: the point cloud would be an unresolved mass of points at one ...
1
Pearson's correlation is usually what is intended when the word "correlation" is used without qualification.
That said, a good starting point is to look at a scatterplot of your two variables. There are plenty of options for how best to assess the negative relationship implied by your question. Depending on distributions you might want to transform your ...
Only top voted, non community-wiki answers of a minimum length are eligible
