Tag Info

4

Maximal Information coefficient is one method that has been used for this. "In statistics, the maximal information coefficient (MIC) is a measure of the strength of the linear or non-linear association between two variables X and Y." Detecting Novel Associations in Large Data Sets. D. Reshef et. al ...

4

Depends on the details, but in general think of each autocorrelation as close to the correlation of a series and itself lagged. An outlier adds two points to the corresponding scatter plot, as the outlier appears first as itself and second as a previous value. The net result will often be difficult to detect. I wouldn't expect a correlogram to be a useful ...

3

Here is the relevan excerpt from the Correlogram wikipedia page: If the autocorrelation is higher (lower) than this upper (lower) bound, the null hypothesis that there is no autocorrelation at and beyond a given lag is rejected at a significance level of \alpha\,. This test is an approximate one and assumes that the time-series is Gaussian. ...

2

Outliers affect the covariance and the variance. The acf is the ratio between the covariance and the variance. Since the variance is inflated, the acf is dampened by the outliers. Effectively, the true acf is masked by the outliers. This is why simple model identification schemes are just too simple.

2

Two facts: (i) Correlation is the covariance of the z-scores. (e.g. see here about four-fifths of the way down the page; alternatively, try zx = scale(x) # this returns z-scores directly, but you can use your form instead zy = scale(y) cov(zx,zy);cor(x,y) to see that covariance of z-scores and correlation are the same. (ii) If you takes z-scores of ...

2

The pearson product correlation coefficient is defined as: $r=\frac{cov(X,Y)}{\sigma_X\sigma_Y}$ In order to estimate it from the sample, you put in the sample estimates of covariance and standard deviation: $r=\frac{\frac{1}{n-1}\sum{(X_i-\bar{X})(Y_i-\bar{Y})}}{\sqrt{\frac{1}{n-1}\sum(X_i-\bar{X})^2}\sqrt{\frac{1}{n-1}\sum(X_i-\bar{X})^2}}$ ...

2

The term "marginal" is very old. If you go back far enough in history, there were no scientific journals (evidently they started circa 1665). Instead, interim results were communicated via hand-written letters, and final results were written in books. There didn't tend to be much in the way of data graphics before Playfair, but books might often have ...

1

The comment made by @user32164 still stands as I write: "highly correlated with a poor $R^2$" is contradictory. Regardless of what you consider as highly correlated, a high correlation means a high $R^2$. I am assuming that you measured color somehow so that it may fairly be used as a quantitative predictor in a regression model. Whether that's so is an ...

1

first of all when you calculate correlations, don't just use the default correlation functions in any package, they all refer to pearson correlations. also calculate the kendall's tau and spearman's rho. (Use R). Next for the case of stock prices, only correlation may be one property, but a more useful property is called cointegration which is used in a ...

1

Unfortunately, answering this question in full really requires you to be more specific about what type of pattern you are hoping to uncover. One type of pattern that you may want to consider is one which often arises in the context of time series data (e.g., data sets consisting of a single quantity which rises and falls over time, such as the price of a ...

1

I found Kolenikov and Angeles "The Use of Discrete Data in Principal Component Analysis" working paper to be helpful. Slides here as well.

1

You could use Spearman's, which is based on ranks and therefore OK for ordinal data. You would then have six results. If you want to take a different approach, you could get complex and look at a multilevel model, with subject being repeated. It sounds like "accuracy" would depend on "preference". So, a mixed model could look at that and account for the ...

1

You can certainly do a t-test between the heights of Europeans and Asians. The two sample t-test requires independent data; you seem, somehow, that it requires dependent data. You may be confusing dependent/independent data with dependent/independent variables. See this blog post of mine Independent samples t-tests compare the means of the same variable ...

1

You should not use any form of ANOVA since the number of goals per game is going to be a count, probably with a fairly small mean (I presume you are talking about what Americans call soccer, not what Americans call football). Therefore, you need some sort of count regression model: Candidates could be Poisson, negative binomial and possibly zero-inflated ...

1

Don't call the unweighted item sums "factor scores". Even if the items were continuous and you were using a weighted sum, the scores would not be the same as, or even necessarily good estimates of, the theoretical factors in a factor analysis. Call them "scale scores" (or something similar). Such scales are the norm, rather than the exception, and treating ...

1

To expand on my comment: I am aware that there is a matched-pairs t-test between two lists of counts. Matched pairs of counts might be tested in other ways. Once issue with using a paired t-test on counts is that the assumption of equal variance for observations (and even of differences) will almost certainly be violated. However, I cannot find ...

1

I think your request for the "overall correlation" may be asking the wrong question. If you already know that you have varied factor1 and factor2, the correlations you want to look for are conditional the combination of those factors. It is unlikely the independent variables have absolutely 0 effect on the dependent variables, so looking at the total ...

1

Here's my solution to the problem. I calculate all possible combinations of k of n items and calculate their mutual dependencies by transforming the problem in a graph-theoretical one: Which is the complete graph containing all k nodes with the smallest edge sum (dependencies)? Here's a python script using the networkx library and one possible output. Please ...

Only top voted, non community-wiki answers of a minimum length are eligible