2
Two features can be considered similar with respect to a response if they provide similar information about the response. In other words: if they're redundant for the purpose of predicting the response. I'll describe how to formalize this intuition using information theory, and use it to construct a measure of dissimilarity between features.
Proposed ...
1
The most probable cause for what you are seeing is collinearity, i.e. your 3 independent variables are correlated.
Collinearity in Normal Linear Regression
One assumption of the linear regression is "no or little (Multi-)Collinearity". If we violate this assumption we get biased estimates (coefficients). Sometimes this is exactly what we want, e.g. ...
1
PLS regression (Partial Least Squares or Projection to Latent structures - both mean the same thing and underlying process), as suggested by user257566 is a very useful way to go for this type of query. Unlike PCA it uses external information to guide the act of data reduction, meaning that underlying data processes related to that external information are ...
1
In principle, Factor Analysis was devised to deal with exactly your problem. It tries to achieve three aims:
Identify groups of strongly correlated features.
Identify "irrelevant" features, i.e. features not "contributing" to any of the groups.
Replace each group by a new value called factor that is a linear combination of the features of the respective ...
1
Summary: Standard triplots for ecological data analyzed by canonical correspondence analysis (CCA*) provide a way to gauge both the strengths of relationships of individual environmental variables to species distributions and the similarities among environmental variables in these respects. You might, however, want to do some dimension reduction on the set ...
1
I would run a simple linear regression of the following form:
$R_i = \beta _0 + \beta _1D_i + \beta _2G_i + \beta _3G_i\times D_i + \epsilon_i,$
where i would be a pair of two individuals $R_i$ - the measure of their relatedness $D_i$ - the distance between them, $G_i$ - a dummy variable which is one if both belong to the same group and zero if they dont. ...
1
This problem has often been discussed; one keyword is comparision of "centroid-rotation" vs. "pc-rotation", another one is "parcels" vs. "item-factor-analysis" (or so, I may not be up-to-date).
The computation of the items-mean is here in principle equivalent with determining the "centroid", and if multiple means are taken from multiple subsets of items, ...
Only top voted, non community-wiki answers of a minimum length are eligible
