EDIT: I have modified my data set so as to make it effectively decontextualized, and so now have data and example code if anyone wishes to reproduce this effect. You can find it at the bottom of the post.
I am currently running a path analysis on a set of data (I am using the "sem" function of the "lavaan" package in R).
The path diagram itself looks something like this:
Where "Antecedent," "Opportunity," and "Propensity" refer to groups of observed variables (no latent factors are being used, for the moment) with paths between variables following the paths in the diagram (e.g. any variable in the "propensity" group is predicted by all variables in the "antecedent" group and the "opportunity" group, and predicts all variables in the "Pre-K Mathematics" group), and with "Pre-k Mathematics" being our primary research endpoint (it is not actually Pre-K mathematics in this study, but I could not find a suitably-simple picture with a generic endpoint).
(Note: being a lowly grad student, I do not get to choose the model we are using in this case; please do not respond with detailed questions regarding the motivation for this model, as I will not be able to answer them).
Now, I am observing something curious: the $R^2$ values for some of the endogenous variables (notably for our primary endpoint, but also for several others) are absolutely, unreasonably high (>.99). This seems to me an obvious red flag that something is going wrong..
Even more-interesting is that if I lump our "antecedent," "propensity," and "opportunity" variables into a single category and just do a simple linear regression for our endpoint, the $R^2$ drops to a much more reasonable (read: not particularly noteworthy) value of ~.15 (I strongly suspect there is not much of interest to see in this particular data set). Yet, the actual path coefficients for each variable to the endpoint do not change at all. This seems totally baffling - how can the variable's $R^2$ change drastically when the equation predicting it has remained the same?
It occurs to me that I must be missing something about how $R^2$ is calculated for endogenous variables in a SEM. Does anyone have any insight as to what might be causing this behavior? I am at a loss.
EDIT: As mentioned at the top, here is a sample data set which demonstrates the effect, along with a matching R script. The script reads the relevant files containing the data and model specifications (linked below), and outputs relevant summaries. Both the full path analysis and the simple linear regression are done using lavaan's "sem" method; for verification, the simple linear regression is also done using the "lm" method.
Data:
Model specifications:
renamedModel.txt renamedModelSimple.txt
Script:
library(lavaan)
renamedData <- read.csv("renamedData.csv")
renamedModel<-readLines("renamedmodel.txt")
renamedOut<-sem(renamedModel, renamedData)
summary(renamedOut, rsquare=T, standardized=T)
renamedModelSimple<-readLines("renamedmodelsimple.txt")
renamedOutSimple<-sem(renamedModelSimple, renamedData)
summary(renamedOutSimple, rsquare=T, standardized=T)
verification <- lm(data=renamedData, Out ~ A1 + A2 + O1 + O2 + O3 + O4 + O5 + O6 + O7 + P1 + P2 + P3 + P4 + P5)
summary(verification)
Note that I was not quite correct when I noted earlier that the path coefficients barely change at all - rather, they appear to be rigorously identical. I suppose that is somewhat suspicious, as well - but I have no idea how to interpret that, either.
