How to generate variables with common latent factor? I'm dealing with Kalnins (2018) work about multicollinearity. I can't comprehend one thing - how can I get variables that share a common latent factor? (not only correlate with each other).
I can generate any set of data with fixed correlation values between random variables, but despite the high correlation I do not get (multi)collinearity breaking regression asumptions. I understand that this Kalnins' collinearity (when regression model turns bad) occurs when the variables are not only correlated but also have a common latent factor. In fact, I am a bit helpless, because I do not understand in what mathematical relations the variables should remain with each other in order to obtain such a common latent factor.
Since I work in the social sciences, I would like to understand this concept with some example in social sciences (if possible). I know that I can generate a set of data with simulated questionnaire results (four variables, integer values in the range 1-20), like this:
28,28,26,26
22,16,26,8
29,17,27,29
24,12,14,24
22,6,4,8
20,28,28,28
27,17,29,27
16,30,30,4
14,14,8,2
2,8,6,6
2,2,18,14
7,11,7,7
10,10,2,12
3,1,1,5
23,27,25,29
23,7,11,13
15,25,13,27
14,18,18,16
27,21,25,29

and then how to get common latent factor? Should some individual cases have identical results for all variables? Maybe the point is that the variables should be in some linear relation to each other? But they are, they are correlated and quite strongly! Maybe all these variables should be a function of some other variable? How to understand it, how to get it?
Thank you very much in advance for any tips
 A: The diagram below (from here) sketches out the most commonly-used kind of latent factor model.

Scores on the $x$ variables are modelled as the score on the latent factor $z$, weighted by the factor weights $\lambda$, plus the noise terms $\delta$ (each of which has a mean of zero and a specific standard deviation).
This is pretty simple to simulate (using R, and keeping the code simple):
N = 1000
n_variables = 3
z = rnorm(N, 0, 1)
weights = runif(n_variables, .5, 1) # Random weights

noise1 = rnorm(N, 0, .5)
noise2 = rnorm(N, 0, .2)
noise3 = rnorm(N, 0, .7)
noise_y = rnorm(N, 0, .1)

# Observed variables reflect latent factor plus noise
x1 = z * weights[1] + noise1
x2 = z * weights[2] + noise2
x3 = z * weights[3] + noise3

# y is predicted by x1 and x2
y = .1 * x1 + .1 * x2 + noise_y

X = data.frame(x1, x2, x3, y)


m = lm(y ~ x1 + x2 + x3, data = X)
m_coefs = coef(summary(m))
round(m_coefs, 3)
#             Estimate Std. Error t value Pr(>|t|)
# (Intercept)    0.004      0.003   1.380    0.168
# x1             0.108      0.006  17.550    0.000
# x2             0.095      0.007  13.404    0.000
# x3             0.000      0.004   0.083    0.934

