1
$\begingroup$

I can understand how it works when there are two variables in the linear regression model(the shaded circles represent the observed variables, and the white ones the latent variables):

enter image description here

We can draw the following scatter plot for the above graph(or linear regression function):

enter image description here

But when it comes a little bit complex(we have still one observed variable but two latent ones):

enter image description here

I don't understand why its diagram becomes this one(analogous to the scatter plot of the linear regression above):

enter image description here

And how can I model the so-called complex regression model? Could anyone please recommend any related articles for me? It just becomes too abstract to me. Thanks in advance.

My questions, to be precise, include the following:

1) what is the so-called complex regression models? I have searched Google and found nothing related to it. I thought maybe someone can tell what probabilistic graphical model it is according to the graph and the scatter plot.
2) how can I model that which has two hierarchical latent variables(targets)?
3) how are the scatter plot and graph pair related? I saw that on a slide which obviously shows that they are representing the same underlying mechanism but in two different ways.

$\endgroup$
  • $\begingroup$ Is this supposed to illustrate an interaction? In that case, you can simply model it by including a multiplicative effect of $x$ and $z$ (in R: lm(y ~ x * z)). $\endgroup$ – Frans Rodenburg Jun 30 '19 at 23:01
  • $\begingroup$ Sorry, but I don't understand what your question is or what the last diagram is supposed to represent. $\endgroup$ – Peter Flom - Reinstate Monica Jul 1 '19 at 10:54
  • $\begingroup$ I understand, let me answer. $\endgroup$ – cure Jul 1 '19 at 10:56
  • $\begingroup$ @PeterFlom The shaded circles are observed variables and white ones are latent ones as explain in the first two diagrams. The first diagram is the graph of the linear regression and the second one is the scatter plot of the linear regression. The same applies to the third and fouth. $\endgroup$ – Lerner Zhang Jul 1 '19 at 12:53
  • 1
    $\begingroup$ The thing you present is the spline regression. Unfortunately this is mostly basic econometrics, so I know it is hard to ask a proper question if you do not know the terms and many mechanisms (nevertheless you should try to in the future). I think it was unfair to hold it. In my opinion the reason to close it was of lack of understanding and curiosity of the readers. If you want to look about the spline, try here: analyticsvidhya.com/blog/2018/03/… $\endgroup$ – cure Jul 1 '19 at 14:42
0
$\begingroup$

It's hard to say anything for sure without knowing the exact context of the pictures you have provided. Below is my guess on what is going on in the pictures.

The case of the first two pictures is clear. To ease myself the explanation of the second case I write down the situation in the pictures in the form of a mathematical equation:

$y_n = \alpha + \beta x_n + \epsilon$

I guess two latter plots present a situation which can be described with two equations (one for each node in the graph pointed by arrows):

$ \begin{cases} P(z_n = 1) = \frac{1}{1 + \exp \left( - \left( \alpha + \beta x_n \right) \right) } \\ y_n = \gamma + z_n\delta - (-1)^{z_n} x_n + \epsilon \end{cases} $

OK. So what's going on here?

I guess the second situation presents an example of a regime-switching model. It's a class of models which kind of consist of multiple submodels (regimes) and are able to switch between them depending on the situation. I guess in your case the $z_n$ variable indicates the regime of your model. It may have one of two values: 0 or 1 each value representing one regime. I used logistic regression as an example to model the probability of $z_n$ being 1 depending on the value of $x_n$, but there are also other choices possible. The second equation describes the effect of $x_n$ variable on the $y_n$ variable accounting for both regimes the model may be in. These regimes are represented by the $z_n$ variable. If $z_n = 0$ this equation reduces to $y_n = \gamma - x_n + \epsilon$. For $z_n = 1$ the equation is equivalent to $y_n = \gamma + \delta + x_n + \epsilon$.

Now the interpretation of both pictures becomes clearer:

The first picture (the graph) shows which variables are influenced by which. The input variables (source of the arrow) are on the right side of equations and output variables (target of the arrow) are on the left side of the equations.

The second picture is actually two pictures overlaid. One for each equation. Dashed lines present the way the $x_n$ variable influences $z_n$: the probability of $z_n$ being 0 or 1 depending on the value of $x_n$ as described in the first equation. Solid lines and x's is the second picture describing the two regimes of how $x_n$ influences $y_n$. In this example situation regimes switch in the middle of x axis.

And answering your question about "complex regression models": I don't recognize this as a statistical term. I guess in your context it just means "more complex than the simple regression model" which is the first case you described.

Finally, one thing which attracted my attention: in the beginning of your question you say that the white circles represent latent variables. I guess you meant output variables which is a different thing than latent variable. The meaning of latent variable is easy to google so I won't explain the difference here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.