0
$\begingroup$

I'm looking for an existing algorithm which carries out the task shown in the title.

My use-case in other words: I have a set of continuous independent variables (X) and a continuous dependent variable (y). I want to find a linear component from X along which y varies the most.

Conceptually, I think I'm looking for something analog to LDA but in my case the dependent variable is continuous. I'd want to maximize the standard deviation of y instead of mean difference between groups. There are no groups in my case.

Thanks!

Edit:

Adding a visual example with synthetic data to clarify what I'm looking for.

Suppose I have 2 independent variables (X) and one dependent variable (y). X is two dimensional, y is color coded. I added a small amount of noise to y.

What I'm looking for is an algorithm which finds the axis along which the trend in y varies the most.

If I take trends in y along the two original axes, I get a flat line for each (apart from noise):

However, if I take axes along the diagonals (space rotated by 45 degrees):

I get much stronger trends:

enter image description here

I'm looking for something that would automate this search, without assuming anything about the shape of the trend apart from it being the most different from flat.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ I am having trouble distinguishing your question from ordinary least squares (OLS) regression. Could you clarify what you mean by "linear component from $X$" and how you wish to quantify the amount by which "$y$ varies"? What is confusing is that $y$ does not appear to change in this setting: its standard deviation is what it is. What OLS does is to find a "linear component" $X\beta$ (a linear combination of the $X$ variables) that maximizes the correlation with $y,$ which is one natural sense of "along which $y$ varies the most." $\endgroup$
    – whuber ♦
    Jul 21 at 21:08
  • $\begingroup$ Thanks for dedicating your time to this question, I already learned a lot from this. Sure, I'll try to clarify. By "linear component from X", I meant a variable which is computed from a linear combination of features of X. Analog to a principal component in a PCA scenario. By "y varies", I meant variance of y. Thanks to you pointing it out, I now realize the standard deviation (or variance) of y will remain unchanged in this setting, no matter what the axes are of our coordinate system. $\endgroup$
    – CanisLupusOccidentalis
    Jul 21 at 22:11
  • $\begingroup$ With that said my question still persists after some refinements. With simple terms, I'm looking for an axis along which y has the "most varying trend". I thought of OLS but it would constrain this trend to be a linear one, which is my concern. In the context of my problem, I don't know the nature of this trend, and it's unlikely to be linear. I just know there's an interesting trend of y along an axis (which is a mixture of my x variables) and I want to find this axis. Do I have to assume a model in this setting for this question to make sense? $\endgroup$
    – CanisLupusOccidentalis
    Jul 21 at 22:13
  • 1
    $\begingroup$ I don't follow this. Nothing about how you have formulated the question gives us any clue what you might mean by "interesting" trend. The only information you clearly give is that the "mixture of my X variables" must be linear. That's what OLS supplies. If you want something more complex, you must create it in the usual way by constructing additional features via nonlinear transformations of the original variables--and then applying OLS. No probability model is assumed; the only thing you are asking, and the only thing OLS does, is to maximize the variance of the fitted values. $\endgroup$
    – whuber ♦
    Jul 21 at 22:29
  • 1
    $\begingroup$ It continues to sound like a standard OLS problem. Maybe you could post a small example showing what your data look like and presenting what you think a good answer would be? $\endgroup$
    – whuber ♦
    Jul 22 at 14:20

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.