I have a data set of N points, n X variables plus Y variable. $$ (x^{(i)}_1,...,x^{(i)}_n,Y^{(i)}),\,\,\,\,i = 1,...,N $$ generated by a mixture of $k$ linear dependencies; with this, I mean that there exist $k$ vectors of coefficients $$ a(j) = (a(j)_1,...,a(j)_n),\,\,\,\,j = 1,...,k $$ such that for each $i$, there exists a $j \in \{1,...,k\}$ such that $$ Y^{(i)} = \epsilon + a(j)_1x^{(i)}_1+...+ a(j)_nx^{(i)}_n $$ where $\epsilon$ is a random noise. So, in theory, for each $i\in \{1,...,N\}$, there exists a $j \in \{1,...,k\}$ such that $a(j)$ describes the relation between the $x^{(i)}$ and the $Y^{(i)}$
However I don't know the values of the $a(j)$: I only have the $N$ points and the number of mixture components $k$.
Now, I want to group together the individual samples that have been generated by the same linear function, that is, for each $i\in \{1,...,N\}$ find the most suitable $j \in \{1,...,k\}$.
A possibility may be the following: For each $i\in \{1,...,N\}$ I do a linear regression and find coefficients $$ b^{(i)} = (b^{(i)}_1,...,b^{(i)}_n) $$ such that $$ Y^{(i)} = b^{(i)}_1x^{(i)}_1+...+ b^{(i)}_nx^{(i)}_n $$
Then I cluster in $k$ groups the various $b^{(i)}$ to get the clusters, using k-means.
My questions:
- Is this a safe approach?
- There exists a standard procedure to cluster points generated by mixtures of linear functions?
For each i...I do a linear regression and find coefficients
How are you going to do a multiple "linear regression" for each point i separately? What error will such regression be minimizing? $\endgroup$