Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
How can I do regression when the variables have no specific relationship (linear or non-linear)?
I looked at scatter plots and correlation tests to discover any linear relationship. I also did a curve estimation test to test for existence of non-linear relationships in spss. But there seems to be no specific relationship between the variables (linear or non-linear).
In this cases how we still do regression? (Sampling was costly and I can't just leave the data.)
It sounds like you are treating regression analysis as a tool to show that there is a relationship between two variables. But the goal of regression, or correlation analysis, or looking at a scatter plot, is to figure out if there is a relationship between variables or not. Often times the relationships we think exist (or that we WANT) to exist do not actually exist in real life. That's why we do these tests in the first place!
It looks like you've checked a bunch of methods to see if there is a relationship between these variables and haven't found one. If you only have two variables then running a regression analysis (linear or otherwise) isn't going to much you anything more than looking at a scatter plot or running a correlation. You could try it, but my guess is that the result will not be significant.
If all these analyses are telling you that there is no significant relationship between the variables, then that's your result. In this case you are not "leaving" the data - you are accurately describing what the data show. It doesn't matter how expensive it was to collect, if there's no relationship in the data it's not your job to create one.
You can assess the predictive power of a regression model using cross-validation regardless of what the true relationship between the variables is. So you can do linear or nonlinear regression, and cross-validation will tell you how good your model is at prediction (at least within the range of the observed data) no matter whether the assumed model relationship is really true. (You need to assume independence and the same relation valid for all data though.)
So if your cross-validation predictive power is good enough (which to assess depends on subject matter knowledge), the model is useful regardless of whether it's true.
On top of that, if you run a linear model and look at diagnostic residual plots, this may give you a better idea of whether linearity is a good assumption or what other relation could hold than looking at raw scatterplots and correlations.
