1
$\begingroup$

The data is from Udacity DAND Project 3. I finished this project but I am still wondering about some interesting discoveries.

I got the scatterplot below and there must be something over there.

  1. The distribution is clearly consisted of multiple straight lines.
  2. On the direction from top left to bottom right, the ProsperScore tends to be decreasing but seems to be bimodal or multimodal for lower range scores (1-5).

I am not quite sure how to make this pattern clear and interpretable. How to model it? Can anyone help me?

Data download link

Scatter Plot with Multiple Linear Relationship

$\endgroup$

3 Answers 3

3
$\begingroup$

As far as the multiple straight lines bit goes, I'm basically copying and pasting from another comment I put up a while back.

This seems like a perfect candidate for ANCOVA. Like you said, there appear to be multiple linear relationships within the same scatter plot. That seems to indicate that there is another (categorical) variable that controls which of these linear relationships you end up with.

As an example of what I'm talking about, suppose we consider an experiment where students are subjected to learning math from a good teacher, a bad teacher or from no teacher at all and we have their pre class math scores and their post class math scores. We then look at a scatter plot of preclass scores to post class scores. We would expect preclass scores to have a high positive correlation with post class scores but we would also expect that students in the good teacher treatment would score better than students in the no teacher treatment or bad teacher treatment.

Your plot clearly has a categorical variable with more than 3 levels controlling its slope but the idea is the same. This is a perfect scenario for ANCOVA.

As far as the color pattern goes, this looks similar to a plot of clustering results where not enough dimensions were used in the plot (or perhaps the wrong dimensions were used). Have you plotted this in 3 dimensions and seen what it looks like?

$\endgroup$
2
$\begingroup$

This interesting graph indicates that there is another factor that is causing the scatter to align with different intercepts. Since the slopes of all these clusters are almost same, y/x ratio will separate these clusters.

Hence if a new variable of EstimatedEffectiveYield/BorrowerAPR is created and then converted to suitable categories, it will group different points into clusters indicated by above scatterplot.

Alternatively, cluster analysis techniques can be applied to separate these groups.

$\endgroup$
1
$\begingroup$

You really need two plots here. One without the fill factor variable and then another that would facet on it.

Without the factor variable it is fairly clear there is positive correlation between APR and Estimated Yield (modeled ?).

It's unclear from this visualization alone how the Prosper Score plays out and that's why I recommend looking at it by each bucket.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.