1
$\begingroup$

I have a data set measuring rock detection depths $Y$ based on the distance from some point of interests $X$, which are classified based on geophysical criteria. Each observation $Y$ is set after plowing the ground to determine if A) there is any chance of rock formation at all and B) what is the distance of the rock from the ground. In most cases the detection is unsuccessful and $Y$ is set to 0. Otherwise $Y$ it's set to a measure of rock depth.

There is a linear relationship between $\log(Y)$ and $\log(X)$ as shown in the plot, for $Y \ne 0$. To allow for the $Y$ log transformation I added a small constant value, 0.00001 which is an order of 10 lower than $Y$ smallest value other than 0.

: X ~ Y

How should I model this data for prediction?

I thought about a convoluted solution using logistic regression to determine whether Y is higher than 0.0001 or not, then OLS to predict Y for each X when the result of the logistic estimator is higher than 50%. Is there a more sound approach?

$\endgroup$
10
  • 2
    $\begingroup$ Those data sure look a little screwy. Can you say more about what they are & where they come from? What do you want the model for? $\endgroup$ Oct 3, 2014 at 22:03
  • $\begingroup$ Possibly censored regression would apply. I learned a lot from stats.stackexchange.com/questions/49443/… $\endgroup$
    – rolando2
    Oct 3, 2014 at 22:25
  • $\begingroup$ @gung Added more information about the data source. $\endgroup$ Oct 3, 2014 at 23:07
  • $\begingroup$ My first thought would be a zero-inflated glm (say gamma with log-link and log(X) as a predictor) .... but looking at your log-log plot, the non-zero part of the distribution looks like a mixture of discrete and continuous, which makes me wonder if there's some important feature of the data that I've missed. $\endgroup$
    – Glen_b
    Oct 4, 2014 at 5:44
  • $\begingroup$ @Glen_b That is correct, data is a mixture of discrete (any rock detection) and continuos (depth). $X$ is the distance from the POI, which linearly predicts rock depth $only$ when rock is actually present. $\endgroup$ Oct 4, 2014 at 9:51

2 Answers 2

2
$\begingroup$

Unless there is a method that is tailored to what the 0.0001 values really mean, a good model to consider is one of the cumulative probability ordinal models such as the proportional odds ordinal logistic model. Such models can handle continuous $Y$ and also deal with arbitrarily large amounts of clumping at specific $Y$ values. The R rms package orm function is designed to handled continuous $Y$ fairly efficiently even when the model has thousands of intercepts. In ordinal models you need one intercept per unique value of $Y$ less one.

$\endgroup$
1
$\begingroup$

You can certainly specify a mixture model which will jointly model the probability that Y is 0.0001 alongside the linear relationship (given that Y is greater than this quantity) between X and Y. On the other hand, I'd first start with some questions about what the underlying process is.

I don't see a lot of variability (though it's hard to tell from the plot) in the proportion of 0.0001 counts over the range of X. Are they actually related, or is there some other sort of censoring process going on?

Also, I'd like to know more about those (seemingly perfect) linear patterns in the non-degenerate Y's. What kind of data is this?

$\endgroup$
4
  • $\begingroup$ Rephrased my question. $\endgroup$ Oct 3, 2014 at 23:06
  • $\begingroup$ I'm still not clear on what X is. You say that it represents a point, but what distinguishes high values of X from low values of X? $\endgroup$ Oct 4, 2014 at 4:19
  • $\begingroup$ $X$ is mainly a measure of distance from some POI of interesets. The idea is that the farther the probe from the POI, the higher the chance that the probe will detect a certain kind of rock formation. Indeed that's confirmed by the linear relationship in the plot, except for those cases where no rock was found at all. $\endgroup$ Oct 5, 2014 at 13:52
  • $\begingroup$ That still seems screwy to me. Think about the actual geological implication of the data as pictured - is the probe standing at the peak of a set of perfectly shaped, though somewhat porous, stacked cones of rock layers? Also, for problems like this, wouldn't a two dimensional location metric be more useful? $\endgroup$ Oct 5, 2014 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.