Modelling the logarithm of a response

Question

My response variable is positive and I decided to model the logarithm of the response.

Some of the values are zero. For this reason I modelled $Z = \log(Y + 0.1)$. When I transform back, some of my predictions are negative.

I.e. $Y = \exp(Z) - 0.1$ is negative for some predictions. Am I missing something here or is it expected that some predictions may be negative when transforming back to the raw scale?

Perhaps I should be considering $Y' = Y + 0.1$. I can then model $Z = \log(Y')$. When transforming back to the raw scale, $Y'$ will be positive. Perhaps the only guarantee is that $Y'$ will be positive?

One other thing that I tried (which I know is not ideal) is to replace all zero values with a small number $\varepsilon$. This way the transformation was $Z = \log(Y)$ and hence the predictions on the raw scale, $\exp(Z) \geq 0$.

Edit: Consider that the output being modelled is rain in mm per hour. It is possible to observe 0 ml of rain in a given hour.

Consider the following:

y=c(3,1.9,1.2,0.5,0.3,0.2,0.1,0.05,0.03,0.01,0)
y = y+0.01
plot(y, ylab = "y", xlab = "Time", type = "o")
plot(log(y), ylab = expression(log(y)), xlab = "Time", type = "o")

$\log(y)$ is linear and could be modelled using a linear regression. The slope is steep and so it would not be unusual for a prediction to predict lower than $\log(0.01)$. What could be done in this case?

Answer 1

A generalized linear model with log link assumes only that the mean response is positive. That is consistent with some observed zero values (or even some observed negative values).

One flavour of GLMs with logarithmic link is popular as Poisson regression, which, contrary to myth, is not restricted to count-like responses.

An appropriate GLM is a much better approach than fudging the data so that logarithms can be calculated for all the values of the response. It's logical and respects knowledge of the generating process as well as the data as they arrive.

However, the occurrence of zeros might need to be accommodated in some more complicated model, usually called zero-inflated. All depends on your response, which evidently is like rainfall, but may not be. For example, the distribution of cigarette smoking, measured as cigarettes per day, often shows a very large spike at zero, dominated by people who do not smoke at all, with another mode say around 20, for smokers who smoke about a pack a day. Although the marginal distribution of the response need not drive the form of a model, spikes at zero are often special.

Note: your response vector

(3,1.9,1.2,0.5,0.3,0.2,0.1,0.05,0.03,0.01,0)

includes a zero but generalized linear models with log link happily report a coefficient of $-0.4113917$ equivalent to a "geometric mean" of $0.66272729$. I give here a ridiculous number of decimal places so that anyone who cares can check with their own software what mine (Stata) shows.

Naturally we can't comment on your real data as yet.

Answer 2

There are two approaches I've seen to the problem of handling 0s. One is how the OP did with taking logarithms and adding a small value for 0s to ameliorate the $\text{log}(0) = -\infty$ issue. The other is to use a distribution with a point mass mixture at 0. One common approach is a Tweedie distribution with $1<p<2$. There is support for this in R and Python as well as common statistical software such as SAS.

One of the challenges with using logarithms is that the function is not linear whereas expectations are, the implication is that $\mathbb{E}(\text{log}(y_i)) \neq \text{log}(\mathbb{E}(y_i))$. This means if you're using least squares as an objective function you'll get some bias because of the non-linearity. Under certain distribution assumptions (e.g. lognormal) you're actually modeling the median and not the mean. If in general you want invariance to monotonic transformations in your solution method (least squares does not give you this property in general) you need to exit the least squares fitting approach and use a quantile regression approach.

Quantile regression can be used in the programming languages and statistical software mentioned above too. Although quantile regression comes with its own set of challenges it may be worthwhile to consider if you want this sort of invariance.

Answer 3

You are likely extrapolating too far. The reason some of the predictions may be negative is because the predicted $Z$ is smaller than $\ln(0.1)$. When this is the case, your transform will return a $Y$ smaller than 0.1, so subtracting 0.1 makes a negative number.

One approach is to use a different likelihood. Ask yourself why the outcomes are 0, and are they adding information to your model. Consider an example of someone modelling customer spend. Well, in order for customers to spend then need to buy something, so why would I include customers who do not buy something in my data? In that case, my data generating process is more complex (the processes of deciding to buy, and then the amount they spend) and I should model that directly with something like a hurdle model. I think something similar can be done in your case, but I'd need to know more about the data in order to comment intelligently.