I have a multiple regression problem, which I tried to solve using simple multiple regression:

model1 <- lm(Y ~ X1 + X2 + X3 + X4 + X5, data=data)

This seems to be explaining the 85% of variance (according to R-squared) which seems pretty good.

However what worries me is the weird looking Residuals vs Fitted plot, see below:

enter image description here

I suspect the reason why we have such parallel lines is because the Y value has only 10 unique values corresponding to about 160 of X values.

Perhaps I should use a different type of regression in this case?

Edit: I've seen in the following paper a similar behavior. Note it's a one-page only paper so when you preview it you can read it all. I think it explains pretty well why I observe this behavior but I'm still not sure if any other regression would work better here?

Edit2: The closest example to our case I can think of is the change in interest rates. FED announces new interest rate every few months (we don't know when and how often). In the meantime we gather our independent variables on the daily basis (such as daily inflation rate, stock market data, etc.). As a result we will have a situation where we can have many measurements for one interest rate.

  • 1
    $\begingroup$ You almost certainly do need some other form of regression. If the Y data are ordinal (which I suspect) then you probably want ordinal logistic regression. One R package that does this is ordinal, but there are others as well $\endgroup$ – Peter Flom Sep 27 '12 at 15:20
  • $\begingroup$ Actually the Y is the price we try to predict, which changes every few months. We have weekly-recorder variables (X) for the corresponding price (Y) that changes every few months. Would logistic regression work in this case when we don't know future price? $\endgroup$ – Datageek Sep 27 '12 at 15:51
  • 2
    $\begingroup$ You're right about the explanation; your reference nailed it. But your situation looks unusual: it appears you have only ten or so independent responses (which lie on a continuous scale, not a discrete one) but you are using multiple explanatory variables that vary over time. This is not a situation contemplated by most regression techniques. More information about what these variables mean and how they are measured might help us identify a good analytical approach. $\endgroup$ – whuber Sep 27 '12 at 16:09

One possible model it one of a "rounded" or "censored" variable : let $y_1,\ldots y_{10}$ being your 10 observed values. One could suppose that there is a latent variable $Z$ representing the "real" price, which you do not fully know. However, you can write $Y_i=y_j\Rightarrow{}y_{j-1}\leq{}Z_i\leq{}y_{j+1}$ (with $y_0=-\infty, y_{11}=+\infty$, if you forgive this abuse of notation). If you are willing to risk a statement about the distribution of Z in each of these intervals, a Bayesian regression becomes trivial ; a maximum likelihood estimation needs a bit more work (but not much, as far as I can tell). Analogues of this problem are treated by Gelman & Hill (2007).

| cite | improve this answer | |
  • 1
    $\begingroup$ This is a good idea. It takes care of the phenomenon but I wonder whether it might miss a bigger problem: even if the prices can be considered censored, they most likely are highly serially correlated. $\endgroup$ – whuber Sep 27 '12 at 20:10
  • $\begingroup$ I've tried the censReg R package but wasn't able to make it working. It's possible that I didn't understand your idea though. The thing is that we know all dependent variable so we don't have a situation where Y = 0 (censored), it's just that the Y stays stable for few months. I just made another edit so hopefully this explains better our use case. $\endgroup$ – Datageek Sep 28 '12 at 15:18
  • 1
    $\begingroup$ Radek, I think the idea is this: suppose the price $Y(t)$ depends on time but only changes at discrete times $t_1,t_2,\ldots$. We conceive of this as the manifestation of some unobserved underlying variable (the "real price") $Z(t)$ and we hope that between times $t_i$ and $t_{i+1}$ $Z(t)$ will always lie between $Y(t_i)$ and $Y(t_{i+1})$. In effect, then, we view the observed price at any time $t$ in this interval as being $Z(t)$ as censored both at the left and the right by $Y(t_i)$ and $Y(t_{i+1})$. (I must emphasize "hope": this is the "risky statement" referred to.) $\endgroup$ – whuber Sep 28 '12 at 17:00
  • 1
    $\begingroup$ whuber : you are right. The original post didn't allude to a time series, so I overlooked that. I think that in order to answer the question, we have to risk two statements : one about the distribution of $Z$ in the intervals $(y_{j-1}, y_{j+1}$, and one about the shape of the temporal model, i. e. the function f binding $Z(t)$ to $f(Z(1), Z(2,\ldots,Z(t-1))$. In a BUGS model, both of these aspects would be expressed in statements about $Z$. Not so simple anymore... $\endgroup$ – Emmanuel Charpentier Sep 28 '12 at 18:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.