12
$\begingroup$

I have a multiple regression problem, which I tried to solve using simple multiple regression:

model1 <- lm(Y ~ X1 + X2 + X3 + X4 + X5, data=data)

This seems to be explaining the 85% of variance (according to R-squared) which seems pretty good.

However what worries me is the weird looking Residuals vs Fitted plot, see below:

enter image description here

I suspect the reason why we have such parallel lines is because the Y value has only 10 unique values corresponding to about 160 of X values.

Perhaps I should use a different type of regression in this case?

Edit: I've seen in the following paper a similar behavior. Note it's a one-page only paper so when you preview it you can read it all. I think it explains pretty well why I observe this behavior but I'm still not sure if any other regression would work better here?

Edit2: The closest example to our case I can think of is the change in interest rates. FED announces new interest rate every few months (we don't know when and how often). In the meantime we gather our independent variables on the daily basis (such as daily inflation rate, stock market data, etc.). As a result we will have a situation where we can have many measurements for one interest rate.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ You almost certainly do need some other form of regression. If the Y data are ordinal (which I suspect) then you probably want ordinal logistic regression. One R package that does this is ordinal, but there are others as well $\endgroup$
    – Peter Flom
    Commented Sep 27, 2012 at 15:20
  • $\begingroup$ Actually the Y is the price we try to predict, which changes every few months. We have weekly-recorder variables (X) for the corresponding price (Y) that changes every few months. Would logistic regression work in this case when we don't know future price? $\endgroup$
    – Datageek
    Commented Sep 27, 2012 at 15:51
  • 2
    $\begingroup$ You're right about the explanation; your reference nailed it. But your situation looks unusual: it appears you have only ten or so independent responses (which lie on a continuous scale, not a discrete one) but you are using multiple explanatory variables that vary over time. This is not a situation contemplated by most regression techniques. More information about what these variables mean and how they are measured might help us identify a good analytical approach. $\endgroup$
    – whuber ♦
    Commented Sep 27, 2012 at 16:09

1 Answer 1

Reset to default
5
$\begingroup$

One possible model it one of a "rounded" or "censored" variable : let $y_1,\ldots y_{10}$ being your 10 observed values. One could suppose that there is a latent variable $Z$ representing the "real" price, which you do not fully know. However, you can write $Y_i=y_j\Rightarrow{}y_{j-1}\leq{}Z_i\leq{}y_{j+1}$ (with $y_0=-\infty, y_{11}=+\infty$, if you forgive this abuse of notation). If you are willing to risk a statement about the distribution of Z in each of these intervals, a Bayesian regression becomes trivial ; a maximum likelihood estimation needs a bit more work (but not much, as far as I can tell). Analogues of this problem are treated by Gelman & Hill (2007).

Improve this answer
$\endgroup$
4
  • 2
    $\begingroup$ This is a good idea. It takes care of the phenomenon but I wonder whether it might miss a bigger problem: even if the prices can be considered censored, they most likely are highly serially correlated. $\endgroup$
    – whuber ♦
    Commented Sep 27, 2012 at 20:10
  • $\begingroup$ I've tried the censReg R package but wasn't able to make it working. It's possible that I didn't understand your idea though. The thing is that we know all dependent variable so we don't have a situation where Y = 0 (censored), it's just that the Y stays stable for few months. I just made another edit so hopefully this explains better our use case. $\endgroup$
    – Datageek
    Commented Sep 28, 2012 at 15:18
  • 1
    $\begingroup$ Radek, I think the idea is this: suppose the price $Y(t)$ depends on time but only changes at discrete times $t_1,t_2,\ldots$. We conceive of this as the manifestation of some unobserved underlying variable (the "real price") $Z(t)$ and we hope that between times $t_i$ and $t_{i+1}$ $Z(t)$ will always lie between $Y(t_i)$ and $Y(t_{i+1})$. In effect, then, we view the observed price at any time $t$ in this interval as being $Z(t)$ as censored both at the left and the right by $Y(t_i)$ and $Y(t_{i+1})$. (I must emphasize "hope": this is the "risky statement" referred to.) $\endgroup$
    – whuber ♦
    Commented Sep 28, 2012 at 17:00
  • 2
    $\begingroup$ whuber : you are right. The original post didn't allude to a time series, so I overlooked that. I think that in order to answer the question, we have to risk two statements : one about the distribution of $Z$ in the intervals $(y_{j-1}, y_{j+1}$, and one about the shape of the temporal model, i. e. the function f binding $Z(t)$ to $f(Z(1), Z(2,\ldots,Z(t-1))$. In a BUGS model, both of these aspects would be expressed in statements about $Z$. Not so simple anymore... $\endgroup$
    – Emmanuel Charpentier
    Commented Sep 28, 2012 at 18:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.