0
$\begingroup$

I have data on hospital treatment times. I would like to fit a polynomial to the data using least-squares.

In a previous question raised before I have already been advised against this, but unfortunately due to other restrictions, I will need to stay with polynomials.

My problem is understanding what causes the oscillation, or the Runge's phenomenon, in my data so I can minimize it.

I realize that keeping the order of the polynomial low or fitting cubic splines might work, but unfortunately I've tried those options and they will not work for me. It will probably be a polynomial of order 5 or 6 that I will need to fit.

However, I have some wiggle-room in the range of the data that is included in the analysis, therefore I need to understand, in non-technical terms, what is causing the oscillation.

  1. Is it that my data is not evenly spaced? The distribution I am trying to fit has a very long tail with only a few observations near the end. Would eliminating areas with less observations help?

  2. In general, would having less (or more) observations help?

EDIT:

I'm attaching a picture as illustration. enter image description here

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Can you please add a graph showing the oscillation you refer at? This will make the question clearer. (For your question: No, the data being unevenly spaced should have nothing to do with this) $\endgroup$
    – usεr11852
    Sep 13, 2019 at 16:19
  • $\begingroup$ Hospital treatment times are notoriously skewed - you may have to consider modelling the median treatment time rather than the mean treatment time. $\endgroup$
    – Isabella Ghement
    Sep 13, 2019 at 16:24

1 Answer 1

Reset to default
1
$\begingroup$

The oscillations from Runge's phenomenon are inherent in fitting high-degree polynomials. It's unlikely that any adjustment to your data in terms of spacing or numbers of observations would help much at all.

That's why restricted cubic splines are often used for smoothing. They even work well on Runge's example function (analyzed at even spacing in the x-axis), as illustrated here. Unlike fitting a single infinitely smooth smooth polynomial to cover the entire range of data, restricted cubic splines are a set of individual cubic polynomials joined at a set of knots around which the restriction of complete mathematical smoothness is relaxed. You can specify the locations of the knots, which might be important in your situation given the "hump" in the data near your "time 0" from your previous question.

That said, it's not clear whether smooth curve fitting of polynomials, even with restricted cubic splines, will best address your main research question as described in your prior question:

The hump around the zero point is a policy-related distortion and is my main interest. I use a polynomial function excluding this region to identify what the distribution would look like without this distortion.

As noted in comments and an answer to that previous question, your data would seem to be reasonably fit by an exponential or Weibull function except around the "hump," provided that you choose an appropriate time=0 instead of your shifting the time axis so that the "hump" occurs at time=0. If you are willing to "exclude this region" for polynomial fitting then you can do the same for exponential or Weibull fitting.* That would seem to be both a better and a more defensible approach than high-degree polynomial fitting when you need to present your work to a skeptical audience.

*I'm not sure that this focus on the frequencies of treatment times is the best way to proceed. Looking at your data again, I think you might be better served by a form of survival analysis, focusing on the cumulative incidence of treatment times. That is, starting from a true time=0 (e.g., admission or first encounter with a physician), you examine the fraction of patients who finished treatment before each specified time thereafter. That could smooth out the curve quite well on its own, and such analysis is quite amenable to modeling with Weibull hazard functions, of which the exponential is a special case.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your comment. I thought about your idea to use cumulative indicence, but this would completely smooth out my curve. That is I might see signs of the upturn in frequencies around the disturbance, but definitely will not see signs of the downturn, at least not visually. And since it is the size of the upturn/downturn that I want to estimate, I don't see how this would help me out. $\endgroup$
    – Stata_user
    Sep 13, 2019 at 17:35
  • $\begingroup$ @Stata_user you will not see a downturn in cumulative incidence before your time of interest (as CI curves are non-decreasing), but you might well see a flattening of the cumulative-incidence curve followed by an uptick. Alternatively, as your interest seems to be in decisions made around that time of interest, you could consider limiting your analysis to times surrounding that time during which the policy-related distortion might be playing a role, rather than working with the entire distribution of treatment durations. $\endgroup$
    – EdM
    Sep 13, 2019 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.