4
$\begingroup$

The title may be entirely inappropriate for this question: that depends on whether I am on the right track. I am developing a statistical model to evaluate flower temperature based on air temperature. I'm not very good at anything statistics related, but I'm decent in programming in Python so I was thinking of building a linear regression in there.

However, I have difficulties figuring out how to build my regression and whether even a linear regression is a good choice at all.

The following plot shows how temperature and flower data varies with the time (Month-day Hour).

The black line = Air temperature.

The coloured lines = various flowers whose temperatures were measured.

Perhaps important, but I can't figure out how to make use of them: the arrows at the top show the wind and direction, while the blue line shows the incident solar radiation.

The y-axis shows the temperature (in Celsius) while the x-axis shows the time.

enter image description here

If I were to compute a linear regression, it would have to be valid for any hour of the day, hence I can't just evaluate how the flower temp varies with the air temp, because a 20 degree air temperature at 1 pm won't give out the same flower temperature as a 20 degree air temperature at 1 am. I tried separating night and day (day varies in between 6 am-7 pm), but even then the results were too chaotic.

After reading this post: Is time of the day (predictor in regression) a categorical or a continuous variable?, I thought maybe that using a categorical approach would work, but I am getting from this is that I would have 24 different equations for each hour of the day, which seems a bit much. I suppose that I am prepared to attempt such an approach, but I was hoping to get some advice before pressing on?

Perhaps I should simply use solar radiation instead of the time? But even then, the shape is periodic and I have no idea how to integrate a periodic component within a linear regression!

| cite | improve this question | |
$\endgroup$
  • 2
    $\begingroup$ Nicely phrased post. The solution is going to be found not in some trick within linear regression, but in an entirely different family of models. You'll want to get familiar with time series models, seasonality, autocorrelation and cross-correlation. Some authors to look at: Rob Hyndman and Mark Pickup. Good luck! $\endgroup$ – rolando2 Oct 26 '18 at 17:14
  • $\begingroup$ How much data do you have? If you have a lot, say several hundred observations, more options are open to you than if you're more limited. $\endgroup$ – jbowman Oct 26 '18 at 17:38
  • $\begingroup$ @jbowman, what i'm showing right now is about a 2200x5 matrix and goes for 6 days (the flower period only lasts about 2 weeks). I also have about 200-300 observations that extend throughout 3 years, but I intended to use those as a way to verify the regression. $\endgroup$ – MorningGlory Oct 26 '18 at 18:03
  • $\begingroup$ If you scale individual 24-hour periods from minimum temperature = 0.0 and maximum temperature = 1.0 and then plot them together, that should be easy to do and might be instructive in your study. $\endgroup$ – James Phillips Oct 26 '18 at 18:12
  • 1
    $\begingroup$ You can also think of the time of day as a circular predictor. See also this post. $\endgroup$ – Kees Mulder Oct 29 '18 at 12:30
3
$\begingroup$

I think a partially linear modeling framework may be suitable for your problem. If you focus on one flower at the time, note that both the flower data and the air temperature data exhibit strong temporal cycles which peak roughly at the same time. So the simplest partially linear model you could consider for one flower would look like this:

FT_h = beta0 + beta1*AT_h + m(h) + epsilon_h,

where FT_h is the flower temperature for the chosen flower at hour h, AT_h is the air temperature at hour h, m() is a smooth, unknown function meant to capture the temporal cycles you see in the temperature data and epsilon_h is an unknown error term. Here, h = 1, 2, 3, ..., H is an index which counts how many hours you have represented in total in your flower data. In other words, this index counts your hours from the first to the last. If you have 9,000 hours represented in your data, for example, then H = 9,000. In this model, beta1 represents the hourly effect of air temperature on flower temperature, after controlling for temporal effects.

The model can be expanded by adding a linear effect for incident solar radiation (ISR):

FT_h = beta0 + beta1*AT_h + m(h) + beta2*ISR_h + epsilon_h.

If you wanted to throw in wind direction as well, you could code this variable as taking the values North, South, East, West (or add variations like North-East, North-West, etc.) and include it in your model using dummy variables. For example, if you only code this variable as taking the values North, South, East or West, the flower-specific model could be expressed as: 

FT_h = beta0 + beta1*AT_h + m(h) + beta2*ISR_h +
       beta3*NorthDummy_h + beta3*EastDummy_h + beta4*WestDummy_h +       
       epsilon_h,

where South is treated as the reference direction against which all others will be compared and NorthDummy_h is set to 1 if wind direction was North at hour h and 0 otherwise, EastDummy_h is set to 1 if wind direction was East at hour h and 0 otherwise and WestDummy_h is set to 1 if wind direction was West at hour h and 0 otherwise.

The challenging aspects of these models are:

  1. The need to estimate the (unknown) degree of smoothness of the (unknown) temporal effect m() carefully, given that this is just a nuisance effect and the real interest is in estimating beta1;

  2. The possibility that the error terms epsilon_h might be temporally correlated, which in turns can affect how item 1. above is addressed.

Many years ago, I conducted research on this very topic - see, for example - http://www.ghement.ca/217.pdf. However, I have not stayed current on the topic so it's possible there have been several advances on ways to handle item 1.

Intuitively, the temporal signal seen in the data is really strong while the air temperature signal is likely tiny by comparison. So you need to find the right balance when determining the degree of smoothness of the temporal effect, so as not to throw the baby with the bath water.

If you are interested in comparing effects of air temperature across flowers, you can expand the model even further. But I would start small to make sure I get a handle first on the simpler, flower-specific models.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Wow... I didn't expect to receive such a well-developed answer! Thank you so much for sharing with me this knowledge about partially linear models with error coefficients. The subject seems quite complex, but it seems very linear algebra oriented, which is a domain I'm much more at ease with than with statistics in general (even though both are quite related!). I hope to make the most out of it and, even if that’s not very pertinent, if it works, then I will come back to make this as the acceptable answer. Cheers! $\endgroup$ – MorningGlory Oct 31 '18 at 14:41
  • 1
    $\begingroup$ Good luck, @MorningGlory - it might be easier for you to switch to R for this analysis, as it has the right tools for what you need via the mgcv package and its gam() or gamm() function. $\endgroup$ – Isabella Ghement Oct 31 '18 at 15:27
  • 1
    $\begingroup$ By the way, if you suspect AT (air temperature) to have different effects on FT (flower temperature) in the first half of the day compared to the second half, you can just include a main effect for HD and an interaction between HD and AT in your model, where HD = 1 if a specific hour h in your data falls on the first half of the day and 0 if it falls on the second half of the day. $\endgroup$ – Isabella Ghement Oct 31 '18 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.