How can I analyse an annual cyclic dataset with large gaps between some readings?

Question

I have solar panels and have been logging their output in a Google spreadsheet. The only data I collect is the following:

Reading Date    kWh

Where kWh is the cumulative reading on the meter.

I don't perform a reading every day, sometimes weeks or months can pass without me taking a reading.

In total I have 144 readings stretching back to 11th May 2013, the average gap between readings is 10 days, but the max is 106. Therefore the data isn't of a very high quality.

What analyses can I perform on this, principally to see whether my panels are deteriorating over time? Is my only option to calculate an average reading per day and then compare different years or is there something else I can do?

I'm coming from a programming background by the way - I am far from being a statistician or a mathematician!

Answer 1

First, I wonder if you have enough data to deal with the variation in the amount of incoming radiation between years. Do you know whether the number of hours and intensity of sunshine in your location has overall increased or decreased since 2013? This between years factor would be useful to know.

Over the course of each individual year, the output of the panels is likely to be somewhat sinusoidal due to the annual changes in the angle, intensity and number of hours of sunshine. You could separate each year's data - interpolation to separate the years might be least error prone in winter, when the output is generally very low anyway. Then, try to fit a sinusoid (or even a Gaussian/normal distribution) to each year's data, and then look at the differences between the fitted models. Is there a consistent pattern of reducing amplitude or area under the curve, that is inconsistent with the climate data for your location?

Edit: Your data is a cumulative sum, so rather fitting than a sinusoid or Gaussian to estimated per-period data, it might be better to fit a cosine, or logistic function to your cumulative data.

My other thought is about collecting new data. When I observe the output from my panels, it tends to max out on a sunny day in summer, when the sun is high - the panels' performance rises in the morning, reaches a plateau sometime during the morning and doesn't increase past that plateau, then reduces in the late afternoon. If this is the case for you too, then you might get the most direct measure of performance change by comparing periods when the performance is maxed out (take readings at 11am, noon and 1pm on some bright summer days, at least in the future, and see if this changes the following year).

Because you don't have intra-day measurements for previous years, an alternative might be to try to extract estimates from your data for the performance in July in each year by interpolating between the periods you do have readings for (for instance - if July is usually sunny where you are). You could even do this separately for every month, and compare these data sets to see whether they all tell the same story of decreasing (or increasing!) performance.

Answer 2

With only this kind of data, you should/could start out with averages over time, based on the available data within specific periods of time. Note that this will force you to decide on periods with or without regular spacing. Further, you will have varying (un)certainty around your estimate of the mean, or even missing periods, due to the varying frequency of your measurements.

Alternatives - handling missing data

An alternative would be to assume something about your missing values. As a start you could assume that the amount of KWh you've measured since the previous measurement is collected at a constant rate in the period in between the last and previous measurement (essentially averaging the KWhs over the amount of days in between the measurements). This way you'd have values for every day and would be able to assess visually the KWh return of your solar panels. This would still not give you more certainty than the first suggestion however. For that you will more data. In your case, I feel you could actually try to obtain more data! As we are talking solar panels, we know the weather influences the KWh return. If you could get a daily (or weekly) read-out of cloudiness, hours of sun, rain, temperature and other meteorological measurements which might affect solar energy return, you could use multiple imputation to estimate (multiple times) the missing measurements, based on the associations from the days you have measured. You could even think of imputation models which would take into account that you know the cumulative returns exactly. This would take into account the uncertainty of replacing missing values and will give you estimates with a proper sense of uncertainty around them.