I have a problem which has me stumped on what to try next.
I have some data from a farm, related to yield over a period of days, for a defined area.
I have daily-resolution data, where the day's harvest efforts yield a certain amount for a given square area, the original data looks similar to like this:
- Day 1: 1.9 kg/m^2
- Day 2: 5.9 kg/m^2
- Day 3: 15.8 kg/m^2
- Day 4: 39.5 kg/m^2
- Day 5: 15.8 kg/m^2
- Day 6: 5.9 kg/m^2
- Day 7: 1.9 kg/m^2
Due to the low resolution sampling, the growth curve looks like a pointy spear, but because this is natural organism the distribution of age/growth/time-to-yield is fairly reasonable to assume is Normally distributed, with a population's mean for maturity being around the 4th day, and the rest scattered either earlier or later than this in a fairly symmetrical fashion.
For modelling the growth related to process control, I am trying to take the assumption of normal distribution and re-sample the original 7-day small set of data points and scale it down to more of an hourly approximation of kg/m^2 becoming 'ready', from T0 to T168 hours.
Edit: Note: My Minitab stats check shows the dataset follows a normal distribution with P = 0.048, which is close enough for me!
Attempt at defining normal distribution and using that to get intermediate hourly values: First, I tried to quantify the data set as having a mean of around 12.5 and std deviation of similar value, and to center this around the 4th day and compute all the yield around that.
However I don't think this properly captures the essence of re-sampling it, it can just help me fit missing data between the days. It seemed that the accumulated yield was higher (area under curve) doing it this way, because it still had the large peak value.
I have to be honest, what I'm looking for is a way to "flatten the curve". I expect that the peak value of 39.5kg/m^2 for the peak of day 4 is actually less if it's smoothed out into the later part of day 3 and early part of day 5 properly. One of the concerns for the process control is the peak loading, and how to quantify it properly is my main goal.
So the accumulated data over days 3,4, and 5 period of time is most likely not as sharp as it looks, because the data is only collected once.
Re-sampling manually by linear interpolation and 3-sample average filter: My next attempt was based on creating a set of data which was x2 increments of the existing data, so 12-hour bins instead of 24 hour bins, and linearly interpolating the values in between and scaling down all original bin values so the accumulated yield was still the same (but now spread out into higher resolution datapoints), linearly interpolated between the major data points.
Then I applied a 3-sample moving average to create a 'smoothed' version of the original dataset, and ensured the total accumulated yield was still accurate.
This was all a very manual process, and in Excel is prone to error.
Can you more learn-ed gentlefolk please suggest a better way I can wrap my head around this data and what i'm trying to do, and help me find the correct terminology and processes/methods I can research to make this into a more formula-driven/standard methodology? Or is my method I'm doing completely wrong and terrible and I should try something else?