2
$\begingroup$

I am using Google Charts to plot a large data set. The database contains one record for every two seconds; five minutes' worth of data yields 150 records (data points) and the result is acceeptable. However, my client wants to be able to visualize a whole day or even a month, which is a lot of data for an Ajax request, and the resulting graph doesnt look good. I tried averaging the values, but this attenuates the peaks. I need to compress the data while preserving the extrema.results The data represent the voltage in each phase of an electric grid. The first graph ("A") in the image displays 5 minutes' worth of data (150 datapoints), with a peak of around 600 volts at 15:32. The second ("B") is for 10 minutes, also 150 datapoints with averaging. Notice the peak now appears to be a little over 400V. The third graph is for 60 minutes, no averaging; the peaks are preserved, but there are now 1800 datapoints and the graph appears crowded. The last graph is an example of the desired result. Somehow it manages to display a month's worth of data without appearing jumbled up. This is an example of the desired result.

example data 280 points

http://expti.com.br/getlog.asp

example

$\endgroup$
13
  • 4
    $\begingroup$ This is much too general. There is no way to suggest ideas about compressing the data and about creating concise graphics without knowing what sort of data and issues it is about. Could you show examples, and better explain which features you would like to wish to present in a graph. $\endgroup$ – Sextus Empiricus Sep 30 '19 at 13:50
  • 2
    $\begingroup$ An example of an answerable question of this type appears at stats.stackexchange.com/questions/35220. It reveals why this question must be made more specific, as requested by @Martijn. $\endgroup$ – whuber Sep 30 '19 at 14:20
  • 1
    $\begingroup$ Having worked as a submarine nuclear reactor operator in the U.S. Navy, I find your question to be clearly stated. My recommendation is to present the data in two parts: voltages within the normal range are not directly plotted, but rather displayed as a "within normal range" band horizontal line. These voltages are not of concern by definition, as they are normal. Only voltages outside the normal range are plotted by value. This will convert the plot from a display of "all voltages" to "abnormal voltages". You might allow choice of "normal range" with some preset default value. $\endgroup$ – James Phillips Sep 30 '19 at 15:27
  • 2
    $\begingroup$ So you have a slow-moving trend plus fast and sharp peaks. You could treat the two separately. Filter all the peaks and then display the relatively continuous part of the data by subsampling (or some other data reduction or filter method). Then put them together 'filtered trend + peaks'. Two questions: The peaks are always a single point? This is not clear from your graphs which seem to add more detail (some wavy curve), more than what 150 points provide. In addition, what features are important about the peaks, only the height, and not the width? $\endgroup$ – Sextus Empiricus Sep 30 '19 at 15:36
  • 1
    $\begingroup$ @MartijnWeterings I should have mentioned that the data in the plot were simulated. I should think real peaks are not always a single point, and their width does matter. And you're right about the wavyness. It's a feature of Google Charts. When disabled, the graph presents sharp edges. I also might have mentioned I have no experience with statistical methods whatsoever. How does one go about subsampling? $\endgroup$ – developer1405 Sep 30 '19 at 16:23