I am working on simple statistical prediction. However, whatever i did, i come up with a high range. I have used standard deviation and also percentile. Yes they work well in the given data but the outcome has very wide range.

Now my question is, say i have a perfect bell shaped curve. Does it mean that the data is random and can not be predictable? Note that I'm not using any ML or AI models.

I am trying to predict income of a coffee shop. And I'm trying to predict next days or any given day's income.

Here's the sample data i have:

    Day  Income
0     0  -192.0
1     1  1632.0
2     2 -1368.0
3     3   456.0
4     4  1608.0
5     5   792.0
6     6    48.0
7     0 -2352.0
8     1 -1224.0
9     2 -1224.0
10    3   936.0
11    4   -48.0
12    5    72.0
13    6  -576.0
14    0 -1728.0
15    1   528.0
16    2 -1464.0
17    3  2664.0
18    4  -480.0
19    5 -1896.0
20    6 -4824.0
21    0    72.0
22    1 -1440.0
23    2 -1488.0
24    3  1224.0
25    4  -984.0
26    5  1032.0
27    6 -2640.0
28    0    96.0
29    1  -816.0

I have excluded salaries but added all other expenses. And here's how returns look: PS: First 3 years data not included and I only have returns data.

enter image description here

As you can see, the returns have perfect bell shape. So business is not doing great i think. Anyways, when i use standard deviation, i am getting a very wide range. Same with using percentile. Is there a better proven way to make this predictions? Also when i apply ML (Linear Regression), Most of the time i am getting closer value of previous day. I think because of normal distribution.

I am also adding the description of the dataset:

count     4974.000000
mean        23.815199
std       1793.574762
min      -8376.000000
25%       -960.000000
50%         24.000000
75%       1008.000000
max      11832.000000

As you know that 68% of data is within 1 std and all i can say is that; there's 68% probability that profit (or loss) will be between -1769.76 / 1817.38 which doesn't make so much sense.

What else can i say:

1. 50% the return will be more or less then $24
2. 75% will be more than -$960 and less than $1008
3. Maximum expected loss is less than -$8376
4. Maximum expected profit is less than $11832

How exactly can you analyse such data? Or what else can you say using this data?

Thank you very much in advance

  • $\begingroup$ Did you look at your auto-covariance function? Using for instance R's acf() function. There may be some covariance structure in your data, to help you predict next day's income. $\endgroup$ – wiwh Oct 13 at 17:30
  • $\begingroup$ Thank you for your recommendation. No I haven't checked that. Let me see if i can find any ways to check it in python. $\endgroup$ – Don Coder Oct 13 at 17:34
  • $\begingroup$ In a large sample from a 'perfect' normal dist'n Q1 and Q3 should be at about $\bar X \pm .67S.$ Your dist'n may have a higher than "normal" SD. I don't know what prediction method you're using, but that could account for the 'wide range' in your results. // Have you tried making a 'density' histogram of your data and seeing how well a normal dist'n with $\mu = 24, \sigma=1800$ fits? // I guess 'Day' in your data is day of week (Mon, Tues,...). Do you take day of wk into account when trying to predict next day's performance? // Weekly predictions might work better than daily. Still useful? $\endgroup$ – BruceET Oct 13 at 18:04
  • $\begingroup$ @BruceET I have not used any predictions method. All I have used is to calculate std and percentile. So nothing else is done here. I think that most of predictions method ends up giving a value close to mean. It's more like a guessing game. I couldn't find a way to predict more accurately. $\endgroup$ – Don Coder Oct 14 at 7:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.