2
$\begingroup$

Below is a pair plot of the types of distributions (Time Series) I've been attempting to run models upon. Two of the features are strongly collinear (the distributions of last 2 on the diagonal of the plot) and are heavily positively skewed shown below along with another fairly heavily negatively skewed distribution (the 2nd on the diagonal).

Enclosed is the data I'm working with:

https://drive.google.com/file/d/1A9zxOSV3WJZIfS6qVKZIYwNnNa65O-Q3/view

enter image description here

enter image description here

I've run a few attempts of multiple linear regression, ridge, random forest, and have tried log transforming the data to explore the behaviour better.

The coefficients of determination are far from what we'd like (~0.31) running on Python's Statsmodel OLS + sklearn regressors. I've also tried modeling the features separately to the target feature with just as bad (if not worse) outcomes.

When dealing with this type of situation is the best way to deal with it to use non log transformations to some of the features and not the others? I've also thought about possibly breaking up the models into 3 thresholds to break the skewedness.

I've also thought about looking more into Gumbel distributions or Intensity Duration Frequency modeling as those seem to deal directly with these types of scenarios and require a great deal of research on my part (willing to do if it's a step in the right direction).

Any suggestions on how to proceed with these distributions?

enter image description here

$\endgroup$
4
  • 1
    $\begingroup$ You might find some useful suggestions for distributions using my online Python open source statistical distribution fitter, it fits data to over 80 of the continuous distributions in scipy. The URL is zunzun.com/StatisticalDistributions/1 $\endgroup$ Commented Oct 30, 2018 at 20:17
  • $\begingroup$ Cool but not really what I'm looking for. $\endgroup$ Commented Oct 30, 2018 at 22:03
  • $\begingroup$ These are time series? $\endgroup$
    – Glen_b
    Commented Oct 30, 2018 at 22:04
  • $\begingroup$ Yes. Editing to include that information. Thank you. $\endgroup$ Commented Oct 30, 2018 at 22:07

1 Answer 1

0
$\begingroup$

enter image description hereThe whole idea is that the residuals from a useful model are roughly normal. The distribution of the observed series is not an issue. Post an example of your time series data and I will try and help further, See stats.stackexchange.com/questions/18844/… for a discussion of time series data , outlier adjustment and power transforms –

For example your first histogram might simply be reflecting day-of-the-week effects . After adjusting for day-of-the-week effects the conditional histogram ( i.e. the errors from the fixed effects model ) might be quite normal or nearly so.

AFTER RECEIPT OF DATA:

I submitted your data to AUTOBOX which has an automatic transfer function modelling option and obtained enter image description here and enter image description here . The residuals from this model are here enter image description here and here enter image description here . The forecasts are here for the next 12 periods enter image description here

The Actual/Fit and Forecast graph is here enter image description here

$\endgroup$
10
  • $\begingroup$ Thanks, @IrishStat. I've shared the time series I'm interested in as a Google drive document. $\endgroup$ Commented Oct 30, 2018 at 22:34
  • $\begingroup$ where is the data for month 1 of each year ? Are you trying to analyze/predict each of these series separately ? $\endgroup$
    – IrishStat
    Commented Oct 31, 2018 at 0:21
  • $\begingroup$ I must have skipped them on accident. I can correct them later. I'd like to predict Energy on all 5 of the other features. I'd also like to build a correlation matrix and check on feature importances at some point. $\endgroup$ Commented Oct 31, 2018 at 1:36
  • $\begingroup$ So you wish to predict energy using the other 5 . I suggest that you code the data and if you don't I subsequently will.as that is good practice. $\endgroup$
    – IrishStat
    Commented Oct 31, 2018 at 2:42
  • $\begingroup$ Predict energy market values of Europe based off the population, energy industry index, economy, and two air pollutant counts. High importance and interest on air pollutant counts correlations. $\endgroup$ Commented Oct 31, 2018 at 3:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.