Questions tagged [statsmodels]

A Python module for exploration, testing, and estimation. Do not use this tag for general statistical modeling questions! Nb, questions only about the module itself, Python, or coding will likely be off topic.

Filter by
Sorted by
Tagged with
52
votes
2answers
30k views

Pandas / Statsmodel / Scikit-learn

Are Pandas, Statsmodels and Scikit-learn different implementations of machine learning/statistical operations, or are these complementary to one another? Which of these has the most comprehensive ...
45
votes
3answers
88k views

Logistic Regression: Scikit Learn vs Statsmodels

I am trying to understand why the output from logistic regression of these two libraries gives different results. I am using the dataset from UCLA idre tutorial, predicting ...
25
votes
3answers
71k views

Analyse ACF and PACF plots

I want to see if I am on the right track analysing my ACF and PACF plots: Background: (Reff: Philip Hans Franses, 1998) As both ACF and PACF show significant values, I assume that an ARMA-model ...
21
votes
4answers
29k views

Difference between statsmodel OLS and scikit linear regression

I have a question about two different methods from different libraries which seems doing same job. I am trying to make linear regression model. Here is the code which I using statsmodel library with ...
16
votes
2answers
15k views

Ordinal logistic regression in Python

I would like to run an ordinal logistic regression in Python - for a response variable with three levels and with a few explanatory factors. The statsmodels package ...
13
votes
4answers
16k views

Statsmodels says ARIMA is not appropriate because series is not stationary, how is it testing that?

I have a time series that I am trying to model with Python's statsmodels ARIMA api. When I apply the following: ...
12
votes
1answer
9k views

Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to ...
11
votes
1answer
9k views

Why does statsmodels.api.OLS over-report the r-squared value?

I am using statsmodels.api.OLS to fit a linear regression model with 4 input-features. The shape of the data is: ...
10
votes
2answers
8k views

Logistic regression with binomial data in Python

This is probably trivial but I couldn't figure it out. I want to fit a logistic regression model, where my dependent variable is not a Bernoulli variable, but a binomial count. Namely, for each $X_i$, ...
10
votes
1answer
2k views

Why is forecasting of ARMA models performed by Kalman filter

What are the advantages of expressing an ARMA model as a state-space-model and do forecasting using a Kalman filter? This methodology is for example used in the SARIMAX implementation of python-...
9
votes
1answer
8k views

Assessing the Contribution of each Predictor in Linear Regression

Say I build a linear regression model to identify linear dependencies between variables in my data. Some of these variables are categorical variables. If I want to evaluate the contribution of a ...
9
votes
2answers
1k views

Dummy/baseline models for time series forecasting

I am working on an evaluation of time series forecasting models in Python, more specifically with statsmodels, scikit-learn and tensorflow. I think it makes sense to first compare the model ...
9
votes
2answers
4k views

The identity link function does not respect the domain of the Gamma family?

I am using using a gamma generalized linear model (GLM) with an identity link. The independent variable is the compensation of a particular group. Python's statsmodels summary is giving me a warning ...
8
votes
2answers
6k views

Can we say 50% of data will be between 25th-75th percentile?

Let's say we have the following dataframe: ...
8
votes
1answer
21k views

Proving similarities of two time series

Let's assume an analytical model predicts an epidemic trend over time, i.e. number of infections over time. We also have a computer simulation results over time to verify the performance of the model. ...
6
votes
1answer
16k views

Testing for Granger Causality

I have two time series (Stocks and GDP) that I want to check for Granger causality. After reading the literature and documentations of various statistics software documentations (py statsmodels), I'm ...
6
votes
1answer
715 views

For random forest, what's the difference between out-of-bag error and k-fold cross validation?

I am trying to forecast a time series using random forest. In order to validate my model, I came cross two tests: OOB and K-fold cross validation. My question is: For a time series, is out-of-bag ...
6
votes
2answers
406 views

ARIMA model has trouble forecasting next month

I am implementing a statsmodels.tsa.arima_model.ARIMA model with daily advertising spend data. I have a dataset that ranges from Jan 1 2016 - present. Using a ...
6
votes
0answers
7k views

Assessing A/B test results using Python

Problem's setting Assume having results of an A/B test. You let you users experience two variants of your website and you counted how many converted: ...
5
votes
1answer
5k views

Wildly different $R^2$ between statsmodels linear regression and sklearn linear regression

My question is related to: Difference between statsmodel OLS and scikit linear regression I essentially have the same problem, except my results are even more substantially different. Performing ...
5
votes
2answers
4k views

Why is gradient descent so bad at optimizing polynomial regression?

As part of a self-study exercise, I am comparing various implementations of polynomial regression: Closed form solution Gradient descent with Numpy Scipy optimize Sklearn Statsmodel When the ...
5
votes
2answers
124 views

How to find the 95% confidence interval when there is outliers?

I know how to find the 95% confidence interval for normal distributions. But how to find it when there is outliers? Question: The health-care costs, in thousands of dollars for 20 males aged 75 or ...
5
votes
2answers
352 views

Specifying Mixed Effects Model Formulas

I have a dataset of experiment groups, different tests, schools, and students (note students are in the dataset multiple times due to multiple tests so they're not iid either) and want to specify a ...
5
votes
2answers
296 views

Heteroscedasticity in Linear Regression

I implemented a linear regression model on some dataset. When I plotted the scatter plot of residual v/s predicted y (i.e., yhat), I observed heteroscedasticity in the plot. What can I do about it? x-...
5
votes
2answers
321 views

Why do you set 1 as intercept in linear regression model in python?

I'm learning linear regression in Udacity as a beginner. I know statsmodels.regression.linear_model.OLS() needs an intercept but why do you set ...
5
votes
2answers
7k views

Partial correlation in panda dataframe python [closed]

I have a data in pandas dataframe like: ...
5
votes
2answers
3k views

How do I test how “good” my Poisson Model is?

Background: coding in Python, utilizing pandas, statsmodels and scikitlearn (the former ...
5
votes
1answer
10k views

How to optimise an automatic ARIMA-model selection?

I've been using statsmodels.tsa.arima_model to fit the residual component of some data. I've written an algorithm to automatically select the ARIMA model. Results ...
5
votes
1answer
965 views

LOWESS implementation in Stata vs. R (and Python)

Recently I was comparing the output of LOWESS regressions performed in R (and using Python's statsmodels module) and Stata. I realized that some of the values obtained by Stata seem to be off; ...
5
votes
1answer
170 views

Adjusting time-series before a sudden increase(the reason & time of the increase are known)

There are 2 increases in magnitude (one between feb and march, and one at the beginning of september) (the chart has daily resolution btw) Assuming weekly seasonality, is it ok to get the exact ...
5
votes
2answers
14k views

Standard error of regression and of predictions in python (these are available in R) [closed]

I'm working with R and confirming my results in Python with the overwhelming majority of the work matching between the two quite well. There are two outputs coming out of R that I'm not seeing how to ...
5
votes
0answers
128 views

Why does uncertainty of the autocorrelation coefficient increase as lag increases?

The Python module statsmodels contains functions for ACF and PACF. Below is an example from the docs with a plot that shows the (zero-centered) confidence ...
5
votes
0answers
1k views

statsmodels: quantreg convergence cycle warning

I am getting the same Convergence cycle detected warning running a quantile regression with ...
4
votes
2answers
543 views

Gamma GLM - Derive prediction intervals for new x_i

In a Gamma GLM, the statistical model for each observation 𝑖 is assumed to be $Y_i \sim Gamma(shape, scale)$, where $E(Y_i) = \mu_i = f(X_i\beta)$, and $f$ is the link function. I've used MLE to ...
4
votes
2answers
779 views

Does the dependent variable in a GLM have to be transformed before running the model or does the model do it?

I'm trying to fit a Tweedie model with statsmodels and was wondering if I have to transform the dependent variable before I run the model or if ...
4
votes
1answer
5k views

Multinomial logit: mlogit vs statsmodels

I am doing a comparison between mlogit in R and statsmodels in python and have had trouble getting them to produce the same result. I'm wondering if the difference is a result of libraries or I am ...
4
votes
1answer
89 views

Python Mixed Linear Model Regression Interpretation of the Results

I cannot seem to find a proper guide on how to interpret the results from a Mixed Linear Model Regression. ...
4
votes
1answer
497 views

Interpreting the intercept of a Linear Mixed Model Results in Python - Statsmodel Package

Using python package statsmodel and the code in this link: If a linear mixed model has a random variable with x groups. then why when one would run this code: ...
4
votes
1answer
8k views

ARIMA with seasonality in Statsmodels

I'd like to have a seasonal ARIMA model implemented with Statsmodels ARIMA. Specifically, I'd like to log before the weekly seasonality and then be able to make forecasts. Perhaps an example with ...
4
votes
1answer
58 views

Appropriate Distribution for Survival Probability Parameters

I have a model that assumes a probability of survival over discrete time (example: decades) ...
4
votes
1answer
1k views

“Zero-lag” for ARMA model

I am reading a statsmodels tutorial for generating and fitting ARMA models here. The author says The conventions of the arma_generate function require that we specify a 1 for the zero-lag of the AR ...
4
votes
2answers
6k views

Understand Summary from Statsmodels' MixedLM function

I am using MixedLM to fit a repeated-measures model to this data, in an effort to determine whether any of the treatment time points is significantly different from the others. I get the following ...
4
votes
0answers
5k views

ARMA Model fitting using statsmodels.tsa.ARMA()

Two questions. 1.) When I use the statsmodels.tsa.ARMA() module, I enter my parameters and fit a model as follows: ...
3
votes
2answers
105 views

Can't decide if my data is normally distributed

I have a problem comparing results from shapiro test and qqplot. Shapiro tells me my data doesn't have normal distribution characteristics (pvalue = 1.94...e-08 <= 0.05) however when I look on QQ ...
3
votes
5answers
138 views

is logistic regression stochastic like neural network?

I have observed that neural network models (using Keras TensorFlow) can be very unstable (when my sample size is small) in the sense that if I were to train 999 NN models, there might only be 99 with ...
3
votes
2answers
5k views

Different output for R lm() and python statsmodel OLS for linear regression

I'm exploring linear regressions in R and Python, and usually get the same results but this is an instance I do not. I added the sum of Agriculture and ...
3
votes
2answers
172 views

Unable to recreate Statsmodels ARIMAX (1, 1, 0) forecasts by hand

I have fit an ARIMAX (1, 1, 0) model to a timeseries dataset consisting of 1 endogenous timeseries ("Y") and 1 exogenous timeseries ("X"). My exogenous timeseries in the model was ...
3
votes
1answer
1k views

Apply statsmodels PCA to new data

I am working on a personal project, and I want to use Statsmodels' PCA on a dataset. The ultimate goal is to then perform a linear regression and evaluate its prediction. I know scikit-learn may be ...
3
votes
2answers
641 views

How can I program correction for multiple comparisons using lsmeans in R while not correcting for more than necessary

I posted this question on stack overflow here enter link description here but did not get useful feedback. So, I'll post it again here, in the hopes that this is a better venue for it. I'm wondering ...
3
votes
2answers
3k views

SciPy chisquare applied on continuous data?

chi2 seems to be used to compare distributions between categorical features. Is it possible to use SciPy chisquare function to compare two distributions of continuous data ? For example : ...

1
2 3 4 5 6