Tagged Questions

Problems focusing on (but not necessarily limited to) using Python for statistical computation.

learn more… | top users | synonyms

1
vote
1answer
23 views

Simple model selection example in PYMC

I am currently experimenting with PYMC and I am trying out a simple example so that I start learning how things work (I am also a Python beginner, but an experienced machine learner). I have set ...
1
vote
1answer
37 views

What are some useful robust and scalable approaches towards anomaly detection of a time series data?

What are some useful robust and scalable approaches/methods towards anomaly detection of a time series data? I am mainly looking for some practical approaches carried out using Python, R, Java, etc. ...
0
votes
0answers
19 views

How to plot a precision-recall curve when doing cross-validation?

I'm using cross-validation to evaluate the performance of a classifier with scikit-learn and I want to plot the Precision-Recall curve. I found an example on ...
0
votes
0answers
6 views

Borgatti Key Player Problem (KPP) in Python

Has anyone come across any Python implementation of Borgatti's proposed Key Player Problem (KPP) algorithms? I'm interested in solutions using NetworkX and particularly interested in implementations ...
0
votes
0answers
19 views

Given a covariance matrix from a Linear regression, how do I calculate the standard error of the coefficients?

I have an OLS with autocorrelation in the residuals. I'm using python statsmodels, and found that there is the sandwich_covariance matrix, which can cal Reference to Newey-West covariance matrix: ...
0
votes
1answer
28 views

Convert Daily Data to Monthly Data in Python : Time Series Analysis

I am new to data analysis with python. I have daily data of flu cases for a five year period which I want to do Time Series Analysis on. Am using the Pandas library. It is easy to plot this data and ...
1
vote
0answers
10 views

Approximating cox model with time varying covariates using poisson

How do you reformat a dataset in order to perform a cox regression with time-varying covariates as a poisson regression. I'm trying to run a survival analysis regression in python with time varying ...
0
votes
0answers
14 views

What is the best way to do a seasonal ARMA (or ARIMA) in python?

Scikit learn and statsmodels don't seem to support this type of ARMA. I tried to use rpy2 python library, but that proved to be far too difficult to integrate, as my IDE was not able to recognize my ...
1
vote
2answers
47 views

Time Series Anomaly Detection with Python

I need to implement anomaly detection on several time-series datasets. I've never done this before and was hoping for some advice. I'm very comfortable with python, so I would prefer the solution be ...
0
votes
0answers
11 views

Slow Lasso Performance Using sklearn

I am trying to use scikit-learn's LassoCV and/or ElasticNetCV functions to model a dataset with a large (>800) number of ...
0
votes
0answers
27 views

Which naive Bayes?

I am attempting to use a naïve Bayes classifier in python (using scikit-learn), with two examples. The first example has 6 classes and 2 hypotheses, the 2nd example has 2 classes and 6 hypotheses. ...
1
vote
0answers
22 views

A log-normal distribution in Python

I have seen several questions in stackoverflow regarding how to fit a log-normal distribution. Still there are two clarifications that I need known. I have a ...
0
votes
0answers
12 views

Weka SMOreg and LIBSVM with linear kernel problems

I want to test a dataset in weka using either LIBSVM with an e-SVR or SMOreg for regression. I also choose a linear kernel in both (in SMOreg i use an exponent=1 in a non normalized polykernel). ...
0
votes
0answers
17 views

Inverse Box-Cox transform in Python

I am using SciPy's boxcox function to perform a Box-Cox transformation on a continuous variable. ...
1
vote
0answers
69 views

Any advice on how to improve my accuracy rate in text classification?

I'm trying to do a text classification task. Here are some specs: Context file size = 1M+ documents already labeled Number of top-labels = 17 Number of sub-labels = around 130 Each document is ...
4
votes
1answer
61 views

How to interpret autocorrelation plot in MCMC

I am getting familiar with Bayesian statistics by reading the book Doing Bayesian Data Analysis, by John K. Kruschke also known as the "puppy book". In chapter 9, hierarchical models are introduced ...
3
votes
0answers
25 views

Fitting Multivariate Bernoulli distribution

I want to fit a model to a number of observations, each of them being a k-dimensional binary vector $(x_1, x_2, ..., x_k)$ where $x_i \in \{0,1\}$. Naturally I would like to fit a multivariate ...
0
votes
1answer
28 views

what does a negative logloss value indicate

I am using logloss python function provided here and I am getting results as -2.99 when I use a machine learning algorithm on my dataset. What does that mean? The algorithm's predictions are bad (or) ...
0
votes
0answers
47 views

Python: Compute the Generalized Impulse Responses

I am using the tsa.vector_ar in Python, StatsModel Package and when I do a Forecast Error Variance Decomposition using the command ...
1
vote
0answers
19 views

Align two datasets in Python

I want to develop some python code to align datasets obtained by different instruments recording the same event. As an example, say I have two sets of measurements: ...
0
votes
0answers
40 views

How can you print the decision tree of a RandomForestClassifier

Recently, I have noticed that there is a method sklearn.tree.export_graphviz documented here. However, I do not know how I can apply it to a ...
1
vote
2answers
149 views

very high frequency time series analysis (seconds) and Forecasting (Python/R)

I have high frequency data (observations separated by seconds), which I'd like to analyse and eventually forecast short-term periods (1/5/10/15/60 min ahead) using ARIMA models. My whole data set is ...
0
votes
0answers
20 views

How do I get a classification report for my cross validated scores using sklean

I am running a logistic regression model using sklearn with 2 classes (1 and 0). Here is my code: ...
0
votes
0answers
17 views

How can I improve my sklearn logistic regression model

My objective is to classify sentences into useful (denote in boolean as 1) and not useful (denote in boolean as 0) categories. I have about 525 features where 300 features are the most frequent and ...
0
votes
0answers
63 views

Anyone use R or Python as statistical tool in industry? How to get coefficients of gradient booting models?

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the coefficients of the model. For gbm in R, it seems one can get the tree structure, ...
0
votes
1answer
32 views

what does the numbers in the classification report of sklearn mean?

I have below an example i pulled from sklearn 's sklearn.metrics.classification_report documentation. What i dont understand is why there are f1-score, precision and recall values for each class ...
1
vote
1answer
33 views

The likelihood that a time series is generated by certain ARMA(p,q) ?

I have a group ( only 20 of them, each one has 170 time pointers) of time series that I can consider as "GOOD", meaning, they have consistent statistical characteristics. I am not sure how they are ...
1
vote
1answer
20 views

Logistic-Regression: Prior correction at test time

Using sklean.linear_model.LogisticRegression for a binary classification problem. My classes are unbalanced. The positive class comprises about 20% of the training set. When fitting the model I use: ...
0
votes
0answers
9 views

Is it possible to create the data object in Python for SPSS [migrated]

I have a python script that is reading in an XML file into an array (in a CSV format I created). I'd like to be able to use that data directly instead of saving to a file. Is this possible? So it ...
0
votes
0answers
4 views

Printing nested lists using nested functions [migrated]

n = [1, [2, [3, [4, 5]]], [6, [7, [8, [9]], 10]]] In python, I want to write a nested function within the function 'print_list' that calls itself recursively for each sub-list. The nested function ...
4
votes
2answers
100 views

Tournament Plotting: Who is good against whom?

I would like to get insight into who is good against whom. Imagine a tournament setting* of 40 contestants and you're interested in seeing who is good against who in the top 10: how could we make this ...
2
votes
2answers
133 views

Different output for R lm() and python statsmodel OLS for linear regression

I'm exploring linear regressions in R and Python, and usually get the same results but this is an instance I do not. I added the sum of Agriculture and ...
0
votes
0answers
24 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...
0
votes
0answers
53 views

More recognizable Python implementation of Linear Discriminant Analysis?

I have been using scikit-learn's LDA implementation to do some experiments, and recently wanted to test out some modifications to the LDA derivation. I was looking at the Python implementation that ...
1
vote
0answers
46 views

Proper normalized cross-correlation

I'm confused with the widely used approach to compute the normalized cross-correlation which is something like this: Standardize the argument vectors (of equal length $n$). Slide one over the other ...
1
vote
0answers
40 views

Bias-variance decomposition with sklearn BaggingRegressor

There is an example given on the Scikit-Learn site that compares the bias-variance decomposition of the rmse of a single SVR model against a bagging ensemble. Unfortunately, the data is being ...
1
vote
0answers
19 views

How do I filter insignificant modes from a kernel density estimation

How do I detect the number of significant modes in a set of data? I have a set of 1 dimensional data where the number of modes are unknown. I'm familiar with python and scikit learn so I use the ...
0
votes
0answers
16 views

converting textblob object into DataFrame

Can we convert blob.noun_phrases into Pandas's DataFrame? The data type of blob.noun_phrases is class 'textblob.blob.WordList' type(blob.noun_phrases) class ...
0
votes
0answers
38 views

Plot 2D view of 3D joint ecdf with isolines

I have this matrix, histDict, which is a dictionary of data of variable X for each value of variable ...
4
votes
1answer
131 views

How to apply a Gaussian radial basis function kernel PCA to nonlinear data?

I have an assignment to implement a Gaussian radial basis function-kernel principal component analysis (RBF-kernel PCA) and have some challenges here. It would be great if someone could point me to ...
0
votes
0answers
27 views

Python module request: Spectral density estimation for multivariate time series

I have worked with scipy.signal.welch and spectrum.pptm to calculate power spectral density with Welch and Multitaper methods. However as far as I can see these functions are meant for one dimensional ...
0
votes
0answers
36 views

Time-series detection algorithm for multi-seasonal data using Python

My data: I have two seasonal patterns in my hourly data... daily and weekly. For example... each day in my dataset has roughly the same shape based on hour of the day. However, certain days like ...
2
votes
1answer
45 views

PYMC Confusion: are observed nodes fixed or stochastic?

I've been trying to gain a better understanding of factor potentials in PYMC. In reading this article by Cam Davidson-Pilon on Yhat, I got confused about how observed nodes are understood by PYMC. ...
0
votes
0answers
17 views

Choosing between different methods when the first one raises error message for linear regression

I have a linear regression problem $$Ax=b$$ My initial approach that helped to solve some of my questions was using SVD and obtaining the chi-square and some other values that I am interested but it ...
1
vote
0answers
48 views

Optimization in R vs Python, constrained, unconstrained and automatic differentiation?

I am an economics/stat guy who uses quite a bit of optimization (maximum likelihood, simulated maximum likelihood), constrained optimization (mathematical programming w/ equilibrium conditions), ...
1
vote
1answer
38 views

How to sample from “biased” binomial distribution (ideally in python/numpy)

I need to sample a variable from a distribution that's like a binomial distribution except with a "bias", I'm not sure what it may be called: $p(X=k)$ is proportional to $k.B(n,p)(k)$ where $B(n,p)$ ...
0
votes
1answer
36 views

Autoclass in R/Python? [closed]

Are there any packages that implement the Autoclass/ Naive Bayes Clustering algorithm in R or Python? Alternatively, what are some other clustering algorithms that can handle both categorical and ...
0
votes
0answers
14 views

PyMC: Which parameter is passed to the model by sampler? How to find the value passed by custom sampler?

I am relatively new for PyMC and as a whole I have some less knowledge in probability. As far as I understand MCMC algorithm, its main task is to propose samples of parameters to the model (in ...
0
votes
1answer
76 views

Defining the Stochastic and Deterministic variables with pymc3

I am trying to use write my own stochastic and deterministic variables with pymc3, but old ...
0
votes
0answers
63 views

Numerical estimation of MLE in Python — normal distribution and gradient is close to zero away from the mean

I am exploring how to model a data set using normal distributions with both mean and variance defined as linear functions of independent variables. Something like $\mathcal{N} \sim \left (f(x), ...