Problems focusing on (but not necessarily limited to) using Python for statistical computation.

learn more… | top users | synonyms

0
votes
0answers
5 views

Fit multivariate gaussian distribution to a given dataset [on hold]

I need to fit multivariate gaussian distribution i.e obtain mean vector and covariance matrix of the nearest multivariate gaussian for a given dataset of audio features in python. The audio features ...
1
vote
1answer
33 views

Classification score for Random Forest

I'm learning about the Decision Tree and Random Forests. But there is something I don't really understand. I have a training set and a cross-validation set. I need to train different Random Forests, ...
0
votes
0answers
13 views

Trend analysis on time intervales

I am trying to do trend analysis on tweets. I am still confused. The classical model is based on computing the frequency of items, to say that the item with big number of frequency is the trending ...
0
votes
1answer
19 views

How to find optimal penaltyparameter C for SVM (regression)

I am training an svm regressor using python sklearn.svm.SVR From the example given on the sklearn website, the above line of code defines my svm. ...
0
votes
0answers
11 views

Python “feature_importances” for most important factors

I'm a little unsure as whether this belongs in stackoverflow or cross validated. I have found a few posts on this topic , but I have not found the following question. Is it accurate to run the feature ...
0
votes
0answers
26 views

lowess: R (& python-statsmodels) vs MATLAB (& biopython)

There seems to be two different interpretations of what LOWESS really means: one from R (also used by python-statsmodels), and one from MATLAB (also used by biopython), see comparison below. Could ...
2
votes
2answers
53 views

Reproduce Cross Correlation results in Python

I'm trying and failing to reproduce the results found in this paper, where the cross correlation between two distributions is calculated: ...
0
votes
0answers
4 views

Is there a way to get the p-values/standard errors via a function in scikit learn?

Can't seem to figure this out very easily. I am looking for a similar output as that of statsmodels; however, statsmodels doesn't support some of the regressions that I want to do. I basically want ...
0
votes
0answers
18 views

Regression Analysis with String values in column of DataFrame

I have a dataset which is having multiple columns where some column has string values with comma seperated ...
0
votes
1answer
13 views

Scikit-learn model representation

Suppose, i have some models created and fitted in scikit-learn. ...
0
votes
0answers
16 views

NLP- Sentiment analysis

Most of solutions are based on training classifier to detect the sentiment of text. I am trying to analyze political reviews, I don't wanna divide my data on train/Test. What is the approach that ...
0
votes
0answers
9 views

Obtain the graph of the autocorrelation function in ARIMA models [migrated]

I am implementing an ARIMA model in Python for forecasting U.S. GDP. I am interested in obtaining the graph for the autocorrelation function. I obtained the values for ACF but I can not see the ...
0
votes
0answers
13 views

Convert time scalar field to velocity vector field

I am trying to figure out if it's possible to create vector velocity field from some raster containing spatial distribution of scalar variable (timestamp of some mass-media event in my case). Here is ...
0
votes
0answers
16 views

Using PyBrain after training a network

I'm using PyBrain to create a neural network. I'm still pretty new to neural networks and their concepts. I've so far only run train() over the network, as ...
0
votes
2answers
58 views

two samples z test in Python [closed]

Does Scipy have a z test to compare the mean of two samples? I searched this page but couldn't find one.
0
votes
0answers
30 views

Random Forest Regressor - Incorporating Sample Weights in scikit-learn

I am running randomforestregressor in python. The target variable I am modeling is the frequency of an event occurring per unit of time. Each record in my data includes whether or not the event ...
0
votes
2answers
72 views

Predict interesting articles: increase accuracy

I'm trying to write a gui to display articles, and predict which articles I could like, based on the articles I previously liked. This post is the continuation of this one: ...
0
votes
0answers
24 views

How to fit an equation to a regression (and get coefficients) using scikit or statsmodels?

This may be simple but I can't find out how to do this. Suppose I have the following equation: f = a*(1+e^(c*x))^(-1) + by + d f, x, y, and d are vectors. I want to get estimates of the ...
0
votes
0answers
18 views

How do I implement custom sparse connections in a neural network?

We'd like to implement neural network which is not fully connected - we want to explicitly set which output neurons connect to which input neurons (there's no hidden layer). We use Theano but we can ...
1
vote
1answer
73 views

Simple model selection example in PYMC

I am currently experimenting with PYMC and I am trying out a simple example so that I start learning how things work (I am also a Python beginner, but an experienced machine learner). I have set ...
1
vote
1answer
72 views

What are some useful robust and scalable approaches towards anomaly detection of a time series data?

What are some useful robust and scalable approaches/methods towards anomaly detection of a time series data? I am mainly looking for some practical approaches carried out using Python, R, Java, etc. ...
0
votes
0answers
42 views

How to plot a precision-recall curve when doing cross-validation?

I'm using cross-validation to evaluate the performance of a classifier with scikit-learn and I want to plot the Precision-Recall curve. I found an example on ...
0
votes
0answers
11 views

Borgatti Key Player Problem (KPP) in Python

Has anyone come across any Python implementation of Borgatti's proposed Key Player Problem (KPP) algorithms? I'm interested in solutions using NetworkX and particularly interested in implementations ...
0
votes
0answers
31 views

Given a covariance matrix from a Linear regression, how do I calculate the standard error of the coefficients?

I have an OLS with autocorrelation in the residuals. I'm using python statsmodels, and found that there is the sandwich_covariance matrix, which can cal Reference to Newey-West covariance matrix: ...
0
votes
1answer
61 views

Convert Daily Data to Monthly Data in Python : Time Series Analysis

I am new to data analysis with python. I have daily data of flu cases for a five year period which I want to do Time Series Analysis on. Am using the Pandas library. It is easy to plot this data and ...
1
vote
0answers
20 views

Approximating cox model with time varying covariates using poisson

How do you reformat a dataset in order to perform a cox regression with time-varying covariates as a poisson regression. I'm trying to run a survival analysis regression in python with time varying ...
0
votes
1answer
67 views

What is the best way to do a seasonal ARMA (or ARIMA) in python?

Scikit learn and statsmodels don't seem to support this type of ARMA. I tried to use rpy2 python library, but that proved to be far too difficult to integrate, as my IDE was not able to recognize my ...
1
vote
2answers
109 views

Time Series Anomaly Detection with Python

I need to implement anomaly detection on several time-series datasets. I've never done this before and was hoping for some advice. I'm very comfortable with python, so I would prefer the solution be ...
0
votes
0answers
18 views

Slow Lasso Performance Using sklearn

I am trying to use scikit-learn's LassoCV and/or ElasticNetCV functions to model a dataset with a large (>800) number of ...
0
votes
0answers
37 views

Which naive Bayes?

I am attempting to use a naïve Bayes classifier in python (using scikit-learn), with two examples. The first example has 6 classes and 2 hypotheses, the 2nd example has 2 classes and 6 hypotheses. ...
1
vote
0answers
41 views

A log-normal distribution in Python

I have seen several questions in stackoverflow regarding how to fit a log-normal distribution. Still there are two clarifications that I need known. I have a ...
0
votes
0answers
26 views

Weka SMOreg and LIBSVM with linear kernel problems

I want to test a dataset in weka using either LIBSVM with an e-SVR or SMOreg for regression. I also choose a linear kernel in both (in SMOreg i use an exponent=1 in a non normalized polykernel). ...
0
votes
0answers
30 views

Inverse Box-Cox transform in Python

I am using SciPy's boxcox function to perform a Box-Cox transformation on a continuous variable. ...
1
vote
0answers
84 views

Any advice on how to improve my accuracy rate in text classification?

I'm trying to do a text classification task. Here are some specs: Context file size = 1M+ documents already labeled Number of top-labels = 17 Number of sub-labels = around 130 Each document is ...
4
votes
1answer
89 views

How to interpret autocorrelation plot in MCMC

I am getting familiar with Bayesian statistics by reading the book Doing Bayesian Data Analysis, by John K. Kruschke also known as the "puppy book". In chapter 9, hierarchical models are introduced ...
3
votes
0answers
40 views

Fitting Multivariate Bernoulli distribution

I want to fit a model to a number of observations, each of them being a k-dimensional binary vector $(x_1, x_2, ..., x_k)$ where $x_i \in \{0,1\}$. Naturally I would like to fit a multivariate ...
0
votes
1answer
39 views

what does a negative logloss value indicate

I am using logloss python function provided here and I am getting results as -2.99 when I use a machine learning algorithm on my dataset. What does that mean? The algorithm's predictions are bad (or) ...
0
votes
0answers
52 views

Python: Compute the Generalized Impulse Responses

I am using the tsa.vector_ar in Python, StatsModel Package and when I do a Forecast Error Variance Decomposition using the command ...
1
vote
0answers
26 views

Align two datasets in Python

I want to develop some python code to align datasets obtained by different instruments recording the same event. As an example, say I have two sets of measurements: ...
0
votes
1answer
78 views

How can you print the decision tree of a RandomForestClassifier

Recently, I have noticed that there is a method sklearn.tree.export_graphviz documented here. However, I do not know how I can apply it to a ...
1
vote
2answers
234 views

very high frequency time series analysis (seconds) and Forecasting (Python/R)

I have high frequency data (observations separated by seconds), which I'd like to analyse and eventually forecast short-term periods (1/5/10/15/60 min ahead) using ARIMA models. My whole data set is ...
0
votes
0answers
26 views

How do I get a classification report for my cross validated scores using sklean

I am running a logistic regression model using sklearn with 2 classes (1 and 0). Here is my code: ...
0
votes
0answers
29 views

How can I improve my sklearn logistic regression model

My objective is to classify sentences into useful (denote in boolean as 1) and not useful (denote in boolean as 0) categories. I have about 525 features where 300 features are the most frequent and ...
0
votes
0answers
78 views

Anyone use R or Python as statistical tool in industry? How to get coefficients of gradient booting models?

I tried gradient boosting models using both gbm in R and sklearn in Python. However, neither of them can provide the coefficients of the model. For gbm in R, it seems one can get the tree structure, ...
0
votes
1answer
63 views

what does the numbers in the classification report of sklearn mean?

I have below an example i pulled from sklearn 's sklearn.metrics.classification_report documentation. What i dont understand is why there are f1-score, precision and recall values for each class ...
1
vote
1answer
34 views

The likelihood that a time series is generated by certain ARMA(p,q) ?

I have a group ( only 20 of them, each one has 170 time pointers) of time series that I can consider as "GOOD", meaning, they have consistent statistical characteristics. I am not sure how they are ...
1
vote
1answer
28 views

Logistic-Regression: Prior correction at test time

Using sklean.linear_model.LogisticRegression for a binary classification problem. My classes are unbalanced. The positive class comprises about 20% of the training set. When fitting the model I use: ...
4
votes
2answers
103 views

Tournament Plotting: Who is good against whom?

I would like to get insight into who is good against whom. Imagine a tournament setting* of 40 contestants and you're interested in seeing who is good against who in the top 10: how could we make this ...
2
votes
2answers
207 views

Different output for R lm() and python statsmodel OLS for linear regression

I'm exploring linear regressions in R and Python, and usually get the same results but this is an instance I do not. I added the sum of Agriculture and ...
0
votes
0answers
36 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...