Problems focusing on (but not necessarily limited to) using Python for statistical computation.

learn more… | top users | synonyms

0
votes
0answers
9 views

Is it possible to create the data object in Python for SPSS [migrated]

I have a python script that is reading in an XML file into an array (in a CSV format I created). I'd like to be able to use that data directly instead of saving to a file. Is this possible? So it ...
0
votes
0answers
4 views

Printing nested lists using nested functions [migrated]

n = [1, [2, [3, [4, 5]]], [6, [7, [8, [9]], 10]]] In python, I want to write a nested function within the function 'print_list' that calls itself recursively for each sub-list. The nested function ...
3
votes
3answers
91 views

Tournament Plotting: Who is good against whom?

I would like to get insight into who is good against whom. Imagine a tournament setting* of 40 contestants and you're interested in seeing who is good against who in the top 10: how could we make this ...
2
votes
2answers
75 views

Different output for R lm() and python statsmodel OLS for linear regression

I'm exploring linear regressions in R and Python, and usually get the same results but this is an instance I do not. I added the sum of Agriculture and ...
0
votes
0answers
11 views

unary classification in PyBrain

I've just started using PyBrain for some data classification work, and I've gotten it working pretty well where I have data from all possible classes and I can train the network using all the classes. ...
0
votes
0answers
29 views

More recognizable Python implementation of Linear Discriminant Analysis?

I have been using scikit-learn's LDA implementation to do some experiments, and recently wanted to test out some modifications to the LDA derivation. I was looking at the Python implementation that ...
1
vote
0answers
32 views

Proper normalized cross-correlation

I'm confused with the widely used approach to compute the normalized cross-correlation which is something like this: Standardize the argument vectors (of equal length $n$). Slide one over the other ...
1
vote
0answers
29 views

Bias-variance decomposition with sklearn BaggingRegressor

There is an example given on the Scikit-Learn site that compares the bias-variance decomposition of the rmse of a single SVR model against a bagging ensemble. Unfortunately, the data is being ...
1
vote
0answers
16 views

How do I filter insignificant modes from a kernel density estimation

How do I detect the number of significant modes in a set of data? I have a set of 1 dimensional data where the number of modes are unknown. I'm familiar with python and scikit learn so I use the ...
0
votes
0answers
11 views

converting textblob object into DataFrame

Can we convert blob.noun_phrases into Pandas's DataFrame? The data type of blob.noun_phrases is class 'textblob.blob.WordList' type(blob.noun_phrases) class ...
-1
votes
0answers
30 views

tutorial on sampling methods and MC

I'm looking for good tutorials that cover the various sampling methods: simple sampling, MCMC, Gibbs Sampling, and Metropolis Hastings Algorithm. I barely know what is an MCMC. I would like to learn ...
0
votes
0answers
26 views

Plot 2D view of 3D joint ecdf with isolines

I have this matrix, histDict, which is a dictionary of data of variable X for each value of variable ...
4
votes
1answer
78 views

How to apply a Gaussian radial basis function kernel PCA to nonlinear data?

I have an assignment to implement a Gaussian radial basis function-kernel principal component analysis (RBF-kernel PCA) and have some challenges here. It would be great if someone could point me to ...
0
votes
0answers
19 views

Python module request: Spectral density estimation for multivariate time series

I have worked with scipy.signal.welch and spectrum.pptm to calculate power spectral density with Welch and Multitaper methods. However as far as I can see these functions are meant for one dimensional ...
0
votes
0answers
23 views

Time-series detection algorithm for multi-seasonal data using Python

My data: I have two seasonal patterns in my hourly data... daily and weekly. For example... each day in my dataset has roughly the same shape based on hour of the day. However, certain days like ...
0
votes
0answers
8 views

Script for downloading Medline abstracts with PMID [migrated]

I am looking for a script that will enable me to download a set of abstracts (or article meta-data of an article) from PubMed by supplying it with a list of PubMed ID (PMID) numbers (e.g., from a .csv ...
2
votes
1answer
39 views

PYMC Confusion: are observed nodes fixed or stochastic?

I've been trying to gain a better understanding of factor potentials in PYMC. In reading this article by Cam Davidson-Pilon on Yhat, I got confused about how observed nodes are understood by PYMC. ...
0
votes
0answers
16 views

Choosing between different methods when the first one raises error message for linear regression

I have a linear regression problem $$Ax=b$$ My initial approach that helped to solve some of my questions was using SVD and obtaining the chi-square and some other values that I am interested but it ...
1
vote
0answers
39 views

Optimization in R vs Python, constrained, unconstrained and automatic differentiation?

I am an economics/stat guy who uses quite a bit of optimization (maximum likelihood, simulated maximum likelihood), constrained optimization (mathematical programming w/ equilibrium conditions), ...
1
vote
1answer
36 views

How to sample from “biased” binomial distribution (ideally in python/numpy)

I need to sample a variable from a distribution that's like a binomial distribution except with a "bias", I'm not sure what it may be called: $p(X=k)$ is proportional to $k.B(n,p)(k)$ where $B(n,p)$ ...
0
votes
1answer
31 views

Autoclass in R/Python? [closed]

Are there any packages that implement the Autoclass/ Naive Bayes Clustering algorithm in R or Python? Alternatively, what are some other clustering algorithms that can handle both categorical and ...
0
votes
0answers
12 views

PyMC: Which parameter is passed to the model by sampler? How to find the value passed by custom sampler?

I am relatively new for PyMC and as a whole I have some less knowledge in probability. As far as I understand MCMC algorithm, its main task is to propose samples of parameters to the model (in ...
0
votes
1answer
43 views

Defining the Stochastic and Deterministic variables with pymc3

I am trying to use write my own stochastic and deterministic variables with pymc3, but old ...
0
votes
0answers
45 views

Numerical estimation of MLE in Python — normal distribution and gradient is close to zero away from the mean

I am exploring how to model a data set using normal distributions with both mean and variance defined as linear functions of independent variables. Something like $\mathcal{N} \sim \left (f(x), ...
0
votes
1answer
52 views

Best optimization package for employee scheduling problem? [closed]

I am looking to solve the optimization problem described below. Which optimization software package would be best suited for this, considering the requirements specified below? Requirements: 1) Can ...
1
vote
2answers
82 views

Time series - correlation and lag time

I am studying the correlation between a set of input variables and a response variable, price. These are all in time series. 1) Is it necessary that I smooth out the curve where the input variable is ...
3
votes
1answer
47 views

Standard Deviation of an Exponentially-weighted Mean

I wrote a simple function in Python to calculate the exponentially weighted mean: ...
1
vote
1answer
57 views

How do I perform this complicated ANOVA type analysis in R?

I have the following type of dataset: | | | variable_r | subject | gender | age_group | Cond_1 | Cond_2 | ...
0
votes
0answers
26 views

What library in Python implements a Logistic Model Tree?

I am looking for something that implements a an algorithm along the lines of Landwehr et al.'s 2003 "Logistic Model Trees". I've found LMT() in the RWeka package implements the algorithm in R, but am ...
1
vote
0answers
38 views

Test means of populations with lots of zeros, is sampling the way to go?

I am trying to test whether means of two populations with lots of zeros are different. Here is the following python code example: ...
0
votes
1answer
54 views

Multivariate Linear Regression in Python

How to compute the overall standard error of a linear regression model using Python? Which library should I use? I am looking for something like this, however, I can't see how to get the overall ...
0
votes
0answers
41 views

Find linear SVM feature weights using libsvm

I'm trying to use linear SVM to do some feature selection. I'm using libsvm, but I cannot figure out how to find feature weights. The model file created looks something like this: ...
0
votes
0answers
60 views

Autoregressive distributed lag models ADL(p,q): seeking “how to” preferably in Matlab (Stata/R/Python/C# etc)

Could anyone provide me the details of how to determine the lag order of the distributed lags for an $\text{ADL}(p,q)$ model in Matlab or another statistical package (and very much preferably in ...
1
vote
1answer
158 views

Plot the probability mass function

I am trying to plot the probability mass function of a sample of a discrete metric. If it was continuous, I know that using pandas it would be as simple as calling: ...
2
votes
1answer
48 views

finding global minima of a random forest estimator

I have a random forest regression model with 1000 trees, having 16 parameters (using python scikit-learn). The estimator can predict a target value with cross validated r2 score of 0.87 +/- 0.03. I ...
0
votes
1answer
15 views

Feature selection with a binary dependent variable

Given we have a binary dependent variable and 100s of features and ~50k observations, is there a generally accepted way to trim the features via some type of machine learning concept? I was trying a ...
0
votes
1answer
98 views

Calculating the information gain on the features with python

I'm looking for a python library that computes the information gain for the features given a training matrix. Are you aware of any?
0
votes
3answers
107 views

Biased coin toss simulation — which random generator is most appropriate?

I have a doubt whether the uniformly distributed random generator is the most appropriate to use for simulating biased coin toss. ...
0
votes
0answers
27 views

How to calculate the correlation between a variable and a matrix of variables

I'm trying to improve a factory quality control. I have some variables from the melting process (something like ten control variables) that changes trough time (a matrix of the values of those ...
0
votes
0answers
43 views

Chi-squared test with scipy: what's the difference between chi2_contingency and chisquare?

I'd like to run a chi-squared test in Python with scipy. I've created code to do this, but I don't know if what I'm doing is right, because the scipy docs are quite sparse. Background first: I have ...
0
votes
1answer
44 views

How to get both MSE and R2 from a sklearn GridSearchCV?

I can use a GridSearchCV on a pipeline and specify scoring to either be 'MSE' or 'R2'. I can then access ...
0
votes
2answers
84 views

Plotting a Gaussian in Python

I am trying to plot a histogram of my data, and I seem to be a little confused here. I am using matplotlib in Python. Here is the code from their website: ...
0
votes
1answer
50 views

Alternative to forecast() and ets() in Python?

I'm looking for a Python alternative to R's ETS() from forecast(). It's my understanding that ETS() is one of the best performing forecasting program and I would like to use it. However I am ...
0
votes
1answer
32 views

What is R squared for a neural network and what does it signify?

I calculated R square for my neural network based on a formula I found somewhere, which goes something like: http://i.stack.imgur.com/DojZC.png It should be something around 0.98-0.99. But, when I ...
7
votes
3answers
352 views

Why Python's scikit-learn LDA results are different from LDA in R or a step-by-step approach

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious about the results. I am wondering ...
0
votes
1answer
30 views

PDF not matching histogram of synthetic ratios of independent beta

The PDF of the ratios of independent beta variables is described in http://www.tandfonline.com/doi/abs/10.1080/03610920008832632#.U9J02vldUcC To explore the implications, i created an implementation ...
3
votes
1answer
104 views

Training a Tic Tac Toe brain - am I on the right track?

My only experience with Machine Learning is Andrew Ng's Coursera course, but I did work through that just fine and passed with 100%. I decided to practice by making up some problems and solving them. ...
0
votes
1answer
77 views

Fisher's z-transform in Python?

I want to test a sample correlation $r$ for significance ($n=16$), using p-values, in Python. As I have understood from this question, I can achieve that by using Fisher's z-transform. Is there a ...
0
votes
1answer
89 views

What is the equation for random forest?

I need an equation for random forest so that I can score fresh data I receive every week, based on beta estimates I got after building model using this ensemble methodology. Every week I do not want ...
2
votes
3answers
122 views

Methods in R or Python to perform feature selection in unsupervised learning

What are the available methods/implementation in R/Python to discard/select unimportant/important features in data? My data does not have labels (unsupervised). The data has ~100 features with mixed ...