Questions tagged [python]

Python is a programming language commonly used for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `Python`.

1,561 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
34
votes
1answer
2k views

Link Anomaly Detection in Temporal Network

I came across this paper that uses link anomaly detection to predict trending topics, and I found it incredibly intriguing: The paper is "Discovering Emerging Topics in Social Streams via Link Anomaly ...
12
votes
1answer
9k views

Cause of a high condition number in a python statsmodels regression?

I'm pretty new to regression analysis, and I'm using python's statsmodels to look at the relationship between GDP/health/social services spending and health outcomes (DALYs) across the OECD. Just to ...
7
votes
0answers
2k views
6
votes
0answers
7k views

Assessing A/B test results using Python

Problem's setting Assume having results of an A/B test. You let you users experience two variants of your website and you counted how many converted: ...
5
votes
1answer
1k views

How do I get the p value of AD test using the results of scipy.stats.anderson()

I am conducting an Anderson Darling test for normality using scipy.stats.anderson() command in python. I am getting test statistic, critical values at various significance levels as the output. The ...
5
votes
0answers
2k views

What is the difference between Lasso regression in glmnet (in R) and Sklearn lasso (in Python)?

A similar post was discussed here regarding Ridge Regression: What are the differences between Ridge regression using R's glmnet and Python's scikit-learn? My question is what is this ...
5
votes
0answers
1k views

Odds ratio MLE calculation: unconditional vs conditional?

I am running an odds ratio calculation for site methylation amongst cases and controls. In this situation is it preferable to use a conditional or unconditional MLE? I am asking because R uses a ...
5
votes
0answers
1k views

What is the intuition for testing seasonal difference with OCSB test and its correct application?

I have daily time series data of a shop's revenue. Now I would like to test for seasonal differencing with the OCSB test originally intrduced in (Osborn et al. (1988): Seasonality and the Order of ...
5
votes
0answers
326 views

How to impose spatial continuity constraint onto clustering?

Statistics version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: $f(a,b,c;x,y)$, with $f$ being a vector of several quantities. I would like ...
5
votes
0answers
1k views

statsmodels: quantreg convergence cycle warning

I am getting the same Convergence cycle detected warning running a quantile regression with ...
5
votes
0answers
936 views

Asymmetric confidence intervals on bootstrap estimates

I've performed bootstrapping on my leastsq parameters and now I have a load of data from which I can get the mean and standard deviation for each parameter. Lovely. But when I look at a histogram of ...
5
votes
0answers
959 views

Bayesian model selection in PyMC

I'm trying to do model selection using PyMC (v2.2), but having difficulty assessing the models using various Information Criteria and/or Bayes Factor. My model is similar to a typical regression, with ...
5
votes
0answers
242 views

Difficulty with MCMC implementation

I could really use some guided help! I'm having difficulty understanding an MCMC implementation in terms of modeling a data set. I'm working on generating parameters from stellar light curves, and was ...
4
votes
0answers
259 views

How to draw an ellipsoid corresponding to the eigenvectors and eigenvalues of covariance matrix?

I'm doing PCA with Python with dataset decathlon in which I'm interested in 3 variables 100m, ...
4
votes
0answers
78 views

How to explain random forest ML algorithm doesn't learn at all, while logistic regression learns very well?

My prediction task is as follows: Use name to predict people's ethnicity (into 4 categories: "English", "French", "Chinese", and "All others") as a multiclass classification problem. The name ...
4
votes
1answer
2k views

Improve precision/recall for class imbalance?

Trying to get better precision/recall for both classes ... any tips? I have heterogeneous features [a few num vars, a few cat vars, and 2 text vars] Target is a binary classification w/ class ...
4
votes
0answers
111 views

Method to forecast correlate univariate time-series (with trend, seasonality) via regression

I have two univariate time-series with seasonality and trend--dt1 and dt2. I believe that dt1 and dt2 are strongly correlated, both through a few statistical test (see below) and that in my field dt2 ...
4
votes
0answers
3k views

What is the difference between dice loss vs jaccard loss in semantic segmentation task?

What is the difference between dice loss vs jaccard loss in semantic segmentation task? Dice loss: ...
4
votes
0answers
3k views

Correct number of components in GMM according to BIC and AIC plots

I have applied GMM(Gaussian Mixture Model) to my data set and I have plotted the resulting BIC(Bayesian Information Criterion) and AIC(Akaike Information Criterion) for different number of components. ...
4
votes
0answers
569 views

Medium Frequency Trading - Better labelling strategy?

The mid-price at time $t$ is denoted by $$p_t = \frac{s_t^{a,1} + s_t^{b,1}}{2}.$$ This mid-price can evolve in minimum increments of half a tick but is almost always observed to move at ...
4
votes
0answers
3k views

Transfer learning on faster rcnn and tensorflow

I am trying to do transfer learning to reuse a pretrained neural net. I got the tensorflow faster rcnn official example to work, and now i would like to reuse it to detect my own classes. This is the ...
4
votes
2answers
5k views

How to train a LSTM model for a next basket recommendation problem?

I try to use a LSTM model for a problem of next basket recommendation. I would like to apply the same approach as this article in Python using Keras : A Dynamic Recurrent Model for Next Basket ...
4
votes
0answers
979 views

Time series analysis with rare events / anomalous event detection

I have a time series which indicates a certain demand over time. In my project I detect anomalies in this time series by applying Robust Principal Component Analysis (RPCA) which gives my something ...
4
votes
0answers
1k views

Gaussian process regression and optimizing an RBF kernel for forecasting?

I'm using gaussian process regression with an RBF kernel to forecast a time series. I'm using GaussianProcessRegression in ...
4
votes
0answers
961 views

Row Correlation Heatmap Pandas

I'm trying to find any relationship/patterns between a large number of rows in a dataset (~2000) and I'm thinking of using a correlation heatmap. However, after transforming the df using ...
4
votes
0answers
1k views

Robustness Weights in LOESS behaving strangely

I've been playing around with writing my own LOESS module in Python (2 reasons: first, I wanted the practice, and second, the implementation in statsmodels doesn't ...
4
votes
0answers
5k views

ARMA Model fitting using statsmodels.tsa.ARMA()

Two questions. 1.) When I use the statsmodels.tsa.ARMA() module, I enter my parameters and fit a model as follows: ...
4
votes
0answers
427 views

Sampling from Wishart distributions with scalar degrees of freedom $(\upsilon>p-1)$

Let $\upsilon$ be the degrees of freedom of a Wishart distribution and $p$ the dimensions of its scale matrix. If the degrees of freedom is a scalar, then its range is: $$ \upsilon > p−1 $$ and ...
4
votes
0answers
1k views

Slow Lasso Performance Using sklearn

I am trying to use scikit-learn's LassoCV and/or ElasticNetCV functions to model a dataset with a large (>800) number of ...
4
votes
0answers
827 views

Fitting Multivariate Bernoulli distribution

I want to fit a model to a number of observations, each of them being a k-dimensional binary vector $(x_1, x_2, ..., x_k)$ where $x_i \in \{0,1\}$. Naturally I would like to fit a multivariate ...
4
votes
0answers
591 views

detecting circadian rhythm in a time series

I have a sensor that can detect minute changes in distance. It produces a time series. I would like to point it at people and detect things like their sleeping pattern. How would one build a system ...
4
votes
2answers
2k views

How do I find multiple change points in an online dataset?

I am trying to develop a Python based script connected to a SQLite3 database to identify distinct system changepoints in an "online" datastream. Changepoint must be identified in less than 2 minutes ...
3
votes
1answer
52 views

Will a model with a low R² value have higher RMS error rates?

I'm testing different models and my general expectation is that models that have a high coefficient of determination should roughly also have a lower error rate (RMSE in that case) than those with a ...
3
votes
0answers
69 views

SVM Scaling problem with One-Class SVM

I'm trying to mess around with a one-class SVM implementation I hacked together from ArduinoSVM. I'm using an RBF kernel and training the model with just "in" datapoints with sklearn. First, as is ...
3
votes
0answers
364 views

Why are the Polynomial Features used in a Linear Regression model not working as expected as the degree increases?

I'm fitting several polynomial regression models of varying degree. The smaller degree models (1 to 7) are behaving as expected. However, as the model increases in degree (8 or higher), the fitted ...
3
votes
1answer
397 views

Rare value prediction in Regression

Im working on a project which is to estimate blood pressure from independent variables. The problem I have is that the Blood Pressure data is gaussian in nature since most of the people are having ...
3
votes
2answers
1k views

How to make train/test split with given class weights

I am doing simple multi class classification ML problem. I was given train data with perfectly balanced classes. However the data I must predict is not balanced. I was able to deduct the class ...
3
votes
0answers
329 views

James Stein Estimator for more than one Sample

I have a hard time understanding the James-Stein Estimator. I show you how I tried to comprehend it by using a python example. I take a normally distributed random vector with mean $(0.1, 0.2, 0.3, 0....
3
votes
0answers
116 views

How to use Approximate Bayesian computation to estimate the parameters of a function?

I am new in bayesian analysis and I want to use Approximate Bayesian computation in order to convert an odd giving to me by a bookmaker to a probability that the event occurs. Here is the Python code ...
3
votes
0answers
54 views

Calculate percentiles of angles/bearings in Python

I have a dataset with a variable that is the angle of the event (record) from a central point (e.g. crime events in relation to a defined city center). I want to calculate percentiles for the variable....
3
votes
0answers
228 views

OneVsRestClassifier and predict_proba

I have an interesting problem. I am working with a MULTICLASS problem (~90 classes), and have settled on using OneVsRestClassifier wrapper around a RandomForestClassifier. When I call a ....
3
votes
1answer
2k views

What is MultiOutputRegressor and how does it work?

What is MultiOutputRegressor in scikit-learn and how does it work? Are they predicting multi-output using a model trained for each target variable?
3
votes
0answers
72 views

Why does lasso return unstable features when using the same data?

I am using scikit-learn to shrink my data set having around 800 features. It is a very noisy data (market and economic data) To my best knowledge, lasso returns same features for the same data set. ...
3
votes
0answers
80 views

Is there any strategy for validating the result of a general comparison between several confusion matrices?

Disclaimer: Recently we have developed a python library named PyCM specialized for analyzing multi-class confusion matrices. A compare system has been added in <...
3
votes
0answers
287 views

Noise in ARIMA Model In-Sample Predictions

I am working on fitting some financial data into an ARIMA model to give me a forecast of the next time period. I am using pyramid's auto_arima function to get a ...
3
votes
0answers
116 views

Python Imblearn - How to track original data records

My data is severely unbalanced, so I'm using Python imbalanced-learn here to make data balanced: https://imbalanced-learn.readthedocs.io/en/stable/generated/imblearn.combine.SMOTEENN.html#imblearn....
3
votes
0answers
22 views

Tools/resources on probabilistic inference and mixed-integer programming

Here's the problem I'm trying to solve: Given a pool of gene ID and their read counts -- some genes are unique to one particular species while some are shared--what is the most likely combination of ...
3
votes
0answers
206 views

Calculating the probability of winning a game of Rainbow Six Siege

I'm currently developing an application in Python that calculates the probability of winning a game of Rainbow Six Siege based on the skill level of the players in the game. Siege is a 5 vs. 5 ...
3
votes
0answers
7k views

Understanding Quantile Regression with Scikit-Learn

I have a case where I want to predict a time value in minutes. This is the problem of regression. I also want to predict the upper bound and lower bound. I can do it two ways: Train 3 models: one ...
3
votes
0answers
5k views

Find RMSE from StatsModels OLS Results

I playing around with some regression analyses in Python using StatsModels. I am getting a little confused with some terminology and just wanted to clarify. I have run a regression and get the ...

1
2 3 4 5
32