Questions tagged [python]

Python is a general purpose programming language designed for ease of use. It is a commonly used platform for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `Python`.

0
votes
0answers
6 views

Reduce scoring time for test sample of K-Nearest Neighbors regression for a time series dependent variables

My dependent variable is values of $ balance amount for an entity over next 24 months after the entity is subjected to a specific treatment. I am trying to predict these 24 values (bal1-bal24) for ...
0
votes
0answers
5 views

Using mutual information for feature selection between feature maps (python)

I want to do feature selection between 512 3X3 feature maps from convolutional layers. I want to calculate a 512X512 MI matrix and choose 256 feature maps with the lowest MI values between themselves (...
0
votes
1answer
17 views

Visualising 2 Independent Variables and 1 Dependent Variable

I'm trying to come up with the best way to graphically display misfits from multiple geophysical inversion runs, where I am fixing two independent variables and analysing the misfit of each test. For ...
0
votes
1answer
35 views

What statistical technique could be used to extrapolate CO2 data

I am new to statistics but scraped data from NOAA's web site to plot CO2 level vs time. Is there a good statistical technique to extrapolate past limit of data X axis to near future like 2050? One ...
0
votes
0answers
8 views

Kernel-weighted local polynomial smoothing in Python? (Replicating Stata/R functionality) [on hold]

I am currently using LPOLY from Stata (https://www.stata.com/manuals13/rlpoly.pdf) to create a smooth curve between different Political Pollsters: ...
0
votes
0answers
7 views

Electronic equipment failure prediction/abnormal behavior detection model

I have a data set of alarm logs and event logs of network equipment with timestamps and I'm looking to build a machine learning failure prediction/ abnormal behavior detection model using python. ...
0
votes
0answers
10 views

What change should I make in the Image Data Generator to solve the error? [on hold]

I am trying to perform Mixup Augmentation (details here) but I am getting a value error as follows : ...
3
votes
0answers
14 views

OneVsRestClassifier and predict_proba

I have an interesting problem. I am working with a MULTICLASS problem (~90 classes), and have settled on using OneVsRestClassifier wrapper around a RandomForestClassifier. When I call a ....
0
votes
0answers
12 views

Python Factor Analyzer and PCA

I am performing PCA and I need to extract squared loadings. I found this python library, Factor Analyzer, that can extract eigenvalues and squared loadings, etc., but the results are different from ...
0
votes
0answers
13 views

MLRose Specific Min and Max Values for each value Random Optimization

I'm currently working on a project where I am creating a genetic algorithms using the MLRose library (link) to find optimal values for a set of insurance factors. ...
0
votes
0answers
7 views

Repeated measure ANOVA with between-subject factor in Python?

I'm performing repeated measure ANOVA on a 3x3 within-subject factor experiment using statsmodels's AnovaRM. It's a response time experiment, so each participant went through a lot of trials. This ...
-1
votes
0answers
10 views

Matplotlib shows date in POSIX time [migrated]

My data contains order dates and gross sales. I have the problem that xticks shows POSIX time for X which is not readable for humans. Do you have any idea how to ...
0
votes
0answers
27 views

Multiple Time series Forecasting Using LSTM in python

Assume I have a m dimensional input feature vector and I would like to perform multiple steps time series forecasting. I have about 500 files which each one is has 100 observations for example ...
0
votes
0answers
8 views

Hyperparameter-free method for Moving Average/ Exponential smoothing?

I want to find hyperparameter-free method for Moving Average/ Exponential smoothing. Is there any related paper or python code? S(t)= alpha * F(t) + (1-alpha) * S(t-1) Any methods can avoid the ...
-2
votes
0answers
25 views

What Would Be The Best Scikit Learn Alogriths For Lottery Numbers? [on hold]

I want to start a machine learning model on lottery results using ScikitLearn. Below is the head of my database. If anyone has any suggestions I would appreciate it. Thanks. Head of Database: ...
0
votes
0answers
14 views

onehotencoder issue [duplicate]

New to DNN. I have the following dataset My target value is "Purchase" column. I know I need to drop "product_id" column and create dummies for "Gender" and "City_category", but do i need to do ...
0
votes
2answers
53 views

Using bagging and random forests together

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of ...
0
votes
1answer
52 views
+100

Trying to predict continuation of curves using LSTM

I have an application where I get a large set of smooth curves (2D). Those curves are represented by sample points on that curve. Sometimes, those curves cross or get close to each other and it is not ...
0
votes
0answers
6 views

Timeseries Forecast with log-normalized and differentiated data

i posted a similar, but more confusion question already. I have a weekly timeseries so far, which looks like this (pls ignore the red line): My original data is (e.g.): ...
0
votes
0answers
20 views

Increasing image size in pytorch celebrity generating GAN?

complete newbie here, bear with me. I'm making my way through this tutorial: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html Upon attempting to make a simple change to the image ...
0
votes
1answer
17 views

Cross Validation Score difference between test/train individual r-square

I am new to Machine Learning, as part of my learning, I created a Linear Regression Model using two attributes for Boston Housing Price dataset. I am having doubts calculating : MSE and R-Square. So, ...
1
vote
0answers
50 views

Calculating standard errors of non-OLS regressions in python and scipy [on hold]

Scipy/statsmodels provides the ability to get a number of metrics from the output of an OLS regression model, a few examples of which (from statsmodels) are the following: ...
-2
votes
0answers
24 views

Convert R's rgamma random sampling to Python scipy's gamma.rvs

Is there a way to translate from R's rgamma(MSamples, shape = kA, scale = thetaA) gamma random distribution, to Python scipy's ...
0
votes
0answers
18 views

SciPy Beta Distribution rounding errors [closed]

I'm using stats.beta.ppf(prob,alpha,beta) in Python with very high probability values, for example prob=0.999999805056895. This appears to be producing incorrect results, returning a value of 1 (it ...
1
vote
2answers
22 views

How to classify time series trends into 2 groups: “contain seasonality” and “doesn't contain seasonality”

I'm optimizing prediction model for time series data trends. Each trend may have seasonality effect or may not. I want to classify each trend into one of the following groups: "seasonality" or "no ...
0
votes
0answers
23 views

Timeseries with Log and differianted does not fit to predicted data [closed]

I am trying to built a data model with Knime, where I use functions in python for data wrangling and the metanodes (java/R) in Knime for forecasting. Untill now I discovered that I do not have ...
0
votes
0answers
29 views

Which model type for a distribution of discrete outcomes?

I am doing work on predicting household types (income group, number of workers, presence of children) based on a forecast of a number of economic variables from a regional economic model, such as a ...
0
votes
0answers
15 views

Using Decomposition to Extrapolate seasonality, cycle and trends of predictors

I'm creating a dynamic regression model in which macroeconomic indicators are predictors/features in the model. I need to forecast these features n-steps into the future. I am planning to decompose ...
-1
votes
0answers
35 views

Online training with Random Forest Classifier [on hold]

Suppose I have a huge set of training data related to online transactions. I have trained a random forest classifer on full set of that transaction data. Now, consider an online learning kind of ...
0
votes
0answers
38 views

What Are The Best Time Series Algorithms? [closed]

I was wondering what the best time series algorithms were, I want to use Scikit Learn, for the machine learning. Here below is the head of my database, the data = Lottery Results. ...
0
votes
0answers
35 views

Warning: Maximum number of iterations has been exceeded. Statsmodels Logistic Regression [closed]

I am trying to train a Logit model using the statsmodels api and I get the following error message. ...
0
votes
1answer
25 views

Trying to Understand the Statistical Notation [closed]

Does x need to be randomly sampled from a normal population and what is d?
1
vote
0answers
11 views

Wald Chi-square with survey weights in Python [closed]

I am trying to perform a Wald (Chi-square) test in Python using survey data with sample weights. I have looked in Scikit learn and statsmodels, but neither seem to support this test where the sample ...
-1
votes
0answers
11 views

Assigning tags to posts using predefined set of tags [closed]

I want to tag the text of a post with predefined set of tags. A post could have multiple tags such as health, addiction etc. I want to recommend up to 5 tags. Total 60 tags are present. Nearly 50 ...
2
votes
3answers
122 views

predictions for AR(1) model

I don't understand how predictions can trace the actual data so closely (see the code below)? Does that make sense? The model is $Y_t = \theta Y_{t-1} + Z_t$ where $Z_t$ is random noise. Hence the ...
0
votes
0answers
18 views

Hierachical Bayesian Linear Regression using PyMC3 is super slow [migrated]

I am trying to write some code for implementing HBM in the case of logistic regression using the adults dataset from the UCI repository. I have already written the code, but sampling is super slow, ...
0
votes
0answers
9 views

Use raw count, Term Frequency or Log Normalized with Cosine Similarity?

For the task of finding duplicate documents within a corpus of documents I have been using a TF-IDF vectorizer (from sklearn in Python) combined with the Cosine Similarity between the document vectors....
0
votes
0answers
10 views

Question About Preprocessing And Regression Models [on hold]

My first question is weather or not the head of data below needs to be further processed, or if the format is good the way it is. ...
0
votes
0answers
22 views

What's the bug in my implementation/understanding of backpropagation?

For learning purposes, I'm trying to implement a simple neural network with only linear layers followed by logistic activation. As far as I understand, the backpropagation algorithm exploits the ...
1
vote
1answer
28 views

Removing multiple seasonalities from time series

I'm using statsmodels.tsa.seasonal.seasonal_decompose to remove seasonality from a time series. I can remove a seasonal component in this way: ...
2
votes
3answers
66 views

How to visualize an evolution of a distribution in time?

Suppose you have a record of distribution for each day in some period. For example, some distribution which depends on a parameter which evolves over time. Suppose we have dozens or hundreds of days. ...
0
votes
0answers
15 views

Efficient algorithm for finding optimal number of breakpoints for piecewise regression

I aim to implement a module in python that does the following: 1) Upon taking training data, fit a piecewise regression with $n$ breakpoints. 2) Determine how well of a fit it is to the data (I ...
1
vote
0answers
18 views

Multiple Non-linear Regression with Function-based regression and Machine Learning models

I'm working on an application of Multi-nonlinear regression. Initially, I tried this algorithmically by creating a polynomial of the form A(x^p * y^q * z^r). I saw ...
1
vote
1answer
53 views

Independent and dependent variables Machine Learning

I want to start a regression model on lottery numbers, my database consist of the dates, and results(numbers). In order to do a regression model I believe you need independent and dependent variables, ...
0
votes
0answers
5 views

Subsetting columns that are two groups of consecutive number [migrated]

Hi I know in R if I have a data frame, df, if I want to just cal the first ten columns and the 12th-17th columns, I can do this: df[,c(1:10,12:17)]. Is there a way to do this in python with the pandas ...
0
votes
0answers
13 views

Downsampling before or after train/test split?

I've seen people split into train and test sets before applying upsample techniques (to only the training data), but applying downsampling to both training and test data (i.e. before the train/test ...
0
votes
0answers
21 views

Regression model for ML hyperparameter tuning

Problem definition: Having predictor variables such as: learning rate(continous, range 0-1), number of iterations(continous), number of hidden nodes(continous), LossFunction(categorical) and ...
1
vote
1answer
60 views

How to find the relationship between two variables using regression? [closed]

I have a Raspberry Pi hooked up to a sensor, which will send a data to the device every second. As you may have guessed, what I want is to find a relation between this independent variable time and ...
0
votes
0answers
7 views

Maximize ELBO in Keras

When we train a Variational Autoencoder we say that we want to maximize the ELBO. However, from the Keras documentation, it seems that we are actually minimizing the ELBO: ...
1
vote
1answer
23 views

How to choose a model for survival regression when data does not fit assumptions?

I am trying to perform survival regression (prediction) on a dataset of lifetimes, which is highly concentrated around 1, with a significant right skew. The below photo is how it looks when log-...