Questions tagged [python]

Python is a programming language commonly used for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `Python`.

Filter by
Sorted by
Tagged with
1
vote
0answers
10 views

How to use weights from our data set properly? Should we use them at all?

The census data set I'm using: https://archive.ics.uci.edu/ml/datasets/Adult So, I'm currently using this census data to make observations and predict whether someone is married or not. However, when ...
2
votes
0answers
16 views

Sample subset characteristics

When drawing a random sample from data, it is typically tested to check if the sample is properly representative of the total population. Assume a scenario where a subset exists within a population, ...
0
votes
1answer
13 views

Tensorflow matmul error when predict is called [closed]

I got the following error when running this code for the MNIST dataset. The code implements an autoencoder with tied weight. error: ...
0
votes
0answers
3 views

What can I do when the values for my attributes is mostly 0

I was trying to do classification for census-income (KDD Data Set) (https://archive.ics.uci.edu/ml/datasets/Census-Income+%28KDD%29). The aim is to classify people who gain income >50 k and <50k. ...
0
votes
1answer
11 views

How to add bigram stopwords when text clustering with tf-idf and k-means [closed]

Summary: I'm doing text clustering with using tf-idf and k-means clustering. I have followed this article to do so. While I haven't fully understood what's going on with the clustering side, the ...
0
votes
1answer
28 views

What are the weights assigned to the features (coefficients in the primal problem) in SVR?

In case of linear kernel, how to interpret the weights in the formulations of nu-SVR? I am using nu-SVR to estimate the parameters in GARCH(1,1) model:$ \sigma_t^2 = \omega + \alpha y_{t-1}^2 + \...
0
votes
0answers
12 views

What is the use of x_train.astype('float32') in CNN? [closed]

I an working with the mnist dataset through some tutorial using CNN.After loading the data, x_train is converted into float32.I cant see the significance of it as if i divide it as x_train/255, the ...
0
votes
0answers
17 views

Correlogram With Points Beyond -1 and +1?

I made a partial correlation plot of the residual of a dataset using python. I got the pacf graph below with some correlation points going to 2 and -1.5. Everyone finds this strange, does someone have ...
3
votes
1answer
41 views
+50

Is there a standard measure of fit to validate Exploratory factor analysis?

I am modeling Exploratory Factor Analysis in R, Python, Mplus, and SPSS with maximum likelihood method and Varimax orthogonal rotation. However, each software gives different measures of fit and I am ...
0
votes
0answers
17 views

Analysing arrays of image data with Machine Learning Models

I am trying to do Machine Learning on arrays or vectors describing images. The target variable is a category I am trying to predict. I have multiple features that each contain arrays describing the ...
-1
votes
0answers
18 views

Does a pca plot gives me a relationship about my future importances?

I am searching for outliers and wanna have some graphical support. Let's assume I have a original dataset with several columns (numbers for example 7). And I do a pca decompisation. So my code would ...
0
votes
1answer
9 views

Custom Loss combining multiple outputs Keras/Tensorflow 2.0 [closed]

I am trying to use the keras/tensorflow 2.0 API to implement the negative log likelihood loss from equation (8) in this paper: #https://arxiv.org/pdf/1703.04977.pdf I am having difficulty combining ...
1
vote
0answers
17 views

Choosing a space function for hyperopt

I was originally doing a grid search for my parameter optimization and with 7 parameters being optimized, it would take ages. So I am choosing to use hyperopt at this point. I am however confused on ...
0
votes
0answers
23 views

what should be best technology stack for R project? [closed]

I want to create an application which will generate the graphs using R in which I will take data from database and pass to R project, R will do some operation and pass it back to front end? I have ...
0
votes
0answers
6 views

Changing the movement of the window in spectogram [migrated]

What I would like to do is to plot the spectogram of a given timeseries. My problem is: I don't want to move my window for calculation of the amplitude of the frequences by 1 index, but by for ex. 128 ...
0
votes
0answers
9 views

Trouble optimizing Five Parameter Logistic (5PL) Standard Curve for ELISA data using Python

Cross-posted from Bioinformatics SE here. The program is used for regression of a bioassay standard curve, but the question I'm asking is statistical in nature. I am attempting to write a tool to be ...
-1
votes
1answer
74 views

Why am I getting accuracy of 100 percent using SVM

I am working on Credit card data set for fraud detection. When I apply SVM for it, I am getting the accuracy as 100 %. What might be going wrong here? Here is the code ...
0
votes
1answer
16 views

Brown-Forsythe using python

I'm analysing what's the pattern of sales of a product. For that I grouped the sales by week (1st week, 2nd week, 3rd week and 4th week). The sample sizes differ significantly from each other, so do ...
0
votes
0answers
3 views

How to convert GEKKO solutions into a float [migrated]

I am using GEKKO to solve a system of non-linear differential equations. It is able to solve the equations and give me solutions, however the the solutions seem to be saved in some GEKKO object kind ...
-2
votes
0answers
13 views

Play raw bytes audio [closed]

I have recorded my audio using pyaudio and stored those bytes data in a dictionary. I want to play those raw bytes data without converting it to a file. Please help me
0
votes
0answers
16 views

Why are frequencies not sorted by default by matplotlib? [closed]

I was toying with the british national corpus out of curiosity and found something odd happening to me. I used the following code to prepare a linear plot of the word frequencies (with a ...
0
votes
0answers
10 views

Dealing with missing levels in categorical variables when batch scoring in Python [closed]

I have encountered a unique problem. My model was trained on DNN framework and the model parameters are saved which I'm using now to score the data. Since my data is quite huge, I'm scoring the data ...
0
votes
0answers
10 views

Statistical tests suitable for a pandas dataframe [closed]

I have a large dataframe in the form of : ...
0
votes
1answer
32 views

Random Forest classifier with mean accuracy of 1? Sounds fishy

I have a small dataset with many features, but unfortunately only 19 observations from 2 categories. The idea is that I can determine feature importance in classifying samples in one of 2 categories. ...
0
votes
0answers
11 views

Is it possible to determine a certain characteristic point in a picture based on existing pictures?

The problem I try to solve is actually something I want to apply to an existing use case. To give some background information on that, there's this site botb.com that works kinda like a lottery, ...
0
votes
0answers
17 views

Calculating EER with anomaly detection using LSTM in python

I have dataset features evaluated from the touch screen and built-in sensors on smartphones. I want to implement an anomaly detection code using LSTM autoencoder in python to compute EER value (Equal ...
1
vote
3answers
29 views

What methods of forecasting should I be looking at to forecast sales?

I am wanting to forecast sales of different products within a business. I have a good background in mathematics (but mainly focused on analysis, group theory, algebra etc. as opposed to statistics). ...
0
votes
0answers
12 views

Implementing a Kernel Adaptive Filtering model explained in a paper

In this paper, Stock price prediction using kernel adaptive filtering within a stock market interdependence approach, the authors propose a method for predicting stock prices by combining the ...
0
votes
0answers
15 views

Adding features lowered accuracy score Gaussian Naive Bayes algorithm, Python

I am in third year of university, this simple py program is meant to use the Gaussian Naive Bayes algorithm to create a model and evaluate it. ...
0
votes
0answers
9 views

how to split the given condition? [closed]

condition=((selection1 AND (selection2 OR selection3) AND selection4 AND selection5 AND selection6 AND selection7) OR (selection8 AND selection9 AND selection10 AND selection11 AND selection12))....
0
votes
0answers
9 views

How to run a nested RM-ANOVA?

I have some data in the following format: 4 conditions, within subject, multiple measurements per subject. Condition is the fixed effect, and subject is the random effect, which is nested into ...
0
votes
1answer
12 views

Feeding multiple rows of data into ANN [closed]

I've built an ANN from scratch, that works with one row of data with any number of neurons and hidden layers. For the setup I am using 2 hidden layers, 5 neurons (just while building). The network ...
0
votes
0answers
6 views

Is it possible to achieve both stratified sampling and keeping the same train/test dataset split across different runs?

Generally, it is suggested to sample a dataset such that test set and train set remain the same when running the code multiple times, for comparison but also to hide your algorithm the whole dataset. ...
0
votes
0answers
20 views

Train unsupervised algorithm to known output [closed]

I have a big dataset consisting of coordinates of particles and coordinates of centers; these centers belong to underdense regions between all these particles. Given the particle positions I want to ...
0
votes
0answers
13 views

Listing factors and their corresponding R for a dataset [closed]

I have a dataset (Skewed Left) with customer survey scores and then a mix of mainly categorical and some numerical features. I am attempting to see how any of these features correspond to the ...
0
votes
0answers
16 views

Can knowing the sklearn code help? [closed]

I'm a machine learning amateur and since the very beginning was wondering if understanding the sklearn code would help to understand how to prepare data the way it ...
2
votes
0answers
22 views

Is there a library to fit a Threshold Autoregressive Model (TAR) in Python?

I tried to look into Statsmodel but I couldn't find it. I know that in R there is the TAR package. I would like to find something similar for Python. My entire project is written in Python and I've ...
1
vote
1answer
78 views

Time Series Forecasting - Daily data

I'm relatively new to time series forecasting. I've been assigned with the task of forecasting operation time of an industrial equipment based on a daily data (3 years of daily data). The prediction ...
1
vote
0answers
17 views

Betweenness Centrality giving different results with Package Igraph R vs Networkx Python (and Gephi) [closed]

Basically, I don't understand what is going on. When I calculate the normalized betweenness centrality with R using this function: ...
0
votes
0answers
6 views

Python: Imbalanced multi-classification machine learning [closed]

Can I use SVM, logistics regression, tree based methods(GradientBoostingClassifier, RandomForest) after oversampling SMOTE? ''' X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,...
1
vote
0answers
19 views

How to compare four conditions with nested subgroups of measurements in python? [closed]

I'd like to ask for some advice on what statistical inference I should run on my data with the following design: There are 4 conditions. Each subject was tested for each condition (->within subject ...
0
votes
0answers
16 views

LDA for binary classification?

My plan was to use LDA as a feature selection method instead of PCA since I have labels. The problem I'm facing is my problem changed to binary classification. So the output of my LDA model is one ...
4
votes
1answer
119 views

What does Sparse PCA implementation in Python do?

I am interested on using sparse PCA in python and I found the sklearn implementation. However, I think this python implementation solves a different problem than the original sparse pca algorithm ...
0
votes
0answers
6 views

Performance Statistics Of Each Class in ML Classification problem

I am comparing the performance of two neural network architectures for a classification problem with three possible outputs. I am able to track the overall progress (validation accuracy/loss) of ...
0
votes
0answers
21 views

How can I interpret very high values of lambda in a BoxCox transformation?

I got a skewed datset. (I also tested it with the python method .skew() and the value was 398) So I wanted that my datset is not that skewed anymore and found the method BoxCox. I then implemented ...
0
votes
0answers
10 views

Using MCA/PCA together?

If I have a large dataset with continuous, discrete, and categorical data, is it appropriate to use MCA on the categorical features and PCA on the continuous, separately? I'm preprocessing my data ...
0
votes
1answer
22 views

Correct z-score implementation for a hypothesis test in Python?

I have been getting some bizarre z-score calculations and wanted to confirm whether my concept and/or implementation is correct. I have a pandas DataFrame ...
0
votes
0answers
22 views

After applying SMOTE, the class distribution doesn't match the real world. Is this a problem? [duplicate]

I have an extremely unbalanced dataset with two classes: 1: 1,800 # class 1 0: 40,000 # class 0 This is real world customer data of churned/not churned If I ...
1
vote
1answer
33 views

Linear regressions of fluorescence and Discrete Chlorophyll

I'm pretty much clueless here. I'm doing a project in Springboard's Data Science Career Track. Most of the data science makes sense; the ML part, not so much. I have a data set from USGS - water ...
1
vote
0answers
33 views

Using qualitative data (numerical) for prediction

Let's try and look at this with an example before I get into my questions. Let's say you're trying to predict the number of Oscars a movie will win. With qualitative data we can clean and plug in a ...