Python is a general purpose programming language designed for ease of use. It is a commonly used platform for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `...

learn more… | top users | synonyms

1
vote
0answers
5 views

Title and Occupation Correlation Model

I have a large set of data that contains both a person's title (e.g., "CEO", "Bank Teller", "Fireman", etc.). and occupation codes (e.g, "A1": Managerial, "A2": Laborer, etc.). I need to build an ...
0
votes
0answers
13 views

Data analysis with python [on hold]

To APPLY for a post data scientist, the employer ask me to analyze data and find patterns and relationships. For that, I started preparing the data, and extract correlated variables. I used Python. I ...
0
votes
0answers
12 views

Using sentiment data to predict stock prices

I have implemented a model to predict "last" price of stock of 5 days ahead. I have used RandomForestRegressor. In addition to this, I have a data set of sentiments. It is the data from social media ...
0
votes
0answers
7 views

Evaluation of a binary classifier in regards of rounding, taking average scores

I'm using scikit-learn to evaluate a binary classifier: There are 2 Methods to get the Precision, Recall and F1 scores: 1.classification_report: ...
0
votes
0answers
10 views

Applying support vector regression to vector-valued functions?

So, just as a preface: this is my first time posting to this site, and I am also a machine learning beginner, so I apologize if my question is dumb or if I do something wrong, format-wise. Alright, ...
1
vote
0answers
39 views

Reversing SVD back to the original variables [on hold]

I have a large collection of text documents I am trying to group using Singular Value Decomposition for dimension reduction(popularly known as LSA) followed by k-means clustering. Subsequently, I ...
0
votes
0answers
9 views

Programs to read an arbitrary image and convert it to the format of MNIST dataset [on hold]

I want to make a program that turns a given image into the format of the MNIST dataset, as a kind of exercise to understand the various preprocessing steps involved. But since the description of how ...
0
votes
0answers
16 views

How to do sentence boundary detection? [on hold]

I have a string as My name is sushil feeling bad My age is 30 and hobbies are none thanks for asking me name varun i am good at cricket and my age is 40 i have done a lot of work in research area , ...
2
votes
1answer
27 views

How to avoid k-means assigning different labels on different run?

I have unlabeled dataset. I am running k-means flat cluster with 2 number of clusters. Every time I run the below program the labels are different. How can I make labels not to change. Is it even ...
0
votes
0answers
11 views

How to group/cluster variables/features using Python? [closed]

I have 200 variables and 1 million records. I want 20 clusters, so that I can pick out top variable (based on Information Values of the variables) in each cluster.
0
votes
0answers
18 views

Why are LSTMs performing worse on the training set with higher cell size?

I'm attempting to train a (single-layer) LSTM-based autoencoder using the seq2seq framework within Tensor Flow. The input is sequences of numerical values. The sequences each contain 40 values. As a ...
-2
votes
1answer
52 views

If wrong retry - negating automatically incorrect answers [closed]

Say the first event is someone buying product, and the second them selling it. I need to predict how long they'll have the product for from today, but my dataset and my model only predict the overall ...
0
votes
0answers
7 views

Why is xgboost giving bad scores and taking a very long time? [migrated]

I am completely new to xgboost so thank you very much for any help in advance. I have this simple script. ...
2
votes
0answers
39 views

How to know which statistical model to use for categorical data?

I'm new to statistical analysis. I'm trying to conduct an analysis of datapoints and possible correlations between them using Python's sci-kit learn library. My data is categorical. For example, a ...
3
votes
1answer
41 views

Neural network library for Python for Microsoft Windows

I have been having trouble with selecting a good library for Neural network algorithms in Python. TensorFlow isn't supported on windows. Theano is still in the beta phase of development. PyBrain too ...
0
votes
0answers
20 views

Cross-validating an ordinal logistic regression in R (using rpy2) [migrated]

I'm trying to create a predictive model in Python, comparing several different regression models through cross-validation. In order to fit an ordinal logistic model (...
0
votes
1answer
16 views

Subset documents based on tfidf weights

I am new with text mining therefore please bare me if this question sound too easy for others. But I tried to find out the solution with no success. I am working on a project of document ...
0
votes
0answers
15 views

Python Scipy expanded Shapiro Wilks test

Does anyone know for certain if scipy.stats.shapiro uses the expanded Shapiro-Wilks test approach by J.P. Royston? Should I only use this function for sample sizes ...
0
votes
0answers
11 views

PyMC: Using maximum a posteriori on vector/matrix-valued variables

I am using PyMC to find a maximum a posteriori estimate for some data. The data is all vector-valued. I am trying to estimate my variable x based on a matrix-valued ...
0
votes
0answers
12 views
0
votes
0answers
8 views

Data preparation of a new record on the fly [migrated]

I am facing problem in the implementation of Data Preparation of a single record on the fly. I am loading the model from disk and I need to to make prediction against it. Lets Say, I have 3 ...
0
votes
1answer
25 views

What do you expect a 2-sample t-test for the same two samples to return?

I am using scipy in Python and the following return a nan value for whatever reason: ...
0
votes
0answers
47 views

Sigmoid cross-entropy loss layer: What is wrong with the gradient?

Could you help me find out what is wrong with my gradient? I am trying to implement a sigmoid cross-entropy loss layer in Python (similar to caffe's, but my c++ is not good http://caffe....
0
votes
1answer
20 views

Net surgery to convert to FCN: am I doing it right?

I am kind of stuck here, hopefully someone can throw some light on it. To demonstrate my problem I will try and simplify my network architecture for sake of convenience. I have a pipeline: Training: ...
1
vote
0answers
25 views

Row Correlation Heatmap Pandas

I'm trying to find any relationship/patterns between a large number of rows in a dataset (~2000) and I'm thinking of using a correlation heatmap. However, after transforming the df using ...
1
vote
0answers
25 views

Determine the feature weights with a regression?

I have a set of houses with their features (location, size, number of rooms, etc … and the y is the price). In the future I will have a new house without the price. My goal is to find the 20 closest ...
1
vote
2answers
69 views

How to categorize (lookup) brands with spelling mistakes

I have a dataset with around 4 million rows. The data has only column which contains around 50000 unique brand names (e.g. IBM, Google, Adobe, Microsoft etc.). I also have a lookup table which ...
1
vote
0answers
16 views

Difference between two methods of random point generation [migrated]

In order to do a monte carlo simulation to estimate expected distance between two random points in $n$ dimensional space I discovered the following two similar looking methods to generate random ...
0
votes
0answers
21 views

Finding patterns in rows instead of columns

Let's say I have a dataset and instead of looking at the patterns and relationships between columns, such as plotting the distributions or a correlation matrix, I want to look at the relationship ...
2
votes
2answers
84 views

How can I calculate variance of a very large random variable?

I'm implementing an algorithm which recieves as input samples from a random variable with an unknown distribution. The random variable is extremely large so my input is logarithmic, and still large (...
-1
votes
0answers
29 views

Time Series Novelty Detection

I am performing very frequent analysis on many datasets of one variable, 15 minute increment, time series data. I want to check if the most recent observation of each dataset is anomalous. The various ...
1
vote
1answer
23 views

SVM labeling more classes does not result in higher accuracy?

I used SVM to predict the ranking score of muffin recipes. X is a numpy array of ingredient amounts of a certain recipe and y is the label according to the online ranking score. First I labelled my ...
0
votes
0answers
19 views

Number of Sinusoids Feature Extraction for MUSIC Algorithm

I am a complete novice to machine learning, but I wanted to ask this question to get me started on a path to solving this problem. I have been working on a radar project where I am receiving a signal ...
2
votes
0answers
29 views

How does Principal Coordinate Analysis (PCoA) work, as compared to PCA? [duplicate]

I am familiar with PCA from Making sense of principal component analysis, eigenvectors & eigenvalues where you either normalize the data (to standard normal or ...
0
votes
1answer
15 views

How do I learn a simple cut-off value between 2 classes given one-dimensional data?

Given a set of data which consists of a single real number and a class, I want to find a value (i.e., the inflection point if we were talking about logistic regression) which would lie right at the ...
0
votes
2answers
87 views

random forest how to use the results

I used the package for random forest. It is not clear to me how to use the results. In logistic regression you can have an equation as an output, in standard tree some rules. If you receive a new ...
0
votes
0answers
28 views

Smart sampling techniques to ensure boundary/edge cases are included

In an effort to reduce the time it takes to test software against various data sets I would like to do create an application that looks at a data set and creates a "smart sample" from the it. The ...
0
votes
0answers
17 views

How to use weights with Elasticnet regression in python?

I am using Elasticnet from scikit-learn in python, I've also used Glmnet package in R for prototyping. I want to use weights in Elasticnet which apparently is not available as an option/argument in ...
0
votes
0answers
11 views

NEAT Neural Network Digit Recognition Mean Square Error 0.9

I'm new to machine learning and am trying to create a neural network using the NEAT algorithm in python that can recognize 0-9 digits from the sklearn data set. There are ten output neuron in the nn, ...
0
votes
0answers
26 views

Nonlinear Multiple Regression

I have a dataset that has multiple x predictor values. To fit a model, I was going to use multiple regression but I looked at the scatter plots for each x value and the y dependent variable and they ...
0
votes
0answers
29 views

A time-series analysis problem

I am trying to run an analysis where I am looking at the time it takes for a building lot to go from having submitted its first construction job application to having earned its first construction ...
1
vote
0answers
28 views

Line of best fit for data with error in x and y

I have a two dimensional dataset which comes from prior statistical analysis. Each point $(x_n,y_n)$ in the dataset has an error estimate $\sigma_{x,n}, \sigma_{y,n}$ for both coordinates. The $x_n$ ...
4
votes
1answer
49 views

Why are points uniformly distributed on a sphere in 3D uniformly distributed in component coordinates?

I've generated uniformly random points on a sphere (in 3D). As expected, all azimuthal angles are drawn with equal probability and it's less likely to draw points close to the poles: However, when ...
1
vote
0answers
41 views

Convolutional neural network fails at the easiest task

This is my first attempt at making a convolutional neural network, and I'm having trouble making it perform the easiest task. Even though each separate part of the algorithm seems to work as expected (...
0
votes
0answers
30 views

How to set mini-batch size in SGD in keras

I am new to Keras and need your help. I am training a neural net in Keras and my loss function is Squared Difference b/w net's output and target value. I want to optimize this using Gradient Descent....
7
votes
4answers
263 views

Why does lasso not converge on a penalization parameter?

To explore how the LASSO regression works, I wrote a small piece of code that should optimize LASSO regression by picking the ...
1
vote
3answers
96 views

Which is the best classifier and with what performance measures?

I tried to implement a Classifier comparison like in the scikit-learn for text classification. I used an 81 instances as a training sample and a 46 instances as a test sample. I tried several ...
0
votes
1answer
21 views

Combining two or more (SVC) models in Python/scikit.learn

I have some data which I use SVC models with 10 fold cross validation and a parameter grid search on (scikit.learn). I observed that the predictions of some folds have low accuracy, whereas remained ...
0
votes
0answers
7 views

What should I expect biweight weighting values to be?

I am working on a project that involves taking the biweight sample variance of a velocity dataset. This is defined as $\sigma_{BI}^2 = N\dfrac{\sum_{|u_i|<1}(1-u_i^2)^4(v_i-\bar{v})}{D(D-1)}$ ...