Python is a general purpose programming language designed for ease of use. It is a commonly used platform for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `...

learn more… | top users | synonyms

0
votes
0answers
7 views

TensorFlow: Implementing Spearman Distance as the Objective Function

In order to make my issue reproducible, I have generated the following .csv file using iris flower data set (10 arbitrary rows, all columns standard normalized) and ...
0
votes
0answers
3 views

In ARIMA with differencing (I>0), how are prediction intervals for forecasts calculated for the original (undifferenced) series?

In ARIMA with differencing (I>0), when producing forecasts, how can I go from prediction intervals for the differences to prediction intervals for the undifferenced series? My guess is, since the ...
0
votes
0answers
8 views

Scikit Learn Python PCA and Feature Agglomeration feature indices?

I am using Scikit learn to perform PCA and Feature Agglomeration of my survey data. I have done the following : ...
1
vote
0answers
18 views

How to calculate p-value for significance test of variables in linear regression

I am trying to find the significance of predictors while using different linear regression models (I am using Python scikit-learn). Scikit-learn does not provide the pvalues of predictors( at least i ...
0
votes
1answer
29 views

Prove Causation [on hold]

Could anyone share me some good material to start to read How one can prove causation or the steps to prove it using statistical test via code e.g python or R? Thanks in advanced.
0
votes
0answers
6 views

Find features that contribute to the cluster formation in survey data?

My survey has the following components: 1. Some descriptive data such as gender, employment, age 2. All other data is collected on a equally spaced scale of 1 to 5 I want to do a test to see if there ...
0
votes
0answers
9 views

Getting survival function estimates group by attribute level in Lifelines

I have a challenge with using Lifelines for Kaplan Meier estimates. I have a variable column called worker type (Full Time, Part Time, etc) that I would like to group the KM estimates for, then append ...
-1
votes
0answers
23 views

How to put model into Database [on hold]

I am trying to create system recommendation. I implemented some algorithms from the ebook "Program Collective Intelligence" and "A Programmer's Guide to Data Mining" to create recommendation. My ...
-1
votes
0answers
17 views

Data science : where to find Project Examples using Python with SQL? [on hold]

I’ve learnt Python and SQL, but was suggested to “work in Python to analyze data sets stored in MySQL”. I am following CS109 which uses Python, but it does not really use SQL https://www.quora.com/...
1
vote
1answer
32 views

Regression + Time series

I have time series data about sales/day, but i also want to include other data (static/dynamic) to forecast the time series. Is it possible to combine ARIMA model and regression models to achieve the ...
1
vote
0answers
9 views

Title and Occupation Correlation Model

I have a large set of data that contains both a person's title (e.g., "CEO", "Bank Teller", "Fireman", etc.). and occupation codes (e.g, "A1": Managerial, "A2": Laborer, etc.). I need to build an ...
0
votes
0answers
16 views

Data analysis with python [on hold]

To APPLY for a post data scientist, the employer ask me to analyze data and find patterns and relationships. For that, I started preparing the data, and extract correlated variables. I used Python. I ...
0
votes
0answers
12 views

Using sentiment data to predict stock prices

I have implemented a model to predict "last" price of stock of 5 days ahead. I have used RandomForestRegressor. In addition to this, I have a data set of sentiments. It is the data from social media ...
0
votes
0answers
8 views

Evaluation of a binary classifier in regards of rounding, taking average scores

I'm using scikit-learn to evaluate a binary classifier: There are 2 Methods to get the Precision, Recall and F1 scores: 1.classification_report: ...
0
votes
0answers
12 views

Applying support vector regression to vector-valued functions?

So, just as a preface: this is my first time posting to this site, and I am also a machine learning beginner, so I apologize if my question is dumb or if I do something wrong, format-wise. Alright, ...
1
vote
0answers
50 views

Latent Semantic Analysis- Theme extraction [closed]

I have a large collection of text documents I am trying to group using Singular Value Decomposition for dimension reduction(popularly known as LSA) followed by k-means clustering. To be able to ...
0
votes
0answers
10 views

Programs to read an arbitrary image and convert it to the format of MNIST dataset [closed]

I want to make a program that turns a given image into the format of the MNIST dataset, as a kind of exercise to understand the various preprocessing steps involved. But since the description of how ...
0
votes
0answers
17 views

How to do sentence boundary detection? [closed]

I have a string as My name is sushil feeling bad My age is 30 and hobbies are none thanks for asking me name varun i am good at cricket and my age is 40 i have done a lot of work in research area , ...
2
votes
1answer
29 views

How to avoid k-means assigning different labels on different run?

I have unlabeled dataset. I am running k-means flat cluster with 2 number of clusters. Every time I run the below program the labels are different. How can I make labels not to change. Is it even ...
0
votes
0answers
13 views

How to group/cluster variables/features using Python? [closed]

I have 200 variables and 1 million records. I want 20 clusters, so that I can pick out top variable (based on Information Values of the variables) in each cluster.
0
votes
0answers
18 views

Why are LSTMs performing worse on the training set with higher cell size?

I'm attempting to train a (single-layer) LSTM-based autoencoder using the seq2seq framework within Tensor Flow. The input is sequences of numerical values. The sequences each contain 40 values. As a ...
-2
votes
1answer
52 views

If wrong retry - negating automatically incorrect answers [closed]

Say the first event is someone buying product, and the second them selling it. I need to predict how long they'll have the product for from today, but my dataset and my model only predict the overall ...
0
votes
0answers
7 views

Why is xgboost giving bad scores and taking a very long time? [migrated]

I am completely new to xgboost so thank you very much for any help in advance. I have this simple script. ...
2
votes
0answers
40 views

How to know which statistical model to use for categorical data?

I'm new to statistical analysis. I'm trying to conduct an analysis of datapoints and possible correlations between them using Python's sci-kit learn library. My data is categorical. For example, a ...
3
votes
1answer
43 views

Neural network library for Python for Microsoft Windows

I have been having trouble with selecting a good library for Neural network algorithms in Python. TensorFlow isn't supported on windows. Theano is still in the beta phase of development. PyBrain too ...
0
votes
0answers
20 views

Cross-validating an ordinal logistic regression in R (using rpy2) [migrated]

I'm trying to create a predictive model in Python, comparing several different regression models through cross-validation. In order to fit an ordinal logistic model (...
0
votes
1answer
17 views

Subset documents based on tfidf weights

I am new with text mining therefore please bare me if this question sound too easy for others. But I tried to find out the solution with no success. I am working on a project of document ...
0
votes
0answers
16 views

Python Scipy expanded Shapiro Wilks test

Does anyone know for certain if scipy.stats.shapiro uses the expanded Shapiro-Wilks test approach by J.P. Royston? Should I only use this function for sample sizes ...
0
votes
0answers
13 views

PyMC: Using maximum a posteriori on vector/matrix-valued variables

I am using PyMC to find a maximum a posteriori estimate for some data. The data is all vector-valued. I am trying to estimate my variable x based on a matrix-valued ...
0
votes
0answers
15 views
0
votes
0answers
8 views

Data preparation of a new record on the fly [migrated]

I am facing problem in the implementation of Data Preparation of a single record on the fly. I am loading the model from disk and I need to to make prediction against it. Lets Say, I have 3 ...
0
votes
1answer
25 views

What do you expect a 2-sample t-test for the same two samples to return?

I am using scipy in Python and the following return a nan value for whatever reason: ...
0
votes
0answers
58 views

Sigmoid cross-entropy loss layer: What is wrong with the gradient?

Could you help me find out what is wrong with my gradient? I am trying to implement a sigmoid cross-entropy loss layer in Python (similar to caffe's, but my c++ is not good http://caffe....
0
votes
1answer
20 views

Net surgery to convert to FCN: am I doing it right?

I am kind of stuck here, hopefully someone can throw some light on it. To demonstrate my problem I will try and simplify my network architecture for sake of convenience. I have a pipeline: Training: ...
1
vote
0answers
26 views

Row Correlation Heatmap Pandas

I'm trying to find any relationship/patterns between a large number of rows in a dataset (~2000) and I'm thinking of using a correlation heatmap. However, after transforming the df using ...
2
votes
1answer
35 views

Determine the feature weights with a regression?

I have a set of houses with their features (location, size, number of rooms, etc … and the y is the price). In the future I will have a new house without the price. My goal is to find the 20 closest ...
1
vote
2answers
71 views

How to categorize (lookup) brands with spelling mistakes

I have a dataset with around 4 million rows. The data has only column which contains around 50000 unique brand names (e.g. IBM, Google, Adobe, Microsoft etc.). I also have a lookup table which ...
1
vote
0answers
16 views

Difference between two methods of random point generation [migrated]

In order to do a monte carlo simulation to estimate expected distance between two random points in $n$ dimensional space I discovered the following two similar looking methods to generate random ...
0
votes
0answers
21 views

Finding patterns in rows instead of columns

Let's say I have a dataset and instead of looking at the patterns and relationships between columns, such as plotting the distributions or a correlation matrix, I want to look at the relationship ...
2
votes
2answers
84 views

How can I calculate variance of a very large random variable?

I'm implementing an algorithm which recieves as input samples from a random variable with an unknown distribution. The random variable is extremely large so my input is logarithmic, and still large (...
-1
votes
0answers
29 views

Time Series Novelty Detection

I am performing very frequent analysis on many datasets of one variable, 15 minute increment, time series data. I want to check if the most recent observation of each dataset is anomalous. The various ...
1
vote
1answer
24 views

SVM labeling more classes does not result in higher accuracy?

I used SVM to predict the ranking score of muffin recipes. X is a numpy array of ingredient amounts of a certain recipe and y is the label according to the online ranking score. First I labelled my ...
0
votes
0answers
21 views

Number of Sinusoids Feature Extraction for MUSIC Algorithm

I am a complete novice to machine learning, but I wanted to ask this question to get me started on a path to solving this problem. I have been working on a radar project where I am receiving a signal ...
2
votes
0answers
30 views

How does Principal Coordinate Analysis (PCoA) work, as compared to PCA? [duplicate]

I am familiar with PCA from Making sense of principal component analysis, eigenvectors & eigenvalues where you either normalize the data (to standard normal or ...
0
votes
1answer
16 views

How do I learn a simple cut-off value between 2 classes given one-dimensional data?

Given a set of data which consists of a single real number and a class, I want to find a value (i.e., the inflection point if we were talking about logistic regression) which would lie right at the ...
0
votes
2answers
90 views

random forest how to use the results

I used the package for random forest. It is not clear to me how to use the results. In logistic regression you can have an equation as an output, in standard tree some rules. If you receive a new ...
0
votes
0answers
29 views

Smart sampling techniques to ensure boundary/edge cases are included

In an effort to reduce the time it takes to test software against various data sets I would like to do create an application that looks at a data set and creates a "smart sample" from the it. The ...
0
votes
0answers
17 views

How to use weights with Elasticnet regression in python?

I am using Elasticnet from scikit-learn in python, I've also used Glmnet package in R for prototyping. I want to use weights in Elasticnet which apparently is not available as an option/argument in ...