0
votes
2answers
43 views

Ensemble of models with different feature spaces

BACKGROUND I have data in which the dependent variable is binary with a highly-skewed distribution: <1% records are 1 (doers), >99% records are 0 (non-doers). I'm using logistic regression to ...
3
votes
2answers
101 views

How to calculate p values in logistic regression with gradient descent algorithm

In logistic regression, the gradient descent algorithm for calculating coefficients can be described this way: Until convergence, do $$ \beta := \beta + \alpha \frac{\partial L}{\partial \beta} ...
0
votes
1answer
154 views

Beginner - How can I use ranked values in my Logistic Regression?

I am running a Logistic Regression on some data to predict if a webpage is "good" or "bad". I got the dataset from a finished Kaggle competiton here (train.tsv). I extract the second column of this ...
0
votes
1answer
61 views

Bootstrap aggregation (bagging) of logistic regression classifiers

So I'm taking N bootstrap samples and training N logistic regression classifiers on these samples. Each classifier gives me some probability of being in a binary class and then I average these N ...
0
votes
0answers
39 views

How to decide the number of Hidden layers and number of neurons in an Artificial neural network?

How to decide the number of Hidden layers and number of neurons in an Artificial neural network ? I am having an artificial neural network for a rainfall prediction application which is having four ...
1
vote
2answers
92 views

How to improve a Fraud Classification Model?

I built a classification model (Logistic Regression) in order to classify data in Fraud or Not Fraud. This data is related with online CNP (Card Not Present) transactions and after choosing some ...
1
vote
1answer
189 views

Logistic Regression\SVM implementation in Mahout

I am currently working on sentimental analysis of twitter data for one of telecom company data.I am loading the data into HDFS and using Mahout's Naive Bayes Classifier for predicting the sentiments ...
0
votes
1answer
84 views

Machine learning with trinomial features

I have 100,000 students who have each answered some multiple choice questions. Given their performance I want to work out what the chances are of a particular student answering the next question ...
2
votes
3answers
145 views

Machine Learning to Predict Class Probabilities

I am looking for classifiers that output probabilties that examples belong to one of two classes. I know of logistic regression and naive Bayes, but can you tell me of others that work in a similar ...
1
vote
2answers
89 views

Correlated features produce strange weights in Logistic Regression

I have a data set with highly positively correlated features that I'm classifying with LR. AFAIK correlated weights are not a problem in the same way they are in Naive Bayes - overcounting will not ...
9
votes
2answers
457 views

Intuition behind logistic regression

Recently I began studying machine learning, however I failed to grasp the intuition behind logistic regression. The following are the facts about logistic regression that I understand. As the ...
1
vote
1answer
122 views

Machine Learning System Design: a practical advice

Recently, I stared working on a machine learning competition hosted on Kagge.com. As the first step, a quick and dirty system was developed using Logistic Regression (LR). After running the system ...
5
votes
1answer
261 views

Why does my ROC curve look like this (is it correct?)

I have a ROC curve generated for a multivariate logistic regression. Does it look correct? This is what I've done: Solve $\theta_0 + \theta_1X_1 + \theta_2X_2 ... = Y$ for the $\theta$s Iterate ...
1
vote
0answers
90 views

Validating logistic regression - formulas needed for simulation

This is related to a data classification problem having a Boolean output variable. Summary: Once I perform the ML task using Logistic regression, I get the required coefficients. I use the ...
4
votes
3answers
399 views

How do you report percentage accuracy for glmnet logistic regression?

I am using glmnet where my dependent variable is binary (class 0, class 1). I want to report percentage accuracy of the model. So I use the ...
1
vote
2answers
148 views

Is there overfitting in my modelling approach despite cross-vaidation?

My model is predicting a binomial dependent variable with a rich feature space of 20,000 independent variables. I am using the penalized logistic regression from the glmnet package, which works for ...
5
votes
3answers
99 views

Sharing a model trained on confidential data

I have a regularized logistic regression model using scikit-learn and would like to share it with others, however the data it is trained on is confidential and must remain protected. The model uses ...
3
votes
0answers
357 views

Online logistic regression? [duplicate]

Here is my problem: I am developing an embedded system for some classification task. I am using Logistic Regression as my classifier. Now I train my classifier, and download my model on to my machine. ...
0
votes
0answers
134 views

Fitting a logistic regression using lassoglm in matlab [duplicate]

I am fitting a logistic regression model using lassoglm in matlab. I issued the following command ...
-1
votes
1answer
65 views

How does Logistic regression classifier modelize the dataset?

I'm working on a system that be able to detect the hand contour. So I have 270 instance in my dataset: 7 class of hand contour, 8 feature vectors of each instance. Firstly, I used Weka to determine ...
2
votes
1answer
436 views

Logistic regression as classifier and overfitting

I am using logistic regression to classify data into two classes. The variable to predict (Y) is either 0 or 1. I have found a rather good estimation of Y by logistic regression, and ended up using ...
3
votes
0answers
107 views

Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
1
vote
0answers
165 views

Explain ridge in the log-likelihood for Logistic Regression classifier

What does the ridge parameter change in a Logistic Regression classifier as for example implemented in Weka Logistic classifier "Parameter -R ridge". The paper describing the underlying theory: Ridge ...
3
votes
1answer
462 views

Logistic regression: how to choose negative examples for training set

I want to predict the probability of rain based on the measured weather parameters like temperature, humidity, etc. Let's not get into why I want to do that despite the fact that weather websites ...
4
votes
2answers
565 views

Importance of variables in logistic regression

I am probably dealing with a problem that has probably been solved a hundred times before, but I'm not sure where to find the answer. When using logistic regression, given many features $x_1,...,x_n$ ...
5
votes
2answers
1k views

How to combine results of logistic regression and random forest?

I am new to machine learning. I applied logistic regression and random forest on a same dataset. So I get variable importance (absolute coefficient for logistic regression and variable importance for ...
7
votes
1answer
164 views

Logistic input with Gaussian noise

Both the logistic function and standard deviation are usually denoted $\sigma$. I'll use $\sigma(x) = 1/(1+\exp(-x))$ and $s$ for standard deviation. I have a logistic neuron with a random input ...
3
votes
2answers
490 views

SVM prediction sensitivity when compared to neural networks and logistic regression

Basically I want to classify a rather rare status (about 2% of the 2000) with some predictors. I have used logistic regression, neural network, and Support Vector Machines to do it. All the ...
2
votes
1answer
391 views

Logistic Regression with weighted instances

I'm working on implementing a logistic regression algorithm in code. It's based this link. Unfortunately, the paper doesn't talk about weighting the individual examples $x_{i}$. I think the relevant ...
0
votes
1answer
742 views

Logistic Regression Algorithm?

I am studying on some Machine Learning concepts. I am looking for logistic regression(multiclass) and logistic regression classifier and I should learn how to change it to penalize large weights. I ...
0
votes
1answer
120 views

Algorithm convergence with logistic classifier

I am doing a college classification project, in which I am required to classify some handwritten digits. Assume that my input is a N*D where D is the number of features in each input sample and I need ...
0
votes
2answers
72 views

logistic regression always yielding increasing f'n when should sometimes be decreasing (using R)

I'm modeling a set of outcome data the depends on two parameters: time, T -100 < A < 100 I've done logistic regression using R with the command: ...
3
votes
0answers
150 views

Shifted intercepts in logistic regression

I have a question about the effects of shifting the intercept in a logistic fit on the mean of a particular transformation of the scores. Here is the notation I will be using for the question. The ...
2
votes
0answers
173 views

Standardize/normalize power law distribution for machine learning

If my data follows a normal distribution I can standardize it for a machine learning algorithm, e.g. logistic regression, by subtracting the mean and dividing the result by the standard deviation. ...
5
votes
2answers
245 views

Assumptions of GAM

I am looking to understand the assumptions of using a generalized additive model. 1) Are the assumptions the same as the assumptions for each equivalent link function in a generalized linear model - ...
0
votes
1answer
77 views

machine learning to predict equations / parameters of equations

I'm not sure if machine learning is the best way to do this, but I'm interested in seeing if this problem is feasible. Normally you use machine learning for classification. ie.given the size of a ...
2
votes
4answers
539 views

logistic regression. How to get dual function?

Given pairs $(x_i, y_i), x_i \in R^n , y_i \in R$ we want to solve minimization problem (logistic regression):$\min \frac{1}{2} ||w||^2 + \sum_i^{i=m}\log(1+\exp(-y w\cdot x_i))$. How to do that? I ...
4
votes
5answers
304 views

Performance metric for algorithm predicting probability of low probability events

I am writing a program to predict the click through rates of online ads. Two important notes about this problem: click through rates are very small (like 0.1%) click through rates depend on several ...
4
votes
2answers
1k views

Kernel logistic regression

I heard Kernel Logistic Regression is a classical combination of kernel methods and Logistic regression, but I cannot find any major reference (book, or paper) on this topic. Can you give me any ...
0
votes
1answer
143 views

Are there popular dimensionality reduction tools for classification type supervised learning?

I am familiar with PCA as a linear transformation in order to align the axis of the IV space with the directions of maximal variance in order to possibly be able to reduce the dimensionality of the ...
6
votes
0answers
574 views

Maximum entropy classifier and sentiment analysis

I am doing a project work in sentiment analysis (on Twitter data) using machine learning approach. In order to find the 'best' way to this I have experimented with naive Bayesian and maximum entropy ...
2
votes
1answer
148 views

Remove duplicates from training set for classification

Let us say I have a bunch of rows for a classification problem: $$X_1, ... X_N, Y$$ Where $X_1, ..., X_N$ are the features/predictors and $Y$ is the class the row’s feature combination belongs to. ...
6
votes
1answer
820 views

Solving for regression parameters in closed-form vs gradient descent

In Andrew Ng's machine learning course, he introduces linear regression and logistic regression, and shows how to fit the model parameters using gradient descent and Newton's method. I know gradient ...
7
votes
0answers
231 views

Updating classification probability in logistic regression through time

I am building a predictive model that forecasts a student's probability of success at the end of a term. I’m specifically interested in whether the student succeeds or fails, where success is usually ...
3
votes
1answer
211 views

Weights of radial basis function networks

If I use radial basis function networks (RBFNs) for probability estimation by plugging the output of the RBFNs into the Logistic function are weights between 0 and 1 sufficient?
2
votes
1answer
54 views

Adjusting existing algorithm - likelihood for presence-only data

Logistic regression fits a model that predicts a binary variable whilst performing a logit transformation of the linear combination (LC) of predictors: 1/1 + exp(-LC). I have a working machine ...
4
votes
3answers
1k views

How do I handle predictor variables from different distributions in logistic regression?

I am using logistic regression to predict y given x1 and x2: z = B0 + B1 * x1 + B2 * x2 y = e^z / (e^z + 1) How is logistic regression supposed to handle cases ...
1
vote
1answer
138 views

Identifying culs-de-sac (circular housing arrangements) with GIS data

I'm thinking about a new project, so I don't have data yet, but I plan on collecting GIS information for houses within a state. Usually in the U.S., these dead-end streets will have a large circle ...
4
votes
2answers
397 views

GDA and LDA terminology

Can the terms LDA (Linear Discriminant Analysis) and GDA (Gaussian Discriminant Analysis) be used interchangeably? Do they often refer to the same thing?
2
votes
3answers
2k views

Logistic regression with LBFGS solver

Is there any open source library or code which implements Logistic Regression using L-BFGS solver? I would prefer Python, but other languages are welcome, too.