All Questions

Tagged with
Filter by
Sorted by
Tagged with
0
votes
1answer
37 views

If logistic regression is a linear classifier why does it fail on linearly separable data?

Logistic regression is a linear model, decision boundary generated is linear. If the data points are linearly separable, then why does Logistic regression fail? Shouldn't it perform better on data ...
1
vote
1answer
36 views

Can a linear and logit model have the same shape?

While I was working on an exercise based this book, I discovered something interesting. When I fit a logit and simple linear probability model on the data (see code below), the predictions are almost ...
2
votes
1answer
2k views

glm.fit: fitted probabilities numerically 0 or 1 occurred however culprit feature is numeric

I've been receiving the warning message in the title and have reviewed posts such as e.g. this one. I would like to understand how this feature has perfect separation with the target variable, since ...
0
votes
0answers
16 views

Can visualization help to identify a dataset is linearly separable with polynomial features?

This data set cannot be linearly separable. If the polynomial and interaction features $X_1^2, X_2^2, X_1 \times X_2$ are used, can the data set linearly separable? I wanted to know there is any way ...
0
votes
0answers
38 views

Visualising Generalised linear models

I read about linear regression where we assume, the response is linear and the noise $\epsilon$, follows $N(0, \sigma^2)$ (Gaussian noise model), this leads us to conclude $E[Y|X] = b^*x$ and that the ...
0
votes
1answer
59 views

Curved regression lines

I had already asked a similar question here, but I'm experiencing the same problem for a different data-set and for a different family of mixed models. My response variable is a binary outcome of ...
1
vote
0answers
53 views

Discrete-time survival model as linear probability model

Discrete-time survival (event history) models are typically estimated using a nonlinear transformation such as logit, probit, or completementary log-log. Logit assumes proportional odds, and similarly ...
0
votes
1answer
34 views

What does ${}^{\mathsf{T}}$ mean in the context of the binomial distribution?

In a presentation of the binomial distribution on this website, the probability of the success of a trial is presented as $\pi=g^{-1}(\eta)$, with $g$ the inverse link function. $\eta=\alpha+\...
1
vote
2answers
211 views

Binomial(n,p) GLM - Can n be passed as an explanatory variable?

For example, lets say I'd like to fit a binomial GLM to predict D, the number of deaths in a hospital on a given day, where n is the number of patients, and p is the probability a patient dies. What ...
5
votes
2answers
812 views

How to have a “None of the above” category in a Logistic Regression?

I have a Logistic Regression supervised classifier that is trained on n=365 observations and m=179 attributes with ...
1
vote
0answers
53 views

why is colinearity not an issue with linear modeling using the hashing trick (ie vw, fasttext)?

Usually when I use linear models I have to do feature selection or use pca to prevent the detrimental effects of collinearity. I am wondering why algorithms using linear models with the hashing ...
-1
votes
2answers
455 views

R, linear regressions and contingency tables

I'm sorry this may be very simple and stupid but I'm new to R. I've tried everywhere on the net to find a solution but nothing. I have this data set: ...
0
votes
0answers
42 views

Regression on non-normal DV + Heteroskedasticity

I have a DV scored from 1 to 3 from a questionnaire: respondents were given an integer score according to their responses. I need to compare the mean scores between two groups using a regression (...
5
votes
2answers
6k views

How do I avoid computationally singular matrices in R?

I'm fitting a logistic regression model (with R's caret package) to data here. I aim to predict whether Hillary or Trump will ...
0
votes
1answer
35 views

How to pick two features that can linearly separate two classes as much as possible

I have a few dataset, each has about 120 features/dimensions and two classes (e.g. A and B) in it. Now I'd like to visualize the dataset from just two dimensions without doing any dimensionality ...
1
vote
0answers
32 views

Please help. my parameter estimates are different from what is expected.

I'm building a binary logit model to estimate the crossing behaviour of pedestrians on the freeway. The alternatives (Y|0,1) are: 0 = cross directly and 1 = use a facility. Generic attributes include ...
0
votes
1answer
34 views

Can Relative Risk mislead us when choosing predictors for a logistic model?

My friend taught me to use Relative Risk as a guide to check if my coefficients make sense. For example, I have a propensity to default model, where the variable fl_default is equal to 1 if the ...
1
vote
0answers
123 views

How can I use residual plots to determine whether or not a linear classifier is suitable?

I read on this page, that residual plots can be used to determine whether or not a linear model for regression is suitable. I am working on a binary classification problem, with the classes 0 and 1. ...
14
votes
2answers
8k views

Matrix notation for logistic regression

In linear regression (squared loss), using matrix we have a very concise notation for the objective $$\text{minimize}~~ \|Ax-b\|^2$$ Where $A$ is the data matrix, $x$ is the coefficients, and $b$ ...
2
votes
0answers
883 views

Marketing/Sales Mix/Response Models: approaches and comparisons

CV/SO Community: I am probably skirting (or crossing) the line of the preference for questions that can be answered vs. those that can (only) be discussed. That said, I'm trying to wrap my head ...
2
votes
0answers
1k views

How to know when to use Kernel SVM and not Linear SVM?

If I have more than 3 features in a dataset, then I can't visualize them to see if my classes are scattered in a non linear fashion. So how do I know when is the right way to use linear model with non-...
6
votes
1answer
10k views

Why KNN is a non linear classifier ?

How do we decide if a classifier is linear or non linear ? What property/characteristic makes a classifier linear or non linear ? Eg: Why SVM is a linear classifier ? Why Logistic Regression is ...
2
votes
0answers
226 views

Logistic regression vs. linear regression on class probabilities

I have a bunch of data points, each of which represent a success or failure. Each data point is from one of ~40 conditions, each of which contains approximately 40 data points. All of my predictor ...
0
votes
0answers
349 views

Logistic regression: Estimation of marginal effects of predictors

I ran a logistic regression analysis with 12 independent variables (predictors). I heard that I could estimate the average marginal effects of these predictors using a linear regression model. Could ...
0
votes
0answers
139 views

Linear regression for classification

Suppose, I have a classification problem with 2 classes (0 and 1) and evaluation criteria is AUC. I used the following method: fit a linear regression and then pass its predictions through the ...
1
vote
1answer
102 views

How to determine whether a dataset can be learned by Logistic regression?

As far as I know, Logistic Regression can deal with data in which positive and negative samples can be separated by a linear hyperplane. But if the data cannot be separated by a hyperplane, it cannot ...
0
votes
1answer
62 views

Modelling interaction

How does adding interaction term in the model adjust for it or why do we need to add interaction? I am working on logistic regression model with treatment and race as predictors. I have added ...
1
vote
1answer
649 views

adjustment of covariates in linear model

I am trying to understand the adjustment of covariates in the linear model such as multiple logistic regression. How does adding a covariate adjusts the coefficients for that covariate (any intuitive ...
3
votes
0answers
643 views

Latent variables motivation for ordinal and binary logistic regression [duplicate]

The latent variable motivation for losgistic regression goes thus. There exist $Y^*=\beta^tX+\epsilon$ which is continuous. We can only observe $Y$ at specific thresholds of $Y^*$, say at $Y^*\leq \...
0
votes
1answer
21 views

single categorical DV, single countinuous IV

Background: I have a set of student answers to some questions and also their scores, whether the answer is correct or not (1 - not correct, 2 - somewhat correct, 3 - correct). I also have for each ...
1
vote
1answer
973 views

Can univariate linear regression be used to identify useful variables for a subsequent multiple logistic regression?

Does the $R^2$ (or some other statistic) from a univariate linear regression tell me anything about how it would work in a logistic model? What if I normalized the data to mean zero? I'm doing ...
1
vote
1answer
2k views

Big coefficient difference between logit/probit and linear probability model

Setting: Large N, short T panel dataset. Very few 1's (probably 1 percent), most zeroes. I estimate a logistic regression and find a negative significant coefficient on the IV of interest. ...
0
votes
1answer
570 views

When to take logarithms of a variable such as the Herfindahl Index?

Currently I am skimming through a couple of papers in well established journals! I became curious when I found papers with linear regression models using the Herfindahl index as the dependent ...
0
votes
2answers
2k views

Inclusion of significant interaction term in logistic regression table versus stratification for data presentation

This is a general question on logistic regression result reporting for a publication. We have an example where two well correlated ($r=0.4, p=0.001$) blood parameters (...
1
vote
1answer
656 views

How to determine the best relationship (linear, log, etc.) between input predictor variable(s) and output variable for multiple linear regression?

I am trying to determine the most accurate relationship between two variables (each predictor versus the output eventually). I want to know if the relationship is linear, or log-linear, or log-log, or ...
122
votes
3answers
257k views

What is the difference between linear regression and logistic regression?

What is the difference between linear regression and logistic regression? When would you use each?
3
votes
1answer
324 views

Difference between linear regression prediction intervals and logistic regression targets

I am trying to understand the difference between logistic regression probabilities and linear regression prediction intervals. For example, let's say we have a database of student test scores in the ...