# Tag Info

20

Reasons to prefer zero-one coding of binary variables: The mean of a zero-one variable represents the proportion in the category represented by the value one (e.g., the percentage of males). In a simple regression $y = a + bx$ where $x$ is the zero-one variable, the constant has a straightforward interpretation (e.g., $a$ is the mean of $y$ for females). ...

14

Let $y_{ij}, {\boldsymbol x}_{ij}$ denote the response and predictor vector (respectively) of student $i$ in school $j$. (1) For binary data, I think the standard way to do variance decompositions analogous to those done for continuous data is what the authors call Method D (I'll comment on the other methods below) in your link - envisioning the binary ...

13

I take it the focus of the question is less on the theoretical side, and more on the practical side, i.e., how to implement a factor analysis of dichotomous data in R. First, let's simulate 200 observations from 6 variables, coming from 2 orthogonal factors. I'll take a couple of intermediate steps and start with multivariate normal continuous data that I ...

12

It makes it easier to interpret the results. Suppose you had some height data: Woman A: 165 Woman B: 170 Woman C: 175 Man D: 170 Man E: 180 Man F: 190 and you took a regression of the form Height = a + b * Gender + Residual. With the 0,1 dummy variable you would get an estimate of a of 170 being the average height of the women and of b of 10 being the ...

11

Regarding shopping cart analysis, I think that the main objective is to individuate the most frequent combinations of products bought by the customers. The association rules represent the most natural methodology here (indeed they were actually developed for this purpose). Analysing the combinations of products bought by the customers, and the number of ...

11

To sum up, with n=45 subjects you're left with correlation-based and multivariate descriptive approaches. However, since this questionnaire is supposed to be unidimensional, this always is a good start. What I would do: Compute pairwise correlations for your 22 items; report the range and the median -- this will give an indication of the relative ...

8

A popular approach towards solving class imbalance problems is to bias the classifier so that it pays more attention to the positive instances. This can be done, for instance, by increasing the penalty associated with misclassifying the positive class relative to the negative class. Another approach is to preprocess the data by oversampling the majority ...

8

The question of dichotomous or binary variables in PCA or Factor analysis is eternal. There are polar opinions from "it is illegal" to "it is alright", through something like "you may do it but you'll get too many factors". My own current opinion is as follows. First, I deem that binary observed variable is descrete and that it is improper to treat it in any ...

8

One answer is "no." Another is, "of course." No To simplify notation, let $\lambda(x) = 1/(1 + \exp(-x))$, the inverse logit. Because $\lambda(x) = 1 - \lambda(-x)$, $$\beta_0 + \beta_1 \lambda(x) = (\beta_0 + \beta_1) - \beta_1 \lambda(-x)).$$ Therefore it is impossible to distinguish the parameters $(\beta_0, \beta_1, \beta_2, \beta_3)$ from ...

8

Your thinking is good. John Tukey recommended binning by halves: split the data into upper and lower halves, then split those halves, then split the extreme halves recursively. Compared to equal-width binning, this allows visual inspection of tail behavior without devoting too many graphical elements to the bulk of the data (in the middle). Here is an ...

7

You are decribing "categorical variables" (represented in R a factors). These can be incorporated into almost any statistical model by being assigned levels. You would need to give more detail about your particular problem in order to be advised on a particular method. Edit If the response variable has two possible outcomes, you might consider binomial ...

7

As touched upon by @scortchi the reviewer was operating under the false impression that it is not possible to model nonlinear effects of predictors on the logit scale in the context of logistic regression. The original model was quick to assume linearity of all predictors. By relaxing the linearity assumption, using for example restricted cubic splines ...

6

Did you read the original Olsson (1979) paper? I believe it still provides the best description of what polychoric correlations are (although I've probably skimmed only 10% of the existing literature, I have to admit; at some point, it just gets too repetitive of the limited number of ideas though). Polychoric correlations are ML estimates of the ...

6

This is a method using Monte Carlo to show whether a result is correct. Our Null Hypothesis H_0 is that our dataset does not have an interesting clustering. Our alternative hypothesis H_1 is that our dataset contains an interesting clustering. Hereby we think of interesting as, more interesting than the clustering structure of a random dataset with the ...

6

Yes, you can get the predicted probability that an observation is yes from a logistic regression model. If you have the estimated coefficients from your model fit, you can use those to get the predicted probabilities thusly: $$\widehat{p(y_i=1)} = \frac{\exp(\beta_0 + \beta_AA + \beta_BB + \beta_CC)}{1+\exp(\beta_0 + \beta_AA + \beta_BB + \beta_CC)}$$

5

What information is lost: It depends on the variable. Generally, by dichotomizing, you're asserting that there is a straight line of effect between one variable and another. For example, consider a continuous measure of exposure to a pollutant in a study on cancer. If you dichotomize it to "High" and "Low", you assert that those are the only two values that ...

5

With regards to whether you should compute agreement for each item, this depends somewhat on how you plan to analyse the data. If you plan to compute scale scores (e.g., sum up the binary responses or sum up the likert responses) to form a scale, then you could perform a reliability analysis on the scale scores. In this situation, you may be starting to ...

5

Let $Y_i$ be $1$ if patient $i$ is positive and $0$ otherwise and let $X_{i1}, ..., X_{ip}$ be the $p$ predictor variables for patient $i$. A standard tool for determining whether variables are related to an individual's probability that $Y_i = 1$ is the logistic regression model:  \log \left( \frac{ P(Y_i = 1 | X_i) }{P(Y_i = 0| X_i)} \right) = \beta_0 ...

5

Yes, this is common with an imbalance in training data and some types of relationships. Suppose bad students pass a tough course with probability $0$, while good students pass the course with probability $1/3$. If the only information you get to observe is whether the student is good or bad, then your most accurate prediction is that the student will fail ...

5

In some sense, I think it depends on what you mean by "adding value," but let's assume you are simply trying to judge whether your particular model is doing better than guessing 0 for all your cases or even randomly guessing. Since you are talking about AUC, I'm assuming you are drawing an ROC curve based on the cutoff for a logistic classifier. If you ...

5

A binary logistic regression is generally used for fitting a model to a binary output, but formally the results of logistic regression are not themselves binary, they are continuous probability values (pushed to zero or 1 by a logit transformaion, but continuous between 0 and 1 nonetheless). It sounds like the software you are using is rounding the output ...

5

There are quite a few ways to work around it. Jittering the variables mildly to smear the lines apart First, since both age and the outcome are nicely discrete, we can afford to mildly jitter them in order to show some trends. The trick is to use transparency in the line color so that it's easier to discern the magnitude of overlapping. library(geepack) ...

4

You are conducting a one-sided test of a difference of proportions. Because all four outcomes--A, not A, B, not B--occur often (70 times or more in this case), the Normal approximation to the Binomial distribution will be just fine. Let $a$ be the number of occurrences of A, $b$ the number of occurrences of B, and $n$ the total sample (about 350). Under ...

4

Your addendum suggests that A and B are dependent samples since they come from the same "instance", i.e., patient. In that case, I propose the McNemar-Test which tests for a (two-sided) hypothesis of unequal proportions: > N <- 350 > A <- factor(rbinom(N, size=1, p=0.6), labels=c("pos", "neg")) > B <- factor(rbinom(N, size=1, ...

4

It sounds like you are trying to predict your boolean response, yes? This is called classification. Logistic Regression is the obvious choice here, but there are other methods too. You can't do traditional regression, because the response is not a real number. The lookup variables are called nominals, and can be dealt with in regression by using "dummy" ...

4

Your class sample sizes do not seem so unbalanced since you have 30% of observations in your minority class. Logistic regression should be well performing in your case. Depending on the number of predictors that enter your model, you may consider some kind of penalization for parameters estimation, like ridge (L2) or lasso (L1). For an overview of problems ...

4

Dichotimization adds magical thinking to data analysis. This is very rarely a good idea. Here is an article by Royston, Altman and Sauerbrei on some reasons why it is a bad idea. My own thoughts: if you dichotomize a dependent variable, say, birth weight at 2.5 kg (this is done all the time) then you are treating babies who are born at 2.49 kg just like ...

4

That it doesn't work does not come from the unbalanced size of the groups, but from the smallness of one of the groups. Downsampling the larger group is ok but does not help with overfitting. (BTW, there is an easy and elegant way to correct the predictions from the downsampled model, by adding ±log(r) to the linear terms where r is the downsampling ratio.) ...

4

This problem surfaces in virtually all classification approaches, whether logistic regression, support vector classification, or Naive Bayes classification. There are two intertwined issues: A model trained on an imbalanced dataset may overfit in the sense of acquiring a bias in favour of the majority class. When evaluating this model on a test dataset ...

4

When the repeated-measures or related-samples data are dichotomous Friedman nonparametric test degenerates into Cochran's Q test (Friedman's chi-square statistic becomes identical to Cochran's Q statistic) which is the extension of McNemar's test from 2 to several related samples. McNemar's uses exact binomial computation of p-value, while Cochran relies on ...

Only top voted, non community-wiki answers of a minimum length are eligible