Questions tagged [binary-data]

A binary variable takes one of two values, typically coded as "0" and "1".

Filter by
Sorted by
Tagged with
66
votes
4answers
48k views

Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
38
votes
1answer
50k views

Doing principal component analysis or factor analysis on binary data

I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
34
votes
1answer
32k views

Is there Factor analysis or PCA for ordinal or binary data?

I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
61
votes
5answers
114k views

Is it meaningful to calculate Pearson or Spearman correlation between two Boolean vectors?

There are two Boolean vectors, which contain 0 and 1 only. If I calculate the Pearson or Spearman correlation, are they meaningful or reasonable?
60
votes
7answers
40k views

Binary classification with strongly unbalanced classes

I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
34
votes
2answers
32k views

How to use both binary and continuous variables together in clustering?

I need to use binary variables (values 0 & 1) in k-means. But k-means only works with continuous variables. I know some people still use these binary variables in k-means ignoring the fact that k-...
6
votes
2answers
31k views

How do you estimate correlations between ordinal and binary data in SPSS?

I'm struggling with my lab report. I have a questionnaire which contains Likert responses (1-7) as well as questions that require only yes or no answers. I want to see if there is a correlation ...
26
votes
2answers
27k views

Similarity Coefficients for binary data: Why choose Jaccard over Russell and Rao?

From Encyclopedia of Statistical Sciences I understand that given $p$ dichotomous (binary: 1=present; 0=absent) attributes (variables), we can form a contingency table for any two objects i and j of a ...
5
votes
1answer
28k views

Hierarchical or TwoStep cluster analysis for binary data?

(This question is an edited version of a question I previously posted which one user recommended would benefit from more focus). I have 2000 questionnaires from respondents which ask 33 different ...
46
votes
4answers
51k views

Would PCA work for boolean (binary) data types?

I want to reduce the dimensionality of higher order systems and capture most of the covariance on a preferably 2 dimensional or 1 dimensional field. I understand this can be done via principal ...
27
votes
7answers
55k views

Why is gender typically coded 0/1 rather than 1/2, for example?

I understand the logic of coding for data analysis. My question below is on the use of a specific code. Is there a reason why gender is often coded as 0 for female and 1 for male? Why is this coding ...
9
votes
3answers
7k views

Binary time series

I have a binary time series: We have 2160 data (0=didn't happen, 1=happened) for one-hour period in 90 days. I want to forecast after these 90 days, where the next 1 will happen, and also Extend ...
6
votes
1answer
6k views

How to interpret the significant interaction of two non-significant main predictors?

I have two non-significant main binary variables in my binary logistic regression model, but their interaction is significant. The variables are centered and no multicollinearity is the case (all VIFs ...
8
votes
2answers
10k views

Can binary data be ordinal?

Binary data is often mentioned as a nominal sub-category, especially in such examples as female/male, smoker/non-smoker, etc. However, binary data with such values as pass/fail, correct/incorrect, ...
12
votes
2answers
2k views

Modelling auto-correlated binary time series

What are the usual approach to modelling binary time series? Is there a paper or a text book where this is treated? I think of a binary process with strong auto-correlation. Something like the sign of ...
10
votes
1answer
2k views

What are dangers of calculating Pearson correlations (instead of tetrachoric ones) for binary variables in factor analysis?

I do research on educational games, and some of my current projects involve using data from BoardGameGeek (BGG) and VideoGameGeek (VGG) to examine relationships between design elements of games (i.e., ...
3
votes
1answer
3k views

Coefficient of determination for binary responses

D.R. Cox and Nanny Wermuth seem to suggest that the coefficient of determination (R squared) is misleading when you have binary responses, in fact if I am understanding well, they are saying that the ...
30
votes
2answers
25k views

Clustering a binary matrix

I have a semi-small matrix of binary features of dimension 250k x 100. Each row is a user and the columns are binary "tags" of some user behavior e.g. "likes_cats". ...
12
votes
3answers
17k views

Distance Metrics For Binary Vectors

I have vectors of same length consisting of 1 and 0. I am trying to find out how similar they are. So far I am using hamming distance that I calculate sum of one vector then sum of second vector and ...
9
votes
3answers
23k views

Calculating Jaccard or other association coefficient for binary data using matrix multiplication

I want to know if is there any possible way to calculate Jaccard coefficient using matrix multiplication. I used this code ...
14
votes
4answers
7k views

Forecasting binary time series

I have a binary time series with 1 when the car is not moving, and 0 when the car is moving. I want to make a forecast for a time horizon up to 36 hours ahead and for each hour. My first approach ...
15
votes
3answers
19k views

What is the effect of dichotomising variables?

When dichotomising variables, what information is lost in the process? How does a dichotomisation help in the analyses?
59
votes
10answers
19k views

Measuring entropy/ information/ patterns of a 2d binary matrix

I want to measure the entropy/ information density/ pattern-likeness of a two-dimensional binary matrix. Let me show some pictures for clarification: This display should have a rather high entropy: ...
23
votes
3answers
6k views

Visualizing the calibration of predicted probability of a model

Suppose I have a predictive model that produces, for each instance, a probability for each class. Now I recognize that there are many ways to evaluate such a model if I want to use those ...
15
votes
1answer
13k views

Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
16
votes
3answers
25k views

Probit two-stage least squares (2SLS)

I was told that it's possible to run a two-stage IV regression where the first stage is a probit and the second stage is an OLS. Is it possible to use 2SLS if the first stage is a probit but the ...
3
votes
2answers
11k views

Predicted Probabilities vs. Marginal Effects using at means or asObserved in Stata 14 Margins [closed]

This is a complicated question, and I'd very much appreciate any help I can get. I've been running on it nonstop for two or three days, and am running out of time. To be clear: This isn't so much a ...
4
votes
3answers
6k views

Does it make sense to interact 2 dummy variables?

I am running a logistic regression, where the dependent variable is marital status. And I assume that educated women are different from women with e.g. high school degree and are less likely to marry. ...
40
votes
5answers
37k views

Should you ever standardise binary variables?

I have a data set with a set of features. Some of them are binary $(1=$ active or fired, $0=$ inactive or dormant), and the rest are real valued, e.g. $4564.342$. I want to feed this data to a ...
19
votes
2answers
11k views

optimizing auc vs logloss in binary classification problems

I am performing a binary classification task where the outcome probability is fair low (aroung 3%). I am trying to decide whether to optimize by AUC or log-loss. As much as I have understood, AUC ...
7
votes
3answers
20k views

How to deal with a skewed class in binary classification having many features?

I am doing data analysis in the mobile ad targeting domain. I have around 18 features and for a combination of these features, the result is either True or False (1/0) depending on whether the ...
18
votes
5answers
5k views

How to do one-class text classification?

I have to deal with a text classification problem. A web crawler crawls webpages of a certain domain and for each webpage I want to find out whether it belongs to only one specific class or not. That ...
8
votes
3answers
3k views

GLM with logit link and Gaussian family to predict a continuous DV between 0 and 1

Can you run a GLM using a logit link with a continuous DV (between 0 and 1)? Generally it's suggested to use a binomial family with a logit link, but I'm guessing that is because the model assumes a ...
6
votes
5answers
1k views

theoretical basis for logistic regression

Sorry if this is obvious but why does it seem that some papers/studies (usually in economics discipline) use utility theory as a basis for discrete choice models while others (often social science) do ...
6
votes
4answers
22k views

Comparing two binary variables of unequal sizes

I measured a binary variable from two different populations, and now I'm trying to find out whether the different populations differ with regards to this variable. I could use a Chi-Square test, but ...
5
votes
1answer
11k views

How to detect multicollinearity in a logistic regression where all the independent variables are categorical and binary?

I'm doing a multivariate logistic regression where all my independent variables are categorical and binary. I have transformed all my categorical variables into dummies in order to have reference ...
3
votes
2answers
5k views

Why don't dummy variables have the continuous adjacent category problem in cluster analysis?

I know that if we use categorical variables in cluster analysis we would assume that the scale is continuous and we don't have this concept of distance between two adjacent categories. But what is the ...
6
votes
3answers
2k views

Visualizing variability from graph

It is written in the book Applied Logistic Regression, Second Edition. By David W. Hosmer and Stanley Lemeshow , p.2 that a problem with the above graph is that the ...
5
votes
4answers
3k views

Evaluate statistical significance of difference between outcomes of tests

I have 400 patients and I apply 4 tests to all of them. This gives me a number of positive and negative for each test. I want to compare the tests 2 by 2 to find out if there is any statistically ...
4
votes
1answer
156 views

Granger Causality Analog for Binary Time Series

Is there a generalized form of granger causality that can be applied to two binary time series? By binary time series I mean an ordered series of values that take values 0 or 1.
4
votes
0answers
6k views

Test for effects of two categorical variables on a binary response variable?

I am looking for a test similar to a 2-way ANOVA that would work on a binary response variable. My response variable is survival of plant seedlings (alive or dead). My explanatory variables are ...
4
votes
1answer
9k views

(Multiple) Correspondence Analysis for count data entered as binary variables

I have a data set, 1014 cases and 55 variables which are binary and is in the form of ...
7
votes
1answer
1k views

Probabilistic classification and loss functions

I am trying to compare several binary classifiers. These classifiers (Gaussian Processes in my case, but it shouldn't matter) give me probabilistic predictions. Let's introduce some notations: $$y_i \...
4
votes
1answer
2k views

Is it acceptable to rotate factors with PCA for binary data?

What issues, if any, might there be in rotating factors in order to obtain factor/component loadings of binary data? Is it acceptable to rotate the factors when doing a traditional PCA? (Assuming I’m ...
4
votes
1answer
1k views

Cluster large boolean dataset

I got a dataset with about 5,000 columns and about 135,000 rows - all fields contain boolean (binary) data. I am looking to classify each of these columns into one of 50 groups, based on similarity. ...
10
votes
2answers
7k views

Recommended procedure for factor analysis on dichotomous data with R

I have to run a factor analysis on a dataset made up of dichotomous variables (0=yes, 1= no) and I don´t know if I'm on the right track. Using tetrachoric() I ...
3
votes
3answers
342 views

Test for aggregation of binary events/successes (binomial/glm??)

this has been vexing me for a while and I can't seem to solidify an answer beyond vague thoughts about Poisson distributions. I think this is a simple problem and I'm missing something obvious. Any ...
1
vote
0answers
1k views

What analysis (in SPSS) should I use for a repeated measures design with dichotomous outcome variable? [closed]

Parts of my question may seem rudimentary but the more I read, the more confused I get so I am going to completely spell out my experiment. In my study, I have created 4 conditions by crossing 2 ...
7
votes
2answers
1k views

What's the name of this correlation/association measure between binary variables?

There are several measures of association (or contingency or correlation) between two binary random variables $X$ and $Y$, among others Pearson's phi coefficient Cramér's V I wonder how the ...
4
votes
2answers
8k views

Covariance of binary and continuous variable

Suppose $y$ is a continuous random variable and $d$ is a binary random variable that takes the value $1$ with probability $p$ and $0$ with probability $1-p$. How do I show that $\text{Cov}(y,d)=(E[y|...