Questions tagged [binary-data]

A binary variable takes one of two values, typically coded as "0" and "1".

1,176 questions
Filter by
Sorted by
Tagged with
46k views

Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will ...
37k views

Binary classification with strongly unbalanced classes

I have a data set in the form of (features, binary output 0 or 1), but 1 happens pretty rarely, so just by always predicting 0, I get accuracy between 70% and 90% (depending on the particular data I ...
18k views

Measuring entropy/ information/ patterns of a 2d binary matrix

I want to measure the entropy/ information density/ pattern-likeness of a two-dimensional binary matrix. Let me show some pictures for clarification: This display should have a rather high entropy: ...
105k views

Is it meaningful to calculate Pearson or Spearman correlation between two Boolean vectors?

There are two Boolean vectors, which contain 0 and 1 only. If I calculate the Pearson or Spearman correlation, are they meaningful or reasonable?
47k views

Would PCA work for boolean (binary) data types?

I want to reduce the dimensionality of higher order systems and capture most of the covariance on a preferably 2 dimensional or 1 dimensional field. I understand this can be done via principal ...
49k views

Doing principal component analysis or factor analysis on binary data

I have a dataset with a large number of Yes/No responses. Can I use principal components (PCA) or any other data reduction analyses (such as factor analysis) for this type of data? Please advise how I ...
33k views

Should you ever standardise binary variables?

I have a data set with a set of features. Some of them are binary $(1=$ active or fired, $0=$ inactive or dormant), and the rest are real valued, e.g. $4564.342$. I want to feed this data to a ...
31k views

How to use both binary and continuous variables together in clustering?

I need to use binary variables (values 0 & 1) in k-means. But k-means only works with continuous variables. I know some people still use these binary variables in k-means ignoring the fact that k-...
30k views

Is there Factor analysis or PCA for ordinal or binary data?

I have completed the principal component analysis (PCA), exploratory factor analysis (EFA), and confirmatory factor analysis (CFA), treating data with likert scale (5-level responses: none, a little, ...
24k views

Clustering a binary matrix

I have a semi-small matrix of binary features of dimension 250k x 100. Each row is a user and the columns are binary "tags" of some user behavior e.g. "likes_cats". ...
52k views

Why is gender typically coded 0/1 rather than 1/2, for example?

I understand the logic of coding for data analysis. My question below is on the use of a specific code. Is there a reason why gender is often coded as 0 for female and 1 for male? Why is this coding ...
26k views

Similarity Coefficients for binary data: Why choose Jaccard over Russell and Rao?

From Encyclopedia of Statistical Sciences I understand that given $p$ dichotomous (binary: 1=present; 0=absent) attributes (variables), we can form a contingency table for any two objects i and j of a ...
12k views

Generate random correlated data between a binary and a continuous variable

I want to generate two variables. One is binary outcome variable (say success / failure) and the other is age in years. I want age to be positively correlated with success. For example there should be ...
6k views

Visualizing the calibration of predicted probability of a model

Suppose I have a predictive model that produces, for each instance, a probability for each class. Now I recognize that there are many ways to evaluate such a model if I want to use those ...
4k views

How to do one-class text classification?

I have to deal with a text classification problem. A web crawler crawls webpages of a certain domain and for each webpage I want to find out whether it belongs to only one specific class or not. That ...
10k views

optimizing auc vs logloss in binary classification problems

I am performing a binary classification task where the outcome probability is fair low (aroung 3%). I am trying to decide whether to optimize by AUC or log-loss. As much as I have understood, AUC ...
18k views

What is the effect of dichotomising variables?

When dichotomising variables, what information is lost in the process? How does a dichotomisation help in the analyses?
12k views

Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
2k views

Variance partitioning and longitudinal changes in correlation with binary data

I am analysing data on 300,000 pupils in 175 schools with a logistic linear mixed effects model (random intercepts). Each pupil occurs exactly once and the data spans 6 years. How do I partition ...
424 views

Comparison of statistical tests exploring co-dependence of two binary variables

Suppose we observe data $(X_i,Y_i)_{i=1,...,n}$ on two binary variables: $X\in\{0,1\}$ and $Y\in\{0,1\}$. We would like to test if $X$ and $Y$ are co-dependent (related). Standard suggestions in ...
24k views

Probit two-stage least squares (2SLS)

I was told that it's possible to run a two-stage IV regression where the first stage is a probit and the second stage is an OLS. Is it possible to use 2SLS if the first stage is a probit but the ...
1k views

How would I bias my binary classifier to prefer false positive errors over false negatives?

I've put together a binary classifier using Keras' Sequential model. Of its errors, it predicts with false negatives more frequently than false positives. This tool is for medical application, where I'...
31k views

How to plot binary (presence/absence - 1/0) data against continuous variables [duplicate]

I am trying to plot presence/absence (1/0) of a sample species against various environmental variables. I have put presence/absence on the y-axis and the environmental variable (in this case ...
7k views

Forecasting binary time series

I have a binary time series with 1 when the car is not moving, and 0 when the car is moving. I want to make a forecast for a time horizon up to 36 hours ahead and for each hour. My first approach ...
1k views

Is there an R package for continuous time longitudinal binary responses?

The bild package appears to be an excellent package for serial binary responses. But it is for discrete time. I would like to specify a smooth function of time ...
7k views

How can I test whether my clustering of binary data is significant

I'm doing shopping cart analyses my dataset is set of transaction vectors, with the items the products being bought. When applying k-means on the transactions, I will always get some result. A random ...
15k views

Distance Metrics For Binary Vectors

I have vectors of same length consisting of 1 and 0. I am trying to find out how similar they are. So far I am using hamming distance that I calculate sum of one vector then sum of second vector and ...
13k views

What algorithm should I use to cluster a huge binary dataset into few categories?

I have a large (650K rows * 62 columns) matrix of binary data (0-1 entries only). The matrix is mostly sparse: about 8% is filled. I would like to cluster it into 5 groups - say named from 1 to 5. I ...
2k views

Why we use log function for cross entropy?

I'm learning about a binary classifier. It uses the cross-entropy function as its loss function. $y_i \log p_i + (1-y_i) \log(1-p_i)$ But why does it use the log function? How about just use linear ...
7k views

Logistic Regression and Inflection Point

We have data with a binary outcome and some covariates. I used logistic regression to model the data. Just a simple analysis, nothing extraordinary. The final output is supposed to be a dose-response ...
2k views

How to choose optimal bin width while calibrating probability models?

Background: There are some great questions/answers here on how to calibrate models which predict probabilities of an outcome happening. For example Brier score, and its decomposition into resolution, ...
10k views

How to interpret second-stage coefficient in instrumental variables regression with a binary instrument and a binary endogenous variable?

(fairly long post, sorry. It includes lots of background info, so feel free to skip to the question at the bottom.) Intro: I am working on a project where we are trying to identify the effect of a ...
3k views

How to visualize Bayesian goodness of fit for logistic regression

For a Bayesian logistic regression problem, I have created a posterior predictive distribution. I sample from the predictive distribution and receive thousands of samples of (0,1) for each observation ...
2k views

Modelling auto-correlated binary time series

What are the usual approach to modelling binary time series? Is there a paper or a text book where this is treated? I think of a binary process with strong auto-correlation. Something like the sign of ...
16k views

logit - interpreting coefficients as probabilities

I seem to be missing some vital piece of information. I am aware that the coefficient of logistic regression are in log(odds), called the logit scale. Therefore to interpret them, ...
946 views

Indicator variable for binary data: {-1,1} vs {0,1}

I am interested in treatment-covariate interactions in the context of experiments/randomized controlled trials, with a binary treatment assignment indicator $T$. Depending on the specific method/ ...
8k views

For a classification problem if class variable has unequal distribution which technique we should use?

for eg. if I have a class variable credit scoring with two classes good and bad, where #(good) = 700 and #(bad)= 300. I do not want to shorten my data. which technique I should use? I was using SVM ...
7k views

Recommended procedure for factor analysis on dichotomous data with R

I have to run a factor analysis on a dataset made up of dichotomous variables (0=yes, 1= no) and I don´t know if I'm on the right track. Using tetrachoric() I ...
359 views

Is it ever a good idea to give “partial credit” (continuous outcome) in training a logistic regression?

I am training a logistic regression to predict which runners are most likely to finish a grueling endurance race. Very few runners complete this race, so I have severe class imbalance and a small ...
2k views

What are dangers of calculating Pearson correlations (instead of tetrachoric ones) for binary variables in factor analysis?

I do research on educational games, and some of my current projects involve using data from BoardGameGeek (BGG) and VideoGameGeek (VGG) to examine relationships between design elements of games (i.e., ...
225 views

Correlation between two binary variables within one categorical variable

The Problem: I have measured two binary variables within 1 categorical variable with 5 levels. Initially, I thought I'd be able to use Fisher's Exact test or some $N \times M \times K$ version of it. ...
22k views

Calculating Jaccard or other association coefficient for binary data using matrix multiplication

I want to know if is there any possible way to calculate Jaccard coefficient using matrix multiplication. I used this code ...
103k views

Is nominal, ordinal, & binary for quantitative data, qualitative data, or both?

I'm getting wrapped around data types and I need some help: If you look at the picture above (taken from here), it has the data types like this: Quantitative (Discrete, Continuous) Qualitative (...
428 views

How should I approach this binary prediction problem?

I've got a dataset with the following format. There's a binary outcome cancer/no cancer. Every doctor in the dataset has seen every patient and given an independent judgment on whether the patient ...
7k views

How to measure a classifier's performance when close to 100% of the class labels belong to one class?

In my data, I have a class variable, denoted as $C$. This class variable values are ${0, 1}$ (binary). Almost all observations of $C$ are 0 (close to 100%, more precisely, 97%). I would like a "...
6k views

Binary time series

I have a binary time series: We have 2160 data (0=didn't happen, 1=happened) for one-hour period in 90 days. I want to forecast after these 90 days, where the next 1 will happen, and also Extend ...
761 views

Monotone machine learning

I have a binary classification (supervised learning) problem, where all my features are boolean, with the following twist: I want to learn a classifier $f:\{0,1\}^n \to \{0,1\}$ that is monotone. In ...
3k views

GLM with logit link and Gaussian family to predict a continuous DV between 0 and 1

Can you run a GLM using a logit link with a continuous DV (between 0 and 1)? Generally it's suggested to use a binomial family with a logit link, but I'm guessing that is because the model assumes a ...
I am establishing a model for prediction of a binary variable (Yes/No) depending on three continuous variables ($A$,$B$,$C$). I applied logistic regression analysis for a learning dataset vith the ...