# All Questions

141,712 questions
Filter by
Sorted by
Tagged with
2 views

### Calibration curve of XGBoost for binary classification

I'm working on a binary classification problem, with imbalanced classes (10:1). Since for binary classification, the objective function of XGBoost is ...
5 views

### How to make Gradient Boosting Regression *more* sensitive to outliers?

So, I know this is sort of backwards from the normal qustion. Mostly people want to remove outliers in order to improve their models. However, the purpose of my model is to find peaks in a predicted ...
15 views

### Why is it bad to dichotomize categorical variables in regression?

Say we have a research question of: "Does a specific cancer type increase the risk of seizure?" We then have 10 cancer types and a seizure 0/1 (no/yes) dependent variable. One way we could answer ...
7 views

### Confusion about PACF in time series analysis

My confusion is how can we subtract the future value x(t-1) from the past value x(t-2), I think this implicitly means that the future value will affect the past value which I think is impossible? ...
8 views

### Modelling the heat exchange between a steel cylinder and the surrounding medium

Massively refactoring my question after @user2974951 feedback below. I have a set of measurements (time, temperature) that correspond to a certain physical process. Both time and temperature ...
19 views

### How to interpret the following equations

I am following the below-mentioned blog post from instagram: https://instagram-engineering.com/trending-on-instagram-b749450e6d93 In the link they discuss the following equation. Equation 1 (which ...
10 views

### Create a new column with conditions

I have a data frame with multiple rows and columns. I want to create a new column based on two existing columns. The new data I want is like the following: ...
7 views

### How should I incorporate random effect of site into my GLMM?

I have a dataset that consists of 30 sites in a city, covering a gradient of urbanisation. Each contains 3 trees, from which I sampled arthropods and measured 5 habitat variables. After exploring my ...
10 views

### How to select the best dataset after multiple imputation in MICE to build other models

I carried out multiple imputation using MICE with m=10. The R code is shown below: RainfallData <- mice(rainfall,m=10,maxit=10,meth='pmm') modelFit1 <- with(RainfallData,lm(Total.Rainfall~Wind....
15 views

### Building A Model: Autoregressive?

I've recently read a code about a fisheries productivity model where someone tries to predict the value at time t+1 from its value at time t. There're 23 recorded productivity of tuna from 1967 to ...
5 views

### How do you “correct” for another predictor when investigating correlations?

I have a large data set of survey responses in which test takers, after taking a test, reported on resources they used, hours spent studying each month, etc. I also have access to the test taker's ...
5 views

### Two-step residualization in fuzzy RD

I'm adopting a two-step residualization approach in fuzzy RD, should my first stage also include a polynomial of running variable (age)?
13 views

### Do randomization tests for difference in means assume independent groups?

I performed a randomization test and found that the means of groups A and B are significantly different. I am trying to prove that group B is "special", that there is something interesting going on ...
14 views

15 views

9 views

### How to predict values in one variables using previous observations of another variable, when observation times are different across participants?

We have observations of individuals at various points in their life. We are curious if Variable A predicts later values of Variable B (i.e., if being high on A early on means you will be high on B ...
21 views

### how to cut/group variables in Cox model

Considering that for a logistic regression one approach is to cut the numeric variables (or group the categorical ones) with some algorithm before running the logistic model (to allow the algorithm to ...
11 views

### Using k-means clustering to train radial basis neural network for highly imbalanced dataset

I am trying to find prototype neurons for my radial basis neural network. My dataset has 30 attributes (of which 28 of them are the result of a single PCA) and 300.000 observations. It is a binary ...
9 views

### Differences between glmmadaptive Vs lme4 and glmmTMB in ICC measurement

This is my first question, so please be kind... I am currently modelling a GLMM with a binary outcome with many (500+) clusters but cluster size of 2 (by design - there can be no more than 2 per ...
7 views

### Anomaly detection through distribution-based hypothesis testing

I have a table of event logs that contains several categorical variables (gender, age bucket, city of residence, and education level), and I'd like to retroactively identify if a given hour of logs ...
22 views

### What are some still maintained packages for doing GEE in R? [on hold]

The geepack, geeglm, gee packages are not really updated regularly by some author anymore or maintained. Are there any other packages that can replace geepack/geeglm,gee?
8 views

### Right truncation and right censoring

Is is possible for a survival data to be **right truncated and right censored **. If so, please leave an example for better understanding. Thanks in advance!
17 views

### Hoes does laplace smoothing in Naive Bayes control high bias and high variance?

I'm trying to understand how laplace smoothing exactly helps to balance between overfitting and underfitting. I know that Laplace smoothing is used as a fail safe probability if there's a any ...
22 views

### How are Wald type tests better asymptotically? [on hold]

I heard that Wald tests are better asymptotically than other tests. But what does this mean? Does it mean that it's better at testing for population level differences? Is it better for "big data" and ...
37 views

### How to normalize if variance is zero?

How do I normalize a dataset to z-scores is some features have variance zero? Is throwing them away beforehand the only solution?
7 views

### Finding the weights of individual children items given the actual parent assembly/unit weights

I'm jumping head first into learning machine learning by trying to solve an actual problem at work. I'm trying to find individual weights of items/options that make up the total weights of assemblies/...
15 views

### Choosing which chi-square test to use when working with count data

I am analyzing data that compares responses to various questions across a variety of demographics. I want to see how the count of each response varies by each demographic group. The most basic test to ...
41 views

### Why estimate using GEEs inspite of their disadvantages over using ML?

I know sometimes I want to know the population-level estimates, but the problem with GEEs is I can't calculate the likelihood, and therefore all models I make with it aren't comparable, and I don't ...
17 views

### Non-linear Least squares fitting in R

I want to fit a curve to a set of 3 data points (1,65), (200,70), (800,75) in the form $y = \frac{ax}{b+x}$ Using the set of points I have this set of equations $a-65b = 65$ $200a-70b = 14000$ ...
4 views

### Estimating false accept rates from imposter scores below a threshold

I have a system that compares two items and produces a match score. Scores below a threshold are manually inspected to determine if they match or don't(imposter). Scores above the threshold are ...
6 views

### How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
This inspiring answer describes a variant of hypothesis, and I want to analysis its property further. Basically, it considers a two-sided test and interprets the $p$-value as a measure of how strong ...