All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
2 views

Calibration curve of XGBoost for binary classification

I'm working on a binary classification problem, with imbalanced classes (10:1). Since for binary classification, the objective function of XGBoost is ...
0
votes
0answers
5 views

How to make Gradient Boosting Regression *more* sensitive to outliers?

So, I know this is sort of backwards from the normal qustion. Mostly people want to remove outliers in order to improve their models. However, the purpose of my model is to find peaks in a predicted ...
0
votes
1answer
15 views

Why is it bad to dichotomize categorical variables in regression?

Say we have a research question of: "Does a specific cancer type increase the risk of seizure?" We then have 10 cancer types and a seizure 0/1 (no/yes) dependent variable. One way we could answer ...
0
votes
0answers
7 views

Confusion about PACF in time series analysis

My confusion is how can we subtract the future value x(t-1) from the past value x(t-2), I think this implicitly means that the future value will affect the past value which I think is impossible? ...
0
votes
0answers
8 views

Modelling the heat exchange between a steel cylinder and the surrounding medium

Massively refactoring my question after @user2974951 feedback below. I have a set of measurements (time, temperature) that correspond to a certain physical process. Both time and temperature ...
0
votes
0answers
19 views

How to interpret the following equations

I am following the below-mentioned blog post from instagram: https://instagram-engineering.com/trending-on-instagram-b749450e6d93 In the link they discuss the following equation. Equation 1 (which ...
0
votes
0answers
10 views

Create a new column with conditions

I have a data frame with multiple rows and columns. I want to create a new column based on two existing columns. The new data I want is like the following: ...
0
votes
0answers
7 views

How should I incorporate random effect of site into my GLMM?

I have a dataset that consists of 30 sites in a city, covering a gradient of urbanisation. Each contains 3 trees, from which I sampled arthropods and measured 5 habitat variables. After exploring my ...
0
votes
1answer
10 views

How to select the best dataset after multiple imputation in MICE to build other models

I carried out multiple imputation using MICE with m=10. The R code is shown below: RainfallData <- mice(rainfall,m=10,maxit=10,meth='pmm') modelFit1 <- with(RainfallData,lm(Total.Rainfall~Wind....
0
votes
1answer
15 views

Building A Model: Autoregressive?

I've recently read a code about a fisheries productivity model where someone tries to predict the value at time t+1 from its value at time t. There're 23 recorded productivity of tuna from 1967 to ...
0
votes
0answers
5 views

How do you “correct” for another predictor when investigating correlations?

I have a large data set of survey responses in which test takers, after taking a test, reported on resources they used, hours spent studying each month, etc. I also have access to the test taker's ...
0
votes
0answers
5 views

Two-step residualization in fuzzy RD

I'm adopting a two-step residualization approach in fuzzy RD, should my first stage also include a polynomial of running variable (age)?
0
votes
1answer
13 views

Do randomization tests for difference in means assume independent groups?

I performed a randomization test and found that the means of groups A and B are significantly different. I am trying to prove that group B is "special", that there is something interesting going on ...
0
votes
0answers
14 views

How to find the optimal classifier for a given loss function?

For a binary classification problem (labels being 0 and 1) and a classifier $g$ we consider the loss function $L(g)=P[Y\neq g(X)]$. It is known that the optimal classifier $g^*$ that minimizes $L$ is $...
1
vote
2answers
33 views

Why is it okay to take the log (or any other transformation) of the dependent variable?

Why is it common practice to take the log of the dependent variable Y? To be clear, I understand that under appropriate circumstances that taking the log can help normalize the distribution/linearize ...
0
votes
0answers
5 views

How to obtain Covariance-variance matrix for VECM

I am trying to conduct a causality analysis with a VECM, and I am looking for ways to extract the Covariance-Variance matrix and the correlation matrix from the fitted VECM using the already existing ...
0
votes
0answers
9 views

Understanding the stochastic part of rank-based inverse normal transformation

I'm checking out some methods for rank-based inverse normal transformation in Python and found this: https://github.com/edm1/rank-based-INT/blob/master/rank_based_inverse_normal_transformation.py ...
0
votes
0answers
16 views

Creating confidence intervals for rates per 100,000 with very small numerators?

I am conducting a study comparing rates per 100,000 of new HIV diagnoses over time in certain regions. I intend to calculate 95% confidence intervals for each rate. However, some of my numerators (HIV ...
0
votes
0answers
7 views

Bayesian random slope model with divergent transitions. Help in setting stronger priors

I have a model with which I have convergence problems. This are the model specifics (brms package): ...
1
vote
1answer
16 views

Geometric distribution with multiple trials

Not sure how to word this question, sorry if it's dumb. But I was looking into geometric distributions to find the probability of the first success of some random variable X. So if p = 0.04, the ...
0
votes
0answers
9 views

Bandit Problem where only the sum of the rewards is known

Say you have a bandit problem where the only feedback is a) the number of times you pulled arm A b) the number of times you pulled arm B c) the sum total reward associated with A and B on a daily ...
0
votes
0answers
7 views

Error during model comparison :arguments imply differing number of rows: 3, 2 [on hold]

I'm doing a model comparison or ordinal mixed models from clmm() using anova() and I keep getting the error Error in data....
0
votes
0answers
7 views

VAR obtained from vec2var() and and regular VAR giving different IRF and OIRF

I am currently trying to generate the orthogonal impulse response functions (OIRF) of a VECM with two variables. Both variables are I(1) and there is definitely a cointegration at all levels as tested ...
0
votes
0answers
17 views

Multidimensional, Multivariate Delta method

Suppose I have an estimator $B\in\mathbb{R}^m$ converging to $\beta$, such that $$ \sqrt{n}(B-\beta)\rightarrow\mathcal{N}(0,\Sigma). $$ I am interested in a quantity $\mathbf{h}(B):\ \mathbb{R}^m\...
0
votes
0answers
15 views

ARMA canonical form

I am a little confused about ARMA equations. Suppose that I have the following AR(1) process: $$y_t = \phi y_{t-1} + \epsilon_t$$ From the same equation I can deduce: $$y_{t-1} = \phi y_{t-2} + \...
1
vote
1answer
15 views

Are there any problems with using an average of percentages for a 2 proportion z-test?

Let's say I have a 2 groups of test scores (ranging from 0% to 100%), and I wanted to compare their means to see if there was anything significant. I know I can average the test scores to obtain a ...
0
votes
0answers
27 views

Can data ever be too high dimensional for the Lasso?

I'm trying to implement Lasso on high dimensional textual data. Format of Data: p ~= 45,000, n~=4,000 When running the Lasso, I get a training score of 0 and the number of features selected as 0. ...
0
votes
0answers
8 views

Why is this extremely high class_weight not affecting number of false negatives?

I'm using an extremely high class_weight for positive class in a standard Sklearn Random Forest Classifier (Params have been included below the Model Report). The reason I'm up-weighting the positive ...
0
votes
1answer
17 views

Regression multi-level meta-analysis with binomial data in R: no existing package?

Almost all is in the title. I'm trying to do a regression multilevel meta-analysis of data that are binomally distributed in R. I thought of doing it with the ...
0
votes
0answers
8 views

Modeling approach to showing a parabolic effect is greater than that expected by scale boundaries

I have conducted an experiment where raters rate different nations on a DV. Each observation is a different nation. I calculated a mean and SD of the DV for each nation. The data we are working with ...
0
votes
0answers
13 views

Classifying data, and then performing linear regression on the classes

Is it valid to perform a classification on a data set, separate the data by class, and then perform a regression on each of the groups? The reason why I ask is that the histogram for my data looks ...
2
votes
1answer
28 views

Regression in linear equation vs. distribution form

In most introductory textbooks (less 'mathematical') simple linear regression model is formulated as equation. i.e. $$Y_i = \beta_0 + \beta_1 X_{1 i} + \epsilon_i, \quad i =1,n $$ $$ \epsilon_i \sim \...
0
votes
0answers
9 views

How to predict values in one variables using previous observations of another variable, when observation times are different across participants?

We have observations of individuals at various points in their life. We are curious if Variable A predicts later values of Variable B (i.e., if being high on A early on means you will be high on B ...
0
votes
1answer
21 views

how to cut/group variables in Cox model

Considering that for a logistic regression one approach is to cut the numeric variables (or group the categorical ones) with some algorithm before running the logistic model (to allow the algorithm to ...
0
votes
0answers
11 views

Using k-means clustering to train radial basis neural network for highly imbalanced dataset

I am trying to find prototype neurons for my radial basis neural network. My dataset has 30 attributes (of which 28 of them are the result of a single PCA) and 300.000 observations. It is a binary ...
0
votes
0answers
9 views

Differences between glmmadaptive Vs lme4 and glmmTMB in ICC measurement

This is my first question, so please be kind... I am currently modelling a GLMM with a binary outcome with many (500+) clusters but cluster size of 2 (by design - there can be no more than 2 per ...
0
votes
0answers
7 views

Anomaly detection through distribution-based hypothesis testing

I have a table of event logs that contains several categorical variables (gender, age bucket, city of residence, and education level), and I'd like to retroactively identify if a given hour of logs ...
0
votes
0answers
22 views

What are some still maintained packages for doing GEE in R? [on hold]

The geepack, geeglm, gee packages are not really updated regularly by some author anymore or maintained. Are there any other packages that can replace geepack/geeglm,gee?
0
votes
0answers
8 views

Right truncation and right censoring

Is is possible for a survival data to be **right truncated and right censored **. If so, please leave an example for better understanding. Thanks in advance!
0
votes
1answer
17 views

Hoes does laplace smoothing in Naive Bayes control high bias and high variance?

I'm trying to understand how laplace smoothing exactly helps to balance between overfitting and underfitting. I know that Laplace smoothing is used as a fail safe probability if there's a any ...
0
votes
1answer
22 views

How are Wald type tests better asymptotically? [on hold]

I heard that Wald tests are better asymptotically than other tests. But what does this mean? Does it mean that it's better at testing for population level differences? Is it better for "big data" and ...
1
vote
1answer
37 views

How to normalize if variance is zero?

How do I normalize a dataset to z-scores is some features have variance zero? Is throwing them away beforehand the only solution?
-1
votes
0answers
7 views

Finding the weights of individual children items given the actual parent assembly/unit weights

I'm jumping head first into learning machine learning by trying to solve an actual problem at work. I'm trying to find individual weights of items/options that make up the total weights of assemblies/...
0
votes
0answers
15 views

Choosing which chi-square test to use when working with count data

I am analyzing data that compares responses to various questions across a variety of demographics. I want to see how the count of each response varies by each demographic group. The most basic test to ...
0
votes
2answers
41 views

Why estimate using GEEs inspite of their disadvantages over using ML?

I know sometimes I want to know the population-level estimates, but the problem with GEEs is I can't calculate the likelihood, and therefore all models I make with it aren't comparable, and I don't ...
0
votes
0answers
17 views

Non-linear Least squares fitting in R

I want to fit a curve to a set of 3 data points (1,65), (200,70), (800,75) in the form $y = \frac{ax}{b+x}$ Using the set of points I have this set of equations $a-65b = 65$ $200a-70b = 14000$ ...
0
votes
0answers
4 views

Estimating false accept rates from imposter scores below a threshold

I have a system that compares two items and produces a match score. Scores below a threshold are manually inspected to determine if they match or don't(imposter). Scores above the threshold are ...
0
votes
0answers
6 views

How to show or prove a dataset is not linearly separable

I am looking to be pointed in the right direction. I am learning about kernels and I have a homework assignment to use the dual perceptron algorithm to classify datapoints from a spiral dataset, with ...
0
votes
0answers
9 views

Cox proportional hazards model understanding [on hold]

Assuming that we perform a Cox regression as presented in the example here. I am wondering what we should expect in the output of the Cox regression if, for example, we perform the fit again but this ...
2
votes
1answer
28 views

Expanding initial sample when the result isn't significant

This inspiring answer describes a variant of hypothesis, and I want to analysis its property further. Basically, it considers a two-sided test and interprets the $p$-value as a measure of how strong ...

15 30 50 per page