All Questions

Filter by
Sorted by
Tagged with
0
votes
0answers
4 views

Using S statistic in place of Cohen's Kappa: R code provided

This article proposes a new statistic called $S$ statistic to replace Cohen's Kappa. I know I can compute KAPPA for each category using the R function below. But ...
0
votes
0answers
2 views

Correct Estimator for Path Analysis/SEM Using Categorical Covariate

I'm currently building a path analysis model with model-based corrections for measurement error, using a latent variable structural equation model (i.e., each observed variable serves as a single ...
0
votes
1answer
9 views

How to fit a Linear Regression model with linearly related predictor variables

I have three predictor variables x,y and z and a outcome variable "grade". The three predictor variable are linearly related (x+y+z=1). When I fit a linear regression model using these variables, I ...
0
votes
0answers
4 views

Overlapping training data windows for classification

I have a data set that represents snapshots of user behavior over a fixed window length $l$ and attempts to classify the user into one of $k$ groups. With language models it seems like it's fair ...
0
votes
0answers
5 views

How to solve for the minimum KL Divergence when the distribution is discrete?

Say we have a simple case of $p(x,y)$ is a 3x3 matrix: $$\begin{bmatrix} 1/6 & 0 & 0 \\ 1/6 & 3/6 & 0 \\ 0 & 0 & 1/6 \end{bmatrix}$$ And $q(x,y)=...
0
votes
0answers
9 views

quickly finding the moments of a numerically-defined PDF

I have a two-dimensional continuous PDF which is numerically defined. From this PDF I would like to extract the second central moment (variance) of a "slice" of this distribution. The "slice" is to ...
1
vote
0answers
11 views

Does the function $H(a) \equiv \mathbb{P}(f_X(X) \geqslant a)$ have a name? (If it doesn't, what is a *good* name for it?)

Given a random variable $X$ with density function $f_X$, we can define the function $H$ as: $$H(a) \equiv \mathbb{P}(f_X(X) \geqslant a) \quad \quad \text{for all } a \geqslant 0.$$ This function ...
0
votes
0answers
3 views

Save scikit agglomerative clustering iterations

I'm using the scikit-learn module of agglomerative hierarchical clustering to obtain clusters of a three million geographical hexagrid using contiguity constraints and ward affinity. My question is ...
0
votes
0answers
6 views

Combining features linearly before non-linear modelling

I want to construct a model like this: $$y = g(X, Z)$$ $$X = \alpha_1 x_1 + \alpha_2 x_2 + \dots + \alpha_n x_n$$ $$Z = \beta_1 z_1 + \beta_2 z_2 + \dots + \beta_n z_n$$ Where $x_i \text{ and } z_i$ ...
0
votes
0answers
6 views

Hard classification problem - test set accuracy seems to cap out at a certain value regardless of architecture

I've been working on a hard binary classification problem (50-50 split between classes). I've tried a variety of different network architectures and training schemes - dropout, no dropout, batch ...
1
vote
0answers
7 views

If linear combination of two time series processes is non-stationary does it mean one of the two series is non-stationary

Suppose I have 2 time-series processes. If they are jointly weakly stationary then the linear combination is weakly stationary. If the linear combination is non-stationary does it mean at least one ...
1
vote
0answers
27 views

Sampling from/near boundary of a region in R^n

Suppose $\Omega$ is a region in $\mathbb R^n$, and suppose we are given a function $\chi(x)$ with $\chi(x)=1$ if $x\in \Omega$ and $\chi(x)=0$ otherwise. If it helps we can assume $\Omega\subseteq B$ ...
0
votes
0answers
8 views

How can I capture household characteristics?

I have a panel dataset following individuals and households over time. Now, testing for observable changes in the population of individuals is easy. It is less easy for me to test for changes in the ...
0
votes
0answers
23 views

For lognormal distribution which one is preferred? Log 10 or Ln or Log 2?

I want to perform a linear regression analysis. The distributions of data for all continuous variables are not normal. The tail of graph is to the right and thre highest point of graph is due to the ...
0
votes
0answers
5 views

How to reshape input data in RNN model for prediction

While making prediction with a built model using new data, an error returned because the shape is different: Here's the code: ...
1
vote
0answers
6 views

Help with model selection

In my experimental setup, I have a square symmetric matrix of variables (65) which I'll call A. Aij represents some relationship between the variables. I then performed an experiment in which I have ...
0
votes
0answers
4 views

What's the advantage of Toda-Yamamoto's Granger Causality procedure?

There are already quite a few questions here on the Toda-Yamamoto (TY) approach to Granger Causality, i.e., the blog posts by Dave Giles (2011 and 2014). What I would like to clarify is just whether I ...
0
votes
0answers
8 views

Which statistical studies should be carried out for expression analysis of genes when I have fold change data of genes

I am new to R and data analysis. I am working on gene expression. My data frame contains Gene_ID and quantification_value which is the fold change of the gene. ( the value represents the amount by ...
1
vote
0answers
17 views

Regression analysis with multiple categories

I have this data below which I am analyzing using R. First, I am trying to find which predictors (chem1, chem2 and chem3) have effects on yield for this data. I did this model test below and found ...
0
votes
1answer
34 views

How to compute $E[ (|X|) X]$ when $X \sim N(0,1)$?

Any help would be appreciated.
0
votes
0answers
8 views

Including ratios as features in machine learning algorithms

Assume that body mass index (BMI) is a good predictor for early death which we are trying to model using a host of different algorithms. A colleague posed the following to me: would it be better to ...
1
vote
1answer
26 views

Poisson distribution with large mean

The hits in a website can be modeled as a Poisson process with a rate of 5000 hits per day. Let X be the number of hits in 6 days. Give the distribution of X. So, is the percentile x = 6 and the ...
1
vote
0answers
7 views

Land Method for Lognormal Distribution

The Land Method is explain here. Anyone knows how I can look up the H*1-α* It says "Tables of these values are given by Gilbert (1987, Tables A-10 and A-12) and Land (1975). " Gilbert (1987) is not ...
0
votes
0answers
9 views

Multinomial Distribution

I am trying to quantify a belief that the proportions of testing conditions I have subjected my test units to is representative of my population proportions(known). My data is a number of hours spent ...
0
votes
0answers
6 views

Deriving full conditional of ordered probit model (Bayes)

I have a question regarding the following exercise: I am able to compute the complete (full) data likelihood function, the full conditionals of $y^{*}_i$ and $\beta$. However, I do not know how to ...
0
votes
0answers
9 views

Validate raking (post-stratification) weights within sample

Are there standard ways for verifying raking / post-stratification weights? For example, I want to adjust for age, gender and income across some sample of a larger population. Can I then use the ...
0
votes
1answer
21 views

Matching vs simple regression for causal inference?

This is a really simple, newbie question. I am really confused about the notion of matching and when it can be used instead of a multiple regression? Assume I have listed all the confounding ...
1
vote
0answers
20 views

Sum of Two Bivariate Normal Distributions

I'm just confused on how to set up and start this problem. I'm confident that once I start down the right path, I'll have little issue. Let $p_1$ denote a bivariate normal distribution $N(0, 0, 1, 1, ...
0
votes
0answers
6 views

Multi feature matching

this might be a really intro question, But if I have two dataframes like: ...
0
votes
0answers
11 views

Testing significance: single population, two-sample unpaired data

I am looking to test the significance of a training/ education program of home building construction. I am working with unpaired data of a dependent sample and am unsure how to proceed. Since the ...
0
votes
0answers
16 views

Is term “metric” for evaluating machine learning model misnomer?

Term "metric" is used in many popular machine learning articles [1, 2] for describing an evaluation criterium of the performance of a model. Although, mathematically is the term defined as: In ...
2
votes
1answer
25 views

Legitimacy of Regressing Actual Values on Predicted Values for Better Residual Sum of Squares?

A coworker of mine recently performed an analysis where after training a simple linear model, he regressed the actual Y values against his model's predicted Y values, and applied this regression's ...
2
votes
0answers
16 views

How do two of my Wilcoxon signed rank tests have the same p-values?

I'm running a statistical analysis where I'm comparing land surface temperature in matched equal sized areas (project areas and control areas) of different cities. In each city, I do it for two ...
0
votes
0answers
12 views

When assessing normality from a QQ-plot, can we use r (lin. cor. coeff.) to do so?

When assessing normality from a QQ-plot, I see online that many sources compare the linearity by plotting a straight line through the two points of intersection of quantiles 1 and 3. But does it also ...
0
votes
0answers
6 views

Feature Extraction, Moving window approach [on hold]

I am new to the ML and I only have the strongest will power to learn it. Question: I have the time series data and I want to approach the ML algorithm by extracting features, I didn't choose the ...
0
votes
0answers
6 views

Evaluation of predictions: How to combine significances of different distributions?

The question: I have 100 different distributions of 24 values each. Now out of those 100 distributions my predictor selects 1 of those 24 values each. Now I want to test if those 100 selected values ...
-1
votes
0answers
8 views

Product/SKU level time series forecasting [on hold]

Predict demand of product at each outlet for next 6 months 5 – 7 years of sales data at outlet level for each and every brand is available As it is a time series problem we need to design per outlet ...
0
votes
1answer
18 views

Point estimate options with highly skewed data

(Added regression tag since I think that my question overlaps with that area but not sure. Added transformation tag since I discuss log transformations. Tag recommendations welcome) I would like to ...
0
votes
0answers
8 views

GBM Models with Correlated Predictors

I am building GBM models in which Predictor A is more highly correlated with my response variable for certain values of A while Predictor B is more highly correlated with my response variable for ...
0
votes
0answers
29 views

What does this proof mean?

I am self-studying Shai-Shawrts and Shai Ben-David's Understanding Machine Learning book. In chapter 4(Learning via uniform convergence) I have encountered this proof. Can someone explain me the ...
0
votes
0answers
18 views

Scikit-learn hierarchical clustering threshold

I'm using the scikit-learn module of agglomerative hierarchical clustering to obtain clusters of a three million geographical hexagrid using contiguity constraints and ward affinity. My question is ...
0
votes
1answer
11 views

Deviation Coding meaning of test statistic on grand mean (intercept)

I have a dataset with 7 factor levels none of which are a reasonable reference group, so I've chosen to set up my GLM (negative binomial) analysis using deviation coding. I've studied this question ...
0
votes
0answers
5 views

how to estimate hyper-parameters when cross-validating time series forecasting?

I want to evaluate several forecasting methods on the taylor time series using cross-validation. How do I go about selecting the hyper-parameters for the methods? ...
0
votes
1answer
17 views

Add Absolute Value as a Feature

In a machine learning context, does it make sense to have both a measurement and its absolute value transformation as features? There are already ~120 features in this predictive model (an elastic ...
0
votes
0answers
5 views

discriminator function in dual Generative adversarial network

I am reading the following article: https://arxiv.org/pdf/1803.00385.pdf On page 3 they explain how the losses for two discriminators (D1 and D2) is determined. D1 separates true upright image from ...
0
votes
0answers
15 views

Data set with known camera parameters [on hold]

I am looking for a data set containing images of pedestrians/cyclists with static background and given camera calibration information (intrinsic and extrinsic) for a scene. I have already found EPFL ...
0
votes
0answers
7 views

ANOVA/Chi-square analog to GEE

If repeated measures anova is analogous to random-intercepts regression and regular anova is analogous to multiple linear regression because of the F-statistic. Is GEE logit or GEE poisson or ...
0
votes
0answers
3 views

Brown-Forsythe Test on Unbalanced Two-Way ANOVA

I am trying to create code in Matlab that runs a Brown-Forsythe test on an unbalanced two-way ANOVA. I am having difficulty figuring out the actual math behind this test and how to do it. I have read ...
1
vote
0answers
9 views

Translating the odds ratio

I have a question regarding a translation in odds ratio curve OR(x). More specifically, let's say we have an OR(x) which we get using logistic regression. We look at the interval bounded with 25% and ...
-1
votes
0answers
8 views

Problem with non standard question about latent variable applied non-linear discrete choice models

Problem with non standard question about latent variable applied non-linear discrete choice models. I'm having issues because I couldn't find in the material I have in my possession any example ...

15 30 50 per page