# All Questions

136,269 questions
620 views

### What is the reasoning behind standardization (dividing by standard deviation)?

Why does dividing a dataset by sigma make the sample variance equal to 1? Assuming a zero mean for simplicity. What's the intuition behind this? Dividing by the range (max-min) makes intuitive sense....
182 views

### Creating thinned models in during Dropout process

Applying dropout to a neural network amounts to sampling a “thinned” network from it. The thinned network consists of all the units that survived dropout. A neural net with n units can be seen as a ...
32 views

### Time series with python

I've a dataset in following format day,8am,9am,10am,......,8pm 2012-1-1,0.3,0.2,0.21,.....,0.4 2012-1-2,0.32,0.19,0.22,.....,0.45 ................ I'm not sure ...
9 views

### How IBM SPSS calculates exact p value for Pearson Chi-Square statistic?

Usually, I use Fisher's exact test p value when the sample size is small for Pearson chi-square test. Then I realized that IBM SPSS reports exact p values for all the tests (i.e. Pearson Chi-Square, ...
15 views

### Bootstrapping non-random samples

I perform some analysis (with the goal to select some relevant variables that are related between two experiments) on a group of samples (n = 158) but I would like to know how robust are, before going ...
9 views

### Text generation without being Language dependent

Is there a way in which I can generate the text descriptions for products without bothering about the natural language. For example, if I want to describe a product in German, so can I train my model ...
13 views

### Predicting whether a potential business agreements will be won or lost

I am currently working on a project using a sales system and trying to come up with a way to use the current pipeline of potential sales to predict if the business agreements will win or lost based on ...
22 views

### Model for forecasting daily page views of a web page in R

I have to forecast daily page views of a web portal. We have the daily page views of the data for the last 2 years. We have to forecast for next 90 days. I am using a multi-seasonal (with season ...
5 views

### Assign variable name to desired parts in a function in R [on hold]

I am new to R and here. Sorry If I am complicated. I created a function by trial name. There are some functions from my own function below. ...
11 views

### Fitting probability boundaries of a binary output

I work on a problem for which the output is binary. The class are not linearly separable and quite unbalanced (few ones are embedded in a lot of zeroes). I want to classify the data by levels of ...
12 views

### Formula for deviance residuals for Poisson model with identity link function?

I understand the deviance residuals $r_D$ for a Poisson GLM with log link function are given by $r_D = \mu_{ij} \log(\mu_{ij}/\hat{\mu}_{ij}) + (\hat{\mu}_{ij} - \mu_{ij})$ I was wondering though ...
9 views

### what is alternative/opposite to piecemeal statistical approach?

I have only basic knowledge about bio-statistics. For not normally distributed data, I used Kruskal-Wallis test to investigate the statistical significance between different variables. I performed an ...
9 views

### How to transform sorting data into a dissimilarity matrix (in the presence of missing values)?

Imagine you ask 50 experts to categorize objects (e.g. wine) based on their similarity). To capture the answers of each expert, we compile a similarity matrix. $1$ indicates the objects are in the ...
23 views

### Summation of two Gaussian distributed data with different coefficient of mean and variance

I need some help on Gaussian distribution. i have two dataset, both are identical and independent distributed, but having mean as 2μ_1 and μ_2, same scenario for the variance. How can I add them? ...
8 views

### Liner Regression model bulding - Adjusted R2 [on hold]

While building the Linear regression model step by step, is it mandatory Adjuste-R2 value should keep decreasing from its previous step ? In some scenarios, even if we remove Higher Pr+VIF's variables ...
10 views

### How to determine the longitudinal duration of effect of one time series on another?

I have two time series related to app usage: Notifications- a time series of notifications sent to the user from the app. User Activity- time series of the user interaction on the app. I am trying ...
19 views

### Clustering documents based on pairwise similarity and without access to features

I have a set of documents and distances among them. I want to cluster the documents based on pairwise distances/similarities among them. I have only a single parameter as distance. What are the ...
24 views

### Probability density function units of the log-normal distribution

In a discussion on this forum lognormal distribution, standard-deviation and (physical) units the cumulative distribution function (PDF) of the lognormal distribution was analysed. The conclusion ...
35 views

### Can left-censored data be normal distributed?

Very often I read that the distribution of IQ-test results is normal. However, the IQ-scale is left-censored, i.e. test results cannot be lower than zero. The normal distribution, however, is usually ...
13 views

### how to reduce false positive in logistic regression [on hold]

I am getting all the true positives. However, I am also getting lot of false positives. How can I reduce the number of false positives? I have dataset of 18k records out of which true positive = 10 ...
6 views

### Is there a R package of text mining which does string matching using “CTE+CDAM” http://www.vldb.org/pvldb/vol7/p1167-londhe.pdf? [on hold]

I try to use string matching algorithm to calculate string distance, i.e, how similar two strings are. I am using two R packages: tm and stringdist. But I found there is a paper "matching titles with ...
3 views

### Dynamic harmonic regression: estimated coefficients are zero

I fitted a dynamic harmonic regression model with specified features. However, the summary of the training model indiates one of the features has an estimated coefficient of zero and its corresponding ...
8 views

### Can we predict the monthly sales amount of the coming month without knowing the values of the independent variables of the coming month

I have a data set where the monthly sales of TMT bars and various other explanatory variables are present from April 2014-March 2018. I need to predict the monthly sales of the coming/next month. ...
28 views

### Question Regarding Zero Conditional Mean

Hi I am a beginner to econometrics! I have been dealing with bivariate regression. We use the formula $y = \beta_0 + \beta_1 x$. I am told that if $E(u\mid x) \ne 0$ then the estimate of the slope ...
24 views

### How to choose “family” in Generalized Additive Model (GAM)

When modelling a GAM model using mgcv in R, we need to define the family = . I tried some families (e.g., Gaussian, Gamma), R ...
12 views

### R Replicate observations in Dataframe [on hold]

I have 3 factors, the first one is call cat1, range from 'a' to 'd', and the second factor, cat2, has two levels, 'Y'/'N', and the 3rd, cat3, have 3 levels, 0/1/2, i would like to generate below data ...
34 views

### The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
13 views

### Multi-year and multi-location correlation or regression?

I have fire activity data (i.e. number of fires) and a series of factors (e.g. precipitation, tree cover loss, distance to nearest forest, etc...) that can potentially explain it. I have all this data ...
9 views

### How to test whether a statistically significant coefficient is driven by observations in certain periods in panel data?

I estimated a fixed effects model as follows using a unbalanced panel with 10 years of observations. $y_{it}=\beta x_{it}+\delta_i+\gamma_t+\epsilon_{it}$ The coefficient of interest is $\beta$. I ...
11 views

### Do .sav files expire after the trial version expires? [on hold]

I saved a certain file on an SPSS software that expires after 2 weeks of its installation. I wanted to ask if my file would still exist even after the expiration date.
11 views

### Is attempting to discern gender discrimination in a distribution descriptive or inferential?

Let's say I have a given sample population $P$, that describes the traits of a group of people as a predictor variable, and whether or not they are CEOs as a response variable. I am trying to ...
10 views

### A minimum False Discovery Rate of 0.8 when doing multiple hypothesis testing on Logistic Regression Coefficient p-values?

I currently have 8,000 predictor variables and a binary indicator response variable. I then did a logistic regression on each of the predictor variables, i.e., response regressed on predictor variable ...
15 views

### Power analysis: number of participants or number of data points?

This almost answered my question below, but not quite. I want to calculate the sample size required in order to reach a certain level of a priori statistical power in my experiment. My question is ...
8 views

### Contextual bandit - problem with LinUCB parameter estimation

I am trying to understand and code up the contextual LinUCB algorithm with disjoint models from here: https://arxiv.org/pdf/1003.0146.pdf Using pretty simple simulated data wtih code below: ...
9 views

### GMM moment conditions [on hold]

I want to know why it is useful to add the moment condition about $E(X^2Y)$ for deriving parameters in linear regression.
309 views

### Relation between independence and correlation of uniform random variables

My question is fairly simple: let $X$ and $Y$ be two uncorrelated uniform random variables on $[-1,1]$. Are they independent? I was under the impression that two random, uncorrelated variables are ...
129 views

### Why is Bayesian Statistics becoming a more and more popular research topic? [on hold]

Browsing through the research area of the top 100 US News statistics program, almost all of them are heavy in Bayesian statistics. However, if I go to lower tier school, most of them are still doing ...
13 views

### How can I test for endogeneity after having used Panels Corrected Standard errors (PCSEs)?

How can I test for endogeneity after having used Panels Corrected Standard errors (PCSEs)? By endogeneity I mean the situation in which an explanatory variable is correlated with the error term. In ...
19 views

### Does the total of ordinal scores on a questionnaire become scale in SPSS?

I have a questionnaire whose data I've inputted into SPSS. I wanted to ask that do the total scores of likert items become scale and no more ordinal?
48 views

### What is a 'true' model?

A short question, but I am somehow unable to find any concrete answer. I suppose it means that the model is as good as it can be? Containing all relevant variables and hence not suffering from any ...
11 views

### Question of Degrees of freedom negative in SEM [on hold]

I have a "F_A_T_A_L E_R_R_O_R: Degrees of freedom is negative.", what can be the solution?
33 views

I am confused about how to get the posterior distribution of Bayesian Parameters. I have $t = w_1x + w_0 + \epsilon$ with $\epsilon = N(0, \sigma^2)$ How do I find $p(\textbf{w}|x_1, t_1, ...., x_N, ... 0answers 29 views ### probability involves bivariate gaussian I'm working on a spatial project. I need to calculate the probability of a point being the closest to another. Say I'm given four points$y$,$x_1$,$x_2$and$x_3$in 2D plane, and let$Y'=y+Z$, where ... 1answer 69 views ### Is the Chi-Square Test of Independence the best option for 3x2 contingency table? I have some data which in the terms of a 3x2 contingency table look in the following way. ... 1answer 8 views ### Partitioning Around Medoids: Choosing a cluster number larger than the “optimal” one? I asked a number of 71 'experts' to sort 92 different psychological constructs based on their similarity. Based on their answers, I constructed a dissimilarity matrix. Initially, I wanted to analyse ... 0answers 19 views ### Question about the regularization on Deep learning book (Dr. Goodfellow) [on hold] In Chapter 8 page 228, I don't understand why the component of$w^\starthat is aligned with theI$-th eigenvector of$H$is rescaled by a factor of$\lambda_i/(\lambda_i + \alpha)$. And in the ... 1answer 19 views ### How to determine if two directed probabilistic graphical models are I-equivalent? I'm trying to figure out how to determine if these two models are I-equivalent. Google didn't properly give me a solid answer so far. Any idea on to determine it? Thank you. 0answers 26 views ### UMVUE for$g(p) = \mathbb{E}_p[X^2]\$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...