All Questions

5
votes
2answers
620 views

What is the reasoning behind standardization (dividing by standard deviation)?

Why does dividing a dataset by sigma make the sample variance equal to 1? Assuming a zero mean for simplicity. What's the intuition behind this? Dividing by the range (max-min) makes intuitive sense....
4
votes
2answers
182 views

Creating thinned models in during Dropout process

Applying dropout to a neural network amounts to sampling a “thinned” network from it. The thinned network consists of all the units that survived dropout. A neural net with n units can be seen as a ...
0
votes
0answers
32 views

Time series with python

I've a dataset in following format day,8am,9am,10am,......,8pm 2012-1-1,0.3,0.2,0.21,.....,0.4 2012-1-2,0.32,0.19,0.22,.....,0.45 ................ I'm not sure ...
0
votes
0answers
9 views

How IBM SPSS calculates exact p value for Pearson Chi-Square statistic?

Usually, I use Fisher's exact test p value when the sample size is small for Pearson chi-square test. Then I realized that IBM SPSS reports exact p values for all the tests (i.e. Pearson Chi-Square, ...
0
votes
0answers
15 views

Bootstrapping non-random samples

I perform some analysis (with the goal to select some relevant variables that are related between two experiments) on a group of samples (n = 158) but I would like to know how robust are, before going ...
0
votes
0answers
9 views

Text generation without being Language dependent

Is there a way in which I can generate the text descriptions for products without bothering about the natural language. For example, if I want to describe a product in German, so can I train my model ...
0
votes
0answers
13 views

Predicting whether a potential business agreements will be won or lost

I am currently working on a project using a sales system and trying to come up with a way to use the current pipeline of potential sales to predict if the business agreements will win or lost based on ...
1
vote
1answer
22 views

Model for forecasting daily page views of a web page in R

I have to forecast daily page views of a web portal. We have the daily page views of the data for the last 2 years. We have to forecast for next 90 days. I am using a multi-seasonal (with season ...
0
votes
0answers
5 views

Assign variable name to desired parts in a function in R [on hold]

I am new to R and here. Sorry If I am complicated. I created a function by trial name. There are some functions from my own function below. ...
0
votes
0answers
11 views

Fitting probability boundaries of a binary output

I work on a problem for which the output is binary. The class are not linearly separable and quite unbalanced (few ones are embedded in a lot of zeroes). I want to classify the data by levels of ...
0
votes
1answer
12 views

Formula for deviance residuals for Poisson model with identity link function?

I understand the deviance residuals $r_D$ for a Poisson GLM with log link function are given by $r_D = \mu_{ij} \log(\mu_{ij}/\hat{\mu}_{ij}) + (\hat{\mu}_{ij} - \mu_{ij})$ I was wondering though ...
0
votes
0answers
9 views

what is alternative/opposite to piecemeal statistical approach?

I have only basic knowledge about bio-statistics. For not normally distributed data, I used Kruskal-Wallis test to investigate the statistical significance between different variables. I performed an ...
0
votes
0answers
9 views

How to transform sorting data into a dissimilarity matrix (in the presence of missing values)?

Imagine you ask 50 experts to categorize objects (e.g. wine) based on their similarity). To capture the answers of each expert, we compile a similarity matrix. $1$ indicates the objects are in the ...
0
votes
1answer
23 views

Summation of two Gaussian distributed data with different coefficient of mean and variance

I need some help on Gaussian distribution. i have two dataset, both are identical and independent distributed, but having mean as 2μ_1 and μ_2, same scenario for the variance. How can I add them? ...
0
votes
0answers
8 views

Liner Regression model bulding - Adjusted R2 [on hold]

While building the Linear regression model step by step, is it mandatory Adjuste-R2 value should keep decreasing from its previous step ? In some scenarios, even if we remove Higher Pr+VIF's variables ...
0
votes
0answers
10 views

How to determine the longitudinal duration of effect of one time series on another?

I have two time series related to app usage: Notifications- a time series of notifications sent to the user from the app. User Activity- time series of the user interaction on the app. I am trying ...
1
vote
1answer
19 views

Clustering documents based on pairwise similarity and without access to features

I have a set of documents and distances among them. I want to cluster the documents based on pairwise distances/similarities among them. I have only a single parameter as distance. What are the ...
2
votes
0answers
24 views

Probability density function units of the log-normal distribution

In a discussion on this forum lognormal distribution, standard-deviation and (physical) units the cumulative distribution function (PDF) of the lognormal distribution was analysed. The conclusion ...
0
votes
3answers
35 views

Can left-censored data be normal distributed?

Very often I read that the distribution of IQ-test results is normal. However, the IQ-scale is left-censored, i.e. test results cannot be lower than zero. The normal distribution, however, is usually ...
0
votes
0answers
13 views

how to reduce false positive in logistic regression [on hold]

I am getting all the true positives. However, I am also getting lot of false positives. How can I reduce the number of false positives? I have dataset of 18k records out of which true positive = 10 ...
0
votes
0answers
6 views

Is there a R package of text mining which does string matching using “CTE+CDAM” http://www.vldb.org/pvldb/vol7/p1167-londhe.pdf? [on hold]

I try to use string matching algorithm to calculate string distance, i.e, how similar two strings are. I am using two R packages: tm and stringdist. But I found there is a paper "matching titles with ...
1
vote
0answers
3 views

Dynamic harmonic regression: estimated coefficients are zero

I fitted a dynamic harmonic regression model with specified features. However, the summary of the training model indiates one of the features has an estimated coefficient of zero and its corresponding ...
0
votes
0answers
8 views

Can we predict the monthly sales amount of the coming month without knowing the values of the independent variables of the coming month

I have a data set where the monthly sales of TMT bars and various other explanatory variables are present from April 2014-March 2018. I need to predict the monthly sales of the coming/next month. ...
1
vote
3answers
28 views

Question Regarding Zero Conditional Mean

Hi I am a beginner to econometrics! I have been dealing with bivariate regression. We use the formula $y = \beta_0 + \beta_1 x$. I am told that if $E(u\mid x) \ne 0$ then the estimate of the slope ...
2
votes
0answers
24 views

How to choose “family” in Generalized Additive Model (GAM)

When modelling a GAM model using mgcv in R, we need to define the family = . I tried some families (e.g., Gaussian, Gamma), R ...
0
votes
0answers
12 views

R Replicate observations in Dataframe [on hold]

I have 3 factors, the first one is call cat1, range from 'a' to 'd', and the second factor, cat2, has two levels, 'Y'/'N', and the 3rd, cat3, have 3 levels, 0/1/2, i would like to generate below data ...
1
vote
0answers
34 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
0
votes
0answers
13 views

Multi-year and multi-location correlation or regression?

I have fire activity data (i.e. number of fires) and a series of factors (e.g. precipitation, tree cover loss, distance to nearest forest, etc...) that can potentially explain it. I have all this data ...
0
votes
1answer
9 views

How to test whether a statistically significant coefficient is driven by observations in certain periods in panel data?

I estimated a fixed effects model as follows using a unbalanced panel with 10 years of observations. $y_{it}=\beta x_{it}+\delta_i+\gamma_t+\epsilon_{it}$ The coefficient of interest is $\beta$. I ...
0
votes
0answers
11 views

Do .sav files expire after the trial version expires? [on hold]

I saved a certain file on an SPSS software that expires after 2 weeks of its installation. I wanted to ask if my file would still exist even after the expiration date.
0
votes
1answer
11 views

Is attempting to discern gender discrimination in a distribution descriptive or inferential?

Let's say I have a given sample population $P$, that describes the traits of a group of people as a predictor variable, and whether or not they are CEOs as a response variable. I am trying to ...
0
votes
0answers
10 views

A minimum False Discovery Rate of 0.8 when doing multiple hypothesis testing on Logistic Regression Coefficient p-values?

I currently have 8,000 predictor variables and a binary indicator response variable. I then did a logistic regression on each of the predictor variables, i.e., response regressed on predictor variable ...
1
vote
1answer
15 views

Power analysis: number of participants or number of data points?

This almost answered my question below, but not quite. I want to calculate the sample size required in order to reach a certain level of a priori statistical power in my experiment. My question is ...
0
votes
0answers
8 views

Contextual bandit - problem with LinUCB parameter estimation

I am trying to understand and code up the contextual LinUCB algorithm with disjoint models from here: https://arxiv.org/pdf/1003.0146.pdf Using pretty simple simulated data wtih code below: ...
0
votes
0answers
9 views

GMM moment conditions [on hold]

I want to know why it is useful to add the moment condition about $E(X^2Y)$ for deriving parameters in linear regression.
6
votes
1answer
309 views

Relation between independence and correlation of uniform random variables

My question is fairly simple: let $X$ and $Y$ be two uncorrelated uniform random variables on $[-1,1]$. Are they independent? I was under the impression that two random, uncorrelated variables are ...
9
votes
1answer
129 views

Why is Bayesian Statistics becoming a more and more popular research topic? [on hold]

Browsing through the research area of the top 100 US News statistics program, almost all of them are heavy in Bayesian statistics. However, if I go to lower tier school, most of them are still doing ...
0
votes
0answers
13 views

How can I test for endogeneity after having used Panels Corrected Standard errors (PCSEs)?

How can I test for endogeneity after having used Panels Corrected Standard errors (PCSEs)? By endogeneity I mean the situation in which an explanatory variable is correlated with the error term. In ...
0
votes
2answers
19 views

Does the total of ordinal scores on a questionnaire become scale in SPSS?

I have a questionnaire whose data I've inputted into SPSS. I wanted to ask that do the total scores of likert items become scale and no more ordinal?
2
votes
2answers
48 views

What is a 'true' model?

A short question, but I am somehow unable to find any concrete answer. I suppose it means that the model is as good as it can be? Containing all relevant variables and hence not suffering from any ...
1
vote
0answers
11 views

Question of Degrees of freedom negative in SEM [on hold]

I have a "F_A_T_A_L E_R_R_O_R: Degrees of freedom is negative.", what can be the solution?
-1
votes
0answers
33 views

Posterior distribution of Bayesian Parameters

I am confused about how to get the posterior distribution of Bayesian Parameters. I have $t = w_1x + w_0 + \epsilon$ with $\epsilon = N(0, \sigma^2)$ How do I find $p(\textbf{w}|x_1, t_1, ...., x_N, ...
1
vote
0answers
29 views

probability involves bivariate gaussian

I'm working on a spatial project. I need to calculate the probability of a point being the closest to another. Say I'm given four points $y$, $x_1$,$x_2$ and $x_3$ in 2D plane, and let $Y'=y+Z$, where ...
0
votes
1answer
69 views

Is the Chi-Square Test of Independence the best option for 3x2 contingency table?

I have some data which in the terms of a 3x2 contingency table look in the following way. ...
0
votes
1answer
8 views

Partitioning Around Medoids: Choosing a cluster number larger than the “optimal” one?

I asked a number of 71 'experts' to sort 92 different psychological constructs based on their similarity. Based on their answers, I constructed a dissimilarity matrix. Initially, I wanted to analyse ...
0
votes
0answers
19 views

Question about the regularization on Deep learning book (Dr. Goodfellow) [on hold]

In Chapter 8 page 228, I don't understand why the component of $w^\star$ that is aligned with the $I$-th eigenvector of $H$ is rescaled by a factor of $\lambda_i/(\lambda_i + \alpha)$. And in the ...
0
votes
1answer
19 views

How to determine if two directed probabilistic graphical models are I-equivalent?

I'm trying to figure out how to determine if these two models are I-equivalent. Google didn't properly give me a solid answer so far. Any idea on to determine it? Thank you.
3
votes
0answers
26 views

UMVUE for $g(p) = \mathbb{E}_p[X^2]$, where X follows a geometric distribution

I have a random variable X with pmf $$p_\lambda(x) = (1-p)^{x-1}p, \ \ x = 1,2,3,\ldots, \ \ p \in (0,1)$$ and I am trying to find a UMVUE for $$g(p) = \mathbb{E}_p[X^2]$$. Here is my attempt so ...
0
votes
0answers
26 views

Python + Machine Learning : string matching problem

I have been given one problem to solve: The problem is explained below: The company maintains a dataset for specifications of all the products (nearly 4,500 at present) which it sells. Now each ...
0
votes
0answers
14 views

how to calulate the expected shortfall for a t distribution [on hold]

How to prove ESv(p)=(a+(Tq(1-p))^2)/(a-1) * f(Tp(1-p))/(1-p), where v is a t distribution,a is the degree freedom, Tq is the quantile function of T, f is the pdf of the t distribution. I used ...

15 30 50 per page