All Questions

From an AutoML Leaderboard on H2O, I selected a Stacked Ensemble model. I used this model to predict, using a new data set, and now I would like to obtain prediction intervals in addition to point ...

Variable smokes is a binary variable equal to one if a person smokes and zero otherwise. We estimate a linear probability model for smokes: $$\hat{smokes} = .656 - .069\log(cigprice) + .012\log(...

Do I have to test the goodness of fit before applying the paired t test? Why do I have to test linearity in the first place? and how can I perform it on a paired data set?

I have the following data for Maryland and Virginia with six different capacity types along with what score (1, 3, or 5) each person grades that capacity type. Essentially what I want to do is see if ...

Given a distribution X($E[X]$, $Var[X]$) and a target value T, I am wondering how to find a value N such that $|\sum_{i=1}^{i=N}X_i - T|$ is minimized. (i) For a large T, it makes sense to pick $N = ...

I am trying to compare two different watershed's chemical profiles to determine which is less contaminated. I have data from four different lakes in each of the two watersheds, and alkalinity, ...

In a interview i had something similar to this question Random walk on the edges of a cube. But this time there is only a ant, it takes one minute to go through a single edge. The ant can use the ...

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...

I have read http://www.sumsar.net/blog/2018/12/visualizing-the-beta-binomial/ this simple explanation of how the posterior is changing while more data are added: in this visualization there are six ...

I have a dataset of named entities (NER) and text documents. For each entity I have N documents associated, while an entity belongs to one among multiple classes like (here the CoreNLP ones) ...

Suppose I have the following true model, where an individual $i$ at a particular point in time $t$ is either treated ($W=1)$ or untreated ($W=0$). The outcome for individual $i$ at time $t$ under ...

I am interested in examining inventor features on their inventive performance using patent data. I have an unbalanced panel data of 7000 inventor-year observations on 3000 inventors over 15 years ...

In every Local Whittle function I have seen I have to set bandwith parameter usually denoted m such that m = floor(1+T^delta). I am interested in the delta, how do I choose what value to put in the ...

Suppose we have a data set $\{x_i, y_i\}_i$ where $x_i$ is a multi-dimensional tuple and scattered (not on a equally spaced regular lattice). How does one regress or smooth such a scattered data set ...

I have an experiment where subjects reported multiple results (binary) in two treatments. I have compared each subject separately to see if the treatment had an effect on a given subject, but would ...

Relatively new to both R and classification, so sorry if this is a dumb question. I'm working on a "first classification" level project using this dataset. My code below for reference. ...

I initially used Cox PH model for my data for different subgroups of my population, however I do not want to make any comparisons between groups, rather i need to describe the time to event for each ...

When performing stochastic gradient descent, it is necessary for the training loss to decrease a) between iterations in an epoch? (I think the answer is no) b) between epochs? (I think the answer is ...

I have samples $\{x_1, x_2, x_3, \dots , x_n\}$ of a random variable $X$. I compute Maximum Likelihood Estimator $\hat{\theta}_n$ using the sample data. Now, if I collect one more sample $x_{n+1}$ ...

I'm interested in numerical inputs (scalars) and output (scalar) in particular and would greatly appreciate a Python implementation (PyTorch preferably). The topic I'm looking at is a time series ...

I'm trying to solve a problem with Gibbs Sampling, so I'm trying to do: $$ x_1^1 \sim p(x_1 | x_2^0, x_3^0)\\ x_2^1 \sim p(x_2 | x_1^1, x_3^0)\\ x_3^1 \sim p(x_3 | x_1^1, x_2^1)\\ x_1^2 \sim p(x_1 | ...

Please, could you advice me a good paper which talks about the ARIMA's state-of-art? I have already searched on google but I have not found anything interesting Thank you very much

I have a set of data of counts of hares in summer and in winter, which is very unbalanced: winter was too cold, and samplings were taken only in some 20 occasions, while during the pleasant summer the ...

In keeping with the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement, I am trying to fill in their flowchart (Fig. 1) with the details of the ...

Given Consider the upper tail $H_0: \theta \leq 0 \,\,\, \text{vs} \,\,\, H_a: \theta > 0$ sign test with the test statistic $B = \sum_{i=1}^n{\psi_i}$ where $\psi_i = \mathbb{I}(Z_i > \theta)$...

I have an event which happens so many times, I want to know how significantly is 1300$ in comparison to the distribution. which test should I use ?

I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict. The model is ... ...

I'm familiar with the closed-form solution of ordinary least squares which minimizes $\sum_{n=1}^N(y_n - \mathbf{\beta x_n})^2$ for scalar $y_n$. However, in my situation I am trying to fit some data ...

I have a test and control group, of 1,000 customer each. Following a test, the two groups have sales of say £11,000 for the test group and £10,000 for the control. How can i tell if the £1,000 ...

I want to do a small project about the use of optimization in engineering design using any real-world application. I have several topics. single/multi variable optimization algorythm non tradinoal ...

I am trying to find a formula that I can use to find out what my consumption will be based on what the temperature might be, based on the comparison of what the consumption/temperature has been in the ...

Where I am in my analysis; I have concluded that I need to remove some of my variables (I have 13 of them atm) because my model suffers from multicollinearity. After this, I conduct ridge/lasso ...

Suppose an estimate for the conditional mean of $y$ given $x$ is $\hat{E}(y|x)$. Suppose the variance (or the variance estimate) of $\hat{E}(y|x)$ is known to be $V(\hat{E}(y|x))$ for all $x$. The ...

I have a machine learning challenge I may be over thinking. I have a set of 3.5 million products (not unique, there are multiple instances of each product). Each product has a "description" from it's ...

I have modelled organism dynamics and abiotic factors time series in order to understand their seasonal oscillation and trend over time. Now I want to identify if there are any correlation between ...

I am interested in the fixed effects of two within-subject experimental manipulations (i.e., Ambiguity 0 = No / 1 = Yes, and Uncertainty 0 = No / 1 = Yes) on a dichotomized variable (i.e., Punishment, ...

I know that the calculation of parameter values of a standard OLS can be made more efficient using a QR decomposition; i.e. if $X=QR$ and we are using the model $Y=X\beta+\epsilon$; Then it is true ...

I have a sequence of categorical images. For a two category image, each image pixel can have one of two values. I would like to analyze these images using a technique like eigen images. The goal is to ...

In the paper "Self-critical sequence training for image captioning" (link) on page 3 they define the loss function (of the parameters $\theta$) of an image captioning system as the negative expected ...

In this blog describing variational inference under the section KL divergence and ELBO they mention that in the equation $$p(x) = \frac{w(x)}{Z}$$ we can substitute $w$ and $Z$ with: $$Z = p(x;\...

This is a very basic question. I'm looking at a physical problem where one wants to estimate a parameter $\lambda$ of a system. Suppose I perform a measurement on the system. I call the (stochastic) ...

I feel like this is an easy question if I just knew the right words to Google, but apparently I don't, so I'm sorry if it's been asked before. I'm simply looking to determine how long I need to ...

I am facing a problem with running a simple OLS regression with two categorical independent variable. I would like to impose that the sum of the coefficients referred to one variable (x in the ...

I am currently struggling with a somewhat atypical analysis. People have been evaluating their cognitive state at two time periods – when they had attended school and at the present moment. Naturally, ...

I have paired data for two tests. I would like to say that Test A is more sensitive than Best B, but not sure if I'm using the correct methods or if my data support it. I know McNemar's Exact test can ...

Good afternoon, I'd like to ask you for some help in finding a (hopefully citable) way of analyzing panel data when looking for mediation and moderation effects. Problem I have is that my measured ...

For binary classification task, with samples labeled $y=0$ and $y=1$, a neural network has one output node with sigmoid activation function, producing predictions $\hat{y}\in(0;1)$. Is the following ...

I'm new to GBM.Can you help me to understand the interpretation of gbm.perf function? I used following code in R best.iter = gbm.perf(train, method="cv") & got ...

I'm doing time series for my masters thesis, thus I need to interpret som ARIMA models. The data consist of number of recordings per night throughout the summer. I think I'm done with my coding now, ...

I have a good prediction results on KNN-DTW group classification (group labels are either 0 or 1). But I don't know how to explain how discriminative the 2 groups are. Then I tried k-means and tsne ...