All Questions

0
votes
0answers
2 views

How to get prediction interval from a model obtained in h2o?

From an AutoML Leaderboard on H2O, I selected a Stacked Ensemble model. I used this model to predict, using a new data set, and now I would like to obtain prediction intervals in addition to point ...
0
votes
0answers
4 views

Simple Regression Question for Probability of Smoking

Variable smokes is a binary variable equal to one if a person smokes and zero otherwise. We estimate a linear probability model for smokes: $$\hat{smokes} = .656 - .069\log(cigprice) + .012\log(...
0
votes
0answers
3 views

Is it necessary to perform linear regression test when applying paired t test?

Do I have to test the goodness of fit before applying the paired t test? Why do I have to test linearity in the first place? and how can I perform it on a paired data set?
0
votes
0answers
3 views

Interpreting Multiple Correspondence Analysis Output

I have the following data for Maryland and Virginia with six different capacity types along with what score (1, 3, or 5) each person grades that capacity type. Essentially what I want to do is see if ...
0
votes
0answers
3 views

How to find the distance confidence between the sum of random variables and a given target value?

Given a distribution X($E[X]$, $Var[X]$) and a target value T, I am wondering how to find a value N such that $|\sum_{i=1}^{i=N}X_i - T|$ is minimized. (i) For a large T, it makes sense to pick $N = ...
0
votes
0answers
2 views

Multivariate analysis for data concerning lake chemical profiles

I am trying to compare two different watershed's chemical profiles to determine which is less contaminated. I have data from four different lakes in each of the two watersheds, and alkalinity, ...
0
votes
0answers
18 views

Average time for a random walk on the edges of a cube

In a interview i had something similar to this question Random walk on the edges of a cube. But this time there is only a ant, it takes one minute to go through a single edge. The ant can use the ...
0
votes
0answers
6 views

Feature selection in xgboost vs GBM in H2O

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...
0
votes
1answer
5 views

Beta-binomial Model with missing values

I have read http://www.sumsar.net/blog/2018/12/visualizing-the-beta-binomial/ this simple explanation of how the posterior is changing while more data are added: in this visualization there are six ...
0
votes
0answers
5 views

Visualize Named Entities Relations

I have a dataset of named entities (NER) and text documents. For each entity I have N documents associated, while an entity belongs to one among multiple classes like (here the CoreNLP ones) ...
0
votes
0answers
6 views

Linear Regression: Calculating a treatment effect directly in regression vs. averaging potential outcomes

Suppose I have the following true model, where an individual $i$ at a particular point in time $t$ is either treated ($W=1)$ or untreated ($W=0$). The outcome for individual $i$ at time $t$ under ...
1
vote
0answers
8 views

Random effect vs. fixed effect with a huge amount of dummies

I am interested in examining inventor features on their inventive performance using patent data. I have an unbalanced panel data of 7000 inventor-year observations on 3000 inventors over 15 years ...
0
votes
0answers
6 views

Specifying the bandwidth parameter for Local Whittle estimation

In every Local Whittle function I have seen I have to set bandwith parameter usually denoted m such that m = floor(1+T^delta). I am interested in the delta, how do I choose what value to put in the ...
0
votes
0answers
6 views

Wavelet smoothing or regression with scattered data

Suppose we have a data set $\{x_i, y_i\}_i$ where $x_i$ is a multi-dimensional tuple and scattered (not on a equally spaced regular lattice). How does one regress or smooth such a scattered data set ...
1
vote
0answers
11 views

Generalized Linear Mixed Effects Logistic Regression with Repeated Measures

I have an experiment where subjects reported multiple results (binary) in two treatments. I have compared each subject separately to see if the treatment had an effect on a given subject, but would ...
0
votes
0answers
8 views

klaR NaiveBayes warnings “Numerical 0 probability for all classes…”

Relatively new to both R and classification, so sorry if this is a dumb question. I'm working on a "first classification" level project using this dataset. My code below for reference. ...
0
votes
1answer
14 views

What survival model is suitable for not comparing groups?

I initially used Cox PH model for my data for different subgroups of my population, however I do not want to make any comparisons between groups, rather i need to describe the time to event for each ...
0
votes
1answer
9 views

Is training loss guaranteed to decrease for stochastic gradient descent?

When performing stochastic gradient descent, it is necessary for the training loss to decrease a) between iterations in an epoch? (I think the answer is no) b) between epochs? (I think the answer is ...
1
vote
0answers
22 views

Effect of adding more sample data on Maximum Likelihood estimator

I have samples $\{x_1, x_2, x_3, \dots , x_n\}$ of a random variable $X$. I compute Maximum Likelihood Estimator $\hat{\theta}_n$ using the sample data. Now, if I collect one more sample $x_{n+1}$ ...
0
votes
0answers
5 views

Is it possible to create LSTM with attention mechanism for many-to-one problems?

I'm interested in numerical inputs (scalars) and output (scalar) in particular and would greatly appreciate a Python implementation (PyTorch preferably). The topic I'm looking at is a time series ...
0
votes
0answers
16 views

Gibbs sampling where I can only find the mode of conditionals?

I'm trying to solve a problem with Gibbs Sampling, so I'm trying to do: $$ x_1^1 \sim p(x_1 | x_2^0, x_3^0)\\ x_2^1 \sim p(x_2 | x_1^1, x_3^0)\\ x_3^1 \sim p(x_3 | x_1^1, x_2^1)\\ x_1^2 \sim p(x_1 | ...
0
votes
1answer
23 views

state-of-art of ARIMA

Please, could you advice me a good paper which talks about the ARIMA's state-of-art? I have already searched on google but I have not found anything interesting Thank you very much
0
votes
0answers
10 views

Asking opinion about random forests, chi squared and data permutation

I have a set of data of counts of hares in summer and in winter, which is very unbalanced: winter was too cold, and samplings were taken only in some 20 occasions, while during the pleasant summer the ...
0
votes
1answer
6 views

PRISMA Statement: number of records returned by (one/several) databases?

In keeping with the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) Statement, I am trying to fill in their flowchart (Fig. 1) with the details of the ...
0
votes
0answers
16 views

Properties of Nonparametric Test Statistics $B$

Given Consider the upper tail $H_0: \theta \leq 0 \,\,\, \text{vs} \,\,\, H_a: \theta > 0$ sign test with the test statistic $B = \sum_{i=1}^n{\psi_i}$ where $\psi_i = \mathbb{I}(Z_i > \theta)$...
-1
votes
2answers
41 views

can one calculate a p value in a distribution

I have an event which happens so many times, I want to know how significantly is 1300$ in comparison to the distribution. which test should I use ?
0
votes
0answers
4 views

Linear discriminant scores using caret

I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict. The model is ... ...
0
votes
1answer
21 views

Closed-form solution to least squares with a matrix of parameters?

I'm familiar with the closed-form solution of ordinary least squares which minimizes $\sum_{n=1}^N(y_n - \mathbf{\beta x_n})^2$ for scalar $y_n$. However, in my situation I am trying to fit some data ...
0
votes
1answer
13 views

Statistical significance in sales - test vs control

I have a test and control group, of 1,000 customer each. Following a test, the two groups have sales of say £11,000 for the test group and £10,000 for the control. How can i tell if the £1,000 ...
0
votes
0answers
7 views

optimization of engineering design algorithms

I want to do a small project about the use of optimization in engineering design using any real-world application. I have several topics. single/multi variable optimization algorythm non tradinoal ...
0
votes
0answers
7 views

help with formula for prediciting electrical consumption

I am trying to find a formula that I can use to find out what my consumption will be based on what the temperature might be, based on the comparison of what the consumption/temperature has been in the ...
0
votes
0answers
19 views

Ridge/lasso graph interpretation in R [duplicate]

Where I am in my analysis; I have concluded that I need to remove some of my variables (I have 13 of them atm) because my model suffers from multicollinearity. After this, I conduct ridge/lasso ...
0
votes
0answers
17 views

What is the marginal variance of the mean of $y$ if I only know the variance of the conditional mean of $y$?

Suppose an estimate for the conditional mean of $y$ given $x$ is $\hat{E}(y|x)$. Suppose the variance (or the variance estimate) of $\hat{E}(y|x)$ is known to be $V(\hat{E}(y|x))$ for all $x$. The ...
0
votes
0answers
9 views

Machine learning for product names

I have a machine learning challenge I may be over thinking. I have a set of 3.5 million products (not unique, there are multiple instances of each product). Each product has a "description" from it's ...
1
vote
0answers
10 views

Using non-stationary time series in cross-correlation analysis

I have modelled organism dynamics and abiotic factors time series in order to understand their seasonal oscillation and trend over time. Now I want to identify if there are any correlation between ...
2
votes
0answers
20 views

Estimates radically change when including Random Slopes in Multiple Logistic Regression

I am interested in the fixed effects of two within-subject experimental manipulations (i.e., Ambiguity 0 = No / 1 = Yes, and Uncertainty 0 = No / 1 = Yes) on a dichotomized variable (i.e., Punishment, ...
1
vote
0answers
9 views

Generalised least squares using QR decomposition

I know that the calculation of parameter values of a standard OLS can be made more efficient using a QR decomposition; i.e. if $X=QR$ and we are using the model $Y=X\beta+\epsilon$; Then it is true ...
1
vote
0answers
6 views

Basis vectors for categorical images

I have a sequence of categorical images. For a two category image, each image pixel can have one of two values. I would like to analyze these images using a technique like eigen images. The goal is to ...
1
vote
0answers
11 views

Reinforcement Learning partial derivative of loss function w.r.t. input of softmax

In the paper "Self-critical sequence training for image captioning" (link) on page 3 they define the loss function (of the parameters $\theta$) of an image captioning system as the negative expected ...
0
votes
0answers
9 views

variational lower bound confusion

In this blog describing variational inference under the section KL divergence and ELBO they mention that in the equation $$p(x) = \frac{w(x)}{Z}$$ we can substitute $w$ and $Z$ with: $$Z = p(x;\...
0
votes
1answer
28 views

Interpretation of Mean Square Error formula

This is a very basic question. I'm looking at a physical problem where one wants to estimate a parameter $\lambda$ of a system. Suppose I perform a measurement on the system. I call the (stochastic) ...
3
votes
1answer
26 views

Determining a Statistically Significant Result

I feel like this is an easy question if I just knew the right words to Google, but apparently I don't, so I'm sorry if it's been asked before. I'm simply looking to determine how long I need to ...
1
vote
0answers
20 views

R - Linear Regression with parameters constraint - contr.sum contrast

I am facing a problem with running a simple OLS regression with two categorical independent variable. I would like to impose that the sum of the coefficients referred to one variable (x in the ...
0
votes
0answers
6 views

Two correlated ordinal independent variables, several continuous dependent variables

I am currently struggling with a somewhat atypical analysis. People have been evaluating their cognitive state at two time periods – when they had attended school and at the present moment. Naturally, ...
0
votes
0answers
6 views

Conditional Exact Logistic Regression

I have paired data for two tests. I would like to say that Test A is more sensitive than Best B, but not sure if I'm using the correct methods or if my data support it. I know McNemar's Exact test can ...
0
votes
0answers
8 views

Analysing panel data - looking for hypothesized mediation and moderation effects

Good afternoon, I'd like to ask you for some help in finding a (hopefully citable) way of analyzing panel data when looking for mediation and moderation effects. Problem I have is that my measured ...
0
votes
0answers
8 views

Loss function for a risk neutral binary classification

For binary classification task, with samples labeled $y=0$ and $y=1$, a neural network has one output node with sigmoid activation function, producing predictions $\hat{y}\in(0;1)$. Is the following ...
1
vote
0answers
11 views

How to interpret chart generated by gbm.perf function?

I'm new to GBM.Can you help me to understand the interpretation of gbm.perf function? I used following code in R best.iter = gbm.perf(train, method="cv") & got ...
0
votes
0answers
14 views

How to interpret ARIMA (1,0,0) model?

I'm doing time series for my masters thesis, thus I need to interpret som ARIMA models. The data consist of number of recordings per night throughout the summer. I think I'm done with my coding now, ...
0
votes
0answers
20 views

How to explain why KNN has good classification performance (have example)

I have a good prediction results on KNN-DTW group classification (group labels are either 0 or 1). But I don't know how to explain how discriminative the 2 groups are. Then I tried k-means and tsne ...

15 30 50 per page