0
votes
0answers
2 views

Second layer pre-training for a stacked autoencoder has to reconstruct a badly scaled data

For my greedy layer-wise pre-training using sparse autoencoder, the first layer training seems to be okay since it can fairly reconstruct my test set. However, because I use "sparse" autoencoder, the ...
0
votes
0answers
2 views

Compute the infimum of the L^2-distance to the set of Gaussian functions

I am faced with the following question: given a (nonnegative) distribution function $f$ over $\mathbb{R}$ with unit mass and $\int_{\mathbb{R}}f(x)x^2dx=\sigma^2$ given, I am trying to find the ...
0
votes
0answers
3 views

Checking linearity assumption and colinearity assumption

Linear regression is equivalent to ANOVA. Do we need to check for linearity and multicolinearity for ANOVA? I have seen that these assumptions are usually omitted for ANOVA but not for linear ...
1
vote
0answers
6 views

How best to summarize a predictive distribution in a single number?

I have generated a predictive distribution for a future observable outcome, and would like to generate a single value $p$ which we would most likely encounter when we perform the experiment the next ...
0
votes
0answers
6 views

Which is the PACF of an AR(2) =

Which is the PACF(1) of the following AR(2) process? $$ y_t = \phi y_{t-2}+\epsilon_y$$
0
votes
0answers
2 views

error propagation on the median

Suppose to have a set of exact data $x_i, i=1,\dots,N$ and to calculate its median value $m$. Then, a sound way to estimate the error $\delta m$ on $m$ would be bootstrapping. (I think...) But what ...
0
votes
0answers
2 views

Generating weights with a normal distribution (Mediation analysis)

This mediation analysis regards how much of the social inequality (as a binary exposure) in long term sickness absence is mediated through physical work environment. The mediator is the logarithm of ...
0
votes
0answers
2 views

Problem of infinite or missing values in Hessian at convergence in the case of joint longitudinal and survival sub model

I'm estimating a Joint longitudinal and survival model using the JM package. The procedure works well for a subset of my data, but if I try to use the entire data set I get the following warning ...
0
votes
0answers
2 views

Making knowledge discovery services scalable on clouds for big data mining

i'm a research scholar. i need to do research by the title in a new way. i'm entirely blank. what are the methodologies to follow this research. i got the title from ieee.org here's the link. ...
0
votes
0answers
6 views

DIfferent zero-inflated poisson model outputs of Frequentist and Bayesian

I modeled zero-inflated poisson regression using crime data. There are 10034 obs and let assume spatial autocorrelation is not significant. For the Frequentist approach, R was used and code is ...
0
votes
0answers
3 views

Not sure how to model some probabilities on web traffic

So, I have a dataset of web site visits (a tidy one, thankfully), of which the most important columns are user_id and page. I have a training set for the first 11 months of the year, and a test set ...
0
votes
0answers
4 views

Estimating days of usage for a limited time frame

I would like to estimate how many individual days of a week user have been using a feature in my app, but I'm kinda lost as to do that? Basically the set up is an A/B test, where some users get one ...
0
votes
2answers
18 views

how can i calculate $E(\Pi_{i=1}^{n}\displaystyle\frac{X_i}{X_{(n)}})$

suppose $X_1,X_2,...,X_n$ be a random sample of $U(0,\theta)$. how can i calculate $E(\Pi_{i=1}^{n}\displaystyle\frac{X_i}{X_{(n)}})$. $X_{(n)}$ is largest of order statistics
0
votes
0answers
7 views

code for the fixed effect (conditional) and the random effect model in SAS using panel data and a logistic+multinomial logistic model

With respect, I am using a panel data for my thesis and I am Using SAS 9.4 for the analysis. I was wondering if I could have the chance for your advice relative to writing the codes for the model in ...
0
votes
0answers
7 views

Proportional hazard model with fixed effects in stata

I am pretty new with stata. I am running a proportional hazard model and my dependent variable is bankruptcy. I have panel data of 537 firms(co) from 2004 to 2015(t), and I want to control industry ...
0
votes
0answers
5 views

Statistic test that check difference of proportion in paired data

I want to announce that my algorithm (denoted by A) is better than some state-of -the-art algorithm (denoted by B) I have n samples which are sentences from the wall street journal and for each ...
0
votes
1answer
12 views

Calculating standard deviations and testing the null hypothesis in a biological sample with n = 6

I am doing experiments on the viability of cells after treatment with different amounts of a certain substance. I have split my original cell line into 6 separate flasks at 3 days after unfreezing. ...
0
votes
0answers
4 views

specific example of reinforcement learning using linear function approximation

I can think of two cases when linear function approximation in reinforcement learning is useful: when the state space is large enough, or when the state space is continuous. I have been reading many ...
0
votes
0answers
7 views

Random right cenoring

I observe a series of values from different trials ($Y_1, ... Y_N$). All values come from the same distribution ($F(\cdot)$). Trials do not have the same number of values ($N$ differs across ...
0
votes
0answers
7 views

In General, how to calculate “X” levels of significance?

After looking over statistics I have come across many questions which involve finding % level of significance of a set of data. how is this generally solved ?
0
votes
0answers
18 views

my Wilcoxon is contradicting my Spearmans Rho [duplicate]

In brief, I'm trying to find out if 62 people perform better on test one or test two. I ran a spearman's rho and it came back positive and significant with a moderate strength. In other words, when ...
0
votes
0answers
5 views

Relationship Between Correlations and Contour Plots for OLS

In the paper, "Simultaneous Regression Shrinkage, Variable Selection and Supervised Clustering of Predictors with OSCAR" (Bondell, Reich), the authors state: "As the contours are in terms of $X^TX$ ...
0
votes
0answers
7 views

How to proper evaluate the PDF of a Beta Distribution?

On page 40 of "Think Bayes - Bayesian Statistics Made Simple", Allen evaluates the PDF of the Beta distribution as ...
6
votes
2answers
167 views

Are CDFs more fundamental than PDFs?

My stat prof basically said, if given one of the following three, you can find the other two: Cumulative distribution function Moment Generating Function Probability Density Function But my ...
0
votes
0answers
8 views

Inference about two samples

I have two different large samples, one is normally distributed (0,1) and the other one follows a t-distribution (df = 4). Can I compare whether the means or the st. deviations of both samples are the ...
0
votes
0answers
9 views

2 features and 2 principal components

Whats the difference between 2 features and 2 principal components? I know what a PCA is, I just have the following problem: If my data has 2 features, the PCA will produce 2 components. So why does ...
0
votes
0answers
4 views

Do I have to find all the complement subspaces when a matrix A does not have pivots in every column?

Say I have a 2*2 matrix A, it has only one pivot in the first column. I have learnt column, null, left-null, and row spaces but I am confused which space I need to find for a complement vector space ...
0
votes
0answers
4 views

“Likelihood Ratio” for multiclass

If i have 2 classes, the likelihood ratio gives me a boundary like the image below. What if i have 3 or more classes ? What test can I use to classify an input data ? What if it is a mixture of ...
0
votes
1answer
20 views

Feature selection: PCA vs intuition? [on hold]

Which one should I choose? How can I combine them (i.e. in series or parallel)? What if there are dummy features in my data? What if my intuition messes things up?
0
votes
0answers
8 views

What explanation can i give for having a negative correlation coefficient but positive regression coefficient? [duplicate]

The first table is the regression results showing negative correlation coefficients for all variables. However in the regression table below, only EXR shows the same association as the correlation. ...
1
vote
1answer
16 views

large sample and significance

I have a very large sample (790 million rows), and in this case, even small difference are determined to be significant, p <0.0001. The 95% C.I.s are very sharp. In this case, these tests (t-test, ...
0
votes
1answer
9 views

Is there a concern for multiple comparisons when ranking instead of using significance?

Let's say I have many ANOVA tests on the same data using different factor variables each time. Then, I want to rank the F-statistics for all of those tests. Notice that I'm not interested in ...
2
votes
0answers
9 views

How to determine counts of viruses from limiting dilution assay

My question relates to a common experiment in the field of virology. Let me give some background so you understand why I'm asking this question. When producing virus for an experiment you typically ...
2
votes
0answers
12 views

What is the difference between the Monte Carlo Method in R package 'DMwR' and a normal Monte Carlo Method?

I am trying to estimate the performance of a machine learning model on time series data. I saw the example of model evaluation using Monte Carlo Estimates from the book "Data Mining With R Learning ...
0
votes
0answers
19 views

Using confidence intervals with Simple Linear Regression

So simple linear regression is performed on 3000 data points, and 1000 data points are withheld. How can we use confidence intervals, along with the withheld data points, to assess the predictive ...
0
votes
1answer
7 views

Generalized Hurst Exponent - What value to specify for $\tau_{\max}$?

Consider a time series $X: S \to \mathbb{R}$, where $S := \{\nu, 2\nu, 3\nu, \ldots T\}$, and $T$ is a multiple of $\nu > 0$. For each $\tau \in (0, \tau_{\max}] \cap S$ and $q \in \mathbb{N}$, ...
0
votes
0answers
7 views

how to achieve function criterion to be close to zero [on hold]

I wrote this R script to solve solve this system of nonlinear equations using nleqslv package. ...
0
votes
2answers
7 views

How does DeepMind's DQN agent determines target values for sampled episodes?

In DeepMind's paper titled Human-level control through deep reinforcement learning authors present Algorithm 1 (available in the paper reachable with mentioned link) for training of deep Q-learning ...
1
vote
0answers
3 views

Modelling a Static Bayes Net versus Dynamic Bayes Net

I have a Bayes Net with 20 variables, but I found out that one of the Parent variables is dependent on the previous value of its Child as: C(t-1)->P(t)->C(t) C and P are binary (True or False). All ...
1
vote
1answer
15 views

Finding subset of data where the 'B' in A/B testing is true

Suppose that I have implemented a referral program in my product, in which I offer rewards to a user whenever one of their referrals uses my product. Say, we reward him with gift/prize 'A'. I am also ...
0
votes
0answers
8 views

Standard error for an estimate of invented statistic

I created a statistic that is the division of two proportions. Now, I wanted to compute the standard error, which I know how to calculate for each proportion: $$SE = ...
0
votes
0answers
6 views

Find a significance difference between two models

I am trying to determine if immigration status is a determinant of risk preferences. To do this, I am using the 2014 Health and Retirement Study data which has approximately ~20,000 participants and ...
-2
votes
0answers
13 views

Aggregate with custom function [on hold]

I am new to statistics, I have quick doubt regarding the aggregate custom functions. Could you say what would be the sum(x) and length(x) for each row from the second snap? but am able to understand ...
0
votes
1answer
17 views

Why is my Spearman's Rho result contradicting my Wilcoxon matched pairs?

In brief I'm trying to find out if 62 people perform better on test one or test two. I ran a Spearman's rho and it has come back positive and significant with a moderate strength. I.E As the test 1 ...
0
votes
0answers
9 views

Strange predictions from GAMs

I am fitting a GAM model to data of capture success in trapping sites using the mgcv package in R. The independent variables are proportions of land cover types. Here are: the output of my model: ...
0
votes
0answers
8 views

Partial identification for layman

Can someone please explain Manski approach to partial identification in very basic terms? I have gone through a lot of stuff but failed to find an intuitive explanation. I also don't understand how ...
1
vote
1answer
8 views

Loss function/error measurement for allocation problem

I'm working on a prediction rule for an allocation problem. So, it's data like this: ...
0
votes
0answers
9 views

can I use the cluster membership from cluster analysis (or LCA) for classification

We had a survey of >1500 patients, and we did cluster analysis and also LCA, and grouped these patients into 3 classes. My supervisor wants to develop a algorithm to predict class membership of ...
2
votes
0answers
14 views

Chi square test in a contigency table with sampling zeroes

im trying to estimate if an association exists between two variables in a 7x5 contingency table (the type of prosthesis with the type of complication). The table contains a lot of sampling zeroes and ...
0
votes
0answers
20 views

When do we have that $E[X_{T+1} | X_T] = X_T$?

Given a filtered probability space $(\Omega, \mathscr{F}, \{\mathscr{F}_n\}_{n \in \mathbb{N}}, \mathbb{P})$: Let $X = \{X_n\}_{n \in \mathbb N}$ be a $(\{\mathscr{F}_n\}_{n \in \mathbb{N}}, ...

15 30 50 per page