0
votes
0answers
2 views

What determines the cost function / how to find the cost function to use

I'm starting to learn a bit about gradient descent, and that method attempts to minimize the cost function. I've seen examples using linear regression, and the corresponding cost function, but now I'm ...
0
votes
0answers
5 views

Cronbach's Alpha - reliability

In psychometric tests, we sometimes are instructed to end the test after the child has failed a determined number of questions. Thus we end up with children who answered 30 questions and some who ...
3
votes
1answer
139 views

Estimating the prediction variance in kernel ridge regression

I'm trying to estimate the variance of predictions for a kernel ridge regression model. The model is simply kernel ridge regression: $$\hat{y} = K(K+\lambda I)^{-1}y = A y$$ $K$ is the $n \times n$ ...
9
votes
5answers
490 views

SOM clustering for nominal/circular variables

Just wondering if anyone is familiar with clustering nominal inputs. I've been looking at SOM as a solution but apparently it only works with numerical features. Are there any extensions for ...
3
votes
2answers
454 views

How to draw fitted graph and actual graph of gamma distribution in one plot?

Load the package needed. library(ggplot2) library(MASS) Generate 10,000 numbers fitted to gamma distribution. ...
3
votes
3answers
148 views

Logistic Regression: Does my model selection process make sense?

This is kind of a broad question and so I am okay with broad or general answers. In fact, each of these could be their own individual questions, but I think it makes sense to ask them all. Even if you ...
1
vote
0answers
6 views

What is the point of using an initial estimate when sigma-clipping data?

I am reading a paper that is discussing sigma clipping some velocity data, lets call it a set $V$, and it says: The center at each iteration of the $3\sigma$ clipping is the biweight average, and $...
0
votes
1answer
11 views

xgboost: does it support stochastic gradient boosting?

What function/parameter needs to be set to enable stochastic gradient boosting in XGBoost? I ask because in https://statweb.stanford.edu/~jhf/ftp/stobst.pdf, Jerome Friedman shows that stochastic ...
2
votes
1answer
732 views

R - Weibull Distribution Parameters (Shape and Scale) - Confidence Intervals

I am wondering how you would obtain Scale and Shape parameter values on a Weibull Distribution's Confidence Interval bands (95% CI). The following post nicely illustrates confidence interval bands ...
0
votes
1answer
19 views

Bayesian estimation of Dynamic Linear Models with RStan

I'm reading the Dynamic Linear Models with R book, where most of chapter 4 is devoted to bayesian estimation of parameters. They code most of it manually though, and it seems it can get quite tricky ...
1
vote
1answer
16 views

Limiting the memory window of an LSTM network

I'm using an LSTM to explore some properties of long range dependencies in some data. One of the things that I was hoping to do was to examine the effects of limiting the "memory" of the network to ...
0
votes
0answers
5 views

Hard question - Trying to predict one dependent, continuous variable in 2 conditions, both with different correlations, how do I proceed?

I'm trying to explain the variance experienced with cybersickness, roughly put a type of motion sickness experienced inside of Virtual Reality (more specifically, I put participants in a virtual ...
1
vote
0answers
22 views

How to make a customized kernel function

I have a sample set ${(\mathbf{x_i})}_{I=1}^N$, each $\mathbf{x_i}$ $\in R^d$ and $\mathbf{x_i}$ is a column vector with $d$ dimensions. Webpage gives an idea that a kernel function can be build ...
0
votes
0answers
12 views

Threshold to segment count data following different distribution

I have a dataset D, wherein $ D = \{x_i, x_{i+1}, \ldots, x_n\} $. each data point in D is a discrete count data and since this is spatial data, one can expect dependence between contiguous data-...
1
vote
3answers
65 views

Logistic Regression Cutoff Values for Multiple Models

I understand that once the logistic regression model has output probabilities, a cutoff value for classifying probabilities of new observations is decided for a model to optimize some metric like ...
1
vote
1answer
11 views

Why are the weights of RNN/LSTM networks shared across time?

I've recently become interested in LSTMs and I was surprised to learn that the weights are shared across time. I know that if you share the weights across time, then your input time sequences can ...
0
votes
0answers
9 views

relationship between fully connected layer and convolutional layer

When reading about the transforming the fully connected layer into convolutional layer, posted in http://cs231n.github.io/convolutional-networks/#convert. I just feel confused about how to understand ...
0
votes
1answer
177 views

GAM summary F values

Does anyone know how the F values are computed in summary(gam.object). I've looked through the help page, and the gam package, ...
1
vote
1answer
22 views

Are there disadvantages using proportional features instead of absolute values?

I was wondering whether there are disadvantages in using proportional features instead of features with absolute values. For example: I have the following data set, which includes, TV duration, ...
0
votes
0answers
6 views

how to calculate regression coefficients in terms of original variables when i already have regression coefficients in terms of PC1 PC2 etc.? [duplicate]

while doing Principal component regression i take the input data, standardize it calculate pca and use the score matrix to solve the equation Y=score*B where Y is my mean centered known output and B ...
0
votes
0answers
7 views

Method for stochastic data partitioning

I have one task. I have random polynomial like $F(x) = a_0(\omega) + a_1(\omega)x +\cdots+ a_n(\omega)x^n$, where $a_i(\omega)$ is a random variable for each $i = 1, \ldots, n$. Let looks for a ...
0
votes
0answers
11 views

Why do moving averages of a sample with zero mean have non-zero mean?

I came across this (I think) paradox. Say I simulate normal distribution in R x <-rnorm(10000) t.test(x) data: x t = 0.64827, df = 9999, p-value = 0.5168 Then I calculate a sample of ...
0
votes
0answers
6 views

Single pass object detection

Let we have a set of images $\{\mathcal I_i\}_{i=1}^n$ with labels $\{\mathcal B_i\}_{i=1}^n$, where each $\mathcal B_i$ is a set of regions. The problem is to find a function that given image $\...
10
votes
1answer
169 views

What does it mean to say that an event “happens eventually”?

Consider a 1 dimensional random walk on the integers $\mathbb{Z}$ with initial state $x\in\mathbb{Z}$: \begin{equation} S_n=x+\sum^n_{i=1}\xi_i \end{equation} where the increments $\xi_i$ are I.I.D ...
0
votes
0answers
5 views

What should I expect biweight weighting values to be?

I am working on a project that involves taking the biweight sample variance of a velocity dataset. This is defined as $\sigma_{BI}^2 = N\dfrac{\sum_{|u_i|<1}(1-u_i^2)^4(v_i-\bar{v})}{D(D-1)}$ ...
5
votes
5answers
427 views

Boundary or threshold test for regression-type scatter plot

I am looking for a way to test whether a boundary threshold exists in a physiological response – a sample of the data is plotted below. My hypothesis is that the X-variable imposes a physiological ...
0
votes
2answers
22 views

internal reliability

I have been searching for information on how to conduct internal reliability assessments but have been unsuccessful. Is there a book or website that can inform me on how to conduct these analysis? I ...
0
votes
0answers
7 views

Support vector data description (SVDD) [on hold]

How can i explain the dual of the SVDD method?. what's the influence of linear term on the dual objective function?
2
votes
0answers
26 views

Modelling binomial data with censoring in dependent variable and in independent covariate

I am working on a problem in which I have multiple pairs of currently living males i that each have a presumed paternal ancestor ...
3
votes
2answers
338 views

Normalized RMSE

I have several time-series in a VAR(1) and, due to some of them haven't the same unit of measure, I'd like to estimate the RMSE in percentage. I know that it could be done in several ways (see below) ...
0
votes
1answer
4 views

internal reliability scores below .70

When one is assessing the reliability of one instrument and the value is under .70 and you then eliminate the item/items that are mentioned that if eliminated would increase the reliability score, ...
0
votes
0answers
2 views

Confront % changes over time in a time series split by categories

I am working with a problem where I have a time series of sales data and I can look at this data across different categories (for example I can look at sales data over time categorized by products). ...
0
votes
2answers
16 views

Neural network input values belonging to classes

I need help on configuring a neural network. I would like to pass in accelerometer values (x,y,z) from two different sensors, and have the network compute the corresponding angle. I am providing close ...
0
votes
0answers
5 views

Calculate EER from FAR and FRR?

I'm wondering if we have FAR and FRR scores for each threshold if we can compute an EER programatically? Say we have: ...
0
votes
1answer
18 views

Finding correlation between nonlinear variables sets & forecasting

I have two data sets. Data Set A is % over/under budget for a list of non-related projects. Data Set B is % over/under time for the same list of projects. Here is a sample of the data Will I be ...
0
votes
0answers
21 views

Interpreting the regression coefficient when the regressor is polychoric-based principal component

I have a regression where I am trying to interpret the regression coefficient of the first principal component on some outcome variable. The component scores variable was obtained in a polychoric ...
11
votes
4answers
387 views

What is an intuitive explanation for how PCA turns from a geometric problem (with distances) to a linear algebra problem (with eigenvectors)?

I've read a lot about PCA, including various tutorials and questions (such as this one, this one, and this one). The geometric problem that PCA is trying to optimize is clear to me: PCA tries to ...
0
votes
0answers
15 views
0
votes
0answers
6 views

Minimum sample size to check second order stationarity

I can understand second order stationarity refers to weakly stationary processes in which mean, variance, and autocovariance do not depend on time. But is it possible to check weak stationarity/second ...
1
vote
0answers
9 views

How does imputation affect significance test power?

There are many different imputation methods that make different assumptions about the missingness of the data (MCAR, MNAR, MAR). Since this assumption may not be met, I'm interested in knowing how the ...
0
votes
0answers
13 views

Estimating correlation(covariance) matrix when fitting a copula using R copula package [migrated]

I have a question about the R package copula. When using fitCopula to fit a copula to data, ...
0
votes
0answers
6 views

natural breaks arcpy [on hold]

Is there any chance I can create a simple python script to classify my data as Natural Breaks? I have only created a script for quantile classification, but it is not enough for my work. ...
0
votes
0answers
12 views

Correlation with seasonal data

I've got 5+ years of data, with multiple observations per week. I'd like to understand if there is a correlation between my dependent and independent variables. The catch is that I know this data is ...
2
votes
1answer
83 views

How can I experiment with Lagrange multiplier in PCA optimization?

Suppose we want to solve following optimization problem (it is a PCA problem in this post) $$ \underset{\mathbf w}{\text{maximize}}~~ \mathbf w^\top \mathbf{Cw} \\ \text{s.t.}~~~~~~ \mathbf w^\top \...
-1
votes
2answers
29 views

Covariance Matrix for Time Series

I'm trying to investigate how events affect the stock market through econo-physics and I came across a paper that uses the co-variance matrix. What I don't understand is how such a matrix can be ...
0
votes
0answers
4 views

Are estimators of heterogenity variance and of the random-effects mean asymptotically independent?

In random-effects meta-analysis, there are many different methods for estimating the across-study heterogeneity, $\tau^{2}$ (including DerSimonian-Laird's moment estimator, ML and REML estimators, and ...
2
votes
1answer
169 views

Machine learning tutorials / examples on data sets larger than a terabyte

I am trying to gather a list of practical ML examples / tutorials on more than a terabyte of data. I'm particularly interested in feature extraction from large data sets that involves aggregation (the ...
0
votes
0answers
10 views

Dataframes of different lengths [migrated]

I'm looking at the time series of the DJIA and FSTE100 but they are not of the same length because of trading days. How can I fix this in R? I saw a code snippet and I tried to adapt it must my data ...
0
votes
0answers
26 views

Interpret hazard ratio that has huge value

I run a coxph model in R using survival package. Here's the output ...
1
vote
0answers
33 views

predict function is underestimating in cases where polynomials are included in an lmer model

I have been having trouble with the predict function underestimating the predictions from an lmer model with some polynomials. Hopefully my edits make it clearer. I have scaled data that looks like ...

15 30 50 per page