New answers tagged

1

The goal of a hard-margin SVM (you didn't mention slack variables, $\xi$, for soft-margin SVMs) is to minimize the Euclidean norm $||\mathbf{w}||$ to impose the inequality $y_i D(\mathbf{x}_i)/||\mathbf{w}||>1$ (Boser et al, 1992). This is accomplished by using both the margin value $M$ and the weight vector $\mathbf{w}$ and enforcing the constraint $M ||...


0

By writing the joint density in exponential family form we find $$ f(x,\theta) = \frac1{\sqrt{2\pi}} \exp\left\{ -\frac{n}{2}\log\theta - \frac1{2\theta} \sum_i x_i^2 + \sum_i x_i -n\theta^2 \right\} $$ and then the result follows by general exponential family theory, as noted in comments.


0

Please add some context in which you find this term, sometimes terms explain themselves out of context. Without further context my understanding is, that machine learning models are trained on a sample of data and that extra sample or out-of-sample means, that this observation was not used in the training phase of the model.


1

This is just an extended comment. You might want to try a symbolic computation program such as Mathematica, Maple, or MATLAB. Here are the results for the mean and variance using Mathematica. (And maybe there is some simplification available?) dist = TransformedDistribution[Abs[x], x \[Distributed] VarianceGammaDistribution[\[Lambda], \[Alpha], \[Beta],...


1

This is related to Cover's theorem. A nice summary by Emin Orhan is given here. Ps: I would post this in a comment but don't have enough reputation.


3

A quick one: This follows from a property of moments (a rule to transform the center) $$E\left[(x-\hat{x})^n\right] = \sum_{i=0}^n {n \choose i} E\left[(x-a)^i\right] (a-\hat{x})^{n-i}$$ which becomes for $n=2$ and $a=\mu=E[X]$ $$E \left[(x-\hat{x})^2\right] = \underbrace{E \left[(x-\mu)^2\right] }_{=\text{Var}(x)} + 2 \underbrace{E \left[(x-\mu)\right] }...


0

In general, survival as a function of time, $S(t)$, is conveniently expressed in terms of the cumulative hazard function $H(t)$: $$ -\log S(t) = H(t) $$ where (trying to match the symbols in the linked video) $H(t) = \int_0^t \lambda(\tau) d\tau$ is the cumulative hazard function, integrating through time $t$ the instantaneous hazard function $\lambda(\tau)$....


42

Budding statisticians in Statistics 101 with no mathematical skills beyond high-school algebra should consider \begin{align} E\left[(X-a)^2\right] &= E\bigr[\big(X-\mu + \mu -a\big)^2\bigr] & {\scriptstyle{\text{Here,}~\mu ~ \text{denotes the mean of} ~ X}}\\ &= E\bigr[\big((X-\mu) + (\mu -a)\big)^2\bigr]\\ &= E\bigr[(X-\mu)^2 + (\mu -a)^2 &...


1

It depends on the unit your weights are associated with. From what I can understand from the question, your weight vector may be associated in two ways: Weights are associated with y-axis (i.e. rate):   In this case you need to multiply the individual weights with the x-length of your individual rectangles, and your average becomes (9*35.2*0.21) + (...


1

$x_i y_i$ is a size of the rectangle $K_i$, just calculate weighted average of those.


2

The benefit of distributional assumptions is that you can do additional inference rather than just prediction. Using the mean as a prediction of some population level parameter is often good, the mean has lots of very desirable properties and is well studied. That being said, if the underlying distribution is long tailed and you choose to ignore that, then ...


2

I would try a PCA. As you should know that a PC is a linear combination of one or more original features. Once I have fitted the model on PCs and got the feature importances/coefficients, I would use it to proportionately allocate among the individual members of the PCs.


0

A common task in statistics is to perform parameter estimation. The parameter estimator is a function of the sample. For example, consider the following two parameter estimator for the mean of normal distribution: $\hat{\mu}_1=\frac{X_1+X_2}{2}$ $\hat{\mu}_2=X_1$ We have $Var(\hat{\mu}_1)=\frac{\sigma^2}{2}$ but $Var(\hat{\mu}_2)=\sigma^2$. Even though ...


22

The expression is $\mathbb E[(X-a)^2]$. We'll differentiate and equate the expression to $0$: $$\begin{align}\frac{d}{da}\mathbb E[(X-a)^2]&=\mathbb E\left[\frac{d}{da}(X-a)^2\right]\\&=\mathbb E[-2(X-a)]\\&=0\end{align}$$ Then, $\mathbb E[-2X +2a]=0\rightarrow \mathbb E[2X]=\mathbb E[2a]=2a\rightarrow a=\mathbb E[X]$. The second derivative is ...


6

Assuming that you have $n$ values $\{x_1, x_2, \ldots, x_n\}$, the mean squared difference from each value $x_i$ to some number $a$ is: $$ m(a)=\frac{1}{n}\sum_{i=1}^{n}(x_i-a)^2 $$ We could ignore the term $\frac{1}{n}$ but I'm leaving it in. Expand the square and manipulate: $$ m(a)=\frac{1}{n}\left(\sum_{i=1}^{n}x_{i}^{2}-2ax_i+a^2\right)=\frac{1}{n}\left(...


3

It is possible to assume that the population from which data were randomly sampled has a probability distribution with a finite mean $\mu$ and a finite variance $\sigma^2$ without assuming that the distribution has a PDF (density or PMF) of a particular form. Here are some examples in which the form of the population distribution is known. The particulars ...


2

I can't follow your working after $\int_0^1\frac{\left(\frac{\partial f(x;\theta)}{\partial \theta}\right)^2}{f(x;\theta)}d x$, note that this expression is nonnegative as the function being integrated is nonnegative. Let $u = 1 + \theta \log x, \frac{du}{dx}=\frac{\theta}x$, \begin{align} &\int_0^1\frac{\left(\frac{\partial f(x;\theta)}{\partial \theta}\...


3

Seven years of data is quite a lot, so you should be able to detect any trend and seasonality. Then again, five years horizon is also a lot, so note that the long term forecasts will be rather dubious. I would recommend you use a simple and well understood method, like Exponential Smoothing, or possibly ARIMA. Both are implemented in the forecast and fable ...


0

The paragraph adresses the problem of multiple testing, and discusses why we cannot just do $p = 100$ individual hypothesis tests with $H_0: \beta_k = 0$ for each individual predictor at the significance level $\alpha = 0.05$. Even if the null hypothesis is true there is still a chance that the specific $\hat{\beta}_k$ calculated from your sample leads to a ...


2

There are a couple of issues that can be addressed here. First, In general, the expected value of the product of two random variables does not need to be equal to the product of their expectation. This means that $$ E(XY) \neq E(X)E(Y) $$ The equality only holds when the random variables are independent. But... This links directly to the second issue. Second ...


0

Without loss of generality, assume $\mathbb{E}[X] = \mathbb{E}[Y] = 0$ and $\text{Var}(X) = \text{Var}(Y) = 1$. Then $$r_{Y}(X) = \rho_{X, Y} \cdot X$$ and $$r_{X}(Y) = \rho_{X, Y} \cdot Y\text{.}$$ Hence, as long as $\rho_{X, Y} \neq 1$, $$r_{Y}(r_{X}(Y)) = r_{Y}(\rho_{X, Y} Y) = \rho_{r_{X}(Y), Y}\rho_{X, Y}Y $$ Now $$\rho_{r_X(Y), Y} = \rho_{\rho_{X, Y}Y,...


4

In many cases there's a relatively simple, mindless test you can apply. Recall that an Exponential family of distributions has densities of the form $$f(x,\theta) = \exp(\eta(\theta)T(x) + A(\theta)+B(x)).$$ Suppose there is a region of values of $(x,\theta)$ in which $\eta,$ $T,$ $A,$ and $B$ are differentiable. Applying the logarithm and taking ...


0

You can use kernel density estimation (KDE) to get a smoothed function of the histogram of the data (pdf). Here is a link to a working paper I published a while ago that used KDE for the acceptance/rejection method. You could also fit a cubic spline on the histogram bin values to get an approximation of the pdf function. Below is a figure from that working ...


0

Can't comment so forgive my brevity but R^2 is represented as 1 - SSE (sum of squared errors) where the residual is defined by equation you used to represent Z. With regards to the variance of the residuals the assumption for OLS is simply constant variance and mean zero.


0

You don't want a proof showing the MLE is never consistent with an infinite parameter space, because that's not true. There are many settings with countably infinite parameter spaces that have consistent MLEs. There are even many settings with uncountable parameter spaces that have consistent MLEs -- the usual $N(\mu,\sigma^2)$ model with real $\mu$ and ...


0

I think the easiest way is just index chasing, not anything elegant. I'll assume the $u$ are all mean zero; if not, the identity does not hold, and the left-hand side isn't the covariance matrix. A. $E[u_{it}u_{js}]=\sigma_{ij}$ if $t=s$ and zero otherwise. B. Now write $S$ for the matrix on the right-hand side, and $s_{itjs}$ for the entry corresponding to $...


1

One of the most popular ways to learn a (deterministic) function is Gaussian Process (GP) regression. It is commonly phrased in the Bayesian framework so that our 'prior beliefs' are $$ f(\cdot) \sim GP(m(\cdot), C(\cdot, \cdot))$$ where $m(\cdot)$ represents our prior expected value of $f(x)$ for any $x$. It's usually a 'rough and ready' approximation. E.g. ...


0

The easiest way to think about this is as a chain of two Normal random variables. You've said your sensor has a known error variance (or at least estimated from a previous experiment) and known no bias (implied it's symmetric about zero, I'll show you how to incorporate non-zero bias either way though). So you have basically defined the distribution of ...


0

Using physics notation, when adding or subtracting measurements $x,\ldots,w$ which have uncertainties $\delta x, \ldots, \delta w$, and the values are used to compute $q=x + \cdots + z - (u + \cdots + w)$, then the uncertainty in $q$ is $\delta q = \sqrt{(\delta x)^2 + \cdots + (\delta z)^2 + (\delta u)^2 + \cdots + (\delta w)^2}$. In statistical notation ...


5

Revisiting the definition of identification. Although your definition of consistency is fine, I think you're defining identification in a somewhat odd way, especially with regards to the usage of "if we have enough data." Although we can loosely think about identification as having "infinite data," I'd suggest you instead consider a ...


1

How do you want to treat negative individual values? Your constraints also require all pairs to add to values greater than $0$ as well as less than $1$, which suggests to me that you have some issues with negative weights. You could try something like $$\mathrm{scaled}(X_i) = \frac{X_i-\min\{X_j,0\}}{\sum\limits_k\left(X_k-\min\{X_j,0\}\right)}$$ on some ...


0

You don't have to adjust residual error by volatility, since you can remove cluster volatility in the log price returns using a GARCH model before running any kind of regression. Below is a plot of volatility over time, $\sigma(t)$, as well as the SP500 log price returns with cluster volatility removed (red line). Be sure to always use the log price ...


0

I might have figured it out. Since $Z_i(t)$ and $Z_j'(t)$ are zero-mean, $E(Z_i(t)Z_j'(t))=\text{Cov}(Z_i(t),Z_j'(t))$. For $(i,j)\in A$, this quantity is non-negative. Thinking of covariance as an inner product, we see that $Z_i(t)$ can be written as a non-negative multiple of $Z_j'(t)$ plus an orthogonal component $W_{ij}$. In addition, for Gaussian ...


1

It might help to read the introductory chapter(s) of e.g. MIT's Deep Learning Book. They explain the mathematical background of deep learning briefly and, more importantly, introduce the reader to the style of notation commonly found in papers on deep learning. The three notations you mentioned: $x \sim G(z, \,c)$: $x$ is a random variable distributed as ...


1

Not a complete answer, but I hope it steers you towards it. I'll show how it's possible to keep the matrices after the first derivative: $$ \frac{\partial}{\partial\rho}\ln f_{\rho}(\boldsymbol x)= -\frac12\frac{\partial}{\partial\rho}\ln|\Sigma| - \frac12 \frac{\partial}{\partial\rho}\left(\boldsymbol x'\Sigma^{-1}\boldsymbol x\right) = -\frac12\text{tr}((\...


0

A $2 \times 5$ contingency (multiway) table analysis, based on a chi-squared test. It's essentially a test of proportions of the frequencies (count data) to determine if anything "tracks" between the two factors, before/after and grade. Contingency table analysis can be complex, however, wherein different hypotheses can be employed surrounding ...


1

The first thing to try is always Chebyshev's inequality -- especially with well-behaved distributions where moments are likely to be enough to decide the question. $$\mathrm{var}[S_p]=\frac{1}{p^2}\sum_{i,j=1}^p \mathrm{cov}[a_iU_i^2,a_jU_j^2]$$ Now, correlations are bounded by 1,so: $$\left|\mathrm{cov}[a_iU_i^2,a_jU_j^2]\right|\leq \sqrt{\mathrm{var}[...


0

The image below from an answer to another question here might help. Why does regularization wreck orthogonality of predictions and residuals in linear regression? The OLS solution is an orthogonal projection of the observations into a subspace defined by the model. The ridge regression solution is also a sort of orthogonal projection, but now the subspace ...


0

I think Q1 is false and a "simple" simulation can confirm (see below). However, there is probably an equation that can relate $\rho(t_1, t_2)$ to $\rho(x_1, x_2)$. For Q2, I think, it is basically related to the residual error in $y$. The way I see it is basically to take the "complete" linear model: $y=\beta_1x_1+\beta_2x_2+\epsilon _y$,...


0

I am not an expert in bayesian non-parametrics but I think I can help a little. Others please correct me If i am wrong. Based on your model, you would have $\boldsymbol{\pi}, \mu, \Sigma, \boldsymbol{Z}$ as your latent variables and $\boldsymbol{X}$ as your observables. I use bold symbols to represent vectors. The objective is to find the posterior ...


2

You're right: this is not (quite) a one-liner. But I can do a thorough job in just two lines by building on what you already know. Let's make some standard preliminary observations to simplify the work and establish the notation. Your $t_\nu$ is a location-scale family based on the Student $t$ distribution with $\nu$ degrees of freedom with a location ...


6

No $x$ can come out of the integral: $$\begin{align}\mathbb E[X]&=\int_0^\infty \underbrace{x}_u \underbrace{\lambda e^{-\lambda x}dx}_{dv}\rightarrow du=dx, v=-e^{-\lambda x}\\&=\left[xe^{-\lambda x}\right]_0^\infty-\int_0^\infty (-e^{-\lambda x})dx\\&=0 -\left[\frac{e^{-\lambda x}}{\lambda}\right]_0^\infty\\&=\frac{1}{\lambda}\end{align}$$ ...


1

Do you mean to ask if you can solve exactly for the third column? Unless the number of rows is very small, or the covariance matrix $\Sigma$ is singular with a very specific structure, that is not possible. You might be able to estimate the conditional distribution of the last column given the others, and simulate from that.


1

The heuristic answer is that there are $IJ$ numbers in the table, but since they add to the sample size, only $IJ-1$ degrees of freedom. The test estimates the row and column margins in order to compute expected proportions, and this uses up $I-1$ and $J-1$ degrees of freedom respectively, so there are $(IJ-1)-(I-1)-(J-1)$ left. This is actually kind of a ...


0

There isn't much to it actually. You have $$Y=\beta X+\epsilon$$ Where the columns of $X$ are given by: $$X_i=\left[\begin{matrix}1 \\ A_i \\ B_i\end{matrix}\right], A_i \in \{0,1\},B_i \in \{0,1\}$$ $\beta$ can be obtained just like in OLS: $$\beta = YX^T(XX^T)^{-1}$$ We have that: $$XX^T= \left[\begin{matrix} \mathbf 1\\A\\B\end{matrix}\right] \left[\begin{...


1

For two multivariate normal vectors $\mathbf{x}\sim\mathcal N(\mathbf{\mu_x},\Sigma_{\mathbf{x,x}})$ and $\mathbf{Y}\sim\mathcal N(\mathbf{\mu_Y},\Sigma_{\mathbf{Y,Y}})$ we have that, given their covariance $\Sigma_{\mathbf{x,Y}}$: $$E[\mathbf x|\mathbf Y]=\mathbf{\mu_x}+\Sigma_{\mathbf{x,Y}}\Sigma_{\mathbf{Y,Y}}^{-1}(\mathbf{Y-\mu_Y})$$ $$\text{Var}[\mathbf ...


2

See the 2019 preprint Machine Learning meets Number Theory: The Data Science of Birch-Swinnerton-Dyer by Alessandretti, Baronchelli & He. Here is the Abstract: Empirical analysis is often the first step towards the birth of a conjecture. This is the case of the Birch-Swinnerton-Dyer (BSD) Conjecture describing the rational points on an elliptic curve, ...


2

To begin with required structure, suppose I have normal data in vector x with summary statistics as shown: summary(x); length(x); sd(x) Min. 1st Qu. Median Mean 3rd Qu. Max. -2.6342 -1.0172 -0.6990 -0.6650 0.2648 0.4705 [1] 20 # sample size [1] 0.9254546 # sample SD A t test of $H_0: \mu=0$ against $H_a: \mu < 0$ will reject ...


0

Minitab statistical software will accept summarized data, such as you provide. Assuming that data are sufficiently nearly normal and admission groups have nearly equal variances, the following output from Minitab for Welch 2-sample, 2-sided t test might be useful: Two-Sample T-Test and CI Sample N Mean StDev SE Mean 1 193 37.2 35.6 2....


7

The model is a mixture of Bernoullis, with likelihood $$L(p)=\prod_{t=1}^n \{pa_t+(1-p)b_t\}$$ a polynomial of degree $n$ in $p$. Since this distribution is not an exponential family, there is no sufficient statistic of fixed dimension and hence no way to update the maximum likelihood estimator in the way you describe. As an aside, the Bayesian estimation ...


Top 50 recent answers are included