Search Results

TLDR; I want to know the percentage % of explained variance of the dependent variable given a list of D independent variables with crazy different scales -- but I believe that given convexity of regression …

check this for normalized eucledian similarity as a candidate measure: Definition of normalized Euclidean distance The answers already have great details!

To add a more modern (but not very deep) perspective, consider how it's used in deep learning (ha, pun intended...): logit is referred to the output of a function (e.g. a Neural Net) just before it's …

I want to let the gradient be a constant, say $3$ and then regress on the offset. Its obvious that one can do GD (or SGD) on something like the L2 loss of this. But this seems such an easy problem tha …

Recall that pseudo-inverse can be characterized as follows: Solve $$ \| w \|^2 $$ subject to: $$ Xw = y $$ thus it is plausible since its a constrained optimization problem that the solution gener …

Recall we are trying to solve: $$ \text{minimize}_{x}\,\,\,\,\left\Vert Ax-y\right\Vert _{2}^{2}+ \lambda \| x \|^2_2 \,\,\,\,\text{s.t. }x>0 $$ is equivalent to: $$ \text{minimize}_{x}\,\,\,\,\lef …

I was interested in recovering the solution $x$ to a linear system underdetermined $N < D$: $$ Ax = y$$ as accurately as possible to the true $x$. Obviously, this system has infinite number of solut …

I also find Andrew Ng's notes confusing because there is a subtle point that isn't explained. What they say is that the noise $\epsilon$ has Gaussian distribution. This ends up being essential. If you …

I was trying to learn a sine curve $$ f_{target}(x) = sin(2 \pi f_s x )$$ with $f_s = 4$, from 10 points, with linear regression and a polynomial of degree 9: $$f_{model}(x) = \langle w, \Phi(x) \rangle …

I've been trying to approximate a polynomial function in 1D $f_{target}(x) = \sum^{ D_{target} }_{d=1} c_d x^d$ with linear regression: $$ \Phi(x) w = y $$ where $\Phi(x)$ is the Vandermonde matrix ( … I would have expected that the only point the regression problem should really start to be zero $D_{model}+1 > D_{target}+1 = N$. …

I wanted to compare and potentially justify that one model is better than another on function approximation or (regression task). … However, its unclear to me what type of normalization would be sensible for function approximation since it would be nice to not "screw up" the regression task because of normalization. …

I was training a singled layered (shallow) neural network as in: $$ f(x) = \sum^K_{k} c_k\theta(W_k x+b_k)$$ for regression (using squared error loss) or function approximation. …

Consider the case when we are trying to learn a regression function via gradient descent. … However, it seems less obvious how to reason about this in the case of regression and gradient descent. What are people's thoughts? …

The issue I was having is since $e^{(i)}$ is a r.v in terms of $x^{(i)}$ and $y^{(i)}$. i.e. $$e^{(i)} = y^{(i)} - \theta^{T} x^{(i)}$$ Then if we have: $$p_{e}(e^{(i)}) = \frac{1}{\sqrt{2\pi\sigma …

I was reading Andrew Ng's CS229 lecture notes (page 12) about justifying squared loss risk as a means of estimating regressions parameters. Andres explains that we first need to assume that the target …