Search Results

The issue I was having is since $e^{(i)}$ is a r.v in terms of $x^{(i)}$ and $y^{(i)}$. i.e. $$e^{(i)} = y^{(i)} - \theta^{T} x^{(i)}$$ Then if we have: $$p_{e}(e^{(i)}) = \frac{1}{\sqrt{2\pi\sigma …

Consider the case when we are trying to learn a regression function via gradient descent. … However, it seems less obvious how to reason about this in the case of regression and gradient descent. What are people's thoughts? …

I was reading Andrew Ng's CS229 lecture notes (page 12) about justifying squared loss risk as a means of estimating regressions parameters. Andres explains that we first need to assume that the target …

I also find Andrew Ng's notes confusing because there is a subtle point that isn't explained. What they say is that the noise $\epsilon$ has Gaussian distribution. This ends up being essential. If you …

check this for normalized eucledian similarity as a candidate measure: Definition of normalized Euclidean distance The answers already have great details!

I was trying to understand better when we can learn a unique parameter for linear regression and how much data is required to get one. …

Recall stochastic gradient descent (for regression): $\theta = 0 $ $ \text{Randomly select } t \in [1,n]\{\\ \quad \theta^{(k+1)} = \theta^{k} + \eta_{k}(y^{(t)} - \theta \cdot x^{(t)})x^{(t)}\\ … Also an answer that contains a proof, even if its for some special case, say "convex function" or in the case of linear regression or whatever the case is, but that provides some insight (or a proof) why …

To add a more modern (but not very deep) perspective, consider how it's used in deep learning (ha, pun intended...): logit is referred to the output of a function (e.g. a Neural Net) just before it's …

TLDR; I want to know the percentage % of explained variance of the dependent variable given a list of D independent variables with crazy different scales -- but I believe that given convexity of regression …

Recall we are trying to solve: $$ \text{minimize}_{x}\,\,\,\,\left\Vert Ax-y\right\Vert _{2}^{2}+ \lambda \| x \|^2_2 \,\,\,\,\text{s.t. }x>0 $$ is equivalent to: $$ \text{minimize}_{x}\,\,\,\,\lef …

I was training a singled layered (shallow) neural network as in: $$ f(x) = \sum^K_{k} c_k\theta(W_k x+b_k)$$ for regression (using squared error loss) or function approximation. …

I want to let the gradient be a constant, say $3$ and then regress on the offset. Its obvious that one can do GD (or SGD) on something like the L2 loss of this. But this seems such an easy problem tha …

I wanted to compare and potentially justify that one model is better than another on function approximation or (regression task). … However, its unclear to me what type of normalization would be sensible for function approximation since it would be nice to not "screw up" the regression task because of normalization. …

Recall that pseudo-inverse can be characterized as follows: Solve $$ \| w \|^2 $$ subject to: $$ Xw = y $$ thus it is plausible since its a constrained optimization problem that the solution gener …

I was interested in recovering the solution $x$ to a linear system underdetermined $N < D$: $$ Ax = y$$ as accurately as possible to the true $x$. Obviously, this system has infinite number of solut …