Search Results

I want to solve the a standard unconstrained optimization problem: $$ \min_{x \in R^D} f(x) $$ and I want to solve it using RL (Reinforcement Learning). …

I was reading the paper MAML and they say: which we found led to roughly 33% speed-up in network computation. Which I thought was surprising because from my knowledge (https://www.quora.com/Wh …

I was trying to understand the different alternatives there are for halting batch gradient descent and concluding that one has reached some minimum (local or global). In the presence of noisy data (si …

I was trying to understand the sources of having many (equivalent) solutions $w$ with respect to the training set in the context of logistic regression (with a predictor of the form $h_{w}(x) = \text{ …

In my quest to find such an equation I thought I found something promising in optimization lecture by Cambridge University where they had: $$ w^{(t+1)}_i = w^{(t)}_i - \frac{\eta}{\sqrt{\sum^t_{\tau = …

Yes it's coordinate wise. But the trick is that it can be made into a vectorized for as shown here in it's pytorch implementation (https://github.com/markdtw/meta-learning-lstm-pytorch/blob/master/met …

How can I set up the problem so the optimization via (S)GD fits the data perfectly? …

I was reading Optimization as a model for few shot learning and Learning to learn by gradient descent by gradient descent as I noticed both papers use something they call coordinate wise optimzers …

I want to let the gradient be a constant, say $3$ and then regress on the offset. Its obvious that one can do GD (or SGD) on something like the L2 loss of this. But this seems such an easy problem tha …

In particular this is more or less what I conjecture the optimization problem is and its solution (but wasn't sure if I could derive it). … So we would consider the variational optimization problem (similar to the one on the paper): $$ H[f] = \sum^N_{i=1} \frac{1}{2} \| y_i - f(x_i) \|^2 + \lambda \| Pf \|^2$$ where $f(x_i), y_i \in R^m$ …

Recall we are trying to solve: $$ \text{minimize}_{x}\,\,\,\,\left\Vert Ax-y\right\Vert _{2}^{2}+\sum_{k}\lambda_{k}x^{T}R_{k}x+\alpha\left\Vert x\right\Vert _{1}\,\,\,\,\text{s.t. }x>0 $$ This prob …

Let $X$ be a $n \times d$ matrix with users as rows and movies as columns. Each user is a single row $x^{(u)} \in \mathbb{R}^d$ (i.e. for user u there are at most d ratings for the d movies). Also a …

I want to solve: $$ J_{R_K,L1}(x) = ||Ax - y ||^2 + \sum^K_{k} \lambda_k x^\top R_kx + \alpha \| x \|_1, x>0$$ of course, in the case where $R_1 = I$ we get non-negative Elastic Net regularization: …

I've take multiple machine learning classes and I am always told been told say when we do regularization on the training error $\mathcal E(W) = \frac{1}{n} \sum^n_{i=1}Loss(f(x_i),y_i)$: $$ \text{ (1 …

Consider a regularized optimization problem that we want to optimize via SGD: $$ H[w] = J(w; S_N) + \lambda R(w) = \frac{1}{N} \sum^{N}_{n = 1} J(w;x,y) + \lambda R(w) $$ My conjecture is that the correct …