New answers tagged optimization
0
Let there be $k_i$ balls of color $i$ where $i$ ranges from $1$ to $m=10,$ so that the total number of balls is $k_1+\cdots+k_m=n=100,$ and let $p_i$ be the chance that any randomly chosen ball of color $i$ contains a winning ticket. You are invited to sample up to $s \le m$ balls (without replacement). How to maximize your chances of a winning ticket?
To ...
0
Have you tried lowering the learning rate of SGD to 0.001? It seems like the training loss decays too fast (due to a potentially high learning rate) and that potentially explains the divergence of the training and validation loss.
1
There is an interesting paper proposing to maximize not the observed likelihood, but the expected likelihood Expected Maximum Log Likelihood Estimation. In many examples this gives the same results as MLE, but in some examples where it is different, it as arguably better, or at least different in an interesting way.
One example: Take the usual multiple ...
0
The rationale behind using a bayesian framework is not only the Bayes update rule or the availability of (subjective)prior if any exists, but it is due to the availability marginalization and conditioning, which drive the entire modeling process in Bayesian framework. These principles originate due to the quantification of uncertainty in the Bayesian ...
2
I'm not sure I really understand your analogy, but it seems like your analogy doesn't fully capture either the objections or the philosophy of Bayesianism.
If I were going to criticize Bayesianism, I'd point to the fact that there are no guarantees on the results. Frequentists enjoy frequency properties of their estimators. They can say, at least in ...
2
Using vector math re-writing your problem, also replacing b with X:
$$f(X)=(X-Y)^T(X-Y)+\varphi |X|^T 1$$
First Order Conditions:
$$f(X)'=0$$
$$2(X-Y)+\varphi \space\mathrm{sign}(X)=0$$
Solution:
$$X=Y-\mathrm{sign}(X)\varphi/2$$
when $|Y_i|\ge\varphi/2$:
$$X=Y-\mathrm{sign}(Y)\varphi /2$$
when $|Y_i|<\varphi/2$:
$$X=0$$
0
You could just perform 5 independent regressions. If you use scikit-learn, you can do it in one step as LinearRegression supports multiple target variables, as documented
here.
0
It just means that as you increase your sample size (i.e. number of non-censored survival times available to you gets very large), the distribution you observe will not get closer and closer to the "true" distribution of survival times. That is, it will be biased, even as the variance is reduced by increasing your sample size.
1
As you mentioned we do not know what happens with the censored individuals. Let's say the event in question is death. In this case, censorship means we have not observed the patient die, therefore that information is unavailable. Censored patients may die in the near or far future, but the important thing is that we do not know when they will die (or whether ...
3
It is well known that even in 1D, N-R needn't converge at all and it can hop all over the place. (Numerical Recipes has a good discussion of this behavior, as I recall.)
In more than one dimension, consider a function with a global optimum that nevertheless has regions where the Hessian is non-definite. Start the algorithm with a point in one of those ...
2
Feature engineering: working with the available data in order to create/transform good predictors (your $X$). This can usually be done by transforming, averaging, combining, etc. the available columns of your database, in order to obtain the predictors that are the most meaningful (or just work better) for the problem at hand. This can also include feature ...
4
Feature engineering is about data and is the process of finding/creating features that might improve your model's performance. You sometimes engineer new features from raw data you have, use the existing ones and perform univariate/multivariate transformations. So, by engineering more features your linear regression model may for example become $y=w_1x_1+...
1
This is only marginally related to the detailed coding discussion, but seems highly relevant to the original question concerning modeling of the current 2019-nCoV epidemic. Please see arxiv:2002.00418v1 (paper at https://arxiv.org/pdf/2002.00418v1.pdf ) for a delayed diff equation system ~5 component model, with parameter estimation and predictions using ...
1
I just found that inside the algorithm in the "matrix form" the variable it's equal to Wdiag = deriv2(eta) so in this case this variable stays always the same.
So we have to change it to Wdiag = deriv2(eta1). So now both of these algorithms works fine.
There's another error anyway with this algorithm (this is also present in the original question, linked ...
1
\begin{aligned}
\min\limits_{\boldsymbol{b}} \boldsymbol{e}^T\boldsymbol{e} = (\boldsymbol{Y}-\boldsymbol{Xb})^T(\boldsymbol{Y}-\boldsymbol{Xb}) \\
\end{aligned}
FONC:
let $\boldsymbol{u} = \boldsymbol{Y}-\boldsymbol{Xb}$
\begin{aligned}
\frac{\partial \boldsymbol{e}^T\boldsymbol{e}}{\partial{\boldsymbol{b}}} &= 0 \\
\frac{\partial [\boldsymbol{Y}^T-\...
2
Because OLS belongs in a subfield of mathematical optimization called convex optimization, and that field has some nice properties, such as
every local minimum is a global minimum
0
There are many ways to do it.
The simple way would be the following:
You need to identify a cut-off point for which, when surpassed at least $\alpha$% of people
never come back to play. Never can be defined as a) a big time-frame e.g:3 years or b) Uninstall of the game. You fix the $\alpha$ (for this example I'll go for $\alpha=95$) and you then try to ...
2
I'd first take a look at the available data, in particular at the distribution of inactivity days before activity is resumed. Some players pause for a day or two and then become active again, some pause for a week etc. For active users, I'd expect a Poisson or an exponential distribution. Whatever the distribution, I'd try to find its parameters.
The next ...
1
The question is very vague, so I expect they want to see your view on it, how you would solve it. Without more information there are many ways in which you could do this.
One example would be using a logistic regression model, which would not only "classify" the players into churned or not churned, but would also give you the odds of a player doing so, ...
1
Lagrange multipliers are fine but you don't actually need that to get a decent intuitive picture of why eigenvectors maximize the variance (the projected lengths).
So we want to find the unit length $w$ such that $\|Aw\|$ is maximal, where $A$ is the centered data matrix and $\frac{A^TA}{n} = C$ is our covariance matrix.
Since squaring is monotonically ...
4
This is our objective function, composed of a loss function and a regularizer.
$$\mathcal O(w,x,y) = \mathcal L(w,x,y)+\mathcal R(w)$$
So $\mathcal R(w)=\|w\|_2^2=\sum{w_i^2}$ in the case of $\ell_2$ regularization.
Let's perform gradient-based minimization, i.e. we will update params based on the negative partial derivatives of the objective function.
...
2
L2 regularization adds $w_i^2$ term to the loss function. In iterative approaches using gradients, we subtract the gradient of the loss function not the magnitude of the weight itself. And in the loss function, the regularization part's derivative with respect to $w_i$ is going to be ${d\over dw_i}(w_i^2)=2w_i$. Typically, this part is multiplied with a ...
0
I checked them by SAS. If I have to say the answer to my question, only “logLik()” may be wrong but glm() is OK in R. Because SE is same all(glm, optim, SAS). But anyway, I think now "2.474444" would be answer, and I couldn't find where the value “-1.547104” comes from. I don't have confidence about this answer yet, if someone makes it, please give me some ...
1
This is true for any strictly increasing function, and it is quite trivial to prove using the definitions of strict monotonicity and the argmax. In fact, the result does not ]require either of the functions to be differentiable. If $f$ is any strictly increasing real function then, by definition, we have:
$$L > L'
\quad \quad \iff \quad \quad
f(L) >...
3
Additionnally to @forgottenscience answer, let consider the following (more intuitive, to my view) point:
First,
$x \rightarrow log(x)$ is a strictly increasing function
Let call $\theta^*$ the value maximizing $L$ and $\nu^*$ the value maximizing $\log(L)$
(i.e. i.e. $\theta^* =\arg\max_{\theta} L(\theta)$ and $\nu^* =\arg\max_{\nu} \log L(\nu)$).
Then,
...
2
I'll assume that the likelihood is differentiable and has a unique maximum. If $\theta^*$ is the argmax of $L(\theta)$,
$$\frac{\partial}{\partial \theta} L(\theta^*) = 0, $$
while it follows from the chain rule that
$$\frac{\partial}{\partial \theta} \log L(\theta) = \frac{\frac{\partial}{\partial \theta} L(\theta)}{L(\theta)}, $$
so in particular
$$ \frac{...
7
There are several points that you can improve in the code
Wrong boundary conditions
Your model is fixed to I=1 for time zero. You can either changes this point to the observed value or add a parameter in the model that shifts the time accordingly.
init <- c(S = N-1, I = 1, R = 0)
# should be
init <- c(S = N-Infected[1], I = Infected[1], R = 0)
...
3
Because the population of china is so huge, the parameters will be very small.
Since we are in the early days of the infection, and because N is so big, then $S(t)I(t)/N \ll 1$. It could me more reasonable to assume that at this stage of the infection, the number of infected people is approximately exponential, and fit a much simpler model.
3
You might be experiencing numerical issues due to the very large population size $N$, which will force the estimate of $\beta$ to be very close to zero. You could re-parameterise the model as
\begin{align}
{\mathrm d S \over \mathrm d t} &= -\beta {S I / N}\\[1.5ex]
{\mathrm d I \over \mathrm d t} &= \beta {S I / N} - \gamma I \\[1.5ex]
{\mathrm d R \...
answered Jan 28 at 12:35
S. Catterall Reinstate Monica
1,9481 gold badge8 silver badges13 bronze badges
2
This is often called algorithmic design or optimal design. The last name hints on some optimality criteria, and there are many to choose from! Much used is D-optimality. Some related posts here is Motivations for experiment design in statistical learning? and Is DoE applicable to collect data for machine learning model?, look at the links and references in ...
3
This thread may be useful to statisticians and data scientists because it shows how to construct arbitrarily "nasty" functions (that nevertheless are easy to handle mathematically and computationally) for testing algorithms that rely on optimization.
One way to construct functions with specified local properties (like local minima) is to assemble them from ...
1
This is just the result of a bit of tedious algebra. From the other definitions in the article, we have the following identities:
$$w^* = A^{-1}b$$
$$w^k = A^{-1}b + Qx^k$$
One identity they assume you know is that $Q^TQ = I$, which implies:
$$ Q^T A Q = Q^T Q \text{diag}[\lambda_1,\dots,\lambda_n]Q^T Q = \text{diag}[\lambda_1,\dots,\lambda_n] $$
The next ...
Top 50 recent answers are included
