37
votes
Accepted
What are the practical uses of Neural ODEs?
TL;DR: For time series and density modeling, neural ODEs offer some benefits that we don't know how to get otherwise. For plain supervised learning, there are potential computational benefits, but ...
- 594
26
votes
Accepted
Fitting SIR model with 2019-nCoV data doesn't conververge
There are several points that you can improve in the code
Wrong boundary conditions
Your model is fixed to I=1 for time zero. You can either changes this point to the observed value or add a ...
- 86.5k
11
votes
Accepted
Where does the logistic function come from?
I read from Strogatz's book that it was originated from modeling the human populations by Verhulst in 1838. Assume the population size is $N(t)$, then the per capita growth rate is $\dot N(t)/N(t)$. ...
- 1,928
9
votes
Accepted
Differentiation of Cross Entropy
Your $\frac{\partial E}{\partial o_j}$ is correct, but $\frac{\partial E}{\partial z_j}$ should be
$$\frac{\partial E}{\partial z_j}=\sum_i\frac{\partial E}{\partial o_i}\frac{\partial o_i}{\partial ...
- 16.8k
9
votes
Accepted
How does Hamiltonian Monte Carlo work?
Before answering the question about an intuitive way to think about Hamiltonian Monte Carlo, it's probably best to get a really firm grasp on regular MCMC. Let's set aside the satellite metaphor for ...
- 1,173
9
votes
Accepted
How can we efficiently find the fourth moment of a Poisson distribution?
Get the raw moments via the factorial moments
As with many other discrete distributions with simple factorial moments, obtaining the high-order raw moments is simplest when done through the factorial ...
- 133k
8
votes
What are the pros and cons to fit data with simple polynomial regression vs. complicated ODE model?
Just extend time a little bit, we can see how terrible is the polynomial fit:
...
- 37.3k
7
votes
Where does the logistic function come from?
I don't know about its history, but logistic function has a property which makes it attractive for machine learning and logistic regression:
If you have two normally distributed classes with equal ...
- 9,678
6
votes
Fitting SIR model with 2019-nCoV data doesn't conververge
You might be experiencing numerical issues due to the very large population size $N$, which will force the estimate of $\beta$ to be very close to zero. You could re-parameterise the model as
\begin{...
- 4,027
6
votes
What does it mean L1 loss is not differentiable?
I understand that derivative not exist at x=0, but what practical problems can arise from this fact?
$$ L = |x*a - y|; $$
$$ \frac{\partial L}{\partial a} = \dfrac{x\left(xa-y\right)}{\left|xa-y\...
- 3,049
6
votes
What does it mean L1 loss is not differentiable?
$L_1$ loss uses the absolute value of the difference between the predicted and the actual value to measure the loss (or the error) made by the model. The absolute value (or the modulus function), i.e. ...
- 1,599
6
votes
Accepted
How can I differentiate the equation with respect to $\theta$?
Let's assume:
$a=\alpha_H+\alpha-1$
and
$b=\alpha_T+\beta-1$
Therefore,
$\log (\theta^{ a_H+\alpha-1}(1-\theta)^{a_T+\beta-1})=\log\theta^a(1-\theta)^b=\log\theta^a+\log(1-\theta)^b={a\log\theta+...
- 3,524
6
votes
Gaussian process with Matérn kernel on a finite domain with periodic boundary conditions
Indeed it is a fractional SPDE, in particular $u$ is a Matern distributed if it solves:
$$
\tau(\kappa^2-\Delta)^\frac{\alpha}{2}u = \mathcal{W}
$$
where $\mathcal{W}$ is Gaussian white noise, $\alpha=...
- 5,670
6
votes
How can we efficiently find the fourth moment of a Poisson distribution?
I don't think there's any genuinely easy way. The factorial moments are straightforward, as are the cumulants, but then it's messy combinatorial stuff to recover the moments (raw moments or central ...
- 46.8k
5
votes
Does this interpretation $\phi'(x)=-x\phi(x)$ of the normal distribution have any significance?
I like to think of it in a similar way but with slightly different differential equations. (edit: below I managed to make it also intuitive for $\phi'(x) = -x \phi(x)$)
Case: heat equation
$$\frac{d}{...
- 86.5k
5
votes
Accepted
solve a differential equation
Assuming that $F(x)$ and $f(x)$ are differentiable everywhere, this would lead to:
$$f(x)=-f(x)-xf'(x) \Rightarrow f'(x)/f(x) = -2/x.$$
This is separable, and yields solution $f(x)=cx^{-2}$. You now ...
- 14.1k
5
votes
Fitting flexible spline using ODEs
I worked a bit on your question. I could see only the piece of code you shared (no data or extra detail on your problem). I have following "comments" about your code
The way you specify the ...
- 1,171
5
votes
Accepted
Is my understanding of neural ODE correct?
I think you're close.
The neural net is not the solution to the differential equation $\dot{x} = F(x(t))$, but rather the function governing the dynamics. That is to say, the neural net is $F$, not $...
- 39.6k
4
votes
What does it mean L1 loss is not differentiable?
+1 to both Tomasz and Alexey posts.
I would add that a good surrogate for the $L_1$ loss is the Pseudo-Huber loss function: $ L_{\delta }(x) =$ $\delta ^{2}\left({\sqrt {1+(x/\delta )^{2}}}-1\right)$ ...
- 46k
4
votes
Fitting SIR model with 2019-nCoV data doesn't conververge
Because the population of china is so huge, the parameters will be very small.
Since we are in the early days of the infection, and because N is so big, then $S(t)I(t)/N \ll 1$. It could me more ...
- 39.6k
4
votes
What are the pros and cons to fit data with simple polynomial regression vs. complicated ODE model?
I actually wondered the reason of not choosing mechanistic modeling if it models the data well. I would always favor ODE if it is feasible for a known system and good observations.
The primary goal ...
- 1,928
4
votes
Accepted
On solving ode/pde with Neural Networks
The procedure presented in the paper seems to be slightly different from the one above. In the paper the authors make an ansatz that explicitely fulfills the initial conditions. For a second order ...
- 278
3
votes
Does this interpretation $\phi'(x)=-x\phi(x)$ of the normal distribution have any significance?
That differential equation is how Gauss arrived at the normal distribution in 1809.
Gauss wanted to rationalize the choice of the average as an estimator of a location parameter.
He imposed the ...
- 7,110
3
votes
Accepted
Are the parameters $\beta$ and $\gamma$ in (Susceptible, Infected, Recovered) SIR model probability number? Can they larger than 1.0?
The parameters $\beta$ and $\gamma$ of the standard SIR model in the blog post
\begin{align}
{\mathrm d S \over \mathrm d t} &= -\beta {S I }\\[1.5ex]
{\mathrm d I \over \mathrm d t} &= \beta ...
- 4,027
3
votes
Accepted
Exponential distribution as a differential equation
The differential equation $F'(t) = (1-F(t))p$ has general solution $F(t) = 1 + Ce^{-pt}$. Now, since $F$ is a nonnegative distribution, $F(0)=0$, and hence $C=-1$. So the distribution is $F(t) = 1-e^{-...
- 659
3
votes
Why are mixed effect methods more effective when data are limited
A couple of points:
Standard mixed models also minimize an objective function, namely the negative of the log-likelihood function. Hence, you’re making a parametric assumption for the distribution of ...
- 21.5k
3
votes
Why are most epidemic models continuous-time?
This is a really interesting question and I doubt that there is any one 'correct' answer to it, but here are my thoughts on the reasons, which can be split into three categories.
History
I suggest you ...
- 4,027
2
votes
Accepted
Derivation Harvey (1984) Logistic Curve
Let
$$
\begin{align*}
f(t) = \frac{\alpha}{1 + \beta e^{\gamma t}}
\end{align*}.
$$
Then
$$
f'(t) = -\alpha (1 + \beta e^{\gamma t})^{-2} \cdot \beta e^{\gamma t} \cdot \gamma
$$
$$
= - \frac{\alpha }...
- 20.8k
2
votes
How to numerically solve a matrix differential equation in R?
I found this question after a couple of months, so I assume a solution was already found. Nevertheless, here an example, just for the record:
...
- 121
2
votes
Are the parameters $\beta$ and $\gamma$ in (Susceptible, Infected, Recovered) SIR model probability number? Can they larger than 1.0?
About the blog
The linked blog uses the equation
$$I^\prime = \beta SI - \gamma I$$
Instead of
$$I^\prime = \beta \frac{S}{N} I - \gamma I$$
That is why their results are so strange. Their values are ...
- 86.5k
Only top scored, non community-wiki answers of a minimum length are eligible