24
votes
Accepted
What does interpolating the training set actually mean?
Your question already got two nice answers, but I feel that some more context is needed.
First, we are talking here about overparametrized models and the double descent phenomenon. By overparametrized ...
- 140k
17
votes
Why does linear regression use a cost function based on the vertical distance between the hypothesis and the input data point?
When you have noise in both the dependent variable (vertical errors) and the independent variable (horizontal errors), the least squares objective function can be modified to incorporate these ...
- 37.8k
14
votes
Accepted
Why is the arithmetic mean smaller than the distribution mean in a log-normal distribution?
The two estimators you are comparing are the method of moments estimator (1.) and the MLE (2.), see here. Both are consistent (so for large $N$, they are in a certain sense likely to be close to the ...
- 34.3k
14
votes
Confidence intervals around functions of estimated parameters
Usually we take normality assumption for linear regression models. That is, $y_i\sim N(\beta^Tx_i,\sigma^2)$. From this assumption we derive the asymptotic distribution of $\hat{\beta}$, which is also ...
- 4,017
13
votes
Accepted
Algorithms for weighted maximum likelihood parameter estimation
There are a number of ways to handle importance weights. Note that "weights" as a general term can be ambiguous. R's glm method, for instance, takes a weight parameter that is interpreted differently.
...
- 1,614
13
votes
What does interpolating the training set actually mean?
In layman's terms, an interpolator will literally 'join the dots'.
Here's a simple graphical summary of what interpolation can do and why it can be awful. I'd like to stress that interpolation does ...
- 2,927
12
votes
Understanding the Cullen and Frey plot
This plot used to be commonly called a Pearson plot (it also had several other names), though sometimes with skewness rather than its square being plotted. It was used long before Cullen and Frey ...
- 287k
12
votes
What I should do if no distribution fits my dataset?
If you have 26K data, any test on a given distribution will fail. Because for that much data, the testing can detect tiny difference and report it is not coming from that distribution.
I would ...
- 37.2k
12
votes
Accepted
Confidence intervals around functions of estimated parameters
Two common approaches for this problem are to calculate the non-linear combination of the coefficients directly from the regression or to bootstrap it.
The variance in the former is based on the "...
- 37.8k
12
votes
Accepted
What is the meaning of "loc" and "scale" for the distributions in scipy.stats?
Background
The SciPy distribution objects are, by default, the standardized version of a distribution. In practice, this means that some "special" location occurs at $x=0$, while something ...
- 21.3k
10
votes
Accepted
Discrete probability distribution with two 'tails'
There's an infinite number of such distributions -- one need merely a way to get a probability for each value of $i$ and one has just such a distribution. It's a simple matter to spend a leisurely ...
- 287k
10
votes
Why does linear regression use a cost function based on the vertical distance between the hypothesis and the input data point?
One reason is that $$\sum_{i=1}^N(y_i-h_\theta(x_i))^2$$ is relatively easy to compute and optimize, while the proposed cost $$\sum_{i=1}^N \min_{x,y}\big[(y_i-h_\theta(x))^2+(x_i-x)^2\big]$$ has a ...
- 468
10
votes
Accepted
Formal definition of the qqline used in a Q-Q plot
Sort of "both" - the line depends both on the observed quantiles (which define the y-axis of the QQ plot) and the expected/theoretical/reference quantiles (which the define the x-axis). The ...
- 45.1k
10
votes
Accepted
How can I fit a spline to data that contains values and 1st/2nd derivatives?
We will describe how a spline can be used through Kalman Filtering
(KF) techniques in relation with a State-Space Model (SSM). The fact
that some spline models can be represented by SSM and computed ...
- 5,636
10
votes
Accepted
How does logistic growth rate coincide with the slope of the line in the exponential phase of the growth?
Let's do the calculations to see what the answers are.
By changing the units of measurement of $x$ to the origin $x_0$ we may assume $x_0=0$ (to simplify the work and the notation) and--therefore--the ...
- 330k
10
votes
Confidence intervals around functions of estimated parameters
It seems like you are estimating the discriminant of a quadratic function, ie. your function is
$$y = \hat{\beta_0} + \hat{\beta_1} X_1 + \hat{\beta_2} X_2 =
\hat{\beta_0} + \hat{\beta_1} X + \hat{\...
- 82.8k
10
votes
Accepted
What would be a good model to fit to cumulative reputation on Stack Exchange?
This is a textbook case of a time series, so you could bring some well-developed machinery to bear.
The initial challenge is that you have an irregular series. There are far more tools available for ...
- 129k
9
votes
Accepted
Logistic regression: restrict the prediction range
Logistic regression is actually quite inflexible, since it is linear in the logit, a fact that is somewhat obscured by the fact that the plot on the original scale is nonlinear. In the present case, ...
- 129k
8
votes
How can I fit the parameters of a lognormal distribution knowing the sample mean and one certain quantile?
Let $\mu$ and $\sigma$ be parameters of the corresponding Normal distribution (its mean and standard deviation, respectively). Given the lognormal mean $m$ and the value $z$ for percentile $\alpha$, ...
- 330k
8
votes
Reliability of a fitted curve?
This is an ordinary least squares problem!
Defining
$$x = V^{-2/3}, \ w = V_0^{1/3},$$
the model can be rewritten
$$\mathbb{E}(E|V) = \beta_0 + \beta_1 x + \beta_2 x^2 + \beta_3 x^3$$
where the ...
- 330k
8
votes
Accepted
How can I convert a lognormal distribution into a normal distribution?
By definition, a random variable $Z$ has a Lognormal distribution when $\log Z$ has a Normal distribution. This means there are numbers $\sigma\gt 0$ and $\mu$ for which the density function of $X = (...
- 330k
8
votes
What is a good number of treedepth saturations for a fit stan model?
In No-U-Turn-Sampler a maximum tree depth of 10 is a sensible default, but occasionally you have to increase it. In my experience not usually by much. I might try 12 next and I have never had to go ...
- 34.2k
8
votes
What is a good number of treedepth saturations for a fit stan model?
I'll leave this as an "answer" as I don't have enough reputation to "comment" on this post.
This webpage might be of interest to you.
The development team describses here, although quite shortly, ...
- 663
8
votes
An easy decision when to use a spline or a polynomial
My RMS book and course notes go into detail about this. Briefly, polynomials are too restrictive, allow a point in one part of the curve to too greatly influence the fit in other parts of the curve, ...
- 96.4k
8
votes
How can I fit a spline to data that contains values and 1st/2nd derivatives?
You can do spectacularly well with a standard least-squares routine, provided you have a reasonable idea of the relative sizes of the random errors made for each derivative. There is no restriction ...
- 330k
8
votes
What are the pros and cons to fit data with simple polynomial regression vs. complicated ODE model?
Just extend time a little bit, we can see how terrible is the polynomial fit:
...
- 37.2k
8
votes
Model misfit with DHARMa - What needs/can be done?
Interesting problem! In addition to what Florian has suggested, here are my thoughts:
The mixed effects models you fitted may not be the best for teasing out the effect of person-level predictors (...
- 20.7k
7
votes
Accepted
How to fit the SIR and SEIR models to the epidemiological data?
I am going to confine my comments to the SEIR model - the issues for the SIR model are similar and it can be treated as a special limiting case of the SEIR model anyway (for large $\delta$).
What you'...
- 3,987
7
votes
How to choose and perform a goodness-of-fit test?
Great question! (Almost a professional statistician here!) Let's see if I can try to answer your question. Whuber gave a comment on what the typical strategy that someone might follow. I'll try to ...
7
votes
What does interpolating the training set actually mean?
Apart from literal meaning of interpolation, this is related to something called deep learning models totally memorize the training data. Hence, both ...
- 2,590
Only top scored, non community-wiki answers of a minimum length are eligible