15

This type of model is actually much more common in certain branches of science (e.g. physics) and engineering than "normal" linear regression. So, in physics tools like ROOT, doing this type of fit is trivial, while linear regression is not natively implemented! Physicists tend to call this just a "fit" or a chi-square minimizing fit. The normal linear ...


14

Here's how I would express the cross-entropy loss: $$\mathcal{L}(X, Y) = -\frac{1}{n} \sum_{i=1}^n y^{(i)} \ln a(x^{(i)}) + \left(1 - y^{(i)}\right) \ln \left(1 - a(x^{(i)})\right) $$ Here, $X = \left\{x^{(1)},\dots,x^{(n)}\right\}$ is the set of input examples in the training dataset, and $Y=\left\{y^{(1)},\dots,y^{(n)} \right\}$ is the corresponding set ...


8

Estimating that error correctly will be tricky. But I would like to suggest it is more important first to find a better procedure to subtract background. Only once a good procedure is available would it be worthwhile analyzing the amount of error. In this case, using the mean is biased upwards by the contributions from the signal, which look large enough ...


7

You should simply treat your SE as SD, and use exactly the same error propagation formulas. Indeed, standard error of the mean is nothing else than standard deviation of your estimate of the mean, so the math does not change. In your particular case when you estimate SE of $C=A-B$ and you know $\sigma^2_A$, $\sigma^2_B$, $N_A$, and $N_B$, then $$\mathrm{SE}...


7

I take the freedom to answer the question in the title, how would I analyze this data. Given that we have replicates within the samples, mixed models immediately come to mind, which should estimate the variability within each individual and control for it. Hence I fit the model using lmer from lme4. However, as we are interested in p-values, I use mixed ...


6

We will assume that $A\sim\text{Pois}(\lambda)$ and $B\sim\text{Pois}(\mu)$ are independent Poisson variables. Then $$ EA=\text{var} A=\lambda\quad\text{and}\quad EB=\text{var} B=\mu. $$ Let's consider $A+B$. It's a standard property of the Poisson distribution that the sum of independent Poisson variables is again Poisson, with a parameter that is just the ...


6

Let's break this down into easier problems. To keep the post reasonably short, I will only sketch a good confidence interval procedure without going into all the details. What is interesting about this situation is that because $Y$ varies in such a complex, nonlinear fashion with the distribution parameters, a careful analysis and special solution are ...


5

In this type of situation one often assumes that the errors in the phasors are complex circular Gaussian distributed; this model comes up since one assumes that the observed errors are the accumulation of a larger number of smaller errors, and then using the central limit theorem. The key assumptions to watch for are the noise is purely additive and ...


5

What exactly are we summing over? The tutorial is actually pretty explicit: ... $n$ is the total number of items of training data, the sum is over all training inputs... The original single neuron cost function given in the tutorial (Eqn. 57) also has an $x$ subscript under the $\Sigma$ which is supposed to hint at this. For the single neuron case there'...


5

It sounds like you want to calculate a standard error for the unobserved count (i.e. counts of values without the error) in each bin. For each bin you can calculate the probability that a given observation ($x_i^\text{obs}$ with associated standard deviation $\sigma_i$) could have come from any given bin. So the number of observations actually in some ...


5

If I'm understanding your question properly, this sounds like you need Inverse variance weighting. https://en.wikipedia.org/wiki/Inverse-variance_weighting The estimate of your $x'$ that would minimize the variance (so giving you the "best guess") will be given by \begin{equation} \hat{x} = \frac{\Sigma_ix_i/\sigma^2_{x,i}}{\Sigma_i1/\sigma^2_{x,i}} \end{...


5

You did nothing wrong: the first value of $\delta s,$ which was obtained by an approximate method, is a close approximation to the second. Let's compare the two expressions by using Stirling's approximation $$\log\left(\Gamma(z)\right) \approx z \log(z) - z + \log(2\pi)/2 - \log(z)/2 + \frac{1}{12z} + O(z^{-2}).$$ and the Taylor series approximation $$\...


5

The principal objective of this reply is to point out how perilous this enterprise can be. Along the way I'll be able to suggest some approaches as well as provide some ideas for a different analysis. Whether any of this works will depend on the details of your circumstances. The key points to watch for are You need to get the model right. In particular,...


4

In the answers there (if I understood correctly) I learned that within-subject variance does not effect inferences made about group means and it is ok to simply take the averages of averages to calculate group mean, then calculate within-group variance and use that to perform significance tests. Let me develop this idea here. The model for the ...


4

For information, the random-effect model given by @Henrik: > f <- function(x) sqrt(x) > library(lme4) > ( fit1 <- lmer(f(Value) ~ Group + (1|Subject), data=dat) ) Linear mixed model fit by REML ['lmerMod'] Formula: f(Value) ~ Group + (1 | Subject) Data: dat REML criterion at convergence: 296.3579 Random effects: Groups Name ...


4

Apparently, there are many answers to this question: it has its own Wikipedia page and R package. The uncertainty range I described above is the "normal approximation interval": $\displaystyle p + z \sqrt{\frac{p(1 - p)}{M}}$ where $z$ is signed, $z=0$ is the central value ($p = N/M$), $z=1$ is "one sigma" (68% confidence level) above the central value and ...


3

Your analysis is correct. As a check, for example, try calculating $(50+2)^2=2704$, which gives a relative error of $(2704-2500)/2500=8.16\%$ for the output, approximately double that of the $4\%$ relative error of the input. One way of looking at it is like this: if we multiply two numbers together, forming a product $XY$, where $X$ has a relative error of ...


3

Yes, I believe your formula is correct. If $A$ is a random variable which takes positive values the geometric standard deviation of $A$ is by definition the exponential of the ordinary (arithmetic) standard deviation of $\log(A)$. Write $\sigma_g(A)$ for the geometric standard deviation and $\sigma(A)$ for the arithmatic standard deviation. Then: $$\log \...


3

Yes you are correct. For other readers, there are a number of clear summaries of how to propagate errors through a linear system. Here are three of the best ones I found: http://ipl.physics.harvard.edu/wp-uploads/2013/03/PS3_Error_Propagation_sp13.pdf http://www.itl.nist.gov/div898/handbook/mpc/section5/mpc552.htm http://lben.epfl.ch/files/content/sites/...


3

One approach would to model the distribution of the ratios, and the calculate the confidence interval based on this distribution. For example, suppose you repeat A and B many times, and you find that you can model them as independent normal variables $A$ and $B$ (whose parameters you find). The ratio will have a Cauchy distribution. Once you obtain this ...


3

First, $Cov(A,B)=0.5$. Second - you get different answers because you have calculated different things. In first case you calculating variance of $f$ as if it is a random variable. So if we wish to write our $f$ like $Mean\pm Var$ we wold get $0.861\pm 0.975$. In second case you know that $f$ is not random by itself but created from two random variables ...


3

Yes, the variance of the sum is the sum of the variances, in this case. By the law of the unconscious statistician, for $i\neq j$: \begin{align*} E\left[\frac{X_i}{Y_i}\frac{X_j}{Y_j} \right] &= \iiiint \frac{X_i}{Y_i}\frac{X_j}{Y_j} f(x_i)f(x_j)f(y_i)f(y_j)dx_i dx_j dy_i dy_j \\ &= E\left[\frac{X_i}{Y_i}\right] E\left[\frac{X_j}{Y_j} \right]. \end{...


3

To analyse the distribution properly you cannot start with the assumption that the result will be Gaussian. If $$X|\theta,\tau\sim\mathcal N(\theta,\tau^2)$$ and $$\theta\sim\mathcal N(\mu,\sigma^2_\theta)\qquad\tau\sim\mathcal N(\sigma,\omega^2_\tau)$$ then one can write $$X=\theta+\tau\xi=\mu+\sigma_\theta\epsilon_\theta+(\sigma+\omega_{\tau}\epsilon_\tau)\...


2

If you take the standard deviation of your two values (first way you suggested), this does not include the error within each measurement, only the error between the two methods (heating vs. cooling). That is why the first way shows surprisingly low standard deviation- it's only incorporating one of the sources of error. Therefore, the correct way is to ...


2

If you would be OK using R I think you could also use bbmle's mle2 function to optimize the least squares likelihood function and calculate 95% confidence intervals on the nonnegative nnls coefficients. Furthermore, you can take into account that your coefficients cannot go negative by optimizing the log of your coefficients, so that on a backtransformed ...


2

to my knowledge, there is no "default" way for comparing two distributions. It always depends on what you're looking for. Here are few simple suggestions. If I understand your problem correctly, normalizing your distributions will not be helpful - at least if the range/deviation of the values is an "information" in your data (see no. 1 below). If the data ...


2

$x$ is perfectly correlated with itself, so $x-x$ is $0$, with no uncertainty. The example given in the documentation is not really a very good one, since this isn't a common situation and the result is trivial. More importantly, if $x$ and $y$ are correlated, than the documentation is claiming that the code will take this correlation into account when ...


2

Let us assume that the activation function is a logistic regression denoted as $\sigma()$. The idea behind cross-entropy (CE) is to optimise the weights $W = [w_1, w_2,...,w_j,...w_k]$ to maximise the log probability - or to minimise the negative log probability. Here, you are willing to obtain each neuron's derivative of the cost $C^n$ with respect to ...


Only top voted, non community-wiki answers of a minimum length are eligible