53

This question has been partially discussed at this site as below, and opinions seem mixed. What is the difference between fixed effect, random effect and mixed effect models? What is the mathematical difference between random- and fixed-effects? Concepts behind fixed/random effects models All terms are generally related to longitudinal / panel / clustered /...


18

Densities can be hard to work with. Whenever you can, calculate with the total probabilities instead. Usually, histograms begin with point data, such as these 10,000 points: A general 2D histogram tessellates the domain of the two variables (here, the unit square) by a collection $P$ of non-overlapping polygons (usually rectangles or triangles). To each ...


14

A concise introduction is T. Schmidt 2008 - Copulas and dependent measurement. Also noteworthy is Embrechts 2009 - Copulas - A personal view. For Schmidt I could not provide a better summary than the section titles. It provides basic definitions, intuition and examples. Discussion of sampling is bare-bone, and a brief literature review covers the must-have....


11

The term "marginal" is very old. If you go back far enough in history, there were no scientific journals (evidently they started circa 1665). Instead, interim results were communicated via hand-written letters, and final results were written in books. There didn't tend to be much in the way of data graphics before Playfair, but books might often have ...


9

Yes, this is correct. Basically, you have $$f_{X,Y}(x,y) = f_{X|Y}(x|y) f_Y(y),$$ and as you said, you can sample from the joint density. Picking up just the $x$s from the samples leads you to a sample from the marginal distribution. This is because the act of ignoring the $y$ is akin to integrating over it. Lets understand this with an example. Suppose $...


8

If you treat $Z$ as non-random variable, then the marginal effect is $b_1 + b_3 \cdot Z$, a function of $Z$. The variance of a weighted sum is $Var(b_1) + Var(b_3) \cdot Z^2 + 2 \cdot Z \cdot Cov(b_1,b_3),$ and the standard error is just the square root of that. You can get the covariance from the right off-diagonal element of the variance-covariance matrix ...


8

As you correctly pointed out in your question $f_{Y}(y)$ is calculated by integrating the joint density, $f_{X,Y}(x,y)$ with respect to X. The critical part here is identifying the area on which you integrate. You have already clearly showed graphically the support of the joint distribution function $f_{X,Y}(x,y)$. So, now, you can note that the range of $X$ ...


8

The joint density uniquely determines the marginal densities: $$p(x) = \sum_y p(x,y) ~~\text{or}~~ p(x) = \int_{-\infty}^{\infty}p(x,y)\,\mathrm dy$$ (similarly for $p(y)$) and so the conditional densities are also determined uniquely by the joint density. So the answer is No, you cannot construct another group of samples as you desire. If the ...


8

Chris Genest has another introductory paper "Everything You Always Wanted to Know about Copula Modeling but Were Afraid to Ask".


8

How about annealed importance sampling? It has much lower variance than regular importance sampling. I've seen it called the "gold standard", and it's not much harder to implement than "normal" importance sampling. It's slower in the sense that you have to make a bunch of MCMC moves for each sample, but each sample tends to be very high-quality so you don'...


8

The standard way to do the calculation is to expand the argument of the $\exp$ term as a quadratic in $x$, complete the square, and get to the result that $f(y)$ is also a normal density. Or, you can use the fact that you know that $E[Y\mid X=x] = x$ and so $E[Y\mid X]$ is a random variable (it equals $X$) with mean $$E[Y] = E[E[Y\mid X]] = E[X] = \mu_x.$$...


8

One way to create a multimodal bivariate distribution out of unimodal marginals is with a mixture. The idea is that a mixture of Normal distributions can be unimodal, but bivariate versions of the same normals can have distinct peaks provided they are suitably correlated. As a concrete example, let $f$ be an equal mixture of two bivariate Normal ...


8

$P(S=s)$ and $P(R=r)$ both are marginal probabilities from the following table $$ \begin{array}{c|cc|c} & R=0 & R=1 \\ \hline S=0 & 0.20 & 0.08 & 0.28 \\ S=1 & 0.70 & 0.02 & 0.72 \\ \hline & 0.90 & 0.10 & \end{array} $$ Given such table, you can calculate conditional ...


7

If you seek the conditional density of $(X_1,...,X_{n-1})$ given $$S=\sum_{k=1}^n X_k$$ a change of variable from $$(X_1,...,X_{n})\sim\prod_{i=1}^n f(x_i)$$ to $$\left(X_1,...,X_{n-1},S\right)\sim\prod_{i=1}^{n-1}f(x_i)\times f(s-x_1-\cdots-x_{n-1})$$ [with Jacobian equal to 1] shows that this conditional density is proportional to$$f(x_1)\cdots f(x_{n-1})\,...


7

$$ \begin{array}{|c|cc|l|} \hline & 0 & 1 & \\ \hline 0 & 0.1 & 0.2 & 0.3 \\ 1 & 0.3 & 0.4 & 0.7 \\ \hline & 0.4 & 0.6 & 1 \\ \hline \end{array} $$ The table above means \begin{align} \Pr(X=0\ \&\ Y=0) = 0.1 & & & \Pr(X=0\ \&\ Y=1) = 0.2 \\ \Pr(X=1\ \&\ Y=0) = 0.3 & & & \Pr(...


7

Let's apply Bayes theorem: $$f(X_1 \vert X_1+X_2 = d) = \frac{f(X_1+X_2 = d \vert X_1)f(X_1)}{f(X_1+X_2 = d)} = cte \cdot f(X_2=d-X_1)f(X_1)$$ Substituting expressions for Weibull distributions: $$f(X_1 \vert X_1+X_2 = d) = cte \cdot \left(\frac{k}{\lambda}\right) \left(\frac{d-x_1}{\lambda}\right)^{k-1} e^{((d-x_1)/\lambda)^k}\left(\frac{k}{\lambda}\right) \...


6

I recommend this paper as a must read: Li, David X. "On default correlation: A copula function approach." The Journal of Fixed Income 9.4 (2000): 43-54. Here's the PDF. It explains what copula is and how it can be used in the financial application. It's a nice easy read. This should be followed by an article By Felix Salmon "Recipe for Disaster: The Formula ...


6

A good layperson introduction to copulas and its use in quantative fianance is http://archive.wired.com/techbiz/it/magazine/17-03/wp_quant?currentPage=all The concept of correlation of probabilities is illustrated by two elementary school students Alice and Britney. It also discusses how prices of credit default swaps are used as a shortcut to the ...


6

Following up on whuber's comment, $$\begin{align} \left.\left.\frac{1}{2(1-\rho^2)}\right(x^2+y^2-2\rho xy\right) &=\left.\left.\frac{1}{2(1-\rho^2)}\right(x^2-\rho^2x^2+(y^2-2\rho xy + \rho^2x^2)\right)\\ &= \frac{x^2}{2} + \frac{1}{2}\left(\frac{y-\rho x}{\sqrt{1-\rho^2}}\right)^2 \end{align}$$ and so $$\int_{-\infty}^\infty f_{X,Y}(x,y)\,\mathrm ...


6

Yes and yes, and examples are easy to devise. Let $(X,Y)$ have discrete distribution that has probability mass of $0.12$ at $(0,0)$ and $0.08$ at the $4$ points $(\pm 1,\pm 1)$. probability mass of $0.07$ at each of $(\pm 2, \pm 2)$ and $(\pm 3, \pm 3)$. This is a unimodal bivariate distribution with a mode of $0.12$ at $(0,0)$ However, the marginal ...


6

The equation follows from the definition of marginal distribution: $$ p(y) = \int_\theta{p(y, \theta)} $$ And, from factoring the joint probability of data and parameters into conditional probabilities, like so: $$p(y, \theta) = p(y|\theta)p(\theta)$$ (If this is confusing, divide both sides by $p(\theta)$ to get the familiar definition of conditional ...


6

For a continuous variable, the marginal effect of $x_k$ in a logit is $$\Lambda(X'\beta)\cdot \left[1-\Lambda(X'\beta)\right]\cdot\beta_k,$$ where $$\Lambda(z)=\frac{\exp{z}}{1+\exp{z}}.$$ By default, Stata actually calculates the average of this over the estimation sample, but I will use the mean value of x in what follows (marginal effect at the mean ...


6

By definition of conditional probability* we have that: $$P(E=e|A=a)=\frac{P(E=e,A=a)}{P(A=a)}=\frac{\sum_{c}P(E=e,C=c,A=a)}{P(A=a)}$$ In the last step I used marginalization over $c$. Then, again using the definition of conditional probability, this is equal to: $$\sum_{c}P(E=e,C=c|A=a)$$. *Definition of conditional probability: $$P(x_1,...,x_n|y_1,...,...


6

Your question is unclear: when simulating $X\sim f(x)$, one can instead simulate$$(X,U)\sim\mathcal{U}(\{(x,u);\ 0<u<f(x)\})$$which has the joint density$$\mathbb{I}_{(0,f(x)}(u)\mathbb{I}_\mathcal{X}(x)$$and then use only the $X$ component in the simulation. For instance, here is a figure from our book depicting many realisations of such a Uniform ...


6

We want to solve $$ \int \mathcal N(y | Wx, \beta^{-1} I) \mathcal N(x | 0, I) dx $$ $$ = \frac{\beta^{D/2}}{(2\pi)^{D/2}} \cdot \frac{1}{(2\pi)^{q/2}}\int \exp \left(-\frac \beta 2 || y - Wx ||^2 - \frac 12 x^T x\right) dx $$ $$ \propto e^{-\frac \beta 2 y^T y}\int \exp \left(-\frac 12 \left[x^T(\beta W^T W + I)x - 2 \beta y^T W x\right]\right) dx. $$ Let ...


5

In a probit model, $\Pr(y_i=1 \vert x_i,z_i,t_i)=\Phi(\alpha +\beta x_i+\gamma z_i + \psi t_i),$ where $\Phi()$ is the standard normal cdf. The marginal effect is the derivative of that function (using the chain rule): \begin{equation} \frac{\partial \Pr(y_i=1 \vert x_i,z_i,t_i)}{\partial x}=\varphi(\alpha +\beta x_i+\gamma z_i + \psi t_i)\cdot\beta, \end{...


5

Have a look at PyMC (documentation is available as pdf and html) See also this section on using arbitrary factors.


5

About your comment: "How does the law of total expectation apply in a context where there are no conditionals?". You don't see the conditional distribution because a more precise statement of the problem is $$Y\mid P=p\sim \textrm{Geo}(p) \qquad \textrm{and} \qquad P\sim \textrm{Be}(2,1) \, .$$ Now that you see the conditional distribution, just do the ...


5

Draw pictures of the regions of integration. The region where $0 \le x_1 \le 1, 0 \le x_2 \le 1,$ and $x_1 x_2 \le y$ (for $0 \le y \le 1$) looks like the shaded part of The colors denote the varying values of the density $f(x_1,x_2)$, ranging from blue (low) to red (high). The integral of $f(x_1,x_2)dx_1 dx_2 = 4 x_1 x_2 dx_1 dx_2$ is readily found by ...


5

If $X$ and $Y$ are random variables taking on values $x_1, x_2, \ldots, x_m,$ and $y_1, y_2, \ldots, y_n,$ respectively, then the joint probabilities $P\{X = x_i, Y = y_j\}$ (meaning the probability that $X$ has value $x_i$ and simultaneously $Y$ has value $y_j$) can be displayed as an array (a matrix, if you like) with $m$ columns and $n$ rows: $$\...


Only top voted, non community-wiki answers of a minimum length are eligible