22 votes

How to find the mode of a probability density function?

This answer focuses entirely on mode estimation from a sample, with emphasis on one particular method. If there is any strong sense in which you already know the density, analytically or numerically, ...
user avatar
  • 49.2k
20 votes

How to measure the shift between two cumulative distribution functions (CDFs)?

The absolute value of this area is $$\int_{x=-\infty}^\infty \lvert F(x) - G(x)\rvert \,\mathrm{d}x,$$ which note – at least for continuous distributions – is exactly equal to $$\int_{x=-\infty}^\...
user avatar
  • 22.3k
19 votes
Accepted

How can Logistic Regression produce curves that aren't traditional functions?

This is an example of overfitting on the Coursera course on ML by Andrew Ng in the case of a classification model with two features $(x_1, x_2)$, in which the true values are symbolized by $\color{red}...
user avatar
15 votes
Accepted

How to find the mode of a probability density function?

Saying "the mode" implies that the distribution has one and only one. In general a distribution may have many modes, or (arguably) none. If there's more than one mode you need to specify if you want ...
user avatar
  • 261k
9 votes
Accepted

How to understand the geometric intuition of the inner workings of neural networks?

There are two great recent articles on some of the geometric properties of deep neural networks with piecewise linear nonlinearities (which would include the ReLU activation): On the Number of Linear ...
user avatar
  • 6,742
8 votes
Accepted

What is an induced probability function?

Not quite. The setting is a probability space $(\Omega,\mathfrak{F},\mathbb{P})$ and a measurable function $X$ whose domain is $\Omega$ and whose codomain usually is $\mathbb{R}$ with its Borel sigma-...
user avatar
  • 291k
7 votes
Accepted

Arbitrary function approximation in one dimension

Roughness penalty methods seem like a good fit. Here you fit a curve $f$ to a set of data points that minimizes the sum of squared errors with a roughness penalty: $$ \sum_i \left( y_i - f(x_i) \...
user avatar
6 votes

How to find the mode of a probability density function?

If you have samples from the distribution in a vector "x", I would do: mymode <- function(x){ d<-density(x) return(d$x[which(d$y==max(d$y)[1])]) } ...
user avatar
6 votes
Accepted

Let $X$ be a random variable and $f$ an invertible function. Then the CDF of a random variable $Y=f(X)$ always exists?

Your argument is correct, though it would be worth specifying an increasing invertible function, otherwise the inequality would flip. To get the density function, you only need to differentiate the ...
user avatar
6 votes

Limit of Bernoulli R.V.s is a singular distribution

This might be complicated to describe rigorously, using elementary notions, but the underlying concept is simple: almost all real numbers between $0$ and $1$, when written in binary, have equally many ...
user avatar
  • 291k
6 votes
Accepted

How to model positive S-shaped-function?

The sigmoid, S-shaped or ogive curve shown in your plot is ubiquitous in nature. Geoffrey West's recent book Scale: The Universal Laws of Growth, Innovation, Sustainability, and the Pace of Life in ...
user avatar
  • 9,832
6 votes
Accepted

Does $\mathbb{P}(X < a) = \mathbb{P}(f(X) < f(a))$?

Consider the set of $x$, call it $S$, where $x<a$. You seek for the probability, $P(S)$. Any expression that lead to $S$ produces the exact same probability, $b$, irrespective of its decleration. ...
user avatar
  • 52.2k
6 votes
Accepted

Generating uniform points inside an $m$-dimensional ball

A simple and efficient method for this problem uses a variation of the well-known Box-Mueller transform, which connects the normal distribution to the uniform distribution on a ball. If we generate a ...
user avatar
  • 97.7k
5 votes
Accepted

How to measure the shift between two cumulative distribution functions (CDFs)?

Think about what a CDF represents in terms of probability. Let the variables on the x-axis be referred to as $x$ and y-axis values be referred to as $y$. By definition the cumulative distribution ...
user avatar
5 votes
Accepted

Showing Bayesian updating in R

You aren't updating the "new" prior with the posterior! And it's much harder to do with functions, unless you're good with R functional programming, than with discretized values for $p$. I'm ...
user avatar
  • 33.5k
5 votes

Difference between function and distribution?

Small corrections and notes: y should be $\frac{1}{\sqrt{\pi}}exp(-x^2)$ to be a density function. This density, as you say, represents a Gaussian distribution with specific mean/variance. ...
user avatar
  • 52.2k
4 votes
Accepted

Decomposing a random variable with random mean into a sum

This question, and all the responses by the OP, seems not to jibe in various ways. $X\sim \mathcal{N}(0,\sigma^2+1)$. $Z$, a gaussian with mean $X$, I assume this means that the ...
user avatar
4 votes

Do optimization problems written in argmax form represent a function?

To view argmax as a function, other than the trivial "constant" function, you would need to make it a function of one or more arguments, which could be, for instance, input data specifying a ...
user avatar
4 votes
Accepted

How to write diff() mathematically?

Assume you want to apply diff to a vector $(x_1, \dots, x_n)$ of length $n$. The result will be the vector $(d_1, \dots, d_{n-1})$ of length $n-1$ with entries $$ ...
user avatar
4 votes
Accepted

Are dependent variables necessarily functions of one another?

This question is interesting because it concerns the general problem of regression in the sense of characterizing (or estimating based on data) the conditional distribution of one variable, $X_1,$ ...
user avatar
  • 291k
4 votes
Accepted

Optimization: Convex function

It simplifies the notation to work with $$g(\mathbf{y}) = f(\mathbf{y}+\mathbf{x}) - f(\mathbf{x})$$ because (as you can readily compute) $$g(\mathbf{0}) = 0;\ \nabla g(\mathbf{0}) = \nabla f(\...
user avatar
  • 291k
4 votes

How do you find the asymptotic distribution of a function of the sample mean?

The question is a little bit too general in its present form to get a useful result. Nevertheless, with some slight restrictions we can get a useful general form for the asymptotic distribution using ...
user avatar
  • 97.7k
4 votes

Generating uniform points inside an $m$-dimensional ball

The simplest and least error-prone approach - for low dimensions (see below!) - would still be rejection sampling: pick uniformly distributed points from the $m$-dimensional hypercube circumscribing ...
user avatar
4 votes
Accepted

Is there a name for $\sum P(x) \frac{P(x)}{Q(x)}$ ? (P and Q are pmf)

It is basically $\chi^2(P,Q)+1$, where the chi-squared divergence between two distributions is defined as $\chi^2(P,Q)=\sum_x {(P(x)-Q(x))^2\over Q(x)}=\sum_x {P^2(x)\over Q(x)}-1$. Note it is not ...
user avatar
3 votes
Accepted

What is this formula?

I think your first expression should be $|\frac{\bar y}{y}-1|$, because negative values are meaningless in terms of error. Note that $|\frac{\bar y}{y}-1|=|\frac{y-\bar y}{y}|$. So the difference ...
user avatar
  • 478
3 votes
Accepted

Apply function generating random numbers to a matrix (R)

Your code does exactly what you have required of it. Function my.function returns only one value, and you asked yo put it in everywhere based on condition. I ...
user avatar
  • 33.5k
3 votes

Can a neural network learn a functional, and its functional derivative?

Neural nets can approximate continuous mappings between Euclidean vector spaces $f : \mathbb{R}^M \to \mathbb{R}^N$ when the hidden layer becomes infinite in size. That said, it's more efficient to ...
user avatar
3 votes

Can a neural network learn a functional, and its functional derivative?

This is a good question. I think it involve theoretical mathematical proof. I have been working with Deep Learning (basically neural network) for a while (about a year), and based on my knowledge from ...
user avatar
  • 11.6k
3 votes
Accepted

Given the distribution of a variable X, what is the distribution of f(X)?

Suppose that $f()$ is a smooth function (if not infinitely differentiable, we'll need it to have a least a couple of continuous derivatives.) Also, assume that $X$ has $E[X]=\mu$, and $Var(X)=E[(X-\...
user avatar
3 votes
Accepted

What is the name of the resulting function?

Sorry to say, but this is a totally confused mix up. First, it appears that $f(\epsilon)$ is a probability density function. If this is the case it must intagrate to unity, over the specified domain ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible