A stochastic process describes evolution of random variables/systems over time and/or space and/or any other index set. It has applications in areas such as econometrics, weather, signal processing, etc. Examples - Gaussian process, Markov Process, etc.

learn more… | top users | synonyms

3
votes
0answers
37 views

Capturing an Escaping Prisoner? (Something I thought about in my car)

So, I was in my car listening to the radio when I was listening to a story about the captured New York prison escapees, and it had me thinking: In a hypothetical large fixed area of land, let us ...
0
votes
0answers
7 views

Recommended Study Area For Processes

I am looking for a machine learning area that deals in processes for logistics. If anyone can show me some use cases or even point me in the direction of a couple of algorthims. Im currently using R ...
9
votes
4answers
892 views

Why does the variance of the Random walk increase?

The random walk that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term. You can prove that ...
2
votes
0answers
26 views

Hodrick-Prescott Filter, Time Series, SDE, and Ito Isometry

The background of this question is a paper written by Morten O.Ravn and Harald Uhlig, titled "On Adjusting The Hodrick-Prescott Filter For The Frequency of Observations" Consider the decomposition of ...
5
votes
1answer
44 views

Runs of the same type within a deck of cards - distribution of runs of different length

My background is in Physics, not statistics, so forgive any suspect terminology or notation, but I hope the problem is clearly set out below. Secondly, my statistics is not good enough to recognise ...
2
votes
1answer
37 views

How does one do Stochastic Gradient Descent (SGD) on an objective function that has a regularizer?

I know that for Stochastic Gradient Descent, one picks a data point $(x_n, y_n)$ at random from the training set $S_N$ and then updates the parameter of the model in question. If the cost function ...
0
votes
0answers
15 views

Occupation Time for a Uniform Stochastic Process

Let $\{\mathcal{X}(\theta ):\theta \in \Theta\}$ be a stochastic process with continuous sample paths, where $\Theta$ is a compact set, and $\mathcal{X}(\theta )$ is uniformly distributed on $[0,1]$. ...
6
votes
1answer
87 views

Expected number of times you spent in a state of an absorbing markov chain, given the eventual absorbing state

It's well known that, if $Q$ is the matrix of transient state transition probabilities, and $$ N = \sum_{n=0}^{\infty} Q^n = (I - Q)^{-1}$$ then $N_{ij}$ describes the expected number of times the ...
0
votes
2answers
36 views

Wanting to learn Dynamic Programming for stochastic optimal control, I need help getting started

I have an optimal stopping and control problem for which the dynamic programming equation is written. I am totally new to this field and type of problem but I have bases in Stochastic Calculus and ...
-1
votes
0answers
24 views

Trend modelling interpretation

I have estimated the following two models: $$ Δy_t=0.015-0.410Δy_{t-1}-0.220Δy_{t-2} $$ and $$ Δy_t=0.400+0.00145t-0.150y_{t-1}-0.325Δy_{t-1}-0.220Δy_{t-2} $$ (Note that $y_t$ is the log of monthly ...
2
votes
0answers
17 views

Nonparametric changepoint detection for a point process

I have a bunch of point processes that are generated by some unknown model. There is a marked pause that seems to begin and end at the same time in each process. I would like to measure this pause. I ...
0
votes
0answers
18 views

How can I form an SDE with two Levy noises using the YUIMA package in R?

I'm trying to use the YUIMA package in R to simulate a two-dimensional process with two distinct Levy noises, but I can't seem to get it to work. I've searched online for documentation or examples ...
0
votes
0answers
11 views

Complicated SDE

I have following problem. Let $Y_{t}$ be an exponential Levy Process. That is: $$Y_{t} = Y_{0}e^{X_{t}}$$ Where $X_{t}$ is Levy process. I have a function of $Y_{t}$, $f$ :$\mathbb{R}_{+} \times ...
0
votes
1answer
51 views

Deriving Itō's Process with a drift for Geometric Brownian Process

Can anyone show how to derive Itō's Process if given a Geometric Brownian Process $\Delta S/S=\mu\Delta t + \sigma\epsilon\sqrt{\Delta t}$, where $\Delta S$ = change in stock price, $\mu$=expected ...
1
vote
0answers
22 views

Probability of losing everything in N games

Consider a gambler who starts with an initial amount of money of $£i$, obtain $£R$ with probability $p$ and lose $£J$ with probability $q=1-p$. What is the probability that it loses everything if he ...
0
votes
0answers
19 views

Prove that $N(\tau),V_t,Z_2,\ldots$ are independent in Poisson process

We define a Poisson process is a renewal process in which the interarrival intervals $X_n$'s have an exponential distribution with parameter $\lambda$. Denote $N(t)$ is the number of arrivals in ...
0
votes
0answers
18 views

Prediction of the probabilities in the stochastic process

The system can have n different states. At every time period it might either stay in the previous state or move to another state due to two possible reasons (A and B). I need to predict three ...
0
votes
0answers
7 views

Binomial representation of stochastic process

It is common knowledge that a random walk can be represented in the form of a binomial process. Is it possible to represent any generic stochastic process (including non-linear) of the form $dX = adt ...
0
votes
0answers
19 views

Set probability of Discrete Random variable

I want to simulate outcomes of 3 discrete events. For example lets say I have a spinner and there are 3 outcomes. The outcomes of the spinner are 3 numbers: 1, 2 and 3. The probability of getting ...
1
vote
1answer
28 views

Identifying a stochastic trend model

My question is a bit general Say I am given a time series $X_t$, In what ways I can use in order to check whether the sequence behaves like a stochastic trend model or not? and if yes how can I find ...
0
votes
0answers
4 views

Determining a fit test for a interupted stochastic data

I have 10s of thousands of linear data tracks (DNA sequence abundances) which I am attempting to classify by their adherence to a theoretical model. My first pass approach was quite naive, defining ...
-1
votes
1answer
42 views

markov chain - probability question

Transition matrix has been written like that; $$\mathcal P = \begin{bmatrix} 1/3 & 0 & 2/3 \\ 1/3 & 1/3 & 1/3 \\ 0 & 0 & 1 \end{bmatrix}$$ the initial vector is that ...
1
vote
1answer
15 views

Calibrating irregularly sampled Ornstein-Uhlenbeck process

I would like to recover the parameters of a Ornstein-Uhlenbeck process from observations that are irregularly spaced. Estimation via linear regression and maximum likelihood is demonstrated here for ...
0
votes
0answers
18 views

How to show that $\pi^{X(n)}_{o}$ for $n \geq 0$ and $X(n)$ a branching process, is a martingale?

If I let $X(n)$ be defined as the size of a branching process at the $n$th generation, and $\pi_{o}$ as the probability that the process will eventually go extinct, I'd like to show that ...
2
votes
3answers
62 views

How to show $M_n = X_n^2-n$ is a martingale?

Let $X_n, n = 0, 1, 2, . . .$ denote an unbiased Normal Random Walk. $X_0 = 10$, and $X_{n+1} = X_n + Y_{n+1}$, with $\{Y_n\}$ are i.i.d. $N(0, 1)$. Then how can I show that: A) $M_n = X_n^2-n$ is a ...
0
votes
1answer
51 views

How can I find the expected time until a random variable is greater than some constant?

I have random variables $X_1, X_2, X_3, ....$ that are i.i.d. with the same distribution F. If I define $k$ to be a constant and $T$ to be the time until any $X_i$ is greater than $k$, what would be ...
0
votes
0answers
42 views

Does a renewal process have the Markov property?

I have seen three proofs for the renewal equation as used in studying renewal processes. The one that has bothered me the most is the one on wikipedia. It states the following: Wikipedia The ...
2
votes
0answers
43 views

Does a stationary process necessarily have to be mean-reverting?

I wonder about if a stationary process is by definition mean-reverting too. I know the formal definition of a stationary process, but I'm not sure about the definition of a mean-reverting process. ...
0
votes
0answers
25 views

Limiting distribution of a Markov chain?

I have the problem below. There are n identical machines. They are all operational at time 0. The lifetime of each one is an exponential random variable with rate L. There are r repairmen (1 ≤ r ≤ ...
0
votes
0answers
16 views

Improvement of Minimum description length (MDL) estimate

I earnestly request apology if this question is inappropriate for the forum. The question has two parts one technical and the other is not technical. I would appreciate any response. Let me consider ...
-1
votes
1answer
22 views

Proof of Markov Chain property

Suppose that $X_n$ is a Markov Chain.Then for $m,n \in N$ such that $m<n$ $Pr[X_n=j_n|X_m=j_m,X_{m-1}=j_{m-1},...=X_0=j_0]=Pr[X_n=j_n|X_m=j_m]$ When proving for n=3,m=1 case we have to show ...
3
votes
2answers
37 views

Finding $b$ such that $e^{5B_t - bt}$ is a martingale

I have $X_t = e^{5B_t}$ and Where $B_t$ is brownian motion at time $t$. $M_t = X_t \cdot e^{-bt}$ I need to find a value for $b$ such that $M_t$ is a martingale. I am encountering difficulty, ...
0
votes
0answers
23 views

Finding the best predictor Brownian motion

I want to find the best predictor of $(B_3-B_2)(B_4-B_{\pi})$ given an observation of $B_1$ Where $B_t$ is brownian motion for time $t \geq 0$. I am not sure how to approach this. I know it will be ...
0
votes
0answers
83 views

Show $∫_0^t X(t,s)dB(s)$ is a Gaussian random variable $Y(t)$ [duplicate]

Show that if $X(t)$ is non-random (does not depend on $B(t)$) and is a function of $t$ and $s$ with $\int_0^t X^2(t,s)ds<\infty$, then $\int_0^t X(t,s) dB(s)$ is a Gaussian random variable $Y(t)$. ...
1
vote
1answer
114 views

Fit and evaluate a second order transition matrix (Markov Process) in R?

I already built 1 first order discrete state Markov Chain model. It was built with R using the function 'markovchainFit()' in ...
3
votes
0answers
83 views

simulating birth death process with random numbers from negative binomial

I am trying to generate random deviates for the population size at time $t$ for a birth-death process with constant birth and death rates per individual and initial size $N_0 \gt 0$. For the simple ...
2
votes
0answers
72 views

a question on 0-1 valued stochastic process [closed]

Consider a stochastic process $X_{t}$ taking values in the set $\{0,1\}$ according to the probability measure $\mu$. Let $$Y_{t} = \mu\left(\limsup_{T \rightarrow \infty}\frac{1}{T}\sum_{t = 0}^{T - ...
0
votes
1answer
18 views

Birth & Death process - Combining Transition rates

I think I'm missing a fundamental step in regards to how to combine two exponential distributions in the context of this problem. If we have a birth and death process where birth rate ~ ...
3
votes
3answers
103 views

Stochastic Differential Equations - A Few General Questions

I just have a few questions about stochastic differential equations. I generally did a lot of pure math but signed up for a course on probability models and stochastic differential equations because I ...
1
vote
1answer
58 views

How is $P[X_t\le x_t | X_1,\ldots, X_{t-1}]=P[X_t\le x_t]$ when $X_t\sim WN(0,\sigma^2)$?

In this slide , p.30 , p.31 , it is written that : White noise : $X_t\sim WN(0,\sigma^2)$ i.e., ${\{X_t}\}$ uncorrelated, $\mathbb E[X_t]=0, \mathbb V[X_t] =\sigma^2$ Example : i.i.d noise : ...
11
votes
1answer
186 views

What are the main differences between Granger's and Pearl's causality frameworks?

Recently, I ran across several papers and online resources that mention Granger causality. Brief browsing through the corresponding Wikipedia article left me with the impression that this term refers ...
0
votes
1answer
68 views

writing down markov chain transition matrix

Question: An experimental animal can stay in room-A until 1 minute,and it can stay in room-B until 2 minutes. There exist deadly gases in room-C. One room among these three rooms is being randomly ...
6
votes
0answers
118 views

A question related to Borel-Cantelli Lemma

Note: Borel-Cantelli Lemma says that $$\sum_{n=1}^\infty P(A_n) \lt \infty \Rightarrow P(\lim\sup A_n)=0$$ $$\sum_{n=1}^\infty P(A_n) =\infty \textrm{ and } ...
4
votes
2answers
45 views

Definition of $X_t$ in the context of Stochastic process and Time Series

In the book An Introduction to Stochastic Modeling , Stochastic process is defined as : A stochastic process is a family of random variable(s) , $X_t$ , where $t$ is a parameter running over a ...
0
votes
0answers
48 views

Detrending or Differencing in order to make a series stationary?

I got several time series for which I want to find out if they are stationary or not. So I computed for each series the kpss.test(). But before making further ...
0
votes
0answers
16 views

Class of semimartingales for which all characteristics can be estimated?

I'm going to ask the question for Ito semimartingales rather than semimartingales in general, but more general answers would be great. An Ito semimartingale is a martingale for which the ...
2
votes
1answer
49 views

Proof of Chapman Kolmogorov equation

In the proof of Chapman Kolmogorov Equation $p_{ij}^{(m+n)}=\sum_{k=0}^{\infty}p_{ik}^{(n)}p_{kj}^{(m)}$ Proof: $p_{ij}^{(m+n)}=P[X_{m+n}=j|X_0=i]$ By the total probability it says ...
3
votes
1answer
57 views

How can I calculate this probability: $P(W_1<cW_2$) and $c\geq 0$?

Let $(W_t)_{t\geq 0}, $ be a Brownian motion. I want to calculate the following: $P(W_1<cW_2$) and $c\geq 0$ For $c=1$ it is easy. I just write it as an increment, but how can I do it when $c$ ...
1
vote
1answer
58 views

Frequency distribution of Chinese Restaurant Process?

Set-up I was simulating the Generalized Chinese Restaurant Process as shown on the wikipedia page [link] with a discount, $\alpha$, and concentration parameter $\theta$ For $n=5$ total customers ...
0
votes
0answers
37 views

Bayesian Ridge vs Stochastic Gradient Descent

I was running some Regression algorithms on a dataset and it just so happens, that the Bayesian ridge Regression techniques is performing not so well as the SGD (Stochastic Gradient Descent) ...