Return to Question

$\TeX$, spelling, grammar, punctuation, clarity, consolidation, reorganization, brevity, code formatting; converted URL to hyperlinked text

Source Link

edited Mar 28, 2014 at 15:26

Nick Stauner

12.4k
5
53
110

I was faced with a problem of automatically detecting regions of continuity in a vector. I have a lot of these vectors, hundreds. oneOne example of such vector is here: http://pastebin.com/UiSkJ7Dahere

So basically. Basically looking at the vector, one can see that there are regions of continuity, then it breaks, jumps some value, and then has another region of continuity and so on.

this works wonderfully and uses the 95% confidence interval to find the start and end points of the indexes of the regions.

I I know it worksworks; I just want to know why and how it works.

Statistics Statistics is not my strong point, so I have been reading in detail anything I can find related to the provided solution. Can anyone strong in statistics shed some lightStatistics is really fascinating me as to how one can make assumptions intelligently based on thisa sample and find them to be true.

I want to fully understand the working of the solution provided. Based on what I have read and understood, this is my understanding of the explanation (Pleaseplease note I have no base in stats at all, so I may be going very basic here). Can anyone strong in statistics shed some light on this?

So to begin aA 95% confidence interval is basically saying that I am 95% sure that given mean of a probability is within these limits [a, b].

And And this can be calculated with the formula:

x +- 1.96 (std/sqrt(n))

where x $x \pm 1.96 (std/\sqrt n)$ where $x$ is the mean, std$std$ is tehthe standard deviation and n$n$ is the sample size.

And And this confidence interval applies to a normal distribution only.

So now that I have my basics done, let me try to explain my understanding of the solution.

judgingJudging by my vector and, the samples were observed to be normally distributed samples (between the jumps). Knowing that these are normal distributions, simply finding their confidence intervals will give the start and end pointendpoint of a normally distributed sample.

And And that is what is going on in this line:

where find,find finds the indexesindices of the points where the confidence interval equation holds true, but why the differecedifference vector, dv dv? and why use it in the condition statement?

Why Why even create a difference vector? andAnd why use it in the line above?

This is based on what I read today, so I may be way off. Sorry for such a long read but amAm I on the right path? Is my understanding correct.

Statistics is really fascinating me as to how intelligently one can make assumptions based on a sample and find them to be true.

? Thank you.

I was faced with a problem of automatically detecting regions of continuity in a vector. I have a lot of these vectors, hundreds. one example of such vector is here: http://pastebin.com/UiSkJ7Da

So basically looking at the vector one can see that there are regions of continuity, then it breaks, jumps some value and then has another region of continuity and so on.

this works wonderfully and uses the 95% confidence interval to find the start and end points of the indexes of the regions.

I know it works I just want to know why and how it works.

Statistics is not my strong point, so I have been reading in detail anything I can find related to the provided solution. Can anyone strong in statistics shed some light on this.

I want to fully understand the working of the solution provided. Based on what I have read and understood this is my understanding of the explanation (Please note I have no base in stats at all so I may be going very basic here).

So to begin a 95% confidence interval is basically saying that I am 95% sure that given mean of a probability is within these limits [a, b].

And this can be calculated with the formula:

x +- 1.96 (std/sqrt(n))

where x is the mean, std is teh standard deviation and n is the sample size.

And this confidence interval applies to a normal distribution only.

So now that I have my basics done, let me try to explain my understanding of the solution.

judging by my vector and the samples were observed to be normally distributed samples (between the jumps). Knowing that these are normal distributions simply finding their confidence intervals will give the start and end point of a normally distributed sample.

And that is what is going on in this line:

where find, finds the indexes of the points where the confidence interval equation holds true, but why the differece vector, dv? and why use it in the condition statement?

Why even create a difference vector? and why use it in the line above?

This is based on what I read today so I may be way off. Sorry for such a long read but am I on the right path? Is my understanding correct.

Statistics is really fascinating me as to how intelligently one can make assumptions based on a sample and find them to be true.

Thank you

I was faced with a problem of automatically detecting regions of continuity in a vector. I have a lot of these vectors, hundreds. One example of such vector is here. Basically looking at the vector, one can see that there are regions of continuity, then it breaks, jumps some value, and then has another region of continuity and so on.

this works wonderfully and uses the 95% confidence interval to find the start and end points of the indexes of the regions. I know it works; I just want to know why and how it works. Statistics is not my strong point, so I have been reading in detail anything I can find related to the provided solution. Statistics is really fascinating me as to how one can make assumptions intelligently based on a sample and find them to be true.

I want to fully understand the working of the solution provided. Based on what I have read and understood, this is my understanding of the explanation (please note I have no base in stats at all, so I may be going very basic here). Can anyone strong in statistics shed some light on this?

A 95% confidence interval is basically saying that I am 95% sure that given mean of a probability is within these limits [a, b]. And this can be calculated with the formula: $x \pm 1.96 (std/\sqrt n)$ where $x$ is the mean, $std$ is the standard deviation and $n$ is the sample size. And this confidence interval applies to a normal distribution only.

Judging by my vector, the samples were observed to be normally distributed samples (between the jumps). Knowing that these are normal distributions, simply finding their confidence intervals will give the start and endpoint of a normally distributed sample. And that is what is going on in this line:

where find finds the indices of the points where the confidence interval equation holds true, but why the difference vector dv? and why use it in the condition statement? Why even create a difference vector? And why use it in the line above?

This is based on what I read today, so I may be way off. Am I on the right path? Is my understanding correct? Thank you.

Source Link

asked Mar 28, 2014 at 15:00

StuckInPhDNoMore

Is my understanding of this “confidence interval” solution correct?

I was faced with a problem of automatically detecting regions of continuity in a vector. I have a lot of these vectors, hundreds. one example of such vector is here: http://pastebin.com/UiSkJ7Da

So basically looking at the vector one can see that there are regions of continuity, then it breaks, jumps some value and then has another region of continuity and so on.

I was provided with this solution by some nice fellow on the newsreader:

v = [ 9 18 21 58 59 60 63 66 69 70 72 74 ...
dv = diff([0 v]);                                       % Create difference vector
dvs = [mean(dv)  std(dv)];                              % Determine mean & std
dvd = [0 find(dv > 1.96*dvs(2)+dvs(1)) length(v)];      % Use ‘dvs’ to detect discontinuities & create index reference vector
for k1 = 1:length(dvd)-1
    vs{k1} = v(dvd(k1)+1:dvd(k1+1)-1);                  % Create cell array of regions-of-interest
    vi{k1} = [dvd(k1)+1 dvd(k1+1)-1];                   % Create reference array of start-end indices for ‘vs’ regions
end

this works wonderfully and uses the 95% confidence interval to find the start and end points of the indexes of the regions.

I know it works I just want to know why and how it works.

Statistics is not my strong point, so I have been reading in detail anything I can find related to the provided solution. Can anyone strong in statistics shed some light on this.

So to begin a 95% confidence interval is basically saying that I am 95% sure that given mean of a probability is within these limits [a, b].

And this can be calculated with the formula:

x +- 1.96 (std/sqrt(n))

where x is the mean, std is teh standard deviation and n is the sample size.

And this confidence interval applies to a normal distribution only.

So now that I have my basics done, let me try to explain my understanding of the solution.

And that is what is going on in this line:

dvd = [0 find(dv > 1.96*dvs(2)+dvs(1)) length(v)];

where find, finds the indexes of the points where the confidence interval equation holds true, but why the differece vector, dv? and why use it in the condition statement?

Why even create a difference vector? and why use it in the line above?

This is based on what I read today so I may be way off. Sorry for such a long read but am I on the right path? Is my understanding correct.

Statistics is really fascinating me as to how intelligently one can make assumptions based on a sample and find them to be true.

Thank you