Questions tagged [overlapping-data]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
0 answers
24 views

How to quantify temporal overlap of species abundance?

I want to compare the extent of overlap of species abundances in the following two secanrios. In the first scenario, species barely overlap so the index or overlap value should be zero. in the second ...
user avatar
3 votes
1 answer
51 views

Probability of outcomes from overlapping samples of a random variable

I have a feeling there is a straight forward answer to my question, but I'm not sure what the appropriate term to search for is. I'm trying to get a sense of what the probability is of finding some ...
user avatar
1 vote
0 answers
23 views

What are pros and cons of AR and other time series models with overlapping datapoints

Anyone have any links or resources on pros/cons of building a timeseries model with overlapping data points? Generally, a lot of text build models on daily returns, but let's say the daily variable ...
user avatar
  • 2,523
0 votes
0 answers
61 views

Overlapping data in time series model?

I have some time series data that I am trying to model - basically historical elasticity reads with daily sales data. I really am just interested in finding the 'mean' value - not necessarily predict ...
user avatar
  • 2,523
0 votes
1 answer
13 views

Population overlap of random distributions of binary choices

I’m trying to figure out how to calculate the overlap (concurrence) of certain building features, for example: high-efficiency windows, thick insulation, efficient lights, etc. I know the probability ...
user avatar
0 votes
0 answers
25 views

How can I visualize and receive data about time overlap of several actions?

I am observing a person walking, running, and yo-yoing and want to know how much time a person spends yo-yoing while they walk. Calculating total time spent doing each activity is relatively easy (...
user avatar
  • 1
1 vote
2 answers
106 views

Average count of overlap between random samples

If I randomly sample n numbers from 1..m, x times, what would be the average count of ...
user avatar
1 vote
1 answer
48 views

Properties of OLS estimator for overlapping observations' regression

I am running regressions of long-horizon gross financial returns $$R_{t+k} = \prod_{i=1}^{k} R_{t+i}$$ where $R_t=1+r_t$, on current dividend-price ratios $DP_t=\frac{D}{P}_t$: $$R_{t+k}=\alpha +\beta ...
user avatar
  • 207
1 vote
1 answer
75 views

How to deal with visually overlapping clusters

I have done hierarchal clustering (average) and silhouette method shows 6 as the best k. However visualising I see that most of the clusters are overlapping. Any way to deal with the overlapping ...
user avatar
2 votes
2 answers
46 views

How to get the estimated means and standard deviations of two overlapping circular normal distributions?

I have the following plot: Overlapping circular normal distributions I would like to estimate the means and standard deviations of the apparent overlapping normal distributions. This is slightly ...
user avatar
0 votes
0 answers
20 views

How to do hypothesis test for overlapping data?

I am now studying the effects of two different drugs on a specific outcome. The problem is some of my subjects could take these two drugs at the same time, which makes it hard to estimate individual ...
user avatar
  • 1
0 votes
0 answers
24 views

Appropriate to rescale kernel densities to radians and use watson's two sample test of homogeneity?

I'm looking at activity budgets of animals derived from camera traps and gps collars. I'm pooling multiple cameras or gps collars within each group before comparing the activity of two groups against ...
user avatar
  • 103
0 votes
0 answers
58 views

Overlapping vs. non-overlapping financial returns in machine learning algorithms

Dear all I have a question with respect to overlapping observations when using any machine learning algorithm. When I want to test predictability say of weekly or monthly returns with some set of ...
user avatar
  • 27
0 votes
0 answers
17 views

How to measure the overlap quality of a feature?

For a communication related project, I want to measure the overlapping quality of three features based on three applications. I have a data set as follows: There are three types of activities: Audio, ...
user avatar
  • 21
0 votes
0 answers
29 views

How to quantify the overlap of 2 distribution

I have three events in time that follows a Lognormal distribution and that, in theory, occur in order T0 then T1 then ...
user avatar
  • 31
2 votes
1 answer
34 views

Why do two confidence intervals only overlap when their difference $(\theta_1 − \theta_2) \pm z(SE_1 + SE_2)$ includes 0?

I am able to check if the interval contains 0 which I can find out by just plugging numbers into the above formula but I am curious as to why this interval containing zero means that the two CI's of $\...
user avatar
  • 23
1 vote
1 answer
327 views

Quantifying overlap between two categorical distributions with some non-identical categories

I'm looking for a way to measure the overlap (or general similarity) between two categorical distributions in which some of the categories are shared between each and some are not. For example, if the ...
user avatar
1 vote
0 answers
122 views

Robust standard errors under overlapping observations: confusing simulation results from alternative methods

I have a time series of $h$-step-ahead forecasts $\hat{y}_{t+h|t}$ for $t=1,\dots,T-h$ with $h>1$. I also have the corresponding realized values of their targets $y_{t+h}$ and the corresponding ...
user avatar
0 votes
0 answers
47 views

Stationarity in quarterly overlapping data

If I have a time series of quarterly default rates (e.g. of mortgages), by definition they will be 75% correlated, thus leading in non stationarity. What should I do in this case? Can I ignore lags up ...
user avatar
  • 473
2 votes
1 answer
80 views

Bayesian framework - Prior and Likelihood independence

I've a question on how, if ever needed, one can deal with the possible overlapping dependence of prior information and current data. Basically, I was given the output parameters of a regression that ...
user avatar
  • 317
2 votes
0 answers
95 views

Correction for sample overlap in a meta-analysis of prevalence (metafor package)

I want to conduct a meta-analysis of around 30 studies (from a systematic review). Some background of the studies: The quantity of interest is the prevalence of RSV infection. Different studies ...
user avatar
2 votes
0 answers
76 views

Standard Error Bias from Monte Carlo Simulations

I am currently working through this paper on overlapping observations by Britten-Jones, Neuberger and Nolte 1, which presents an estimation method for correcting standard errors when performing ...
user avatar
  • 143
2 votes
0 answers
152 views

How to adjust R^2 and the Coefficient of Determination when using Overlapping Observations?

I am currently working through the paper Improved Inference and Estimation in Regression With Overlapping Observations [1] which presents an elegant way to do inference on $\beta$ in a linear ...
user avatar
  • 143
2 votes
1 answer
109 views

Estimating moments of annual return distribution

I am interested in estimating moments of annual return distributions of indices with limited historical data. I.e. equity/fixed income indices which have only 10-20 years of daily data available. The ...
user avatar
2 votes
0 answers
225 views

Effective sample size of a time series of overlapping observations

Question: How do I calculate the effective sample size of a time series under overlapping observations? By overlapping observations of $x_t$ where the overlap is of length $k$ I mean a case where $...
user avatar
3 votes
1 answer
149 views

Bias of overlap sampling

Say I have a series $x_1, ...., x_N$ independently drawn from $Normal(0,1)$ Say I create sub-samples $y_1 = x_1+..x_n$, $y_2 = x_2+..x_{n+1}$, $y_m = x_{N-n}+..x_N$, I was expecting $std(\{y\}) =...
user avatar
1 vote
0 answers
36 views

Is it allowed to reduce a dataset of moving averages to run an AR(1) model properly?

I run a simple AR(1) and AR(2) model with the following code: ar.ols(df$y, order.max = 1) ar.ols(df$y, order.max =2) My dataset is as follows: I do have yearly ...
user avatar
  • 39
0 votes
1 answer
82 views

How to change the observation for the first lag in an AR(1) model?

I run a simple AR(1) model in my analysis using ols: ar.ols(df$y, order.max = 1)) However, I work with generations as my unit of analysis. Therefore, the first ...
user avatar
  • 39
6 votes
1 answer
345 views

$R^2$ and adjusted $R^2$ in presence of overlapping observations

Given a linear model $$ y=X\beta+\varepsilon, $$ the population value of $R^2$ is $$ R^2=1-\frac{\text{Var}(\varepsilon)}{\text{Var}(y)}. $$ The vanilla estimator of $R^2$ is $$ \hat R^2=1-\frac{\...
user avatar
1 vote
0 answers
13 views

Multiple Comparisons Problem for non-independant subsets

I'm investigating whether a program was effective or not using BACI analysis and noticed that overall it had no effect (The 2-way ANOVA and non-parametric approach gave a p-value of about 0.5). I want ...
user avatar
0 votes
1 answer
37 views

Cross checking AI Selections vs Random Chance

Fifty Three Images AI Selects 20 images based on various criteria. I Select 20 images based on personal criteria. In a two group selection of all 53 items what is the likely overlap due to random ...
user avatar
0 votes
1 answer
249 views

Survival Analysis on recurrent behavior time series predictor

We are trying to build a credit model to predict the default time (or finally closed the loans as censored). The predictor is a high-dimension time series of current observed previous payment behavior....
user avatar
  • 106
3 votes
0 answers
139 views

Spatial regression with overlapping areas

Is it statistically correct to calculate a regression with overlapping areas? I have market areas as the spatial unit with different sociodemographic and (macro)economic variables and I´m examining ...
user avatar
  • 131
1 vote
0 answers
31 views

(Another) What Statistical Test Should I Use?

If I ask 600 different people to rate 3 different images from 0-10 (i.e 3 samples of 200 people per image) and I calculate the mean scores per image as well as the overall mean for all 3 images, can I ...
user avatar
1 vote
0 answers
86 views

Probability of Finding Two Matching Subsequences in a Sequence

I'm currently studying DNA sequencing and am trying to find a formula which gives the probability that a subsequence of length $k$ appears twice in a sequence of $L$ bases (characters); this is pretty ...
user avatar
  • 11
0 votes
0 answers
52 views

How to analyze combined data from studies with overlap in subjects?

I have data from multiple placebo-controlled cross-over trials (each with $n \simeq 20$) with unique phenotypes measured during the placebo phase. I'd like to combine the data from these measurements ...
user avatar
4 votes
2 answers
106 views

The minimum number of classes in the case of overlapping classes

I've a set of elements that should be grouped in the minimum number of groups, each element belongs to overlapping classes, for eg: a belongs to the classes x , y , z; b belongs to the classes x , y; ...
user avatar
  • 93
2 votes
2 answers
314 views

95% CI for GMC did not overlap but paired Wilcoxon signed rank test gave a p-value > 0.05

I measured antibodies for a pathogen on the same group of subjects measured at month 2 and month 7. Since antibody concentrations vary a lot from subject to subject, we decided to use Geometric Mean ...
user avatar
4 votes
0 answers
181 views

Forecasting with annual data that has a rolling quarter

I want to fit an ARIMA model for forecasting on a quarterly basis, but my data is a rolling year, updated quarterly, how can I use this most effectively? I'm really interested in the best estimate of ...
user avatar
3 votes
0 answers
476 views

Correcting estimate of covariance/correlation with overlapping/autocorrelated data

I'm wondering how to correct for the presence of autocorrelation/heteroscedasticity in estimating sample covariance matrices from data. I know there have been some questions on this before: ...
user avatar
  • 2,220
0 votes
1 answer
199 views

how to merge multiple overlapping datasets with different units of measurement

I have many overlapping datasets of molecules and their respective binding affinities to a receptor structure. Each dataset comes from a different experimental assay and conditions, therefore the ...
user avatar
0 votes
0 answers
48 views

Intersection of significant differences

My data are signals in which each data point represents a pair of time point and location on the brain. I have a categorical variable with three levels: conditions A, B, and C. I have run non-...
user avatar
  • 131
1 vote
1 answer
2k views

What is the probability (area) of overlap of two normal distributions having EQUAL variance

Can you please provide a math formula or an Excel example (math operations and Excel functions jointly used) to compute the probability (area) of overlap of two normal distributions having EQUAL ...
user avatar
  • 11
0 votes
0 answers
217 views

Overlapping data and QQ plot

Problem Have a time series of data and want to look at changes over long periods so want to use overlapping changes, then want to compare these overlapping changes to overlapping changes from a normal ...
user avatar
  • 49
2 votes
0 answers
473 views

How to analyse multiple choice and overlapping questions in MCA? [closed]

I need to analyze a survey about entrepreneurship which has around 50 categorical variables. Therefore, after some univariate analysis, I want to apply Multiple Correspondence Analysis (MCA) in order ...
user avatar
0 votes
2 answers
2k views

Comparing overlap and matches in different datasets

My goal: a measure/test/score/etc. that can give me a numerical value for how similar two datasets are. Let's say I have two datasets, each with a differing number of datapoints, where each datapoint ...
user avatar
  • 137
2 votes
0 answers
42 views

How to estimate treatment effect with overlapping treatments?

I've taken over a project with an overlapping treatment design and I'm wondering how to estimate the overall effect. The description: Our organization sends 4 separate promotional postcards to our ...
user avatar
1 vote
0 answers
51 views

calculate percent of sample in overlapping distributions

I'm looking at 80k samples representing results of a test after a fixed period of time. I think it looks like a combination of normal curves. I see a fair amount of documentation about combining ...
user avatar
  • 111
0 votes
0 answers
37 views

Can the data from two resources be mixed?

There are several sources of temperature-dependent viscosity data, which goes as following: |:--------------------------------| | 273.15 | 320.5 | 330.25 | |---------------------------------| | 1....
user avatar
  • 67
3 votes
0 answers
495 views

Robust standard errors for a time series regression

Suppose we have time series data $(p_t, (x_{i,t})_{i = 1...n})$, where $p_t$ is a price and $x_{i,t} \in \mathbb{R}^n$ regressors. From $p_t$ we construct $r_t = \frac{p_{t+30}}{p_t} -1$, the one-...
user avatar
  • 3,033