Questions tagged [aggregation]
Refers to "lumping together" potentially inhomogeneous groups of data.
208
questions
0
votes
1answer
34 views
Aggregation Estimation Issues
I'm analyzing the effect of a law enforcement measure over time on reported violence for districts in a city. If I consider the city as a whole I have access to a lot of possible control variables (e....
2
votes
1answer
27 views
Looking for R's way of doing Stata's svy tabulate with “tab” argument [closed]
Stata's svy command allows to get standard error estimates from two way contingency tables and the tab(varname) option makes it ...
0
votes
1answer
19 views
How to use weight variables within aggregated results, for group proportions
If I have the following data, where each row represents and individual
response, how would I go about reporting the proportions of different groups?
(here the groups are ...
0
votes
0answers
28 views
Explain Dempster Shafer Equation
I have a question about the Dempster Shafer theory application. I have four models where the output is of abstract level (crisp). I understand I have to use the confusion matrix (precision/recall) to ...
0
votes
0answers
17 views
How to aggregate multiple MAF files together for analysis?
I have cancer genomic data (tumor/normal whole exome sequencing) from 50
patients that received the same type of treatment, half of whom responded. These come in the form of 50 .maf files, along with ...
0
votes
0answers
15 views
Creating a fraud score for an image fraud detection workflow
So to set the context. I'm working on an image fraud detection workflow for an insurance company. The idea basically is:
There are a number of cases (car accidents).
Each case contains a set of ...
0
votes
0answers
6 views
Choosing the best data aggregation function for downsampling over variable collected over an irregular time series
I was hoping that you can provide me with some expert advice concerning the problem described in title.
I have a time series of tweets that contains sentiment values with range (0,1) in which I am ...
0
votes
0answers
15 views
can i aggregate 3 conditions into 1? What test to use? & unequal data
I've run an experiment with 2 independent variables and I need some advice for what the best form of statistical test to use is, and whether one of the IVs is even possible to do.
IV1: between-...
0
votes
0answers
10 views
How to avoid bucketing in creating time series data with specific characteristics
Based on a dataset containing the sell prices of a house, along with its characteristics (such as nr of bedrooms, living space, etc), I would like to forecast housing prices one year ahead in the ...
1
vote
1answer
21 views
Calculating estimates of aggregate totals from percentile data
I'm working with wage data and would like to compute total aggregate wages earned for individual job classifications. The survey data I'm working from does not report aggregate wages, but it does ...
1
vote
1answer
26 views
Using (summaries of) multiple time series as the independent variable in a regression
I am faced with quite a challenge.
I would like to run a regression to see if variable X can predict variable Y.
Variable Y (DV) is continuous (0-100) and I have 120 data points, so it's all simple ...
1
vote
0answers
17 views
How to analyze data for spatial aggregation and niche overlap with no specific GPS points over time?
I study soil insects, and sample monthly for insects. Each month, I sample at 8 different sites. Each site is divided roughly into 4 meter square quadrants (shown in figure). From each quadrant, I ...
0
votes
0answers
12 views
No of trials needed for a dependent event
I am from risk management. An event is defined as a system going down. Issue is a vulnerability in the system.
Higher the severity of the issue , greater the probability that system is going down.
...
0
votes
0answers
38 views
Aggregating individual-level correlations to make an aggregate-level claim
I have a correlation between X and Y for different groups (they are states in the US, fwiw):
I'd like to make some claim about the aggregate correlation: something like "Overall, as X increases, Y ...
0
votes
0answers
18 views
Comparison using MASE,MAPE for aggregated data with different points of forecast
Despite the fact that it's necessary to analyze the data first, is it possible to compare accuracies (MAPE/MASE) for differents points of forecast from aggregated data (flow, as in Silvestrini and ...
0
votes
0answers
11 views
Modeling Aggregated Variable from Disaggregated Exogenous
I'm trying to do a bit of research regarding what my options are with the following problem. I am trying to predict a nationwide value for retail supply chain volume, but my exogenous data is at the ...
0
votes
0answers
7 views
Aggregating a multivariate distribution
I'm currently trying to investigate how to 'quickly' measure the distance between a sample of data and a larger distribution of data. I have looked at the Kolmogorov-Smirnov test, which in some paper ...
0
votes
0answers
14 views
How can I measure how my clumped my spatial data are?
If I have spatial data how can I measure the degree to which the points are clumped?
...
1
vote
0answers
19 views
Average/Aggregation of 10 Discrete Probabilty Distributions
I have 2 experiments, where I have repeated each 10 times (20 experiments in total).
I want to compare experiment A with experiment B. I create a probability distribution from each exp (20 prob ...
1
vote
0answers
14 views
How should I think about model selection when I will be aggregating predictions?
I'm estimating a model and then using it to predict values in another dataset. After predicting the values, I will aggregate the values to find averages within income quantiles.
I know the ...
0
votes
1answer
34 views
Aggregating sales forecasts from mutually exclusive segments
I am using generalized additive models (GAMs) to forecast sales for 16 mutually exclusive and exhaustive customer segments. There are naturally correlations in these 16 series, including seasonal ...
0
votes
0answers
10 views
Why Does emmeans add_grouping average over the levels of the reference factor?
I've implemented a learning technology to reduce the gender gap(the difference between male and female student grades) in CS education. I conducted a quasi-experiment across six semesters, with ~200 ...
1
vote
0answers
10 views
Aggregation of daily futures prices to quarterly prices
We are working to incorporate the EIA future prices into our quarterly data set of financial balance sheet data. There are four different contracts, which are available at the EIA website.
The idea is ...
0
votes
0answers
17 views
How to calculate the precision of the mean of univariate time series data, based on the precision of the discrete measurements it consists of?
Assume that the precision of the measurements is fixed and independent – each discrete measurement has the same degree of random error, and is independent of all other measurements.
For example, ...
2
votes
0answers
27 views
How can I use spatial heterogeneity within regions as predictors about those regions?
Please forgive if I am misusing the term "spatial heterogeneity". I am trying to predict county level crop productivity using satellite acquired vegetative health (NDVI). Since this is pixel data, I ...
2
votes
0answers
30 views
Inference from small sample size with aggregated country data
I have a data set with country aggregate patient data for yearly total number of diseases, treatments provided by health services, and some other covariates for two countries over 10 years. The ...
1
vote
0answers
22 views
How can I aggregate the estimates for the rate of decay from different experimental runs?
I have data for a device that dispenses material and I want to use an exponential decay model in python to relate the flow rate to the mass left inside the device, in particular $flow = \beta_0 + \...
0
votes
1answer
272 views
Using independent t test on aggregated data?
Stats noob here. Is it possible and appropriate to use an independent t test to compare attrition rates of one department to another department? For example, Department A has 15 teams and Department B ...
0
votes
0answers
29 views
Estimating and doing inference about proportions when individual indicators are themselves estimated
I have a sample of $N$ units and $T$ observations per unit. For each of these units $i$, I estimate a measure of dependence between two variables,$Y_{it}$ and $X_{it}$.
Call the estimated measure $\...
0
votes
1answer
91 views
A/B Testing on Aggregate Measures
Suppose we have built a new version of the "Recommended" page of our media website (e.g. Netflix) and are running a test to evaluate its effectiveness.
We can collect data in the following schema:
<...
0
votes
0answers
16 views
How to interpret hourly rate data given in one miniute intervals
I have time series data on natural gas flow, which is in units of "tonnes/hour". But the data are given in one minute intervals (each row represents a single minute of time duration). Here is a sample:...
1
vote
0answers
45 views
Monthly data vs aggregating weekly data to monthly in linear regression
Let's suppose I have weekly data available for making a forecast using a linear regression model. Let's also assume that the weekly forecasts can be aggregated to monthly ones. Would it make sense to ...
1
vote
0answers
34 views
Correlation of Aggregate Data on a monthly to quarterly basis
I am working with a data set of 42 countries of monthly migration. I want to extract factors using PCA, and find non stationary errors of my model, so I am working in first difference. However there ...
0
votes
1answer
13 views
Best Way to Aggregate further and Analyze a Data Point that is Already an Average
I want to understand how much time (on average) each Customer Service Agent (CSA) is taking to reply to messages that are sent by a Customer on a single chat.
E.g., "Charles took on average, 22s to ...
2
votes
0answers
153 views
Regressions with different periodicity
I am trying to estimate a regression with variables of different periodicity. The dependent variable is given monthly, whereas most other independent variables are also given monthly, but some are ...
0
votes
0answers
189 views
Machine-learning using an aggregation function
I am looking for a machine-learning algorithm (ANN, random forest,...) that is able to account for an aggregation function during the learning step.
I want to estimate variable Y from variables X1,X2,...
0
votes
1answer
69 views
Multiple data points per subject? Aggregation to mean with or without considering subjects
I am currently arguing with someone on how to correctly treat data with multiple observations per subject. More specifically data was gathered from 100 participants 8 times per day for 5 days (...
0
votes
0answers
35 views
Is it valid to get the average of the corresponding elements of two time series?
Say I have two time series that have the same length within the same time period. I need to replace the two time series with a single time series that explains the variations in both. So what I ...
0
votes
0answers
44 views
Recommended model to analyze relationship with aggregate rate
In order to analyze the relationship between some independent variables and a binary dependent variable, it is standard to use logistic regression, which allows us to model the probability of the ...
1
vote
1answer
427 views
Aggregation of “tree results” in random forest regression
I would like to know how the results of different decision trees are aggregated (average) in random forest regression.
If I have a numeric target variable and 10 predictors, each decision tree is ...
2
votes
0answers
103 views
How to aggregate histograms for density estimation
Within a very large sensor network, each node does take measurements derived from a fixed number of samples taken at a high frequency from an instrument. The number of measurements send to an ...
0
votes
1answer
817 views
Machine Learning model with aggregated data as training
I would like to predict a LABEL: A,B or C using a classification machine learning model.
My data to train the model is like:
...
-2
votes
1answer
139 views
Aggregating Cluster Analysis Results [closed]
I want to write something about cluster analysis. I have started to read something about it and I faced with a concept which is called ''Aggregating''. What does it mean for cluster analysis? Do I ...
2
votes
1answer
56 views
Aggregation of overlapping intervals
Hello anyone and everyone,
I have a data set of traffic flow data, particularly intensity data. I have the traffic counts per minute as the base data and then I am aggregating them into 3 and 5 ...
0
votes
0answers
28 views
Combining distributions
I have a ruler and a set of poles $S$. My ruler is not perfect, so a length measurement of one pole yields in fact a sample of a probability distribution (e.g. natural scientists usually find normal ...
0
votes
1answer
646 views
How do I aggregate percentiles from previously aggregated percentiles?
I have previously aggregated data for every 5 minutes with a count of sensors at that location and 5% percentiles of a value of interest: 5th, 10th, ..., 90th, 95th. A sensor can have multiple ...
0
votes
0answers
85 views
Time Series Data: Unequal Sample sizes
I have a dataset which consists of daily observations over a 3 month period for different variables. The data are aggregated so I do not have access to user level data.
The problem is that the ...
3
votes
2answers
167 views
Aggregate repeated measures data or model it?
The structure of my data is causing me some trouble because I'd prefer to aggregate it but I'm not sure of the implications.
It's from an experiment comparing dwell time. The subjects were randomly ...
2
votes
1answer
187 views
Forecasting proportions of aggregations vs direct forecasting
Assume one has to predict the sales of a shop on a daily basis. The sales per day $S(t)$ clearly follow a strong weekly pattern, but also a yearly pattern, a general trend and of course some noise are ...
1
vote
1answer
39 views
How to summarize items and still calculate percentiles?
My company wants to track request latencies for the project I'm working on.
Specifically, the report I'm building should have the 95th and 99th percentile values of the latencies over all events.
...
