Cross Validated

Questions tagged [aggregation]

Ask Question

Refers to "lumping together" potentially inhomogeneous groups of data.

190 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

Why Does emmeans add_grouping average over the levels of the reference factor?

I've implemented a learning technology to reduce the gender gap(the difference between male and female student grades) in CS education. I conducted a quasi-experiment across six semesters, with ~200 ...
1
vote
0answers
8 views

Aggregation of daily futures prices to quarterly prices

We are working to incorporate the EIA future prices into our quarterly data set of financial balance sheet data. There are four different contracts, which are available at the EIA website. The idea is ...
0
votes
0answers
11 views

How to calculate the precision of the mean of univariate time series data, based on the precision of the discrete measurements it consists of?

Assume that the precision of the measurements is fixed and independent – each discrete measurement has the same degree of random error, and is independent of all other measurements. For example, ...
2
votes
0answers
27 views

How can I use spatial heterogeneity within regions as predictors about those regions?

Please forgive if I am misusing the term "spatial heterogeneity". I am trying to predict county level crop productivity using satellite acquired vegetative health (NDVI). Since this is pixel data, I ...
0
votes
0answers
10 views

Inference from small sample size with aggregated country data

I have a data set with country aggregate patient data for yearly total number of diseases, treatments provided by health services, and some other covariates for two countries over 10 years. The ...
1
vote
0answers
17 views

How can I aggregate the estimates for the rate of decay from different experimental runs?

I have data for a device that dispenses material and I want to use an exponential decay model in python to relate the flow rate to the mass left inside the device, in particular $flow = \beta_0 + \...
0
votes
1answer
70 views

Using independent t test on aggregated data?

Stats noob here. Is it possible and appropriate to use an independent t test to compare attrition rates of one department to another department? For example, Department A has 15 teams and Department B ...
0
votes
0answers
29 views

Estimating and doing inference about proportions when individual indicators are themselves estimated

I have a sample of $N$ units and $T$ observations per unit. For each of these units $i$, I estimate a measure of dependence between two variables,$Y_{it}$ and $X_{it}$. Call the estimated measure $\...
0
votes
1answer
43 views

A/B Testing on Aggregate Measures

Suppose we have built a new version of the "Recommended" page of our media website (e.g. Netflix) and are running a test to evaluate its effectiveness. We can collect data in the following schema: <...
0
votes
0answers
14 views

How to interpret hourly rate data given in one miniute intervals

I have time series data on natural gas flow, which is in units of "tonnes/hour". But the data are given in one minute intervals (each row represents a single minute of time duration). Here is a sample:...
0
votes
0answers
10 views

Estimating yearly diagnosis rate from multiple studies

I want to estimate yearly diagnosis rates from some 8 studies. Each of the 8 studies contains the following data: $n$ = total number of patients (e.g., 100) $t_{start}$ = start timestamp of the ...
1
vote
0answers
21 views

Monthly data vs aggregating weekly data to monthly in linear regression

Let's suppose I have weekly data available for making a forecast using a linear regression model. Let's also assume that the weekly forecasts can be aggregated to monthly ones. Would it make sense to ...
1
vote
0answers
21 views

Correlation of Aggregate Data on a monthly to quarterly basis

I am working with a data set of 42 countries of monthly migration. I want to extract factors using PCA, and find non stationary errors of my model, so I am working in first difference. However there ...
0
votes
0answers
31 views

How to deal with a potentially multiple categorical variable

I'm building a model that has, as inputs, some categorical variables. I had already dealt with this sort of data before, and applied different techniques as creation of dummy variables and factor ...
0
votes
1answer
13 views

Best Way to Aggregate further and Analyze a Data Point that is Already an Average

I want to understand how much time (on average) each Customer Service Agent (CSA) is taking to reply to messages that are sent by a Customer on a single chat. E.g., "Charles took on average, 22s to ...
2
votes
0answers
78 views

Regressions with different periodicity

I am trying to estimate a regression with variables of different periodicity. The dependent variable is given monthly, whereas most other independent variables are also given monthly, but some are ...
0
votes
0answers
107 views

Machine-learning using an aggregation function

I am looking for a machine-learning algorithm (ANN, random forest,...) that is able to account for an aggregation function during the learning step. I want to estimate variable Y from variables X1,X2,...
0
votes
1answer
47 views

Multiple data points per subject? Aggregation to mean with or without considering subjects

I am currently arguing with someone on how to correctly treat data with multiple observations per subject. More specifically data was gathered from 100 participants 8 times per day for 5 days (...
0
votes
0answers
34 views

Is it valid to get the average of the corresponding elements of two time series?

Say I have two time series that have the same length within the same time period. I need to replace the two time series with a single time series that explains the variations in both. So what I ...
0
votes
0answers
26 views

True Mean of an Observation

I have a list of aggregated data with 100 data points. The information I have is the mean and the number of observations used to calculate the mean. Moreover I have the raw data from which the ...
0
votes
0answers
35 views

Recommended model to analyze relationship with aggregate rate

In order to analyze the relationship between some independent variables and a binary dependent variable, it is standard to use logistic regression, which allows us to model the probability of the ...
1
vote
1answer
182 views

Aggregation of “tree results” in random forest regression

I would like to know how the results of different decision trees are aggregated (average) in random forest regression. If I have a numeric target variable and 10 predictors, each decision tree is ...
2
votes
0answers
85 views

How to aggregate histograms for density estimation

Within a very large sensor network, each node does take measurements derived from a fixed number of samples taken at a high frequency from an instrument. The number of measurements send to an ...
0
votes
1answer
488 views

Machine Learning model with aggregated data as training

I would like to predict a LABEL: A,B or C using a classification machine learning model. My data to train the model is like: ...
-2
votes
1answer
96 views

Aggregating Cluster Analysis Results [closed]

I want to write something about cluster analysis. I have started to read something about it and I faced with a concept which is called ''Aggregating''. What does it mean for cluster analysis? Do I ...
2
votes
1answer
37 views

Aggregation of overlapping intervals

Hello anyone and everyone, I have a data set of traffic flow data, particularly intensity data. I have the traffic counts per minute as the base data and then I am aggregating them into 3 and 5 ...
0
votes
0answers
26 views

Combining distributions

I have a ruler and a set of poles $S$. My ruler is not perfect, so a length measurement of one pole yields in fact a sample of a probability distribution (e.g. natural scientists usually find normal ...
0
votes
1answer
444 views

How do I aggregate percentiles from previously aggregated percentiles?

I have previously aggregated data for every 5 minutes with a count of sensors at that location and 5% percentiles of a value of interest: 5th, 10th, ..., 90th, 95th. A sensor can have multiple ...
0
votes
0answers
62 views

Time Series Data: Unequal Sample sizes

I have a dataset which consists of daily observations over a 3 month period for different variables. The data are aggregated so I do not have access to user level data. The problem is that the ...
3
votes
2answers
117 views

Aggregate repeated measures data or model it?

The structure of my data is causing me some trouble because I'd prefer to aggregate it but I'm not sure of the implications. It's from an experiment comparing dwell time. The subjects were randomly ...
2
votes
1answer
111 views

Forecasting proportions of aggregations vs direct forecasting

Assume one has to predict the sales of a shop on a daily basis. The sales per day $S(t)$ clearly follow a strong weekly pattern, but also a yearly pattern, a general trend and of course some noise are ...
1
vote
1answer
35 views

How to summarize items and still calculate percentiles?

My company wants to track request latencies for the project I'm working on. Specifically, the report I'm building should have the 95th and 99th percentile values of the latencies over all events. ...
3
votes
2answers
38 views

Many samples of the same identity for machine learning

This seems to be a simple question, yet I couldn't find a clear answer on the net. I'm using machine learning to classify ill/healthy patients based on their medical record (probably using a Random ...
0
votes
1answer
27 views

Feature selection: Should we keep all metrics (like mean. median,..) of a distribution or they make redundant features?

There are many cases in which I don't know which metric I should use to calculate the value of a feature when the value has a distribution for each data point instead of an absolute number. for ...
1
vote
0answers
42 views

OLS difference between individual level and aggregate level estimates

I am new here, so please accept my apologies if this does not belong here. My question essentially concerns what appears to me a special case of the ecological fallacy. Consider a simple model that ...
2
votes
0answers
17 views

Fuzzy Aggregation

Each observation in the data consists of a binary outcome variable $Y$, a set of labelled measurements $(M_1,L_1), (M_2,L_2)...$ (the set size can vary between observations) where each label is one of ...
1
vote
0answers
100 views

Calculating/Averaging/Aggregating Effect Size Across Multiple Measurements

What is the standard practice for calculating a total effect size (using Cohen's d) across multiple measurements in one study? Do you calculate the effect size for each measurement and then take the ...
1
vote
0answers
480 views

R mlogit and McFadden R^2 (strange?) value

I was a little bit surprised by the high value of the McFadden R^2 given by the "mlogit" R package for this simple model: f = mFormula(mode ~ log(cost) | 1 | 1) ...
0
votes
1answer
30 views

Summarise data with R - other than n counts [closed]

I've got donor segments Donors aged 34 and younger, vs. donors aged 35 and older Donors with a high frequency of donations vs. donors with a low frequency of donations Every row in my data frame ...
3
votes
0answers
60 views

Is it dangerous to take the mean of ratio measurements?

I'm going through a Computer Architecture course and in one of the lectures they mention that when measuring the difference in runtime performance between two systems, using for example multiple ...
4
votes
1answer
886 views

Regularized GLM with aggregated data

I am fitting a poisson GLM to model claim rates. Since I have 1.5m+ records, I have aggregated my data (to improve efficiency). My understanding is that using aggregated data with a poisson GLM will ...
1
vote
0answers
252 views

Aggregating normalised data

I want to aggregate Google Trends data relating to separate queries that share a thematic category - e.g. "beer" and "wine". I don't care about the relative popularity of the terms - it is more ...
0
votes
1answer
45 views

Averaging bang for the buck

I'm building a table/graph to compare multiple video cards according to a given benchmark. The benchmark consists of multiple measurements using games. For each benchmark measurement, I build a ...
1
vote
0answers
67 views

When is it appropriate to pool trials within subjects for statistical comparisons?

Usually I have small number of subjects (e.g., 4-6 rodents) and I record electrophysiological activity during presentation of a stimulus or performance of a task. A general question that arises when ...
0
votes
0answers
29 views

Aggregated Means Differ in Regression

I try to run a difference-in-difference model based on single observations which I have to aggregate in order to generate an outcome variable "w.mortf" = mortality. My original data set provides a ...
5
votes
2answers
77 views

Privacy through moving averages?

I am considering the following hypothetical situation: I have a time series of data. In general, 'the public' should have access to features of this data. However, making the time series available ...
3
votes
1answer
171 views

Linear regression on aggregate and missing data

I have data from a survey with $m$ quantitative questions (rating 0 to 100). There was $n$ participants. I want to apply linear regression to predict $X_m$ based on $X_1,\dots,X_{m-1}$. There are, ...
-1
votes
1answer
26 views

How to represent multiple data by single value

I have three inputs x1,x2,x3 and to each single input there are three outputs y1, y2, y3. (1) x1 --y1, y2, y3 (2) x2 --y1, y2, y3 (3) x3 --y1, y2, y3 The Whole whole set has to be represented by ...
0
votes
1answer
108 views

Is it good statistical practice to include an “aggregated” categorical variable in LM?

I am dealing with a large data set (~100000 observations, roughly 13 variables). Most variables are of categorical nature and are stable across different observations concerning the same individual/...
1
vote
2answers
1k views

Weighted Majority Rule

I have to implement a weighted majority vote which is an aggregation technique to generate one final prediction. In fact, several classifiers make local predictions. These are then collected and ...

1 2 3 4