I have a distribution that assigns 3 numbers to each data point instead of one scalar number. These numbers can be e.g. the weight of a person measured at 3 specific ages. (The distribution can of course be though of as 3 different distributions.)
How can I use tests such as the t-test or one way ANOVA with such a distribution? Of course I can perform the test for each of the components and then multiply the F-scores and (separately) the p-values but I wanted to know what is the customary way to do such a test.

  • $\begingroup$ Do a Google search for "MANOVA". That's the general method. However, if your data consists of multiple measurements made on the same person, then "repeated measures" ANOVA may be more specifically tuned to your problem. $\endgroup$ Feb 4 '17 at 23:16
  • $\begingroup$ OK great. Is there also a mutivariate generalization of the paired t-test? $\endgroup$
    – Reza
    Feb 4 '17 at 23:32
  • $\begingroup$ The paired t-test is just a two-way ANOVA without interaction. So MANOVA provides a direct generalization. $\endgroup$ Feb 5 '17 at 0:33
  • $\begingroup$ Oops I meant pairwise t-test. $\endgroup$
    – Reza
    Feb 5 '17 at 18:48


Let's imagine, that your 3 numbers it is just coordinates in 3D space. Than You have a question whether your groups have sense or not (like in usual ANOVA). Idea is well described here and here. My implementation of numeric ANOVA below:

Body of the numeric ANOVA function

MATLAB code:

function F = fAnova( Dataset )
%% INPUT: First 3 columns - coordinates, 4th column - group index   
%% OUTPUT F-statistic

Find unique groups IDs and their GroupIDXs (from 1 to number of groups)

% Get number of different Groups
% Get number of observations, equal to number of rows
% Count number of observations in each group. It is the same as nItems, but
% lets imagine we work not with simulated data
NumberOfObservationsInEachGroup=accumarray(GroupIDXs,Dataset(:,4),[],@(x) numel(x)); 
% Calculate pair-wise distances
% make distance matrix
% leave only low triangle
% Get grouping matrix. 0 when observations from different groups, ones when
% from same group.
% Calculate matrix with number of observations per group. Equal to
% GroupMatrix, but instead of 1 for each group we have number of
% observations.
for iGroup=1:NumberOfGroups
end %iGrooup
% Make zeros for cells from different groups
% Get SS_T
% Get SS_W


% Get SS_A

% Get F

end % Function

Permutations and example

Let's test our function

%Initialize array of 4 groups of 3D points

nItems=[5 7 9 11]; % Number of dots in each group
Dataset=[];        % First 3 columns - coordinates, 4th column - group index       
for iGroup=1:4
    % normrnd(mu,sigma,n,m) generates matrix with n rows and m columns. 
    % Values are normally distributed with parameters mu and sigma 
    % ramrnd(a,b,n,m)  generates matrix with n rows and m columns. Values
    % have gamma distribution with parameters a and b
    Dataset=[Dataset;[coords1, coords2, coords3, GroupIDX]];
end %iGroup

Normalize Your data if it is necessary. Normalization depends from your data. I could not give you any suggestions. Let's normalize to standart deviation

for iColumn=1:3
end %iColumns

Thus, we are ready to get observed F

Fobserved=fAnova( Dataset );

Let's create null-distribution under assumption that groups are distributed randomly. To do it you may sample 4th column with (more conservative) or without (less conservative) replacement.

for iSample=1:10^5
    Fsample(end+1)=fAnova( SampleDataset );
end % iSample

Computed p-value will be number of samples where F-statistic was equal or higher, than observed divide to number of samples +1. We should add 1, as our observation is also "Inside" population.


Now comes some graphic output

fPlotHist(Fobserved,Fsample,100) %fPlotHist(Observed value,Null distribution,number of bins for histogramm)
title(['ANOVA p-value:' num2str(pValueANOVA,2)])
xlabel('F statistic')
legend({'null-distribution','', 'Fsample>=Fobserved','Observed F statistic'},'FontSize',14)

enter image description here

Links to code

ANOVA function


fPlotHist function


You can't

First things first: there are generalizations available for your case, but it's not that straight forward. Let me explain:

What you have is, as it seems, a multivariate distribution. Every data-point consists of several (not completely independent I assume) values. Pay attention by saying:

(The distribution can of course be though of as 3 different distributions.)

NO! You do have one multivariate distribution in 3 dimensions. Of course you can make 3 projections which look at first glance like three distributions, but they do not represent the same data-set as your original distribution! See below for an example. The projections (1-dimensional distributions on the side) look like they are the same, but the 2-dimensional view tells otherwise.

Two-dimensional distribution with projections Completely different two-dimensional distribution with the same projections

And here starts the problem: if you would just use 1-dimensional tests to tell whether the blue and the red one are the same, they would conclude that they are. Which they surely are not!

So why don't just generalize to multi-dimensions? Because some things are not defined in multivariate distributions. For example, there does not exist an order (which does for the 1-dimensional case, you can sort them by their value). That's why comparing multidimensional distributions gets quite tricky.

One of the best answers would be to use machine learning: train a classifier on your data and on your null-hypothesis. Use a good classifier, the better the classifier performs (CV) on the data, the more different they are.

There are also other test outside, more statistically "approved" ones ;) But they are not a direct counter-part to their 1-dimensional smaller brother and it may even is better to use a completely, for multivariate analysis "created" test instead of a generalized 1-dimensional test.

So find out what you really need, it also depends strongly on your data and the problem you are tackling.

  • $\begingroup$ Why do you think that my approach is incorrect. I don't understand it from Your answer. $\endgroup$
    – zlon
    Feb 11 '17 at 21:47
  • $\begingroup$ I don't say your answer is wrong. But I wanted to highlight that the comparison of a multivariate distribution versus the comparison of a one dimensional distribution cannot, in general, be done straight forward but adds some additional implications. He has to be aware of that. $\endgroup$
    – Mayou36
    Feb 14 '17 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.