# Tests of significance for vector-valued distributions

I have a distribution that assigns 3 numbers to each data point instead of one scalar number. These numbers can be e.g. the weight of a person measured at 3 specific ages. (The distribution can of course be though of as 3 different distributions.)
How can I use tests such as the t-test or one way ANOVA with such a distribution? Of course I can perform the test for each of the components and then multiply the F-scores and (separately) the p-values but I wanted to know what is the customary way to do such a test.

• Do a Google search for "MANOVA". That's the general method. However, if your data consists of multiple measurements made on the same person, then "repeated measures" ANOVA may be more specifically tuned to your problem. – Gordon Smyth Feb 4 '17 at 23:16
• OK great. Is there also a mutivariate generalization of the paired t-test? – Reza Feb 4 '17 at 23:32
• The paired t-test is just a two-way ANOVA without interaction. So MANOVA provides a direct generalization. – Gordon Smyth Feb 5 '17 at 0:33
• Oops I meant pairwise t-test. – Reza Feb 5 '17 at 18:48

# Principle

Let's imagine, that your 3 numbers it is just coordinates in 3D space. Than You have a question whether your groups have sense or not (like in usual ANOVA). Idea is well described here and here. My implementation of numeric ANOVA below:

# Body of the numeric ANOVA function

MATLAB code:

function F = fAnova( Dataset )
%% INPUT: First 3 columns - coordinates, 4th column - group index
%% OUTPUT F-statistic


Find unique groups IDs and their GroupIDXs (from 1 to number of groups)

[GroupLevels,~,GroupIDXs]=unique(Dataset(:,4));
% Get number of different Groups
NumberOfGroups=numel(GroupLevels);
% Get number of observations, equal to number of rows
NumberOfObservations=size(Dataset,1);
% Count number of observations in each group. It is the same as nItems, but
% lets imagine we work not with simulated data
NumberOfObservationsInEachGroup=accumarray(GroupIDXs,Dataset(:,4),[],@(x) numel(x));
% Calculate pair-wise distances
PairWiseDistances=pdist(Dataset(:,1:3));
% make distance matrix
DistMatrix=squareform(pdist(Dataset(:,1:3)));
% leave only low triangle
DistMatrix=tril(DistMatrix);
% Get grouping matrix. 0 when observations from different groups, ones when
% from same group.
GroupMatrix=repmat(Dataset(:,4)',NumberOfObservations,1)==repmat(Dataset(:,4),1,NumberOfObservations);
% Calculate matrix with number of observations per group. Equal to
% GroupMatrix, but instead of 1 for each group we have number of
% observations.
NumberOfObservationsMatrixRedundant=[];
for iGroup=1:NumberOfGroups
NumberOfObservationsMatrixRedundant=[NumberOfObservationsMatrixRedundant,repmat(NumberOfObservationsInEachGroup(iGroup),NumberOfObservations,NumberOfObservationsInEachGroup(iGroup))];
end %iGrooup
% Make zeros for cells from different groups
NumberOfObservationsMatrix=NumberOfObservationsMatrixRedundant.*GroupMatrix;
%Redundant
% Get SS_T
SSt=sum(sum(DistMatrix.^2))/NumberOfObservations;
% Get SS_W

SSw=nansum(nansum(((DistMatrix.*GroupMatrix).^2)./NumberOfObservationsMatrix));

% Get SS_A
SSa=SSt-SSw;

% Get F
F=abs((SSa/(NumberOfGroups-1))/(SSw/(NumberOfObservations-NumberOfGroups)));

end % Function


# Permutations and example

Let's test our function

%Initialize array of 4 groups of 3D points

nItems=[5 7 9 11]; % Number of dots in each group
Dataset=[];        % First 3 columns - coordinates, 4th column - group index
for iGroup=1:4
% normrnd(mu,sigma,n,m) generates matrix with n rows and m columns.
% Values are normally distributed with parameters mu and sigma
coords1=normrnd(iGroup,1,nItems(iGroup),1);
coords2=abs(log10(normrnd(iGroup,1,nItems(iGroup),1)));
% ramrnd(a,b,n,m)  generates matrix with n rows and m columns. Values
% have gamma distribution with parameters a and b
coords3=gamrnd(iGroup,1,nItems(iGroup),1);
GroupIDX=repmat(iGroup,nItems(iGroup),1);
Dataset=[Dataset;[coords1, coords2, coords3, GroupIDX]];
end %iGroup


Normalize Your data if it is necessary. Normalization depends from your data. I could not give you any suggestions. Let's normalize to standart deviation

for iColumn=1:3
Dataset(:,iColumn)=Dataset(:,iColumn)./std(Dataset(:,iColumn));
end %iColumns


Thus, we are ready to get observed F

Fobserved=fAnova( Dataset );


Let's create null-distribution under assumption that groups are distributed randomly. To do it you may sample 4th column with (more conservative) or without (less conservative) replacement.

Fsample=[];
SampleDataset=Dataset;
for iSample=1:10^5
SampleDataset(:,4)=datasample(Dataset(:,4),length(Dataset(:,4)),'Replace',true);
Fsample(end+1)=fAnova( SampleDataset );
end % iSample


Computed p-value will be number of samples where F-statistic was equal or higher, than observed divide to number of samples +1. We should add 1, as our observation is also "Inside" population.

pValueANOVA=sum(Fsample>=Fobserved)/(numel(Fsample)+1);


Now comes some graphic output

fPlotHist(Fobserved,Fsample,100) %fPlotHist(Observed value,Null distribution,number of bins for histogramm)
title(['ANOVA p-value:' num2str(pValueANOVA,2)])
xlabel('F statistic')
ylabel('Counts')
legend({'null-distribution','', 'Fsample>=Fobserved','Observed F statistic'},'FontSize',14)
set(gca,'FontSize',14)


ANOVA function

Example

fPlotHist function

You can't

First things first: there are generalizations available for your case, but it's not that straight forward. Let me explain:

What you have is, as it seems, a multivariate distribution. Every data-point consists of several (not completely independent I assume) values. Pay attention by saying:

(The distribution can of course be though of as 3 different distributions.)

NO! You do have one multivariate distribution in 3 dimensions. Of course you can make 3 projections which look at first glance like three distributions, but they do not represent the same data-set as your original distribution! See below for an example. The projections (1-dimensional distributions on the side) look like they are the same, but the 2-dimensional view tells otherwise.

And here starts the problem: if you would just use 1-dimensional tests to tell whether the blue and the red one are the same, they would conclude that they are. Which they surely are not!

So why don't just generalize to multi-dimensions? Because some things are not defined in multivariate distributions. For example, there does not exist an order (which does for the 1-dimensional case, you can sort them by their value). That's why comparing multidimensional distributions gets quite tricky.

One of the best answers would be to use machine learning: train a classifier on your data and on your null-hypothesis. Use a good classifier, the better the classifier performs (CV) on the data, the more different they are.

There are also other test outside, more statistically "approved" ones ;) But they are not a direct counter-part to their 1-dimensional smaller brother and it may even is better to use a completely, for multivariate analysis "created" test instead of a generalized 1-dimensional test.

So find out what you really need, it also depends strongly on your data and the problem you are tackling.

• Why do you think that my approach is incorrect. I don't understand it from Your answer. – zlon Feb 11 '17 at 21:47
• I don't say your answer is wrong. But I wanted to highlight that the comparison of a multivariate distribution versus the comparison of a one dimensional distribution cannot, in general, be done straight forward but adds some additional implications. He has to be aware of that. – Mayou36 Feb 14 '17 at 13:05