After some time, I came up with what may be a solution for the problem.

Setup: The data is given by a small collection of $m$-dimensional ellipsoid hyperplane segments in an $n\ge m$-dimensional space - they are just collections of means, eigenvectors and singular values. Each ellipsoid $E_i$ has weight $w_i$. The task is to find $m$ principal components of the data. The ellipsoids' dimensions and positions are large in comparison to the expected singular values for the whole data.

Idea: Form covariance matrices from the ellipsoid data.

Approach: The core of the idea is to replace sums in the PCA with the integrals. Very generally, given a measurable subset $X\subset \Bbb R^n$ and a weight function $w(x)\colon X \to \Bbb R$, we may center $X$ by computing the mean $\mathbf{y} = \int_X w(y)dy$. Assuming the data to be centered, the covariance matrix for $X$ is computed by $Cov(X) = \int_X w(y)yy^Tdy$.

With $X$ now being a finite union of weighted ellipses, centering the data is trivial. The covariance matrix is then given by

$Cov(X) = \sum_i c_i\int_{E_i}(x_i + y)(x_i + y)^Tdy$

where $E_i$ is the $i$'th ellipsis centered at the origin, while $x_i$ is its center and $c_i = \frac{w_i}{\int_{E_i}dy}$ where $w_i$ is the "weight" of the ellipsis (in my case - the number of points from which it was obtained by a honest PCA procedure). Now the most pleasant part: since $E_i$ is symmetric with respect to the origin and $x_i$ is constant, we have $\int_{E_i}x_iy^Tdy=\int_{E_i}yx_i^Tdy=0$. Thus

$Cov(X) = \sum_i (w_ix_i^Tx_i + c_i\int_{E_i} yy^Tdy) = \sum_iw_i(x_ix_i^T + Cov(E_i))$

The covariance matrix of $E_i$ is given by conditions. Basically, this means, that we have established a new covariance matrix, for which principal components may be calculated.

Problem is that, when the data is multidimensional, while we are interested in just a few main components, we still need need to make PCA for the matrix rather then infer them by some nice "averaging" of the eigenvalues for each ellipsis. Though the whole data needs not be stored any more and covariance matrix needs not be recalculated.

For myself, I have found another solution that suits for my particular use-case. But I'll just let this answer stay here, perhaps it may be inspirational.

After some time, I came up with what may be a solution for the problem.

Setup: The data is given by a small collection of $m$-dimensional ellipsoid hyperplane segments in an $n\ge m$-dimensional space - they are just collections of means, eigenvectors and singular values. Each ellipsoid $E_i$ has weight $w_i$. The task is to find $m$ principal components of the data. The ellipsoids' dimensions and positions are large in comparison to the expected singular values for the whole data.

Idea: Form covariance matrices from the ellipsoid data.

Approach: The core of the idea is to replace sums in the PCA with the integrals. Very generally, given a measurable subset $X\subset \Bbb R^n$ and a weight function $w(x)\colon X \to \Bbb R$, we may center $X$ by computing the mean $\mathbf{y} = \int_X w(y)dy$. Assuming the data to be centered, the covariance matrix for $X$ is computed by $Cov(X) = \int_X w(y)yy^Tdy$.

With $X$ now being a finite union of weighted ellipses, centering the data is trivial. The covariance matrix is then given by

$Cov(X) = \sum_i c_i\int_{E_i}(x_i + y)(x_i + y)^Tdy$

where $E_i$ is the $i$'th ellipsis centered at the origin, while $x_i$ is its center and $c_i = \frac{w_i}{\int_{E_i}dy}$ where $w_i$ is the "weight" of the ellipsis (in my case - the number of points from which it was obtained by a honest PCA procedure). Now the most pleasant part: since $E_i$ is symmetric with respect to the origin and $x_i$ is constant, we have $\int_{E_i}x_iy^Tdy=\int_{E_i}yx_i^Tdy=0$. Thus

$Cov(X) = \sum_i (w_ix_ix_i^T + c_i\int_{E_i} yy^Tdy) = \sum_iw_i(x_ix_i^T + Cov(E_i))$

The covariance matrix of $E_i$ is given by conditions. Basically, this means, that we have established a new covariance matrix, for which principal components may be calculated.

Problem is that, when the data is multidimensional, while we are interested in just a few main components, we still need need to make PCA for the matrix rather then infer them by some nice "averaging" of the eigenvalues for each ellipsis. Though the whole data needs not be stored any more and covariance matrix needs not be recalculated.

For myself, I have found another solution that suits for my particular use-case. But I'll just let this answer stay here, perhaps it may be inspirational.

After some time, I came up with what may be a solution for the problem.

Setup: The data is given by a small collection of $m$-dimensional ellipsoid hyperplane segments in an $n\ge m$-dimensional space - they are just collections of means, eigenvectors and singular values. Each ellipsoid $E_i$ has weight $w_i$. The task is to find $m$ principal components of the data. The ellipsoids' dimensions and positions are large in comparison to the expected singular values for the whole data.

Idea: Form covariance matrices from the ellipsoid data.

Approach: The core of the idea is to replace sums in the PCA with the integrals. Very generally, given a measurable subset $X\subset \Bbb R^n$ and a weight function $w(x)\colon X \to \Bbb R$, we may center $X$ by computing the mean $\mathbf{y} = \int_X w(y)dy$. Assuming the data to be centered, the covariance matrix for $X$ is computed by $Cov(X) = \int_X w(y)y^Tydy$.

With $X$ now being a finite union of weighted ellipses, centering the data is trivial. The covariance matrix is then given by

$Cov(X) = \sum_i c_i\int_{E_i}(x_i + y)^T(x_i + y)dy$

where $E_i$ is the $i$'th ellipsis centered at the origin, while $x_i$ is its center and $c_i = \frac{w_i}{\int_{E_i}dy}$ where $w_i$ is the "weight" of the ellipsis (in my case - the number of points from which it was obtained by a honest PCA procedure). Now the most pleasant part: since $E_i$ is symmetric with respect to the origin and $x_i$ is constant, we have $\int_{E_i}x_i^Tydy=0$. Thus

$Cov(X) = \sum_i (w_ix_i^Tx_i + c_i\int_{E_i} y^Tydy) = \sum_iw_i(x_i^Tx_i + Cov(E_i))$

The covariance matrix of $E_i$ is given by conditions. Basically, this means, that we have established a new covariance matrix, for which principal components may be calculated.

Problem is that, when the data is multidimensional, while we are interested in just a few main components, we still need need to make PCA for the matrix rather then infer them by some nice "averaging" of the eigenvalues for each ellipsis. Though the whole data needs not be stored any more and covariance matrix needs not be recalculated.

For myself, I have found another solution that suits for my particular use-case. But I'll just let this answer stay here, perhaps it may be inspirational.

After some time, I came up with what may be a solution for the problem.

Setup: The data is given by a small collection of $m$-dimensional ellipsoid hyperplane segments in an $n\ge m$-dimensional space - they are just collections of means, eigenvectors and singular values. Each ellipsoid $E_i$ has weight $w_i$. The task is to find $m$ principal components of the data. The ellipsoids' dimensions and positions are large in comparison to the expected singular values for the whole data.

Idea: Form covariance matrices from the ellipsoid data.

Approach: The core of the idea is to replace sums in the PCA with the integrals. Very generally, given a measurable subset $X\subset \Bbb R^n$ and a weight function $w(x)\colon X \to \Bbb R$, we may center $X$ by computing the mean $\mathbf{y} = \int_X w(y)dy$. Assuming the data to be centered, the covariance matrix for $X$ is computed by $Cov(X) = \int_X w(y)yy^Tdy$.

With $X$ now being a finite union of weighted ellipses, centering the data is trivial. The covariance matrix is then given by

$Cov(X) = \sum_i c_i\int_{E_i}(x_i + y)(x_i + y)^Tdy$

where $E_i$ is the $i$'th ellipsis centered at the origin, while $x_i$ is its center and $c_i = \frac{w_i}{\int_{E_i}dy}$ where $w_i$ is the "weight" of the ellipsis (in my case - the number of points from which it was obtained by a honest PCA procedure). Now the most pleasant part: since $E_i$ is symmetric with respect to the origin and $x_i$ is constant, we have $\int_{E_i}x_iy^Tdy=\int_{E_i}yx_i^Tdy=0$. Thus

$Cov(X) = \sum_i (w_ix_i^Tx_i + c_i\int_{E_i} yy^Tdy) = \sum_iw_i(x_ix_i^T + Cov(E_i))$

The covariance matrix of $E_i$ is given by conditions. Basically, this means, that we have established a new covariance matrix, for which principal components may be calculated.

Problem is that, when the data is multidimensional, while we are interested in just a few main components, we still need need to make PCA for the matrix rather then infer them by some nice "averaging" of the eigenvalues for each ellipsis. Though the whole data needs not be stored any more and covariance matrix needs not be recalculated.

For myself, I have found another solution that suits for my particular use-case. But I'll just let this answer stay here, perhaps it may be inspirational.