Questions tagged [spectral-analysis]
For questions involving spectral clustering algorithms, frequency domain analysis or correlated subjects. May include Fourier transform and graph theoretic questions.
202
questions
0
votes
0
answers
32
views
PLS Regression on data with high number of zeros in dependent variable
I want to perform a PLS regression on a data set coming from spectral images (NIRS). My goal is to relate the different spectra to the total amount of a compound. To do this, I have a dataset ...
- 31
0
votes
0
answers
20
views
Interpretation of time series spectral entropy values wrt forecastability by a general neural network
I recently started using spectral entropy to analyze time series (already windowed). I'm having difficulty for interpreting the results, the entropy of the last 25% of a series is 0.19, and the ...
- 51
0
votes
0
answers
11
views
What is the spectral density matrix, in the context of vector stochastic processes
I cannot seem to find any good/easy to read resources on the spectral theory, and in particular for multivariate stochastic processes.
I want to know:
Any resources explaining spectral theory
...
- 359
0
votes
0
answers
34
views
Singular Spectrum Analysis Decomposition on single input signal using PyTS module
I read this paper and was curious to apply it on a single-channel audio recording of mixed sources. It is about Singular Spectrum Analysis (SSA).
The paper mentions that a key component of the ...
0
votes
0
answers
15
views
Opposite facing loadings of seemingly the same features
I am performing PCA analysis of NIR spectra for the analysis of the progression of a film coating process
I have noticed that one of my components goes up with the film coating process (with upwards ...
- 1
5
votes
3
answers
460
views
Hypothesis testing for detecting a (damped) sinusoidal signal in noise
I have a signal in white noise that has the following form:
\begin{equation}
r[t_i] = A e^{-t_i/\tau} \sin{(\omega t_i + \phi)} + n[t_i]
\end{equation}
I would like to test whether the signal (1st ...
- 161
0
votes
0
answers
29
views
Is Fisher transformation appropriate for magnitude squared coherence and phase locking value?
Fisher transformation, or hyperbolic arctangent, is recommend before performing arithmetic on correlation coefficients (e.g., estimating confidence intervals), because it makes their distribution ...
- 83
1
vote
0
answers
91
views
The logic behind the derivation of the autocovariance generating function?
I am working with complicated sets of time series, where an observed system output is the sum of a number of individual ARMA, error and noise series. I am therefore looking at ways to identify these ...
- 704
1
vote
1
answer
41
views
Availability of Linear Grouping Algorithms to Linearly Cluster Datasets
I have been trying to cluster a scatter plot that has a triangular graph, ideally the proper clustering plot should have a linear form, as shown below:
I tried using Spectral Clustering:
and ...
- 39
0
votes
0
answers
20
views
The sampling frequency or sampling period for financial time series in doing DFT
The discrete Fourier transform (DFT) is widely utilized in computer engineering, and its formula is as follows:
$$X(k)=\sum_{n=0}^{N-1}x[n]e^{-i\frac{2\pi}{N}nk},$$
where the result $X(k)$ refers to ...
- 53
0
votes
0
answers
64
views
Efficient way to encode a set of large covariance matrices
I have a computational model that involves having a set of $K$ covariance matrices, $\{\Sigma_1, ..., \Sigma_K\}$ with each $\Sigma_i \in R^{n \times n}$. Storing all these full covariance matrices is ...
- 1,290
0
votes
0
answers
18
views
How to deal with correlation in spectral data?
Currently I'm tasked with analysing data taken with optical emission spectroscopy (OES).
These measurements serve the purpose of identifying regions on the sample rich in certain chemical elements (...
- 175
0
votes
0
answers
26
views
Find spectral density of a process
Let $X_t = C \cos(2πw t) + D \sin(2πwt) + U_t$ where $C, D$ and $U_t$ are Gaussian random variables with unit variance, and where all the variables are independent, and $U_t$ is white noise. Determine ...
- 435
0
votes
0
answers
42
views
Energy of a signal contained within a given band
Suppose I wish to compute the energy contained in a given frequency range using FFT. So, first, I compute FFT and take the squares of the absolute values of $x_k$. Can I then use discrete integration ...
- 111
1
vote
0
answers
60
views
FFT: zero-padding and mean subtraction
When computing FFT of a finite-time discrete sample with a non-zero mean, I get two different results depending on the order of operations: whether I zero-pad and then subtract the mean or subtract ...
- 111
5
votes
1
answer
291
views
Why is spectral analysis based on autocovariance function instead of in the original sequence?
I have some questions about the spectral analysis in time series. For a zero-mean and covariance stationary time series $\{X_t\}$ with autocovariance function $\gamma(h)=Cov(X_{t+h},X_t)$. The $\...
- 53
2
votes
1
answer
753
views
What does UMAP do on a 3d data?
I have a high dimensional data and when I applied UMAP on the whole data, it didn't seem to find a low dimensional manifold. Out of curiosity, I chose 3 important features and applied UMAP on that ...
- 93
1
vote
0
answers
100
views
Is maximal spectral entropy of residuals a poor loss function because phase information is lost?
Suppose I define a custom loss function SpectralEntropy as follows:
...
- 9,522
1
vote
0
answers
31
views
What is the probability that multiple spectral lines are just noise?
In spectroscopy spectral line emissions are related to each other, with fixed widths and heights that correlate with all lines based on fixed physical conditions. If I detect one spectral line at a 3 ...
- 11
1
vote
0
answers
18
views
Are the classical moments consistently estimated from a single realization drawn from a given PSD?
Given a sequence $\{x_k\}_{k=-N}^{N}$ having power spectral density $S(f)$, we know that that "single realization PSD"
$$
\frac{\Delta t^2}{T} \left| \sum_{k=-N}^{N} x_n \exp(-2\pi i f n \...
- 215
1
vote
0
answers
28
views
Obtaining frequency content of time sequence
I have a long sequence of time values (instants in time where events happened) and I would like to detect perodicities in this data. It's possible to sample these points onto a gigantic grid and take ...
- 529
0
votes
0
answers
31
views
Building up to the periodogram
I have seen in multiple sources that if we start with writing a time series as follows (assuming $n$ is odd):
$$x_t = a_o + \sum_{j=1}^{(n-1)/2}{[a_jcos(2\pi tj/n) + b_jsin(2\pi tj/n)]}$$
Then we will ...
- 318
1
vote
0
answers
37
views
Discover a pattern in a set of spectrograms given a real rating
I'm trying to find patterns in a sorted set of spectrograms, each with a rating. The main idea is to train the CNN in such a way that it understands how the pattern "evolves" at each step, ...
- 11
1
vote
0
answers
54
views
PCA to identify patterns in the data, forced to a particular variable?
Dataset: I have a hyperspectral dataset that consists 250 wavelength bands (x1,...x250) and corresponding reflectance measurements (y) for each band. Plotting X vs Y yields a spectral profile. I have ...
- 11
0
votes
0
answers
15
views
multitaperTrend function in R
How would you determine the "B" in multitaperTrend function?
I have been trying to estimate trend for Southern Oscillation index data using ...
1
vote
0
answers
30
views
Fast Approximate Spectral Clustering in Practice?
Spectral clustering takes $O(n^3)$ time. Over the last fifteen years, a huge number of heuristics have been published on speeding up spectral clustering (e.g. Nystrom method, PCA, sample size ...
- 21
2
votes
1
answer
205
views
Spectral Graph Convolutions: What are the spectral filters functions
I am trying to understand the mathematical meaning of one of the steps that appear in the Convolution Theorem (Step 4 here).
To give some context, this is related to applying the convolution theorem ...
- 298
1
vote
1
answer
192
views
Why is coherence of this wavelet transform almost always near 1?
I'm trying to understand the different aspects of a wavelet transform. Wavelet power has made enough sense to me as an analogy of the covariance. However, the wavelet coherence does not make sense to ...
- 233
1
vote
0
answers
99
views
What is the sampling rate of quarterly time series data?
I am doing spectral analysis of quarterly data. I am very new to spectral analysis and have been using Shumway & Stoffer's textbook as a guide, but I am not understanding exactly what is going on.
...
- 21
1
vote
0
answers
80
views
The length of spectral density is longer than the data using spectrum() in R
I'm using spectrum(method = "pgram") in R to calculate the spectral density in my time series.
spectrum() returns the spectral density for each frequency(from 1/n, 2/n to 1/2, n is the time ...
- 41
1
vote
0
answers
148
views
External loss functions for Spectral/Density-based clustering
In this article, Abou-Mustafa and Schuurmans proposed a method that makes it easy to decide what unsupervised learning algorithm generalizes 'better' to the entire dataset. In particular, this needs ...
- 11
0
votes
0
answers
69
views
How to compute significance between powers of spectrum components (Fourier frequencies) of two EEG signals
We have two EEG signals, and ran standard NeuroDSP Spectral Analyses to compute the power spectral densities of both signals. Moreover, because NeuroDSP uses standard Welch’s spectrogram and averages ...
- 101
0
votes
0
answers
21
views
Normalization, centering and PCA [duplicate]
I have a feature matrix composed of frequency responses (in dB) from individual acoustic events. Frequencies in the columns, events in the rows and the matrix is the response
The responses decrease ...
- 268
1
vote
0
answers
219
views
Relation between Fourier transforms and coherence of signals
My overall aim is to compare the edges of two images by comparing their Fourier Transforms (FFT) and to calculate one number as a key performance indicator that describes how much they are similar to ...
1
vote
0
answers
37
views
Identifying a single dominant number with high probability in a data point array of 4 to 20 numbers
Example readings, and second graph identified Actual Number I am looking for
EDIT: These numbers on the left are period time values in micro seconds (us) in the "time domain" and they need ...
- 11
2
votes
1
answer
2k
views
How to translate eigenvectors and eigenvalues to the number of clusters in spectral clustering?
I have generated this output, where L is the Laplacian Matrix, D is the degree and A is the adjacency matrix:
I can see the eigenvalues and eigenvectors are returned. I am unsure how to interpret ...
- 83
0
votes
0
answers
98
views
How to Understand Autoregressive Process MATLAB Code?
This is supposed to be a code to calculate the true PSD of a 4th order autoregressive process:
...
- 1
0
votes
1
answer
101
views
Finding autocorrelation coefficients given PSD values at 2 frequencies
Assuming that $S_X(w)$ denotes powers spectral density function at
frequency $w$, we are given
$$S_X\left(\frac{\pi}{4}\right)=10+3\sqrt{2},\quad
S_X\left(\frac{\pi}{6}\right)=11+3\sqrt{3}.$$ We also ...
- 105
2
votes
0
answers
110
views
Reference request: (spectral) convergence rate of sample covariance matrix with fixed dimension $p$
I am looking for a reference on convergence of sample covariance matrix (in some reasonable sense) when the dimension $p$ is fixed, but the number of samples $n$ goes to infinity. The ideal result I ...
- 21
0
votes
0
answers
51
views
Frequency of timeseries greater than half the number of datapoints in timeseries
as suggested in the following thread:
Period detection of a generic time series
I'm testing the function findfrequency() to automatize the estimation of the period ...
- 111
2
votes
0
answers
954
views
Why does fast graph convolution need Chebyshev polynomials?
I'm reading the paper Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering and find it difficult to understand the motivation for using Chebyshev polynomials.
With localized ...
- 153
1
vote
2
answers
3k
views
How to calculate the expected value of a time series just from the data
in this question Can stationary time series contain regulary cycles and periods with different fluctuations I was told that stationary time series do not have regular cycles and that having constant ...
- 392
4
votes
1
answer
1k
views
Solver for the true auto-covariance function in AR(p)
Suppose I have the following $AR(p)$ model.
$$X_t = \sum_{i=1}^{p} \phi_i X_{t-i} + \epsilon_t\,, $$
where $\epsilon_t$ has mean 0 variance $\sigma^2$. I am in the situation where the $\phi$s are ...
- 15.9k
2
votes
1
answer
1k
views
Coherence using FFT: how to calculate coherence for one frame
I want to calculate coherence between two time series that are of equal lengths.
Since coherence is given by
Pxy/(sqrt(Pxx)*sqrt(Pyy))
, I did the following steps.
Step 1: Divide both time series (...
- 376
5
votes
1
answer
249
views
Probability density from Hilbert-Schmidt integral operator
The Hilbert-Schmidt integral operator determines the underlying measure, if a universal kernel is used. Now, do eigenvalues of the Hilbert-Schmidt integral operator determine the underlying measure up ...
- 181
1
vote
0
answers
33
views
Are people still researching the use of spectral decomposition on finite groups for data analysis?
In A GENERALIZATION OF SPECTRAL ANALYSIS WITH APPLICATION TO RANKED DATA (Diaconis 1989), the author discusses a dataset of election results. There were 5 candidates, and each voter was asked to rank ...
- 393
2
votes
1
answer
123
views
K means clustering breakup---galaxy spectrum data set
I have a spectrum data set (total 22000). Similar to an electronic wave data, two dimensional (Flux vs Wavelength). A typical set of wavelength plot looks like below
Now I am doing kmeans on this ...
- 171
2
votes
1
answer
406
views
Years as continuous variable [duplicate]
Can I use "years" as a continuous variable ("years" as calendar years from 1984 to 2014) to see if NDVI (normalized difference vegetation index), of the same area at the same time (...
- 21
5
votes
1
answer
3k
views
Are colored noises correlated / uncorrelated?
Let, $x$ be a random variable (r.v) that is white Gaussian, has a flat power spectrum. $y$ can be any colored noise. I think another term for uncorrelated is i.i.d (identically and independently ...
- 231
2
votes
1
answer
646
views
For which clustering algorithms is the Gap statistic useful?
How can i know for which clustering algorithms (with a parameter that represents number of clusters) it makes sense to use the Gap statistic? I've read in the paper by Tibshirani, Walter & Hastie ...
- 451