Skip to main content

Questions tagged [spectral-analysis]

For questions involving spectral clustering algorithms, frequency domain analysis or correlated subjects. May include Fourier transform and graph theoretic questions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Interpretation of time series spectral entropy values wrt forecastability by a general neural network

I recently started using spectral entropy to analyze time series (already windowed). I'm having difficulty for interpreting the results, the entropy of the last 25% of a series is 0.19, and the ...
Marco's user avatar
  • 51
0 votes
0 answers
10 views

What is the spectral density matrix, in the context of vector stochastic processes

I cannot seem to find any good/easy to read resources on the spectral theory, and in particular for multivariate stochastic processes. I want to know: Any resources explaining spectral theory ...
Dylan Dijk's user avatar
0 votes
0 answers
23 views

Singular Spectrum Analysis Decomposition on single input signal using PyTS module

I read this paper and was curious to apply it on a single-channel audio recording of mixed sources. It is about Singular Spectrum Analysis (SSA). The paper mentions that a key component of the ...
user3320707's user avatar
0 votes
0 answers
14 views

Opposite facing loadings of seemingly the same features

I am performing PCA analysis of NIR spectra for the analysis of the progression of a film coating process I have noticed that one of my components goes up with the film coating process (with upwards ...
phil27's user avatar
  • 1
5 votes
3 answers
437 views

Hypothesis testing for detecting a (damped) sinusoidal signal in noise

I have a signal in white noise that has the following form: \begin{equation} r[t_i] = A e^{-t_i/\tau} \sin{(\omega t_i + \phi)} + n[t_i] \end{equation} I would like to test whether the signal (1st ...
P. Egli's user avatar
  • 161
0 votes
0 answers
24 views

Is Fisher transformation appropriate for magnitude squared coherence and phase locking value?

Fisher transformation, or hyperbolic arctangent, is recommend before performing arithmetic on correlation coefficients (e.g., estimating confidence intervals), because it makes their distribution ...
Neuromancer's user avatar
1 vote
0 answers
75 views

The logic behind the derivation of the autocovariance generating function?

I am working with complicated sets of time series, where an observed system output is the sum of a number of individual ARMA, error and noise series. I am therefore looking at ways to identify these ...
hydrologist's user avatar
1 vote
1 answer
40 views

Availability of Linear Grouping Algorithms to Linearly Cluster Datasets

I have been trying to cluster a scatter plot that has a triangular graph, ideally the proper clustering plot should have a linear form, as shown below: I tried using Spectral Clustering: and ...
NOT-A-CS-GUY's user avatar
0 votes
0 answers
20 views

The sampling frequency or sampling period for financial time series in doing DFT

The discrete Fourier transform (DFT) is widely utilized in computer engineering, and its formula is as follows: $$X(k)=\sum_{n=0}^{N-1}x[n]e^{-i\frac{2\pi}{N}nk},$$ where the result $X(k)$ refers to ...
X.H.'s user avatar
  • 53
0 votes
0 answers
42 views

Efficient way to encode a set of large covariance matrices

I have a computational model that involves having a set of $K$ covariance matrices, $\{\Sigma_1, ..., \Sigma_K\}$ with each $\Sigma_i \in R^{n \times n}$. Storing all these full covariance matrices is ...
dherrera's user avatar
  • 1,268
0 votes
0 answers
16 views

How to deal with correlation in spectral data?

Currently I'm tasked with analysing data taken with optical emission spectroscopy (OES). These measurements serve the purpose of identifying regions on the sample rich in certain chemical elements (...
YPOC's user avatar
  • 165
0 votes
0 answers
21 views

Spectral analysis of timestamps

I have a dataset that contains only the timestamps of occurring events, i.e. I have for ~10^5 events the millisecond-precise time when the respective event happened (i.e. the data is unary, containing ...
lrb's user avatar
  • 1
0 votes
0 answers
26 views

Find spectral density of a process

Let $X_t = C \cos(2πw t) + D \sin(2πwt) + U_t$ where $C, D$ and $U_t$ are Gaussian random variables with unit variance, and where all the variables are independent, and $U_t$ is white noise. Determine ...
Kilkik's user avatar
  • 355
0 votes
0 answers
41 views

Energy of a signal contained within a given band

Suppose I wish to compute the energy contained in a given frequency range using FFT. So, first, I compute FFT and take the squares of the absolute values of $x_k$. Can I then use discrete integration ...
Dmitry's user avatar
  • 111
1 vote
0 answers
44 views

FFT: zero-padding and mean subtraction

When computing FFT of a finite-time discrete sample with a non-zero mean, I get two different results depending on the order of operations: whether I zero-pad and then subtract the mean or subtract ...
Dmitry's user avatar
  • 111
5 votes
1 answer
255 views

Why is spectral analysis based on autocovariance function instead of in the original sequence?

I have some questions about the spectral analysis in time series. For a zero-mean and covariance stationary time series $\{X_t\}$ with autocovariance function $\gamma(h)=Cov(X_{t+h},X_t)$. The $\...
X.H.'s user avatar
  • 53
2 votes
1 answer
655 views

What does UMAP do on a 3d data?

I have a high dimensional data and when I applied UMAP on the whole data, it didn't seem to find a low dimensional manifold. Out of curiosity, I chose 3 important features and applied UMAP on that ...
mathlover's user avatar
1 vote
0 answers
88 views

Is maximal spectral entropy of residuals a poor loss function because phase information is lost?

Suppose I define a custom loss function SpectralEntropy as follows: ...
Galen's user avatar
  • 8,784
1 vote
0 answers
29 views

What is the probability that multiple spectral lines are just noise?

In spectroscopy spectral line emissions are related to each other, with fixed widths and heights that correlate with all lines based on fixed physical conditions. If I detect one spectral line at a 3 ...
Loperh's user avatar
  • 11
1 vote
0 answers
18 views

Are the classical moments consistently estimated from a single realization drawn from a given PSD?

Given a sequence $\{x_k\}_{k=-N}^{N}$ having power spectral density $S(f)$, we know that that "single realization PSD" $$ \frac{\Delta t^2}{T} \left| \sum_{k=-N}^{N} x_n \exp(-2\pi i f n \...
user14717's user avatar
  • 215
1 vote
0 answers
28 views

Obtaining frequency content of time sequence

I have a long sequence of time values (instants in time where events happened) and I would like to detect perodicities in this data. It's possible to sample these points onto a gigantic grid and take ...
Mastiff's user avatar
  • 529
0 votes
0 answers
31 views

Building up to the periodogram

I have seen in multiple sources that if we start with writing a time series as follows (assuming $n$ is odd): $$x_t = a_o + \sum_{j=1}^{(n-1)/2}{[a_jcos(2\pi tj/n) + b_jsin(2\pi tj/n)]}$$ Then we will ...
benja616's user avatar
  • 303
1 vote
0 answers
37 views

Discover a pattern in a set of spectrograms given a real rating

I'm trying to find patterns in a sorted set of spectrograms, each with a rating. The main idea is to train the CNN in such a way that it understands how the pattern "evolves" at each step, ...
dustedduke's user avatar
1 vote
0 answers
53 views

PCA to identify patterns in the data, forced to a particular variable?

Dataset: I have a hyperspectral dataset that consists 250 wavelength bands (x1,...x250) and corresponding reflectance measurements (y) for each band. Plotting X vs Y yields a spectral profile. I have ...
hcrawford's user avatar
0 votes
0 answers
15 views

multitaperTrend function in R

How would you determine the "B" in multitaperTrend function? I have been trying to estimate trend for Southern Oscillation index data using ...
Yilma Tefera's user avatar
1 vote
0 answers
30 views

Fast Approximate Spectral Clustering in Practice?

Spectral clustering takes $O(n^3)$ time. Over the last fifteen years, a huge number of heuristics have been published on speeding up spectral clustering (e.g. Nystrom method, PCA, sample size ...
Timothy Chu's user avatar
2 votes
1 answer
199 views

Spectral Graph Convolutions: What are the spectral filters functions

I am trying to understand the mathematical meaning of one of the steps that appear in the Convolution Theorem (Step 4 here). To give some context, this is related to applying the convolution theorem ...
Gonzalo Polo's user avatar
1 vote
1 answer
170 views

Why is coherence of this wavelet transform almost always near 1?

I'm trying to understand the different aspects of a wavelet transform. Wavelet power has made enough sense to me as an analogy of the covariance. However, the wavelet coherence does not make sense to ...
Mark Miller's user avatar
1 vote
0 answers
92 views

What is the sampling rate of quarterly time series data?

I am doing spectral analysis of quarterly data. I am very new to spectral analysis and have been using Shumway & Stoffer's textbook as a guide, but I am not understanding exactly what is going on. ...
username97's user avatar
1 vote
0 answers
75 views

The length of spectral density is longer than the data using spectrum() in R

I'm using spectrum(method = "pgram") in R to calculate the spectral density in my time series. spectrum() returns the spectral density for each frequency(from 1/n, 2/n to 1/2, n is the time ...
Fuhan YANG's user avatar
1 vote
0 answers
130 views

External loss functions for Spectral/Density-based clustering

In this article, Abou-Mustafa and Schuurmans proposed a method that makes it easy to decide what unsupervised learning algorithm generalizes 'better' to the entire dataset. In particular, this needs ...
drommedaris's user avatar
0 votes
0 answers
60 views

How to compute significance between powers of spectrum components (Fourier frequencies) of two EEG signals

We have two EEG signals, and ran standard NeuroDSP Spectral Analyses to compute the power spectral densities of both signals. Moreover, because NeuroDSP uses standard Welch’s spectrogram and averages ...
rth's user avatar
  • 101
0 votes
0 answers
21 views

Normalization, centering and PCA [duplicate]

I have a feature matrix composed of frequency responses (in dB) from individual acoustic events. Frequencies in the columns, events in the rows and the matrix is the response The responses decrease ...
Tino D's user avatar
  • 236
1 vote
0 answers
208 views

Relation between Fourier transforms and coherence of signals

My overall aim is to compare the edges of two images by comparing their Fourier Transforms (FFT) and to calculate one number as a key performance indicator that describes how much they are similar to ...
MolineraNegra's user avatar
1 vote
0 answers
37 views

Identifying a single dominant number with high probability in a data point array of 4 to 20 numbers

Example readings, and second graph identified Actual Number I am looking for EDIT: These numbers on the left are period time values in micro seconds (us) in the "time domain" and they need ...
TommyS's user avatar
  • 11
2 votes
1 answer
1k views

How to translate eigenvectors and eigenvalues to the number of clusters in spectral clustering?

I have generated this output, where L is the Laplacian Matrix, D is the degree and A is the adjacency matrix: I can see the eigenvalues and eigenvectors are returned. I am unsure how to interpret ...
Ioannes's user avatar
  • 83
0 votes
0 answers
96 views

How to Understand Autoregressive Process MATLAB Code?

This is supposed to be a code to calculate the true PSD of a 4th order autoregressive process: ...
rose_t's user avatar
  • 1
0 votes
1 answer
99 views

Finding autocorrelation coefficients given PSD values at 2 frequencies

Assuming that $S_X(w)$ denotes powers spectral density function at frequency $w$, we are given $$S_X\left(\frac{\pi}{4}\right)=10+3\sqrt{2},\quad S_X\left(\frac{\pi}{6}\right)=11+3\sqrt{3}.$$ We also ...
User32563's user avatar
  • 105
2 votes
0 answers
99 views

Reference request: (spectral) convergence rate of sample covariance matrix with fixed dimension $p$

I am looking for a reference on convergence of sample covariance matrix (in some reasonable sense) when the dimension $p$ is fixed, but the number of samples $n$ goes to infinity. The ideal result I ...
Derpsilon's user avatar
0 votes
0 answers
51 views

Frequency of timeseries greater than half the number of datapoints in timeseries

as suggested in the following thread: Period detection of a generic time series I'm testing the function findfrequency() to automatize the estimation of the period ...
nonoDa's user avatar
  • 111
2 votes
0 answers
916 views

Why does fast graph convolution need Chebyshev polynomials?

I'm reading the paper Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering and find it difficult to understand the motivation for using Chebyshev polynomials. With localized ...
youkaichao's user avatar
1 vote
2 answers
3k views

How to calculate the expected value of a time series just from the data

in this question Can stationary time series contain regulary cycles and periods with different fluctuations I was told that stationary time series do not have regular cycles and that having constant ...
PeterBe's user avatar
  • 372
4 votes
1 answer
1k views

Solver for the true auto-covariance function in AR(p)

Suppose I have the following $AR(p)$ model. $$X_t = \sum_{i=1}^{p} \phi_i X_{t-i} + \epsilon_t\,, $$ where $\epsilon_t$ has mean 0 variance $\sigma^2$. I am in the situation where the $\phi$s are ...
Greenparker's user avatar
  • 15.6k
2 votes
1 answer
1k views

Coherence using FFT: how to calculate coherence for one frame

I want to calculate coherence between two time series that are of equal lengths. Since coherence is given by Pxy/(sqrt(Pxx)*sqrt(Pyy)) , I did the following steps. Step 1: Divide both time series (...
Kanmani's user avatar
  • 366
5 votes
1 answer
229 views

Probability density from Hilbert-Schmidt integral operator

The Hilbert-Schmidt integral operator determines the underlying measure, if a universal kernel is used. Now, do eigenvalues of the Hilbert-Schmidt integral operator determine the underlying measure up ...
Uzu Lim's user avatar
  • 181
1 vote
0 answers
32 views

Are people still researching the use of spectral decomposition on finite groups for data analysis?

In A GENERALIZATION OF SPECTRAL ANALYSIS WITH APPLICATION TO RANKED DATA (Diaconis 1989), the author discusses a dataset of election results. There were 5 candidates, and each voter was asked to rank ...
Syllabear's user avatar
  • 393
1 vote
1 answer
121 views

K means clustering breakup---galaxy spectrum data set

I have a spectrum data set (total 22000). Similar to an electronic wave data, two dimensional (Flux vs Wavelength). A typical set of wavelength plot looks like below Now I am doing kmeans on this ...
Ayan Mitra's user avatar
2 votes
1 answer
386 views

Years as continuous variable [duplicate]

Can I use "years" as a continuous variable ("years" as calendar years from 1984 to 2014) to see if NDVI (normalized difference vegetation index), of the same area at the same time (...
PDS's user avatar
  • 21
5 votes
1 answer
3k views

Are colored noises correlated / uncorrelated?

Let, $x$ be a random variable (r.v) that is white Gaussian, has a flat power spectrum. $y$ can be any colored noise. I think another term for uncorrelated is i.i.d (identically and independently ...
Sm1's user avatar
  • 231
2 votes
1 answer
583 views

For which clustering algorithms is the Gap statistic useful?

How can i know for which clustering algorithms (with a parameter that represents number of clusters) it makes sense to use the Gap statistic? I've read in the paper by Tibshirani, Walter & Hastie ...
ira's user avatar
  • 451

1
2 3 4 5