# Questions tagged [noise]

noise is a term used for the error term in statistical models and in signal processing. It could be white noise, colored noise or otherwise.

74 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
30 views

### Formula to detect non uniformity noise

I have this line profile, how is the best way to detect this kind of non uniformity? I must detect a sudden change, as opposed to the last section that decreases slowly.. But the sudden difference ...
45 views

### Get noise model from true and distorted data

I have been given two data sets: Set A: a small data set containing data randomly drawn form an underlying distribution Set B: a very large data set containing data randomly drawn from the same ...
117 views

### Correcting for noise in gene expression data

I have a training set of RT-qPCR gene expression data (not run in triplicate) for a batch of samples with two phenotypes $A$ and $B$ on which I've trained a logistic regression classifier. I also ...
78 views

### Expressing Confidence in Conclusions from Noisy Data

I’m working on improving the robustness of a software engineering process that measures performance of programming language compiler and standard library, a.k.a. benchmarks — in the computing sense of ...
72 views

### What is known about 2nd order estimation biases due to correlated noise between response variables?

I have recently run into a statistical bias in a type of analysis that I believe is somewhat common in my field, and not typically corrected for. I would like to know if this is more well-known in ...
51 views

### Representation of Noise in Fourier transform

I perform an experiment where I sample $M(k)$ which is in theory related to $|f(x)|^2$ via $M(k)=\int e^{ikx}|f(x)|^2\,dx$. I perform the discrete FT on my data in order to obtain $|f(x)|^2$. Without ...
183 views

### Detection of noise and outliers

I am measuring the number of cells with a mutation in a series of 106 subjects. For each position of the genome, the method will output the total number of cells analysed and the number of cells with ...
250 views

### How to determine rise time of a signal from its noisy background timeseries?

I have temperature vs. time data from a thermometer. The data was recorded using a DAQ system, has a stable background level, and some random noise. At a certain time, the temperature begins to rise ...
100 views

289 views

### Awful performance of LSTM on noisy time series after stationarisation

Note. The post is quite long because I added some thought process for the sake of seeing the big picture. So grab a coffee and indulge yourself. For tldr the actual question on the bottom. I put my ...
27 views

### Noisy conditional simulation

A conditional random field $Z_C(x)$ is a random field whose realisations $z_C(x)$ always take the same values $z_C(x_a)$ at locations $x_a$. Realisations of $Z_C(x)$ can be produced as follows (...
79 views

### How to properly treat feature (attribute) data from multiple sensors with different measurement noise (for classification)?

I have a classification problem in which the input feature data are derived from multiple sensors. If the quality of the feature attributes as measured by each sensor varies (for example, because some ...
67 views

### Noise estimation in LTE using bandpass filter

Can noise estimation in LTE be done using bandpass filter? As per my study in wireless systems to estimate noise power, if pilot sequence is known is done as |y(k)-p(k)h(k)|^2, where p(k) is pilot ...
80 views

### Cleaning the signal from noise

I have a following signal, and I want to correct the part which has a sharp jump, based on the value of the signal sometimes before the jump and its future values. Instead of that jump I want ...
915 views

### Training a RNN on time series: How to cope with different sequence origins?

I am wondering if I should apply a recurrent neural network on my data. Data is EEG from sleep, and thus there is much information hidden in the temporal domain. Ergo, RNNs make sense. Intro: I have ...
991 views

### Deriving mean and variance of the posterior distribution

I have a simple linear model: $y_{i}=\mu+e_{i}$ for $i=1,...,n$, where $P(e_{i})=w\mathcal{N}(0,\sigma^2) + (1-w)\mathcal{N}(0,k^2\sigma^2)$ with $w=0.9$, $k=10$ and $\sigma=0.1$. It can be understood ...
20 views

### Finding the spectral density of a fraction of two noisy variables

I have a certain expression $\Phi=C \frac{V}{R}$ where $C$ is a constant, and both $V$ and $R$ are noisy variables; they are set to some working point value, but they have some small noise component (...
613 views

### Dealing with high noise data?

I am working with sensors on trucks which give me data with a lot of noise. The goal is to make a prediction to prevent failure in the system. I am having trouble with this for several reasons: The ...
51 views

### For each x, I observe A and know P(C). What can I say about E(A|C)?

For each subject x in a population, I observe x's age, A(x). I can calculate the probability that x has some property of interest c, $P[C(x) = 1]$, where C(x) is a binary variable indicating whether ...
265 views

### How to specify K cluster in Hierarchical clustering with noisy data?

I'm new in Mining and Clustering and I wonder how to cut off the hierarchical clustering Dendrogram to obtain a specific number of clusters. The problem is here that the data is noisy and the ...
308 views

### R: Is it possible to estimate the poisson noise?

I have a dataset of many discrete counts (RNAseq read counts per base), which contain both real signals and background noise. The noise is random, and should be poisson distributed. What I would ...
87 views

### Techniques to apply Discrete Wavelet Transform (DWT) to denoise and predict time series

I just started playing with wavelets and have been using this library (https://github.com/rafat/wavelib) to further my understanding and see if 'denoising' the series at all possible levels is ...
195 views

### Adding noise to time series data to increase training data

I am dealing with a weekly time series forecasting problem and I am currently investigating the use of an LSTM to make a multi-step forecast for a univariate time series. I actually have a ...
35 views

### Sum of correlated non-Gaussian random variables with same distribution as individual terms

Let's say there is a sum $s$ of $N$ zero-mean correlated random variables $\{x_i\}$: $$s = x_1 + x_2 + \ldots +x_N,$$ where the correlations $C_{ij} = \mathbb E[ x_i x_j]$ are known. Assume that ...
29 views

### Using a neural network for a regression problem, where the model to be learned suffers from awgn

I have currently a neural network to learn a (relatively non complex) system model (vector regression). Its problem is that the outputs of the system suffer from arbitry additional white gaussian ...
47 views

### How do you use the predictive distribution with noise in Bayesian Optimization?

I have been reading a paper on Bayesian Optimization, and I was reading the section on adding Gaussian noise to your Gaussian process. The article is: Brochu, Cora and de Freitas (2010). A Tutorial ...
29 views

### MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...
37 views

### How does Noise affect the results of Transfer Entropy?

I was reading about Transfer Entropy and came across this package: https://cran.r-project.org/web/packages/TransferEntropy/TransferEntropy.pdf The code in the package: ...
41 views

### How to add noise to obfuscate patterns in data

I have a program that generates output data depending on the inputs it is given. Lets say the data generated is a list of n items where each item is a natural number between 1 and k. I need to release ...
41 views

### Time series clustering/segmentation based on pattern

I'm currently working with a database which contains several large PPG (pulse oximetry) and ECG time series. These series, however, contain segments within them which are highly contaminated by noise, ...
31 views

81 views

### Noise identification in Kalman filtering procedure

Suppose I have a standard state-space model. The sample is, say, 1990-2015, quarterly data. I assume that in period 1990-2000 there were two sources of noise in the measurement equation, while in 2001-...
25 views

### Where can I find good references regarding to noise filtering and prediction in time series?

I want to model the error structure of every certain time period obtained from the past errors produced by the predictions of nonlinear time series. I would like to know if someone knows specialized ...
152 views

### Non-Causal time-series filtering techniques for standard noise with unkown variance. (EM vs. weiner vs. kalman)

This is a quick question about filtering stored time-series data using kalman/weiner filtering techniques or expectation maximization. I'm just hoping to fix some confusion about questioning ...
142 views

### Lagrange Multipliers in practice

Say we want to minimize the function $f^2({\bf{x}})$, under the constraint $g({\bf{x}})=0$. The classic solution (Method I) is to introduce a Lagrange Multiplier, and solve: \frac{\partial f^2({\bf{...
47 views

### Signal processing techniques for unevenly spaced and repeated measures series

I am considering using signal processing techniques to find the minimum on a noisy 1D response line. More specifically I have a simulation that requires one parameter, but also includes randomness, ...
11 views

### Filtering noise in signal prior to SVM

I am using an SVMs to detect faults in various signals. In brief these are the steps I am taking: 1) from clean signal data 2) inject Gaussian noise into signal 3) inject error into signal 4) train ...
15 views

### General Advice - Neural Network Optimization for Noisy Label Training

I'm new to Neural Networks. Trying to get some general advice. Multi Class, 3 classes Has noisy labels, with somewhere between 60 and 80 percent accuracy Huge amount of training with the issues ...
20 views

### Find plausible peaks in streamed data

i have got a signal of a streamed source which produces values like in the picture. I want to get the "real" peaks (blue circles). But the noisy peaks (red circles) mess up the peak search. The ...
26 views

### Modeling of signal with noise near zero

I would like to know if there is a standard formula/representation for handling/modeling something like Gaussian noise near zero. If you have a signal that is always positive which has some noise on ...
6 views

### power spectrum density (V**2/Hz) for nonuniform logarithmic array?

I have nonuniform logarithmic time data array which have 1000pts/decade. that means sampling rate is changing in each intervals or decade. How can I calculate and plot power spectrum density (V**2/Hz)...
13 views

### RNN outputs noisy predictions

I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...