# Questions tagged [noise]

noise is a term used for the error term in statistical models and in signal processing. It could be white noise, colored noise or otherwise.

74 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2answers
30 views

### Formula to detect non uniformity noise

I have this line profile, how is the best way to detect this kind of non uniformity? I must detect a sudden change, as opposed to the last section that decreases slowly.. But the sudden difference ...
0answers
45 views

### Get noise model from true and distorted data

I have been given two data sets: Set A: a small data set containing data randomly drawn form an underlying distribution Set B: a very large data set containing data randomly drawn from the same ...
0answers
117 views

### Correcting for noise in gene expression data

I have a training set of RT-qPCR gene expression data (not run in triplicate) for a batch of samples with two phenotypes $A$ and $B$ on which I've trained a logistic regression classifier. I also ...
0answers
78 views

### Expressing Confidence in Conclusions from Noisy Data

I’m working on improving the robustness of a software engineering process that measures performance of programming language compiler and standard library, a.k.a. benchmarks — in the computing sense of ...
0answers
72 views

### What is known about 2nd order estimation biases due to correlated noise between response variables?

I have recently run into a statistical bias in a type of analysis that I believe is somewhat common in my field, and not typically corrected for. I would like to know if this is more well-known in ...
0answers
51 views

### Representation of Noise in Fourier transform

I perform an experiment where I sample $M(k)$ which is in theory related to $|f(x)|^2$ via $M(k)=\int e^{ikx}|f(x)|^2\,dx$. I perform the discrete FT on my data in order to obtain $|f(x)|^2$. Without ...
0answers
183 views

### Detection of noise and outliers

I am measuring the number of cells with a mutation in a series of 106 subjects. For each position of the genome, the method will output the total number of cells analysed and the number of cells with ...
0answers
250 views

### How to determine rise time of a signal from its noisy background timeseries?

I have temperature vs. time data from a thermometer. The data was recorded using a DAQ system, has a stable background level, and some random noise. At a certain time, the temperature begins to rise ...
0answers
100 views

### What is the difference between Noise, error and residuals?

I was reading about Kalman filter. http://web.mit.edu/kirtley/kirtley/binlustuff/literature/control/Kalman%20filter.pdf They talk about additive noise and error. I need to understand difference ...
0answers
289 views

### Awful performance of LSTM on noisy time series after stationarisation

Note. The post is quite long because I added some thought process for the sake of seeing the big picture. So grab a coffee and indulge yourself. For tldr the actual question on the bottom. I put my ...
0answers
27 views

### Noisy conditional simulation

A conditional random field $Z_C(x)$ is a random field whose realisations $z_C(x)$ always take the same values $z_C(x_a)$ at locations $x_a$. Realisations of $Z_C(x)$ can be produced as follows (...
0answers
79 views

### How to properly treat feature (attribute) data from multiple sensors with different measurement noise (for classification)?

I have a classification problem in which the input feature data are derived from multiple sensors. If the quality of the feature attributes as measured by each sensor varies (for example, because some ...
0answers
67 views

### Noise estimation in LTE using bandpass filter

Can noise estimation in LTE be done using bandpass filter? As per my study in wireless systems to estimate noise power, if pilot sequence is known is done as |y(k)-p(k)h(k)|^2, where p(k) is pilot ...
0answers
80 views

### Cleaning the signal from noise

I have a following signal, and I want to correct the part which has a sharp jump, based on the value of the signal sometimes before the jump and its future values. Instead of that jump I want ...
3answers
915 views

### Training a RNN on time series: How to cope with different sequence origins?

I am wondering if I should apply a recurrent neural network on my data. Data is EEG from sleep, and thus there is much information hidden in the temporal domain. Ergo, RNNs make sense. Intro: I have ...
0answers
991 views

### Deriving mean and variance of the posterior distribution

I have a simple linear model: $y_{i}=\mu+e_{i}$ for $i=1,...,n$, where $P(e_{i})=w\mathcal{N}(0,\sigma^2) + (1-w)\mathcal{N}(0,k^2\sigma^2)$ with $w=0.9$, $k=10$ and $\sigma=0.1$. It can be understood ...
0answers
20 views

### Finding the spectral density of a fraction of two noisy variables

I have a certain expression $\Phi=C \frac{V}{R}$ where $C$ is a constant, and both $V$ and $R$ are noisy variables; they are set to some working point value, but they have some small noise component (...
0answers
613 views

### Dealing with high noise data?

I am working with sensors on trucks which give me data with a lot of noise. The goal is to make a prediction to prevent failure in the system. I am having trouble with this for several reasons: The ...
0answers
51 views

### For each x, I observe A and know P(C). What can I say about E(A|C)?

For each subject x in a population, I observe x's age, A(x). I can calculate the probability that x has some property of interest c, $P[C(x) = 1]$, where C(x) is a binary variable indicating whether ...
1answer
265 views

### How to specify K cluster in Hierarchical clustering with noisy data?

I'm new in Mining and Clustering and I wonder how to cut off the hierarchical clustering Dendrogram to obtain a specific number of clusters. The problem is here that the data is noisy and the ...
0answers
308 views

### R: Is it possible to estimate the poisson noise?

I have a dataset of many discrete counts (RNAseq read counts per base), which contain both real signals and background noise. The noise is random, and should be poisson distributed. What I would ...
0answers
87 views

### Techniques to apply Discrete Wavelet Transform (DWT) to denoise and predict time series

I just started playing with wavelets and have been using this library (https://github.com/rafat/wavelib) to further my understanding and see if 'denoising' the series at all possible levels is ...
0answers
195 views

### Adding noise to time series data to increase training data

I am dealing with a weekly time series forecasting problem and I am currently investigating the use of an LSTM to make a multi-step forecast for a univariate time series. I actually have a ...
0answers
35 views

### Sum of correlated non-Gaussian random variables with same distribution as individual terms

Let's say there is a sum $s$ of $N$ zero-mean correlated random variables $\{x_i\}$: $$s = x_1 + x_2 + \ldots +x_N,$$ where the correlations $C_{ij} = \mathbb E[ x_i x_j]$ are known. Assume that ...
0answers
29 views

### Using a neural network for a regression problem, where the model to be learned suffers from awgn

I have currently a neural network to learn a (relatively non complex) system model (vector regression). Its problem is that the outputs of the system suffer from arbitry additional white gaussian ...
0answers
47 views

### How do you use the predictive distribution with noise in Bayesian Optimization?

I have been reading a paper on Bayesian Optimization, and I was reading the section on adding Gaussian noise to your Gaussian process. The article is: Brochu, Cora and de Freitas (2010). A Tutorial ...
0answers
29 views

### MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...
0answers
37 views

### How does Noise affect the results of Transfer Entropy?

I was reading about Transfer Entropy and came across this package: https://cran.r-project.org/web/packages/TransferEntropy/TransferEntropy.pdf The code in the package: ...
0answers
41 views

### How to add noise to obfuscate patterns in data

I have a program that generates output data depending on the inputs it is given. Lets say the data generated is a list of n items where each item is a natural number between 1 and k. I need to release ...
0answers
41 views

### Time series clustering/segmentation based on pattern

I'm currently working with a database which contains several large PPG (pulse oximetry) and ECG time series. These series, however, contain segments within them which are highly contaminated by noise, ...
0answers
31 views

1answer
81 views

### Noise identification in Kalman filtering procedure

Suppose I have a standard state-space model. The sample is, say, 1990-2015, quarterly data. I assume that in period 1990-2000 there were two sources of noise in the measurement equation, while in 2001-...
0answers
25 views

### Where can I find good references regarding to noise filtering and prediction in time series?

I want to model the error structure of every certain time period obtained from the past errors produced by the predictions of nonlinear time series. I would like to know if someone knows specialized ...
0answers
152 views

### Non-Causal time-series filtering techniques for standard noise with unkown variance. (EM vs. weiner vs. kalman)

This is a quick question about filtering stored time-series data using kalman/weiner filtering techniques or expectation maximization. I'm just hoping to fix some confusion about questioning ...
0answers
142 views

### Lagrange Multipliers in practice

Say we want to minimize the function $f^2({\bf{x}})$, under the constraint $g({\bf{x}})=0$. The classic solution (Method I) is to introduce a Lagrange Multiplier, and solve: \frac{\partial f^2({\bf{...
0answers
47 views

### Signal processing techniques for unevenly spaced and repeated measures series

I am considering using signal processing techniques to find the minimum on a noisy 1D response line. More specifically I have a simulation that requires one parameter, but also includes randomness, ...
0answers
11 views

### Filtering noise in signal prior to SVM

I am using an SVMs to detect faults in various signals. In brief these are the steps I am taking: 1) from clean signal data 2) inject Gaussian noise into signal 3) inject error into signal 4) train ...
1answer
15 views

### General Advice - Neural Network Optimization for Noisy Label Training

I'm new to Neural Networks. Trying to get some general advice. Multi Class, 3 classes Has noisy labels, with somewhere between 60 and 80 percent accuracy Huge amount of training with the issues ...
0answers
20 views

### Find plausible peaks in streamed data

i have got a signal of a streamed source which produces values like in the picture. I want to get the "real" peaks (blue circles). But the noisy peaks (red circles) mess up the peak search. The ...
0answers
26 views

### Modeling of signal with noise near zero

I would like to know if there is a standard formula/representation for handling/modeling something like Gaussian noise near zero. If you have a signal that is always positive which has some noise on ...
0answers
6 views

### power spectrum density (V**2/Hz) for nonuniform logarithmic array?

I have nonuniform logarithmic time data array which have 1000pts/decade. that means sampling rate is changing in each intervals or decade. How can I calculate and plot power spectrum density (V**2/Hz)...
0answers
13 views

### RNN outputs noisy predictions

I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...
0answers
17 views

### What is the effect of noisy labels in distant-supervision?

I am just learning about distant supervision. I read the paper of Mintz et al. and trying to get some intuition of how the noise influences the classification. My general assumption is, that having ...
0answers
18 views

### Bayesian vs Frequentist inference in the presence of noisy data

I'm wondering how Bayesian inference and Classical/Frequentist inference fair towards noisy data. I can't seem to find too much literature addressing this issue and it seems the conclusion is usually ...