Questions tagged [noise]

noise is a term used for the error term in statistical models and in signal processing. It could be white noise, colored noise or otherwise.

Filter by
Sorted by
Tagged with
2
votes
1answer
29 views

What does add Gauss Random Noise mean?

I'm reading a article about LSSVM and in numerical part they created an artificial example and it says: The two features of samples are uniformly distrubuted in $[0,1)$ and $(1,2]$ with 50% of Gauss ...
0
votes
0answers
22 views

Modeling background noise with a half normal distribution

I'm working with radiation measurement data and have background noise data with $(\mu, \sigma)$ < 1 and all background measurement values >= 0. If I wanted to approximate/model this noise, would a ...
2
votes
1answer
63 views

Removing noise using a control sample

I have some measurements of displacement vs time of movement of a particle. Between these measurements, I have some particles that are supposed to not be moving at all (these are the control samples). ...
6
votes
1answer
46 views

How i add uniformly distributed noisy attributes to data set?

I want to add some artificial outliers to my data set by follow same method below. so, how i can add contaminated data statistically to real data set like Pima Indians Diabetes? info: Pima Indians ...
0
votes
1answer
16 views

General Advice - Neural Network Optimization for Noisy Label Training

I'm new to Neural Networks. Trying to get some general advice. Multi Class, 3 classes Has noisy labels, with somewhere between 60 and 80 percent accuracy Huge amount of training with the issues ...
0
votes
0answers
22 views

Find plausible peaks in streamed data

i have got a signal of a streamed source which produces values like in the picture. I want to get the "real" peaks (blue circles). But the noisy peaks (red circles) mess up the peak search. The ...
0
votes
0answers
27 views

How can I reduce the noise of prediction graph? [duplicate]

I am trying to use LSTM to predict a time series data as you can see in the following image, the predicted graphs is very noisy: The original data is looking like this: That I normalized it like ...
0
votes
0answers
26 views

Modeling of signal with noise near zero

I would like to know if there is a standard formula/representation for handling/modeling something like Gaussian noise near zero. If you have a signal that is always positive which has some noise on ...
0
votes
0answers
8 views

power spectrum density (V**2/Hz) for nonuniform logarithmic array?

I have nonuniform logarithmic time data array which have 1000pts/decade. that means sampling rate is changing in each intervals or decade. How can I calculate and plot power spectrum density (V**2/Hz)...
0
votes
0answers
15 views

RNN outputs noisy predictions

I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...
1
vote
0answers
108 views

Techniques to apply Discrete Wavelet Transform (DWT) to denoise and predict time series

I just started playing with wavelets and have been using this library (https://github.com/rafat/wavelib) to further my understanding and see if 'denoising' the series at all possible levels is ...
0
votes
0answers
19 views

What is the effect of noisy labels in distant-supervision?

I am just learning about distant supervision. I read the paper of Mintz et al. and trying to get some intuition of how the noise influences the classification. My general assumption is, that having ...
0
votes
0answers
18 views

Bayesian vs Frequentist inference in the presence of noisy data

I'm wondering how Bayesian inference and Classical/Frequentist inference fair towards noisy data. I can't seem to find too much literature addressing this issue and it seems the conclusion is usually ...
0
votes
1answer
26 views

Recovering a distribution after Gaussian noise is added

I have a large dataset (400k rows) in which I suspect the data has been obfuscated by the addition of a Gaussian distribution. My guess is that some of the data had categorical variables (based on the ...
1
vote
2answers
57 views

Sensitivity of regression parameters to noise

How sensitive are the parameters obtained from OLS, logistic or other regression methods to noise ? By noise, I mean minor changes. For e.g. adding a small noise $-1<\Delta<1$ to $\beta_1$ in $...
0
votes
0answers
29 views

How to perform feature scaling on noise removel process?

i'm working on dataset contain machinery sensor data. each column(feature) represent different sensor data(pressure, temperature, speed, etc) of the machine part. here task is to predict normal ...
1
vote
2answers
60 views

differentially private release of histograms (non-negative valued queries)

Two practical questions arise when releasing differentially private histograms/counts via addition of Laplace/Gaussian noise: 1) Is the result of noise addition truncated/rounded (since we know that ...
0
votes
0answers
45 views

Noise reduction with known noise distribution

I have a time signal with a known noise distribution parameters (gaussian, sd is known). I would like to estimate the true value statistically and in the best case obtain a confidence interval. As I ...
0
votes
0answers
70 views

Prediction of noisy target variable

The target variable for my regression problem has a very high noise, so measurement error is very high and trends can only be seen in longer time periods. What are good approaches to address this ...
3
votes
2answers
155 views

Regression algorithm on [0,1] with lots of mislabeled data

I have a training set mapping some Likert-scale variables (integers between 1 and 7, rescaled to real numbers between 0 and 1) to predict a continuous variable between 0 and 1. The data set is ...
2
votes
1answer
47 views

Apply 3 sigma formula in gamma distribution?

Let say i have some data that follows gamma distribution, and i calculated the Mean and Standard deviation of the gamma distribution. I also know that there are some outliers(Noise) in the data i ...
0
votes
1answer
305 views

Noise and Outliers in DBSCAN

Why are noise and outliers treated as the same concept in DBSCAN (density-based spatial clustering of applications with noise)?
2
votes
1answer
263 views

Sum of Bernoulli random variables with Gaussian noise

This relates to a question asked recently where (one of the edits of) the question asked what happens when a sum of Bernoulli random variables has some form of noise on the probability parameter. ...
0
votes
3answers
69 views

How can I understand the concept of a noise in machine learning?

In Bishop's book, one of the first examples is shown here Essentially, the data $x$ are randomly generated, and $t$ are generated by running $x$ through a function $\sin(2\pi x)$, then Gaussian ...
0
votes
0answers
28 views

How sensitive are Neural Networks to weight changes?

Let's consider the space of feedforward neural networks with a given structure: $L$ layers, $m$ neurones per layer, ReLu activation, input dimension $d$, output dimension $k$. Which means I'm ...
1
vote
1answer
209 views

Calculate threshold value for Poisson distributed noise

I need to calculate a threshold value to get rid of Poisson distributed noise in an image to perform a cluster analysis on the image. The image is the representation of a signal, whose datapoints ...
2
votes
0answers
201 views

Adding noise to time series data to increase training data

I am dealing with a weekly time series forecasting problem and I am currently investigating the use of an LSTM to make a multi-step forecast for a univariate time series. I actually have a ...
1
vote
0answers
29 views

Using a neural network for a regression problem, where the model to be learned suffers from awgn

I have currently a neural network to learn a (relatively non complex) system model (vector regression). Its problem is that the outputs of the system suffer from arbitry additional white gaussian ...
1
vote
1answer
39 views

Incorporating noise into machine learning models?

Usually, in machine learning textbooks the $X$ dataset and the target $y$ are defined with exact values. How about the case if the values of both $X$ and $y$ have noises: for instance, we only know ...
1
vote
0answers
35 views

Sum of correlated non-Gaussian random variables with same distribution as individual terms

Let's say there is a sum $s$ of $N$ zero-mean correlated random variables $\{x_i\}$: $$ s = x_1 + x_2 + \ldots +x_N, $$ where the correlations $C_{ij} = \mathbb E[ x_i x_j]$ are known. Assume that ...
2
votes
2answers
55 views

What is the probability that sample variance decreases by adding random Gaussian noise to the variable?

If we assume WLOG that our variable X has mean zero (mean-centered), then this can be stated $Pr \bigg(\sum x^2 > \sum (x-n)^2 \bigg)$ for some random variable $n$ distributed under $N \sim N(0, \...
0
votes
1answer
34 views

How to remove cofounding effect on a variable?

I'm working in a team that is collecting data by bicycle : We have biometric t-shirts that measure our ventilation rate. The problem is that during the last data collection, participants used masks to ...
0
votes
1answer
12 views

Supervised learning with error-range in labels?

I am working in a problem where labels have an error range (we know the range). For instance, a label can be expressed as $y_i \pm e_i$ with $e_i$ is the error range for the label of the instance $i^{...
0
votes
0answers
16 views

What determines whether relative or absolute Gaussian noise should be added to data?

Suppose I have some data vector, $\mathbf{d}$ with length $N$. I want to add "Gaussian noise" to the data. My understanding is that there are two ways to do this. 1) Relative noise percentage $$\...
0
votes
0answers
769 views

“Add White Gaussian Noise with SNR” vs. “Add 5% Gaussian Noise”

I have a noise-free dataset which is a vector of numbers, $\mathbf{d}$, with length $N$. I want to "add noise" to this data. My understanding is that there are two ways to do this. 1) Add some ...
3
votes
2answers
30 views

Formula to detect non uniformity noise

I have this line profile, how is the best way to detect this kind of non uniformity? I must detect a sudden change, as opposed to the last section that decreases slowly.. But the sudden difference ...
4
votes
2answers
619 views

Correcting Kullback-Leibler divergence for size of datasets

We have the following implementation of KLD: ...
0
votes
1answer
39 views

Why adding noisy predictors improves my random Forest prediction?

I am doing some tests with randomForest and I cannot understand the following results : ...
0
votes
0answers
27 views

Optimization of a noisy function with binary output

What frameworks exist to optimize the following problem: $\theta^\ast = \operatorname*{argmin}\limits_{\theta \in \Theta} f(\theta)$ with the characteristic that $f$ is noisy and has a binary output ...
1
vote
0answers
49 views

How do you use the predictive distribution with noise in Bayesian Optimization?

I have been reading a paper on Bayesian Optimization, and I was reading the section on adding Gaussian noise to your Gaussian process. The article is: Brochu, Cora and de Freitas (2010). A Tutorial ...
2
votes
0answers
130 views

What is the difference between Noise, error and residuals?

I was reading about Kalman filter. http://web.mit.edu/kirtley/kirtley/binlustuff/literature/control/Kalman%20filter.pdf They talk about additive noise and error. I need to understand difference ...
1
vote
0answers
32 views

MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...
1
vote
0answers
37 views

How does Noise affect the results of Transfer Entropy?

I was reading about Transfer Entropy and came across this package: https://cran.r-project.org/web/packages/TransferEntropy/TransferEntropy.pdf The code in the package: ...
2
votes
0answers
309 views

Awful performance of LSTM on noisy time series after stationarisation

Note. The post is quite long because I added some thought process for the sake of seeing the big picture. So grab a coffee and indulge yourself. For tldr the actual question on the bottom. I put my ...
2
votes
1answer
84 views

Ridge Regression as Robust Optimization

We were told to assume in class that the below optimization formulations are equivalent- $$\min_w\max_{\delta:||\delta||_F\leq\epsilon}||(X+\delta)w-y||_2^2$$ $$\min_{w}||Xw-y||_2^2+\lambda||w||_2^2 ...
0
votes
1answer
69 views

Can redundant/irrelevant features be called a Noise?

Let's say we want to predict job applicant' salary. We have a dataset with following features: {Age, Experience, Education, Astrological_Sign, Weather_Today} 5 features in total. In this set, ...
1
vote
0answers
43 views

How to add noise to obfuscate patterns in data

I have a program that generates output data depending on the inputs it is given. Lets say the data generated is a list of n items where each item is a natural number between 1 and k. I need to release ...
10
votes
2answers
782 views

Generate uniform noise from a p-norm ball ($||x||_p \leq r$)

I am trying to write a function which generates uniformly distributed noise which comes from a p-norm ball of $n$ dimensions: \begin{equation} ||x||_p \leq r \end{equation} I found possible ...
3
votes
0answers
45 views

Get noise model from true and distorted data

I have been given two data sets: Set A: a small data set containing data randomly drawn form an underlying distribution Set B: a very large data set containing data randomly drawn from the same ...
4
votes
2answers
2k views

Noise in regression data

How can I compare which distribution has more noise than the other. If for example I generate some data, how do I know that it has a large percentage of noise? Here I have a small sample code that ...