Questions tagged [noise]
noise is a term used for the error term in statistical models and in signal processing. It could be white noise, colored noise or otherwise.
173
questions
0
votes
0answers
3 views
Estimate Data Noise formula in Pythhon [closed]
How can I create the above formula in Python?
I would create a function that would take input x (matrix), output y (vector), and weights (vector). Any guidance is appreciated.
Many thanks
0
votes
0answers
10 views
Adding noise to existing noise
I have been working on a little project for myself. It involves adding the following noise types to images: Gaussian blur, Salt and Pepper, and Speckle. I know that there exists some fancy formula for ...
0
votes
0answers
18 views
Laplacian noise applications
I would like to know whether Laplacian distribution can be used to model a Poisson noise.
I have met this case while checking this book and here what it says (see the picture below)
As far as I ...
0
votes
1answer
37 views
Median or mode for measurements with erroneous outliers
Background: I am working with real measurements that likely contain two sources of error, (1) measurements that were performed incorrectly, and (2) natural variability of the measured quantity and ...
1
vote
0answers
50 views
Log of cost function converges to 0 and is extremely “noisy”
I'm writing my very first NN algorithm to solve a regression problem. I have a signal I would like to use a NN to find its correlation with eight other signals (along the line as f = f(x1, x2,..., x8))...
0
votes
0answers
21 views
Maximum level of label noise for binary classification so that dataset is “Learnable”?
Assume we have an imbalanced dataset (minority label frequency 1-20%), where subset of samples have their labels randomly flipped. Now, of all samples with positive label (the minority class) in this ...
0
votes
0answers
14 views
Addings noise to emperical distribution
Say I have the following samples from a joint distribution (note I only have access to the samples and not the functional form of the distribution it self)
...
0
votes
1answer
12 views
Is there a good measure of “paternless-ness” in a set of data?
This may be a slightly open-ended question, or even a bad one, but are there any measures of looking at a set of data and seeing whether there is any kind of pattern or not?
For some context my ...
0
votes
0answers
5 views
Analyzing 1000s of time-stamped tweets. How to automatically identify spikes in terms?
I have about 30,000 tweets from a corporate customer service account, all harvested according to the Twitter TOC. Naturally, each message is time-stamped and not extremely long.
I'm trying to ...
1
vote
0answers
14 views
Timing Jitter of Counter Query … verify hypothesis / noise reduction
I am measuring traffic on an interface every 5 minutes, but the data platform is having an issue with the SNMP data. I think it is caused by timing jitter ... where the data platform queries the ...
0
votes
1answer
36 views
Why noise with normal distribution is used as input to GAN?
Why noise with normal distribution is used as input to GAN?
What will happen if we will use uniform noise or just random binary vector?
0
votes
0answers
19 views
Bounding variance of noise in a noisy voting scheme
I am looking at the accuracy of a method of human yes/no voting.
Essentially, I have the vote totals for a number of binomial processes, which represent different "elections" this method was ran on. ...
0
votes
0answers
20 views
On the transition probability distribution of Gaussian Brownian motion
I am having trouble understanding certain aspects of the following derivation. I'll first present it, and then follow up with questions. The derivation is as follows:
Consider a random variable $X(t)$...
0
votes
1answer
35 views
Smoothing time series with non-constant variance
I have a discrete time series $x(t)$ $ t = \{0,\Delta t,2\Delta t\dots\}$ in which every point comes with a confidence value $c(t)$ arising from the measurements. You may think of is as the variance ...
0
votes
0answers
4 views
Error Analysis For Denoising?
I have a collection of noisy audio files and their corresponding "clean" versions. I used some statistical methods like low-pass and high-pass filtering to try and remove the noise in these noisy ...
1
vote
0answers
40 views
Adjustment of the forecast of a time series for the analysis of a system
I have a simulation model of a system which receives a forecast of a time series as input. In my scientific work I would like to examine how the performance of the simulation model behaves in relation ...
2
votes
1answer
37 views
What does add Gauss Random Noise mean?
I'm reading a article about LSSVM and in numerical part they created an artificial example and it says:
The two features of samples are uniformly distrubuted in $[0,1)$ and $(1,2]$ with 50% of Gauss ...
0
votes
0answers
28 views
Modeling background noise with a half normal distribution
I'm working with radiation measurement data and have background noise data with $(\mu, \sigma)$ < 1 and all background measurement values >= 0. If I wanted to approximate/model this noise, would a ...
2
votes
1answer
64 views
Removing noise using a control sample
I have some measurements of displacement vs time of movement of a particle. Between these measurements, I have some particles that are supposed to not be moving at all (these are the control samples). ...
6
votes
1answer
48 views
How i add uniformly distributed noisy attributes to data set?
I want to add some artificial outliers to my data set by follow same method below.
so, how i can add contaminated data statistically to real data set like Pima Indians Diabetes?
info:
Pima Indians ...
0
votes
1answer
17 views
General Advice - Neural Network Optimization for Noisy Label Training
I'm new to Neural Networks. Trying to get some general advice.
Multi Class, 3 classes
Has noisy labels, with somewhere between 60 and 80 percent accuracy
Huge amount of training with the issues ...
0
votes
0answers
22 views
Find plausible peaks in streamed data
i have got a signal of a streamed source which produces values like in the picture.
I want to get the "real" peaks (blue circles).
But the noisy peaks (red circles) mess up the peak search.
The ...
0
votes
0answers
64 views
How can I reduce the noise of prediction graph? [duplicate]
I am trying to use LSTM to predict a time series data as you can see in the following image, the predicted graphs is very noisy:
The original data is looking like this:
That I normalized it like ...
0
votes
0answers
27 views
Modeling of signal with noise near zero
I would like to know if there is a standard formula/representation for handling/modeling something like Gaussian noise near zero. If you have a signal that is always positive which has some noise on ...
0
votes
0answers
12 views
power spectrum density (V**2/Hz) for nonuniform logarithmic array?
I have nonuniform logarithmic time data array which have 1000pts/decade. that means sampling rate is changing in each intervals or decade.
How can I calculate and plot power spectrum density (V**2/Hz)...
0
votes
0answers
30 views
RNN outputs noisy predictions
I have an RNN that I've trained and I'm now using to generate new sequences. These sequences are basically discrete state time courses for K different states. The ...
1
vote
0answers
237 views
Techniques to apply Discrete Wavelet Transform (DWT) to denoise and predict time series
I just started playing with wavelets and have been using this library (https://github.com/rafat/wavelib) to further my understanding and see if 'denoising' the series at all possible levels is ...
0
votes
0answers
22 views
What is the effect of noisy labels in distant-supervision?
I am just learning about distant supervision. I read the paper of Mintz et al. and trying to get some intuition of how the noise influences the classification.
My general assumption is, that having ...
0
votes
0answers
18 views
Bayesian vs Frequentist inference in the presence of noisy data
I'm wondering how Bayesian inference and Classical/Frequentist inference fair towards noisy data. I can't seem to find too much literature addressing this issue and it seems the conclusion is usually ...
0
votes
1answer
56 views
Recovering a distribution after Gaussian noise is added
I have a large dataset (400k rows) in which I suspect the data has been obfuscated by the addition of a Gaussian distribution. My guess is that some of the data had categorical variables (based on the ...
1
vote
2answers
71 views
Sensitivity of regression parameters to noise
How sensitive are the parameters obtained from OLS, logistic or other regression methods to noise ?
By noise, I mean minor changes. For e.g. adding a small noise $-1<\Delta<1$ to $\beta_1$ in $...
0
votes
0answers
54 views
How to perform feature scaling on noise removel process?
i'm working on dataset contain machinery sensor data. each column(feature) represent different sensor data(pressure, temperature, speed, etc) of the machine part. here task is to predict normal ...
1
vote
2answers
87 views
differentially private release of histograms (non-negative valued queries)
Two practical questions arise when releasing differentially private histograms/counts via addition of Laplace/Gaussian noise:
1) Is the result of noise addition truncated/rounded (since we know that ...
0
votes
0answers
57 views
Noise reduction with known noise distribution
I have a time signal with a known noise distribution parameters (gaussian, sd is known). I would like to estimate the true value statistically and in the best case obtain a confidence interval.
As I ...
1
vote
0answers
89 views
Prediction of noisy target variable
The target variable for my regression problem has a very high noise, so measurement error is very high and trends can only be seen in longer time periods.
What are good approaches to address this ...
3
votes
2answers
168 views
Regression algorithm on [0,1] with lots of mislabeled data
I have a training set mapping some Likert-scale variables (integers between 1 and 7, rescaled to real numbers between 0 and 1) to predict a continuous variable between 0 and 1. The data set is ...
2
votes
1answer
76 views
Apply 3 sigma formula in gamma distribution?
Let say i have some data that follows gamma distribution, and i calculated the Mean and Standard deviation of the gamma distribution.
I also know that there are some outliers(Noise) in the data i ...
0
votes
1answer
505 views
Noise and Outliers in DBSCAN
Why are noise and outliers treated as the same concept in DBSCAN (density-based spatial clustering of applications with noise)?
3
votes
1answer
366 views
Sum of Bernoulli random variables with Gaussian noise
This relates to a question asked recently where (one of the edits of) the question asked what happens when a sum of Bernoulli random variables has some form of noise on the probability parameter.
...
0
votes
3answers
100 views
How can I understand the concept of a noise in machine learning?
In Bishop's book, one of the first examples is shown here
Essentially, the data $x$ are randomly generated, and $t$ are generated by running $x$ through a function $\sin(2\pi x)$, then Gaussian ...
1
vote
1answer
242 views
Calculate threshold value for Poisson distributed noise
I need to calculate a threshold value to get rid of Poisson distributed noise in an image to perform a cluster analysis on the image.
The image is the representation of a signal, whose datapoints ...
2
votes
1answer
284 views
Adding noise to time series data to increase training data
I am dealing with a weekly time series forecasting problem and I am currently investigating the use of an LSTM to make a multi-step forecast for a univariate time series. I actually have a ...
1
vote
0answers
30 views
Using a neural network for a regression problem, where the model to be learned suffers from awgn
I have currently a neural network to learn a (relatively non complex) system model (vector regression). Its problem is that the outputs of the system suffer from arbitry additional white gaussian ...
1
vote
1answer
41 views
Incorporating noise into machine learning models?
Usually, in machine learning textbooks the $X$ dataset and the target $y$ are defined with exact values.
How about the case if the values of both $X$ and $y$ have noises: for instance, we only know ...
2
votes
0answers
39 views
Sum of correlated non-Gaussian random variables with same distribution as individual terms
Let's say there is a sum $s$ of $N$ zero-mean correlated random variables $\{x_i\}$:
$$
s = x_1 + x_2 + \ldots +x_N,
$$
where the correlations $C_{ij} = \mathbb E[ x_i x_j]$ are known. Assume that ...
2
votes
2answers
60 views
What is the probability that sample variance decreases by adding random Gaussian noise to the variable?
If we assume WLOG that our variable X has mean zero (mean-centered), then this can be stated
$Pr \bigg(\sum x^2 > \sum (x-n)^2 \bigg)$
for some random variable $n$ distributed under $N \sim N(0, \...
0
votes
1answer
37 views
How to remove cofounding effect on a variable?
I'm working in a team that is collecting data by bicycle : We have biometric t-shirts that measure our ventilation rate.
The problem is that during the last data collection, participants used masks to ...
0
votes
1answer
13 views
Supervised learning with error-range in labels?
I am working in a problem where labels have an error range (we know the range). For instance, a label can be expressed as $y_i \pm e_i$ with $e_i$ is the error range for the label of the instance $i^{...
0
votes
0answers
1k views
“Add White Gaussian Noise with SNR” vs. “Add 5% Gaussian Noise”
I have a noise-free dataset which is a vector of numbers, $\mathbf{d}$, with length $N$.
I want to "add noise" to this data. My understanding is that there are two ways to do this.
1) Add some ...
3
votes
2answers
32 views
Formula to detect non uniformity noise
I have this line profile, how is the best way to detect this kind of non uniformity?
I must detect a sudden change, as opposed to the last section that decreases slowly..
But the sudden difference ...
