Questions tagged [filter]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
20 views

Time series with long “idle” periods - is it safe to eliminate those periods?

Suppose that I have as input a time series $T = \{ t_1, t_2, ..., t_M \}$ where each point is sampled at a fixed time interval (e.g. every 10 ms). The problem is that $T$ contains a lot of periods ...
1
vote
0answers
7 views

recommendation based on multiple user

I am learning about the recommendation system. How can I make a system where it takes multiple users as input and based on the rating and another attribute it gives recommendation? I have data sets ...
0
votes
1answer
19 views

Need precision about an example in a book about bayesian filters

My question is about the example here : https://github.com/w407022008/Kalman-and-Bayesian-Filters-in-Python/blob/master/02-Discrete-Bayes.ipynb#Adding-Uncertainty-to-the-Prediction paragraph : ...
1
vote
2answers
51 views

Do earlier hidden layers learn more concepts/features than later ones, in neural networks?

I am wondering whether there is a general statement of the sort "earlier layers in neural networks learn more concepts/features than later layers" or the other way around. The output layer not being ...
1
vote
1answer
59 views

Small noise of state process and filtering

Assume we have a linear state-space model: $$ z_{k} = Hx_{k} + v_{k}\\ x_{k} = F x_{k-1}+ w_{k}. $$ We are interested in filtering, i.e. we aim to estimate $E[x_{n}|z_{0}, \dots, z_{n}]$. If the ...
0
votes
0answers
16 views

Effect of log transformation or standardization of a regressor in the filtering step

We are working with a dataset that has hundreds of biomarkers (many of which are correlated) and often they have many missing values. Our initial goal was to use an elastic net but that would require ...
0
votes
1answer
44 views

Optimality of Bayesian filtering

In Kalman filter, we can show it's a minimum variance filter, which I believe is due to the linearity of system and the Gaussianity of noise. It comes to me that what is the optimality criterion used ...
0
votes
0answers
8 views

How are Filters values in keras decided

When we use CNN in keras, we only specify kernel size for filter in Convlayer. Let's say I chose 3x3 64 filters. But then, How would all these 64 filters have values? how is it automatically given ...
1
vote
0answers
82 views

Kalman filter for AR(1) plus noise

I am working the following AR(1) plus noise state-space model $$ z_{t} = x_{t} + v_{t}\\ x_{t} = \phi x_{t-1} + c + w_{t} $$ Therefore, the transition matrix is $[\phi]$, the observation matrix is $[1]...
1
vote
0answers
35 views

Exponential moving average before computing std

In which cases it would make sense to use exponentially weighted moving average (EWMA) before, for example, computing sample variance or other statistical analysis? Could you give an example when one ...
1
vote
0answers
32 views

Signal Decomposition

I have two time dependent signal sources X & Y. Both can be modeled as having a linear combination of time dependent individual components and common components; so X(t)=a(t)+C(t)+noise, Y(t)=b(t)+...
0
votes
0answers
19 views

Principles for choosing standard deviation of gaussian convolution filter

I have collected data from a chemical analysis, and I need to find the maximum of its first derivative; however it is too noisy to find this by simply taking the maximum. The noise can be smoothed out ...
1
vote
1answer
17 views

Filter out linearly interpolated historical data points

I am reading in historical sensor data from a plant. I found out that there are intermittent periods where between time t1 and time t2, the data points are linearly interpolated. I came to know, that ...
0
votes
0answers
10 views

How to enforce smoothness in guided image filtering techniques ? Any preferable model?

Which one (or more) of these three minimization models is the appropriate way to enforce smoothness in guided filtering framework ? \begin{eqnarray} %\begin{aligned} & \sum\limits_{q \in {N}(p)} {\...
0
votes
1answer
35 views

What is the difference between 1x1 convolutions and convolutions with “SAME” padding?

In general, 1x1 convolutions are used to reduce the dimensionality of filter space. I referred this answer. But we can also reduce the dimensionality of filter ...
0
votes
0answers
30 views

zero-lag filter: size of negative part of filter weights: when in-phase with sinusoid?

This question is about negative weights in causal filters and their effect on the lag, or "synchronization" with a sinusoidal signal. There are a few types of moving averages that use negative ...
0
votes
0answers
18 views

CNN Backpropagation Clarrification

Hi I am just trying to make sure my understanding of backpropagation with CNNs is correct, specifically CNNs that have multiple filters in each layer. This is how I have implemented backpropagation ...
0
votes
0answers
56 views

What YOLO architecture should I use for handwriting detection?

I want to create a cnn to draw bounding boxes around individual handwritten words. Ideally I would input a picture of a filled piece of notebook paper and get a cropped image of each word (they don't ...
0
votes
0answers
12 views

Analyzing Accelerometer and Gyroscope Data on a Drone

I am using an MPU-6050 Gyroscope and Accelerometer for a drone flight controller. I have been able to get raw data from the sensor, and account for bias and use the included Digital Low pass Filter at ...
0
votes
0answers
67 views

What is the methodology behind Filter Based Feature Selection (i.e. Pearson correlation, etc.) on Azure Machine Learning Studio?

Filter Based Feature Selection on Azure Machine Learning Studio supports feature selection and ranking through Pearson Correlation, Kendall Correlation, Spearman Correlation, Mutual Information, Chi ...
0
votes
1answer
210 views

How can we determine the appropriate number of hidden layers, kernels in convolutional neural network (CNN)?

I have checked a lot of questions here and in other websites. What I concluded is that there is no rules for choosing the right number of hyper-parameters in CNN, all what can we do is just trying ...
0
votes
0answers
57 views

Noise reduction with known noise distribution

I have a time signal with a known noise distribution parameters (gaussian, sd is known). I would like to estimate the true value statistically and in the best case obtain a confidence interval. As I ...
1
vote
0answers
27 views

Time series filtering notation

I am looking at some suggested filters for tidal data but am having trouble understanding the notation. For example, Godin (1972) suggests a low-pass filter for tidal data that is a combination of 24-...
1
vote
0answers
94 views

Is there a filter function that performs similar to moving average but does not loose data?

I need to smooth a time series using a low pass filter. A simple moving average is working fine for me, however, using a moving average causes an inevitable loss of data a the beginning of the ...
1
vote
1answer
34 views

Partial spline. Reference

I have a well done and perfectly working protocol to smooth my experimental data. I do the following: I have a variable of size 1000. Iteratively I choose random 100 points and spline them using the ...
1
vote
0answers
102 views

Faster RCNN - Pyramid of Filters vs Pyramid of Anchors (Reference Boxes)

I'm reading faster RCNN paper now and trying to understand what is the difference between Pyramid of Filters and Pyramid of Anchors methods from the scale point of view. I mean if I use only one ...
0
votes
1answer
36 views

Filter rows in r having same categorical value in all columns and also rows with all different categories [closed]

I am attempting to filter rows from following dataset where a,b and c give same answers, and also where a, b and c all give different answers from a category of 3 answers. id A B C X1 X2 ...
10
votes
2answers
2k views

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to ...
1
vote
0answers
67 views

Great ways to identify adult content in text

What are some good ways to identify adult content in text. It is definitely a text classification problem, but how do we handle words that are spelt like @$$.
1
vote
2answers
1k views

Removing gaussian noise from a time-series data

I have a noisy time-series data (Figure 1). As you can see the variance in this data set is very high and the "Gaussian noise" needs to be removed for me to analyze this signal. Normally we apply a ...
0
votes
0answers
56 views

In prewhitening, should we develop an ARIMA(p,q,d) for X but use only ARI(p,q) as the filter?

According to the answer in this question by IrishStat, the reason that you pre-whiten X is to identify a filter that can transform Y and X into y and x where x is white noise. Assume that X is ...
1
vote
0answers
15 views

checking the quality of filtering of a temporal series

I have many temporal series of satellite data. I decided to smooth that using Savitzky-Golay filter implemented in R using the "signal" package. I could use also other smoothing algorithms, that's ...
0
votes
0answers
20 views

Appropriate Feature Selection model for Mass Spectrometry data

I have a cancer patients data which consists of more than half million features and my task is to apply feature selection algorithm to extract the most relevant features from it. My question is which ...
1
vote
0answers
170 views

Estimate standard deviation of random-walk using Kalman filter

I created a Kalman filter that takes in time series observations and estimates the mean of that time series. This is simply modeling a random walk. However, I also want to be able to estimate the ...
2
votes
2answers
98 views

some examples of filtering the noise out of a data set

I have a data set which measures 60 data points in a second (60Hz). Clearly, I do not really need all 60 points in one second since this only generates some noise. ABOUT MY DATA: So my data sets ...
1
vote
1answer
242 views

Denoising technique for signal with beforehand known shape (linear and exponential)

I have a noisy signal which is linear and then exponential. I know the type (Gaussian additive noise) and degree (0.01) of noise. Part of the challenge is determining when the signal changed from ...
2
votes
1answer
74 views

What are the downsides of having a large number of convolutional filters

This question is purely about a single convolutional layer in a neural network - not about the number of layers. Aside from computational time what are the downsides of increasing the number of ...
2
votes
0answers
88 views

Who says trading data are noisy? [closed]

We try to denoise our time-series and model inputs with a plethora of methods like Kalman filters, EMA, Kernel filters, Splines, Beziers, etc. But who came up with a theory that trading data is noisy ...
0
votes
2answers
831 views

SMC (Particle Filtering) code [closed]

Does anyone know where I can find particle filtering code for R? In particular I'm looking for code for filtering a forward-rate curve.
1
vote
1answer
4k views

Convolutional neural network with images that have color channels

According to this guide, when applying a kernel to an input volume, the kernel always has to have the same depth as the input volume. When using images, I figured, that means the color channels. ...
2
votes
2answers
183 views

Savitzky-Golay … integrator?

Background: Savitzky-Golay filters (yes they have other names) are robust estimators of slope. Where a small noise can substantially damage the slope estimate of textbook finite difference methods, ...
1
vote
0answers
51 views

Identify this statistical filter?

I have a piece of code that takes a noisy data set and uses some sort of statistical filter to plot a trend line over the noisy data. The original author of the code no longer works at my job, so I ...
1
vote
1answer
77 views

Isn't every linear filter a linear time invariant (LTI) filter?

I'm using the following definition for a linear time invariant digital filter: "A digital filter L that transforms an input sequence $\{ x_{t} \}$ into an output sequence $\{ y_{t} \}$ is called a ...
3
votes
2answers
1k views

Stationarity of AR(1) process, stable filter

This section of the Wikipedia article about the Autoregressive Model reads: An AR(1) process is given by: $$X_t = c + \varphi X_{t-1}+\varepsilon_t$$ where $\varepsilon_t$ is a white noise process ...
1
vote
2answers
91 views

Filtering disturbances from time series

Lets suppose that time series of the following building climate related measures are given for a small building: Solar radiation Outdoor air temperature Heat supply Indoor air temperature The ...
2
votes
0answers
22 views

Prove that a symmetric filter preserve an p-order trend

Let the centered filter $\Theta (L)=\theta _{-m}L^m+...+\theta _{-2}L^2+\theta _{-1}L+\theta _{0}+\theta _{1}L^{-1}+\theta _{2}L^{-2}+...+\theta _{m}L^{-m}$ where $\theta _{-m}+...\theta _{-2}+\...
0
votes
1answer
272 views

How to explain exponential plus Gaussian noise

I have some noisy signal. When I substract real (for calibration it is known) or filtered signal I get residuals. When I plot distribution of these residuals I see that it is sum of exponential and ...
1
vote
0answers
51 views

Introduction to filtering - Kalman filtering

I would like to make a preamble and then introducing the Kalman filter in a work I am writing. In your opinion which are the topics I have to cite? Do you think I should cite Kolmogorov and Wiener ...
1
vote
0answers
36 views

Savitzky-Golay with switching between scales based on slope

I am having fun with Savitzky-Golay (aka H-P, W-H) and I am bumping into limits of the filter. It makes me want a better filter. Is there a filter that adjusts filter order based on "slope" but ...
1
vote
1answer
154 views

Why is it harder to obtain diagnostic knowledge $p(h\mid z)$ than causal knowledge $p(z\mid h)$? [closed]

Causal knowledge is $p(z\mid h)$, i.e. the probability that a certain state $h$ causes a certain state of the observable variable $z$. Diagnostic knowledge on the other hand refers to the knowledge we ...