Questions tagged [filter]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
6 views

How to calculate number of learnable parameters in CNN? [duplicate]

How to calculate number of learnable parameters in CNN when only kernel size and number of filters are given? Lets say the kernel size is x and number of filters is y. In that case in which way I ...
user avatar
0 votes
1 answer
23 views

Approximating a 1-d Kalman Filter with non-Gaussian Observation Noise

I'm looking for a Bayesian filter where observations are generated according to $s_t = \gamma s_{t-1} + w_p$ and $w_p \sim Normal(0, \sigma_p^2)$. Both $\gamma$ and the variance of the process noise $\...
user avatar
-4 votes
1 answer
68 views

Wait so probabilities of 0 or 1 CAN change? $P(A|B)=1$ does not imply $P(A)=1$ because $0 < P(A=B) < 1$?

Nassim Nicholas Taleb says here no probability that is 0 or 1 should ever change. Despite these 6 questions Does an unconditional probability of 1 or 0 imply a conditional probability of 1 or 0 if ...
user avatar
  • 2,176
1 vote
0 answers
52 views

Guessing filters from responses to step signals

Consider a signal $X$ filtered by a kernel $p$ with finite support $[t_0,t_1]$ and $\int_{t_0}^{t_1}p(t)\,\text{d}t = 1$, yielding the response function $$\overline{X}(T) = \int_{t_0}^{t_1} X(T + t)\ ...
user avatar
0 votes
1 answer
30 views

Setting the observation likelihood threshold for outlier detection if you know know the percentage of outliers

Let's assume I have a sensor that gives me measurements $z$ and I know that $50\%$ of the measurements I read are outliers (more than 3 standard deviations away from the real measurement distribution)....
user avatar
  • 340
0 votes
0 answers
13 views

How to optimally select window sizes for filters on spectral data

I am trying to find an efficient method of selecting a window size for a Savitzky-Golay filter. The applications is mainly to find an smooth representation of a spectra containing sharp peaks in noisy ...
user avatar
0 votes
1 answer
53 views

Smoothing sensor signals with response times - how to

I have a Sensor (e. g. for temperature) that has a response time ($t_{99}$ -> this is the time that the sensor needs to give an output of 99 % of the actual value) of let's say 10 seconds. This ...
user avatar
0 votes
0 answers
53 views

When is it acceptable to exclude data from analysis?

I have Alcohol, Tobacco, and Firearms (ATF) trace data that lists categories/offenses related to when a firearm was recovered by law enforcement. However, in many cases there are offenses that are ...
user avatar
1 vote
1 answer
26 views

Estimate the Image Using Multi Many Realizations of Its Convolution with a Known Filters Using Wiener Filter

Suppose we have a corrupted image $Y = H*X + \epsilon$ that is formed by taking an image $X$, convolving it with a point-spread function $H$, and adding gaussian noise $\epsilon$. Then we know that ...
user avatar
1 vote
0 answers
15 views

What is the significance behind having small kernel sizes over having one large kernel size that covers the entire input in a CNN?

I have hardly ever seen anyone cover the entire input image with a filter of the same dimensions. I was wondering why that is the case, and if the performance in say, an image detection application ...
user avatar
  • 111
0 votes
0 answers
24 views

In stochastic filtering, can observations depend on lagged observation values?

Say I have a latent state like $$ dX_t = dW_t $$ and observations like $$ dY_t = f(X_t - Y_t)dt + dZ_t $$ Can I get filtering estimates of $X_t$ using a standard Kalman filter framework despite the ...
user avatar
0 votes
0 answers
32 views

How can a stochastic filter be forecasted?

After the covid-19 lockdown, employment (and other economic indicators) were shocked. I want to model a recovery in employment levels. I used an AR(2) model of quarter-on-quarter (q/q) employment ...
user avatar
  • 207
2 votes
2 answers
169 views

What is a "filter" and what does "filtering" mean in statistics/engineering/computer science?

I see the term "filter" in many neuroscience papers including those with heavy statistical content ("spatial filter", "temporal filter", etc.), as well as those with ...
user avatar
  • 139
0 votes
0 answers
90 views

Help with time series comparisons using periodograms

I have a dataset consisting of time series signals of different lengths obtained from different groups of patients. I am trying to understand the commonalities of the time series of each group. ...
user avatar
  • 123
1 vote
1 answer
94 views

Is there a way to filter a heatmap based on relevance?

I'm currently working with ComplexHeatmap and a very large dataset from RNAseq (~13,000 genes/columns). The heatmap output (based off of clustering) at the moment contains so many columns it's ...
user avatar
5 votes
2 answers
109 views

Seeking recommended literature search terms for a solution to a specific kind of data structure?

Hopefully this isn't considered too off-topic. I'm working in industry these days and came up with a solution to an analysis problem we'd been facing. I'd like to get a sense as to whether said ...
user avatar
1 vote
0 answers
89 views

What is wrong with my approach on a custom way of creating Gabor-filter convolution kernels?

Disclosure: I am not a prominent mathematician (current bachelor student) like others on this website and my approach has been mostly pragmatic. Please do tell me if I can improve the formulation of ...
user avatar
2 votes
0 answers
48 views

How does choosing an even window size actually add a cyclical component to the model?

I am new to Time Series Analysis. Say, we have a time series $(y_{t})_{t}$ that we want to filter with a moving average filter. I have been told that we should choose the window size $L$ of the filter ...
user avatar
  • 139
1 vote
1 answer
32 views

Why "run the filter longer than needed and remove the initial values" will solve the issue of recursive solving equations?

Consider sequence of random variables $w_i$ iid normal(0,1). Given the equation, $x_t=x_{t-1}-0.9x_{t-2}+w_t$ with $t$ discrete, I want to solve for $x_t$ recursively by prescribing $x_1,x_2$. The ...
user avatar
  • 839
1 vote
0 answers
20 views

Does feature detector(filter) has to be a sqaure matrix?

I am going through a course on Convolutional neural networks, where in the convolution step, the feature detector matrix was square shaped. Is there any mathematical significance that Feature ...
user avatar
  • 33
1 vote
0 answers
74 views

How are the values of the Grubb's table calculated for use in the Grubbs filter?

I am looking at Grubb's test for outliers. The approach seems simple enough but for completeness I have two questions. The Grubbs test is defined as $$G_{\rm{test}} = \frac{\left| x_{i} - \bar{x} \...
user avatar
  • 268
1 vote
2 answers
19 views

How can a given conv neural net layer handle filters of different size?

traditional method is to use multiple filters of same dimensions but with different weights and stack the output (basically concatenate them) that is then to be fed into the next conv layer. If I ...
user avatar
0 votes
1 answer
89 views

Time series with long "idle" periods - is it safe to eliminate those periods?

Suppose that I have as input a time series $T = \{ t_1, t_2, ..., t_M \}$ where each point is sampled at a fixed time interval (e.g. every 10 ms). The problem is that $T$ contains a lot of periods ...
user avatar
1 vote
0 answers
22 views

recommendation based on multiple user

I am learning about the recommendation system. How can I make a system where it takes multiple users as input and based on the rating and another attribute it gives recommendation? I have data sets ...
user avatar
0 votes
1 answer
25 views

Need precision about an example in a book about bayesian filters

My question is about the example here : https://github.com/w407022008/Kalman-and-Bayesian-Filters-in-Python/blob/master/02-Discrete-Bayes.ipynb#Adding-Uncertainty-to-the-Prediction paragraph : ...
user avatar
  • 101
2 votes
2 answers
107 views

Do earlier hidden layers learn more concepts/features than later ones, in neural networks?

I am wondering whether there is a general statement of the sort "earlier layers in neural networks learn more concepts/features than later layers" or the other way around. The output layer not being ...
user avatar
  • 1,214
1 vote
1 answer
67 views

Small noise of state process and filtering

Assume we have a linear state-space model: $$ z_{k} = Hx_{k} + v_{k}\\ x_{k} = F x_{k-1}+ w_{k}. $$ We are interested in filtering, i.e. we aim to estimate $E[x_{n}|z_{0}, \dots, z_{n}]$. If the ...
user avatar
  • 406
1 vote
0 answers
63 views

Effect of log transformation or standardization of a regressor in the filtering step

We are working with a dataset that has hundreds of biomarkers (many of which are correlated) and often they have many missing values. Our initial goal was to use an elastic net but that would require ...
user avatar
  • 3,355
1 vote
1 answer
219 views

Optimality of Bayesian filtering

In Kalman filter, we can show it's a minimum variance filter, which I believe is due to the linearity of system and the Gaussianity of noise. It comes to me that what is the optimality criterion used ...
user avatar
1 vote
0 answers
983 views

Kalman filter for AR(1) plus noise

I am working the following AR(1) plus noise state-space model $$ z_{t} = x_{t} + v_{t}\\ x_{t} = \phi x_{t-1} + c + w_{t} $$ Therefore, the transition matrix is $[\phi]$, the observation matrix is $[1]...
user avatar
  • 406
1 vote
0 answers
313 views

Exponential moving average before computing std

In which cases it would make sense to use exponentially weighted moving average (EWMA) before, for example, computing sample variance or other statistical analysis? Could you give an example when one ...
user avatar
  • 406
2 votes
0 answers
39 views

Signal Decomposition

I have two time dependent signal sources X & Y. Both can be modeled as having a linear combination of time dependent individual components and common components; so X(t)=a(t)+C(t)+noise, Y(t)=b(t)+...
user avatar
1 vote
1 answer
23 views

Filter out linearly interpolated historical data points

I am reading in historical sensor data from a plant. I found out that there are intermittent periods where between time t1 and time t2, the data points are linearly interpolated. I came to know, that ...
user avatar
0 votes
1 answer
879 views

What is the difference between 1x1 convolutions and convolutions with "SAME" padding?

In general, 1x1 convolutions are used to reduce the dimensionality of filter space. I referred this answer. But we can also reduce the dimensionality of filter ...
user avatar
  • 171
0 votes
1 answer
811 views

How can we determine the appropriate number of hidden layers, kernels in convolutional neural network (CNN)?

I have checked a lot of questions here and in other websites. What I concluded is that there is no rules for choosing the right number of hyper-parameters in CNN, all what can we do is just trying ...
user avatar
0 votes
0 answers
98 views

Noise reduction with known noise distribution

I have a time signal with a known noise distribution parameters (gaussian, sd is known). I would like to estimate the true value statistically and in the best case obtain a confidence interval. As I ...
user avatar
  • 1,119
1 vote
0 answers
38 views

Time series filtering notation

I am looking at some suggested filters for tidal data but am having trouble understanding the notation. For example, Godin (1972) suggests a low-pass filter for tidal data that is a combination of 24-...
user avatar
  • 205
1 vote
0 answers
229 views

Is there a filter function that performs similar to moving average but does not loose data?

I need to smooth a time series using a low pass filter. A simple moving average is working fine for me, however, using a moving average causes an inevitable loss of data a the beginning of the ...
user avatar
  • 899
1 vote
1 answer
72 views

Partial spline. Reference [closed]

I have a well done and perfectly working protocol to smooth my experimental data. I do the following: I have a variable of size 1000. Iteratively I choose random 100 points and spline them using the ...
user avatar
  • 663
1 vote
0 answers
193 views

Faster RCNN - Pyramid of Filters vs Pyramid of Anchors (Reference Boxes)

I'm reading faster RCNN paper now and trying to understand what is the difference between Pyramid of Filters and Pyramid of Anchors methods from the scale point of view. I mean if I use only one ...
user avatar
0 votes
1 answer
43 views

Filter rows in r having same categorical value in all columns and also rows with all different categories [closed]

I am attempting to filter rows from following dataset where a,b and c give same answers, and also where a, b and c all give different answers from a category of 3 answers. id A B C X1 X2 ...
user avatar
  • 11
15 votes
3 answers
5k views

Wouldn't multiple filters in a convolutional layer learn the same parameter during training?

Based from what I have learned, we use multiple filters in a Conv Layer of a CNN to learn different feature detectors. But since these filters are applied similarly (i.e. slided and multiplied to ...
user avatar
  • 343
1 vote
0 answers
80 views

Great ways to identify adult content in text

What are some good ways to identify adult content in text. It is definitely a text classification problem, but how do we handle words that are spelt like @$$.
user avatar
1 vote
2 answers
3k views

Removing gaussian noise from a time-series data

I have a noisy time-series data (Figure 1). As you can see the variance in this data set is very high and the "Gaussian noise" needs to be removed for me to analyze this signal. Normally we ...
user avatar
0 votes
0 answers
81 views

In prewhitening, should we develop an ARIMA(p,q,d) for X but use only ARI(p,q) as the filter?

According to the answer in this question by IrishStat, the reason that you pre-whiten X is to identify a filter that can transform Y and X into y and x where x is white noise. Assume that X is ...
user avatar
  • 303
1 vote
0 answers
16 views

checking the quality of filtering of a temporal series

I have many temporal series of satellite data. I decided to smooth that using Savitzky-Golay filter implemented in R using the "signal" package. I could use also other smoothing algorithms, that's ...
user avatar
  • 111
0 votes
0 answers
22 views

Appropriate Feature Selection model for Mass Spectrometry data

I have a cancer patients data which consists of more than half million features and my task is to apply feature selection algorithm to extract the most relevant features from it. My question is which ...
user avatar
1 vote
0 answers
239 views

Estimate standard deviation of random-walk using Kalman filter

I created a Kalman filter that takes in time series observations and estimates the mean of that time series. This is simply modeling a random walk. However, I also want to be able to estimate the ...
user avatar
2 votes
2 answers
129 views

some examples of filtering the noise out of a data set

I have a data set which measures 60 data points in a second (60Hz). Clearly, I do not really need all 60 points in one second since this only generates some noise. ABOUT MY DATA: So my data sets ...
user avatar
  • 7,565
1 vote
1 answer
362 views

Denoising technique for signal with beforehand known shape (linear and exponential)

I have a noisy signal which is linear and then exponential. I know the type (Gaussian additive noise) and degree (0.01) of noise. Part of the challenge is determining when the signal changed from ...
user avatar