2
$\begingroup$

I am working on a prognostics task, where I predict the Remaining Useful Life of some equipment (i.e.: time steps remaining until failure). In order to do that, I use multivariate time series sensor data, which contains several run-to-failure recordings for different units. For each time step I can calculate the number of time steps remaining until failure, and use those as a target for a 1D Convolutional Neural Network model.

Therefore, the problem consists of modeling the input-output mapping between a tensor $X \in \mathbb{R}^{n\times d}$ and a scalar $y \in \mathbb{R}$, where $n$ is the length of my sliding time windows and $d$ is the dimensionality of the input data.

The time window lengths are relatively short ($20 \leq n \leq 40)$, and because of that I chose not to use pooling layers, as the convolutions themselves reduce the size of the tensor to some extent already and the dimensions are not too large. The resulting model has multiple 1D Convolution / Dropout layer pairs (the output from the convolution layers goes through a non-linear activation function), followed by one flatten and one dense layer leading up to the output. An example of a model summary is given below (I use Keras):

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_66 (Conv1D)           (None, 34, 16)            336       
_________________________________________________________________
dropout_66 (Dropout)         (None, 34, 16)            0         
_________________________________________________________________
conv1d_67 (Conv1D)           (None, 31, 32)            2080      
_________________________________________________________________
dropout_67 (Dropout)         (None, 31, 32)            0         
_________________________________________________________________
conv1d_68 (Conv1D)           (None, 28, 64)            8256      
_________________________________________________________________
dropout_68 (Dropout)         (None, 28, 64)            0         
_________________________________________________________________
conv1d_69 (Conv1D)           (None, 25, 128)           32896     
_________________________________________________________________
dropout_69 (Dropout)         (None, 25, 128)           0         
_________________________________________________________________
flatten_28 (Flatten)         (None, 3200)              0         
_________________________________________________________________
dense_28 (Dense)             (None, 1)                 3201      
=================================================================
Total params: 46,769
Trainable params: 46,769
Non-trainable params: 0

So my question is: do pooling layers serve any purpose in this type of problem (other than to decrease the tensor length without introducing trainable parameters)?

In other words, am I doing something conceptually wrong or potentially hurting my model's performance by not using pooling layers?

Can you think of any reason why a pooling layer could in fact be beneficial in this scenario?

$\endgroup$
1
  • 2
    $\begingroup$ I mean, I would just try whether it gives good enough performance. Also this kind of data seems to be a better fit for sequence models like LSTM/GRU/RNN (due to variable length), rather than Conv1D. My experience that it is incommon to use Conv1D with pooling. $\endgroup$
    – boomkin
    Jan 6, 2019 at 20:35

2 Answers 2

2
$\begingroup$

Polling layer is used to extract the more finer information from data(and size reduction is its byproduct).

Think this as following, On last dropout layer, you have (None, 25, 128) dims, which is nothing but 128 filters, each of 25 dims. As each filter carries information about input text. Pooling will helps to get rid of redundant or irrelevant information, which further helps Dense layer to focus on the more finer information of data.

Note: Even concatenating outputs from Max-Polling and Global-Average-Polling, gives comparable result, in case of CNN1D. For more info

$\endgroup$
0
$\begingroup$

So my question is: do pooling layers serve any purpose in this type of problem (other than to decrease the tensor length without introducing trainable parameters)?

The pooling operation helps reduce the number of dimensions and helps down sampling. In PHM tasks, this trait of the pooling operation would help remove unnecessary/redundant features (think of the max pooling operation).

In other words, am I doing something conceptually wrong or potentially hurting my model's performance by not using pooling layers?

Pooling operations are not very necessary if data size is sufficient. However, it's good to reduce the number of parameters by the pooling layer, as the data cannot always be sufficient.

Can you think of any reason why a pooling layer could in fact be beneficial in this scenario?

Using a pooling layer (assuming other architectural details are not changed) would help reduce the complexity of the model, therefore reducing the possibility of overfitting. In addition, in RUL estimation using sensor signals, pooling would help extract more from redundant signals (for instance, effects from too high-frequency data can be alleviated to some degree).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.