How does one generate (smooth) varying size output signals with Machine Learning?

Question

I am interested in knowing about generative methods that generate signals (e.g. images) of varying sizes. But the size generation being sort of "smooth/continuous". So for example, generating images of total size 3000 smoothly up to 196,608 (or even more but every number in between matters to me) with a single model.

The only (what I think is totally ridiculous) idea I had is outputting each signal value (e.g. pixel) via an RNN (since RNNs are size-independent). So that completely solves the issue but my RNN might end up being of length 3000 or 196,608 at times (or hopefully even more)...which seemed ridiculous to me. Is there any real machine learning paper (ideally Deep Learning) that solves such a task?

In fact, are there any image generation tasks with such nature? I am not familiar of any but if anyone knows that would be fantastic! Note, however, that this is not really a transfer learning problem. So have 196,608 version of the output layer seems a bit crazy for me (since I might even want more...). The only made-up task on the spot I can come up with for this is generating all size crops for imagenet data set and then trying to generate any of them. I guess that might be an instance of the task I have in mind. Is there a real/serious task like this that the Machine Learning community is working on?

---

Cross posted:

• https://qr.ae/TWvIYP
• https://www.reddit.com/r/MachineLearning/comments/cimpmp/how_does_one_generate_smooth_varying_size_output/

Answer 1

The approach: fully-convolutional generative models

You could try using a fully-convolutional generative model such as a Variational Autoencoder, which has been used for many image generation tasks. Variational Autoencoders (VAEs) are made of an encoder network which compresses an image to a lower-dimensional Gaussian representation and a decoder network which reconstructs the original image. If you feed noise into the decoder network directly you can generate images.

An Example

Since a convolutional filter can be applied to an image of any size, fully-convolutional models can take in arbitrary images and will produce images with output sizes which are a constant fraction (or constant multiple) of the input image size. To use an absurdly very simple example, imagine you trained a VAE with an encoder made of one convolutional layer and a decoder made of one transposed convolutional layer (each with stride 2). If you generated noise of size MxN and fed it into the decoder half of your VAE, you would get an output of size 2Mx2N.

Producing even smoother outputs

This method along wouldn't produce quite smooth (for instance, the model described would only produce even-width/even-height images). If you care about having every single possible pixel dimension you could add an extra convolutional layer at the end with stride 1 (stride 1 will keep the image output size about the same as the input) and pad your image appropriately before passing it into the layer so the output has the desired size.

I couldn't find any papers using a model like this for the type of image generation task you described, but fully convolutional models have been used successfully for semantic segmentation of variable-sized images.