Questions tagged [compression]

Data compression is a process used to reduce the number of bits used to store a "message". Compression can be lossless or lossy. Lossy compression is an option for audio and visual data, whereas many other applications require lossless compression.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

What criteria charactarize the minimum rate for transferring information without loss?

My question in other words is: what is the condition to do data compression and later decompress the data without any loss ?
0
votes
0answers
19 views

Super-resolution approach applied on vibration data

I have low-resolution(LR) vibration data (e.g 3khz), and I mainly want to detect machine fault on that data. Is it a good idea, to train a convolutional neuronal network to create a high-resolution(HR)...
2
votes
0answers
74 views

Data compression for graph plotting [closed]

I am using Google Charts to plot a large data set. The database contains one record for every two seconds; five minutes' worth of data yields 150 records (data points) and the result is acceeptable. ...
0
votes
0answers
17 views

How do I calculate the kth order entropy of a list of strings?

I want to measure the $k$th order entropy of a set of strings $s_1, s_2 \dots s_n$. And I am confused between two possible approaches that can be used. Background For a string $s\in\Sigma^n$, the ($...
1
vote
0answers
127 views

Invertibility of Random Fourier Features

Is it possible to approximately reconstruct a point $ \mathbf{x} $ in a vector space (say $\mathbb{R}^n $) given it's randomized feature map $ z(\cdot) $ and respective projection $ z(\mathbf{x})$ (in ...
2
votes
0answers
17 views

From a deep learning point of view, is there a lower limit on the number of hours of speech needed to train a neural net

From a deep learning practitioner's point of view, is there a lower limit on the number of hours of speech needed to train a neural net to translate speech to text? An estimate from CMU is 3000-5000 ...
1
vote
0answers
20 views

Finding a small description length/compression of variable length lists of rational numbers [closed]

Suppose we have data in form of variable length lists of rational numbers where the order in which the numbers are listed is arbitrary. What is the best way to represent/encode this data so that all ...
2
votes
1answer
25 views

Is there a formal relation between weight regularization and compression?

In my understanding, compression, strictly speaking, means that we diminish the amount of data required to describe something, such as a model. E.g. compressing an image file means to create a file ...
0
votes
1answer
366 views

Minimum Input Dimension for Autoencoder Neural Network

Model: Assume we want to learn patterns using an autoencoder neural network. In the simplest case, such a network is "shallow" with 1 hidden layer, takes a $d$-dimensional numerical input vector $x$, ...
1
vote
0answers
68 views

Regarding the quantics tensor train (QTT) format

I originally posted this question in Data Science Stack Exchange, however, I think this forum may be better for this question. I believe I have a fair understanding of the tensor train (TT) format, ...
1
vote
2answers
69 views

Compression of 18000 curves

I have over $18000$ curves that I need to compress to save $\geq 50\%$ of space. Each curve is described by points $f(1), f(2), ..., f(96)$, each $f(x)$ is 8-bit long. The curves in compressed form ...
2
votes
0answers
83 views

Is there an extension of PCA for data embedded in hyperbolic spaces?

I'm working on a project where we are embedding data into an n-dimensional Poincare ball similar to this paper. However, we'd like to take the additional step of reducing this data to a 2-dimensional ...
2
votes
2answers
59 views

Lossless Compression Of Data Tables Leveraging Exploratory Analysis?

Lossless compression of data tables seems a natural application of automated modeling based on exploratory data analysis methods. While such automatically generated models are not reliable for ...
0
votes
0answers
31 views

Differentiable remover of repetition?

In a pet project to see why automatic phonetic transcription is so hard, I would like to use both audio data where each sample is annotated by the sound to which it belongs (a phonetic segments tier) ...
1
vote
1answer
187 views

Mutual information of BSC Channel

As we all know, formula for mutual information is as follows: $ I(X, Y) = H(Y) - H(Y|X) $ In case of Binary Symmetric Channel with flip probability equal to $f$ and both alphabets of size $2$: $ I(...
2
votes
0answers
141 views

Iterative PCA in C++

I have a time-series data set (10,000 samples/sensors over ~2,000 time-series samples). I would like to keep the covariance structure in the data (sensor-sensor covariance) up to some error, while ...
1
vote
2answers
166 views

Binary Sparse Coding

In this binary sparse coding paper referenced in the Goodfellow/Bengio/Courville deep learning book (https://fias.uni-frankfurt.de/~bornschein/papers/HennigesEtAl_lva2010.pdf), the parameter $\pi=p(...
0
votes
1answer
128 views

Combined PCA and Quantization

I did some PCA on a dataset to reduce the size without compromising too much on their actual information. However, I learnt that quantization is also an effective technique to do this. I guess it is ...
4
votes
2answers
509 views

When and why do we use sparse coding?

Sparse coding is described as "given an input $X$, finding a latent representation $h$ such that h is sparse and the input can be reconstructed as well as possible." (source: https://www.youtube.com/...
0
votes
2answers
784 views

Can white noise be (losslessly) compressed?

Can I expect a vector whose components are i.i.d. Gaussian noise to be compressible (lossless compression)? Why or why not? If so, how much? EDIT: To be clear, I mean "compressible" as in "...
23
votes
3answers
2k views

Kullback-Leibler divergence WITHOUT information theory

After much trawling of Cross Validated, I still don't feel like I'm any closer to understanding KL divergence outside of the realm of information theory. It's rather odd as somebody with a Math ...
3
votes
3answers
2k views

How does SVD save space?

We start with an $m \times n$ matrix before SVD. After SVD, we have three matrices of sizes, $m \times m$, $n \times n$ and $m \times n$. How do we save space then if now we have three matrices ...
2
votes
0answers
235 views

Relation between cross-entropy loss on softmax output and bits per base in DNA sequence compression

When training an autoencoder (softmax in outputlayer), for DNA sequence compression (by minimizing a cross-entropy loss function), how can you calculate the number of bits per base required for the ...
3
votes
1answer
200 views

Compressed Sensing: Missing Fourier Coefficients?

This question is regarding the problem of reconstructing a signal given only a subset of the Fourier coefficients are observed: $$\min_x \|x\|_1 \text{ subject to } y = Ax$$ where $x = (x_1,x_2,\...
2
votes
1answer
103 views

Local redundancy - Product Quantization and global redundancy - Residual Quantization connection

I am currently reading Compressing Deep Convolutional Networks using Vector Quantization paper. The paper states in section 3.2.5 that Product Quantization explores some local redundancy and Residual ...
5
votes
1answer
381 views

Why low rank expansions can exploit the redundancy that exist between different feature channels and filters?

I read Jaderberg et al., 2014 paper about Speeding up Convolutional Neural Network with Low Rank Expansions. In the introduction, it is written in bold font: Our key insight is to exploit the ...
2
votes
0answers
165 views

Should we pre-train neural networks on compressed data?

I believe this is actually related to the topic of "auto-encoding" but my question is trying to express some basic gaps in my knowledge that I have not seen directly addressed anywhere. You can train ...
2
votes
2answers
95 views

Detecting if a File is Compressed using Machine Learning?

Let's say that I want a 'general' way using some machine learning model to classify if a file is possibly compressed or not. I don't need a 100% success rate. How would you approach this? I ...
1
vote
0answers
91 views

Clustering technique and validation for distance based on file compression

I have a distance matrix based on a normalized compression distance between files: $$ d(x, y) = \frac{ C(xy) - \min \{ C(x), C(y) \} } {\max \{ C(x), C(y) \}} $$ Here, $C(xy)$ is the concatenation ...
0
votes
0answers
15 views

Removing minimally informative bits

Each sample in my data set is an $N$-bit bit-vector (say $N$=200). The bits in each sample are not uncorreleated within the sample. I built a matrix $S_{N \times N}$ with each $s_{ij}$ being the ...
2
votes
0answers
164 views

Analyzing 3D data: What can be done?

I am new to this kind of analysis, and I want to know what values I can look at in 3D data. The data itself is a 3D volume $(x,y,z)$ with a floating point value in every coordinate. It is a ...
8
votes
1answer
6k views

Comparison of entropy and distribution of bytes in compressed/encrypted data

I have some question which occupies myself for a while. The entropy test is often used to identify encrypted data. The entropy reaches its maximum when the bytes of the analyzed data are distributed ...
1
vote
1answer
328 views

Distance independent approximation of Nearest Neighbor/k-NN.

Nearest neighbor/k-NN for use with Normalized Compression Distance. I wonder if there exist any approximation of NN/k-NN algorithm which work for all distance measures ? I would like to test ...
1
vote
1answer
517 views

Dense Data Compression (Delta Encoding?)

Let $X $ be a dense integer set such that its elements are closely knit in value. For instance: 1 1 1 1 2 2 2 2 2 3 3 5 5 6 6 6 6 6 7 7 7 7 8 8 8 8 8 8 ... I am ...
4
votes
1answer
108 views

Weights of random sets of random 32-bit strings

I have random sets of $N$ random 32-bit strings, where all bits are i.i.d. with $\mathbb{P}(0) = \mathbb{P}(1) = 1/2$. Define $\ \ \ \ $weight( 32-bit x ) = number of 1 bits in x, i.e. Hamming ...
3
votes
2answers
897 views

How to compress sets of integer series?

I have a set of integer series $S_1$, $S_2$, ... $S_n$. Each series has 3600 data points. Each data point is a positive integer. Each data point is stored as an ...
4
votes
4answers
891 views

Ultimate compression algorithm

I was not sure where to put this question, so I put it here. Feel free to move it to another stack exchange site moderators. Lets say I have a 10 gigs of pictures (or for that matter any type of data,...
7
votes
2answers
255 views

Compression theory, practice, for time series with values in a space of distributions (say of a real random variable)

Example of problem: Part of our research team is working on providing operationally wind power forecast. Usually, since there are different time scalse that interest forecast user, a forecast is ...
2
votes
1answer
647 views

How to compute theoretical compression limit?

Assume we have a sensor field with dimension M*M. In order to apply any data compression technique, first I want to know what is the compression limit or minimum entropy of the entire sensor field. ...
0
votes
2answers
888 views

Suggest a method for statistical data compression

There's a lot of work done in statistics, while state-of-art in lossless data compression is apparently this: http://mattmahoney.net/dc/dce.html#Section_4 Please suggest good methods/models ...