Which image format is better for machine learning .png .jpg or other? I'm trying to train a neural network with images. Since I'm extracting images from a video feed I can convert them either to .png or .jpg. Which format is preferred for machine learning and deep learning. My neural network model contains convolutional layers, max pooling layers and image resizing.
 A: Here is a real-life case: accurate segmentation pipeline for 4K video stream (here are some examples). I do rely on conventional computer vision as well as on neural nets, so there is a need to prepare high-quality training sets. Also, it is somewhat impossible to find training sets for some specific objects:   

(See in action)
Long story short it is about 1TB of data required to create a training set and do additional post-processing. I use ffmpeg and store extracted frames as JPG. There is no reason to use PNG because of the following:


*

*video stream is already compressed

*any single frame from the compressed stream will contain some artefacts

*it might look a bit strange to use lossless compression for lossy compressed data

*there is no reason to consume more space storing the same data

*also, there is no reason to consume additional bandwidth 


Let's do a quick test (really quick). Same 4K stream, same settings, extracting a frame as PNG and as JPG. If you see any difference -- good for you :) Any real-life problem will likely be related to a compressed video stream because bandwidth is critical. 
PNG

JPG

Finally
If you need more details -- use 4K (or 8K if you need even more valuable details). Pretty much all the examples I have are based on 4K input. FPS is what actually matters when you try to deal with real-life scenes and fast moving objects. 

(see in action)
It goes without saying camera and light conditions are the most critical preconditions for getting proper level of the details.
A: JPG performs better for photorealistic images, PNG for drawings with sharp lines and solid colors. For frames of video feed I would definitely use JPG.
UPDATE: Because video is usually compressed in a way similar to JPG, it is unlikely that quality will degrade further than it already has. And dataset size is not unimportant either.
A: As others have indicated in comments, PNG as lossless format is better suited than JPEG. That being said, as an input to your pipeline the answer would be "neither". Almost always you have to preprocess your images, e.g. crop them, subtract mean etc. In a typical scenario, I would run my pipeline ~100 times before I am happy with the results. Preprocessing your images every time would be simply waste of resources, especially since reading images takes time.
Much better idea is to use HDF5, format very well suited to work with numerical data and slices. Typically you would:


*

*Load an image.

*Preprocess it.

*Save to HDF5. You'd create two datasets: one for the image data, second for the label (if present).

*Rince and repeat.


If HDF5 is new to you and may seem somewhat abstract, below you will find an example of class that can be used to store your images to disk using this format. 
import h5py
import os

class HDF5Writer(object):
    def __init__(self, dims, output_path, data_key="images", buf_size=1000):
        self.db = h5py.File(output_path, "w")
        self.data = self.db.create_dataset(data_key, dims, dtype="float")
        self.labels = self.db.create_dataset("labels", (dims[0],), dtype="int")

        self.bufSize = buf_size
        self.buffer = {"data": [], "labels": []}
        self.idx = 0

    def add(self, rows, labels):
        self.buffer["data"].extend(rows)
        self.buffer["labels"].extend(labels)

        if len(self.buffer["data"]) >= self.bufSize:
            self.flush()

    def flush(self):
        i = self.idx + len(self.buffer["data"])
        self.data[self.idx:i] = self.buffer["data"]
        self.labels[self.idx:i] = self.buffer["labels"]
        self.idx = i
        self.buffer = {"data": [], "labels": []}

    def store_class_labels(self, classLabels):
        dt = h5py.special_dtype(vlen=str)
        labelSet = self.db.create_dataset("label_names", (len(classLabels),), dtype=dt)
        labelSet[:] = classLabels

    def close(self):
        if len(self.buffer["data"]) > 0:
            self.flush()
        self.db.close()

I recommend reading a tutorial on h5py, the extra learning curve is worth it. The buffer simply tells you every how many images data should be written to the drive. 
How to use it (in pseudo-code): 
data_dim = (num_of_images, x_dim, y_dim, num_of_colours)
writer = HDF5Writer(data_dim, output_path)

for image_path in paths:
    image = read_image(image_path)
    image = preprocess(image) 
    writer.add([image], [label])

writer.close()

The heavy downside of HDF5 is that it will blow your dataset size 50x if used without any compression. At certain level it may seem not feasible to use it, but then likely you'll have to use specialised compute infrastructure anyway (and then storage space won't be an issue).
