Compressed Image File Formats: JPEG, PNG, GIF, XBM, BMP

This book explains the process of decoding for programmers:

https://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434

The entire process is way to complicated to fit within the space of a SO answer. PNG uses two different methods of compression: LZ and Huffman encoding.

JPEG is somewhat unique in that it involves a series of compression steps. There are two that provide the most opportunities for reducing the size of the image.

The first is sampling. In JPEG one usually converts from RGB to YCbCR. In RGB, each component is equal in value. In YCbCr, the Y component is much more important than the Cb and Cr components. If you sample the later at 4 to 1, a 4x4 block of pixels gets reduced from 16+16+16 to 16+1+1. Just by sampling you have reduced the size of the data to be compressed by nearly 1/3.

The other is quantization. You take the sampled pixel values, divide them into 8x8 blocks and perform the Discrete Cosine transform on them. In 8bpp this takes 8x8 8-bit data and converts it to 8x8 16 bit data (inverse compression at that point).

The DCT process tends to produce larger values in the upper right corner and smaller values (close to zero) towards the lower left corner. The upper right coefficients are more valuable than the lower left coefficients.

The 16-bit values are then "quantized" (division in plain english).

The compression process defines an 8x8 quantization matrix. Divide the corresponding entry in the DCT coefficients by the value in the quantization matrix. Because this is integer division, the small values will go to zero. Long runs of zero values are combined using run-length compression. The more consecutive zeros you get, the better the compression.

Generally, the quantization values are much higher at the lower left than in the upper right. You try to force these DCT coefficients to be zero unless they are very large.

This is where much of the loss (not all of it though) comes from in JPEG.

The trade off is to get as many zeros as you can without noticeably degrading the image.

The choice of quantization matrices is the major factor in compression. Most JPEG libraries present a "quality" setting to the user. This translates into the selection of a quantization matrices in the encoder. If someone could devise better quantization matrices, you could get better compression.

This book explains the JPEG process in plain English:

http://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434/ref=sr_1_1?ie=UTF8&qid=1394252187&sr=8-1&keywords=0201604434

The encoding process depends upon whether you have a sequential scan or a progressive scan. The details of the encoding process are too complicated to fit within an answer here.

I highly recommend this book:

https://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434/ref=sr_1_2?ie=UTF8&qid=1531091178&sr=8-2&keywords=JPEG&dpID=5168QFRTslL&preST=_SX258_BO1,204,203,200_QL70_&dpSrc=srch

It is the only source I know of that explains JPEG end-to-end in plain English.

JPEG is extremely complex. The only source I know of that explains this for programmers is:

https://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434

This is the only book that actually explains the process for programmers without academic mumbo-jumbo

https://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434/ref=sr_1_2?ie=UTF

https://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434/ref=pd_bxgy_14_img_2?_encoding=UTF8&psc=1&refRID=VPB1FTFQG5PJH1AFZ51M

https://www.amazon.com/dp/0201604434/