Musings on image data layout for maximising compression.

[Raizor]

I've been playing around with image formats, so I can store my graphics in RAW format inside my productions without relying on PNG or JPG decompression.  The goal was to come up with a simple format that didn't need decoding, which would save me having to carry a PNG or JPG decompressor.

I'm currently storing the data in simple RGBA or BGRA format, so the data is sequential with a byte for each colour per pixel (i.e. R-G-B-A, R-G-B-A etc). I figured something like Kkrunchy or UPX would have little trouble compression the data when stored inside an exe.

After some initial tests and some reading of how Crinkler works, I started thinking about ordering the data into streams, so each colour is stored in its own stream. This means the file data is ordered R-R-R-G-G-G-B-B-B-A-A-A for example.  Depending on the data, this can greatly increase the compressibility of the file.

I've been using WinRAR to compress the output in order to gauge what sort of compression ratio I'd get with kkrunchy/upx. I'm working on a little program that will convert and re-order the data and will try a number of different orderings to determine which one best suits the file being processed.

This is never going to beat JPG, but I often seem to get a better ratio using my method vs. PNG. Here are some examples:

PNG = size in bytes of orginal PNG file.
RBA(I) = size in bytes using RGBA interleaving (RGBA, RGBA...)
RBA(S) = size in bytes using RGBA streams (R.., G.., B.., A..)
RBA(ISA) = size in bytes using interleaved RGB with trailing Alpha stream (RGB, RGB..., A...)

FILENAME  |  FILE DETAILS                              |  PNG         | RGBA(I)      | RGBA(S)   | RGBA(ISA)
---------------------------------------------------------------------------------------------------------------------------
space.png  | 1024x1024x24bpp  (no alpha)   | 848,420    |  916,300    | 844,453   | 611,094
---------------------------------------------------------------------------------------------------------------------------
fire.png      | 640x640x24bpp  (no alpha)       | 709,003    |  651,211    | 699,500   | 398,535
---------------------------------------------------------------------------------------------------------------------------
logo.png    | 1371x883x24bpp  (alpha)          | 138,071    |  168,119    | 143,732  | 145,732

The results so far are quite interesting (at least to me). I know comparing RAR compression to KKrunchy/UPX is not accurate so I will need to perform some more tests against those two exe packers to be sure, but I'm fairly confident they will yield similar results.

As we can see, for complex images (with no alpha channel) the streaming method allows for better compressibility than PNG. It might be possible to further reduce the PNG images using something like PNG-Crush, which may make this whole exercise moot. It's been interesting playing around with this anyway.

I just thought I'd share this and see if anyone else had any other ideas on different data arrangement patterns to increase compressibility.

Cheers,

Raizor

[ferris]

I've found the best results BY FAR came from employing a basic DCT transform on the image, quantizing, and doing some spatial transforms on the resulting weights; dropping images around 192kb in size to under 3kb with decent (close to jpeg and greyscale) results. It's currently an integral part of my 64k system It's shit-simple to do, too.. so yeah, although you're trying to avoid anything like that, DCT would be a cool thing for you to look into

[Raizor]

I probably should look into DCT Ferris, as doing this my way can only go so far. I will have a look around and try and find some good info and hopefully some example code. Thanks.

[hellfire]

In image processing you mainly have to deal with gradients and transients.
Since the quantization of dct coefficients approximates a frequency spectrum and transients scatter over the whole spectrum, you will have to store rather many values at image discontinuities.
For image compression you're usually much better off using wavelets which store only a single singnificant value at transients.
This property makes even the simplest quantization shemes very efficient.
A simple transform function can be found here.
You should also try to work in yuv instead of rgb as quantization errors are less perceptible.

[ferris]

Interesting stuff about wavelets; never actually played with them before but I know some intros have used them in the past (Kolor's Fresnel series comes to mind)...
Cool stuff, hellfire!

[mind]

here's a little something i was working on a while ago. i'm fairly sure this can be optimized ALOT seeing as how all the data is stored as raw and NOT compressed at all in my results. i guess the link speaks for itself, so here it is: www.synthesi.st/test/apa.html

BTW! might be NSFW, depending on your boss it contains a demoscene image of a half naked chick, so be careful when opening it

[Raizor]

That's very interesting Mind, I didn't expect the quality of the final image to be so good. The only area that really stands out as far as colour-bleed is concerned is the whites of the eyes.

Thanks for sharing

[Kirl]

Thanks for sharing that mind, excellent trick!   

[mind]

you are very welcome.. yeah i think it's a very neat trick and as far as i know(did some extensive research back when i first thought of it) it hasnt been thought of/done before, and seeing as i will never really use the technique for anything i might aswell share it with people who might

consider this aswell: a grayscale image can be saved using only the first 63 entries in an indexed palette, which should help compression even more.

ohh and btw, the photoshop blending mode "color" is a HSV(HSL really) blending mode, combining the Hue and Saturation of the color image with the Lightness of the grayscale one, if i recall correctly.

[hellfire]

as far as i know (did some extensive research back when i first thought of it) it hasnt been thought of/done before

Please check Chroma subsampling

[mind]

as far as i know (did some extensive research back when i first thought of it) it hasnt been thought of/done before

Please check Chroma subsampling

oh god damnit, i thought i was the first to think of this but wtf, 1950? lol.. thx for the link, gotta do some reading up on that, see if i can make it better, faster, stronger.. SCOOTER! err, whatever thx