FRDX File Redux

× PNG vs WebP death match

PNG common types
[todo add dirty transparency explanation, including heatmaps]
Some of the samples used
The unoptimized sample files used here are available in this 1.2 MB archive, the PNG-8 files produced by pngquant 2.1 used in the second part are also available in this 0.4 MB archive.

Round 1 lossless (24-bit + alpha files – aka PNG-32 – or WebP lossless)

File Unoptimized PNGcrush ImageOptim ScriptPNG cryogen4   WebP   Black WebP
sample_00.png 734 558 558 552 547 846 622
sample_01.png 2783 2682 2578 2489 2542 2072 1874
sample_02.png 2805 2720 2652 2565 2557 2032 1912
sample_03.png 3998 3777 3718 3624 3647 3754 3466
sample_04.png 4345 3492 3395 3192 3170 3222 2566
sample_05.png 5150 4543 4263 4202 4248 3602 3112
sample_06.png 5163 4528 4226 4145 4211 3624 3100
sample_07.png 5425 5347 5214 5232 5210 3584 3584
sample_08.png 5570 5055 4982 4865 4842 4124 3808
sample_09.png 5621 5279 5009 4913 4955 4674 4064
sample_10.png 5827 5546 5421 5309 5286 4764 4288
sample_11.png 6538 5424 5295 5268 5271 4782 4280
sample_12.png 6556 5913 5829 5761 5714 5506 5002
sample_13.png 7032 5513 5219 5097 4983 5558 4374
sample_14.png 7258 7213 6985 6982 6964 5042 4888
sample_15.png 7376 5954 5865 5820 5818 5042 4192
sample_16.png 7594 6528 6469 6289 6276 6800 5832
sample_17.png 7778 6906 6468 6202 6277 5828 5296
sample_18.png 8828 8810 8126 7986 8110 6196 6196
sample_19.png 8982 7951 7733 7650 7650 6004 5550
sample_20.png 9871 9145 8732 8593 8641 7688 6964
sample_21.png 10566 6038 5749 5685 5692 7200 4988
sample_22.png 10882 10184 9864 9677 9512 8896 8058
sample_23.png 10995 10218 9497 9425 9397 8102 7394
sample_24.png 11354 10505 10208 9789 9810 8124 7284
sample_25.png 11891 7359 7065 6667 6669 8460 5446
sample_26.png 12269 10062 9595 9356 9348 9474 8118
sample_27.png 12756 11007 10951 10510 10576 8254 7734
sample_28.png 12824 11638 11090 11303 10913 8816 8022
sample_29.png 12942 8999 8613 8017 8373 9354 7020
sample_30.png 14782 14486 13901 13575 13602 11312 10778
sample_31.png 14930 13998 13883 13722 13781 13526 12598
sample_32.png 15168 13449 12881 12672 12727 11616 10516
sample_33.png 15420 13087 12901 12857 12873 12124 10962
sample_34.png 15778 14636 14122 13640 13712 12180 10840
sample_35.png 15799 14629 13957 13561 13600 10858 9782
sample_36.png 16283 15584 15013 14545 14581 13610 12702
sample_37.png 16318 15880 14856 14778 14708 11858 10908
sample_38.png 16805 16038 15128 14734 14764 12058 10952
sample_39.png 16940 13931 13735 13680 13643 12330 10926
sample_40.png 17274 15448 14962 14385 14529 14058 12386
sample_41.png 17548 16177 15641 15317 15385 15242 13796
sample_42.png 17605 14756 14412 13816 13884 14018 12206
sample_43.png 17718 14259 13797 13369 13382 12502 10436
sample_44.png 17777 16852 16392 15980 16182 13900 13238
sample_45.png 18289 18262 17251 17222 17246 13362 13362
sample_46.png 18536 14995 14803 14542 14499 14010 11972
sample_47.png 18584 16423 15961 15639 15615 13636 12134
sample_48.png 20086 17793 17313 16957 17052 15240 13840
sample_49.png 20129 19793 19066 18842 18858 16384 15702
sample_50.png 20302 19086 18531 17532 17950 15248 13838
sample_51.png 20395 19268 18359 18027 17922 14946 13974
sample_52.png 21026 20458 19421 19155 19095 14642 13768
sample_53.png 21557 20422 19776 19432 19454 16186 14772
sample_54.png 23855 22972 22079 21915 21872 18462 16978
sample_55.png 24789 23591 22979 22369 22453 20252 18924
sample_56.png 24897 14151 12784 12896 11851 19110 13238
sample_57.png 25390 22519 22200 22127 22228 19684 18208
sample_58.png 26475 23326 22665 22376 22492 22158 19576
sample_59.png 26733 25264 23985 23586 23627 20468 18476
sample_60.png 29500 29467 27409 27278 27313 21634 20506
sample_61.png 30994 30386 28754 28168 28247 22578 21254
sample_62.png 31938 31314 30617 30575 30567 27458 26554
sample_63.png 34633 31425 30240 30156 30206 24906 22864
sample_64.png 35044 28015 26929 26336 26140 25954 20908
sample_65.png 37644 36994 35217 34698 34761 30282 28440
sample_66.png 39790 29938 26718 26199 25261 30806 26332
sample_67.png 42907 41214 40664 40169 40165 32542 31142
sample_68.png 46395 32913 32209 32059 31895 34454 27172
sample_69.png 50055 44455 43993 42748 43166 39838 39614
sample_70.png 50059 46482 43013 42827 42720 39838 34844
sample_71.png 52806 26815 25692 25729 25267 45340 22540
Total 1290666 1129845 1085578 1067355 1066484 992034 872992
Percent 114.23 100.00 96.08 94.47 94.39 87.80 77.26
Time (seconds) tbd tbd tbd tbd tbd tbd tbd

PNGcrush version 1.7.71
options: -force -brute -blacken -max 1048576

ImageOptim version 1.5.3
All settings to the max, single pass

ScriptPNG dated 1st February 2014
level: 7 High (Max)
ScripPNG produces slightly different results based on CPU speed which is a bit annoying when doing benchmarks. [Update] this is no longer true in the latest version, I'll have to rerun it.
To circumvent this behaviour I had to replace "--max-time=1" by "--max-stagnate-time=0 --max-evaluations=140" in pngwolfz calls.

cryogen4 experimental December 2013
(dog-slow, don’t try this on production environment)

WebP version 0.4.0 + Change-Id I0917a8b91ce15a43199728ff4ee2a163be443bab
options: -lossless -alpha_cleanup -q 90

Black WebP script used to run PNGcrush -blacken followed by cwebp
options: pngcrush -force -blacken, cwebp -lossless -q 90

Commenting on a moving target is bit complicated (I still have to rerun the ScritPNG test) but the advanced alpha cleaning performed by TruePNG and the quick optimal filter search done by pngwolfz look like the keys to ScriptPNG good results.
Cédric Louvrier wrote an article about TruePNG to explain why he considers this tool as the heart of ScriptPNG.
The native alpha cleaning done by WebP is apparently more geared toward lossy compression since replacing it by a simple pngcrush -blacken helps in a big way.
[todo: add computing time and a second table on a separate page with "melt the ice caps" results: patched pngcrush, imageoptim second pass, scriptpng --max-evaluations=1200]

Round 2 lossy (8-bit + transparency files – aka PNG-8 – or WebP lossless)

 Some may argue that these samples could be turned into PNG-8 files to save even more bytes without too much apparent loss. And that’s exactly what has been done using pnquant…

File PNGcrush pngquant +PNGcrush +ImageOptim +ScriptPNG +cryogen4  +WebP 
sample_00.png 558 768 768 570 552 562 622
sample_01.png 2682 2115 2115 2035 2040 2114 1768
sample_02.png 2720 2047 2044 1973 1795 2047 1766
sample_03.png 3777 2384 2384 2296 2290 2384 2084
sample_04.png 3492 2492 2491 2321 2340 2489 2228
sample_05.png 4543 3316 3315 3231 3079 3314 2862
sample_06.png 4528 3311 3310 3222 3048 3308 2824
sample_07.png 5347 3601 3601 3413 3302 3531 3200
sample_08.png 5055 2685 2675 2531 2530 2685 2380
sample_09.png 5279 3305 3305 3221 2965 3305 2880
sample_10.png 5546 2582 2577 2483 2339 2582 2304
sample_11.png 5424 2851 2851 2718 2627 2851 2656
sample_12.png 5913 3172 3166 2952 2843 3171 2828
sample_13.png 5513 2530 2527 2323 2323 2526 2162
sample_14.png 7213 4861 4855 4547 3882 4861 4484
sample_15.png 5954 2874 2874 2776 2790 2873 2628
sample_16.png 6528 3231 3230 3127 3100 3230 2916
sample_17.png 6906 3960 3960 3783 3743 3958 3414
sample_18.png 8810 6303 6303 6002 5283 6034 5294
sample_19.png 7951 4680 4680 4510 4508 4674 4334
sample_20.png 9145 3924 3924 3643 3819 3924 3620
sample_21.png 6038 4336 4336 4216 4187 4336 3834
sample_22.png 10184 4688 4687 4461 4426 4687 3978
sample_23.png 10218 6771 6765 6350 5363 6770 6044
sample_24.png 10505 5646 5644 5313 5081 5646 4934
sample_25.png 7359 4026 4023 3817 3627 4025 3520
sample_26.png 10062 5036 5036 4890 4467 5036 4412
sample_27.png 11007 6742 6742 6554 6115 6738 6152
sample_28.png 11638 5483 5483 5321 5318 5482 5040
sample_29.png 8999 5832 5832 5538 5357 5831 5368
sample_30.png 14486 5752 5750 5449 5007 5752 5170
sample_31.png 13998 5690 5690 5404 5479 5690 5328
sample_32.png 13449 5673 5666 5454 5415 5672 5156
sample_33.png 13087 5130 5123 4979 4814 5130 4896
sample_34.png 14636 5595 5595 5268 5036 5594 5194
sample_35.png 14629 6608 6608 6370 6075 6607 5794
sample_36.png 15584 6579 6579 6082 5420 6579 5916
sample_37.png 15880 6523 6523 6151 5956 6522 5912
sample_38.png 16038 6187 6185 5926 5317 6187 5488
sample_39.png 13931 5385 5385 5058 5077 5385 4896
sample_40.png 15448 5119 5113 4812 4849 5115 4356
sample_41.png 16177 5785 5783 5531 5160 5785 5022
sample_42.png 14756 5428 5428 5277 5116 5426 4926
sample_43.png 14259 6344 6344 6033 5621 6342 5676
sample_44.png 16852 6644 6644 6344 6043 6643 6058
sample_45.png 18262 7254 7249 6986 6854 6974 6430
sample_46.png 14995 6549 6549 6314 5747 6548 6120
sample_47.png 16423 6124 6124 5791 5809 6124 5542
sample_48.png 17793 7509 7505 7251 6162 7507 6664
sample_49.png 19793 6381 6379 5961 6086 6380 5946
sample_50.png 19086 7479 7474 7171 6127 7479 6672
sample_51.png 19268 6417 6417 6127 5921 6417 5926
sample_52.png 20458 10204 10192 9771 8747 10192 9374
sample_53.png 20422 5721 5713 5493 5476 5719 5056
sample_54.png 22972 9693 9679 9100 8488 9680 8800
sample_55.png 23591 7416 7411 7083 6970 7416 6772
sample_56.png 14151 9096 9097 8373 8292 9095 7672
sample_57.png 22519 7789 7789 7369 7108 7788 7014
sample_58.png 23326 7361 7361 6968 6587 7361 6624
sample_59.png 25264 9334 9280 8764 7603 9322 8372
sample_60.png 29467 9669 9657 8891 7917 9657 8756
sample_61.png 30386 9806 9794 9129 8641 9793 8626
sample_62.png 31314 8481 8475 7986 7903 8481 7714
sample_63.png 31425 10301 10289 9482 9070 10289 9280
sample_64.png 28015 10969 10957 10366 10336 10955 9750
sample_65.png 36994 9182 9167 8387 8423 9170 7732
sample_66.png 29938 10121 10109 9443 9087 10108 8852
sample_67.png 41214 13063 13051 12427 12016 13051 12034
sample_68.png 32913 10131 10119 9636 9099 10119 9324
sample_69.png 44455 16446 16434 15927 15098 16433 15580
sample_70.png 46482 16351 16339 14801 13405 16339 14248
sample_71.png 26815 11771 11757 10483 10591 11758 10678
Total 1129845 454612 454286 429755 409087 453558 407892
Percent 100.00 40.23 40.21 38.04 36.21 40.14 36.11
Time (seconds) tbd tbd tbd tbd tbd tbd tbd

The PNG-32 files have been turned into PNG-8 (paletted) files using pngquant 2.1.0 the resulting PNG-8 files then went through the previous optimization tools.
Here again ScriptPNG does a pretty good job, it tries different palette permutations with TruePNG and this apparently helps it to get the krown of PNG compressors. Overall WebP lossless performs even a little bit better but on some files a PNG-8 is still smaller.
[todo: try to compress the output of ScriptPNG using WebP, add lossy -VP8- WebP results]