Now Playing: Mission Control @SomaFM
Topic: CD images
ECMa130 (ECM alternative)
2 years old entry... Adding some new stuff here soon, so, just moving it here from there, to have everything at the same place. ECMa130 can be downloaded here
Initially made as a proof of concept, while discussing possible raw (a.k.a. GDI; 2352 bytes per sector) Dreamcast GD-ROM image (redump.org, Dumpcast) compression, later shaped to be quite capable utility, that I would find myself using daily, because of it's performance advantage over counterpart. Idea is taken from Neill Corlett's ECM and developed further, implementing system specific optimizations, improving container format and more (e.g. latest versions makes use of multithreading for slight performance boost on multi-core machines).
In a few words: this program preprocess raw CD images, commonly produced by CD backup applications (CDRWIN (.bin), CloneCD (.img), ImgBurn, Alcohol 120%, etc.), removing reproducible data (such as ECC), reducing stress on later, usually more complex, processing stages (e.g. delta or compression). This operation is completely lossless and is reverted when decoding, restoring images to their original state.
Dreamcast encoding (data from 15 raw GD-ROM images):
Total size | % from Input | Time (s) | |
Input | 16814462112 (~15.6 GB) | 100 % | 0 |
ECM | 14662611594 (~13.6 GB) | 87.2 % | 685 |
ECMa130 | 9708644352 (~9.0 GB) | 57.7 % | 450 |
Dreamcast compression:
Total size | % from Input | Time (s) | |
Input -> 7-Zip 4.65 | 8156554700 (~7.5 GB) | 48.5 % | 4167 |
ECM -> 7-Zip 4.65 | 6717473687 (~6.2 GB) | 39.9 % | 3521 |
ECMa130 -> 7-Zip 4.65 | 6674638910 (~6.2 GB) | 39.6 % | 3173 |
In illustrated case ECMa130 executes notably faster than ECM (4 min.) and removes a lot more data (30%). This further affects next processing step: 7-Zip compression, since it has far less data to analyze and, while size wise LZMA algorithm ran on ECM output catches up with only about 50 MB difference (it would not always be the case with other algorithms), resulting time saved with ECMa on those 15 images is almost 10 minutes. 9 minutes faster than LZMA over unprocessed data and ~1.4 GB saved.
PlayStation encoding (data from 15 raw CD images):
Total size | % from Input | Time (s) | |
Input | 3878325696 (~3.6 GB) | 100 % | 0 |
ECM | 3498975325 (~3.2 GB) | 90.2 % | 478 |
ECMa130 | 2996472772 (~2.7 GB) | 77.2 % | 116 |
PlayStation compression:
Total size | % from Input | Time (s) | |
Input -> 7-Zip 4.65 | 1638481212 (~1.5 GB) | 42.2 % | 902 |
ECM -> 7-Zip 4.65 | 1417162337 (~1.3 GB) | 36.5 % | 811 |
ECMa130 -> 7-Zip 4.65 | 1406660446 (~1.3 GB) | 36.2 % | 799 |
PlayStation ultra (-mx9) compression:
Total size | % from Input | Time (s) | |
Input -> 7-Zip 4.65 | 1598770703 (~1.4 GB) | 41.2 % | 1422 |
ECM -> 7-Zip 4.65 | 1386606174 (~1.2 GB) | 35.7 % | 1282 |
ECMa130 -> 7-Zip 4.65 | 1372467680 (~1.2 GB) | 35.3 % | 1269 |
In this case 7-Zip's compression with default parameters executes slightly faster on unprocessed data than ECMa + 7-Zip, but with ECMa output takes up about 6 % less space. On ultra compression ECMa + 7-Zip combo produces best overall results, beating 7z by 37 seconds and saving 6 % of data. ECM is by far the slowest, losing more than 6 minutes to ECMa130 and being outperformed even by LZMA algorithm suggests that there likely are certain issues with ECM's CD-ROM XA sector processing implementation.
All code is original, written in pascal with Cyclic Redundancy Check & Reed–Solomon algorithms in assembler. Compiled with Free Pascal. Through last year tested with over 500 images for various systems (DC, PSX, SS, PCE, PC, etc.).
Posted by themabus
at 15:12 EET
Updated: 12/03/2012 00:00 EEST
Post Comment | Permalink | Share This Post
Updated: 12/03/2012 00:00 EEST
Post Comment | Permalink | Share This Post