Nix stuff
I am attempting inclusion in both FreeBSD and Debian. The first seems to have been successful, the second is in progress
Support for BIG ENDIAN CPUs with -DBIG (at compile time)
CPUs are sometimes "weird", now zpaqfranz compatibility is extended (example: PowerPC Mac)
New -DANCIENT switch for running on "old" systems
Compile on gcc 4.0, working for 3.x. Reduce used memory too
xxhash changed. Needing BIG ENDIAN support the used source is different
The implementation I used was very elegant, but I had to take it off
a small... man
Yes, a man zpaqfranz that ... say... "Luke, use the embedded help!". Debian loves man
Windows stuff
The sfx modules are now compressed with... zpaqfranz (!)
I used to compress them with lz4, which is faster, but less efficient. It is now slower to write sfx modules, but both the executable and the source are smaller
findzpaq switch
Look for an existing .zpaq file on all letters. If it finds it, update that
zpaqfranz a z:\prova\ci.zpaq c:\nz\* -findzpaq
WHAAATT?
Suppose you have a USB device used for backups by writing the "ci.zpaq" file in the "prova" folder.
When you put inside USB port, the Windows letter associated with the USB drive may change.
Maybe today the USB stick is called D:\ and tomorrow E:\ (for a thousand reasons).
Using the -findzpaq switch, zpaqfranz will search within the units the .zpaq file. If found, write on this one.
Therefore a batch file (written on the desktop PC) will be able to update backups on USB devices, even if they change letter.
Why do not run from USB device? Because of -key, the password.There is no point in scripting the password in clear text and saving it on the same device
All
Reworked help
Various minor fix (less compiler warnings)
No "init stages" in benchmark
The embedded test zpaq-mime64 encoded is splitted in 50K chunks (ancient compiler)
New command pause, to pause script. For a key, or X seconds.
In *nix there is not something like Windows's pause-batch command
Wait for a key zpaqfranz pause
Wait for 5 seconds (or key) zpaqfranz pause -n 5
Wait 5 seconds (or key) small output zpaqfranz pause -n 5 -pakka
Wait 5 seconds (or key) NO output zpaqfranz pause -n 5 -pakka -silent
New -rename in summa()
Rename files by hashes, with -force to... enforce
Rename all files in z: with -xxh3 hash (do not ask -force)
First run
zpaqfranz sum z:\* -rename -xxh3 -force
Now check if everything match (second run)
zpaqfranz sum z:\* -rename -xxh3 -force
(...)
Creating 32 hashing thread(s) with XXH3
Renamed files 0
NOT renamed files 40.491
The main thing: autotest
There are now three levels of verification to verify operation on different systems
First: hasher quick check (the output must be this one, you can use -n X to extend)
C:\zpaqfranz\release\55_14>zpaqfranz autotest
zpaqfranz v55.14b-experimental-JIT-L (HW BLAKE3), SFX64 v55.1, (Sep 5 2022)
Self-test for correct internal functioning
Quick check chunsize 5: ABCDE
This seems a LITTLE ENDIAN CPU (aka:'normal')
-----------------------------------------------------------------------------------------------------
CALC BLAKE3: 61274278289E9F6233DF34ABB392AAFE03EE7CE9D77167F3A8D9CDE1AD9861C0
CALC CRC-32: 72D31AD5
CALC CRC-32C: B6AFE183
CALC MD5: 2ECDDE3959051D913F61B14579EA136D
CALC SHA-256: F0393FEBE8BAAA55E32F7BE2A7CC180BF34E52137D99E056C817A9C07B8F239A
CALC SHA-3: 034AF02F68F8874B6668CCBEE49143A64BE435610E1282D93BF35FD80ACCE1FB
CALC SHA1-PUT: 7BE07AAF460D593A323D0DB33DA05B64BFDCB3A5
CALC SHA1-WRITE: 7BE07AAF460D593A323D0DB33DA05B64BFDCB3A5
CALC WHIRLPOOL: C73D8F890F181CE6EB9FF431E08828D6BADA50AC2427546BA10A8F8226F527850FB61E638F798CE86028248262DF17D77D9D00FA5FE5E6CE6C94267E1DC2E99C
CALC XXH3: 1C8288B6013152D97B4A5D7E6C7893D4
CALC XXHASH64: E47599E7C7CEF609
CALC XXHASH64Y: E47599E7C7CEF609
-----------------------------------------------------------------------------------------------------
Time 0.00 seconds for bytes 0
Second: more thorough verification, -all
C:\zpaqfranz\release\55_14>zpaqfranz autotest -all
zpaqfranz v55.14b-experimental-JIT-L (HW BLAKE3), SFX64 v55.1, (Sep 5 2022)
Self-test for correct internal functioning
Iteration 0/9 chunksize 1.000.000
Iteration 1/9 chunksize 333.333
Iteration 2/9 chunksize 111.111
Iteration 3/9 chunksize 37.037
Iteration 4/9 chunksize 12.345
Iteration 5/9 chunksize 4.115
Iteration 6/9 chunksize 1.371
Iteration 7/9 chunksize 457
Iteration 8/9 chunksize 152
Iteration 9/9 chunksize 50
This seems a LITTLE ENDIAN CPU (aka:'normal')
BLAKE3 : OK
CRC-32 : OK
CRC-32C : OK
MD5 : OK
SHA-256 : OK
SHA-3 : OK
SHA1-PUT : OK
SHA1-WRITE : OK
WHIRLPOOL : OK
XXH3 : OK
XXHASH64 : OK
XXHASH64Y : OK
Time 25.58 seconds for bytes 1.748.003.691
Third: the paranoid, -all -to
This will write, into -to, a special folder, with a .bat/.sh file
C:\zpaqfranz\release\55_14>zpaqfranz autotest -all -to z:\checkme
zpaqfranz v55.14b-experimental-JIT-L (HW BLAKE3), SFX64 v55.1, (Sep 5 2022)
Self-test for correct internal functioning
Creating autotest folder in <<z:/checkme/>>
Iteration 0/9 chunksize 1.000.000
(...)
Time 31.33 seconds for bytes 1.748.003.691
The test batchfile is: z:\checkme\dotest.bat
31.360 seconds (000:00:31) (all OK)
Getting something like that
Z:\checkme>dir
Il volume nell'unità Z è RamDisk
Numero di serie del volume: 843A-4C36
Directory di Z:\checkme
05/09/2022 11:14 <DIR> .
05/09/2022 11:14 <DIR> ..
05/09/2022 11:13 <DIR> 00
05/09/2022 11:13 <DIR> 01
05/09/2022 11:14 <DIR> 02
05/09/2022 11:14 <DIR> 03
05/09/2022 11:14 <DIR> 04
05/09/2022 11:14 <DIR> 05
05/09/2022 11:14 <DIR> 06
05/09/2022 11:14 <DIR> 07
05/09/2022 11:14 <DIR> 08
05/09/2022 11:14 <DIR> 09
05/09/2022 11:14 2.961 dotest.bat
05/09/2022 11:13 149.066 sha256.zpaq
Now, run the batchfile inside the testfolder (needs ~4GB free space)
You get a lot of things
First test
(...)
Found (9.04 MB) => 9.481.472 bytes (9.04 MB) / 256 files in 0.000000
Creating 1 hashing thread(s) with SHA-256
Renamed files 0
NOT renamed files 256
0.063 seconds (00:00:00) (all OK)
(1) You should read NOT renamed files 256
**** Hit a key to continue ****
NOT renamed files 256 means: yes, I can extract an "alien" .zpaq in this system
After pressing a key you should get here
zpaqfranz v55.14b-experimental-JIT-L (HW BLAKE3), SFX64 v55.1, (Sep 5 2022)
franz:-sha256
Getting SHA-256 ignoring .zfs and :$DATA
Found (976.56 KB) => 1.000.000 bytes (976.56 KB) / 1 files in 0.015000
Creating 1 hashing thread(s) with SHA-256
SHA-256: 67708591460BCE3BC45AE086A342F9F390AD2913A22639EC7AF3646B7D2AEA78 [ 1.000.000] z:/checkme/verifica
0.031 seconds (00:00:00) (all OK)
(2) You should read SHA-256: 67708591460BCE3BC45AE086A342F9F390AD2913A22639EC7AF3646B7D2AEA78
**** Hit a key to continue ****
The situation is: yes, I can make an archive, then extract a single file, and this match
Now a lot of other things and you came here
SUMMARY : 15.58 (total time)
Virtual : 10 (skipped, does not exists in 7.15)
Total : 42.727
GOOD : 42.727 of 42.727 (stored=decompressed)
SURE : 42.727 of 42.727 (stored=decompressed=file on disk)
All OK (paranoid test with check against filesystem)
15.625 seconds (000:00:15) (all OK)
(3) You should read SURE : 42.727 of 42.727 (stored=decompressed=file on disk)
**** Hit a key to continue ****
This means: I made an archive, and this is good
Now the last: can zpaqfranz extract files (on the filesystem)?
Worked on 971.003.691 bytes avg speed (hashtime) 862.347.860 B/s
GLOBAL SHA256: 7B32DB3F6F0180062773C5A046A2C99D17BE7668202379BA136E7A4D134FFC26
2.313 seconds (000:00:02) (all OK)
(4) You should read GLOBAL SHA256: 7B32DB3F6F0180062773C5A046A2C99D17BE7668202379BA136E7A4D134FFC26
**** Hit a key to continue ****