zpaqfranz 65.9

DO NOT USE IN PRODUCTION: IT IS STILL TO BE STABILIZED!

This build (65.9z) is released on purpose as not for production. What
is new about the disk of Windows (ac, xc, their screens, the WinPE) is
young, and it is put out now to get feedback and to debug it along the way
with who uses it, instead of doing it alone. Try it on machines and disks
that can be lost, and tell what happens. Do not make it your only backup!

With this release, I think I've implemented everything I wanted to do; now it's time to focus on stabilization and bug fixes. Please open issues on GitHub so I can fix them and improve the program. The game-changer was finally setting up a cluster of VMs to automate testing, but they require a lot of RAM and disk space, so the more help I get, the better the program will evolve.

This is the release of the PC that comes back. One command saves the
disk of Windows so that it can be put back and boots, one command puts it
back, and there is a first WinPE to do it from:

• ac archive: the disk of Windows, for a restore that boots. C: by
  its used clusters, everything else that is needed to start (the hidden
  partitions, the partition table, the space between the partitions) byte
  by byte, and a map of the disk with the hashes of every piece. MBR and
  GPT. -all: the data partitions too.
• xc archive: the restore. Alone it is the plan, nothing is written;
  -to 2: -force writes a physical disk, -to x.vmdk or x.vhdx a
  virtual one. Every piece is checked before and while it is written. A
  restored Windows 10 was started: it boots.
• A WinPE to restore from (first version): an ISO with zpaqfranz in
  it, to start a PC that has no Windows any more.
• -gui: ac and xc on a screen. Which partitions, which disk,
  with the arrows and the space bar.
• A command without its archive opens a chooser: zpaqfranz t and
  ENTER. Pick the archive on a screen, then the switches, then it runs.
• -ntfs: the list of the files read from the $MFT, not folder by
  folder. The same list, 7 times faster on a C:, 12 times with a
  shadow copy. By itself when the sources are whole NTFS drives.
• Images over 2 TB: they restore to a .vhdx (new), to a .vmdk in
  more files as VMware makes it, with a GPT inside.
• d: what to look for (d u:\*.mp4), filters, a new output in
  groups, hard links told apart, xxh3 for real.
• The shadow copy of an image: by itself for C: only. And an image
  that asked for it and could not have it is an error, not a raw image.
• Fixes that matter: an image on a disk that fills up said "all OK"; an
  elevated run lost the "" and the paths with a space; the old -ntfs
  lost hard links and folders.

This document has three parts:

• Part one: what is new, in short, for everybody
• Part two: the same things in more detail, for power users
• Part three: "lo spiegone", how it works inside, and why, for developers

Part one, what is new, for the user

1. ac: the disk of Windows
2. xc: the restore
3. A WinPE to restore from
4. -gui: choose on a screen
5. A command without its archive
6. -ntfs: the list from the $MFT
7. Images over 2 TB
8. d: what to look for, and a new output
9. The shadow copy of an image
10. Smaller things

1. ac: the disk of Windows

zpaqfranz ac z:\pc.zpaq                   (the disk of Windows)
zpaqfranz ac ""                           (dry run: what would be taken)
zpaqfranz ac z:\pc.zpaq -all              (every partition of that disk)
zpaqfranz ac z:\pc.zpaq -all -not m:      (every partition but M:)
zpaqfranz ac z:\pc.zpaq -gui              (choose on a screen)

An image of C: does not boot: Windows starts from partitions that have
no letter, and from a partition table. ac takes the disk C: is on:

• C: as a archive c: -image does: the used clusters, with a shadow
  copy;
• every partition without a letter (boot, EFI, recovery...) byte by byte;
• the start of the disk with its partition table, the space between the
  partitions, the end of the disk;
• disk_C.txt, a text file: the map of the disk, how each piece was read,
  its size and its hashes.

All in one version of the archive, one for each run. Before anything
is read a table says what will be taken, and what will not:

#            from       size       read  saved                               what
00        0.00  B    1.00 MB    1.00 MB  raw (disk_C_00.raw)                 the start of the disk
01        1.00 MB   50.00 MB   50.00 MB  raw (disk_C_01.raw)                 partition 1, BOOT, NTFS "Riservato per il sistema", active
02       51.00 MB   43.43 GB   11.08 GB  used clusters (image_C.fhd)         partition 2, WINDOWS, C:, NTFS
03       43.48 GB    1.30 MB    1.30 MB  raw (disk_C_03.raw)                 between the partitions
04       43.48 GB   10.00 GB          -  NOT SAVED: data (mounted)           partition 4, D:, NTFS "DATI"
05       53.48 GB    1.00 MB    1.00 MB  raw (disk_C_05.raw)                 between the partitions
06       53.48 GB    5.00 GB          -  NOT SAVED: data (mounted)           partition 5, E:, NTFS "ALTRI"
...
10       59.48 GB  534.00 MB  534.00 MB  raw (disk_C_10.raw)                 partition 3, recovery, NTFS
11       60.00 GB    2.00 MB    2.00 MB  raw (disk_C_11.raw)                 the end of the disk

• The partitions with a letter (D:, E:...) are not taken: they are
  data, and the table says so. Their place in the partition table stays.
• -all takes them too: NTFS, FAT and exFAT by their used clusters,
  anything else (not formatted, encrypted by something else) whole. Not
  the partition the archive is on. -not X: leaves one out.
• ac "" reads everything and writes nothing: how much it is, how
  long it takes.
• An archive of ac is an image archive like the others: image gives
  C: back alone, as a .vhd or on a partition, as before.
• A BitLocker volume that is unlocked is saved in clear (it is told);
  -key encrypts the archive.
• Refused, for now: boot files on another disk than the one of C:,
  dynamic disks.

On the test machine (Windows 10, a disk of 60 GB, 11 GB to read): 36
seconds, an archive of 5.3 GB; a second run right after adds 11 MB.

2. xc: the restore

zpaqfranz xc z:\pc.zpaq                      (the plan: nothing is written)
zpaqfranz xc z:\pc.zpaq -gui                 (choose on a screen what, and where)
zpaqfranz xc z:\pc.zpaq -to 2: -force        (the physical disk 2: OVERWRITTEN)
zpaqfranz xc z:\pc.zpaq -to 2: -force -verify
zpaqfranz xc z:\pc.zpaq -to d:\pc.vmdk       (a disk for a virtual machine)
zpaqfranz xc z:\pc.zpaq -to d:\pc.vhdx -until 3
zpaqfranz drives                             (the disks of this PC, with their numbers)

xc archive alone is the plan: which backup it is, its pieces, where each
one goes.

xc: the backup of DESKTOP-09SNTPI (Windows 10 Pro), taken 2026-10-06 13:07:42 UTC by zpaqfranz v65.9o: version 1 of 1
xc: disk of 60.00 GB (VMware Virtual SATA Hard Drive, SATA), MBR B28ABB93, sectors of 512 bytes, boot by BIOS
#            from       size   to write  restored from                       what
00        0.00  B    1.00 MB    1.00 MB  raw (disk_C_00.raw)                 the start of the disk
01        1.00 MB   50.00 MB   50.00 MB  the whole partition (disk_C_01.raw) partition 1, BOOT, NTFS "Riservato per il sistema"
02       51.00 MB   43.43 GB   11.08 GB  its used clusters (image_C.fhd)     partition 2, WINDOWS, C:, NTFS
...

• Every piece goes back where it was, with the partition table of the
  backup: the same disk signature (or the same GUIDs). Nothing to repair
  to make it boot, no external program.
• The pieces must be the ones the backup wrote: size, CRC-32 and hash
  are checked in the index of the archive before anything is written, and
  again on the data while it is written.
• -to N: -force writes the physical disk N. -force is the consent,
  then a captcha that names the disk (killdrive2 for disk 2) must be
  typed. The disk must be at least as big as the one of the backup, with
  sectors of the same size. Never the disk of the running Windows, nor the
  one the archive is on.
• On the disk of the backup itself what was not in the backup (the
  data partitions) is not touched. On another disk those partitions
  come back empty, to be formatted.
• -to x.vmdk (VMware, VirtualBox, QEMU/Proxmox), -to x.vhdx
  (Windows mounts it, Hyper-V starts it), -to x.raw: the whole disk
  in a file.
• -verify reads back what was written.
• While it writes, a live map of what goes on the disk; at the end one
  line for each piece: "as the backup wrote".

A disk of Windows 10 (MBR, BIOS) restored this way on an empty disk was
then used to start the machine: up in 50 seconds, chkdsk C: clean, the
recovery environment in its place.

On the test machine a restore of 11.8 GB on a disk takes 33 seconds.

3. A WinPE to restore from

DO NOT USE IN PRODUCTION: IT IS STILL TO BE STABILIZED

A restore on the disk Windows runs from cannot be made from that Windows.
So here is the first version of a WinPE with zpaqfranz inside: a small
Windows that starts from a USB stick or a CD, on a PC with an empty disk,
a broken Windows, or none.

https://sourceforge.net/projects/zpaqfranz/files/winpe/beta65_9z/

WinPE_zpaqfranz_65_9z.iso

• About 430 MB.
• What is in it: zpaqfranz.exe, and a script that sets the English
  keyboard (if asked). Nothing else: all the rest is Microsoft's
  WinPE as it is.
• Start the PC (or the virtual machine) from it, and at its prompt:

zpaqfranz drives                         (which disk is which)
zpaqfranz xc e:\pc.zpaq                  (the plain)
zpaqfranz xc e:\pc.zpaq -gui             (choose on a screen, then F8)
zpaqfranz xc e:\pc.zpaq -to 0: -force    (or straight on disk 0)
zpaqfranz xc \\thenas\theshare\thebackup.zpaq -to 0:

The archive must be somewhere WinPE can read: a USB disk, a second
disk. image is there too, for one partition alone, or on a supported network (DHCP)

It is a beginning, and it will grow with the rest. For now it is there:
who tries it and tells how it went (which PC, BIOS or UEFI, which disk,
what the screen said) is doing the debug that one person alone cannot do in a short time.

4. -gui: choose on a screen

zpaqfranz ac z:\pc.zpaq -gui

>[ ] ac: disk 0, 60.00 GB (VMware Virtual SATA Hard Drive, SATA), boot by BIOS
     #        size       used  type  what
 [+] 02   43.43 GB   22.91 GB  NTFS  WINDOWS (C:, NTFS)
 [ ] 04   10.00 GB   97.76 MB  NTFS  DATA    (D:, NTFS "DATI")
 [ ] 06    5.00 GB   85.20 MB  NTFS  DATA    (E:, NTFS "ALTRI")
           TO READ   23.48 GB  9 pieces
         NOT SAVED   15.00 GB  not in the backup: D: E:
 ESC/q=quit SPACE=select a=all v=system ?=help F8/g=START

[+] is what is always taken (Windows, and what it boots with), [X] and
[ ] every other partition: arrows and space. The first line is the whole
disk. The line in cyan is how much will be read. F8 (or g) starts,
ESC leaves and nothing was read.

zpaqfranz xc z:\pc.zpaq -gui

 xc: the backup of DESKTOP-09SNTPI (Windows 10 Pro): 60.00 GB, MBR, BIOS
     taken 2026-10-07 08:41:18 UTC by zpaqfranz v65.9v, version 1 of 1
     #        size   to write  type  what
 [+] 02   43.43 GB   19.84 GB  NTFS  WINDOWS (C:, NTFS)
>[X] 04   10.00 GB  100.00 MB  NTFS  DATA    (D:, NTFS "DATI")
          TO WRITE   20.51 GB  10 pieces
        NOT WRITTEN    5.00 GB  not restored: E:
 ######################################################################
 ######################################################################
 [-] 00 SATA, Fixed VMware Virtual SATA Hard D   60.00 GB  C:, D: "DATI", E: "A
 [-] 01 SATA, Fixed VMware Virtual SATA Hard D   50.00 GB  Z: "ARCHIVI"
 [ ] 02 SATA, Fixed VMware Virtual SATA Hard D   70.00 GB  BACKUP DISK: 5 parti

(As it is on a console of 80 columns: the long lines are cut.)

Above, the partitions of the backup; below, the disks of this PC: one
only. Green: a disk without partitions. Yellow: it has partitions, they
will be lost. Red, [-]: it cannot be (too small, the disk of Windows,
the one of the archive), and why. F8 asks to type the name of the disk
(killdrive2), then it writes. i asks for a file instead (.vhdx,
.vmdk, .raw, .img).

Only with -gui: ac and xc without it are what they were, for scripts
and scheduled tasks.

5. A command without its archive

zpaqfranz t
zpaqfranz x
zpaqfranz i

A command that wants an archive, typed without one on a console, used to
print the help. Now it opens a chooser on the file system:

FILE 00000014|C:/zt/sc/
zpaqfranz t: ENTER on an archive (in cyan) takes it; SPACE marks more than one
>[ ] 00000001| 2026-10-07 12:37:12                 <DIR> dati/
 [ ] 00000006| 2026-10-07 12:37:12                 6.499 bk_00000001.zpaq
 [ ] 00000009| 2026-10-07 12:37:12                 7.799 due.zpaq
 [ ] 00000011| 2026-10-07 12:37:12                 7.804 multi_0001.zpaq
ESC/q F1-F4=Sort all not inv search filter SPACE :line path ?help

The archives are in cyan, the folders in green. The keys are the ones of
the tui command. ENTER on an archive, and the switches of that command
come up:

zpaqfranz t <archive> -checksum
 RUN   -verify   -ssd   -paranoid   -quick  [-checksum]  -collision   -crc32
 -all   other: (none)
arrows=move SPACE/ENTER=on/off (on: reversed) ENTER on RUN=start ESC/q=back

ENTER on RUN, and the command runs: the line is shown as it would have
been typed, to be copied in a script next time.

• What a command cannot do without is asked: x asks where to extract
  (nothing: here).
• A piece of a multipart archive is taken as the whole archive
  (multi_????.zpaq); a file of the backup command too.
• More archives marked with SPACE: one run each (t and x), as
  t "name*" does.
• The password, when there is one, is asked by zpaqfranz as always.
• For: x xx e t l i v p pp w zip image crop trim password ls tui gui xc.

Not on a console (a script, a redirected output): the help, as before.

6. -ntfs: the list from the $MFT

zpaqfranz a z:\1.zpaq c:\ e:\              (by itself: whole NTFS drives, an administrator)
zpaqfranz a z:\1.zpaq c:\data -ntfs        (asked for, on a folder)
zpaqfranz a z:\1.zpaq c:\ -nontfs          (never: folder by folder, as always)

Before a backup, the list of the files. Windows gives it folder by
folder, and it costs for each folder. -ntfs reads the $MFT of the
volume, where NTFS keeps every name, in one go.

• The same list: the same names, the same sizes, dates and
  attributes (but for a file that is open and growing: see part two).
  That was the point, more than the speed.
• By itself when every source is a whole NTFS drive and an
  administrator runs. A folder, a FAT drive, a share among the sources, no
  administrator: the normal scan, and nothing is said. -ntfs asks for it
  on a folder too; -nontfs never.
• The switch was already there, with other code: that one lost the hard
  links and the folders, and without an administrator it listed nothing.
  It is gone.
• Not for an image (-image does not list files).

7. Images over 2 TB

zpaqfranz image z:\big.zpaq g: -to d:\g.vhd      (over 2040 GB: d:\g.vhdx by itself)
zpaqfranz image z:\c.zpaq c: -to d:\c.vhdx       (a .vhdx, of any size)
zpaqfranz image z:\big.zpaq g: -to d:\g.vmdk     (over 2032 GB: g.vmdk, g-s001.vmdk, g-s002.vmdk...)

The backup of a volume over 2 TB was already good. What was missing was
the way out: a .vhd ends at 2040 GB, one sparse .vmdk a little before.

• .vhdx, new: of any size, mounted by Windows the same way as a
  .vhd. Over 2040 GB a .vhd becomes a .vhdx by itself (with x
  too).
• A sparse .vmdk over 2032 GB is written in more files, as VMware
  Workstation makes it.
• A partition over 2 TB does not fit in an MBR: the virtual disk gets
  a GPT.
• The free clusters as zeros in the image (the thin image of 65.8), and
  f X: -test -force -zero -ntfs: over 2 TB too.

Tried on a volume of 4,000 GB with 2,300 GB used: the image in 1 hour and
34 minutes (417 MB/s), then -to x.vhd gave a .vhdx that Windows
mounted, chkdsk clean, the files the same.

8. d: what to look for, and a new output

zpaqfranz d u:\*.mp4                            (only the .mp4, and below)
zpaqfranz d u:\ *.mp4 -minsize 100MB -zero      (as find; the big ones; not the empty files)
zpaqfranz d c:\d0\ -verbose                     (the groups)
zpaqfranz d c:\d0\ -force                       (wet run: the duplicates are DELETED)

• What to look for: d folder\*.mp4, or d folder\ *.mp4 as find
  does. -find, -minsize, -maxsize, -only, -not, -datefrom,
  -dateto. -norecursion: that folder only.
• -zero leaves the empty files alone (by default they are all
  duplicates of each other).
• The output: with -verbose the files in groups, the one that stays
  first (with its date), the ones that go marked ====, then what the
  group gives back and its hash. The smallest saving first, the biggest
  last, where the eye is when it ends. The same total with and without
  -verbose.
• Hard links (more names of the same file) are in yellow and are not
  counted: deleting one of them frees nothing.
• xxh3 is the default, as the help always said. It was SHA-1.
• Two files that could not be read, of the same size, were duplicates of
  each other. Not any more.

It is a dry run without -force.

9. The shadow copy of an image

zpaqfranz a z:\c.zpaq c: -image            (C: a shadow copy, by itself)
zpaqfranz a z:\d.zpaq d: -image            (D: the volume is locked)
zpaqfranz a z:\d.zpaq d: -image -vss       (D: with a shadow copy)

• By itself for C: only. Any other drive is locked while it is read;
  a shadow copy there is asked for, with -vss. The line of the image
  says which one it is: "with VSS", or "without VSS (by itself for C:
  only; -vss to have it)".
• A drive that cannot be locked (something keeps a file open) is read as
  it is, with the warning of always and exit code 1.
• -image -vss when the shadow copy cannot be made (no room for it on
  the volume, a USB disk) is an error now: nothing written, exit code 2.
  It became a raw image of the whole partition, without a shadow copy,
  with exit code 0, after a line that said "with VSS".

10. Smaller things

• An image on a destination that fills up (Windows): the whole source
  was read, and it ended with "(all OK)" and exit code 0, the archive cut
  short. Now it is an error, exit code 2, as it was for files.
• The elevated window lost its arguments. From a normal prompt
  ac "", and any path with a space in it, got to the elevated run
  without the "" and without the quotes: ac "" ended at once with
  "(all OK)". The command line now goes through as it was typed. g ends
  with the exit code of its elevated run (it was always 0).
• written in the progress lines of a is what really went out: on
  a destination that is full it stops, and that is the truth. With ""
  as the archive it shows what would have been written.
• i -verbose shows the backups of ac that are in the archive: the
  last one with its table, with -all every one. On every platform.
• tui: F1 to F4 sort by name, size, date and extension, the same
  key again reverses. (F4 to F6 were the reverse sorts.)
• OpenBSD 7.9: the linker does not complain about rand() any more.
• It builds everywhere it did: Windows (64 and 32 bit), Linux,
  FreeBSD, OpenBSD, macOS (Intel and Apple Silicon), Solaris, the ESXi
  build (gcc 3.4.6), the eleven NAS binaries. Some -Wall warnings of the
  last releases are gone.

Part two, the details, for power users

1. ac: what is taken, and how
2. xc: the rules
3. The WinPE, and what to tell
4. The two screens, key by key
5. The chooser: keys, and the table of the commands
6. -ntfs: when, and where it differs
7. Virtual disks: every combination, again
8. What can change for a script

1. ac: what is taken, and how

The disk is not seen as "EFI, MSR, recovery": it is partitions, and
everything that is outside them. The same rules for MBR and GPT.

• The raw pieces are disk_C_00.raw, disk_C_01.raw...: the number is
  the one of the zone in the table, in the order of the disk.
• With -all every image of a partition with a letter can be taken back
  alone: image z:\pc.zpaq d: -to x.vhd. (a c: -image -all is another
  thing, and it is still there: the whole disk, raw, in one file.)
• The shadow copies: C: always (-novss: none). With -all the
  other partitions are locked, one after the other; one that cannot be
  locked is read as it is, with a warning and exit code 1. -vss: a
  shadow copy for every NTFS. Each one is consistent in itself, not with
  the others.
• The partition Windows boots from is mounted, with files open in it: it
  is read as it is, and the map says so.
• BitLocker: its presence is written in the map. An unlocked volume
  is read through Windows, so in clear; a locked one, with -all, raw as
  it is on the disk.
• -not X: with -all leaves that partition out. -not c: is refused.
  (-not with files, inside an image, still means what it meant.)
• The last line of ac is the name, the size, the CRC-32 and the hash of
  disk_C.txt: the same that l -checksum shows. It ends up in the
  report by e-mail by itself.
• x z:\pc.zpaq disk_C.txt -to z:\out\ to read the map: it is a text.

Refused, with the reason: the boot files on another disk; the boot files
on a data partition with a letter over 1 GB; a dynamic disk; a table of a
logical drive (MBR) that would fall outside what is taken.

2. xc: the rules

Which backup. The last one. -until N for the one of version N; a
drive after the archive (xc arc c:) when the archive has the disks of
more machines.

The check before. Each piece named in the map must be in the index of
the archive, in that version, with that size, CRC-32, hash, and the same
list of fragments. An archive where an a c: -image was added after the
ac has an image_C.fhd that is not the one of the map: xc says which
pieces are not the ones, and which -until to use. The pieces and the
map of two different days are never mixed.

The check while. Size, CRC-32 and hash of every piece as it comes out
of the archive. The small raw pieces (the partition table is always one
of them) are kept in memory and written only when they are right. The
serial number of the volume inside an image must be the one of the map.

The disk of the backup, or another one.

• A bigger disk is fine (on GPT Windows moves the second table to the
  end by itself). A smaller one is refused.
• Refused as a destination: the disk of the running Windows, the one it
  boots from, the one the archive is on, the one of zpaqfranz itself, a
  read-only disk.
• Inside an image, on a physical disk: what the file system counts as
  used but was not saved (pagefile, hiberfil, swapfile) is written as
  zeros. An old hibernation must not be found there.
• At the end, when another disk on line in this PC has the same
  signature (the original next to its copy), the restored disk is kept
  off line, and it is told (66194): put on line there it would get a
  new signature from Windows, and would not boot. Take one of the two
  away.
• -verify: a physical disk and a raw file are read back, piece by
  piece.
• -nocaptcha skips the captcha, for scripts. -force is also the
  consent to write over a destination file that is there.
• An error while writing: 66180, then "RESTORE NOT DONE" (66182), exit
  code 2.

The live map shows what is written, in the order of the disk: green
when written, with -verify blue then green when read back. Redirected,
or with -nodashboard or -noeta: the lines of before.

3. The WinPE, and what to tell

• The ISO is a WinPE as Microsoft makes it: write it on a USB stick or
  attach it to a virtual machine, and boot from it.
• Inside WinPE there is no Windows running from the disk, so the disk
  the PC boots from can be the destination: that is what it is for.
• The numbers of the disks there may not be the ones Windows gave:
  always zpaqfranz drives first.
• The console of WinPE is 80 columns by 25 lines: the screens of -gui
  were made to fit 80 columns, and a list that is longer scrolls.
• The keyboard is the one WinPE starts with (Italian, yep I am from Italy); the script in the ISO sets
  the English one, if asked.

What helps, when something goes wrong: the model of the PC, BIOS or UEFI,
MBR or GPT, the kind of disk (SATA, NVMe, a RAID controller that WinPE
does not see), the output of drives and of xc archive (the plan), and
the last lines on the screen.

I'll make a new github repository with the "source code" of the WinPE (in fact, a single .cmd)
Please be patient, I have to ... work too 😄

4. The two screens, key by key

In the column type: the file system; RAW for a partition that can be
chosen but is read whole; --- for a system partition, always taken.

• [-] in ac: a partition that cannot be taken (the one the archive is
  on), with the reason.
• A disk in xc: green, no partitions; yellow, partitions that will be
  lost; magenta "BACKUP DISK", the disk of the backup itself (or a copy
  of it): its data partitions stay; red [-], it cannot be, and why.
• From the screen no -force is needed: the consent is the name of the
  disk typed by hand.
• xc arc -to 2: -gui: disk 2 already chosen. A partition saved by
  ac -all and left out here is as one that was not in the backup.
• i: a name without what comes before the extension is refused; a file
  that is there: "overwrite it? (y/n)"; a .vmdk is the sparse one.
• Not a console: 66300, exit code 2.

5. The chooser: keys, and the table of the commands

Only .zpaq and .franzen (and the files that are pieces of them) can
be taken. A command for one archive refuses the marks.

• A switch already on the command line (zpaqfranz x -to z:\out -force)
  is on in the panel, and what it wants is not asked again.
• The last item of the panel, "other", is a line typed by hand: its words
  before the first switch go right after the archive (the files to
  extract, the drive of an image), the rest at the end.
• The time on the last line of the run counts from after the choice.
• The chooser starts only when the command is one of the table, no
  archive follows it, and both the input and the output are a console.

6. -ntfs: when, and where it differs

When it started by itself and cannot go on (the volume cannot be opened),
the normal scan, told only with -verbose. One line says what was done:
how many names, in how many seconds, out of how many records.

Where it pays. The normal scan costs for each folder (30 to 60
microseconds when in the cache of Windows, about 280 when not), almost
nothing for each file. -ntfs costs the read of the $MFT (about half a
second for each million records, on that disk) and little more. So: a
whole volume, many folders, and above all a volume that is not in the
cache. A shadow copy is always a new volume for the cache: with -vss
the normal scan is always the slow one. One million files in 10,000
folders, already in the cache: even.

Where it differs, and it is not an error:

• A file that a program keeps open and makes grow is 0 bytes for the
  normal scan until it is closed, and has its true size in the $MFT.
  On a live C:: some tens of names out of 180,000 (logs, traces). On a
  shadow copy: none.
• What only Windows can tell is left to Windows: a folder that cannot be
  listed, paths of 240 characters and more, names that are not Win32,
  folders of a cloud (OneDrive, Dropbox). That subtree goes through the
  normal scan.

7. Virtual disks: every combination, again

• A .vhdx, as a .vhd, keeps whole blocks of 2 MB: on a volume with
  its free space in small pieces it is almost as big as the disk (8,086
  MB for 3,056 MB used on the tortured test volume). A sparse .vmdk
  (grains of 64 KB) stays near the used data.
• Sectors of 512 bytes only.
• xc writes the same kinds of virtual disks, of a whole disk: .vmdk
  sparse or flat, .vhdx, raw.

The volume of 4,000 GB with 2,300 GB used, on the test machine:

Mounted by Windows (GPT, NTFS of 4,000 GB) and by OSFMount, chkdsk
clean, the list of the files the same as on the volume.

8. What can change for a script

Part three, lo spiegone, for developers

1. The disk as zones, and disk_C.txt
2. More images in one run
3. xc: one pass of the engine for each piece
4. franzscelta, and the map of xc
5. franzscegli: one hook before the parser
6. franzntfs2: the $MFT in memory
7. franzvhdx, the .vmdk in pieces, the GPT
8. Asking for the administrator, again
9. How it was tested
10. Fixes
11. Tried and thrown away
12. Known limits and open points

1. The disk as zones, and disk_C.txt

Windows gives the layout of a disk with one call, the same for MBR and
GPT, logical drives included (IOCTL_DISK_GET_DRIVE_LAYOUT_EX). ac
makes of it a list of zones that covers the disk from the first byte
to the last: a partition, or what is outside. Each zone gets a decision
(ac_decidi): an image, raw, not taken and why.

The partition Windows boots from is read in the registry
(HKLM\SYSTEM\Setup\SystemPartition), not guessed from its type: on MBR
it looks like a data partition. BitLocker is seen in the first sector as
the disk has it (-FVE-FS-); anything else under a volume, when the
first sector of the disk is not the one of the volume. No external
program is run (no bcdedit, no reagentc): what is not needed is not
asked.

Why it boots without repairing anything: on MBR the boot configuration
finds its partition by the signature of the disk (4 bytes of sector 0)
and an offset. The same sector 0, the same offsets: it starts. On GPT,
the same GUIDs.

In the archive C: is exactly what a c: -image writes (image_C.fhd
and its companions), so every command that knows an image knows this one.
What is new has another prefix, disk_C_, because the names that begin
with image_ have a meaning by their length and by one letter.

The map is a text, franzdisk 1 on its first line:

franzdisk 1
# every piece named here is in the SAME version of the archive, with the size and the hashes written here.
# A key not known is to be ignored. Numbers are bytes, from the start of the disk.

[run]
program=zpaqfranz v65.9o (2026-10-04)
command=C:/zt/zf9o.exe  ac Z:/tutto.zpaq -all -noeta
computer=DESKTOP-09SNTPI
windows=Windows 10 Pro
firmware=BIOS
started_utc=2026-10-06 13:07:42
seconds=30.3
archive_version=1
hash=XXHASH64B
...

[disk]
number=0
model=VMware Virtual SATA Hard Drive
bytes=64424509440
sector=512
style=MBR
signature=B28ABB93
zones=12
...

[zone 01]
kind=partition
start=1048576
bytes=52428800
mbr_type=07
active=yes
filesystem=NTFS
label=Riservato per il sistema
volume_serial=D446C64B
boot=yes
encrypted=no
saved=raw
file=disk_C_01.raw
layer=disk
note=its volume was mounted, not locked, while it was read
read_errors=0
...

then [files] (name, size, CRC-32, hash with its kind, the hash of the
list of the fragments) and [archive before] (the versions that were
there). It describes facts, it does not give orders: what to do is
decided by the restore, looking at the destination. As VFILE-info: keys
are only added.

It is written last in its version, as a file in memory, so it holds
hashes that zpaqfranz computed while reading: nothing is read twice. The
hasher of a file is closed when the index is written; ac closes it when
the file ends and keeps the result, and the index gets the same one.

The pieces of ac do not get the compression method of the companions of
an image (the one for the small metadata files): on 590 MB it took 41
seconds and 1.3 GB of RAM instead of 18 seconds and 27 MB.

2. More images in one run

add() was made for one image in a run: one reader, one drive letter,
one set of names, one live map. ac -all wants C:, D:, E: by their
used clusters in the same version.

The engine was not rewritten. The files of every image are queued in
order (an image, its companions, the next image..., the raw pieces, the
map last), and when the file in turn is the image of another partition
ac_cambia() puts a new reader in the place of the old one (the
destructor, then a placement new: the shadow copy of before is released),
sets the letter, the kind (used clusters or raw), the names, and the
bases the progress is counted from. The live map starts again from zero
for each image.

When the used clusters of a partition cannot be read after all, its
companions are skipped and image_X.raw takes the place of
image_X.fhd, with a warning.

Only with ac: the path of one image, the one of a c: -image, does not
go through any of this.

3. xc: one pass of the engine for each piece

xc does not have an engine of its own. The extraction to stdout
(extractstdout) is called once for each piece, in the order the
destination wants: a virtual disk is written from its start to its end
(its writer wants whole blocks of 2 MiB, in order: an assembler makes
them); a physical disk and a raw file get the piece with the partition
table last. The fixed cost is about 0.35 seconds a piece.

franzxc is the piece going through: it counts, computes CRC-32 and
hash, and sends the bytes to franzxcuscita (a physical disk, a sparse
raw file, a virtual disk through franzvdisk).

An image of the used clusters is a list of records (512 bytes and a block
of 2 MB), and a map says which block of the partition each record is. It
goes straight to its place on the disk: no .vhd in between.

On a physical disk:

• a volume is locked and dismounted before it is written; with a file
  kept open by another program the lock fails. That is why, on a disk
  that is not the one of the backup, the partition table is wiped first:
  the volumes are gone, in use or not;
• a disk off line can be written raw;
• after the last piece Windows is told to read the disk again; on a GPT
  disk bigger than the original it moves the second table to the end and
  writes the header again by itself.

In WinPE (the MiniNT key in the registry) the partition Windows boots
from is not looked for: there is no such thing there.

4. franzscelta, and the map of xc

franzscelta is a list with check boxes on the console API of Windows:
lines of text, boxes, fixed ones ([+]), forbidden ones ([-]),
one-of-many. It knows nothing of disks. ac_scegli and
xc_scegli fill it and read it back; a question (a file name, the
captcha) is asked on its last lines.

The numbers shown are the true numbers of the zones (02, 04, 06), never
1, 2, 3: they are the ones of the table, of the map, of the names in the
archive.

After F8 nothing special happens: ac decides each partition again
with the same function as without the screen, with "chosen" or "not
chosen" in the place of -all and -not. The map says optional=yes
for a partition that was a choice, so that xc can offer it.

franzxcdash is the live map of xc: cells of what is written, in
the order of the disk, each with its own counters. Not the whole disk: a
disk of 2 TB with 20 GB to write would be a screen of nothing. The lines
of the pieces are kept and printed after the map.

5. franzscegli: one hook before the parser

A table, one line a command:

{"x", "-force -checksum -longpath -flat -utf -fix255 -zero -space -all", "-to", "Where to extract", "./", true},
{"t", "-verify -ssd -paranoid -quick -checksum -collision -crc32 -all", "", "", "", true},
{"xc", "-gui* -verify -verbose", "", "", "", false},

the command, the switches offered (*: on at the start), the switch to
ask a value for, the question, the value of an empty answer, and whether
more archives can be marked. A new command is a new line.

And one hook, in main, before the command line is read: when the
first word is in the table, nothing that is an archive follows, and input
and output are a console, the chooser builds a command line as it would
have been typed, shows it, and puts it in the place of argc and argv.
From there on it is the parser of always. The switches are not "set":
they are read, as if typed. Nothing else in the program knows that a
chooser exists.

More archives: their list is given to multisomething(), the loop that
already runs t "name*" one archive at a time, with its reset of the
state between one and the next.

The chooser is on the primitives of the tui command (ANSI, get_key),
so it is on every platform that has tui; franzscelta is Windows
only, as ac and xc are. get_key got the left and right arrows, and
F1 to F4 as ESC O P...S (xterm, the terminal of Windows): they were
read as an ESC.

6. franzntfs2: the $MFT in memory

One hook in scandir(). The folder gives its volume (its handle, the
final path, the device: through the link of -vss it is the shadow
copy); the pieces of the $MFT are asked to Windows
(FSCTL_GET_RETRIEVAL_POINTERS) and read raw, 4 MB at a time; the
records become a table of names with their parents. The volume is
flushed first (0.03 to 0.13 seconds on a C:).

"The same list" is a strong word, and it is kept by construction:
where only Windows knows what it would answer, Windows is asked. A
folder that cannot be listed (one probe for each different security id,
not for each folder), long paths, odd names, cloud folders: that subtree
goes back to the normal scandir(). A file with a visible reparse point
is asked one by one, with the same calls.

The attributes are the ones Windows gives, not the ones on the disk:
some bits are only on the disk and are taken away; compressed, sparse and
encrypted come from the header of the data, not from the standard
information; the files compressed by Windows itself (WOF) are sparse and
reparse on the disk and plain for a program.

Two things found on the way:

• The kind of a name (Win32, DOS, POSIX) cannot be a signal of "odd
  name": a Windows laid down from an image has the POSIX kind on every
  folder.
• A shadow copy has the same serial number as the live volume: a
  cache of the volumes goes by device, not by serial.

The names come out already in the order of the table of the files and
are inserted with a hint: 0.9 microseconds a name instead of 1.95.

7. franzvhdx, the .vmdk in pieces, the GPT

franzvdisk is the base: something that takes blocks of a disk in order.
franzvmdk (one sparse file, pieces of 4,261,412,864 sectors as
Workstation makes them, flat) and franzvhdx (dynamic, blocks of 2 MB,
CRC-32C on the headers and on the region table, a block table of 64 bit)
are on it.

The image of the used clusters in the archive is not a .vhd: it is
records, a map, and a header and a footer kept to make the .vhd again.
So the backup of a volume over 2 TB was already right, and only the 32
bit block table of that header was in the way: over 2040 GB the .vhd is
not put together, the .vhdx is written instead.

The GPT is made when the virtual disk is written, not kept in the
archive: a protective MBR, the GPT, its copy at the end, a disk 1 MiB
longer.

8. Asking for the administrator, again

The command line for the elevated run was rebuilt from argv, the
arguments joined by spaces. An empty argument is nothing between two
spaces, and a path with a space becomes two arguments. ac "" reached
the elevated window as ac -pause -elevated: no archive, so no command,
so exit code 0. "(all OK)", in one second.

Now the line is the one Windows has (GetCommandLineW), with the name of
the program taken away: what was typed, quotes and all.

9. How it was tested

Two virtual machines for the disks: a Windows 10 on MBR and BIOS (a
reserved partition, C:, an extended one with two logical drives, the
recovery; a second disk as the destination), a Windows 11 on GPT and
UEFI. Windows code runs there over ssh, as an administrator.

• ac: every raw piece taken out of the archive and compared with
  the disk, byte by byte; CRC-32 and hash computed again against the map;
  a second version; an a -image mixed in; -sha256; BitLocker on a
  volume, unlocked and locked; FAT32 and exFAT; a partition without a
  file system; a drive kept open, with and without -vss; each image of
  -all taken back alone, mounted, its files compared by hash. A real
  archive of the Windows 11 disk (GPT, 64 GB): 82 seconds, 32 GB.
• xc: a disk made to look like a disk of Windows, MBR and GPT,
  saved and restored to a raw file, a .vhdx, a flat .vmdk and a
  physical disk, each compared with the original zone by zone and as a
  whole; the same disk with a file kept open on a data partition; dirty
  destination; mixed versions; -key; every refusal. 69 checks. Then the
  real disk of Windows 10: restored on an empty disk, the machine started
  from it.
• The screens: driven in a real console with scripted keys, the
  stream kept and drawn again to look at it; 80 columns.
• -ntfs: 40 forms of the argument; a live C: and its shadow copy;
  records of 4 KB, clusters of 64 KB and 2 MB, a compressed volume; an
  $MFT in 4,652 pieces; a standard user; a real backup both ways, the
  same 1,828 files.
• Over 2 TB: a volume of 4,000 GB on a thin virtual disk, with files
  below and above the 2 TB.
• The battery of the images (138 checks) and autotest -all after
  each step.
• Every platform, at the end: Windows (ucrt64; 32 bit, the open
  build, -DANCIENT, -DNOEMAIL, -DSFTP), Fedora 44 (gcc 16,
  -O1 and -O3), FreeBSD 14.2 (clang 18), OpenBSD 7.9 (clang 19), macOS
  12.7 (x86_64, the arm64 half, with macFUSE and FUSE-T), Solaris 11.4
  (gcc 7.3), the ESXi build (gcc 3.4.6, static), the eleven NAS binaries
  (musl). No errors; autotest where the binary runs.

10. Fixes

• An image on a full destination said "all OK" (Windows): the check
  of what was written against what was expected was skipped for images.
• -image -vss without a shadow copy fell back twice, to a raw image
  and then to a direct read, and ended well.
• The elevated run without its quotes (see above).
• g always ended with 0.
• The old -ntfs: no hard links, no folders, nothing at all without
  an administrator.
• d: the hash was SHA-1 whatever the help said (the default was set
  after the hasher had been chosen); two files not read had the same
  empty hash, so they were duplicates.
• written was counted where the data was handed over, not where it
  was written; with "" it read a counter nobody touched.
• A message with six digits, an error printed as a normal line.
• An extraction to stdout asked Windows for the kind of the disk (a slow
  question) every time: with -verbose only.
• ac writes "Windows 11" in its map (the registry still says 10).
• OpenBSD: arc4random_buf in the place of the last rand() the linker
  saw.
• Functions used on one platform only were compiled, and warned about,
  on the others.

11. Tried and thrown away

-ntfs2, a second switch next to the old -ntfs, with -ntfs2test
that ran both scans and compared them. It was the way to trust the new
code; once trusted, it took the name and the old code went.

A cache of the index of the volume, kept up with the USN journal: one
more thing that can be wrong on the day of the backup. The $MFT is read
every time: it is a second.

A threshold of time to pass from the normal scan to the $MFT on a
folder: by itself only on whole drives, where it always pays.

A shadow copy for the partition Windows boots from: on an EFI
partition (FAT32) it cannot be made anyway. Read as it is, and written
in the map.

Leaving out the space Windows reserves for the shadow copies (20 GB
on a volume of 4 TB, in the image as used clusters): the old shadow
copies would be left out with it.

A .vhd from xc: there is the .vhdx.

The whole disk in the map of xc: see above.

12. Known limits and open points

• The whole of ac, xc and the WinPE is new: see the first lines
  of this document.
• Started for real: a restored Windows 10 on MBR and BIOS, in a
  virtual machine. VM 11: a restored GPT and UEFI disk (it was
  compared with the original byte by byte, not booted); a restore on
  different hardware; a virtual machine started from a .vmdk or a
  .vhdx written by xc.
• Not tested: disks with sectors of 4,096 bytes; a disk of Windows
  over 2 TB; BitLocker on C:; a Windows that is not on C:; a read
  error inside a raw piece; Ctrl+C in the middle of a restore; the
  elevated window with a real UAC question (I do not have UACsa at all); the three refusals of ac
  (no disk made that way at hand).
• A restore on a smaller disk is refused. Linux: ac and xc
  are Windows only. xc, maybe, will become a Linux thing too (for restoring from USB stick. For now WinPE seems good enough)
• ac with -not of big folders inside the image costs a lot (ten
  minutes and 12 GB of RAM for six folders with 768,000 files), as
  a c: -image -not does.
• With -all each partition is consistent in itself, not with the
  others.
• [archive before] in the map has the list of the versions, not the
  hashes of the pieces of the run before: they are not known without
  reading the index again.
• cloud ... -image with the local archive on a disk that fills up still
  says OK, and uploads an archive cut short. cloud with files ends with
  an error, after a banner that says OK.
• On a full destination the whole source is read anyway.
• -ntfs does nothing on the multithread scan (-ssd); the $MFT stays
  in memory until the last source is done. Not tried: cluster of 512
  bytes, more than 1.1 million files, real slow disks.
• d: a symbolic link and its target have the same content; when the
  link is the one that stays, it is left pointing to nothing.
• The chooser: not tried on FreeBSD and macOS; the function keys as PuTTY
  sends them; folders with thousands of files. Folders can be marked, and
  nothing uses them yet: that is for a.
• The limits of 65.8 that are still there: sectors of 4,096 bytes in the
  virtual disks, the image of a *nix device to a raw file only.