zen-fs/core v2.0.0

2.0.0

on GitHub

2.0 is finally here! The main reason for a semver-major release is that a lot of cleanup was needed that involved breaking changes (like removing deprecated APIs). This release does include some big features though!

How many downloads?

Before getting into the details of this release, I wanted to thank the users of ZenFS. ZenFS and Utilium have now received over 1,000,000 downloads (yes, one million). I'm proud of what ZenFS has matured into, and I couldn't have done this without the community. So, thank you all for supporting the project, even if just by downloading it.

Extended attributes / xattr

This release adds full support for extended attributes (a.k.a xattr), which behave the same as on Linux. Node.js doesn't even have an API for this yet! This allows you to store arbitrary metadata on files, though a limit of 1 KiB per attribute value is imposed for performance and sanity. Like Linux, xattr names are namespaced— you will need to use the user namespace. See #189 for the initial issue.

ioctl and inode flags

With the new ioctl and ioctlSync functions, you can do a few cool things:

1. Inode flags

Using the IOC.GetFlags and IOC.SetFlags commands, the flags on an file can be changed. At the moment, these are the flags that affect VFS behavior:

• NoAtime: When set, the atime of files will not be updated. This replaces the experimental disableUpdateOnRead configuration flag.
• Sync: When set, files will be synced to the file system immediately following any changes. This is off by default, meaning any changes will only be synced once the file is closed. This replaces the experimental onlySyncOnClose configuration flag.

These flags are currently only available on a per-file basis, though support for FS-wide flags will likely be added to mount in a minor release in the near future.

2. File "versions"

IOC.GetVersion and IOC.SetVersion manipulate the version field of inodes. While this field isn't used internally, it could be useful for some use cases— go crazy!

3. File system commands

• IOC.GetLabel and IOC.SetLabel can be used to read and write the label of a file system.
• IOC.GetUUID will retrieve the UUID of a file system. At the moment, each file system instance is assigned a random UUID when instantiated. Only SingleBuffer persists the UUID.
• IOC.GetSysfsPath just gives you /sys/fs/${fs.name}/${fs.uuid}. sysfs is very complicated and has not be implemented.
• IOC.GetXattr/IOC.SetXattr are not related to extend attributes. These are for working with a struct fsxattr. This structure has not been fully implemented, you should not use it.

As of this release, there is not mechanism for file systems to register ioctl command handlers. This is planned for a future minor release.

Contexts

This release makes contexts more powerful by introducing context ids, parent/children tracking, context-dependent working directories, and context-dependent file descriptors. The exported boundContexts Map can be used to access all of the bound contexts (a notable use case is /proc). You can create a child of a BoundContext using its bind method.

Also, this release fixed CredentialsInit not being plural, improves the types of BoundContext, and changes chroot to always be in-place and perform more anti-escape checks. mounts is no longer accessible on the fs namespace export, since this could be used to escape a context.

Internally, a "default" context is used for unbound operations. This absorbs the old "default"/unbound credentials.

Internal API changes: metadata, synchronization, and files

This release significantly improves how metadata and synchronization is handled internally.

• The internal File class has been completely absorbed into the VFS. This comes with some nice performance and maintainability benefits. You can look at SyncHandle for the synchronous logic from File.
• The id property of FileSystem has been renamed to type, since it is used to identify the class rather than the instance, which was ambiguous.
• A new touch method has been added to FileSystem. touch only updates metadata. All of the internal metadata updates now go through this new method.
• FileSystem.sync no longer takes any parameters. It is a no-op for current backends, though it can likely be used as a barrier for asynchronous operations in the future.
• All of the internal APIs for passing around metadata now use InodeLike or Inode instead of StatsLike.
• Whenever an inode is updated, its ctime is now changed automatically.
• The upgrade path for format 1-3 inodes has been completely removed. These are from much older versions of ZenFS, if you are using one of these versions, please upgrade to 1.11 first.
• The createFile and mkdir methods of FileSystem have had their mode parameter merged into CreationOptions. (PureCreationOptions has been removed).
• File open flags are now represented using numbers instead of strings internally, which improved performance and simplified some checks and imports.
• Added an internal hasAccess function, so permissions can be checked without copying an inode and creating a new Stats instance, significantly improving performance.

Configuration and logging

• Passing a boolean-returning function for the type of an option is now supported.
• The MountConfiguration used for a backend is now passed to that backend's isAvailable function. (#196)
• Improved backend options type.
• Configuration.log is no longer experimental.
• The built-in log formatting has been overhauled. Read the documentation on log.fancy for more information.
• The error thrown when sync preloading is disabled is no longer written to the log.
• Logging in tests has been improved.

Other changes

• The readLines method of FileHandle has finally been implemented. This actually involved implementing most of the node:readline module, which can be imported from @zenfs/core/readline.js.
• src/vfs/path.ts has been moved to src/path.ts. You can import from @zenfs/core/path.js.
• Added support for opening files with the synchronous flag (s).
• File descriptor numbers now start at 4.
• Access Control Lists (ACLs) have been implemented, though they are not checked for now. (#194)
• Device inode data can now be customized by the device driver.
• Fixed device reads and writes not working correctly with device sizes.
• Fixed multiple problems with ReadStream. (#193)
• All of the code for the Port backend has been consolidated into src/backends/port.ts.
• The prefix in Passthrough is no longer normalized. (#203)
• Refactored vfs/config.ts for performance.
• Fixed the metadata offset not being set on fresh super blocks for SingleBuffer. (#205, thanks @tbrockman)
• Fixed streaming methods not checking if the file was closed.
• Fixed symlinkSync not closing the file it opened.
• Fixed an access check missing in statSync.
• The RPC code used for Port has been refactored.
• The encode/decode UTF8/Raw functions and Concrete type have been moved to Utilium.