moosefs/moosefs v4.56.5

MooseFS 4.56.5

on GitHub

Dear MooseFS Users,

We are thrilled to announce the release of the long-awaited MooseFS 4.x Community Edition! 🎉

New Features and Enhancements

New Format of Chunk Storage: Erasure Codes (EC)

We have introduced a new format of chunk storage with Erasure Codes, which can store data in ARCHIVE and TRASH states as 4 or 8 parts with an additional parity part, allowing data restoration in case a part is lost. This method provides data restoration in case of missing parts while optimizing storage space:

• EC8+1: Uses 112.5% of space,
• EC4+1: Uses 125% of space,

while maintaining the same level of hardware failure protection as standard replication (200%).

New Options in Storage Class Definitions

• TRASH State: files in the MooseFS TRASH, so the deleted files that are retained for a configurable period of time, can now have custom storage definitions within a storage class.
• ARCHIVE State Configuration: ARCHIVE state is now fully configurable:
• Set file archive times based on last atime, mtime, or ctime.
• Choose between FAST archival (ASAP) or PER CHUNK based on mtime.
• Exclude files below a certain length from being archived.
• Logical Expressions in Label Definitions: Support for comprehensive logical expressions in label definitions is now available, allowing users to create more flexible and precise rules, providing greater control over how storage classes and states are applied to files. This includes the ability to use negations.
• State-Specific Label Overrides: Override label modes (STD, LOOSE, STRICT) for specific states: CREATE, KEEP, ARCHIVE, or TRASH.

Storage Class Based on File Patterns

Users can now assign file storage classes dynamically based on custom file name patterns, providing greater flexibility in managing data. For example, files with a .backup extension could be assigned to a specific storage class with lower redundancy, while files starting with important_ might be placed in a high-redundancy class.

Broadcast Mechanism for Server Restarts

Server restarts (both chunk server and master) no longer block I/O until the end of chunk registration process.

Improved Chunk Format

Enhanced handling of control sums for more secure chunk storage.

New Web-based Graphical User Interface with Dark Mode

Additional monitoring charts and system insights, offering improved chart systems and more detailed information. Moreover, dark mode is now available.

Multi-LAN Configuration

Support for clients across different LANs to remain within their LAN IP class while communicating with the master server. Can be used alongside the remap option known from MooseFS 3.x.

Enhanced IP Remap

Customizable IP-to-IP remapping instead of just fixed remapping of leading bits in IP addresses.

Detailed Daemon Info

MooseFS daemons instructed by an info signal can now generate detailed info files in working directory, listing configuration, state, and module data for enhanced troubleshooting.

CS-Master Consistency Checks

New mechanism to ensure consistency between master and chunk server data held in RAM.

New mfshdd.cfg Options

Flags with limits and options for disks with chunks shared with other processes, allowing better calculation of available/occupied space on disks.

New Offline Tool: mfsmetasearch

Allows offline searches for files in metadata based on various conditions such as chunk IDs, inodes, and attributes.

Updated Logging System

Clear division of log messages by severity levels, with customizable logging options and syslog reporting.

New ACL Handling Tools

Useful for operating systems without built-in ACL management tools.

General Algorithm Improvements

Numerous improvements to algorithms for faster and more efficient MooseFS performance, especially in edge-case scenarios.

Quality of Life Improvements

• Faster Metadata Loading at master server startup.
• mfssupervisor Tool for on-demand metadata saves.
• mfsfileinfo Tool can now display chunk paths on chunk servers.
• Enhanced Chunk Rebalance Algorithms.
• More Configuration Options for various operations and storage management.

Community Contributions

During the development of this version, we received valuable feedback and suggestions from our community, many of which have been implemented. A huge thank you to everyone who contributed to shaping this release of MooseFS!

MooseFS PRO

If you are looking for high availability (HA) for master servers, improved disk space management with enhanced data protection through EC configurations with higher redundancy levels, and dedicated support, we also offer MooseFS PRO version. This version is designed for enterprise environments where uptime and data integrity are critical, providing features like automatic failover for master servers, more advanced erasure coding for greater resilience against hardware failures, and professional support services to ensure smooth operation and rapid issue resolution. MooseFS PRO adds an extra layer of reliability and optimization for those managing large-scale, mission-critical storage systems.

We recommend to update your MooseFS clusters to this version.

Thank you,
Piotr / MooseFS Team