lizardfs/lizardfs v3.10.2

Release 3.10.2

on GitHub

Dear Users,

LizardFS 3.10.2 is out!
Featuring:

Master:

• redesign in-memory representation of file system objects - at least 30% reduction in RAM usage
• name storage: a possibility to keep all file names in BerkeleyDB, thus saving even more RAM
• redesign of trash - increased performance, reduced RAM usage and CPU pressure
• huge boost of responsiveness - lengthy operations split into asynchronous bits
• OPERATIONS_DELAY* config entries, which allow postponing metadata operations on restart/disconnect
• mount: make big snapshot, setgoal and settrashtime operations asynchronous to improve responsiveness
• fix improper handling of endangered chunks

Chunkserver:

• memory optimizations - at least 60% reduction in RAM usage
• introduce smart descriptor management

Tools:

• brand new lizardfs command, a unified replacement for mfs* tools with prompt and bash completion

Other:

• various fixes and community requests

Details:

Master’s memory and CPU
With 3.10.2, the master is much more memory-efficient and responsive. RAM usage reduction was accomplished through redesigning the way it keeps metadata in memory, adding a possibility to keep some metadata in on-disk database (see USE_BDB_FOR_NAME_STORAGE in mfsmaster.cfg file for reference). Responsiveness was dramatically increased by recognizing all potentially lengthy operations and splitting their execution in smaller time bits. It applied to both in-master maintenance procedures and client’s requests like snapshotting/changing goal of very big directories.

lizardfs tool:
3.10.2 comes with new client tool: lizardfs utility. See man lizardfs, lizardfs -h for details. Aside from known functionalities like setting/getting goals, making snapshots, etc. it comes with bash-like prompt and bash completion.

Examples:
lizardfs setgoal -r ec32 a/

lizardfs makesnapshot dira/ dirb/

lizardfs fileinfo a/*

lizardfs <<EOF
fileinfo a/*
setgoal -r 3 a
getgoal a/2
EOF

What’s next:
In next release, paths and build system is to be unified with the one from official Debian repository (e.g. configuration directory will officially become /etc/lizardfs). Also, more asynchronous client commands are coming including recursive-remove operation. A so called “minimal goal setting”, which will probably be named MIN_REDUNDANCY_LEVEL is being implemented as well. Finally, more news on native HA (μRaft) will appear.

Best regards,
LizardFS Team