This file lists noteworthy changes in LizardFS.

 * Lizardfs (3.13.0-rc1) (2018-06-28)
 - (all) uRaft HA
 - (all) fixes to EC handling
 - (all) nfs-ganesha plugin changed to use only C code
 - (mount) reduced number of secondary groups retrievals (better performance)
 - (mount) add fuse3 client (better performance, writeback cache)
 - (all) many fixes

 * Lizardfs (3.12.0-rc1) (2017-11-22)
 - (all) C API
 - (all) nfs-ganesha plugin
 - (all) RichACL support (which includes NFSv4)
 - (all) OSX ACL support
 - (master, mount) file lock fixes
 - (mount) client readahead enabled by default
 - (mount) AVX2 extensions support for erasure code goals
 - (chunkserver) more flexible options
 - (all) many fixes

 * Lizardfs (3.11.3) (2017-07-13)
 - (master) fix issues with reporting defective files
 - (mount) fix request size in read cache for empty results

 * Lizardfs (3.11.0) (2017-05-09)
 - (master) improve ACL implementation
 - (master) add option to avoid same-ip chunkserver replication
 - (master) add minimal goal configuration option
 - (master) reimplement directory entry cache for faster lookups
 - (master) add whole-path lookups
 - (master, chunkserver) add chunkserver load awareness
 - (mount) add readahead to improve sequential read perfromance
 - (mount) add secondary groups support
 - (tools) add correct-only flag to filerepair
 - (tools) add -s and -i options to snapshot command
 - (tools) add recursive remove operations (for removing large directories and snapshots)
 - (tools) add tool for stopping execution of tasks (snapshot, recursive remove, etc.)
 - (all) change to semantic versioning system
 - (all) many fixes

 * Lizardfs (3.10.4) (2016-10-07)
 - (master) task manager performance improvements
 - (master) trash fixes

 * Lizardfs (3.10.2) (2016-08-30)
 - (master) redesign in-memory representation of file system objects - at least 30% reduction in RAM usage
 - (master) name storage - a possibility to keep all file names in BerkeleyDB, thus saving even more RAM
 - (master) redesign of trash - increased performance, reduced RAM usage and CPU pressure
 - (master) huge boost of responsiveness - lengthy operations split into asynchronous bits
 - (master) OPERATIONS_DELAY* config entries, which allow postponing metadata operations on restart/disconnect
 - (master) fix improper handling of endangered chunks
 - (chunkserver) memory optimizations - at least 60% reduction in RAM usage
 - (chunkserver) introduce smart descriptor management
 - (tools) brand new `lizardfs` command, a unified replacement for mfs* tools with prompt and bash completion
 - (all) various fixes and community requests

 * Lizardfs (3.10.0-rc) (2016-03-14)
  - (all) Added erasure code goals
  - (all) Added per directory quotas
  - (all) Improved interaction with legacy version (chunkservers, mounts)
  - (all) Ports for OSX and FreeBSD
  - (all) Many fixes

 * Lizardfs (3.9.4) (2015-12-02)
  - (master) Removed master server overload on restarting chunkservers
  - (master) Improved global file locks engine
  - (chunkserver) Fixed leaking descriptors problem
  - (chunkserver) Improved mechanism of moving chunks to new directory layout
  - (chunkserver) Fixed issues related to scanning directories with new chunk format present
  - (mount) Removed hang in mount when chunkserver reported no valid copies of a file
  - (master) Changed handling of legacy (pre-3.9.2) chunkservers in new installations
  - (cgi) Added XOR replication to statistics
  - (all) Removed default linking to tcmalloc library due to performance drop

 * Lizardfs (3.9.2) (2015-10-23)

  - (all) Introduced XOR goal types
  - (all) Added file locks (flock & fcntl)
  - (all) Increased max number of files from 500 million to over 4 billion
  - (all) Introduced managing open file limits by PAM
  - (master) Improved consistency of applying changelogs by shadow masters
  - (master) Redesigned snapshot execution in master
  - (master) Redesigned chunk loop logic
  - (master) Removed hard coded connection limit
  - (master) Added option to limit chunk loop's CPU usage
  - (chunkserver) Added new network threads responsible for handling requests sent by chunkserver's clients
  - (chunkserver) Introduced new more efficient directory layout
  - (chunkserver) Added option to choose if fsync should be performed after each write for increased safety
  - (chunkserver) Removed hard coded connection limit
  - (chunkserver) Added replication network bandwidth limiting
  - (mount) Improved symlink cache and added configurable timeout value
  - (all) Minor bug fixes and improvements

 * Lizardfs (2.6.0) (2015-02-09)

  - (all) Added comments in all config files
  - (all) Improve messages printed by daemons when starting
  - (cgi) A new chunkserver's chart: number of chunk tests
  - (cgi) Fixed paths to static content
  - (cgi) New implementation of the CGI server; mfscgiserv is now deprecated.
  - (cgi) New table: 'Metadata Servers' in the 'Servers' tab
  - (chunkserver) Allowed starts with damaged disks
  - (chunkserver) A new option: HDD_ADVISE_NO_CACHE
  - (chunkserver) Improved handling of disk read errors
  - (chunkserver) Removed 'testing chunk: xxxxxx' log messages
  - (master) A new feature: disabling atime updates (globally)
  - (master) Added possibility to set priority on handling endangered chunks
  - (master) Fixed rotating changelogs and downloading files in shadow mode
  - (master) Fixed handling replication delay on chunkserver disconnection
  - (probe) New commands
  - (probe) Renamed to lizardfs-admin
  - (all) Minor bug fixes and improvements

 * Lizardfs (2.5.4) (2014-11-07)

  - (all) Boost is no longer required to build the source code of LizardFS or use the binary version
  - (all) Added tiering (aka 'custom goal') feature, which allows users to label chunkservers and to request chunks to be stored on specific groups of servers
  - (cgi) "Exports" tabs renamed to "Config", now it also shows goal definitions
  - (cgi) Added new tab "Chunks"
  - (probe) New command "chunks-health" makes it possible to get number of missing or endangered chunks
  - (master) Fixed reporting memory usage in CGI
  - (mount) Fixed caching contents of open directories
  - (mount) Add a .lizardfs_tweaks file
  - (all) Other minor fixes and improvements

 * Lizardfs (2.5.2) (2014-09-15)

  - (master, shadow) Metadata checksum mechanism, allowing to find and fix possible metadata inconsistencies between master and shadow
  - (mount, master) ACL cache in mount, reducing the load of the master server
  - (packaging) Support packaging for RedHat based systems
  - (master) Improved chunkserver deregistration mechanism in order to avoid temporary master unresponsiveness
  - (polonaise) Add filesystem API for developers allowing to use the filesystem without FUSE (and thus working also on Windows)
  - (all) Minor fixes and improvements

* LizardFS 2.5.0 (2014-07-15)

  - (master) High availability provided by shadow master servers
  - (mount, chunkserver) CRC algorithm replaced with a 3 times faster implementation
  - (mount, master) Support for quotas (for users and groups)
  - (mount, master) Support for posix access contol lists (requires additional OS support)
  - (mount, master) Support for global I/O limiting (bandwidth limiting)
  - (mount) Support for per-mountpoint I/O limiting (bandwidth limiting)
  - (adm) New package lizardfs-adm with a lizardfs-probe command-line tool which can be used to query the installation for variuos parameteres
  - (master) New mechanism of storing metadata backup files which improves performance of the hourly metadata dumps
  - (all) A comprehensive test suite added
  - (all) Multiple bugfixes

* LizardFS 1.6.28 (2013-10-16)

  - (all) compile with g++ by default
  - (deb) fix init scripts for debian packages
  - (all) fix build on Mac OS X
  - (cgi) introducing LizardFS logo

* MooseFS 1.6.27 (2012-08-09)

  - (mfsrestore) fixed bug - freeing filenames memory too early
  - (all) added initial support for extra attributes (xattr), which will be introduced in upcoming version 1.7
  - (master+metalogger) better change log synchronization (storage in master memory and sending expected version in metalogger - inspired by Davies Liu)
  - (master) acceptable difference of percent of hdd usage added to configuration (up to this version this parameter was constantly set to 0.01% - patch contributed by Davies Liu)
  - (master) added emergency store metadata to some other places on errors during standard hourly store (inspired by Davies Liu)
  - (cs) default space to be left (256MB) moved to config file (inspired by Davies Liu)
  - (cs) added extra limits in mfshdd.cfg (minimum space to be left, maximum space to be used)
  - (cs) fixed charts overflow issue (overflow in net transfer on about 575Mbps and hdd transfer on about 77MBps)
  - (metalogger) fixed issue: file variable was not clear after close (on rare occasions might cause segfault after disconnecting from master)
  - (all) cfg files moved form PREFIX/etc/ to PREFIX/etc/mfs/
  - (cgiserv) improved CGI handle (added support for custom http responses, such as "302 Found")
  - (master+cgi) showing disconnected chunkservers in "Servers" tab.
  - (deb+rpm) mfscgiserv moved from mfscgi to separate package, changes in init scripts
  - (mount) added option 'mfsdelayedinit' - for being run from fstab/init scripts
  - (master) optimized goal management in chunks
  - (master) fixed rare race-condition on clear/preserve cache during open in mount
  - (mount) fixed compiling problems on Mac OS X
  - (all) changed code to be more compatible with new gcc 4.7 (gcc 4.7 has too strong optimizations - it can generate unpredictable code)
  - (master) sustain session time could be defined in mfsmaster.cfg

* MooseFS 1.6.26 (2012-02-01)

  - (all) fixed signal handling in multithreaded modules
  - (master) added goal and trashtime limits to mfsexports.cfg
  - (metalogger) added simple check for downloaded metadata file (inspired by Davies Liu)
  - (master) better handle disk full (inspired by Davies Liu)
  - (master+metalogger) added keeping previous copies of metadata (inspired by Davies Liu)
  - (all) reload all settings on "reload" (SIGHUP)
  - (cs) disk scanning in background
  - (cs) fixed long termination issue (found by Davies Liu)
  - (master) fixed modify/open cache race

* MooseFS 1.6.25 (2011-12-30)

  - (metadump) fixed dumping big files (>2TiB)
  - (metarestore) fixed bug: nonexisting changelog file caused segv
  - (master+mount) added 'sugidclearmode' and 'mkdircopysgid' compatibility options
  - (master) improved chunk deletion algorithm (soft/hard limits per server)
  - (all) ready for new metadata file format, which will be introduced in upcomoing version 1.7
  - (all) ready for quota handling, which will be introduced in upcoming version 1.7

* MooseFS 1.6.24 (2011-12-06)

  - (master+mount) proxy in mount for mfstools (fixes problems with frequent connect to master)

* MooseFS 1.6.23 (2011-11-08)

  - (master+mount) removed directory cache (didn't increase performance as expected and caused many troubles)
  - (metarestore) added option (-i) - ignore some structure inconsistencies
  - (metarestore) added option (-b) - in case of errors save the best metadata file
  - (mount) more dynamic write cache management (changed condition ib<tb/5 to ib<3*fb where: ib - inode blocks in cache, tb - total blocks in cache, fb - free block in cache)
  - (master) save metadata file to alternative locations in case of error
  - (all) increased file length limit from 2TiB to 128PiB
  - (mfscgiserv) fixed directory traversal vulnerability
  - (mfscgiserv) added lockfile/pidfile and actions such as 'start', 'stop', 'restart' and 'test'.
  - (mfsmount) fixed parsing file with defaults

* MooseFS 1.6.22 (2011-05-09)

  - (mount) added resolving master hostname whenever connection has failed
  - (cs) removed getting attributes of every chunk during initialization - speeds up starting of chunkserver
  - (cs) changed calculating of total/used space. Superuser space now is subtracted from total space instead of adding it to used space
  - (master+mount) fixed directory cache.
  - (debian) rewritten init scripts to use mfscommon commands (start/stop/restart/reload) instead of start-stop-daemon (where stop caused killing all instances of daemon)
  - (debian) changed init scripts to bail out early if MFS*_ENABLE is not true (i.e. given daemon is not scripts-controlled)

* MooseFS 1.6.21 (2011-04-14)

  - (mount) added support of default config file (mfsmount.cfg)
  - (metarestore) fixed snapshot bug
  - (metarestore) improved tolerance for damaged changelog files
  - (master,mount) added full directory (with attributes) cache on client (mfsmount) side
  - (mount) added symlink cache on client (mfsmount) side
  - (mount) added hidden files '.oplog' and '.ophistory' with detailed info about current/historical operations performed by mfsmount
  - (master) added simple net topology support
  - (all) added -D_POSIX_PTHREAD_SEMANTICS to CFLAGS - needed by Solaris-like OSes
  - (cs) fixed detection of 'damaged disk' condition
  - (mount) fixed error caused segfaults during umount on certain conditions
  - (daemon) added 'test' command - checks if process is running and returns its PID

* MooseFS 1.6.20 (2011-01-14)

  - (cs) fixed "packet too big" issue during register to master (split big register packet with all chunks info into small packets)
  - (cs,metalogger,master) added minimum socket timeout (ten seconds)
  - (mount) accepting nop packets during write (for future use)

* MooseFS 1.6.19 (2010-12-15)

  - (debian) separated mfs-common and mfs-metalogger subpackages, added init scripts (contributed by Christopher Lewis)
  - (daemon) fixed return values (return non zero on error)
  - (cs) fixed chunk testing bug (any error during chunk opening caused assigning whole disk as damaged)
  - (cs,metalogger) added resolving master name when connection failed (patch contributed by Davies Liu)
  - (mount) added creating new session when prevoius is lost (inspired by Davies Liu)
  - (cs) added for unused chunks week delay before deletion (inspired by Davies Liu)
  - (cgi) added switching between name and IP in 'path' column in 'Disks' table (inspired by Davies Liu)
  - (master) do not update ctime when goal, trashtime or extra attributes are not changing

* MooseFS 1.6.18 (2010-11-08)

  - (metalogger) added sending metadata after metalogger startup
  - (master,metalogger) added sending two change logs together with metadata
  - (metarestore) imporved merging change logs
  - (all) added a lot of assertions (mainly NULL pointers, and unsuccessful thread functions)
  - (all) fixed some minor bugs and potential race conditions (makes valgrind happy)
  - (cs) added ability to use read-only disks in "marked for removal" mode (to retrieve missing chunks from damaged disks)
  - (cs) added showing scanning progress during startup
  - (all/master) added releasing used resources at the end.
  - (all) changed some malloc/free allocations into mmap/munmap
  - (all) new error messages routines
  - (daemons) added proper handling SIGINT in "foreground" mode
  - (cgiserv) fixed small bug (malformed request could crash cgiserv)

* MooseFS 1.6.17 (2010-07-20)

  - (master,tools) added automatic data cache management
  - (master,tools) new flag "nodatacache"

* MooseFS 1.6.16 (2010-07-19)

  - (master) added clearing suid/sgid bits
  - (master) added check for "sticky" flag during rename and unlink operations
  - (master) fixed posix compatibility (removing empty directory on rename)
  - (master) fixed posix compatibility (proper changing ctime)
  - (master) fixed some constants (better support for larger systems - with millions of files and chunks)
  - (master) fixed error logging (logging is turning off when there are too many messages)
  - (all) fixed some OpenSolaris compile issues (mainly added -D__EXTENSIONS__ to compile flags)
  - (all) fixed OpenSolaris pthreads issue (errno doesn't work correctly without proper compiler flags)
  - (man) added mfscgiserv man page, added "BIND" options descriptions to man pages
  - (cs) fixed hdd stats (overflow on negative time difference)

* MooseFS 1.6.15 (2010-04-09)

  - (daemons) fixed "double free" error
  - (metarestore) fixed bugs in "REPAIR" and "SNAPSHOT" entries
  - (master) fixed bug in "snapshot" command ("mfsmakesnapshot dir dir/" caused master to hung-up)
  - (master) preserving atime and mtime during "snapshot" operation (makes "snapshot" to work more like "cp -Rp" than "cp -R")
  - (cs) ommit "marked for removal" disks during chunk test loop

* MooseFS 1.6.14 (2010-03-19)

  - (all) improved messages
  - (cs) prevent from using localhost as a master IP address
  - (all) remove some BSDisms from code
  - (metalogger) added BIND_HOST option (as in mount and cs)

* MooseFS 1.6.13 (2010-02-08)

  - (master) fixed poll events bug
  - (autotools) added m4 module for posix threads detection
  - (master) added exporting "meta" to default mfsexports.cfg
  - (master,tools) removed "allowdatacache" flag (will be managed automatically in the future)
  - (master,cs) added support for not compressed PNG images in chart module (when build without zlib)

* MooseFS 1.6.12 (2010-01-25)

  - (mount) fixed locking in read module
  - (mount,cs) added ability to bind outgoing sockets to specific IP

* MooseFS 1.6.11 (2009-12-31)

  - (cs) removed dirent.d_type test
  - (cs) fixed some memory leaks
  - (mount) improved attribute cache

* MooseFS 1.6.10 (2009-12-09)

  - fixed errors introduced in 1.6.9
  - added 'reload' command to master
  - split chunk counters into 'all' and 'regular' (new approach to disks
   removal - since this version chunks on 'marked for removal' disks are not
   deleted)

* MooseFS 1.6.9 (2009-11-24)

  - updated manpages
  - added support for mlockall (prevent from swapping)
  - redesigned charts module
  - redesigned locking mechanizm (daemons)
  - added HDD I/O stats
  - removing unfinished jobs from queue on socket close
  - added 'cache file' flag

* MooseFS 1.6.8 (2009-10-26)

  - added options to set nice level and number of I/O retries in mfsmount

* MooseFS 1.6.7 (2009-10-05)

  - fixed error: damaged disk caused SIGSEGV in mfschunkserver

* MooseFS 1.6.6 (2009-10-02)

  - fixed errors introduced in 1.6.5
  - changed directory structure in CS (from 16 subfolders to 256 subfolders)

* MooseFS 1.6.5 (2009-09-23)

  - fixed some performance issues
  - added new module "mfsmetalogger"

* MooseFS 1.6.4 (2009-08-25)

  - rewrite mfs to use poll instead of select
  - fixed some errors in new write module
  - limit cache usage by inode in write module

* MooseFS 1.6.3 (2009-08-24)

  - reduced version increasing
  - increased open files limit in chunkserver
  - decreased time window in damage disk detection

* MooseFS 1.6.2 (2009-08-19)

  - fixed "race" in writedata module
  - increased session timeout in mfsmaster for new sessions

* MooseFS 1.6.1 (2009-08-14)

  - Added 'mapall' option to exports
  - Added flags 'noattrcache' and 'noentrycache'
  - Rewritten "Writedata" module in mfsmount
  - Added source (read) limit to replications
  - Prefer localhost during some operations (reduce network usage)

* MooseFS 1.6.0 (not released - only available in public repository)

 Common:
  - Removed duplicate source files.
  - Strip whitespace at the end of configuration file lines.

 Chunkserver:
  - Rewritten in multi-threaded model.
  - Added periodical chunk testing functionality (HDD_TEST_FREQ option).
  - New -v option (prints version and exits).

 Master:
  - Added "noowner" objects flag (causes objects to belong to current user).
  - Maintaining `mfsdirinfo` data online, so it doesn't need to be calculated
   on every request.
  - Filesystem access authorization system (NFS-like mfsexports.cfg file,
   REJECT_OLD_CLIENTS option) with ro/rw, password and maproot
   functionality.
  - New -v option (prints version and exits).

 Mount:
  - Rewritten options parsing in mount-like way, making possible to use
   standard FUSE mount utilities (see mfsmount(8) manual for new syntax).
   Note: old syntax is no longer accepted and mountpoint is mandatory now
   (there is no default).
  - Updated for FUSE 2.6+.
  - Added password, file data cache, attribute cache and entry cache options.
   By default attribute cache and directory entry cache are enabled,
   file data cache and file entry cache are disabled.
  - opendir() no longer reads directory contents - it's done on first
   readdir() now; fixes "rm -r" on recent Linux/glibc/coreutils combo.
  - Fixed mtime setting just before close() (by flushing file on mtime
   change); fixes mtime preserving on "cp -p".
  - Added statistics accessible through MFSROOT/.stats pseudo-file.
  - Changed master access method for mfstools (direct .master pseudo-file
   replaced by .masterinfo redirection); fixes possible mfstools race
   condition and allows to use mfstools on read-only filesystem.

 Tools:
  - Units cleanup in values display (exact values, IEC-60027/binary prefixes,
   SI/decimal prefixes); new options: -n, -h, -H and MFSHRFORMAT environment
   variable - refer to mfstools(8) manual for details).
  - mfsrgetgoal, mfsrsetgoal, mfsrgettrashtime, mfsrsettrashtime have been
   deprecated in favour of new "-r" option for mfsgetgoal, mfssetgoal,
   mfsgettrashtime, mfssettrashtime tools.
  - mfssnapshot utility replaced by mfsappendchunks (direct descendant of
   old utility) and mfsmakesnapshot (which creates "real" recursive
   snapshots and behaves similar to "cp -r").
  - New mfsfilerepair utility, which allows partial recovery of file with
   some missing or broken chunks.

 CGI scripts:
  - First public version of CGI scripts allowing to monitor MFS installation
   from WWW browser.

* MooseFS 1.5.14

   Not released to public.

* MooseFS 1.5.13

   Not released to public.

* MooseFS 1.5.12 (2009-01-28)

  - Fixed CRC cache in chunkserver

* MooseFS 1.5.11 (2008-12-18)

  - Added CRC-block and file descriptor cache in chunkserver.
  - Removed compilation warnings on 64-bit machines.
  - Optimized chunkserver choice in mfsmount.
  - Better replication algorithm.

* MooseFS 1.5.10 (2008-12-03)

  - Fixed replication algorithm.

* MooseFS 1.5.9 (2008-11-19)

  - Improved rebalancing algorithm.
  - Protections against overriding metadata.mfs.back by older metadata.mfs.
  - Fixes to build mfstools, mfsmaster and mfsmetarestore with Large File
   Support on Linux (previously only mfsmount had LFS enabled).
  - Less verbose logging in mfsmaster.
  - Log chunkserver disconnections in mfsmaster.

* MooseFS 1.5.8 (2008-08-26)

  - Better memory allocation in Master's chunks module.
  - Syslog messages used only for debugging purposes were commented out.

* MooseFS 1.5.7 (2008-08-01)

  - Fix for stack buffer overrun (by 4 bytes) in mfsmount.

* MooseFS 1.5.6 (2008-07-25)

  - Memory allocation fix in mfsmaster.

* MooseFS 1.5.5 (2008-07-10)

  - Mounting options fix for FreeBSD in mfsmount.

* MooseFS 1.5.4 (2008-07-08)

  - Blocking rebalance replications when undergoal chunks exist.
  - Minor bugfixes.
  - Build system improvements.
  - MooseFS and mfsmount version numbers synchronized.

* MooseFS 1.5 (2008-05-30)

  - First version released as Open Source.