Jump to content

List of file systems

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by 2001:d08:2832:b89:85eb:add2:99e0:3a60 (talk) at 14:11, 5 August 2023. The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

The following lists identify, characterize, and link to more thorough information on file systems. Many older operating systems support only their one "native" file system, which does not bear any name apart from the name of the operating system itself.

Disk file systems

Shared-disk file systems

Shared-disk file systems (also called shared-storage file systems, SAN file system, Clustered file system or even cluster file systems) are primarily used in a storage area network where all nodes directly access the block storage where the file system is located. This makes it possible for nodes to fail without affecting access to the file system from the other nodes. Shared-disk file systems are normally used in a high-availability cluster together with storage on hardware RAID. Shared-disk file systems normally do not scale over 64 or 128 nodes.

Shared-disk file systems may be symmetric where metadata is distributed among the nodes or asymmetric with centralized metadata servers.

Distributed file systems

See also: Comparison of distributed file systems

Distributed file systems are also called network file systems. Many implementations have been made, they are location dependent and they have access control lists (ACLs), unless otherwise stated below.

Distributed fault-tolerant file systems

Distributed fault-tolerant replication of data between nodes (between servers or servers/clients) for high availability and offline (disconnected) operation.

  • Coda from Carnegie Mellon University focuses on bandwidth-adaptive operation (including disconnected operation) using a client-side cache for mobile computing. It is a descendant of AFS-2. It is available for Linux under the GPL.
  • Distributed File System (Dfs) from Microsoft focuses on location transparency and high availability. Available for Windows under a proprietary software license.
  • HAMMER and HAMMER2 – DragonFly BSD's filesystems for clustered storage, created by Matt Dillon.[3][4]
  • InterMezzo from Cluster File Systems uses synchronization over HTTP. Available for Linux under GPL but no longer in development since the developers are working on Lustre.
  • LizardFS a networking, distributed file system based on MooseFS[5]
  • Moose File System (MooseFS) is a networking, distributed file system. It spreads data over several physical locations (servers), which are visible to a user as one resource. Works on Linux, FreeBSD, NetBSD, OpenSolaris and macOS. Master server and chunkservers can also run on Solaris and Windows with Cygwin.
  • Scality is a distributed fault-tolerant filesystem.
  • Tahoe-LAFS is an open source secure, decentralized, fault-tolerant filesystem utilizing encryption as the basis for a least-authority replicated design.
  • A FAT12 and FAT16 (and FAT32) extension to support automatic file distribution across nodes with extra attributes like local, mirror on update, mirror on close, compound on update, compound on close in IBM 4680 OS and Toshiba 4690 OS. The distribution attributes are stored on a file-by-file basis in special entries in the directory table.[6][7]

Distributed parallel file systems

Distributed parallel file systems stripe data over multiple servers for high performance. They are normally used in high-performance computing (HPC).

Some of the distributed parallel file systems use an object storage device (OSD) (in Lustre called OST) for chunks of data together with centralized metadata servers.

  • Lustre is an open-source high-performance distributed parallel file system for Linux, used on many of the largest computers in the world.
  • Parallel Virtual File System (PVFS, PVFS2, OrangeFS). Developed to store virtual system images, with a focus on non-shared writing optimizations. Available for Linux under GPL.

Distributed parallel fault-tolerant file systems

Distributed file systems, which also are parallel and fault tolerant, stripe and replicate data over multiple servers for high performance and to maintain data integrity. Even if a server fails no data is lost. The file systems are used in both high-performance computing (HPC) and high-availability clusters.

All file systems listed here focus on high availability, scalability and high performance unless otherwise stated below.

Name By License OS Description
Alluxio UC Berkeley, Alluxio Apache License cross-platform An open-source virtual distributed file system (VDFS).
BeeGFS (formerly FhGFS) Fraunhofer Society GNU GPL v2 for client, other components are proprietary Linux A free to use file system with optional professional support, designed for easy usage and high performance, used on some of the fastest computer clusters in the world. BeeGFS allows replication of storage volumes with automatic failover and self-healing.
CephFS Inktank Storage, a company acquired by Red Hat GNU LGPL Linux kernel, FreeBSD via FUSE[8] A massively scalable object store. CephFS was merged into the Linux kernel in 2010. Ceph's foundation is the reliable autonomic distributed object store (RADOS), which provides object storage via programmatic interface and S3 or Swift REST APIs, block storage to QEMU/KVM/Linux hosts, and POSIX filesystem storage which can be mounted by Linux kernel and FUSE clients.
Chiron FS GNU GPL v3 It's a FUSE-based, transparent replication file system, layering on an existing file system and implementing at the file system level what RAID 1 does at the device level. A notably convenient consequence is the possibility of picking single target directories, without the need of replicating entire partitions. (The project has no visible activity after 2008, a status request in Oct. 2009 in the chironfs-forum is unanswered.)
CloudStore Kosmix Apache License Google File System workalike. Replaced by Quantcast File System (QFS)
dCache DESY and others proprietary (Free for non-commercial usage)[9] A write once filesystem, accessible via various protocols.
General Parallel File System (GPFS) IBM Proprietary AIX, Linux and Windows A POSIX-compliant, high-performance, parallel filesystem. Support synchronous replication between attached block storage, and asynchronous replication to remote filesystems. Also support erasure coding on dual homed SAS attached storage, and distributed over multiple storage nodes.
Gfarm file system Asia Pacific Grid X11 License Linux, macOS, FreeBSD, NetBSD and Solaris Uses OpenLDAP or PostgreSQL for metadata and FUSE or LUFS for mounting.
GlusterFS Gluster, a company acquired by Red Hat GNU GPL v3 Linux, NetBSD, FreeBSD, OpenSolaris A general purpose distributed file system for scalable storage. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. GlusterFS is the main component in Red Hat Storage Server.
Google File System (GFS) Google internal software Focus on fault tolerance, high throughput and scalability.
Hadoop Distributed File System Apache Software Foundation Apache License Cross-platform Open source GoogleFS clone.
IBRIX Fusion IBRIX Proprietary
JuiceFS Juicedata Apache License cross-platform An open-source POSIX-compliant file system built on top of Redis and object storage (e.g. Amazon S3), designed and optimized for cloud native environment.
LizardFS Skytechnology GNU GPL v3 cross-platform An open source, highly available POSIX-compliant file system that supports Windows clients.
Lustre originally developed by Cluster File Systems and currently supported by OpenSFS GNU GPL v2 & LGPL Linux A POSIX-compliant, high-performance filesystem used on a majority of systems in the Top-500 list of HPC systems. Lustre has high availability via storage failover.
MapR FS MapR Proprietary Linux Highly scalable, POSIX compliant, fault tolerant, read/write filesystem with a distributed, fault tolerant metadata service. It provides an HDFS and NFS interface to clients as well as a noSQL table interface and Apache Kafka compatible messaging system.
MooseFS Core Technology GNU GPL v2 and proprietary[10] cross-platform (Linux, NetBSD, FreeBSD, macOS, OpenSolaris) A fault tolerant, highly available and high performance scale-out network distributed file system. It spreads data over several physical commodity x86 servers, which are visible to the user as one namespace. For standard file operations MooseFS acts like any other Unix-like file systems.
ObjectiveFS Objective Security Corporation Proprietary Linux, macOS POSIX-compliant shared distributed filesystem. Uses object store as a backend. Runs on AWS S3, GCS and object store devices.
OneFS distributed file system Isilon Proprietary[11] FreeBSD BSD-based OS on dedicated Intel based hardware, serving NFS v3 and SMB/CIFS to Windows, macOS, Linux and other UNIX clients under a proprietary software.
OIO-FS OpenIO Proprietary Linux OIO-FS provides file-oriented access to OpenIO SDS object storage backend. It is based on FUSE technology and presents a POSIX file system to users. This access can be used locally, or over a network using NFS or SMB.[12]
PanFS Panasas Proprietary Linux, macOS, FreeBSD A POSIX-compliant, high-performance, parallel filesystem used by HPC clusters. It uses erasure coding and snapshots for data protection, is based upon a scale-out object store, and is focused on transparent failure recovery and ease of use.
RozoFS Rozo Systems GNU GPL v2 Linux A POSIX DFS focused on fault-tolerance and high-performance, based on the Mojette erasure code to reduce significantly the amount of redundancy (compared to plain replication).
Scality Scality ring Proprietary Linux A POSIX file system[citation needed] focused on high availability and performance. Also provides S3/REST/NFS interfaces.
Tahoe-LAFS Tahoe-LAFS Software Foundation GNU GPL v2+ and other[13] Windows, Linux, macOS A secure, decentralized, fault-tolerant, peer-to-peer distributed data store and distributed file system.
Storj Storj Labs Affero General Public License v3[14] Linux A cooperative storage cloud platform for secure and decentralized data and object storage. Storage nodes can be run and operated by diverse entities.
XtreemFS Contrail E.U. project, the German MoSGrid project and the German project "First We Take Berlin" BSD 3-Clause[15] Linux, Solaris, macOS, Windows A cross-platform file system for wide area networks. It replicates the data for fault tolerance and caches metadata and data to improve performance over high-latency links. SSL and X.509 certificates support makes XtreemFS usable over public networks. It also supports striping for usage in a cluster.

In development:

Peer-to-peer file systems

Some of these may be called cooperative storage cloud.

  • Cleversafe uses Cauchy Reed–Solomon information dispersal algorithms to separate data into unrecognizable slices and distribute them, via secure Internet connections, to multiple storage locations.
  • Scality is a distributed filesystem using the Chord peer-to-peer protocol.
  • IPFS InterPlanetary File System is p2p, worldwide distributed content-addressable, file-system.

Special-purpose file systems

  • aufs an enhanced version of UnionFS stackable unification file system
  • AXFS (small footprint compressed read-only, with XIP)
  • Barracuda WebDAV plug-in. Secure Network File Server for embedded devices.
  • Boot File System is used on UnixWare to store files necessary for its boot process.
  • cdfs (reading and writing of CDs)
  • Compact Disc File System (reading and writing of CDs; experimental)
  • cfs (caching)
  • Cramfs (small footprint compressed read-only)
  • Davfs2 (WebDAV)
  • Freenet – Decentralized, censorship-resistant
  • FTPFS (FTP access)
  • GmailFS (Google Mail File System)
  • GridFS – GridFS is a specification for storing and retrieving files that exceed the BSON-document size limit of 16 MB for MongoDB.
  • lnfs (long names)
  • LTFS (Linear Tape File System for LTO and Enterprise tape)
  • MVFS – MultiVersion File System, proprietary, used by Rational ClearCase.
  • Nexfs Combines Block, File, Object and Cloud storage into a single pool of auto-tiering POSIX compatible storage.
  • OverlayFS – A union mount filesystem implementation for Linux. Used mainly by Docker for its image layers.
  • romfs
  • SquashFS (compressed read-only)
  • UMSDOS, UVFAT – FAT file systems extended to store permissions and metadata (and in the case of UVFAT, VFAT long file names), used for Linux
  • UnionFS – stackable unification file system, which can appear to merge the contents of several directories (branches), while keeping their physical content separate
  • Venti – Plan 9 de-duplicated storage used by Fossil.

Pseudo file systems

  • devfs – a virtual file system in Unix-like operating systems for managing device nodes on-the-fly
  • procfs – a pseudo-file system, used to access kernel information about processes
  • tmpfs – in-memory temporary file system (on Unix-like platforms)
  • sysfs – a virtual file system in Linux holding information about buses, devices, firmware, filesystems, etc.
  • debugfs – a virtual file system in Linux for accessing and controlling kernel debugging
  • configfs – a writable file system used to configure various kernel components of Linux
  • sysctlfs – allow accessing sysctl nodes via a file system; available on NetBSD via PUFFS,[16] FreeBSD kernel via a 3rd-party module,[17][18][19] and Linux as a part of Linux procfs.[20]
  • kernfs – a file system found on some BSD systems (notably NetBSD) that provides access to some kernel state variables; similar to sysctlfs, Linux procfs and Linux sysfs.
  • wikifs – a server application for Plan 9's virtual, wiki, file system

Encrypted file systems

File system interfaces

These are not really file systems; they allow access to file systems from an operating system standpoint.

  • FUSE (file system in userspace, like LUFS but better maintained)
  • LUFS (Linux userland file system – seems to be abandoned in favour of FUSE)
  • PUFFS (Userspace filesystem for NetBSD, including a compatibility layer called librefuse for porting existing FUSE-based applications)
  • VFS Virtual Filesystem

See also

References

  1. ^ IBM. 4690 OS Programming Guide Version 5.2, IBM document SC30-4137-01, 2007-12-06 ([1]).
  2. ^ Caldera (1997). Caldera OpenDOS Machine Readable Source Kit 7.01. The FDOS.EQU file in the machine readable source kit has equates for the corresponding directory entries.
  3. ^ Koen Vervloesem (2010-04-21). "DragonFly BSD 2.6: towards a free clustering operating system". LWN.net. Retrieved 2019-03-07.
  4. ^ Matt Dillon (2018-12-09). "hammer2/DESIGN". BSD Cross Reference. DragonFly BSD. Retrieved 2019-03-06.
  5. ^ LizardFS.org
  6. ^ IBM (2003). Information about 4690 OS unique file distribution attributes, IBM document R1001487, 2003-07-30. ("IBM Information about 4690 OS unique file distribution attributes - United States". Archived from the original on 2014-05-21. Retrieved 2014-05-20.): "[...] file types are stored in the "Reserved bits" portion of the PC-DOS file directory structure [...] only 4690 respects and preserves these attributes. Various non-4690 operating systems take different actions if these bits are turned on [...] when copying from a diskette created on a 4690 system. [...] PC-DOS and Windows 2000 Professional will copy the file without error and zero the bits. OS/2 [...] 1.2 [...] will refuse to copy the file unless [...] first run CHKDSK /F on the file. After [...] CHKDSK, it will copy the file and zero the bits. [...] when [...] copy [...] back to the 4690 system, [...] file will copy as a local file."
  7. ^ IBM. 4690 save and restore file distribution attributes. IBM document R1000622, 2010-08-31 ("IBM 4690 save and restore file distribution attributes - United States". Archived from the original on 2014-05-21. Retrieved 2014-05-20.).
  8. ^ "net/ceph14: Ceph delivers object, block, and file storage in a unified system". FreshPorts. Retrieved 2021-07-11.
  9. ^ "dCache Software License".
  10. ^ "MooseFS".
  11. ^ "OneFS 8.1 eLicensing and remote support changes".
  12. ^ "Features of OIO-FS — OpenIO SDS 18.10 Object Storage documentation". docs.openio.io. Retrieved 2018-12-20.
  13. ^ "about.rst in trunk/docs – tahoe-lafs". Tahoe-lafs.org. Retrieved 2014-02-09.
  14. ^ "README.md in github.com/storj/storj – Storj". github.com. Retrieved 2023-06-12.
  15. ^ "XtreemFS - License".
  16. ^ "mount_sysctlfs(8)". NetBSD Manual Pages. 2010-04-11. Retrieved 2021-07-10.
  17. ^ Pawel Jakub Dawidek (2002-12-24). "sysctlfs.README". Archived from the original (text/plain) on 2005-02-23. Retrieved 2021-07-10.
  18. ^ Pawel Jakub Dawidek (2002-12-24). "Re: Hmm, sysctlfs". Retrieved 2021-07-10.
  19. ^ "Hacking FreeBSD / sysctlfs / [9a7ced]". SourceForge. Retrieved 2021-07-10.
  20. ^ Terrehon Bowden; Bodo Bauer; Shen Feng. "Chapter 2: Modifying System Parameters - The /proc Filesystem". The Linux Kernel. Retrieved 2021-07-10.

External links

Retrieved from "https://en.wikipedia.org/w/index.php?title=List_of_file_systems&oldid=1168857857"