Gzip: Difference between revisions
Undid revision 964697146 by 208.184.162.133 (talk) |
Rescuing 7 sources and tagging 0 as dead.) #IABot (v2.0.9.5 |
||
(59 intermediate revisions by 45 users not shown) | |||
Line 1: | Line 1: | ||
{{short description|GNU file compression/decompression tool}} |
{{short description|GNU file compression/decompression tool}} |
||
{{Use dmy dates|date= |
{{Use dmy dates|date=June 2023}} |
||
{{Lowercase}} |
{{Lowercase title}} |
||
{{Infobox software |
{{Infobox software |
||
| name = gzip |
| name = gzip (software) |
||
| logo = Gzip-Logo.svg |
| logo = Gzip-Logo.svg |
||
| author = {{plainlist| |
| author = {{plainlist| |
||
Line 11: | Line 11: | ||
| developer = [[GNU Project]] |
| developer = [[GNU Project]] |
||
| released = {{Start date and age|df=yes|1992|10|31}} |
| released = {{Start date and age|df=yes|1992|10|31}} |
||
| latest release version = |
| latest release version = {{wikidata|property|preferred|references|edit|Q283647|P348|P548=Q2804309}} |
||
| latest release date = {{wikidata|qualifier|preferred|single|Q283647|P348|P548=Q2804309|P577}} |
|||
| url = https://lists.gnu.org/archive/html/info-gnu/2018-12/msg00019.html |
|||
| latest preview version = {{wikidata|property|preferred|references|edit|Q283647|P348|P548=Q51930650}} |
|||
| title = gzip-1.10 released [stable] |
|||
| latest preview date = {{wikidata|qualifier|preferred|single|Q283647|P348|P548=Q51930650|P577}} |
|||
| first = Jim |
|||
| operating system = [[Unix-like]], [[Plan 9 from Bell Labs|Plan 9]], [[Inferno (operating system)|Inferno]] |
|||
| last = Meyering |
|||
| publisher = The Free Software Foundation |
|||
| date = 2018-12-29 |
|||
| accessdate = 2018-12-31}}</ref> |
|||
| latest release date = {{Start date and age|2018|12|29|df=yes}} |
|||
| operating system = [[Unix-like]] |
|||
| genre = [[Data compression]] |
| genre = [[Data compression]] |
||
| programming language = [[C (programming language)|C]] |
| programming language = [[C (programming language)|C]] |
||
| license = [[GNU General Public License |
| license = [[GNU General Public License|GPL-3.0-or-later]] |
||
| website = {{URL|//www.gnu.org/software/gzip/}} |
| website = {{URL|//www.gnu.org/software/gzip/}} |
||
| repo = {{URL|//git.savannah.gnu.org/cgit/gzip.git}} |
| repo = {{URL|//git.savannah.gnu.org/cgit/gzip.git}} |
||
}} |
}} |
||
'''gzip''' is a [[file format]] and a [[software application]] used for [[Data compression|file compression and decompression]]. The program was created by [[Jean-loup Gailly]]<!--Person prefers lowercase l (ell), see his article--> and [[Mark Adler]] as a [[free software]] replacement for the [[compress]] program used in early [[Unix]] systems, and intended for use by [[GNU]] (the "g" |
'''gzip''' is a [[file format]] and a [[software application]] used for [[Data compression|file compression and decompression]]. The program was created by [[Jean-loup Gailly]]<!--Person prefers lowercase l (ell), see his article--> and [[Mark Adler]] as a [[free software]] replacement for the [[compress]] program used in early [[Unix]] systems, and intended for use by [[GNU]] (from where the "g" of gzip is derived). Version 0.1 was first publicly released on 31 October 1992, and version 1.0 followed in February 1993. |
||
The decompression of ''gzip'' format can be implemented as [[streaming algorithm]], an important feature for [[HTTP compression|Web protocols]], [[Electronic data interchange|data interchange]] and [[Extract, transform, load|ETL]] (in [[Standard streams|standard pipes]]) |
The decompression of the ''gzip'' format can be implemented as a [[streaming algorithm]], an important feature for [[HTTP compression|Web protocols]], [[Electronic data interchange|data interchange]] and [[Extract, transform, load|ETL]] (in [[Standard streams|standard pipes]]) applications. |
||
== File format == |
== File format == |
||
{{Infobox file format |
{{Infobox file format |
||
| name = gzip |
| name = gzip (file format) |
||
| extension = .gz |
| extension = .gz |
||
| mime = application/gzip<ref>{{cite IETF|title=The 'application/zlib' and 'application/gzip' Media Types|rfc=6713|publisher= |
| mime = application/gzip<ref>{{cite IETF|title=The 'application/zlib' and 'application/gzip' Media Types|rfc=6713|publisher=Internet Engineering Task Force|access-date=2014-03-01}}</ref> |
||
| uniform type = org.gnu.gnu-zip-archive |
| uniform type = org.gnu.gnu-zip-archive |
||
| magic = <code>1f 8b</code> |
| magic = <code>1f 8b</code> |
||
Line 42: | Line 37: | ||
| genre = [[Data compression]] |
| genre = [[Data compression]] |
||
| url = {{URL|http://gzip.org}} (obsolete) |
| url = {{URL|http://gzip.org}} (obsolete) |
||
| |
| open = Yes |
||
}} |
}} |
||
gzip is based on the [[DEFLATE]] algorithm, which is a combination of [[LZ77 and LZ78|LZ77]] and [[Huffman coding]]. DEFLATE was intended as a replacement for [[LZW]] and other [[patent]]-encumbered [[data compression]] [[algorithm]]s which, at the time, limited the usability of |
gzip is based on the [[DEFLATE]] algorithm, which is a combination of [[LZ77 and LZ78|LZ77]] and [[Huffman coding]]. DEFLATE was intended as a replacement for [[LZW]] and other [[patent]]-encumbered [[data compression]] [[algorithm]]s which, at the time, limited the usability of the [[compress]] utility and other popular archivers. |
||
"gzip" is often also used to refer to the gzip file format, which is: |
"gzip" is often also used to refer to the gzip file format, which is: |
||
* a 10-byte header, containing a [[magic number (programming)|magic number]] (<code>1f 8b</code>), the compression method (<code>08</code> for DEFLATE), 1-byte of header flags, a 4-byte timestamp, compression flags and the operating system ID. |
* a 10-byte header, containing a [[magic number (programming)|magic number]] (<code>1f 8b</code>), the compression method (<code>08</code> for DEFLATE), 1-byte of header flags, a 4-byte timestamp, compression flags and the operating system ID. |
||
* optional extra headers as allowed by the header flags, including the original filename, a comment field, an "extra" field, and the lower half of a [[CRC-32]] checksum for the header section.<ref>{{Cite web| |
* optional extra headers as allowed by the header flags, including the original filename, a comment field, an "extra" field, and the lower half of a [[CRC-32]] checksum for the header section.<ref>{{Cite web |last=Deutsch |first=L. Peter |date=May 1996 |title=GZIP file format specification version 4.3 |url=https://tools.ietf.org/html/rfc1952.html |journal= |language=en |publisher=Internet Engineering Task Force |doi=10.17487/RFC1952 |access-date=2019-07-23}}</ref> |
||
* a body, containing a DEFLATE-compressed [[Payload (computing)|payload]] |
* a body, containing a DEFLATE-compressed [[Payload (computing)|payload]] |
||
* an 8-byte |
* an 8-byte trailer, containing a [[Cyclic redundancy check|CRC-32]] checksum and the length of the original uncompressed data, modulo 2<sup>32</sup>.<ref>{{cite web|author1=Jean-loup Gailly|title=GNU Gzip|url=https://www.gnu.org/software/gzip/manual/gzip.html#index-options-4|website=Gnu.org|access-date=11 October 2015|archive-date=15 October 2015|archive-url=https://web.archive.org/web/20151015064748/http://www.gnu.org/software/gzip/manual/gzip.html#index-options-4|url-status=live}}</ref> |
||
[[File:Targzip.svg|thumb|gzip can be combined with the [[tar (file format)|tar]] program to compress multiple files.]] |
|||
[[File:Targzip.svg|frameless|left]] |
|||
Although its file format also allows for multiple such streams to be [[Concatenation|concatenated]] (gzipped files are simply decompressed concatenated as if they were originally one file),<ref>{{cite web| title=GNU Gzip: Advanced usage| url=//www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html|website=Gnu.org| |
Although its file format also allows for multiple such streams to be [[Concatenation|concatenated]] (gzipped files are simply decompressed concatenated as if they were originally one file),<ref>{{cite web| title=GNU Gzip: Advanced usage| url=https://www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html| website=Gnu.org| access-date=2012-11-28| archive-date=24 December 2012| archive-url=https://web.archive.org/web/20121224211417/https://www.gnu.org/software/gzip/manual/html_node/Advanced-usage.html| url-status=live}}</ref> gzip is normally used to compress just single files.<ref>{{cite web| title=Can gzip compress several files into a single archive?| url=http://www.gzip.org/#faq16|website=Gnu.org| access-date=2010-01-27 |archive-url=https://web.archive.org/web/20100722004012/http://www.gzip.org/#faq16 |archive-date=2010-07-22}}</ref> Compressed archives are typically created by assembling collections of files into a single [[tar (file format)|tar]] archive (also called [[tar (file format)|tarball]]),<ref>{{cite web|title=tarball, The Jargon File, version 4.4.7|url=http://www.catb.org/jargon/html/T/tarball.html|website=Catb.org|access-date=2010-01-27|archive-date=20 March 2017|archive-url=https://web.archive.org/web/20170320183848/http://www.catb.org/jargon/html/T/tarball.html|url-status=live}}</ref> and then compressing that archive with gzip. The final compressed file usually has the extension {{mono|.tar.gz}} or {{mono|.tgz}}. |
||
gzip is not to be confused with the [[ZIP (file format)|ZIP]] archive format, which also uses DEFLATE. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed [[tar (computing)|tarballs]] holding the same data, because it compresses files individually and cannot take advantage of redundancy between files ([[solid compression]]). |
gzip is not to be confused with the [[ZIP (file format)|ZIP]] archive format, which also uses [[Deflate|DEFLATE]]. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed [[tar (computing)|tarballs]] holding the same data, because it compresses files individually and cannot take advantage of redundancy between files ([[solid compression]]). |
||
The gzip file format is also not to be confused with the that of the compress utility, based on LZW, with extension {{mono|.Z}}; however, the gunzip utility is able to decompress .Z files.<ref>{{cite web | title=GNU Gzip | website=The GNU Operating System and the Free Software Movement | date=2023-02-05 | url=https://www.gnu.org/software/gzip/manual/gzip.html | access-date=2024-04-03 |quote=gunzip can currently decompress files created by gzip, zip, compress or pack. The detection of the input format is automatic. }}</ref> |
|||
==Implementations== |
==Implementations== |
||
Line 69: | Line 64: | ||
| repo = {{URL|//cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/gzip/}} |
| repo = {{URL|//cvsweb.netbsd.org/bsdweb.cgi/src/usr.bin/gzip/}} |
||
}} |
}} |
||
Various implementations of the program have been written. The most commonly known is the GNU Project's implementation using [[LZ77|Lempel-Ziv]] coding (LZ77). [[OpenBSD]]'s version of gzip is actually the [[compress]] program, to which support for the gzip format was added in OpenBSD 3.4. The 'g' in this specific version stands for ''[[wikt:gratis|gratis]]''.<ref>{{cite web|url=https://man.openbsd.org/gzip.1#HISTORY|title=OpenBSD gzip(1) manual page|website=Openbsd.org| |
Various implementations of the program have been written. The most commonly known is the GNU Project's implementation using [[LZ77|Lempel-Ziv]] coding (LZ77). [[OpenBSD]]'s version of gzip is actually the [[compress]] program, to which support for the gzip format was added in OpenBSD 3.4. The 'g' in this specific version stands for ''[[wikt:gratis|gratis]]''.<ref>{{cite web|url=https://man.openbsd.org/gzip.1#HISTORY|title=OpenBSD gzip(1) manual page|website=Openbsd.org|access-date=2018-02-04|publisher=OpenBSD}}</ref> [[FreeBSD]], [[DragonFly BSD]] and [[NetBSD]] use a BSD-licensed implementation instead of the GNU version; it is actually a [[command-line interface]] for [[zlib]] intended to be compatible with the GNU implementations' options.<ref>{{cite web|url=http://man.freebsd.org/gzip|title=gzip|publisher=Man.freebsd.org|date=2011-10-09|access-date=2014-03-01|archive-date=17 December 2019|archive-url=https://web.archive.org/web/20191217104949/https://www.freebsd.org/cgi/man.cgi?gzip|url-status=live}}</ref> These implementations originally come from [[NetBSD]], and support decompression of [[bzip2]] and the Unix [[pack (compression)|pack]] format. |
||
An alternative compression program achieving 3-8% better compression is [[Zopfli]]. It achieves gzip-compatible compression using more exhaustive algorithms, at the expense of compression time required. It does not affect decompression time. |
An alternative compression program achieving 3-8% better compression is [[Zopfli]]. It achieves gzip-compatible compression using more exhaustive algorithms, at the expense of compression time required. It does not affect decompression time. |
||
<code>pigz</code>, written by Mark Adler, is compatible with gzip and speeds up compression by using all available CPU cores and threads.<ref>{{cite web|url=https://zlib.net/pigz/|author=Mark Adler|website=zlib.net|title=pigz: A parallel implementation of gzip for modern multi-processor, multi-core machines|date= |
<code>pigz</code>, written by Mark Adler, is compatible with gzip and speeds up compression by using all available CPU cores and threads.<ref>{{cite web|url=https://zlib.net/pigz/|author=Mark Adler|website=zlib.net|title=pigz: A parallel implementation of gzip for modern multi-processor, multi-core machines|date=2017|access-date=23 December 2018|archive-date=18 December 2018|archive-url=https://web.archive.org/web/20181218151754/http://zlib.net/pigz/|url-status=live}}</ref> |
||
=== Damage recovery === |
|||
Data in blocks prior to the first damaged part of the archive is usually fully readable. Data from blocks not demolished by damage that are located afterward ''may'' be [[Data recovery|recoverable]] through difficult workarounds.<ref>[https://web.archive.org/web/20180708075208/http://www.gzip.org/recover.txt Recovering a damaged .gz file – Jean-loup Gailly, GZip.org]</ref> |
|||
== Derivatives and other uses == |
== Derivatives and other uses == |
||
The [[tar (computing)|tar]] utility included in most Linux distributions can extract .tar.gz files by passing the {{mono|z}} option, e.g., {{mono|tar -zxf file.tar.gz}}. |
The [[tar (computing)|tar]] utility included in most Linux distributions can extract .tar.gz files by passing the {{mono|z}} option, e.g., {{mono|tar -zxf file.tar.gz}}, where <code>-z</code> instructs decompression, <code>-x</code> means extraction, and <code>-f</code> specifies the name of the compressed archive file to extract from. Optionally, <code>-v</code> (''verbose'') lists files as they are being extracted.<ref>{{cite web |title=How To Extract / Unzip tar.gz Files From Linux Command Line |url=https://phoenixnap.com/kb/extract-tar-gz-files-linux-command-line |website=Knowledge Base by phoenixNAP |access-date=12 January 2022 |date=14 November 2019}}</ref> |
||
[[zlib]] is an abstraction of the DEFLATE algorithm in library form which includes support both for the gzip file format and a lightweight [[Data stream#General|data stream]] format in its API. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952. |
[[zlib]] is an abstraction of the DEFLATE algorithm in library form which includes support both for the gzip file format and a lightweight [[Data stream#General|data stream]] format in its API. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952. |
||
The gzip format is used in [[HTTP compression]], a technique used to speed up the sending of [[HTML]] and other content on the [[World Wide Web]]. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This [[Request for Comments|RFC]] also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because [[Internet Explorer]] does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.<ref>{{cite web |first=Eric |last=Lawrence |title=Compressing the Web |url=http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |date= |
The gzip format is used in [[HTTP compression]], a technique used to speed up the sending of [[HTML]] and other content on the [[World Wide Web]]. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This [[Request for Comments|RFC]] also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because [[Internet Explorer]] does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.<ref>{{cite web |first=Eric |last=Lawrence |title=Compressing the Web |url=http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |date=21 November 2014 |website=MSDN Blogs > IEInternals |publisher=[[Microsoft]] |access-date=2 November 2015 |archive-date=28 October 2015 |archive-url=https://web.archive.org/web/20151028172035/http://blogs.msdn.com/b/ieinternals/archive/2014/10/21/http-compression-optimize-file-formats-with-deflate.aspx |url-status=live }}</ref> |
||
zlib DEFLATE is used internally by the [[Portable Network Graphics]] (PNG) format. |
zlib DEFLATE is used internally by the [[Portable Network Graphics]] (PNG) format. |
||
Since the late 1990s, [[bzip2]], a file compression utility based on a block-sorting algorithm, has gained some popularity as a gzip replacement. It produces considerably smaller files (especially for source code and other structured text), but at the cost of memory and processing time (up to a factor of 4).<ref name="benchmark">{{cite web |url=http://compressionratings.com/comp.cgi?7-zip+9.12b++bzip2+1.0.5++gzip+1.3.3+-5 |title=Comparison Tool: 7-zip vs bzip2 vs gzip | website=compressionratings.com | |
Since the late 1990s, [[bzip2]], a file compression utility based on a block-sorting algorithm, has gained some popularity as a gzip replacement. It produces considerably smaller files (especially for source code and other structured text), but at the cost of memory and processing time (up to a factor of 4).<ref name="benchmark">{{cite web |url=http://compressionratings.com/comp.cgi?7-zip+9.12b++bzip2+1.0.5++gzip+1.3.3+-5 |title=Comparison Tool: 7-zip vs bzip2 vs gzip | website=compressionratings.com |access-date=1 November 2014 |url-status=dead |archive-url=https://web.archive.org/web/20141101212919/http://compressionratings.com/comp.cgi?7-zip+9.12b++bzip2+1.0.5++gzip+1.3.3+-5 |archive-date=1 November 2014 }}</ref> |
||
⚫ | |||
Research published in 2023 showed that simple lossless compression techniques such as gzip could be combined with a [[K-nearest neighbors algorithm|k-nearest-neighbor classifier]] to create an attractive alternative to [[deep neural networks]] for text classification in [[natural language processing]]. This approach has been shown to equal and in some cases outperform conventional approaches such as [[BERT (language model)|BERT]] due to low resource requirements, e.g. no requirement for [[GPU]] hardware.<ref>{{Cite journal |last1=Jiang |first1=Zhiying |last2=Yang |first2=Matthew |last3=Tsirlin |first3=Mikhail |last4=Tang |first4=Raphael |last5=Dai |first5=Yiqin |last6=Lin |first6=Jimmy |date=July 2023 |title="Low-Resource" Text Classification: A Parameter-Free Classification Method with Compressors |url=https://aclanthology.org/2023.findings-acl.426 |journal=Findings of the Association for Computational Linguistics: ACL 2023 |location=Toronto, Canada |publisher=Association for Computational Linguistics |pages=6810–6828|doi=10.18653/v1/2023.findings-acl.426 |s2cid=260668487 |doi-access=free }}</ref> |
|||
⚫ | |||
==See also== |
==See also== |
||
Line 94: | Line 94: | ||
* [[List of archive formats]] |
* [[List of archive formats]] |
||
* [[List of Unix commands]] |
* [[List of Unix commands]] |
||
* [[Libarc]] |
|||
* [[Brotli]] |
|||
==Notes== |
==Notes== |
||
Line 100: | Line 102: | ||
==References== |
==References== |
||
{{Refbegin}} |
{{Refbegin}} |
||
* RFC 1952 – GZIP file format specification version 4.3 |
* [https://tools.ietf.org/html/rfc1952 RFC 1952] {{Webarchive|url=https://web.archive.org/web/20190728213743/https://tools.ietf.org/html/rfc1952 |date=28 July 2019 }} – GZIP file format specification version 4.3 |
||
{{Refend}} |
{{Refend}} |
||
Line 115: | Line 117: | ||
[[Category:Free software programmed in C]] |
[[Category:Free software programmed in C]] |
||
[[Category:GNU Project software]] |
[[Category:GNU Project software]] |
||
[[Category:IBM i Qshell commands]] |
|||
[[Category:Inferno (operating system) commands]] |
|||
[[Category:Lossless compression algorithms]] |
[[Category:Lossless compression algorithms]] |
||
[[Category:Plan 9 commands]] |
|||
[[Category:Unix archivers and compression-related utilities]] |
[[Category:Unix archivers and compression-related utilities]] |
Revision as of 02:37, 11 July 2024
![]() | |
Original author(s) | |
---|---|
Developer(s) | GNU Project |
Initial release | 31 October 1992 |
Stable release | 1.13[1] ![]() |
Repository | git |
Written in | C |
Operating system | Unix-like, Plan 9, Inferno |
Typ | Data compression |
License | GPL-3.0-or-later |
Website | www |
gzip is a file format and a software application used for file compression and decompression. The program was created by Jean-loup Gailly and Mark Adler as a free software replacement for the compress program used in early Unix systems, and intended for use by GNU (from where the "g" of gzip is derived). Version 0.1 was first publicly released on 31 October 1992, and version 1.0 followed in February 1993.
The decompression of the gzip format can be implemented as a streaming algorithm, an important feature for Web protocols, data interchange and ETL (in standard pipes) applications.
File format
Filename extension |
.gz |
---|---|
Internet media type |
application/gzip[2] |
Uniform Type Identifier (UTI) | org.gnu.gnu-zip-archive |
Magic number | 1f 8b |
Developed by | Jean-loup Gailly and Mark Adler |
Type of format | Data compression |
Open format? | Yes |
Website | gzip |
gzip is based on the DEFLATE algorithm, which is a combination of LZ77 and Huffman coding. DEFLATE was intended as a replacement for LZW and other patent-encumbered data compression algorithms which, at the time, limited the usability of the compress utility and other popular archivers.
"gzip" is often also used to refer to the gzip file format, which is:
- a 10-byte header, containing a magic number (
1f 8b
), the compression method (08
for DEFLATE), 1-byte of header flags, a 4-byte timestamp, compression flags and the operating system ID. - optional extra headers as allowed by the header flags, including the original filename, a comment field, an "extra" field, and the lower half of a CRC-32 checksum for the header section.[3]
- a body, containing a DEFLATE-compressed payload
- an 8-byte trailer, containing a CRC-32 checksum and the length of the original uncompressed data, modulo 232.[4]
![](http://upload.wikimedia.org/wikipedia/commons/thumb/4/42/Targzip.svg/220px-Targzip.svg.png)
Although its file format also allows for multiple such streams to be concatenated (gzipped files are simply decompressed concatenated as if they were originally one file),[5] gzip is normally used to compress just single files.[6] Compressed archives are typically created by assembling collections of files into a single tar archive (also called tarball),[7] and then compressing that archive with gzip. The final compressed file usually has the extension .tar.gz oder .tgz.
gzip is not to be confused with the ZIP archive format, which also uses DEFLATE. The ZIP format can hold collections of files without an external archiver, but is less compact than compressed tarballs holding the same data, because it compresses files individually and cannot take advantage of redundancy between files (solid compression). The gzip file format is also not to be confused with the that of the compress utility, based on LZW, with extension .Z; however, the gunzip utility is able to decompress .Z files.[8]
Implementations
Developer(s) | The NetBSD Foundation |
---|---|
Repository | cvsweb |
Written in | C |
Typ | Data compression |
License | Simplified BSD License |
Various implementations of the program have been written. The most commonly known is the GNU Project's implementation using Lempel-Ziv coding (LZ77). OpenBSD's version of gzip is actually the compress program, to which support for the gzip format was added in OpenBSD 3.4. The 'g' in this specific version stands for gratis.[9] FreeBSD, DragonFly BSD and NetBSD use a BSD-licensed implementation instead of the GNU version; it is actually a command-line interface for zlib intended to be compatible with the GNU implementations' options.[10] These implementations originally come from NetBSD, and support decompression of bzip2 and the Unix pack format.
An alternative compression program achieving 3-8% better compression is Zopfli. It achieves gzip-compatible compression using more exhaustive algorithms, at the expense of compression time required. It does not affect decompression time.
pigz
, written by Mark Adler, is compatible with gzip and speeds up compression by using all available CPU cores and threads.[11]
Damage recovery
Data in blocks prior to the first damaged part of the archive is usually fully readable. Data from blocks not demolished by damage that are located afterward may be recoverable through difficult workarounds.[12]
Derivatives and other uses
The tar utility included in most Linux distributions can extract .tar.gz files by passing the z option, e.g., tar -zxf file.tar.gz, where -z
instructs decompression, -x
means extraction, and -f
specifies the name of the compressed archive file to extract from. Optionally, -v
(verbose) lists files as they are being extracted.[13]
zlib is an abstraction of the DEFLATE algorithm in library form which includes support both for the gzip file format and a lightweight data stream format in its API. The zlib stream format, DEFLATE, and the gzip file format were standardized respectively as RFC 1950, RFC 1951, and RFC 1952.
The gzip format is used in HTTP compression, a technique used to speed up the sending of HTML and other content on the World Wide Web. It is one of the three standard formats for HTTP compression as specified in RFC 2616. This RFC also specifies a zlib format (called "DEFLATE"), which is equal to the gzip format except that gzip adds eleven bytes of overhead in the form of headers and trailers. Still, the gzip format is sometimes recommended over zlib because Internet Explorer does not implement the standard correctly and cannot handle the zlib format as specified in RFC 1950.[14]
zlib DEFLATE is used internally by the Portable Network Graphics (PNG) format.
Since the late 1990s, bzip2, a file compression utility based on a block-sorting algorithm, has gained some popularity as a gzip replacement. It produces considerably smaller files (especially for source code and other structured text), but at the cost of memory and processing time (up to a factor of 4).[15]
AdvanceCOMP, Zopfli, libdeflate and 7-Zip can produce gzip-compatible files, using an internal DEFLATE implementation with better compression ratios than gzip itself—at the cost of more processor time compared to the reference implementation.[citation needed]
Research published in 2023 showed that simple lossless compression techniques such as gzip could be combined with a k-nearest-neighbor classifier to create an attractive alternative to deep neural networks for text classification in natural language processing. This approach has been shown to equal and in some cases outperform conventional approaches such as BERT due to low resource requirements, e.g. no requirement for GPU hardware.[16]
See also
- Comparison of file archivers
- Free file format
- List of archive formats
- List of Unix commands
- Libarc
- Brotli
Notes
- ^ Jim Meyering (19 August 2023). "gzip-1.13 released [stable]". Retrieved 20 August 2023.
- ^ The 'application/zlib' and 'application/gzip' Media Types. Internet Engineering Task Force. doi:10.17487/RFC6713. RFC 6713. Retrieved 1 March 2014.
- ^ Deutsch, L. Peter (May 1996). "GZIP file format specification version 4.3". Internet Engineering Task Force. doi:10.17487/RFC1952. Retrieved 23 July 2019.
- ^ Jean-loup Gailly. "GNU Gzip". Gnu.org. Archived from the original on 15 October 2015. Retrieved 11 October 2015.
- ^ "GNU Gzip: Advanced usage". Gnu.org. Archived from the original on 24 December 2012. Retrieved 28 November 2012.
- ^ "Can gzip compress several files into a single archive?". Gnu.org. Archived from the original on 22 July 2010. Retrieved 27 January 2010.
- ^ "tarball, The Jargon File, version 4.4.7". Catb.org. Archived from the original on 20 March 2017. Retrieved 27 January 2010.
- ^ "GNU Gzip". The GNU Operating System and the Free Software Movement. 5 February 2023. Retrieved 3 April 2024.
gunzip can currently decompress files created by gzip, zip, compress or pack. The detection of the input format is automatic.
- ^ "OpenBSD gzip(1) manual page". Openbsd.org. OpenBSD. Retrieved 4 February 2018.
- ^ "gzip". Man.freebsd.org. 9 October 2011. Archived from the original on 17 December 2019. Retrieved 1 March 2014.
- ^ Mark Adler (2017). "pigz: A parallel implementation of gzip for modern multi-processor, multi-core machines". zlib.net. Archived from the original on 18 December 2018. Retrieved 23 December 2018.
- ^ Recovering a damaged .gz file – Jean-loup Gailly, GZip.org
- ^ "How To Extract / Unzip tar.gz Files From Linux Command Line". Knowledge Base by phoenixNAP. 14 November 2019. Retrieved 12 January 2022.
- ^ Lawrence, Eric (21 November 2014). "Compressing the Web". MSDN Blogs > IEInternals. Microsoft. Archived from the original on 28 October 2015. Retrieved 2 November 2015.
- ^ "Comparison Tool: 7-zip vs bzip2 vs gzip". compressionratings.com. Archived from the original on 1 November 2014. Retrieved 1 November 2014.
- ^ Jiang, Zhiying; Yang, Matthew; Tsirlin, Mikhail; Tang, Raphael; Dai, Yiqin; Lin, Jimmy (July 2023). ""Low-Resource" Text Classification: A Parameter-Free Classification Method with Compressors". Findings of the Association for Computational Linguistics: ACL 2023. Toronto, Canada: Association for Computational Linguistics: 6810–6828. doi:10.18653/v1/2023.findings-acl.426. S2CID 260668487.
References
- RFC 1952 Archived 28 July 2019 at the Wayback Machine – GZIP file format specification version 4.3