Jump to content

Opus (audio format): Difference between revisions

From Wikipedia, the free encyclopedia
Content deleted Content added
Add second reference concerning patent claims
m →‎Hardware: Added phone
Line 130: Line 130:
Since version 3.13 [[Rockbox]] enables Opus playback on supported [[portable media players]], including products from the [[iPod]] series by [[Apple Inc.|Apple]], devices made by [[iriver]], [[Archos]] and [[Sandisk]], and on [[Android (operating system)|Android]] devices using "Rockbox as an Application" (RaaA).<ref name="rockbox_v313"/><ref name="RaaA"/> [[Chromecast]] supports Opus decoding.
Since version 3.13 [[Rockbox]] enables Opus playback on supported [[portable media players]], including products from the [[iPod]] series by [[Apple Inc.|Apple]], devices made by [[iriver]], [[Archos]] and [[Sandisk]], and on [[Android (operating system)|Android]] devices using "Rockbox as an Application" (RaaA).<ref name="rockbox_v313"/><ref name="RaaA"/> [[Chromecast]] supports Opus decoding.


[[Grandstream]] GXV3240 video [[IP phone]] supports Opus audio both for encoding and decoding.
[[Grandstream]] GXV3240 and GXV3275 video [[IP phone|IP phones]] supports Opus audio both for encoding and decoding.


== See also ==
== See also ==

Revision as of 14:27, 17 July 2014

Opus
Filename extension
.opus
Internet media type
audio/ogg [1]
Developed byIETF codec working group
Initial releaseSeptember 11, 2012 (2012-09-11)
Type of formatAudio
Contained byOgg, Matroska
Extended fromSILK, CELT
StandardRFC 6716
Free format?Yes
Websiteopus-codec.org
libopus
Developer(s)Xiph.Org Foundation
Initial releaseAugust 26, 2012 (2012-08-26)
Stable release
1.1 / December 5, 2013; 10 years ago (2013-12-05)
Preview release
1.1-rc / November 26, 2013; 10 years ago (2013-11-26)
Written inC89
PlatformCross-platform
TypAudio codec, reference implementation
License3-clause BSD license
WebsiteOpus codec downloads

Opus is a lossy audio coding format developed by the Internet Engineering Task Force (IETF) that is particularly suitable for interactive real-time applications over the Internet.[2] As an open format standardized through RFC 6716, a reference implementation audio codec called opus-tools is provided under the 3-clause BSD license. All known software patents which cover Opus are licensed under royalty-free terms.[3]

Opus incorporates technology from two other audio coding formats: the speech-oriented SILK and the low-latency CELT.[2] Opus can be adjusted seamlessly between high and low bitrates, and internally, it transitions between linear predictive coding at lower bitrates and transform coding at higher bitrates (as well as a hybrid for a short overlap). Opus has a very low algorithmic delay (22.5 ms by default), which is a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Opus permits trading-off quality or bitrate to achieve an even smaller algorithmic delay, down to 5 ms. Its delay is very low compared to well over 100 ms for popular music formats such as MP3, Ogg Vorbis and HE-AAC; yet Opus performs very competitively with these formats in terms of quality per bitrate.[4] Unlike Ogg Vorbis, Opus does not require the definition of large codebooks for each individual file, making it preferable to Vorbis for short clips of audio.[5]

Eigenschaften

Possible bitrate and latency combinations compared with other audio formats

Opus supports constant and variable bitrate encoding from 6 kbit/s to 510 kbit/s, frame sizes from 2.5 ms to 60 ms, and certain sampling rates from 8 kHz (with 4 kHz bandwidth) to 48 kHz (with 20 kHz bandwidth, where the entire hearing range of the human auditory system can be reproduced). An Opus stream can support up to 255 audio channels, and it allows channel coupling between channels in groups of two using mid-side coding.

The inherently low delay in Opus (22.5 ms by default) makes it possible to be used in the same real-time applications required by telephony, Voice over IP and videoconferencing. For high quality audio, the ability to maintain low latency at higher bitrates is made possible by research done by the Xiph.Org Foundation on the CELT codec. During any Opus stream (live or in a file), the bitrate, bandwidth, and delay can be changed quickly and seamlessly without introducing any distortion or discontinuity in the audio.

As an open standard, the algorithms are openly documented, and a reference implementation (including the source code) is published. Broadcom and the Xiph.Org Foundation own software patents on some of the CELT algorithms, and Skype Technologies S.A./Microsoft own some on the SILK algorithms; but each has pledged to make them available royalty-free for use with Opus once the format was accepted as an IETF standard. They also reserve the right to make use of their patents to defend against infringement suits of third parties. The applicability of non royalty-free patent claims from Qualcomm and Huawei to Opus is disputed.[6][7]

Spectrogram of an Opus-encoded music recording at different average bitrates (~32 to ~160 kbit/s) clearly show the lowpass behavior of the encoder and the comparatively good preservation of the band energy with CELT (compare original, Vorbis, MP3, AAC).

The Opus format is based on the low-latency CELT format and the speech-oriented SILK format (both of which have been heavily modified, rendering them incompatible with their original formats). The transform layer (CELT) is based on the modified discrete cosine transform (MDCT) with approaches from CELP (codebooks for excitation, although in the frequency domain). CELT was modified and among other things, support for 20 ms frames was added. The SILK layer that specializes in speech signals is based on linear predictive coding (LPC) and an optional Long-Term Prediction filter. SILK was modified and among other things, support for 10 ms frames was added. To minimize packet overhead, SILK has support for a longer delay of 60 ms (versus CELT's 20 ms). The shared range encoding of both parts of a hybrid stream was taken from CELT.

The format has three different modes, two being for pure speech signals, and a third for general audio (including music and speech). One of the speech modes is capable of reproducing the full spectrum of the human hearing range. In this mode, CELT is used for the upper part of the frequency range (from 8 kHz upwards), and SILK is used for the lower part. For low bitrates (below about 30 kbit/s), the upper frequencies can be cut off and the CELT layer left out. For audio at higher bitrates, the SILK layer that specializes in speech signals is left out, and the non-specialized CELT layer is used.

The reference implementation is written in C and compiles on hardware architectures with or without a floating-point unit.

Opus data can be encapsulated in Ogg containers. The content of such Ogg Opus streams should be specified as audio/ogg; codecs=opus and for Ogg Opus files the .opus filename extension is recommended.[1] Support for encapsulation of Opus in Matroska containers has been completed.[8] Opus is also allowed as a registered format in MPEG-TS.[9]

Sample rates

Following sample rates are defined in Opus Request for Comments:[10]

Abbreviation Audio bandwidth Effective sample rate
NB (narrowband) 4 kHz 8 kHz
MB (medium-band) 6 kHz 12 kHz
WB (wideband) 8 kHz 16 kHz
SWB (super-wideband) 12 kHz 24 kHz
FB (fullband) 20 kHz 48 kHz

History

Opus was proposed for the standardization of a new audio format at the IETF, which was eventually accepted and granted by the codec working group. It is based on two initially separate standard proposals from the Xiph.Org Foundation and Skype Technologies S.A. (now Microsoft). Its main developers are Jean-Marc Valin (Xiph.Org, Octasic, Mozilla Corporation), Koen Vos (Skype), and Timothy B. Terriberry (Xiph.Org, Mozilla Corporation). Among others, Juin-Hwey (Raymond) Chen (Broadcom), Gregory Maxwell (Xiph.Org, Wikimedia), and Christopher Montgomery (Xiph.Org) were also involved.

The development of the CELT part of the format goes back to thoughts on a successor for Vorbis under the working name Ghost. As a newer speech codec from the Xiph.Org Foundation, Opus replaces Xiph's older speech codec Speex, an earlier project of Jean-Marc Valin. CELT has been worked on since November 2007.

The SILK part has been under development at Skype since January 2007 as the successor of their SVOPC, an internal project to make the company independent from third-party codecs like iSAC and iLBC and respective license payments.

In March 2009, Skype suggested the development and standardization of a wideband audio format within the IETF. Nearly a year passed with much debate on the formation of an appropriate working group.[11] Representatives of several companies which were taking part in the standardization of patent-encumbered competing formats stated objections against the start of the standardization process for a royalty-free format: representatives of Polycom and Ericsson—the creators and licensors of G.719—as well as France Télécom, Huawei and the Orange Labs (department of France Télécom), which were involved in the creation of G.718. The working group finally formed in February 2010, and even the corresponding Study Group 16 from the ITU-T pledged to support its work.

In July 2010, a prototype of a hybrid format was presented that combined the two proposed format candidates SILK and CELT. In September 2010, Opus was submitted to the IETF as proposal for standardization. For a short time the format went under the name of Harmony before it got its present name in October 2010.[12][13] At the beginning of February 2011, the bitstream format was tentatively frozen, subject to last changes.[14] Near the end of July 2011, Jean-Marc Valin was hired by the Mozilla Corporation to continue working on Opus.[15] In November 2011, the working group issued the last call for changes on the bitstream format. The bitstream has been frozen since January 8, 2012.[16] On July 2, 2012, Opus was approved by the IETF for standardization.[17] The reference software entered release candidate state on August 8.[18] The final specification was released as RFC 6716 on September 10, 2012.[6][19] and versions 1.0 and 1.0.1 of the reference implementation were released the day after.

On July 11, 2013, the first beta of version 1.1 was released incorporating significant encoder side improvements for quality and efficiency, and significant speed improvements for both encoding and decoding detailed in the release information. Surround sound encoding has better channel bitrate allocation, improved Low-Frequency Effects quality and efficiency, and use of cross-channel masking. Tonality estimation provides a bitrate boost to highly tonal sounds like harpsichord, significantly improving quality in Opus's weakest samples from previous public 64kbps tests in which it had otherwise performed well. Automated speech/music detection is enabled to improve quality. Other improvements include stereo bitrate saving when channels are closely correlated, dynamic allocation to improve quality for transients and high-energy bands adjacent to low-energy bands, DC rejection below 3 Hz to improve bass analysis. True unconstrained VBR improves quality consistency for same target bitrate, while Temporal VBR is a serendipitous discovery that improves quality on a number of samples.

Also on July 11, 2013, stable version 1.0.3 was introduced implementing a backport of the new 1.1 surround API mentioned in beta 1.1 above as well as some minor bug fixes.

Opus 1.1 was officially released on December 5, 2013.

Quality comparison and low latency performance

Comparison of coding efficiency between Opus and other popular audio formats

Opus has been shown to have excellent quality,[4] and at higher bit rates, it turns out to be competitive with audio formats with much higher delay, such as HE-AAC and Vorbis.[20]

In listening tests at around 64 kbit/s Opus shows superior quality compared to HE-AAC codecs, which have so far been dominant in that area because of their use of the proprietary spectral band replication (SBR) technology.[21][22]

Opus has very low algorithmic delay,[2] a necessity for use as part of a low audio latency communication link, which can permit natural conversation, networked music performances, or lip sync at live events. Total algorithmic delay for an audio format is the sum of delays that must be incurred in the encoder and the decoder of a live audio stream regardless of processing speed and transmission speed, such as buffering audio samples into blocks or frames, allowing for window overlap and possibly allowing for noise-shaping look-ahead in a decoder and any other forms of look-ahead, or for an MP3 encoder, the use of bit reservoir.[23]

Total one-way latency below 150 ms is the preferred target of most VoIP systems,[24] to enable natural conversation with turn-taking little affected by delay. Musicians typically feel in-time with up to around 30 ms audio latency,[25] roughly in accord with the fusion time of the Haas effect, though matching playback delay of each user's own instrument to the round-trip latency can also help.[26] It is suggested for lip sync that around 45-100 ms audio latency may be acceptable.[27]

Opus permits trading-off quality or bitrate to achieve an even smaller algorithmic delay (5.0 ms minimum).[28] While the default Opus frame is 20 ms long, a further 2.5 ms lookahead is required for window overlap in the CELT layer and noise shaping in the SILK layer, typically giving a delay of 22.5 ms.[disputeddiscuss] The SILK layer's minimum is 10 ms frames (12.5 ms delay), while the CELT layer's minimum is 2.5 ms (5.0 ms delay).[29]

Support

The format and algorithms are openly documented and the reference implementation is published as free software. Xiph's reference implementation is called opus-tools, and it consists of a separate encoder and decoder; it is published under the terms of a BSD-like license. It is written in C and can be compiled for hardware architectures with or without a floating point unit. The accompanying diagnostic tool opusinfo reports detailed technical information about Opus files, including information on the standard compliance of the bitstream format. It is based on ogginfo from the vorbis-tools and therefore — unlike the encoder and decoder — is available under the terms of version 2 of the GPL.

Software

Most end-user software relies on multimedia frameworks provided by the operating system. Native Opus codec support is implemented in most major multimedia frameworks for Unix-like operating systems, including GStreamer, FFmpeg and Libav libraries.[30][31][32] Although Microsoft Windows DirectShow framework lacks an Opus encoder and decoder, such capabilities may be acquired by installing third-party codecs like LAV Filters[33] or Xiph.org Foundation's OpenCodecs.

While support in multimedia frameworks automatically enables Opus support in software which is built on top of such frameworks, several applications developers made additional efforts for supporting the Opus audio format in their software. Such support was added to AIMP,[34] Amarok,[35] cmus, foobar2000,[36] Mpxplay, MusicBee,[37] SMplayer, VLC media player,[38] Winamp[39] and Xmplay audio players; Icecast,[40] Liquidsoap and Airtime[41] audio streaming software; and Asunder audio CD ripper, CDBurnerXP CD burner, FFmpeg, Libav and MediaCoder media encoding tools. Streaming Icecast radio trials are live since September 2012 and January 2013.[42][43] SteamOS uses Opus or Vorbis for streaming audio.[citation needed]

Opus support is mandatory for WebRTC implementations.[44] Opus is supported in Mozilla Firefox,[45] Chromium and Google Chrome,[46] Blink-based Opera,[47][48] as well as all browsers for Unix-like systems relying on GStreamer for multimedia formats support. Although Internet Explorer doesn't provide Opus playback natively, support for the format may be added by installing OpenCodecs by Xiph.Org Foundation.

Due to its abilities, Opus gained early interest from VoIP software vendors. Several SIP clients, including Acrobits Softphone, Bria X-Lite, CiceroSupra, CSipSimple (via additional plug-in), Empathy (via GStreamer), Jitsi,[49] Line2 (currently only on iOS),[50] Phoner and PhonerLite, SFLphone,[51] Mumble and TeamSpeak 3 voice chat software also support Opus.[52][53][54] TrueConf supports Opus in its VoIP products.[55] As Asterisk lacks builtin Opus support for legal reasons,[56] third-party patch is available for download.[57]

The GatesAir Intraplex IP Link and Tieline Merlin and Genie families of broadcast IP audio codecs and Tieline Report-IT mobile codecs add Opus support to embedded and mobile devices.[58]

Hardware

Since version 3.13 Rockbox enables Opus playback on supported portable media players, including products from the iPod series by Apple, devices made by iriver, Archos and Sandisk, and on Android devices using "Rockbox as an Application" (RaaA).[59][60] Chromecast supports Opus decoding.

Grandstream GXV3240 and GXV3275 video IP phones supports Opus audio both for encoding and decoding.

See also

References

  1. ^ a b "Ogg Encapsulation for the Opus Audio Codec". IETF. 2012-07-16. Retrieved 2012-09-12.
  2. ^ a b c "Opus Codec". Opus (Home page). Xiph.org Foundation. Retrieved July 31, 2012.
  3. ^ Bright, Peter (2012-09-12). "Newly standardized Opus audio codec fills every role from online chat to music". Ars Technica. Retrieved 2014-05-28.
  4. ^ a b Raymond Chen et al. Opus Testing. IETF 80
  5. ^ Terriberry, Timothy (2012-07-19). "Firefox Beta 15 supports the new Opus audio format". Mozilla Hacks. Mozilla Foundation. Retrieved 2012-07-31.
  6. ^ a b "It's Opus, it rocks and now it's an audio codec standard!". Mozilla Hacks. Retrieved 2012-09-12.
  7. ^ "Nokia's VP8 patent claims: we've been here before". OSNews. 2013-05-25. Retrieved 2014-07-17.
  8. ^ Bunkus, Moritz (2013-09-16). "MKVToolNix v6.4.0 released". Matroska users (Mailing list). Retrieved 2013-12-24. {{cite mailing list}}: Unknown parameter |mailinglist= ignored (|mailing-list= suggested) (help)
  9. ^ http://www.smpte-ra.org/mpegreg/Opus.html
  10. ^ Valin, Jean-Marc; Vos, Koen; Terriberry, Timothy B. (2012-09-11). "Opus Codec Overview". Definition of the Opus Audio Codec. p. 7. sec. 2. doi:10.17487/RFC6716. ISSN 2070-1721. RFC 6716. Retrieved 2014-06-21.
  11. ^ "IETF working towards royalty-free audio codec". H-online.com. 2009-11-13. Archived from the original on 25 January 2013. Retrieved 2012-09-12.
  12. ^ Vos, Koen (2010-10-15). "Harmony became Opus". ietfcodec Discussion Archive (Mailing list). Retrieved 2011-06-19. {{cite mailing list}}: External link in |mailinglist= (help); Unknown parameter |mailinglist= ignored (|mailing-list= suggested) (help)
  13. ^ "[codec] Harmony became Opus". Ietf.org. 2010-10-15. Retrieved 2012-09-12.
  14. ^ "IETF Opus codec now ready for testing". Hydrogenaudio. Retrieved 2012-09-12.
  15. ^ Valin, Jean-Marc (2011-08-01). "IETF update, Mozilla". LiveJournal.com. Retrieved 2012-10-05.
  16. ^ "Opus Codec". Opus Codec. Retrieved 2012-09-12.
  17. ^ "Opus approved by the IETF". Jmspeex.livejournal.com. 2012-07-03. Retrieved 2012-09-12.
  18. ^ "[opus] Release candidates for 1.0.0 and 1.0.1 are available". Lists.xiph.org. Retrieved 2012-10-05.
  19. ^ "RFC 6716 on Definition of the Opus Audio Codec". Ietf.org. Retrieved 2012-09-12.
  20. ^ Maxwell, Gregory (2011). "64kbit/sec stereo multiformat listening test — unofficial results page". Xiph.Org Foundation. Retrieved 2011-06-19.
  21. ^ Next-Gen Low-Latency Open Codec Beats HE-AAC, Slashdot-Meldung vom 14. April 2011
  22. ^ Valin, Jean-Marc; Vos, Koen; Skoglund, Jan (2011-10-24). Hoene, Christian (ed.). "Summary of Opus listening test results". Internet Engineering Task Force. Retrieved 2012-01-16.
  23. ^ Lutzky, Manfred; Schuller, Gerald; Gayer, Marc; Krämer, Ulrich; Wabnik, Stefan (2004). "A guideline to audio codec delay" (PDF). In AES 116th convention, Berlin, Germany: 8–11. Retrieved 30 October 2012.
  24. ^ Lironi, F.; Masseroni, C.; Trivisonno, R.; Ball, C.F. (2005). "Multi RAB-based multimedia services over GERAN mobile networks". Vehicular Technology Conference, VTC-2005-Fall. IEEE 62nd. 3: 1662–1666. Retrieved 30 October 2012.
  25. ^ Lago, Nelson Posse; Kon, Fabio (November 2004). "The quest for low latency" (PDF). In Proceedings of the International Computer Music Conference: 33–36. Retrieved 30 October 2012.
  26. ^ Carôt, Alexander (2010). "Low Latency Audio Streaming for Internet-Based Musical Interaction." in Streaming Media Architectures: Techniques and Applications: Recent Advances. IGI Global. pp. 362–383. ISBN 978-1-61692-833-9.
  27. ^ YE, NONG, VAN CHEN, and TONI FARLEY (2003). "Qos Requirements Of Multimedia Data On Computer Networks." Proceedings of the Second International Conference on Active Media Technology, Chongqing, PR China, 29-31 May 2003. World Scientific Publishing Company Incorporated. pp. 183–189. ISBN 978-981-238-343-3.{{cite book}}: CS1 maint: multiple names: authors list (link)
  28. ^ Montgomery, Christopher. "A quick showcase of the bleeding edge... CELT 0.10.0 @ constant PEAQ value, varying latency". CELT v0.10 (latest prior to Opus integration). xiph.org. Retrieved 30 October 2012.
  29. ^ Valin, Jean-Marc; et al. "Opus Codec Overview". IETF RFC 6716. IETF. Retrieved 30 October 2012. {{cite web}}: Explicit use of et al. in: |first= (help)
  30. ^ "GStreamer: news". Gstreamer.freedesktop.org. Retrieved 2012-10-05.
  31. ^ "Audio Codecs". FFmpeg General Documentation. Retrieved 2014-05-28.
  32. ^ "Audio Codecs". Libav General Documentation. Retrieved 2014-05-28.
  33. ^ "lavfilters — Open-Source DirectShow Media Splitter and Decoders". Google Project Hosting. Retrieved 2012-10-05.
  34. ^ "AIMP: Home Page". Aimp.ru. Retrieved 2012-10-05.
  35. ^ "Amarok 2.8 "Return To The Origin" released". [[Amarok (software)|]]. 2013-08-16. Retrieved 2014-05-28.
  36. ^ "foobar2000 v1.1.14 beta". Hydrogenaudio Forums. Retrieved 2012-10-05.
  37. ^ "MusicBee". MusicBee Forum. Retrieved 2013-08-17.
  38. ^ "VLC 2.0.4 Twoflower". VideoLAN. Retrieved 2012-10-19.
  39. ^ "Winamp". Winamp Forums. Retrieved 2013-01-04.
  40. ^ "Icecast 2.4 beta release". Lists.xiph.org. Retrieved 2012-09-12.
  41. ^ Airtime, Changelog. "Airtime/changelog at 2.4.x". Sourcefabric. Retrieved 1 July 2013.
  42. ^ Radio Jackie, Listen Now. "Opus Stream". Radio Jackie. Retrieved 14 May 2013.
  43. ^ Absolute Radio, Listen Labs. "Opus Streaming Trial". Absolute Radio. Retrieved 30 October 2012.
  44. ^ "Zwei Audio-Codecs für Echtzeit-Kommunikation im Browser". Heise.de. Retrieved 2012-09-12.
  45. ^ "Media formats supported by the HTML audio and video elements". Developer.mozilla.org. 2012-09-03. Retrieved 2012-09-12.
  46. ^ "Issue 104241: Support OPUS in Ogg files for the audio tag and Audio object". [[Chromium (web browser)|]] bug tracker. Retrieved 2014-05-28.
  47. ^ "Why no official mention of Opus support?". My.opera.com. 2012-07-19. Retrieved 2012-09-12.
  48. ^ Shankland, Stephen (2012-08-17). "How corporate bickering hobbled better Web audio". CNET News. Retrieved 2012-09-12.
  49. ^ Bocquet, Ludovic (2013-03-07). "Jitsi 2.0 Now Released!". XMPP Foundation. Retrieved 2014-05-28.
  50. ^ "A new Line2 for iOS experience with improved sound quality". Line2. 2012-10-18. Retrieved 2013-03-15.
  51. ^ "SFLphone — Task #14602: [Codec] Implement opus". Savoir-faire Linux. 2012-08-13. Retrieved 2012-09-12.
  52. ^ Brandon (2013-06-19). "Mumble 1.2.4 Has been released". Mumblevoice Blog. Retrieved 2014-05-28.
  53. ^ "TeamSpeak 3 Client 3.0.10 released". TeamSpeak Forums. Retrieved 2013-03-04.
  54. ^ "TeamSpeak 3 Server Version 3.0.7 released". TeamSpeak Forums. Retrieved 2013-03-04.
  55. ^ "TrueConf сделал ставку на аудиокодек Opus" (Press release) (in Russian). Moscow, Russia. 2012-09-19. Retrieved 2014-05-28. {{cite press release}}: Unknown parameter |trans_title= ignored (|trans-title= suggested) (help)
  56. ^ Jordan, Matthew (2013-05-29). "Opus and VP8". asterisk-dev (Mailing list). Retrieved 2014-05-28. {{cite mailing list}}: Unknown parameter |mailinglist= ignored (|mailing-list= suggested) (help)
  57. ^ "Asterisk Opus/VP8 patch". GitHub. Retrieved 2014-05-28.
  58. ^ "Tieline Integrates OPUS into Report-IT and IP codecs". Radio Magazine. 2012-12-18. Retrieved 2012-12-18.
  59. ^ Rockbox Contributors (2013-03-05). "Release notes for Rockbox v.3.13". Rockbox. Retrieved 2013-03-21. {{cite web}}: |author= has generic name (help)
  60. ^ "RockboxAsAnApplication2010 < Main < Wiki". Rockbox.org. Retrieved 2012-10-05.