Abstract: Embodiments feature families of rate allocation and rate control methods that utilize advanced processing of past and future frame/field picture statistics and are designed to operate with one or more coding passes. At least two method families include: a family of methods for a rate allocation with picture look-ahead; and a family of methods for average bit rate (ABR) control methods. At least two other methods for each method family are described. For the first family of methods, some methods may involve intra rate control. For the second family of methods, some methods may involve high complexity ABR control and/or low complexity ABR control. These and other embodiments can involve any of the following: spatial coding parameter adaptation, coding prediction, complexity processing, complexity estimation, complexity filtering, bit rate considerations, quality considerations, coding parameter allocation, and/or hierarchical prediction structures, among others.

Abstract: Given a sequence of images in a first codeword representation, methods, processes, and systems are presented for integrating reshaping into a next generation video codec for encoding and decoding the images, wherein reshaping allows part of the images to be coded in a second codeword representation which allows more efficient compression than using the first codeword representation. A variety of architectures are discussed, including: an out-of-loop reshaping architecture, an in-loop-for intra pictures only reshaping architecture, an in-loop architecture for prediction residuals, and a hybrid in-loop reshaping architecture. Syntax methods for signaling reshaping parameters, and image-encoding methods optimized with respect to reshaping are also presented.

Abstract: The present document relates to methods and apparatus for rendering input audio for playback in a playback environment. The input audio includes at least one audio object and associated metadata, and the associated metadata indicates at least a location of the audio object. A method for rendering input audio including divergence metadata for playback in a playback environment comprises creating two additional audio objects associated with the audio object such that respective locations of the two additional audio objects are evenly spaced from the location of the audio object, on opposite sides of the location of the audio object when seen from an intended listener's position in the playback environment, determining respective weight factors for application to the audio object and the two additional audio objects, and rendering the audio object and the two additional audio objects to one or more speaker feeds in accordance with the determined weight factors.

Abstract: Soundfield signals such as e.g. Ambisonics carry a representation of a desired sound field. The Ambisonics format is based on spherical harmonic decomposition of the soundfield, and Higher Order Ambisonics (HOA) uses spherical harmonics of at least 2nd order. However, commonly used loudspeaker setups are irregular and lead to problems in decoder design. A method for improved decoding an audio soundfield representation for audio playback comprises calculating a panning function (W) using a geometrical method based on the positions of a plurality of loudspeakers and a plurality of source directions, calculating a mode matrix (?) from the loudspeaker positions, calculating a pseudo-inverse mode matrix (?+) and decoding the audio soundfield representation. The decoding is based on a decode matrix (D) that is obtained from the panning function (W) and the pseudo-inverse mode matrix (?+).

Abstract: Described herein is a method (30) of rendering an audio signal (17) for playback in an audio environment (27) defined by a target loudspeaker system (23), the audio signal (17) including audio data relating to an audio object and associated position data indicative of an object position. Method (30) includes the initial step (31) of receiving the audio signal (17). At step (32) loudspeaker layout data for the target loudspeaker system (23) is received. At step (33) control data is received that is indicative of a position modification to be applied to the audio object in the audio environment (27). At step (38) in response to the position data, loudspeaker layout data and control data, rendering modification data is generated. Finally, at step (39) the audio signal (17) is rendered with the rendering modification data to output the audio signal (17) with the audio object at a modified object position that is between loudspeakers within the audio environment (27).

Abstract: A method for multi-band ducking of audio signals is provided. In some implementations, the method involves receiving, at a decoder, an input audio signal, wherein the input audio signal is a downmixed audio signal. In some implementations, the method involves separating the input audio signal into a first set of frequency bands. In some implementations, the method involves determining a set of ducking gains, a ducking gain corresponding to a frequency band of the first set of frequency bands. In some implementations, the method involves generating a broadband decorrelated audio signal, wherein ducking gains of the set of ducking gains are applied to at least one of: 1) a second set of frequency bands prior to generating the at least one broadband decorrelated audio signal: or 2) a third set of frequency bands that separates the at least one broadband decorrelated audio signal.

Abstract: A method for generating a bitstream indicative of an object based audio program is described. The bitstream comprises a sequence of containers. A first container of the sequence of containers comprises a plurality of substream entities for a plurality of substreams of the object based audio program and a presentation section. The method comprises determining a set of object channels. The method further comprises providing a set of object related metadata for the set of object channels. In addition, the method comprises inserting a first set of object channel frames and a first set of object related metadata frames into a respective set of substream entities of the first container. Furthermore, the method comprises inserting presentation data into the presentation section.

Abstract: Some implementations involve obtaining a plurality of echo references, the plurality of echo references including at least one echo reference for each audio device of a plurality of audio devices in an audio environment, each echo reference corresponding to audio data being played back by one or more loudspeakers of one audio device of the plurality of audio devices. Some examples involve making an importance estimation for each echo reference of the plurality of echo references. Making the importance estimation may involve determining an expected contribution of each echo reference to mitigation of echo by at least one echo management system of at least one audio device of the audio environment. Some implementations involve selecting, based at least in part on the importance estimation, one or more selected echo references and providing the one or more selected echo references to the at least one echo management system.

Abstract: In some embodiments, virtualization methods for generating a binaural signal in response to channels of a multi-channel audio signal, which apply a binaural room impulse response (BRIR) to each channel including by using at least one feedback delay network (FDN) to apply a common late reverberation to a downmix of the channels. In some embodiments, input signal channels are processed in a first processing path to apply to each channel a direct response and early reflection portion of a single-channel BRIR for the channel, and the downmix of the channels is processed in a second processing path including at least one FDN which applies the common late reverberation. Typically, the common late reverberation emulates collective macro attributes of late reverberation portions of at least some of the single-channel BRIRs. Other aspects are headphone virtualizers configured to perform any embodiment of the method.

Abstract: Training image pairs comprising training SDR image and corresponding training HDR images are received. Each training image pair in the training image pairs comprises a training SDR image and a corresponding training HDR image. The training SDR image and the corresponding training HDR image in the training image pair depict same visual content but with different luminance dynamic ranges. Training image feature vectors are extracted from training SDR images in the training image pairs. The training image feature vectors are used to train backward reshaping metadata prediction models for predicting operational parameter values of backward reshaping mappings used to backward reshape SDR images into mapped HDR images.

Abstract: Dialogue enhancement of an audio signal, comprising obtaining a set of time-varying parameters configured to estimate a dialogue component present in said audio signal, estimating the dialogue component from the audio signal, applying a compressor only to the estimated dialogue component, to generate a processed dialogue component, applying a user-determined gain to the processed dialogue component, to provide an enhanced dialogue component. The processing of the estimated dialogue may be performed on the decoder side or encoder side. The invention enables an improved dialogue enhancement.

Abstract: A method and a device for encoding and decoding infra prediction are disclosed. An image decoding method for performing intra prediction comprises the steps of: receiving a bitstream including data on prediction modes of a current block and a block adjacent to the current block; extracting the data from the received bitstream so as to confirm the prediction mode of the adjacent block; determining whether a boundary pixel within the adjacent block can be used as a reference pixel for the current block in consideration of the prediction mode of the adjacent block; obtaining the reference pixel of the current block according to the determined result; generating a prediction block predicted in the frame on the basis of the obtained reference pixel; and decoding the current block by using the generated prediction block.

Abstract: The present invention relates to methods and apparatus for encoding an HOA signal representation (c(t)) of a sound field having an order of N and a number O=(N+1)2 of coefficient sequences to a mezzanine HOA signal representation (wMEZZ(t)). The present invention further relates to methods and apparatus for decoding a reconstructed HOA signal representation from the mezzanine HOA signal representation.

Abstract: Disclosed are methods and systems which convert a multi-microphone input signal to a multichannel output signal making use of a time- and frequency-varying matrix. For each time and frequency tile, the matrix is derived as a function of a dominant direction of arrival and a steering strength parameter. Likewise, the dominant direction and steering strength parameter are derived from characteristics of the multi-microphone signals, where those characteristics include values representative of the inter-channel amplitude and group-delay differences.

Abstract: The present invention relates to coding of audio signals, and in particular to high frequency reconstruction methods including a frequency domain harmonic transposer. A system and method for generating a high frequency component of a signal from a low frequency component of the signal is described.

Abstract: Decoding of Ambisonics representations for a stereo loudspeaker setup is known for first-order Ambisonics audio signals. But such first-order Ambisonics approaches have either high negative side lobes or poor localisation in the frontal region. The invention deals with the processing for stereo decoders for higher-order Ambisonics HOA.

Abstract: Methods for transmitting and receiving an audio stream are provided. For transmission, the method involves obtaining a frame of an audio signal, determining a number of source blocks to divide the frame of the audio signal into and a number of parity blocks to generate for forward error correction, wherein the number of source blocks and the number of parity blocks are determined based on characteristics of a wireless communication protocol to be used. The wireless communication protocol may be BLUETOOTH. The parity blocks are usable by a decoder to reconstruct one or more corrupted or missing source blocks and may be obtained by means of Reed-Solomon encoding.

Abstract: Some implementations involve receiving location information for each of a plurality of audio devices in an audio environment, generating, based at least in part on the location information, rendering information for a plurality of audio devices in an audio environment and determining, based at least on part on the rendering information, a plurality of echo reference metrics. Each echo reference metric may correspond to audio data reproduced by one or more audio devices of the plurality of audio devices. The rendering information may include a matrix of loudspeaker activations. Some examples involve making, based at least in part on the echo reference metrics, an importance estimation for each of a plurality of echo references, selecting, based at least in part on the importance estimation, one or more echo references and providing them to at least one echo management system for canceling or suppressing echoes.

Abstract: The present document relates to a method of layered encoding of a compressed sound representation of a sound or sound field. The compressed sound representation comprises a basic compressed sound representation comprising a plurality of components, basic side information for decoding the basic compressed sound representation to a basic reconstructed sound representation of the sound or sound field, and enhancement side information including parameters for improving the basic reconstructed sound representation.

Abstract: A projection system and calibration method therefore relate to a light source configured to emit a light in response to an image data, an optical system configured to project the light emitted by the light source; receiving an input associated with a plurality of light values corresponding to a plurality of primary lightfields; converting the input associated with the plurality of light values to a plurality of projector primary color values; determining a gain map based on the plurality Values of projector primary color values; applying the gain map to an image to perform a chromaticity uniformity correction by adjusting levels of the plurality of primary lightfields so that a primary mixture is the same over an image frame, and projecting the image with the optical system in the image frame, wherein the second image is corrected by the gain map.

Abstract: Methods and systems for improving coding efficiency of video.

Abstract: This disclosure falls into the field of audio coding, in particular it is related to the field of providing a framework for providing loudness consistency among differing audio output signals. In particular, the disclosure relates to methods, computer program products and apparatus for encoding and decoding of audio data bitstreams in order to attain a desired loudness level of an output audio signal.

Abstract: An apparatus and method of pre-conditioning audio for machine perception. Machine perception differs from human perception, and different processing parameters are used for machine perception applications (e.g., speech to text processing) as compared to those used for human perception applications (e.g., voice communications). These different parameters may result in pre-conditioned audio that is worsened for human perception yet improved for machine perception.

Abstract: A method for delivering media content to one or more clients over a distributed system is disclosed. The method may include generating a plurality of network-coded symbols from a plurality of original symbols representing a first media asset. The method may further include generating an original plurality of coded variants of the first media asset. The method may further include distributing a first coded variant of the original plurality of coded variants to a first cache on a first server device for storage in the first cache. The method may further include distributing a second coded variant of the original plurality of coded variants to a second cache on a second server device for storage in the second cache.

Abstract: The present disclosure provides methods, devices and computer program products for encoding and decoding a stereo audio signal based on an input signal. According to the disclosure, a hybrid approach of using both parametric stereo coding and a discrete representation of the stereo audio signal is used which may improve the quality of the encoded and decoded audio for certain bitrates.

Abstract: Sampled data is packaged in checkerboard format for encoding and decoding. The sampled data may be quincunx sampled multi-image video data (e.g., 3D video or a multi-program stream), and the data may also be divided into sub-images of each image which are then multiplexed, or interleaved, in frames of a video stream to be encoded and then decoded using a standardized video encoder. A system for viewing may utilize a standard video decoder and a formatting device that de-interleaves the decoded sub-images of each frame reformats the images for a display device. A 3D video may be encoded using a most advantageous interleaving format such that a preferred quality and compression ratio is reached. In one embodiment, the invention includes a display device that accepts data in multiple formats.

Abstract: Several embodiments of scalable image processing systems and methods are disclosed herein whereby color management processing of source image data to be displayed on a target display is changed according to varying levels of metadata.

Abstract: A method for representing a three-dimensional scene stored as a three-dimensional data set includes determining a set of P depth-plane depths along a viewing direction. The method includes generating, from a three-dimensional data set, a proxy three-dimensional data set including Pproxy images by, for each depth-plane depth: generating a proxy image of the Pproxy images from at least one cross-sectional image of a plurality of transverse cross-sectional images that (i) constitute the three-dimensional data set and (ii) each represent a respective transverse cross-section of the three-dimensional scene at a respective scene-depth.

Abstract: A method for tracking a reference picture on an electronic device is described. The method includes receiving a bitstream. The method also includes decoding a portion of the bitstream to produce a decoded reference picture. The method further includes tracking the decoded reference picture in a decoded picture buffer (DPB) with reduced overhead referencing. The method additionally includes decoding a picture based on the decoded reference picture.

Abstract: A system for mitigating over-suppression of speech and other non-noise signals is disclosed. In some embodiments, a system is programmed to train a first machine learning model for speech detection or enhancement using a non-linear, asymmetric loss function that penalizes speech over-suppression more than speech under-suppression. The first machine learning model is configured to receive an audio signal and generate a mask indicating an amount of speech present in the audio signal. The mask can be adjusted to remedy sharp voice decay resulting from speech over-suppression. The system is also programmed to train a second machine learning model for laughter or applause detection. The system is further programmed to improve the quality of a new audio signal by applying an adjusted mask to the new audio signal except for the portions of the audio signal that have been identified as corresponding to laughter or applause.

Abstract: Methods and systems for frame rate scalability are described. Support is provided for input and output video sequences with variable frame rate and variable shutter angle across scenes, or for input video sequences with fixed input frame rate and input shutter angle, but allowing a decoder to generate a video output at a different output frame rate and shutter angle than the corresponding input values. Techniques allowing a decoder to decode more computationally-efficiently a specific backward compatible target frame rate and shutter angle among those allowed are also presented.

Abstract: An encoding device evaluates a plurality of processing and/or post-processing algorithms and/or methods to be applied to a video stream, and signals a selected method, algorithm, class or category of methods/algorithms either in an encoded bitstream or as side information related to the encoded bitstream. A decoding device or post-processor utilizes the signaled algorithm or selects an algorithm/method based on the signaled method or algorithm. The selection is based, for example, on availability of the algorithm/method at the decoder/post-processor and/or cost of implementation. The video stream may comprise, for example, downsampled multiplexed stereoscopic images and the selected algorithm may include any of upconversion and/or error correction techniques that contribute to a restoration of the downsampled images.

Abstract: A method and apparatus for reconstructing N audio channels from M audio channels is disclosed. The method includes receiving a bitstream containing an encoded audio signal representing the M audio channels and decoding the encoded audio signal to obtain a frequency domain representation of the M audio channels. The method further includes extracting a parameter from the bitstream and reconstructing at least one of the N audio channels using the parameter. The parameter represents an angle between two signals, at least one of which is included in the M audio channels.

Abstract: Apparatuses and methods for data traffic management in multi-source content delivery are described. The apparatus includes a downloader and a controller. The downloader is coupled to servers via communication links. The controller is configured to determine initial download requests for the servers based on predetermined information about a quality of the links. The controller is also configured to send the initial download requests to the servers with the downloader. The controller is further configured to update the information about the quality of the communication links after the downloader receives data associated with a data file from the servers via the communication links. The controller is also configured to determine subsequent download requests for the servers based on the updated information about the quality of the communication links. The controller of further configured to send the subsequent download requests to the servers via the downloader.

Abstract: A projection system and method therefore related to a first projection device; a second projection device; at least one spatial modulator; and an electronic processor configured to: receive a two-dimensional video data, generate a first plurality of intensity values of a first color gamut and a second plurality of intensity values of a second color gamut, subtract a luminance threshold from a plurality of pixel values of the second color gamut to yield a plurality of positive pixel values and a plurality of negative pixel values, set respective ones of the plurality of pixel values to predetermined values to obtain a residual of the second color gamut, add the residual to the first color gamut, add maximized pixel values to the first color gamut and the second color gamut to create an output gamut, and blend the output gamut with a native color gamut to generate a blended gamut.

Abstract: Methods are described to communicate source color volume information in a coded bitstream using SEI messaging. Such data include at least the minimum, maximum, and average luminance values in the source data plus optional data that may include the color volume x and y chromaticity coordinates for the input color primaries (e.g., red, green, and blue) of the source data, and the color x and y chromaticity coordinates for the color primaries corresponding to the minimum, average, and maximum luminance values in the source data. Messaging data signaling an active region in each picture may also be included.

Abstract: A projection system and calibration method therefor relate to a light source configured to emit a light in response to an image data, an illumination optical system configured to steer the light, the illumination optical system including a first mirror and a second mirror, a digital micromirror device (DMD) including a plurality of micromirrors respectively configured to reflect the steered light to a filter as on-state light or to reflect the steered light as off-state light to a light dump; determining a deviation between an actual angle of orientation and an expected angle of orientation of the DMD; calculating a first amount of angle adjustment corresponding to the first mirror and a second amount of angle adjustment corresponding to the second mirror; and actuating the first mirror according to the first amount and the second mirror according to the second amount.

Abstract: Described herein is a method for Convolutional Neural Network (CNN) based speech source separation, wherein the method includes the steps of: (a) providing multiple frames of a time-frequency transform of an original noisy speech signal; (b) inputting the time-frequency transform of said multiple frames into an aggregated multi-scale CNN having a plurality of parallel convolution paths; (c) extracting and outputting, by each parallel convolution path, features from the input time-frequency transform of said multiple frames; (d) obtaining an aggregated output of the outputs of the parallel convolution paths; and (e) generating an output mask for extracting speech from the original noisy speech signal based on the aggregated output. Described herein are further an apparatus for CNN based speech source separation as well as a respective computer program product comprising a computer-readable storage medium with instructions adapted to carry out said method when executed by a device having processing capability.

Abstract: The application relates to HFR (High Frequency Reconstruction/Regeneration) of audio signals. In particular, the application relates to a method and system for performing HFR of audio signals having large variations in energy level across the low frequency range which is used to reconstruct the high frequencies of the audio signal. A system configured to generate a plurality of high frequency subband signals covering a high frequency interval from a plurality of low frequency subband signals is described.

Abstract: The application relates to HFR (High Frequency Reconstruction/Regeneration) of audio signals. In particular, the application relates to a method and system for performing HFR of audio signals having large variations in energy level across the low frequency range which is used to reconstruct the high frequencies of the audio signal. A system configured to generate a plurality of high frequency subband signals covering a high frequency interval from a plurality of low frequency subband signals is described.

Abstract: Methods and audio processing units for generating an object based audio program including conditional rendering metadata corresponding to at least one object channel of the program, where the conditional rendering metadata is indicative of at least one rendering constraint, based on playback speaker array configuration, which applies to each corresponding object channel, and methods for rendering audio content determined by such a program, including by rendering content of at least one audio channel of the program in a manner compliant with each applicable rendering constraint in response to at least some of the conditional rendering metadata. Rendering of a selected mix of content of the program may provide an immersive experience.

Abstract: Methods, systems, and bitstream syntax are described for determining the processing order of metadata messaging, such as supplemental enhancement information (SEI) messaging in MPEG video coding.

Abstract: An apparatus and method of rendering audio. A binaural signal is split on an amplitude weighting basis into a front binaural signal and a rear binaural signal, based on perceived position information of the audio. In this manner, the front-back differentiation of the binaural signal is improved.

Abstract: The disclosure relates to methods of processing a spatial audio signal for generating a compressed representation of the spatial audio signal. The methods include analyzing the spatial audio signal to determine directions of arrival for one or more audio elements; for at least one frequency subband, determining respective indications of signal power associated with the directions of arrival; generating metadata including direction information that includes indications of the directions of arrival of the audio elements, and energy information that includes respective indications of signal power; generating a channel-based audio signal with a predefined number of channels based on the spatial audio signal; and outputting, as the compressed representation, the channel-based audio signal and the metadata.

Abstract: In a method to improve backwards compatibility when decoding high-dynamic range images coded in a wide color gamut (WCG) space which may not be compatible with legacy color spaces, hue and/or saturation values of images in an image database are computed for both a legacy color space (say, YCbCr-gamma) and a preferred WCG color space (say, IPT-PQ). Based on a cost function, a reshaped color space is computed so that the distance between the hue values in the legacy color space and rotated hue values in the preferred color space is minimized. HDR images are coded in the reshaped color space. Legacy devices can still decode standard dynamic range images assuming they are coded in the legacy color space, while updated devices can use color reshaping information to decode HDR images in the preferred color space at full dynamic range.

Abstract: The application relates to HFR (High Frequency Reconstruction/Regeneration) of audio signals. In particular, the application relates to a method and system for performing HFR of audio signals having large variations in energy level across the low frequency range which is used to reconstruct the high frequencies of the audio signal. A system configured to generate a plurality of high frequency subband signals covering a high frequency interval from a plurality of low frequency subband signals is described.

Abstract: Described is a method of processing a media stream.

Abstract: A rendering mode may be determined for received audio data, including audio signals and associated spatial data. The audio data may be rendered for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals. Rendering the audio data may involve determining relative activation of a set of loudspeakers in an environment. The rendering mode may be variable between a reference spatial mode and one or more distributed spatial modes. The reference spatial mode may have an assumed listening position and orientation. In the distributed spatial mode(s), one or more elements of the audio data may each be rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data may be warped such that they span a rendering space of the environment more completely than in the reference spatial mode.

Abstract: Methods for encoding and decoding video data. One method includes receiving video data, the video data composed of a plurality of image frames, each image frame including a plurality of pixels. The method includes determining, for each image frame, a rotation matrix, determining, for each image frame, at least one of a scaling factor and an offset factor, and determining, for each image frame, a reshaping function based on one or more values of each of the plurality of pixels. The method includes generating an output image for each image frame by applying the rotation matrix, the reshaping function, and the at least one of the scaling factor and the offset factor to the respective image frame. The rotation matrix is applied to each pixel of the plurality of pixels for the respective image frame before the reshaping function is applied to the respective image frame.

Abstract: A method for representing a second presentation of audio channels or objects as a data stream, the method comprising the steps of: (a) providing a set of base signals, the base signals representing a first presentation of the audio channels or objects; (b) providing a set of transformation parameters, the transformation parameters intended to transform the first presentation into the second presentation; the transformation parameters further being specified for at least two frequency bands and including a set of multi-tap convolution matrix parameters for at least one of the frequency bands.