-
Super-rays grouping scheme and novel coding architecture for computational time reduction of graph-based Light Field coding
Authors:
Bach Nguyen Gia,
Chanh Minh Tran,
Tho Nguyen Duc,
Tan Phan Xuan,
Eiji Kamioka
Abstract:
Graph-based Light Field coding using the concept of super-rays is powerful to exploit signal redundancy along irregular shapes and achieves good energy compaction, compared to rectangular block -based approaches. However, its main limitation lies in the high time complexity for eigen-decomposition of each super-ray local graph, a high number of which can be found in a Light Field when segmented in…
▽ More
Graph-based Light Field coding using the concept of super-rays is powerful to exploit signal redundancy along irregular shapes and achieves good energy compaction, compared to rectangular block -based approaches. However, its main limitation lies in the high time complexity for eigen-decomposition of each super-ray local graph, a high number of which can be found in a Light Field when segmented into super-rays. This paper examines a grouping scheme for super-rays in order to reduce the number of eigen-decomposition times, and proposes a novel coding architecture to handle the signal residual data arising for each super-ray group, as a tradeoff to achieve lower computational time. Experimental results have shown to reduce a considerable amount of decoding time for Light Field scenes, despite having a slight increase in the coding bitrates when compared with the original non-grouping super-ray -based approach. The proposal also remains to have competitive performance in Rate Distortion in comparison to HEVC-based and JPEG Pleno -based methods.
△ Less
Submitted 10 December, 2023;
originally announced December 2023.
-
Novel projection schemes for graph-based Light Field coding
Authors:
Bach Gia Nguyen,
Chanh Minh Tran,
Tho Nguyen Duc,
Tan Xuan Phan,
Kamioka Eiji
Abstract:
In Light Field compression, graph-based coding is powerful to exploit signal redundancy along irregular shapes and obtains good energy compaction. However, apart from high time complexity to process high dimensional graphs, their graph construction method is highly sensitive to the accuracy of disparity information between viewpoints. In real world Light Field or synthetic Light Field generated by…
▽ More
In Light Field compression, graph-based coding is powerful to exploit signal redundancy along irregular shapes and obtains good energy compaction. However, apart from high time complexity to process high dimensional graphs, their graph construction method is highly sensitive to the accuracy of disparity information between viewpoints. In real world Light Field or synthetic Light Field generated by computer software, the use of disparity information for super-rays projection might suffer from inaccuracy due to vignetting effect and large disparity between views in the two types of Light Fields respectively. This paper introduces two novel projection schemes resulting in less error in disparity information, in which one projection scheme can also significantly reduce time computation for both encoder and decoder. Experimental results show projection quality of super-pixels across views can be considerably enhanced using the proposals, along with rate-distortion performance when compared against original projection scheme and HEVC-based or JPEG Pleno-based coding approaches.
△ Less
Submitted 9 June, 2022;
originally announced June 2022.
-
Subtitle-based Viewport Prediction for 360-degree Virtual Tourism Video
Authors:
Chuanzhe Jing,
Tho Nguyen Duc,
Phan Xuan Tan,
Eiji Kamioka
Abstract:
360-degree streaming videos can provide a rich immersive experiences to the users. However, it requires an extremely high bandwidth network. One of the common solutions for saving bandwidth consumption is to stream only a portion of video covered by the user's viewport. To do that, the user's viewpoint prediction is indispensable. In existing viewport prediction methods, they mainly concentrate on…
▽ More
360-degree streaming videos can provide a rich immersive experiences to the users. However, it requires an extremely high bandwidth network. One of the common solutions for saving bandwidth consumption is to stream only a portion of video covered by the user's viewport. To do that, the user's viewpoint prediction is indispensable. In existing viewport prediction methods, they mainly concentrate on the user's head movement trajectory and video saliency. None of them consider navigation information contained in the video, which can turn the attention of the user to specific regions in the video with high probability. Such information can be included in video subtitles, especially the one in 360-degree virtual tourism videos. This fact reveals the potential contribution of video subtitles to viewport prediction. Therefore, in this paper, a subtitle-based viewport prediction model for 360-degree virtual tourism videos is proposed. This model leverages the navigation information in the video subtitles in addition to head movement trajectory and video saliency, to improve the prediction accuracy. The experimental results demonstrate that the proposed model outperforms baseline methods which only use head movement trajectory and video saliency for viewport prediction.
△ Less
Submitted 6 June, 2022;
originally announced June 2022.
-
Continuous QoE Prediction Based on WaveNet
Authors:
Phan Xuan Tan,
Tho Nguyen Duc,
Chanh Minh Tran,
Eiji Kamioka
Abstract:
Continuous QoE prediction is crucial in the purpose of maximizing viewer satisfaction, by which video service providers could improve the revenue. Continuously predicting QoE is challenging since it requires QoE models that are capable of capturing the complex dependencies among QoE influence factors. The existing approaches that utilize Long-Short-Term-Memory (LSTM) network successfully model suc…
▽ More
Continuous QoE prediction is crucial in the purpose of maximizing viewer satisfaction, by which video service providers could improve the revenue. Continuously predicting QoE is challenging since it requires QoE models that are capable of capturing the complex dependencies among QoE influence factors. The existing approaches that utilize Long-Short-Term-Memory (LSTM) network successfully model such long-term dependencies, providing the superior QoE prediction performance. However, the inherent drawback of sequential computing of LSTM will result in high computational cost in training and prediction tasks. Recently, WaveNet, a deep neural network for generating raw audio waveform, has been introduced. Immediately, it gains a great attention since it successfully leverages the characteristic of parallel computing of causal convolution and dilated convolution to deal with time-series data (e.g., audio signal). Being inspired by the success of WaveNet, in this paper, we propose WaveNet-based QoE model for continuous QoE prediction in video streaming services. The model is trained and tested upon on two publicly available databases, namely, LFOVIA Video QoE and LIVE Mobile Stall Video II. The experimental results demonstrate that the proposed model outperforms the baselines models in terms of processing time, while maintaining sufficient accuracy.
△ Less
Submitted 20 March, 2020;
originally announced March 2020.
-
FAURAS: A Proxy-based Framework for Ensuring the Fairness of Adaptive Video Streaming over HTTP/2 Server Push
Authors:
Chanh Minh Tran,
Tho Nguyen Duc,
Phan Xuan Tan,
Eiji Kamioka
Abstract:
HTTP/2 video streaming has caught a lot of attentions in the development of multimedia technologies over the last few years. In HTTP/2, the server push mechanism allows the server to deliver more video segments to the client within a single request in order to deal with the requests explosion problem. As a result, recent research efforts have been focusing on utilizing such a feature to enhance th…
▽ More
HTTP/2 video streaming has caught a lot of attentions in the development of multimedia technologies over the last few years. In HTTP/2, the server push mechanism allows the server to deliver more video segments to the client within a single request in order to deal with the requests explosion problem. As a result, recent research efforts have been focusing on utilizing such a feature to enhance the streaming experience while reducing the request-related overhead. However, current works only optimize the performance of a single client, without necessary concerns of possible influences on other clients in the same network. When multiple streaming clients compete for a shared bandwidth in HTTP/1.1, they are likely to suffer from unfairness, which is defined as the inequality in their bitrate selections. For HTTP/1.1, existing works have proven that the network-assisted solutions are effective in solving the unfairness problem. However, the feasibility of utilizing such an approach for the HTTP/2 server push has not been investigated. Therefore, in this paper, a novel proxy-based framework is proposed to overcome the unfairness problem in adaptive streaming over HTTP/2 with the server push. Experimental results confirm the outperformance of the proposed framework in ensuring the fairness, assisting the clients to avoid rebuffering events and lower bitrate degradation amplitude, while maintaining the mechanism of the server push feature.
△ Less
Submitted 19 March, 2020;
originally announced March 2020.
-
Convolutional Neural Networks for Continuous QoE Prediction in Video Streaming Services
Authors:
Tho Nguyen Duc,
Chanh Minh Tran,
Phan Xuan Tan,
Eiji Kamioka
Abstract:
In video streaming services, predicting the continuous user's quality of experience (QoE) plays a crucial role in delivering high quality streaming contents to the user. However, the complexity caused by the temporal dependencies in QoE data and the non-linear relationships among QoE influence factors has introduced challenges to continuous QoE prediction. To deal with that, existing studies have…
▽ More
In video streaming services, predicting the continuous user's quality of experience (QoE) plays a crucial role in delivering high quality streaming contents to the user. However, the complexity caused by the temporal dependencies in QoE data and the non-linear relationships among QoE influence factors has introduced challenges to continuous QoE prediction. To deal with that, existing studies have utilized the Long Short-Term Memory model (LSTM) to effectively capture such complex dependencies, resulting in excellent QoE prediction accuracy. However, the high computational complexity of LSTM, caused by the sequential processing characteristic in its architecture, raises a serious question about its performance on devices with limited computational power. Meanwhile, Temporal Convolutional Network (TCN), a variation of convolutional neural networks, has recently been proposed for sequence modeling tasks (e.g., speech enhancement), providing a superior prediction performance over baseline methods including LSTM in terms of prediction accuracy and computational complexity. Being inspired of that, in this paper, an improved TCN-based model, namely CNN-QoE, is proposed for continuously predicting the QoE, which poses characteristics of sequential data. The proposed model leverages the advantages of TCN to overcome the computational complexity drawbacks of LSTM-based QoE models, while at the same time introducing the improvements to its architecture to improve QoE prediction accuracy. Based on a comprehensive evaluation, we demonstrate that the proposed CNN-QoE model can reach the state-of-the-art performance on both personal computers and mobile devices, outperforming the existing approaches.
△ Less
Submitted 19 March, 2020;
originally announced March 2020.