UM  > Faculty of Science and Technology
Affiliated with RCfalse
Learning a Deep Agent to Predict Head Movement in 360-Degree Images
YUCHENG ZHU1; GUANGTAO ZHAI1; XIONGKUO MIN1; JIANTAO ZHOU2
2020-12-16
Source PublicationACM Transactions on Multimedia Computing, Communications and Applications
ISSN1551-6857
Volume16Issue:4
Other Abstract

Virtual reality adequately stimulates senses to trick users into accepting the virtual environment. To create a sense of immersion, high-resolution images are required to satisfy human visual system, and low latency is essential for smooth operations, which put great demands on data processing and transmission. Actually, when exploring in the virtual environment, viewers only perceive the content in the current field of view. Therefore, if we can predict the head movements that are important behaviors of viewers, more processing resources can be allocated to the active field of view. In this article, we propose a model to predict the trajectory of head movement. Deep reinforcement learning is employed to mimic the decision making. In our framework, to characterize each state, features for viewport images are extracted by convolutional neural networks. In addition, the spherical coordinate maps and visited maps are generated for each viewport image, which facilitate the multiple dimensions of the state information by considering the impact of historical head movement and position information. To ensure the accurate simulation of visual behaviors during the watching of panoramas, we stipulate that the model imitates the behaviors of human demonstrators. To allow the model to generalize to more conditions, the intrinsic motivation is employed to guide the agent's action toward reducing uncertainty, which can enhance robustness during the exploration. The experimental results demonstrate the effectiveness of the proposed stepwise head movement predictor.

Keyword360 Degree Deep Reinforcement Learning (Drl) Head Movement Prediction Omnidirectional Panoramic Saliency Vr
DOI10.1145/3410455
URLView the original
Indexed BySCIE ; SSCI
Language英語English
WOS Research AreaComputer Science
WOS SubjectComputer Science, Information Systems ; Computer Science, Software Engineering ; Computer Science, Theory & Methods
WOS IDWOS:000614088800014
Scopus ID2-s2.0-85100305639
Fulltext Access
Citation statistics
Cited Times [WOS]:0   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
CollectionFaculty of Science and Technology
Corresponding AuthorYUCHENG ZHU
Affiliation1.Institute of Image Communication and Information Processing, Shanghai Jiao Tong University, Shanghai, China
2.Department of Computer and Information Science, University of Macau, Macau, China
Recommended Citation
GB/T 7714
YUCHENG ZHU,GUANGTAO ZHAI,XIONGKUO MIN,et al. Learning a Deep Agent to Predict Head Movement in 360-Degree Images[J]. ACM Transactions on Multimedia Computing, Communications and Applications,2020,16(4).
APA YUCHENG ZHU,GUANGTAO ZHAI,XIONGKUO MIN,&JIANTAO ZHOU.(2020).Learning a Deep Agent to Predict Head Movement in 360-Degree Images.ACM Transactions on Multimedia Computing, Communications and Applications,16(4).
MLA YUCHENG ZHU,et al."Learning a Deep Agent to Predict Head Movement in 360-Degree Images".ACM Transactions on Multimedia Computing, Communications and Applications 16.4(2020).
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[YUCHENG ZHU]'s Articles
[GUANGTAO ZHAI]'s Articles
[XIONGKUO MIN]'s Articles
Baidu academic
Similar articles in Baidu academic
[YUCHENG ZHU]'s Articles
[GUANGTAO ZHAI]'s Articles
[XIONGKUO MIN]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[YUCHENG ZHU]'s Articles
[GUANGTAO ZHAI]'s Articles
[XIONGKUO MIN]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.