UM  > Faculty of Science and Technology
Affiliated with RCfalse
The Prediction of Saliency Map for Head and Eye Movements in 360 Degree Images
Yucheng Zhu1; Guangtao Zhai1; Xiongkuo Min1; Jiantao Zhou2,3
2019-12-12
Source PublicationIEEE Transactions on Multimedia
ISSN1520-9210
Volume22Issue:9Pages:2331-2344
Abstract

By recording the whole scene around the capturer, virtual reality (VR) techniques can provide viewers the sense of presence. To provide a satisfactory quality of experience, there should be at least 60 pixels per degree, so the resolution of panoramas should reach 21600 × 10800. The huge amount of data will put great demands on data processing and transmission. However, when exploring in the virtual environment, viewers only perceive the content in the current field of view (FOV). Therefore if we can predict the head and eye movements which are important behaviors of viewer, more processing resources can be allocated to the active FOV. But conventional saliency prediction methods are not fully adequate for panoramic images. In this paper, a new panorama-oriented model, to predict head and eye movements, is proposed. Due to the superiority of computation in the spherical domain, the spherical harmonics are employed to extract features at different frequency bands and orientations. Related low- and high-level features including the rare components in the frequency domain and color domain, the difference between center vision and peripheral vision, visual equilibrium, person and car detection, and equator bias are extracted to estimate the saliency. To predict head movements, visual mechanisms including visual uncertainty and equilibrium are incorporated, and the graphical model and functional representation for the switch of head orientation are established. Extensive experimental results on the publicly available database demonstrate the effectiveness of our methods.

Keyword360 Degree Center And Peripheral Vision Head-eye Motion Saliency Scanpath Spherical Harmonics Vr
DOI10.1109/TMM.2019.2957986
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaComputer Science ; Telecommunications
WOS SubjectComputer Science, Information Systems ; Computer Science, Software Engineering ; Telecommunications
WOS IDWOS:000562310200009
Scopus ID2-s2.0-85090134764
Fulltext Access
Citation statistics
Cited Times [WOS]:8   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
CollectionFaculty of Science and Technology
Affiliation1.Institute of Image Communication and Information Processing Shanghai Jiao Tong University, Shanghai 200240, China
2.State Key Laboratory of Internet of Things for Smart City
3.Department of Computer and Information Science, University of Macau, Macau 999078, China
Recommended Citation
GB/T 7714
Yucheng Zhu,Guangtao Zhai,Xiongkuo Min,et al. The Prediction of Saliency Map for Head and Eye Movements in 360 Degree Images[J]. IEEE Transactions on Multimedia,2019,22(9):2331-2344.
APA Yucheng Zhu,Guangtao Zhai,Xiongkuo Min,&Jiantao Zhou.(2019).The Prediction of Saliency Map for Head and Eye Movements in 360 Degree Images.IEEE Transactions on Multimedia,22(9),2331-2344.
MLA Yucheng Zhu,et al."The Prediction of Saliency Map for Head and Eye Movements in 360 Degree Images".IEEE Transactions on Multimedia 22.9(2019):2331-2344.
Files in This Item:
There are no files associated with this item.
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Yucheng Zhu]'s Articles
[Guangtao Zhai]'s Articles
[Xiongkuo Min]'s Articles
Baidu academic
Similar articles in Baidu academic
[Yucheng Zhu]'s Articles
[Guangtao Zhai]'s Articles
[Xiongkuo Min]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Yucheng Zhu]'s Articles
[Guangtao Zhai]'s Articles
[Xiongkuo Min]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.