UM  > Faculty of Science and Technology
Affiliated with RCfalse
Status已發表Published
IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention
Wu, Haiwei; Zhou, Jiantao
2022-03-01
Source PublicationIEEE Transactions on Circuits and Systems for Video Technology
ISSN1051-8215
Volume32Issue:3Pages:1172-1185
Abstract

Deep learning (DL) has demonstrated its powerful capabilities in the field of image inpainting, which could produce visually plausible results. Meanwhile, the malicious use of advanced image inpainting tools (e.g. removing key objects to report fake news, erasing visible copyright watermarks, etc.) has led to increasing threats to the reliability of image data. To fight against the inpainting forgeries (not only DL-based but also traditional ones), in this work, we propose a novel end-to-end Image Inpainting Detection Network (IID-Net), to detect the inpainted regions at pixel accuracy. The proposed IID-Net consists of three sub-blocks: the enhancement block, the extraction block and the decision block. Specifically, the enhancement block aims to enhance the inpainting traces by using hierarchically combined special layers. The extraction block, automatically designed by Neural Architecture Search (NAS) algorithm, is targeted to extract features for the actual inpainting detection tasks. To further optimize the extracted latent features, we integrate global and local attention modules in the decision block, where the global attention reduces the intra-class differences by measuring the similarity of global features, while the local attention strengthens the consistency of local features. Furthermore, we thoroughly study the generalizability of our IID-Net, and find that different training data could result in vastly different generalization capability. By carefully examining 10 popular inpainting methods, we identify that the IID-Net trained on only one specific deep inpainting method exhibits desirable generalizability; namely, the obtained IID-Net can accurately detect and localize inpainting manipulations for various unseen inpainting methods as well. Extensive experimental results are presented to validate the superiority of the proposed IID-Net, compared with the state-of-the-art competitors. Our results would suggest that common artifacts are shared across diverse image inpainting methods. Finally, we build a public inpainting dataset of 10K image pairs for future research in this area.

KeywordDeep Neural Networks Generalizability Inpainting Forensics
DOI10.1109/TCSVT.2021.3075039
URLView the original
Indexed BySCIE
Language英語English
WOS Research AreaEngineering
WOS SubjectEngineering, Electrical & Electronic
WOS IDWOS:000766700400023
Scopus ID2-s2.0-85104569937
Fulltext Access
Citation statistics
Cited Times [WOS]:3   [WOS Record]     [Related Records in WOS]
Document TypeJournal article
CollectionFaculty of Science and Technology
Corresponding AuthorZhou, Jiantao
AffiliationState Key Laboratory of Internet of Things for Smart City, University of Macau, Taipa, Macao
First Author AffilicationUniversity of Macau
Corresponding Author AffilicationUniversity of Macau
Recommended Citation
GB/T 7714
Wu, Haiwei,Zhou, Jiantao. IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention[J]. IEEE Transactions on Circuits and Systems for Video Technology,2022,32(3):1172-1185.
APA Wu, Haiwei,&Zhou, Jiantao.(2022).IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention.IEEE Transactions on Circuits and Systems for Video Technology,32(3),1172-1185.
MLA Wu, Haiwei,et al."IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention".IEEE Transactions on Circuits and Systems for Video Technology 32.3(2022):1172-1185.
Files in This Item: Download All
File Name/Size Publications Version Access License
IID-Net_Image_Inpain(6238KB)期刊论文作者接受稿开放获取CC BY-NC-SAView Download
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[Wu, Haiwei]'s Articles
[Zhou, Jiantao]'s Articles
Baidu academic
Similar articles in Baidu academic
[Wu, Haiwei]'s Articles
[Zhou, Jiantao]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Wu, Haiwei]'s Articles
[Zhou, Jiantao]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: IID-Net_Image_Inpainting_Detection_Network_via_Neural_Architecture_Search_and_Attention.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.