Tuesday, June 3, 2014

Image Completion using Planar Structure Guidance

First row: input image (region to fill shown in red). Second row: our results. Input images (from left to right) are from ©Flickr users micromegasNicu BuculeiTheen Moy, used under the Creative Common license.

Filling or replacing regions in images with plausibly synthesized content is a common image editing technique. This task, known as image completion, is used in applications ranging from the removal of unwanted objects in personal photos to movie post-production. It is also an important step in many graphics algorithms, e.g., for generating a clean background plate or reshuffling image contents.

While much progress has been made, image completion remains a challenging problem. This is because some amount of higher level understanding of the scene is often required. The state-of-the-art automatic algorithms typically rely on low-level cues; they synthesize an image that locally appears like the known input everywhere.

In this paper (to be presented in SIGGRAPH 2014), we show how image completion can be substantially improved by automatically guiding the low-level synthesis algorithm using mid-level structural analysis of the known region. Our algorithm significantly improves performance for challenging man-made scenes such as those of architecture and indoors. In the absence of any detected structural cues, e.g., for most natural landscape images, our algorithm falls back to standard unconstrained completion, e.g., Photoshop content-aware fill.

Limitations of current state-of-the-art methods. 

Here we show a few examples to visualize the completion process in a coarse-to-fine fashion.

Image credit: ©Flickr user Nicu Buculei
Image credit: ©Flickr users aigle_dore
Image credit: ©Flickr user cedrennes 
Image credit: ©Flickr user the_dugghouse
Image credit: ©Flickr user sunshinepictures

However, the method still has difficulty on generating satisfactory results on quite a few challenging scenes. Among them, this building in London being demolished to make way for the construction of "The Cheesegrater" is particularly interesting to me. Here, I tried to recovery the deconstruction process. Yet, our algorithm is not able to find the correct demarcation lines between different perspective planes when the unknown region is large.

Image credit: ©Flickr user addictive_picasso

You can find more completion result comparisons below. Please also visit our project page for more information! :D

Here we show sample comparisons with prior works.




Failure cases


No comments :

Post a Comment