Slides of "Image Completion using Planar Structure Guidance" are online

Just came back from SIGGRAPH 2014 last week. It was such a fun event. Here I share the slide I used to present our work.

Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Johannes Kopf, Image Completion using Planar Structure Guidance ACM Transactions on Graphics (Proceedings of SIGGRAPH 2014), 33(4)

Code available on Github: https://github.com/jbhuang0604/StructCompletion


Below are samples comparison with existing works. Please visit our project webpage for more details.

  

  

  

  

  

  

  

  

    

   

   



    

Here are some cases where our method fails

  

   

Five Principles to Find Good Research Problems

I was attending SIGGRAGPH to present our paper on image completion. During the conference, I attended a great talk by Prof. Thomas Funkhouser from Princeton University. He talked about five approaches find good research problems. Even though we are from different fields, I think the main ideas are universal and could be applied to many other fields. I would like to share my notes from his lecture. 

1. Consider non-traditional problems

There are four types of research problems.
a) Improving existing methods

b) Develop new approaches to existing algorithms

c) Define new problems

d) Expand the field

It is easier for junior students to work on type (a) problems because many resources are available. However, try to work on b-d) to achieve the maximum impact.

2. Anticipate disruptive technologies
The research progress is not linear in time. Sometimes there will be disruptive technologies that completely change the game. Anticipate these disruptive technologies in the near future and develop groundwork ahead for the rest. For examples, fast graphics hardware, server-based networking, repositories of 3D models, consumer depth cameras, and internet.

3. Think beyond algorithms
Computer science communities are often obsessed with developing algorithms. However, other forms of work might be impactful as well. For example, analysis and evaluate of the problem, survey study, benchmark development, and dataset collection.

4. Work on real problems

Try to work on real problems. In solving real world problems, along the way we might also develop something useful for other problems in the field.

The computer scientist as toolsmith, Keynote from SIGGRAPH 1994
http://www.cs.unc.edu/~brooks/Toolsmith-CACM.pdf

5. Work with good collaborators

Talk, connect with people, and find good collaborators.

Image-based Height Enhancement

I came across an interesting app (Spring) last week that uses user-specified landmark positions to reshape the body proportions. Their results are quite fascinating. 

There are a few works on parametric modeling of human shapes that achieve awesome results, e.g.,

- Shizhe Zhou Hongbo Fu, Ligang Liu, Daniel Cohen-Or, and Xiaoguang Han, Parametric reshaping of human bodies in images, SIGGRAPH 2010 

- Arjun Jain Thorsten Thormählen Hans-Peter Seidel Christian Theobalt, MovieReshape: Tracking and Reshaping of Humans in Videos, SIGGRAPH Asia 2010

However, due to the use of continuous meshes for warping, artifacts are often obvious in the background if you need more dramatic results. 

To avoid such artifacts and limit the reshape to enhance the height, I implemented a simple variant for (semi-) automatic 1D body reshaping. The proportion could either be automatically determined or adjusted by the user through an 1D slider bar. Such automated approach allows us to process many images quickly. Here are some samples results downloaded from the Google image search. 

(Somehow I couldn't locate the sources from google image search. If you don't want your pictures to be shown here, please let me know and I will take it down immediately. Or, I could add them below the pictures.)

The original image is on the left. The reshaped version is on the right.

Source: Link

Source: Link

Source: Link

Source: Link

Btw, it also works for dogs! XD

Digital Image Forgery Detection with Image Registration

Blending between the two images help identify the differences. (It may take a few seconds to load the gif file.) You could see that Zhao Wei's left hand was left in the digitally manipulated image.

How did we do it? 

First step, download the following two images. The first one is the poster downloaded from the official webpage (with circular marks labeled by someone else), the other is the picture taken from this link. Presumably, they are taken at the same time at the same event.

Image source: link

Image source: link

Feature Detection

We detect Speeded Up Robust Features (SURF) features (shown in the green circle) from bot images. Note that we crop part of the first image to avoid spurious features for detection and matching. 

Feature matching and Image Registration

With the detected features from the two images, we can now estimate the geometric transformation (homography) between two images using the direct linear transform and RANSAC. We can thus align the images to help better show the modification.