Thursday, April 25, 2013

Miss Daegu 2013 Contestants Face Morphing

Recently, there is a debate on plastic surgery from the post on Reddit titled "Korea's plastic surgery mayhem is finally converging on the same face. Here are the miss Korea 2013 contestants.

To better see the similarity among them, I first took the images from one Japanese blog. After simple normalization and registration, I can get an aligned, animated GIF looping through the 20 contestants as shown below.

                                       

These images remind me of a course project in computational photography I did two years ago. I wonder how the "averaged" face of these 20 contestants look like. Thus, I morph all the face images into a mean face shape, then average over the picture values using the code I developed in the course project. Here is the result.


Using the shape of the face images, I can generate a short movie of face morphing from contestant number 1 to 20. We can now see how one face is smoothly transformed into another.





Another application is to synthesize faces with new shapes. For example, I can morph each face a bit toward the mean shape. This results in a stabilized version of animated GIF looping from contestant 1 to 20 as shown below. 





So far, the above visualization figures are qualitative, we may be able to get more insights by quantitative analysis. 


First, we construct the eigenspaces from these contestants faces. However, the standard Principle Component Analysis cannot be directly applied due to the slight pose variation and occlusion by the hair. Instead, we perform a robust version of PCA, e.g., [Robust PCALow-Rank Matrix Recovery], to factor the low rank part and the sparse errors. Specifically, I use the implementation provided in [Low-Rank Matrix Recovery].
[The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, Z. Lin, M. Chen, L. Wu, and Y. Ma (UIUC Technical Report UILU-ENG-09-2215, November 2009)]
Here are three sample results.



Second, by performing singular value decomposition, we get the eigenfaces and the corresponding eigenvalues. Here we can see that the eigenvalues vanish after 7, suggesting that the rank of the image data is 6.

Here we visualize the largest 6 eigenfaces. These eigenfaces encodes the variation within the 20 contestants.




Third, by projecting faces onto the eigenspace, we can analyze how these faces are distributed in the eigenspace. We show here the coefficients of the 20 contestants. Note that most of the energies are concentrated on first two eigenvalues.
Thus, we plot the coefficients corresponding to the first two eigenvectors. We can now see how similar the appearance of these contestants are.



 Fourth, we can try to construct the affinity matrix which encodes the pairwise similarity. (blue indicates similar and red indicates dissimilar).


This affinity matrix facilitates many interesting applications. For example, by summing up over columns or rows, we get a measure of how a contestant is different from the rest. In the plot below, x-axis denotes the contestant index (from 1- 20). The y-axis indicates how distinct the contestant is. This measure can also be interpreted as the salient object among the 20 contestants using the surround center information divergence.
(If you are interested in saliency, please see our paper on ICPR or slide.)

From the plot above, we can see that contestant 1, 2 and 6 are more distinct (or more salient) than the rest of the contestants. On the other hand, contestant 7, 12, and 15 are more common (i.e., closed to the averaged face) than others. 

Salient contestants (1, 2, 6)



It is interesting that one actually used these most salient contestants (1, 2, 6) to support their opinions on how they are dissimilar [link]. (I am not sure whether they did the same eigen analysis or not.)

Common contestants (7, 12, 15)

Here are the pictures of all contestants.


Another application of using the affinity matrix is to come up with a displaying order with maximized or minimized similarity from one to another. You may use one of them to support your opinion on whether they are similar or not.

Consider a complete graph of 20 vertices with edges encode the pairwise similarity, the problem of finding a displaying order is the problem of finding a (weighted) Hamiltonian circuit in an undirected graph [see Hamiltonian path problem]. As the problem is known to be NP-hard, we could use a greedy heuristic to find approximated solutions. 

The heuristic algorithm works as follows. Start from a random contestant, choose the next contestant (that has not been selected before) with maximum or minimum similarity with the previous contestant. Repeat the process until we find the ordering.


Here is the result of ordering with maximized pairwise similarity. Note that the transition between contestants are more smooth.



Here is the result of minimized pairwise similarity. They look more dissimilar in this displaying order. 



Resources: 
  • If you would like to try it out on your own, you can download my code (in Matlab) here (12.7 MB)
  • You can also just download the cropped, aligned, and mean faces here (12.6 MB)
Have fun! :D



Further readings:



  • Average faces
    • Face of tomorrow project by Mike Mike. [link]
    • Face Research has a online demo for you to make an average face [link]
  • Face morphing [wiki]
  • Principle component analysis
  • Low-Rank Matrix Recovery and Completion via Convex Optimization [link]
  • Saliency
    • Salience (neuroscience) [wiki]
  • Hamiltonian path/circuit [wiki]

Updates (May 4, 2013)

1. The title of the original post was wrong. They are in fact contestants for Miss Daegu. I had corrected the title. Sorry for the confusion.

2. The poll results are out and is available on the official website


The top candidates are contestant number 5, 1, 15, 20, and 19.  We may be able to "guess" how the quantitative analysis of similarity is related to the result.

The first thing I observed is that one of the most salient (No. 1) and most common contestants (No. 15) won the 2nd and the 3rd places, respectively. This result seems to coincide with the psychological study on averaged faces. 

Thomas R. Alley and Michael R. Cunningham, Psychological Science, 1991

We then plot the coefficients corresponding to the first two eigenvectors for top 5 contestants (marked in red). It seems that the selection process somehow avoided choosing both candidates with similar appearances. 

Update (July 20, 2013)

One of the most salient contestant (number one), Yoo Ye-bin, crowned the Miss Korea 2013.

95 comments :

  1. Cool! Thanks for sharing interesting work.

    ReplyDelete
  2. Nice and fun project. Cheers.

    ReplyDelete
  3. Nice,
    However, I'd have been interested with a study of the difference with a group on "normal" (i.e. not models) Korean girls.

    ReplyDelete
  4. the candidates for this test should be proven that they underwent surgical procedure so we can see the "convergence" of their faces after surgery. also, sample pictures should have been makeup-less since they tend to use the same style of makeup and therefore be more similar especially in the eyes part.

    interesting, though.

    ReplyDelete
    Replies
    1. I agree. This post is very far from a serious study on the effect of plastic surgery. If more sample faces (under controlled lighting, poses, and w/wo makeup) are available, then we may be able to see more interesting results.

      Delete
  5. WOW this post is so coooooool!! Thought computer vision is not my major, I cannot understand it fully, I am considering whether it's possible to add a section to compare the average of the variance between the "normal people" and that between the "20 contestants." I believe we can see more interesting and inspiring results XD

    ReplyDelete
    Replies
    1. Yes, that will be an interesting comparison. Mike Mike had a project which computes average faces for people in different cities.

      http://www.businessinsider.com/faces-of-tomorrow-2011-2#altogether-now-23

      You can see the mean face of Korean near the top-right corner.

      Delete
    2. i think your talking about ANOVA (Analysis of statistical variance)

      Delete
    3. Although the comparison is interesting, it would only tell you about korean preference or what is considered beautiful in korean society, not anything about plastic surgery necessarily.

      Delete
  6. Makeups do make these pageants contestants look very similar rather plastic surgery. Here is the link of them before and after makeups:
    http://big5.xinhuanet.com/gate/big5/news.xinhuanet.com/photo/2013-04/28/m_124642965.htm

    ReplyDelete
  7. That explain my problem watching korean movies: I can't differentiate one girl from another. Men I have no problems; women look all the same to me.

    BTW, american porn have the same problem: all women have the same long hair, eyes makeup and overall face shape. It's the same face repeated thousand times.

    ReplyDelete
  8. It's not a real picture. They used Photoshop.

    http://pantip.com/topic/30421514

    ReplyDelete
  9. You want to rewrite the code in R for me? :)

    ReplyDelete
    Replies
    1. wow what a lazy ass! :)

      Delete
    2. http://stackoverflow.com/questions/5527145/convert-matlab-code-to-r

      Delete
  10. Replies
    1. science dude! fucking science.

      Delete
    2. I'm in Korea on business now haha... I keep falling in love

      Delete
  11. So, I have a question. Did one 7, 12, or 15 end up winning?

    ReplyDelete
    Replies
    1. my theory would be that, since they are all clearly trying to achieve an ideal, that the "most similar" of them should have the best chances, i.e. contestants 7, 12 and 15

      Delete
    2. The poll results are out and available on the official website
      http://misskorea.mpluskorea.com/missdaegu2013_poll

      Seems like both typical and atypical faces are popular.

      Delete
  12. 可以用它來作判定美醜的基準嗎,例如,

    1.與範本的近似權值,越接近 "理想值" 就越美...
    2.臉型眼眉鼻口相對位置與比例...整容業者會很想知道呢
    3.弄個 App 把照片 "自動美容"...應該會大受歡迎
    4.調整某些參數...拿來製作 美女臉蛋 的 3D 模型 ...

    ReplyDelete
    Replies
    1. There was a work on enhancing facial attractiveness using data-driven approach. This might be close to what you have in mind.

      http://www.youtube.com/watch?v=lVbrUuwK-8g

      Delete
  13. 可以用它來作判定美醜的基準嗎,例如,

    1.與範本的近似權值,越接近 "理想值" 就越美...
    2.臉型眼眉鼻口相對位置與比例...整容業者會很想知道呢
    3.弄個 App 把照片 "自動美容"...應該會大受歡迎
    4.調整某些參數...拿來製作 美女臉蛋 的 3D 模型 ...

    Agree!

    ReplyDelete
  14. Thus, we have mathematically proven that they arr rook same.

    ReplyDelete
    Replies
    1. Actually, I didn't prove anything in the post. We could not draw any conclusion from such limited samples.

      Delete
  15. Amazing. Nice to see practical applications of scientific analysis.

    ReplyDelete
  16. Very interesting and nice job!
    I am wondering whether you can derive how unusual
    their dis-similarity distribution is in a quantitive way?
    such as, the standard deviation of their distribution is much smaller than a random sample!

    ReplyDelete
    Replies
    1. Thanks! Yes, if we have some sample images of normal people in Korea, then, we might be able to see some interesting results. However, it may be hard to draw meaningful conclusion as we didn't control the shooting environment, e.g., lighting, pose, tone mapping.

      Delete
  17. Cool! You can even find out which surgery clinic is shared by different contestants.

    ReplyDelete
  18. 你好,请问一下对齐的脸是自己处理的吗?还是你拿到的数据就是脸部(尤其是双眼)已经对齐的了呢?如果是您自己处理的对齐,能否说下使用的方法吗?
    我从网上找了这些照片,用了一些方法检测到每张照片中双眼的位置,但是每个人双眼距离不同,所以双眼无法对齐,平均脸就看着不像一张脸了,非常想知道对齐方法,谢谢您!

    ReplyDelete
    Replies
    1. Hi Kevin,

      Nope, the raw data is not aligned.
      To align the images using eye positions.
      1. I manually labeled the eye positions for each faces.
      2. Compute mean locations of these eye positions.
      3. Compute similarity transformation from the eye positions in one image to the averaged position.
      4. Wrap each image according to the similarity transformation computed in (3)

      To compute the averaged face, I did something more. I first compute the mean shape. Then, wrap each image to the mean shape. Lastly, perform pixel-wise average.

      You may take a look of my code to see exactly how these steps are performed.

      Delete
  19. Yes, they have very talented hair and makeup people, but more importantly, which one had the best ballet technique? Which one played the concert harp? Which one wrote the best essay on the future of Korean feminism? Which one is an anthropology major and talented violinist? Which one is most likely to re-write your code?

    ReplyDelete
  20. Hi! This is really well done and smart working..I'm impressed much.
    Thanks you for sharing us.It's pleasure to follow you.
    Lots of loves from Turkey.

    ReplyDelete
  21. This comment has been removed by a blog administrator.

    ReplyDelete
  22. Howdy! This is my 1st comment here so I just wanted to give a quick shout out and say I truly enjoy reading your posts. Thanks a lot!beauty surgery

    ReplyDelete
  23. This comment has been removed by the author.

    ReplyDelete
  24. Great work I enjoyed this post! I'm going to download and try the code now just for fun :) Thanks for sharing, really! Looking forward to more posts :)

    ReplyDelete
  25. Remarkable blog! I have no words to praise, it has really allured me.PerfectNose UK

    ReplyDelete
  26. This is an interesting blog on redefining open mind and I have learned a lot from it. In deed the modeling profession has greatly transformed and the participants are not only beauty queens but also intellectuals. Thanks for sharing this article with us and in case you need writing services you can hire our services by clicking on Paper Redoing Assistance.

    ReplyDelete
  27. Interesting this products are really good thanks for sharing information

    ReplyDelete
  28. Thanks for the great post on your blog, it really gives me an insight on this topic.

    korea company registration

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  29. Your texts on this subject are correct, see how I wrote this site is really very good. carbon theory charcoal soap

    ReplyDelete
  30. I'm extremely impressed along with your writing skills and also with the format in your weblog. Is this a paid topic or did you modify it your self? Anyway keep up the excellent quality writing, it's rare to see a nice blog like this one nowadays..

    ReplyDelete
  31. Greate pieces. Keep posting such kind of info on your site. Im really impressed by your site.
    Hey there, You have performed a fantastic job. I will certainly digg it and personally recommend to my friends. I am sure they will be benefited from this site.

    ReplyDelete
  32. Hey! I'm at work surfing around your blog from my new iphone 4! Just wanted to say I love reading your blog and look forward to all your posts! Carry on the superb work!

    ReplyDelete
  33. Trekking in Nepal with Alpine Ramble is one of the most fascinating activities that takes you to the most iconic part of the Himalayan land. Langtang Valley Trek , Ruby valley homestay Trek ,
    Ruby valley homestay Trek

    ReplyDelete
  34. Thanks for sharing such informative article. Doctor of Nursing Practice (DNP) project essentially examines how well one understands the field of nursing practice and one’s ability to solve a real world public health problem. Owing to the fact that working on this type of project can be somehow difficult, some scholars opt to order writing help from our DNP Capstone Project Writers.

    ReplyDelete
  35. Detailed Finding. You must have done a great research before.

    ReplyDelete
  36. Hiring a personal Local Trekking Guide/porters or porter cum Guide for Langtang valley trek, Tamang Heritage Home stay trek, Langtang Gosaikunda Pass trek, Gosaikunda lake trek, Helambu trek, Chisopani Helambu trek and Ganja la high pass trek.
    https://www.himalayanlocalguide.com/local-trekking-guide-service-for-langtang-valley-trek

    ReplyDelete
  37. https://www.vertexholiday.com/trips/ghorepani-poon-hill-trek

    ReplyDelete
  38. The trek to Everest Base camp involves 2 weeks of walking at high altitude. With some physical training and a proper acclimatization schedule it is something that most reasonably fit people can do. https://www.summitclimb.com/trek/everest-base-camp-trek

    ReplyDelete
  39. Thank you for sharing this informative information. If you are also willing to know about Everest Base Camp Trek or Annapurna Base Camp Trek Manaslu Circuit Trek , let us know.

    ReplyDelete
  40. I hope you will post more like that in the future. If anyone interested in Mountaineering then check it out.
    Everest Base Camp Trek
    Everest Nepal
    Aconcagua
    Ama Dablam
    Manaslu
    Lhotse

    ReplyDelete
  41. You made such an interesting piece to read, I really appreciate it. Thanks for sharing useful information with us. I am looking forward to more from you. If want to know about Cosmetic Surgery In Korea so you can visit them.

    ReplyDelete
  42. https://www.vertexholiday.com/blogs/top-10-best-trekking-trails-in-nepal

    ReplyDelete
  43. The information you have provided is very helpful at all thank you very much for sharing useful information with us.
    Appreciated by Excellent Himalaya. Excellent Himalaya Trek and Expedition is the leading trekking tour operator of Nepal. We specialize in Everest Base Camp Trek, Annapurna Base Camp Trek, Langtang Trek, Annapurna Circuit Trek, Mardi Himal Trek etc. Trekking Tours are core activities that Excellent Himalaya Trek & Expedition do. On top of them we organize peak climbing, mountain expeditions, helicopter tours, mountain biking tours, pilgrimage tours and various adventure trips such as Trishuli Rafting, canoeing, Paragliding in Pokhara, Bungee Jump in Pokhara, Ultralight Flight Pokhara and Chitwan Jungle Safari.

    ReplyDelete
  44. 안성초콜릿
    부천초콜릿
    남양주초콜릿
    포천초콜릿
    수원초콜릿
    성남초콜릿
    안산초콜릿Its always good to have tips on good blog posting. As I just started posting comments for blog and faced a lot of rejections. I think your suggestion would be helpful for me. I will let you know if this works for me.

    ReplyDelete
  45. This comment has been removed by the author.

    ReplyDelete
  46. This comment has been removed by the author.

    ReplyDelete
  47. Everest Trek, also known as the Everest Base Camp Trek, is a popular and iconic trekking adventure in the Everest region of Nepal. It takes you through breathtaking landscapes, remote Sherpa villages, and challengingmountain trails,ultimately leading to the base camp of Mount Everest—the highest peak in the world.

    ReplyDelete
  48. https://www.teamhimalaya.com/

    ReplyDelete
  49. I appreciate you providing this important information to us; the information you have given is helpful.
    From Team Himalaya, with gratitude. The trekking trip operator of choice in Nepal is Team Himalaya Trek and Expedition. Our areas of expertise are treks to Everest Base Camp, Annapurna Base Camp, Langtang, Annapurna Circuit, Mardi Himal, and other similar destinations. Team Himalaya Trek & Expedition's main business is providing trekking tours. Above all, we arrange peak climbing, mountain expeditions, mountain biking, helicopter, and pilgrimage tours, in addition to a variety of adventure experiences like canoeing, ultralight flying in Pokhara, paragliding, mountain biking, peak climbing, and Chitwan jungle safari.

    ReplyDelete