Everyone loves taking selfies! However, sometimes it is not easy to take satisfactory ones. It is well-known that the posing of the face significantly affects the perceived appearance. Lots of guidelines and tips could found on the internet that teach you how to strike a nice. For examples,
This video clip clearly demonstrates how pose can change the perceived appearances.
- Julie Chang (張齊郡), facebook ID: julie1989320
- 張香香, facebook ID: m40925
- Mika 黃杏蕙, facebook ID: model.mika17
The reasons for taking these models as examples are two-fold. First, all three of them are very popular on facebook in Taiwan (with hundreds of thousands followers), suggesting that many people really enjoy reading their facebook status updates (photo updates in particular). Second, they consistently upload selfies in a wide variety of environments, providing sufficient data samples (on the order of several thousands samples) for analysis.
Data Collection and Processing
To understand how people pose on facebook, we need to estimate their relative pose with respect to the camera.
- First, we use a free software for downloading publicly available photos in batch mode from their facebook profile pages.
- Second, we use the Viola-Jones face detector in OpenCV to localize faces with more than 40 x 40 pixels.
- Third, we use off-the-shelf face alignment method (IntraFace from CMU) to align the facial landmarks and estimate the 3D pose.
Note that the 3D poses are expressed in terms of Pitch, Yaw, and Roll, as illustrated below with an airplane.
Image source: Link
Roughly speaking, the "Pitch" indicate whether the nose is down (Pitch > 0) or up (Pitch < 0).
The "Yaw" indicate whether the left (Yaw > 0) or right (Yaw < 0) face is toward the camera. The "Roll" indicate the face tilt angle on the image plane.
Marginal Distributions for Pitch, Yaw, and Roll Angles
Given a set of samples images crawled from facebook, we estimate the face pose in each image. We can then visualize the one-dimensional marginal distributions of Pitch, Yaw, and Roll for each person (estimated using the Kernel Density Estimation). Please see the figures below (on the left-hand side).
What can we learn from these marginal distributions? First, all of the pitch angle marginal distribution have strong peaks around 15 degrees. This observation agrees with the conventional wisdom that one should pose their faces a bit downward so that the face would appear slimmer. Here we give a quantitative estimation of the optimal Pitch angle is probably 15 degree.
Second, we can see the Yaw angles in posing differ from person to person. In Julie Chang (張齊郡)'s case, her right faces are toward the cameras most of the time. In contrast, 張香香 often put her left faces toward the cameras. We did not see a clear preference in Mika 黃杏蕙's case, one guess may be that many of her photos are not selfies (e.g., modeling for clothes). Therefore, it's the photographers who decide the pose. In both left and right faces, it seems that Yaw angles around 20 degrees are probably good choices. One should probably do a bit experiments to figure out best sign for them (i.e., whether left or right face is more attractive).
Third, we could see that the Roll distributions are centered at zero degrees with different variances for each person. This may be the bias of using Viola-Jones face detector, which mainly detects frontal faces. However, we could still see some interesting facts. For example, Julie Chang (張齊郡) uses larger roll angles in her selfies than others.
Pitch-Yaw Joint Distributions
As marginal distributions may not capture the true variations in the data, we visualizing the joint distributions of Pitch and Yaw variables on the right-hand-side. We can see somewhat interesting patterns in the joint distributions, motivating us to a more detailed look of the data.
From the joint distributions of Pitch-Yaw angles, we see that the distributions do not take a simple parametric form such as a Gaussian distribution. We use the Mean-Shift algorithm to detect the "modes" in the Pitch-Yaw-Roll 3D distributions. Here we show the clustering results (coded in different colors) in the 3D space. Note that we throw away clusters with less than 3 samples.
Discovering Visual Subcategories
With the detected modes, we can then compute the average to visualize the "prototypes" in the large photo collections. We show 12 visual subcategories visualization for each person. This gives us a rough idea of how they strike nice poses from these examples. They also provide visualizations on the statistics we observation. For examples, we could see from the set of prototypes that Julie Chang (張齊郡) often shows her right faces and has larger variations in Roll angles. We could also see that 張香香 prefer her left faces over right faces.
Pose Changing Sequences
By organizing the poses (e.g., the Yaw angle) in decreasing or increasing order, we could create a photo sequence that the person appear to be slowly rotating their head, providing an interesting way to explore the personal photo collections.
The need of More Data
While the analysis of the selected three people's faces shows some interesting patterns, it is still far from being statistical significant. If you know other people who are good at taking selfies and have lots of photos available online, please let me know and I could try to include them as well. Thanks!
for all pairs of corresponding points (x' and x), where F is the 
