Pose and Illumination Invariant Face Recognition

Home People Contact Research Publications Teaching Jobs

Face Identification by Fitting a 3D Morphable Model
using Linear Shape and Texture Error Functions

Sami Romdhani, Volker Blanz and Thomas Vetter

Adobe Portable Document Format (pdf) [800k]

Compressed Postscript (ps.gz) [561k]

Powerpoint Presentation (.ppt) [1.6M]

Abstract:
This paper presents a novel algorithm aiming at analysis and identification of faces viewed from different poses and illumination conditions. Face analysis from a single image is performed by recovering the shape and textures parameters of a 3D Morphable Model in an analysis-by-synthesis fashion. The shape parameters are computed from a shape error estimated by optical flow and the texture parameters are obtained from a texture error. The algorithm uses linear equations to recover the shape and texture parameters irrespective of pose and lighting conditions of the face image. Identification experiments are reported on more than 5000 images from the publicly available CMU-PIE face image database which includes faces viewed from 13 different poses and under 22 different illuminations. Extensive identification results are available for future comparison with novel algorithms.

Identification of a face viewed from any pose and illuminated from any direction:
1. Initialisation with a pose and a light direction estimate.
2. Model Fitting using our novel algorithm called LiST. Accurate pose, light and individuality parameters (3D shape and texture) are recovered.
3. Using the recovered parameters a synthetic face can be generated.
4. Recognition is performed using the identity parameters computed at step 2.

Sponsor:	DARPA - HumanID at a distance program European Research Office of the US Army
Grant:	N68171-01-C-9000
Copyright:	Springer-Verlag
BibTeX	@conference{RomBlaVet02, author = "Sami Romdhani and Volker Blanz and Thomas Vetter", title = "Face Identification by Fitting a 3D Morphable Model using Linear Shape and Texture Error Functions", booktitle = "Computer Vision -- ECCV'02", address = "Copenhagen, Denmark", volume = "4", pages = "3--19", year = 2002 }

Identification Results Files

To enable a fair comparison of the identification performances obtained by LiST with other algorithms, we provide detailed identification results. The identification experiments reported here were performed on two portions of the CMU-PIE face image database: 1. on the pose variation with ambient light portion (884 images) and 2. on the combined pose and illumination variation (4488 images).

Pose Variation:
This portion of the database contains 13 images for all 68 individual viewed from different poses. One identification experiments was performed by choosing one pose for the gallery set and the other poses for the probe set. Hence the gallery set contains a single image per individual. 13 identification experiments were carried out by choosing different poses for the gallery set. There is one identification result file per experiment. See the detailed description of the file format, but briefly, the files contain one line per probe image. Each line begins by the tag of the probe individual and its pose number. Then follows a series of pairs of gallery individual tag and their distance from the probe. The list of gallery tag is ordered by their closeness to the probe.

Gallery pose	Identification Results Files
2	pose_gal_02.csv
5	pose_gal_05.csv
7	pose_gal_07.csv
9	pose_gal_09.csv
11	pose_gal_11.csv
14	pose_gal_14.csv
22	pose_gal_22.csv
25	pose_gal_25.csv
27	pose_gal_27.csv
29	pose_gal_29.csv
31	pose_gal_31.csv
34	pose_gal_34.csv
37	pose_gal_37.csv

Pose & Illumination Variation:
This portion of the database holds 66 images for all 68 individual viewed from 3 poses and illuminated from 22 directions. We made 3 experiments. The gallery sets contained one image per individual. The 3 gallery sets hold each a different pose. All the image of the gallery set were illuminated from direction number 12.

There is one identification result file per experiment. See the detailed description of the file format, but briefly, the files contain one line per probe image. Each line begins by the tag of the probe individual its pose number and the flash light number. Then follows a series of pairs of gallery individual tag and their distance from the probe. The list of gallery tag is ordered by their closeness to the probe.

Gallery pose	Identification Results Files
27 (front)	light_gal_27_12.csv
5 (side)	light_gal_05_12.csv
22 (profile)	light_gal_22_12.csv

Description of the Identification Result File:
An Identification Result File stores the results of an identification experiment. There is one line per probe image. Each line has two parts. The first part pertain to the probe image, the second part lists the gallery images along with their distance from the probe image. The list of gallery image is ordered by their closeness to the probe.
The probe part varies depending on the type of experiment:

For pose-only experiements, the probe part is composed of the individuality number and the camera number (i.e. the pose number):
```
<id number>, <camera number>
```
For pose and illumination experiments, the probe part is composed of the individuality number, the camera number and the flash number:
```
<id number>, <camera number>, <flash number>
```

The gallery part is a list of id - distance pair:

<id number>, <distance>, ..., <id number>, <distance>

We chose the Comma Separated Values (CSV) file format because it is a human readable format and supported by a large range of software including Matlab, MS Excel, Gnumeric, Star Office and any spreadsheet tool. Each Identification Result File contains a header of 11 lines describing briefly its content.

Download all 16 experiments as a compressed file (.tar.gz) [5.0M]

Other papers reporting results on the PIE face image database

From our group:

Face Identification across different Poses and Illuminations with a 3D Morphable Model
Volker Blanz, Sami Romdhani, and Thomas Vetter
Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, 2002.
pdf [585k], ps.gz [1.1M],

From CMU:

Eigen Light-Fields and Face Recognition across Pose
Ralph Gross, Iain Matthews, and Simon Baker
Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, May, 2002

Quo Vadis Face Recognition ?
Ralph Gross, Jianbo Shi, and Jeffrey Cohn
Third Workshop on Empirical Evaluation Methods in Computer Vision, December, 2001.

Sami Romdhani

Last modified: Wed May 22 13:52:00 CEST 2002