Admin
18- 06- 2007, 21:19
Sebastien Marcel on Face Recognition
http://blog.outer-court.com/files/sebastien-marcel.gif
Hi Sebastien! Can you tell us a little bit about yourself – how old are you, where do you live, and what do you do?
I am 34. I leave in Martigny, Switzerland, and I work in a Research Institute called IDIAP (http://www.idiap.ch/). I am a research scientist in Image Processing and Pattern Recognition applied to Biometric Authentication and Multimodal Interaction. See my homepage (http://www.idiap.ch/~marcel) for more details. I am particularly interested in the problem of face detection and recognition in images and videos.
What are some of the typical applications where people need image recognition?
Photo cataloging is an obvious one, but also biometric authentication (access to a bank account or cash machine, PIN code replacement on the cell phone or any cards), more secured applications such as identification of people such as terrorists (if possible), and more general applications in the domain of multi-modal interaction where a user can interact with a computer using the voice but also using gestures... you need to be able to recognize facial expressions, or to monitor elderly people and for that you need image processing and more particularly face detection and recognition.
Is face recognition very different from recognizing say a car, or an elephant, within a picture?
In practice yes. We are using very specific algorithms, it is very different with other objects. For instance, cars and other objects can be seen from any point of view while for a face this is more restricted. It is frontal most of the time, rotated in the plane or out-of-the plane of the image.
http://blog.outer-court.com/files/google-portrait.jpg
I see. Let’s take one of your applications, Google Portrait (http://www.idiap.ch/googleportrait/), which returns a list of faces only when you enter a search query. Can you explain the broad concept of how the face detection algorithm behind that works?
Yes. There are 3 main concepts:
scanning
classification
and merging
Scanning consists of searching in the image at any scale, location and eventually in-plane orientation for a face within a fixed size sub-window.
Classification consists of deciding if this sub-window is a face or not and if yes, its pose (frontal, +- 25 degrees rotated, +- 45 degrees, +- 90 degrees, ...).
Merging consists of taking all sub-windows classified as faces and merge them when they overlap.
Classification is the most important step. Indeed, it has to be very fast because you need to test several thousands of sub-windows per image depending on the size of the image and on the size of faces you want to find.
Of course you can use tricks to reduce the number of sub-windows like focusing on skin color but this is not applicable in most of the cases because you can have gray-scale images, and also because of illumination that affects the color. If you take 2 movies such as MI2 [AVI] (http://www.idiap.ch/~marcel/demos/video/mi2-mv1.avi) and Matrix [AVI] (http://www.idiap.ch/~marcel/demos/video/matrix2-mv1.avi) for instance, in the first case you can have skin everywhere at the beginning because you have rocks with skin-like color, and in Matrix all faces are “greenish” and people wear sunglasses (See more examples (http://www.idiap.ch/~marcel/demos.php#demo32).)
http://blog.outer-court.com/files/mi-face-detection.jpg
More (http://blog.outer-court.com/archive/2007-06-18-n53.html)
http://blog.outer-court.com/files/sebastien-marcel.gif
Hi Sebastien! Can you tell us a little bit about yourself – how old are you, where do you live, and what do you do?
I am 34. I leave in Martigny, Switzerland, and I work in a Research Institute called IDIAP (http://www.idiap.ch/). I am a research scientist in Image Processing and Pattern Recognition applied to Biometric Authentication and Multimodal Interaction. See my homepage (http://www.idiap.ch/~marcel) for more details. I am particularly interested in the problem of face detection and recognition in images and videos.
What are some of the typical applications where people need image recognition?
Photo cataloging is an obvious one, but also biometric authentication (access to a bank account or cash machine, PIN code replacement on the cell phone or any cards), more secured applications such as identification of people such as terrorists (if possible), and more general applications in the domain of multi-modal interaction where a user can interact with a computer using the voice but also using gestures... you need to be able to recognize facial expressions, or to monitor elderly people and for that you need image processing and more particularly face detection and recognition.
Is face recognition very different from recognizing say a car, or an elephant, within a picture?
In practice yes. We are using very specific algorithms, it is very different with other objects. For instance, cars and other objects can be seen from any point of view while for a face this is more restricted. It is frontal most of the time, rotated in the plane or out-of-the plane of the image.
http://blog.outer-court.com/files/google-portrait.jpg
I see. Let’s take one of your applications, Google Portrait (http://www.idiap.ch/googleportrait/), which returns a list of faces only when you enter a search query. Can you explain the broad concept of how the face detection algorithm behind that works?
Yes. There are 3 main concepts:
scanning
classification
and merging
Scanning consists of searching in the image at any scale, location and eventually in-plane orientation for a face within a fixed size sub-window.
Classification consists of deciding if this sub-window is a face or not and if yes, its pose (frontal, +- 25 degrees rotated, +- 45 degrees, +- 90 degrees, ...).
Merging consists of taking all sub-windows classified as faces and merge them when they overlap.
Classification is the most important step. Indeed, it has to be very fast because you need to test several thousands of sub-windows per image depending on the size of the image and on the size of faces you want to find.
Of course you can use tricks to reduce the number of sub-windows like focusing on skin color but this is not applicable in most of the cases because you can have gray-scale images, and also because of illumination that affects the color. If you take 2 movies such as MI2 [AVI] (http://www.idiap.ch/~marcel/demos/video/mi2-mv1.avi) and Matrix [AVI] (http://www.idiap.ch/~marcel/demos/video/matrix2-mv1.avi) for instance, in the first case you can have skin everywhere at the beginning because you have rocks with skin-like color, and in Matrix all faces are “greenish” and people wear sunglasses (See more examples (http://www.idiap.ch/~marcel/demos.php#demo32).)
http://blog.outer-court.com/files/mi-face-detection.jpg
More (http://blog.outer-court.com/archive/2007-06-18-n53.html)