LIP algorithm page   LapTop presentation on-line   Html LIP Tracking version


The LIP automatic segmentation algorithm test test



It is commonly observed that visual information provides a precious help to the listener under degraded acoustic conditions. Visual cues are effectively used by human beings to improve speech intelligibility. The motivation of the present work is to extract information for automatic audiovisual lipreading.
Here, an original algorithm is proposed for lip segmentation under natural conditions, the only requirement being that the micro-camera is mounted on a light helmet worn by the speaker in order to be fixed w.r.t. the head. The video sequence should contain the region of the face spanning from chin to nostrils.



Left: An RVB image sequence of mouth movements (soft red make-up lips).
Right: Sequence of final lip shape superposed on the initial sequence.


Automatic Lip Tracking: Bayesian Segmentation and Active Contours In A Cooperative Scheme

Main Authors: M.Liévin and P.Delmas.

Abstract

An algorithm for speaker's lip contour extraction is presented here. A color video sequence
of speaker's face is acquired, under natural lighting conditions and without any particular make-up.
First, a logarithmic color transform is performed from RGB to HI (hue, intensity) color space. A
statistical approach using Markov random field modelling helps to segment the mouth area, integrating red hue and motion into a spatiotemporal neighbourhood. Simultaneously, a Region Of Interest (ROI) and relevant boundaries points are automatically extracted. Next, an active contour using spatially varying coefficients is initialised with the results of the preprocessing stage. Performance of active contours are greatly improved when initialisation is close to the desired
features. Finally, an accurate lip shape with inner and outer borders is obtained with good quality
results in this challenging situation.
 

Results


Top Left: An RVB image sequence of mouth movements.
Top Right: Sequence of final lip shape superposed on the initial sequence.
Bottom Left: Initial points and mouth corners (see P.Delmas)
Bottom Right: Final shape of the mouth using active contours (see P.Delmas)
 


Top Left: An RVB image sequence of mouth movements.
Top Right: Sequence of final lip shape superposed on the initial sequence.
Bottom Left: Initial points and mouth corners (see P.Delmas)
Bottom Right: Final shape of the mouth using active contours (see P.Delmas)