Face recognition by multi-frame fusion of rotating heads in videos

Abstract

This paper presents a face recognition study that implicitly utilizes the 3D information in 2D video sequences through a multi-sample fusion process. The approach is based on the hypothesis that continuous and coherent intensity variations in video frames caused by a rotating head can provide information similar to that of explicit face models or shapes from range images. The multi-frame fusion was performed on both the image and score levels. Both types of fusion showed large improvements in the recognition rates. The image level fusion showed improvements from 91%, using one frame, to 100%, using 7 frames, under regular lighting. An improvement from 63%, using one frame, to 85%, using 7 frames, was noticed under strong shadow. The score level fusion of two frames also showed an improvement in the recognition rate.