Revision as of 14:25, 26 February 2019

Hello reader, Tis article describes the current state of the face detection algorithms of digiKam and the desired outcome of the corresponding GSoC project.
It is recommended to read Faces Management workflow improvements, as this describes the entire face management workflow. Thus it helps to understand the scope of these algorithms and where it need clarification about its structure and interfaces with other parties (code modules).

Currently, there are four different methods using the corresponding algorithm, which are more or less functional. The used algorhtmy can be chosen in the one Face Scan dialog.
The goal is to be able to recognize automatically faces in images, which are not tagged, using a previous face tag registered in the face recognition database. The algorithms are complex but explained in more detail below.

=currently implemented face recognition algorythms

Deep Neural Network (DNN) DLib

This is an experimental implementation of neural network to perform faces recognition.
This DNN is based on DLib code, a low level library used by OpenFace project. This code works, but it slow and complex to maintain. It is rather a proof of concept than being used for productive use.
Moreover the documentation in the source code is non-existant.

The code of Dlib is mostly the machine learning core implementation of http://dlib.net/ml.html and https://sourceforge.net/p/dclib/wiki/Known_users.

OpenCV - lbph

This is the most complete implementation of a face detection algorythm. Moreover it is the oldest implementation of such an algorythm in digiKam. It's not perfect and require at least 6 face already tagged manually by the user to identify the same faces in non-tagged images.
This algorithm records a histogram of the face in the database, which is used later to perform the comparisons against new/non-tagged faces. This one use OpenCV backend.https://towardsdatascience.com/face-recognition-how-lbph-works-90ec258c3d6b

OpenCV - Eigen Faces

An alternative algorithm what uses the OpenCV backend. It was introduced to have different source of results for face detection, enabling to proof the DNN approaches.

OpenCV - Fisher Face

Another algorithm what uses the OpenCV backend. It was introduced for the same pruposes as Eigen Faces.
According rumors this one is not finalized, it is said that not all methods are implemented.

Theree is paper explainingl the difference between Fisher and Eigen Faces, see http://disp.ee.ntu.edu.tw/~pujols/Eigenfaces%20and%20Fisherfaces.pdf

why so many differnt approaches?

The idea why four different algorythm were implmented is simply to be able make an comprehnsive assesment of the currntly avaialbe tehcnolgoeies applicalbe ni digiKam and eventuelly choose the best one.
The student who worked on the DNN project few years ago has concluded that DNN was the best method to recognize with less error as possible. Unfortuentley the training and recognition process took too longs and slowed down the application. Regardless that fallback, it is agreed that DNN is the best way to go, but not using the current implementation based on DLib.

prevoius work

DNN

All code code was introduced by a student Yingjie Liu <[email protected]> in a previous GoSC project

code

All the low level steps to train and recognize faces are done in this class. For the middle level codes, muti-threaded and chained, started by the face scan dialog, all is here: https://cgit.kde.org/digikam.git/tree/core/utilities/facemanagement?h=development/dplugins

Under Construction
This is a new page, currently under construction!

Why 4 kind of recognition algorithms

With 3.x versions, OpenCV has introduced a DNN API. I shall be used instead of the others approaches as done for the face detection

6/ Which kind of info are stored in database? This depend of recognition algorithm used. Histograms, Vector, Binary data, all responsible of algorithm computation, and of course all not compatible. Typically, when you change the recognition algorithm in Face Scan dialog, the database must be clear as well. But in fact this kind of database mechanism must be dropped, when DNN algorithm will be finalized, and only this one retained to do the job. As i said previously, 4 algorithms are implemented to choose the best one. At end, only one must still in digiKam face engine, and all the code must be simplified.

@@ Line 3: / Line 3: @@
 It is recommended to read [https://community.kde.org/Digikam/GSoC2019/FacesManagementWorkflowImprovements Faces Management workflow improvements], as this describes the entire face management workflow. Thus it helps to understand the scope of these algorithms and where it need clarification about its structure and interfaces with other parties (code modules).
-Currently, there are four different methods based on the corresponding algorithm, which are more or less functional. The goal is to be able to recognize automatically faces in images, which are not tagged, using a previous ''face tag'' registered in the face recognition database. The algorithms are complex but explained in more detail below
+Currently, there are four different methods using the corresponding algorithm, which are more or less functional. The used algorhtmy can be chosen in  the one Face Scan dialog. <br>
+The goal is to be able to recognize automatically faces in images, which are not tagged, using a previous ''face tag'' registered in the face recognition database. The algorithms are complex but explained in more detail below.
-{{construction}}
+=currently implemented face recognition algorythms
+#Deep Neural Network (DNN) DLib
+This is an experimental implementation of neural network to perform faces recognition. <br>
+This DNN is based on DLib code, a low level library used by OpenFace project. This code works, but it slow and complex to maintain. It is rather a proof of concept than being used for productive use. <br>
+Moreover the documentation in the source code is non-existant.
-# Deep Neural Network DLib
+The code of Dlib is mostly the machine learning core implementation of http://dlib.net/ml.html and https://sourceforge.net/p/dclib/wiki/Known_users.
-digiKam has already an experimental implementation of Neural Network to perform faces recognition
-This DNN is based on DLib code, a low level library used by OpenFace project. his code work, but it slow and is complex to maintain. It's more and less a proof of concept. The documentation in source code is inexistant.
-The code from Dlib is mostly the machine learning core implementation : http://dlib.net/ml.html https://sourceforge.net/p/dclib/wiki/Known_users/  This DNN code was introduced by a student with a previous GoSC project : Yingjie Liu <[email protected]>. This code work, but it slow and is complex to maintain. It's more and less a proof of concept. The documentation in source code is inexistant.
+#OpenCV -  lbph
-b Deep Neural Network  lbph :
+This is the most complete implementation of a face detection algorythm. Moreover it is the oldest implementation of such an algorythm in digiKam. It's not perfect and require at least 6 face already tagged manually by the  user to identify the same faces in non-tagged images. <br>
-this is the most complete implementation and the older one implemented in digiKam. It's not perfect and require at least 6 face already tagged manually by end user to identify the same person. This algorithm record a histogram of face in database, which is used later to perform the comparisons.
+This algorithm records a histogram of the face in the database, which is used later to perform the comparisons against  new/non-tagged faces.
 This one use OpenCV backend.https://towardsdatascience.com/face-recognition-how-lbph-works-90ec258c3d6b
-b  Eigen Faces :
+#OpenCV - [https://en.wikipedia.org/wiki/Eigenface Eigen Faces]
-Another algorithm what uses OpenCV backend. https://en.wikipedia.org/wiki/Eigenface. It was used to compare it with the DNN approaches.
+An alternative algorithm what uses the OpenCV backend. It was introduced to have different source of results for face detection, enabling to proof the DNN approaches.
-c Fisher Faces :
-Another algorithm what uses OpenCV backend. https://en.wikipedia.org/wiki/Eigenface. It was used to compare it with the DNN approaches.
+#OpenCV - Fisher Face
-According rumors this one is not finalized, as i remember some method not implemented.
+Another algorithm what uses the OpenCV backend. It was introduced for the same pruposes as Eigen Faces.  <br>
-This paper explain well the difference between fisher and eigen faces http://disp.ee.ntu.edu.tw/~pujols/Eigenfaces%20and%20Fisherfaces.pdf
+According rumors this one is not finalized, it is said that not all methods are implemented.
+Theree is paper explainingl the difference between Fisher and Eigen Faces, see http://disp.ee.ntu.edu.tw/~pujols/Eigenfaces%20and%20Fisherfaces.pdf
-All code code was introduced by a student with a previous GoSC project : Yingjie Liu <[email protected]>
+==why so many differnt approaches?==
-The 4 kind of recognizer algorithm are instanced and right one is used depending of user choice from Face Scan dialog.
+The idea why four different algorythm were implmented is simply to be able make an comprehnsive assesment of the currntly avaialbe tehcnolgoeies applicalbe ni digiKam and eventuelly choose the best one.  <br>
+The student who worked on the DNN project few years ago has concluded that DNN was the best method to recognize with less error as possible. Unfortuentley the training and recognition process took too longs and slowed down the application.
+Regardless that fallback, it is agreed that DNN is the best way to go, but not using the current implementation based on DLib.
+=prevoius work=
+#  DNN
+All code code was introduced by a student Yingjie Liu <[email protected]> in a previous GoSC project
+=code=
 All the low level steps to train and recognize faces are done in this class.
 For the middle level codes, muti-threaded and chained, started by the face scan dialog, all is here:
 https://cgit.kde.org/digikam.git/tree/core/utilities/facemanagement?h=development/dplugins
+{{construction}}
 Why 4 kind of recognition algorithms
-To compare and choose the best one. The student working on DNN project few year ago has concluded that DNN was the best method to recognize with less error as possible. But the training and recognition process take age and slow down the application.
-It is agreed that DNN is the best way to go, but not using the current implementation based on DLib.
 With 3.x versions, OpenCV has introduced a DNN API.
 I shall be used instead of the others approaches as done for the face detection