Media library face recognition tools

• Detect faces in photo/video files
• Match faces to face patterns
• Patterns manipulation
• Face database search
• Tagging faces in the Plex Media Server library

• Debian based Linux (Other Linux versions not officially supported, but might work)
• Python 3.6+
• NVIDIA GPU (optional)
• Plex Media Server (optional)

• OpenCV + python3 bindings (better with CUDA support)
• DLib + python3 bindings (better with CUDA support)
• TensorFlow + python3 binding (optional)
• deepface python3 library (optional)
• numpy python3 library
• face_alignment python3 library
• face_recognition python3 library
• python3-piexif python3 library

For building dlib and others

sudo apt-get install build-essential cmake python3-pip

sudo apt-get install python3-opencv

Installation of libraries with CUDA is not so easy and may be differ for different OS versions. Some useful links for CUDA setup

https://developer.nvidia.com/cuda-downloads

https://medium.com/@sb.jaduniv/how-to-install-opencv-4-2-0-with-cuda-10-1-on-ubuntu-20-04-lts-focal-fossa-bdc034109df3

https://www.pyimagesearch.com/2019/12/09/how-to-install-tensorflow-2-0-on-ubuntu/

pip3 install deepface

Install my face_recognition clone (there are some modification to support exteranal matched faces)

git clone https://github.com/sashacmc/face_recognition.git; cd face_recognition; pip3 install .; cd ..

git clone https://github.com/sashacmc/face-rec-tools.git; cd face-rec-tools; pip3 install .; cd ..

Use one from predefined config files or prepare you own.

Predefined config files located in package's cfg folder (e.g. ~/.local/lib/python3.6/site-packages/face_rec_tools/cfg/)

Copy it to default config location (~/.face-rec.cfg) or specify with command line switch "-c"

cp ~/.local/lib/python3.?/site-packages/face_rec_tools/cfg/frontal.cfg ~/.face-rec.cfg

# recognize single image and print output (useful for debug)
face-rec-cli -a recognize_image -i imagefile.jpg

# recognize single video and print summary output (useful for debug)
face-rec-cli -a recognize_video -i videofile.mp4

# recognize folder and store the result in the database
face-rec-cli -a recognize_folder -i /folder/with/images

# remove folder recognition the result from the database
face-rec-cli -a remove_folder -i /folder/with/images

# match/rematch unmatched faces in database
face-rec-cli -a match_unmatched

# match/rematch all faces in database
face-rec-cli -a match_all

# match/rematch faces from folder in database
face-rec-cli -a match_folder -i /folder/with/images

# save faces from folder (must be previously recognized)
# cachedb must be disabled (otherwise they will saved inside cachedb)
face-rec-cli -a save_faces -i /folder/with/images -o /folder/for/faces

# find face from image in database and save them in folder
# cachedb must be disabled (otherwise they will saved inside cachedb)
face-rec-cli -a get_faces_by_face -i image_with_one_face.jpg -o /folder/for/faces

Run the server from the command line

face-rec-server 

Open the browser with you hostname/ip_adress and port 8081

For recognize a new folder click to "Recognition"->"Add new files...". And specify a folder which you want to recognize.

First time the recognition will take a while, because of loading necessary models.

After first recognition you will get all faces as unknown.

Click to the face (or select several with Shift/Ctrl) to add it as a pattern.

Previously added names will be saved, and you don't need to input it again. If you want to change the default face logo you can add it as 0_face.jpg for each person subfolder.

After patterns adding, start the match again (e.g. "Match"->"Rematch folder...")

After matching with patterns you will have matched persons and "weak" matched persons, it means that it not fully matched and will not be used for sync or search. You need to check them and add to patterns.

If you need some additional info about the face you can click the icon to see the source file, or the to see the pattern which it has been matched to.

If you have a big amount of faces you can simplify patterns separation by mean of clusterization in menu Clustering it will separate near faces to groups and will allow to add it to patterns together.

Face Recognition Tools allow syncing recognition results with Plex Media Server by means of tags. Recognized files will tagged with tags "person:[PERSON_NAME]"

# set all tags to the Plex database 
face-rec-plexsync -a set_tags

# clear all tags from the Plex database
face-rec-plexsync -a remove_tags

# recognize all files which present in the Plex database
# but not recognized yet and store the result in the database 
face-rec-plexsync -a sync_new

# delete from database all files which not present in the Plex database
face-rec-plexsync -a sync_deleted

If you have no Plex, or want to use some more complex search you can use follow command

(it will search all files in folder 2020 which contains faces of persons Name1 and Name2)

face-rec-db -a find_files_by_names -f 2020 -n Name1,Name2

Thanks to everyone who tested and gave advice.

Bug reports, suggestions and pull request are welcome!