Facial Image Data Prep for a Deep Learning Model

Image for post
Image for post

I wanted to figure out how to best prepare images of faces, to normalise them for use in a Deep Learning model.

Upfront thinking

I started by asking myself, what is it that I am trying to achieve?

I have a face and regardless of which direction it is facing and its position in the whole image, I want to align it so it is facing forward and is at the centre of the image.

So how do I first of all detect which way the face is looking and where it is? For this I could use the location of the eyes, nose and mouth and use them as a guideline.

After a bit of searching I found out that this technique of finding facial features is called facial landmark detection. So now I have a starting point. Find the coordinates on the image of a bunch of facial landmarks.

I can then use these to skew the image so these points are transformed and appear aligned so they are in the centre of the image.

How did I do it?

The idea was to use OpenCV: a python library, to find the coordinates for each of the facial landmarks. Once I’d done that it was then figuring out how to do the transformation of these points to the centre regardless of their starting point.

This sounds well on paper, but in practice trying to find the Haarcascade metadata files for each of these coordinates, running it against the image, extracting each set of coordinates and then feeding them individually into a transformation function just seemed like a whole lot of effort. I spent some time playing with this then I decided to look around to see if there was a better solution.

Turns out I was looking for dlib. Dlib is a library that basically has facial landmark recognition built in and works perfectly with another python library called openface that helps you transform and skew an image.

I found this wonderful example below on a great blog series named Machine Learning is Fun!

import sys
import dlib
import cv2
import openface
# You can download the required pre-trained face detection model here:
# http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
predictor_model = "shape_predictor_68_face_landmarks.dat"
# Take the image file name from the command line
file_name = sys.argv[1]
# Create a HOG face detector using the built-in dlib class
face_detector = dlib.get_frontal_face_detector()
face_pose_predictor = dlib.shape_predictor(predictor_model)
face_aligner = openface.AlignDlib(predictor_model)
# Take the image file name from the command line
file_name = sys.argv[1]
# Load the image
image = cv2.imread(file_name)
# Run the HOG face detector on the image data
detected_faces = face_detector(image, 1)
print("Found {} faces in the image file {}".format(len(detected_faces), file_name))# Loop through each face we found in the image
for i, face_rect in enumerate(detected_faces):
# Detected faces are returned as an object with the coordinates
# of the top, left, right and bottom edges
print("- Face #{} found at Left: {} Top: {} Right: {} Bottom: {}".format(i, face_rect.left(), face_rect.top(), face_rect.right(), face_rect.bottom()))
# Get the the face's pose
pose_landmarks = face_pose_predictor(image, face_rect)
# Use openface to calculate and perform the face alignment
alignedFace = face_aligner.align(534, image, face_rect, landmarkIndices=openface.AlignDlib.OUTER_EYES_AND_NOSE)
# Save the aligned image to a file
cv2.imwrite("aligned_face_{}.jpg".format(i), alignedFace)

When running the above with my face image it gets skewed and transformed so all my facial features are aligned. As you can see below, it not only aligns the features but increases the size of the face too. I can then do this with multiple facial images so the features are always in the same place.

Image for post
Image for post

My data should then be ready to use in a Deep Learning model. Now all the features are aligned this should improve the accuracy of the model as position of features shouldn’t affect how it predicts who’s face it is!

I’ll be looking at implementing that in the near future.

Written by

Data and Productivity Writer — Data Architect at easyfundraising.org.uk

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store