Building K-pop Idol Identifier with Amazon Rekognition

That is Tzuyu indeed!Creating CollectionNow we can detect face from a picture, and find the most similar face to the source picture from the target picture.

But, these are all one-off call, and we need something more to store the information of each member’s face and their name, so that when we send a new picture of Twice, it can retrieve data and detect each member’s face and display their names.

In order to implement this, we need to use what Amazon calls “Storage-Based API Operations”.

There are two Amazon-specific terms for this type of operations.

The “collection” is a virtual space where Rekognition stores information about detected faces.

With a collection, we can “index” faces, which means to detect faces in an image, then store the information in the specified collection.

What’s important is that the information Rekognition stores in the collection is not actual images, but feature vectors extracted by Rekognition’s algorithm.

Let’s see how we can create a collection and add indexes.

collectionId='test-collection'rekognition.

create_collection(CollectionId=collectionId)Yes.

It is as simple as that.

Since this is a new collection we just created, we don’t have any information stored in the collection.

But, let’s double check.

rekognition.

describe_collection(CollectionId=collectionId)In the above response, you can see ‘FaceCount’ is 0.

This will change if we index any face and store that information in the collection.

Indexing FacesIndexing faces is again as simple as one line of code with Rekognition.

sourceFile='Tzuyu.

jpeg' imageSource=open(sourceFile,'rb')rekognition.

index_faces(Image={'Bytes':imageSource.

read()},ExternalImageId='Tzuyu',CollectionId=collectionId)From the above code, you can see that I am passing ExternalImageId parameter and give it the value of string “Tzuyu”.

Later when we try to recognise Tzuyu from a new picture, Rekognition will search for faces that are matching any of the indexed faces.

As you will see later, when indexing a face, Rekognition will give it a unique face ID.

But I want to display the name “Tzuyu” when a matching face is found from a new picture.

For this purpose, I am using ExternalImageId.

Now we if we check our collection, we can see 1 face has been added to the collection.

rekognition.

describe_collection(CollectionId=collectionId)Search Faces by ImageNow with Tzuyu’s face indexed in our collection, we can send a new unseen picture to Rekognition and find the matching face.

But a problem with search_faces_by_image function is that it can only detect one face (the largest in the image).

So if we want to send a group picture of Twice and find Tzuyu from there, we will need to do an additional step.

Below we will first detect all the faces in the picture by using detect_faces, then with the bounding box information of each face, we will call search_faces_by_image one by one.

First let’s detect each face.

imageSource=open('twice_group.

jpg','rb')resp = rekognition.

detect_faces(Image={'Bytes':imageSource.

read()})all_faces = resp['FaceDetails']len(all_faces)Rekognition detected 9 faces from the group picture.

Good.

Now let’s crop each face and call serach_faces_by_image one by one.

image = Image.

open("twice_group.

jpg")image_width,image_height = image.

sizefor face in all_faces: box=face['BoundingBox'] x1 = box['Left'] * image_width y1 = box['Top'] * image_height x2 = x1 + box['Width'] * image_width y2 = y1 + box['Height'] * image_height image_crop = image.

crop((x1,y1,x2,y2)) stream = io.

BytesIO() image_crop.

save(stream,format="JPEG") image_crop_binary = stream.

getvalue()response = rekognition.

search_faces_by_image( CollectionId=collectionId, Image={'Bytes':image_crop_binary} ) print(response) print('-'*100)Among the 9 search_faces_by_image calls we have made, Rekognition has found one face that matches the indexed face in our collection.

We only indexed one face of Tzuyu, so what it has found is Tzuyu’s face from the group picture.

Let’s display this on the image with the bounding box and the name.

For the name part, we will use the ExternalImageId we set when we indexed the face.

By the way, from the search_faces_by_image response, ‘FaceMatches’ part is an array, and if there are more than one matches found from the collection, then it will show all the matches.

According to Amazon this array is ordered by similarity score with the highest similarity first.

We will get the match with the highest score by specifying the first item of the array.

from PIL import ImageFontimport ioimage = Image.

open("twice_group.

jpg")image_width,image_height = image.

size for face in all_faces: box=face['BoundingBox'] x1 = box['Left'] * image_width y1 = box['Top'] * image_height x2 = x1 + box['Width'] * image_width y2 = y1 + box['Height'] * image_height image_crop = image.

crop((x1,y1,x2,y2)) stream = io.

BytesIO() image_crop.

save(stream,format="JPEG") image_crop_binary = stream.

getvalue()response = rekognition.

search_faces_by_image( CollectionId=collectionId, Image={'Bytes':image_crop_binary} ) if len(response['FaceMatches']) > 0: draw = ImageDraw.

Draw(image) points = ( (x1,y1), (x2, y1), (x2, y2), (x1 , y2), (x1, y1)) draw.

line(points, fill='#00d400', width=2) fnt = ImageFont.

truetype('/Library/Fonts/Arial.

ttf', 15) draw.

text((x1,y2),response['FaceMatches'][0]['Face']['ExternalImageId'], font=fnt, fill=(255, 255, 0)) display(image)Hooray!.Again the correct answer!Identifying All Group Members of TwiceNow let’s expand the project to identify all members from the group picture.

In order to do that, we first need to index faces of all members (there are 9 members).

I have prepared 4 pictures of each member.

I have added multiple pictures of the same person following the logic of Amazon tutorial written by Christian Petters.

According to Petters, “adding multiple reference images per person greatly enhances the potential match rate for a person”, which makes intuitive sense.

From the Github link I’ll share at the end, you will find all the pictures that are used in this project.

collectionId='twice'rekognition.

create_collection(CollectionId=collectionId)import ospath = 'Twice'for r, d, f in os.

walk(path): for file in f: if file != '.

DS_Store': sourceFile = os.

path.

join(r,file) imageSource=open(sourceFile,'rb') rekognition.

index_faces(Image={'Bytes':imageSource.

read()},ExternalImageId=file.

split('_')[0],CollectionId=collectionId)rekognition.

describe_collection(CollectionId=collectionId)OK.

It seems like all 36 pictures are indexed in our “twice” collection.

Now it’s time to check the final result.

Can Rekognition be enhanced to identify each member of Twice?from PIL import ImageFontimage = Image.

open("twice_group.

jpg")image_width,image_height = image.

size for face in all_faces: box=face['BoundingBox'] x1 = box['Left'] * image_width y1 = box['Top'] * image_height x2 = x1 + box['Width'] * image_width y2 = y1 + box['Height'] * image_height image_crop = image.

crop((x1,y1,x2,y2)) stream = io.

BytesIO() image_crop.

save(stream,format="JPEG") image_crop_binary = stream.

getvalue()response = rekognition.

search_faces_by_image( CollectionId=collectionId, Image={'Bytes':image_crop_binary} ) if len(response['FaceMatches']) > 0: draw = ImageDraw.

Draw(image) points = ( (x1,y1), (x2, y1), (x2, y2), (x1 , y2), (x1, y1)) draw.

line(points, fill='#00d400', width=2) fnt = ImageFont.

truetype('/Library/Fonts/Arial.

ttf', 15) draw.

text((x1,y2),response['FaceMatches'][0]['Face']['ExternalImageId'], font=fnt, fill=(255, 255, 0))display(image)YES!.It can!.It identified all the members correctly!Thank you for reading.

You can find the Jupyter Notebook and the pictures used for the project from the below link.

https://github.

com/tthustla/twice_recognition.. More details

Leave a Reply