PPE Detection with AWS Rekognition and Storing the Results in DynamoDB

Question

I'm using the PPE Detection from AWS Rekognition to identify PPE on people through images. Every time I upload a picture or several pictures in S3, a message is sent to SQS and my lambda function is triggered from there to capture the image name(s).

The lambda function calls the AWS Rekognition PPE Detection API to scan images in S3 from the image names captures from SQS.

I received the response from AWS Rekognition PPE and now I want to store part of the response in DynamoDB. Below is sample of a response from DynamoDB:

    {
   "ProtectiveEquipmentModelVersion": "1.0",
   "Persons": [
       {
           "BodyParts": [
               {
                   "Name": "FACE",
                   "Confidence": 99.85384368896484,
                   "EquipmentDetections": [
                       {
                           "BoundingBox": {
                               "Width": 0.12469039857387543,
                               "Height": 0.20445917546749115,
                               "Left": 0.5978690981864929,
                               "Top": 0.18556605279445648
                           },
                           "Confidence": 95.17121887207031,
                           "Type": "FACE_COVER",
                           "CoversBodyPart": {
                               "Confidence": 98.84524536132812,
                               "Value": true
                           }
                       }
                   ]
               },
               {
                   "Name": "LEFT_HAND",
                   "Confidence": 98.26607513427734,
                   "EquipmentDetections": [
                       {
                           "BoundingBox": {
                               "Width": 0.13546951115131378,
                               "Height": 0.18359044194221497,
                               "Left": 0.47036099433898926,
                               "Top": 0.5242195725440979
                           },
                           "Confidence": 77.47138214111328,
                           "Type": "HAND_COVER",
                           "CoversBodyPart": {
                               "Confidence": 97.84107208251953,
                               "Value": true
                           }
                       }
                   ]
               },
               {
                   "Name": "HEAD",
                   "Confidence": 99.99432373046875,
                   "EquipmentDetections": []
               }
           ],
           "BoundingBox": {
               "Width": 0.5233333110809326,
               "Height": 0.9821428656578064,
               "Left": 0.3733333349227905,
               "Top": 0.01785714365541935
           },
           "Confidence": 99.49939727783203,
           "Id": 0
       },
       {
           "BodyParts": [
               {
                   "Name": "LEFT_HAND",
                   "Confidence": 93.59660339355469,
                   "EquipmentDetections": []
               }
           ],
           "BoundingBox": {
               "Width": 0.46666666865348816,
               "Height": 0.9226190447807312,
               "Left": 0.0033333334140479565,
               "Top": 0.0535714291036129
           },
           "Confidence": 98.97230529785156,
           "Id": 1
       },
       {
           "BodyParts": [
               {
                   "Name": "FACE",
                   "Confidence": 77.40711212158203,
                   "EquipmentDetections": []
               },
               {
                   "Name": "HEAD",
                   "Confidence": 97.54975891113281,
                   "EquipmentDetections": []
               }
           ],
           "BoundingBox": {
               "Width": 0.08666666597127914,
               "Height": 0.1726190447807312,
               "Left": 0.5633333325386047,
               "Top": 0.761904776096344
           },
           "Confidence": 94.70215606689453,
           "Id": 2
       }
   ],
   "Summary": {
       "PersonsWithRequiredEquipment": [],
       "PersonsWithoutRequiredEquipment": [
           0,
           2
       ],
       "PersonsIndeterminate": [
           1
       ]
   }
}

From the above, the Equipment Detected are: FACE_COVER and HAND_COVER. Every time I attempt to save the data for each image, I can see only either FACE_COVER or HAND_COVER, but never both.

Here's my code so far:

import json
import boto3
from decimal import Decimal

def lambda_handler(event, context):
    s3_client = boto3.client('s3')
    client = boto3.client('rekognition')

    for msg in event["Records"]:
        msg_payload = json.loads(msg["body"])

        if "Records" in msg_payload:
            bucket = msg_payload["Records"][0]["s3"]["bucket"]["name"]
            image = msg_payload["Records"][0]["s3"]["object"]["key"].replace("+", " ")
            response = client.detect_protective_equipment(Image={'S3Object':{'Bucket':bucket,'Name':image}},SummarizationAttributes={'MinConfidence':80, 'RequiredEquipmentTypes':['FACE_COVER', 'HEAD_COVER']})
            
            for person in response["Persons"]:
                bp = person["BodyParts"]
                for ed in bp:
                    name = ed["Name"]
                    ppe = ed["EquipmentDetections"]
                    for type in ppe:
                        types = type["Type"]
                        confidence = str(type["Confidence"])
                        covers_body = type["CoversBodyPart"]["Value"]

                        data = {
                          "Details": 
                           [
                            {
                            "Body Part": ed["Name"],
                            "Confidence": str(type["Confidence"]),
                            "Cover Type": type["Type"],
                            "Covers Body Part": type["CoversBodyPart"]["Value"]
                            },
                           ],
                          "Image_Name": image
                          }
                
                        table = boto3.resource('dynamodb').Table("PPE_Detection")
                        table.put_item(Item={'Image_Name': image, 'Labels': data})

Below is how I want the data to be stored in DynamoDB for each image.

[
{
"Details": [
  {
    "Body Part": "FACE",
    "Confidence": 99.59647361,
    "Cover Type": "FACE_COVER",
    "Covers Body Part": true
  },
  {
    "Body Part": "HEAD",
    "Confidence": 92.464736,
    "Cover Type": "HEAD_COVER",
    "Covers Body Part": true
  }
],
"Image_Name": "image1.jpg"
  }
]

I would be grateful if I could get some help on this. Thanks!

user_271415 · Answer

Be aware that you have multiple people being labelled in your images. If you want the details of each individual person with PPE detected to be a distinct item in your results list then you could use something like this:

results = []

for person in response["Persons"]:
    bp = person["BodyParts"]
    result = { 'Image_Name': image, 'Details': [] }

    for ed in bp:
        name = ed["Name"]
        ppe = ed["EquipmentDetections"]

        for ppe_type in ppe:
            types = ppe_type["Type"]
            confidence = ppe_type["Confidence"]
            covers_body = ppe_type["CoversBodyPart"]["Value"]

            person_details = {
                "Body Part": name,
                "Confidence": confidence,
                "Cover Type": types,
                "Covers Body Part": covers_body
            }

        result['Details'].append(person_details)

    if len(result['Details']) > 0:
        results.append(result)

That would yield the following:

[
  {
    "Details": [
      {
        "Body Part": "FACE",
        "Confidence": 95.17121887207031,
        "Cover Type": "FACE_COVER",
        "Covers Body Part": true
      },
      {
        "Body Part": "LEFT_HAND",
        "Confidence": 77.47138214111328,
        "Cover Type": "HAND_COVER",
        "Covers Body Part": true
      }
    ],
    "Image_Name": "image1.jpg"
  }
]

PPE Detection with AWS Rekognition and Storing the Results in DynamoDB

1 Answers