After the pandemic most of the offices will start for on-site work and schools will start, during this time we also need to take care if any employee or student is having correct body temperature and is wearing mask. Instead of hiring someone and risk their life to check temperature and mask for so much employees and students is tough task.
So to solve this problem we want to create a device which monitors their heart rate and checking are they wearing a mask using Edge Impulse model on AWS IoT Edukit and Embedded Camera.
All the Computer Vision will be completely done over ESP32 Cam using Deep Learning model. The Mask Detection model is trained on Edge Impulse and the generated library is used in ESP32 CAM.
The Data is sent to AWS IoT suing MQTT and major alerts like Person is Not Wearing Mask is sent as email to Admin using AWS SES
- The IR Distance sensor will check when user is standing in front of Device and start camera and heart sensor
- Read the Image and inference it on Mask Model.
- Check whether Employee/Person is wearing Mask using Mask Classification Model trained on Edge Impulse
- Send Predictions to M5Core2 using UART communication.
- Then Check the Pulse Rate using Pulse Sensor and send to M5Core2 to display on screen.
- If results are showing that Person is not healthy then that case will be reported to admin using AWS IoT and SMTP via SES.
1. We created our own dataset by using own photos with mask and no mask and uploaded on Edge Impulse.
2. The created image classification impulse on Edge Impulse
3. After training the model we got accuracy of 87%
4. Deploy the Model using Arduino Library
First the OV2640 Camera on ESP32Cam read the image of user's face and stores it in a buffer. That buffer is then transferred to neural network and processed. THe Predictions are sent to M5COre2 via UART ( 13, 14 Pins on M5COre2).
THis is sample prediction, by using esp32 cam sending data to local ip address webpage. The espcam takes around 2-3 seconds per prediction.
Whenever the user is detected the predictions are sent to AWS via MQTT to AWS IoT Cloud and when there are any people who are not wearing mask then it will also send via eMAIL using SMTP of AWS SES service.
1. Initially Create Thing in AWS IoT and set up the certificate and MQTT URL.
2. It will generate unique URL for host url and also give a topic in form of "$aws/things/NAME/FUNCTION"
3. Set Up the Credentials in code as:
int PORT = 8883;
char MQTT_SUB = "$aws/things/NAME/shadow/update";
char MQTT_PUB = "$aws/things/NAME/shadow/update";
Also set up the credentials of certificates of CA Certificates, Client Certificate and private Key downloaded earlier.
4. Using the Pubsubclient library the M5Stack spublishes data to IoT Cloud.
1. Verify your email and set up HTTPS API of AWS SES.
2. The request url is of form :
3. Sent the request with other parameters like subject of mail and body of mail.
The circuit was simply developed on perf board to create a compact body.
Circuit build on Perf Board to make things easy
- The major features of this gadget are portability and ease of setup, which opens up a large market for it. Another important consideration is the cost savings compared to more expensive health care systems.
- Secure work environments will assist to decrease the risk of illnesses like COVID-19 spreading.
- The product may be utilized at any level, be it at the institution level or at the organization level. It will be able to identify proper circumstances, and if any outliers are found, it will notify the concerned party right away.
- To test out the device in nearby schools and our college entrance.
- Using more AWS Services like Object Detection, Lambada. Currently we didn't have enough AWS credits to do it.
- Adding IR Temperature Sensor to Detect Body Temperature.