ESP32 cam Person Detection

 

 

 

 

https://eloquentarduino.com/posts/esp32-cam-person-detection

ESP32 cam Person Detection

 

 

 

Person detection running on our cheap ESP32 cam has become a common task nowadays.

Only a few years ago this would have been simply impossible because of lack of both hardware (Arduino boards used to feature a mediocre 16 kb RAM) and software (neural networks support for embedded systems was simply non-existent).

As of today, lots of things has changed and person detection has even become one the get started projects for TinyML. Nonetheless, many tutorials on the subject are a bit convoluted and pretty hard to follow, let alone modify and integrate into your very own project.

But it doesn't have to be that way.

I'm going to show you how you can add person detection on your very own ESP32 camera project in 5 minutes!

Software requirements

You will need to install 3 libraries:

1. tlfm_esp32: the TensorFlow runtime for the ESP32
2. EloquentTinyML: a wrapper around TensorFlow
3. EloquentEsp32Cam: to interact with camera easily

You can install them all from the Arduino Library Manager.

Be sure you have the latest version of each!

Arduino sketch

This part is easy, you don't have to write much code nor configure many options.

See source

Filename: PersonDetectionExample.ino

/**
 * Run person detection on ESP32 camera
 *  - Requires tflm_esp32 library
 *  - Requires EloquentEsp32Cam library
 *
 * Detections takes about 4-5 seconds per frame
 */
#include <Arduino.h>
#include <tflm_esp32.h>
#include <eloquent_tinyml.h>
#include <eloquent_tinyml/zoo/person_detection.h>
#include <eloquent_esp32cam.h>

using eloq::camera;
using eloq::tinyml::zoo::personDetection;


void setup() {
    delay(3000);
    Serial.begin(115200);
    Serial.println("__PERSON DETECTION__");

    // camera settings
    // replace with your own model!
    camera.pinout.freenove_s3();
    camera.brownout.disable();
    // only works on 96x96 (yolo) grayscale images
    camera.resolution.yolo();
    camera.pixformat.gray();

    // init camera
    while (!camera.begin().isOk())
        Serial.println(camera.exception.toString());

    // init tf model
    while (!personDetection.begin().isOk())
        Serial.println(personDetection.exception.toString());

    Serial.println("Camera OK");
    Serial.println("Point the camera to yourself");
}

void loop() {
    Serial.println("Loop");

    // capture picture
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    // run person detection
    if (!personDetection.run(camera).isOk()) {
        Serial.println(personDetection.exception.toString());
        return;
    }

    // a person has been detected!
    if (personDetection) {
        Serial.print("Person detected in ");
        Serial.print(personDetection.tf.benchmark.millis());
        Serial.println("ms");
    }
}

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364

Copy

The sketch can be broken into the following parts.

Configure

// configure camera
camera.pinout.freenove_s3();
camera.brownout.disable();
// only works on 96x96 (yolo) grayscale images
camera.resolution.yolo();
camera.pixformat.gray();

// init camera
while (!camera.begin().isOk())
    Serial.println(camera.exception.toString());

// init tf model
while (!personDetection.begin().isOk())
    Serial.println(personDetection.exception.toString());

1234567891011121314

Copy

Run

// capture picture
if (!camera.capture().isOk()) {
    Serial.println(camera.exception.toString());
    return;
}

// run person detection
if (!personDetection.run(camera).isOk()) {
    Serial.println(personDetection.exception.toString());
    return;
}

1234567891011

Copy

Test

// test if a person is detected
if (personDetection) {
    // do whatever you want here...
}

1234

Copy

Customize

Customizing the sketch is easy. You can add your own logic to be performed when a person is detected inside the if block above.

A few use cases you can implement:

• a smart door bell that activates when it recognizes a person in front of it
• a smart relay that turns the lights on when you're back at home
• count the number of people passing in front of your camera

Feel free to let me know what you implemented with the form below.

If you're stuck, don't hesitate to ask for help.

Appendix. Camera config

To run the above code, set these configurations in your Tools menu (left is for ESP32, right form ESP32S3).