Software apps and online services
Hand tools and fabrication machines
Nowadays, we hear everywhere about the "Smart city" concept. It is a concept that is booming and many people and many businesses are focusing on this in order to make cities smarter and more efficient.
One of the things that I miss and that causes me more headaches (I guess like many other people), is the issue of parking in the city.
Smart cities should be prepared to control and monitor city parking spaces, thus improving traffic and making people's lifes easier.
One of the biggest problems I have in my day to day, is the difficulty to park in the area where I live and know if there are free places and when.That's why I decided to address this issue by making this project.THE IDEA
A possible idea to solve this problem could be through small wireless devices that monitor parking spaces (e.g. anchored to street lamps), and send the data to a central station to simultaneously display in real time the current parking status of a street.
When you get to the street, you could see through a panel the number of free spaces in that street, you could even get that same data from the Internet and know it remotely before arriving at the street itself.
To develop this idea I have decided to make a proof of concept using the Sony Spresense board. It has a camera and I can add machine learning models to detect objects (in this case, free parking places and cars), thanks to Edge Impulse.
For this proof of concept I have decided toplace the device on the balcony of my house, from where I can see perfectly the parking area of my street.
In the following image you can see the device checking the parking zone.
The central pillar of this project resides in the Sony Spresense board that, together with a few added components, will allow me to make an ideal prototype for this proof of concept.
Used Hardware Spresense key features:
- Spresense main board
- Spresense camera
- Spresense LTE
- Spresense SDCard
- Spresense RTC
- Spresense LEDs
Used other components:
- 3D printed case/base/support
Used Spresense Firmware libraries:
Other Firmware libraries used for the project:
- Edge impulse
- Arduino IDE
- AWS IoT
The project could be divided into 3 parts:
- Physical prototype design.
- Image capture and ML model creation with Edge Impulse. (Needed to make the device intelligent and allow it to detect cars and free spaces).
- Free parking spacesdetection firmware program.
Knowing already, which are the physical components that are necessary for the project, the first thing is to make a prototype with which to make the tests.
The 3D model for printing has been made in tinkercad and in the following images you can see the parts have been generated (Static and Rotor Base, LTE board case). The 3D files are ….
Once we have the physical part created, it is necessary to start with the programming part.
In this part the idea is to capture a large amount of image samples of the street (with Sony Spresense board)in order to be able to transfer them to Edge Impulse and train the model to detect cars and free spaces.
This program will perform the following functions:
- Move the camera to various angles to capture a wider range of the whole street. The servomotor is used for this purpose.
- Capture images and save them on the SD card.
- Program the RTC alarm to perform the action periodically, so we can capture photos at various times and get images of the car park throughout the day.
- Copy the images to Edge Impulse and generate a model capable of detecting cars and free spaces.
In order to select this project, simply uncomment the PROJECT_TAKE_PHOTOS directive.
There is a section at the end of this documentation where I explain how the code is organised to make it more understandable.
After running this project we will get a large amount of images on the SDCard and we can go to Edge Impulse to load and tag them to generate the model capable of detecting the cars.
These are the steps to follow to generate the intelligent model that detects cars and free spaces through Edge Impulse.
Go to Data acquisition and upload all the images.
Label all photos indicating what is a car and what is a free space (the more samples there are the more accurate the detection).
Create the impulse indicating the image size to be used.
Generate features from the Image section, and then push Start training from the Object detection section. This will tell you how much space it takes up and the accuracy of the model.
In the deployment section we will autogenerate the code to introduce in our board (It will generate a.zip file that will be necessary to introduce it to the arduino IDE as a library).
This is the final firmware that connects to AWS IoT using the MQTT protocol to warn of free parking spaces. It has the following twofeatures:
- Launch a request through AWS to scan the street. (scans the street only once).
- Enable automatic monitoring through AWS (scans the street periodically depending on the sent time in seconds).
In order to select this project, simply uncomment the PROJECT_CAR_PARKING_SPACES directive.
In order to connect to AWS IoT it is important to have the CERTS folder on the SD card with your own certificates and add the address of your broker. The topic I have used is 'spresense/freeParking', but you can feel free to change the names in this.h with whatever you think is more suitable for your application.
The 'request'parameter is used to trigger or not the request to scan the street. ('0' do nothing, '1' ask the board to scan the street and return the current parking status).
The 'alarm' parameter is used to launch a periodical check of the street automatically according to the value in seconds that we set.(In this case we ask it to check the parking in the street every 30 seconds).
I would like to explain small details of the code to make it more understandable for anyone who wants to pick it up/check it.
The file structure is the next one:
It is a very easy to understand code that is based on two main files:
- spresense.h (in BSP folder): This is the file used to configure everything necessary for the project. In this file you can choose the project you want to compile (TAKE_PHOTOS or CAR_PARKING_DETECTION), and on the other hand it contains all the configuration of the peripherals and features used (camera configuration, AWS certificates, LTE, servomotor pin, RTC times, MQTT...).The code is totally modular, so anyone who wants to do without a specific peripheral because they want to use another one or don't need it, can simply comment out any of the directives xxxxx_PRESENT.That's why anyone who needs to tweak the code, can start from this file for most of the important changes and to configure it to their liking.
- ProyectoSpresenseParking.ino:This is the base file where the setup and loop functions typical of the Arduino IDE are located.This is where all the firmware logic resides.
The other files are simply used to maintain a better organisation of the code and separate the functions for each peripheral used by the Spresense board.
Note:The library used for MQTT is the ArduinoMqttClient library (version 0.1.5). The ArduinoJSON library version used is the 6.14.1. In newer versions, the compilation crashes, so I've decided to use this one.