Autonomous drone for delivery

This is the source for my autonomous drone for a simple delivery task project.

It makes use of Stereo Vision from two RPi camera v2 modules for detecting objects with depth using the YOLOv8 model and then uses a subsumption architecture to send control signals via MAVLink to a drone to perform simple delivery missions and avoid obstacles or land automatically on the landing pad.

The full project is intended to run on a Raspberry Pi 5 but should work on any platform connecting to 2 cameras simultaneously and talking via MAVLink.

• INFO: This project was built in 6 months, it includes many stages and complex knowledge. But basically, it works well, and be prepared with enough knowledge to dive into the code and fix things.
• WARNING: I take no responsibility for any problems if you run any of this code on your own drone. You need to have everything ready to avoid dangerous situations

Hardware design

To start this project. Make sure you have enough hardware for your system, and you need to fully connect the hardware components, including setting up and configuring your drone to operate stably. To ensure that the project can continue in the next steps.

• Reference: Ardupilot
• Setup Software: MissionPlanner

The figure below is my entire design for this project including an overview of the connected hardware and a 3D drawing built on Solidworks.

• NOTE: The hardware and design can be changed depending on the needs and purposes of your project.

Getting Started

First, install the ultralytics package from https://docs.ultralytics.com/guides/raspberry-pi/ on Raspberry Pi 5 and you can grab this repo with:

cd ~
git clone https://github.com/caothu2k1/drone_delivery.git

Once done, connect the Camera to the Raspberry Pi 5 and try running a simple python object detection program with a single camera using the yolov8.pt model, You can use another model if there is better performance.

pip install picamera2 

cd models
python test.py

• Next, export the yolov8.pt model to NCNN format to achieve better FPS performance on Raspberry Pi 5 by running the following command in your terminal.

  python export_model.py

• To test the performance, continue running the test.py with NCNN model yolov8n_ncnn_model.

Stereo Vision

Let's start by connecting two Cameras with RPi 5 and taking pictures of the Chessboard at many different angles and positions, do the following:

cd StereoVision
python chessboard_img.py

Once completed, it is necessary to calibrate both cameras from the previously collected images by running the following command:

python stereo_calibration.py

After successfully calibrating the camera, we can check the accuracy of the stereo camera system using the following two methods:

• Depth Map: Let's try out an example Depth Map using the Semi-Global Matching method, you can run:

  python depth_map.py

  • Additionally, you can adjust the Depth Map by running the program tune.py or changing the parameters set in the file 3dmap_set.txt
  • Reference: more information from https://github.com/realizator/stereopi-tutorial or OpenCV

• Distance Measurement: Now, let's try measuring the distance of objects using the yolov8n.pt or yolov8n_ncnn_model model with a simple program, you can run.

  cd camera
  python test.py

The figure below shows the results obtained from the stereo camera in this project, including building a Disparity Map and measuring the distance of the landing_pad at each different distance.

Templates

Instead of using a Ground Station Control like MissionPlanner, I built a simple control website combined with a 4G IoT system equipped on the Drone. To connect the Website to your own drone, you need to add your adminsdk in your-gcs-gg-firebase-adminsdk.json and your API_URL in Key_API.py and install firebase_admin library, you can run:

pip install firebase_admin
pip install retrying

We can remotely control drones via the internet, data will be stored and retrieved on the Firebase Database, Realtime Database parameters are configured in firbase_parameters_export.json.

Let's enter GOOGLE_MAP_KEY_API in index.html and FIREBASE_KEY_API in firebase.js, and open the website on your PC, the website will have the interface as shown below.

Simulation

Before starting the actual testing, we need a simulation to make sure all the code works correctly through the SITL library and install dependent packages on RPi 5:

pip install dronkit
pip install dronekit-sitl 
pip install pymavlink
pip install mavproxy

Then, open MissionPlanner on your PC and run a simple example test.py on RPi 5. You need to change your TCP address to connect RPi 5 to MissionPlanner in vehicle = connect('tcp:192.168.1.125:5762', wait_ready=True) and run:

cd copter
python test.py

Now, Let's open the website control and test the main code, do the following:

python main.py --vehicle_port your_tcp --model_path /path/to/your/model

Set up a flight trajectory, monitor, and control your drone via the website or MissionPlanner. If SITL doesn't set mode auto through your code, maybe the problem is resolved here:

Testing

Now, move to a large area to conduct an outdoor test flight with your code.

• WARNING: You need to check everything works well before doing this:

python main.py --vehicle_port your_port --model_path /path/to/your/model

The figure below is some outdoor test images from my project.

License

Licensed under GPLv3.