Real-time Object Detection on Android using Paddle-Lite

Recently I have created a repository on GitHub named Real-time Object Detector. It’s a Flutter App using the Paddle-Lite framework and Yolo v3 model to detect specific objects. However, I found that some mobile inference framework popular in China like Paddle-Lite by Baidu and MNN by Alibaba is not so well-known by developers out of China. So I decided to write this article explaining how to create your Deep Learning model and make it run on your phone.

If you don’t know how a model looks like inside, it doesn’t matter. There should be no barrier to a starter. Let’s start.

Install PaddlePaddle and PaddleDetection

If you want to reproduce all the work here, make sure you have already installed PaddlePaddle and PaddleDetection. Paddlepaddle is an equivalent to Pytorch or TensorFlow, and it’s easy to use. While PaddleDetection is a toolkit for train high-performance object detection model based on PaddlePaddle.

In most cases, you can install PaddlePaddle in a Conda environment by just one line:

1
$ conda install paddlepaddle-gpu cudatoolkit=10.0

For more details, you should check the official document here. After done, run this to check if there’s anything wrong.
You are supposed to get a welcome message! Then it’s PaddleDetection’s turn. Just follow this tutorial, and don’t forget to add it to your PYTHONPATH.

Train Fruit Detection Model

Anyway, ensure you’re running the command at where PaddleDetection installed. If you have multiple display cards, you can specify one.

1
2
$ export PYTHONPATH=$PYTHONPATH:.
$ export CUDA_VISIBLE_DEVICES=0

Here’s an example script to fetch the fruit image dataset originated from Kaggle. But feel free to use your datasets, the official guide can tell you how.

1
$ python dataset/fruit/download_fruit.py

Then we can train the model. The model we chose is YOLO v3 mobile. The official has already configured many network settings. So you only need to modify them to meet your needs.

Type this to start the training process.

1
2
3
4
$ python -u tools/train.py -c configs/yolov3_mobilenet_v1_fruit.yml \
--use_tb=True \
--tb_log_dir=tb_fruit_dir/scalar \
--eval

If you have tensorboard installed on your device, you can monitor real-time loss value and mAP.

1
$  tensorboard --logdir tb_fruit_dir/scalar/ --host <host_IP> --port <port_num>

Sometimes, the script would run out of memory and end up killed. You can modify the configuration to limit the work number or batch size. Or restart your computer, log in to other non-graphic tty channels.

In the end, we can get the best model in the output-dir. That’s what we are going to use.

Optimize the Model

Although you can directly use the model we got in the last step to detect objects, it’s still not efficient enough to run on a device like your smartphone. What we should do is export the model and optimize it using the opt tool. You can download it from Paddle-Lite’s GitHub release page.

1
2
3
4
5
6
7
$ python tools/export_model.py -c configs/yolov3_mobilenet_v1_fruit.yml \
--output_dir=./inference_model \
-o weights=./output/yolov3_mobilenet_v1_fruit/best_model
$ ./opt --model_file=./inference_model/yolov3_mobilenet_v1_fruit/__model__ \
--param_file=./inference_model/yolov3_mobilenet_v1_fruit/__params__ \
--optimize_out_type=naive_buffer \
--optimize_out=model

Almost there! We successfully get the model file.

Run It on Your Phone

As I had pulled the source code on GitHub, doing some modifying jobs can make it run.

  • Change the parameters about the input size given in main.dart and MainActivity.java
  • Change the parameters used at the normalization step in MainActivyty.java
  • Change the path of the label file and the model file.

The final model and source code are here. Thanks for reading. :)

  • 版权声明: 本博客所有文章除特别声明外,均采用 CC BY-NC-SA 许可协议。转载需要标明作者,并以超链接的方式指向原文。
  • Copyrights © 2020 Kevin Li
  • 访问人数: | 浏览次数:

请我喝杯咖啡吧~