Posts made by zhengbicheng

I'm looking forward to it.

@jugg 解码和播放运行第一条命令应该就有了？

@jugg 硬解码CPU占用能到50?

@jugg https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu-ports/ 换成这个源试试

@MSUMMERS999 Maybe you need to try burning the image again?

@qwe 具体报什么错误，截图看看

Introduction

YOLOv5 is the world's most loved vision AI, representing Ultralytics open-source research into future vision AI methods, incorporating lessons learned and best practices evolved over thousands of hours of research and development.

FastDeploy is an open source and very easy to use warehouse, which can quickly deploy YOLOv5 on coolpi-4B.

Model Speed Table

In addition to YOLOv5, we also support the deployment of YOLOX, YOLOv7, and PPYOLOE. The following speeds are the end-to-end speeds of the model:

C++ Demo

It is difficult to develop AI applications using C++, but it can be matched with other useful libraries on CoolPI-4B. For your convenience, we provide a demo of C++.

// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision.h"

void RKNPU2Infer(const std::string& model_file, const std::string& image_file) {
  auto option = fastdeploy::RuntimeOption();
  option.UseRKNPU2();

  auto format = fastdeploy::ModelFormat::RKNN;

  auto model = fastdeploy::vision::detection::RKYOLOV5(
      model_file, option,format);

  auto im = cv::imread(image_file);

  fastdeploy::vision::DetectionResult res;
  fastdeploy::TimeCounter tc;
  tc.Start();
  if (!model.Predict(im, &res)) {
    std::cerr << "Failed to predict." << std::endl;
    return;
  }
  auto vis_im = fastdeploy::vision::VisDetection(im, res,0.5);
  tc.End();
  tc.PrintInfo("RKYOLOV5 in RKNN");
  std::cout << res.Str() << std::endl;
  cv::imwrite("vis_result.jpg", vis_im);
  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

int main(int argc, char* argv[]) {
  if (argc < 3) {
    std::cout
        << "Usage: infer_demo path/to/model_dir path/to/image run_option, "
           "e.g ./infer_model ./picodet_model_dir ./test.jpeg"
        << std::endl;
    return -1;
  }

  RKNPU2Infer(argv[1], argv[2]);

  return 0;
}

Python Demo

In addition to using C++, you can also use FastDeploy's Python API for rapid development.

# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import fastdeploy as fd
import cv2
import os


def parse_arguments():
    import argparse
    import ast
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_file", required=True, help="Path of rknn model.")
    parser.add_argument(
        "--image", type=str, required=True, help="Path of test image file.")
    return parser.parse_args()


if __name__ == "__main__":
    args = parse_arguments()

    model_file = args.model_file
    params_file = ""

    # 配置runtime，加载模型
    runtime_option = fd.RuntimeOption()
    runtime_option.use_rknpu2()

    model = fd.vision.detection.RKYOLOV5(
        model_file,
        runtime_option=runtime_option,
        model_format=fd.ModelFormat.RKNN)

    # 预测图片分割结果
    im = cv2.imread(args.image)
    result = model.predict(im)
    print(result)

    # 可视化结果
    vis_im = fd.vision.vis_detection(im, result, score_threshold=0.5)
    cv2.imwrite("visualized_result.jpg", vis_im)
    print("Visualized result save in ./visualized_result.jpg")

Result

input image

输入图片

output image

@retroman https://github.com/yanyitech/coolpi_4B_docs

@retroman CoolPI Forum is an open forum where developers from all countries communicate. We have created Github Docs Repo. If you have other documents that we need to sort out, please refer to issues.

@ccjjww1222 您误会我的意思了，我的意思是你可以用电脑利用转接器连接Cool-pi开发板的串口。使用串口输入账号密码后就是命令行了。

@ccjjww1222 不是可以串口连接吗？不可能电脑屏幕也没有把。。。

@bbsvs2000 目标追踪？指deepsort一类的吗？

Brief Introduction

With the improvement of AI computing power, Face Recognition algorithms have gradually shifted from machine learning to deep learning. Through this tutorial, you will learn how to use FastDeploy to quickly implement Face Recognition on Coolpi.

What is ArcFace

ArcFace is a human face recognition model. Please refer to InsightFace Github Repo for detailed parameters and model introduction. For developers' testing, models exported by InsightFace are provided below. Developers can download and use them directly. The accuracy metric is sourced from the model description in InsightFace. Refer to the introduction in InsightFace for more details.

How to transform the model

If you need to convert your ONNX model to RKNN model, FastDeploy also provides the corresponding method. You only need
to modify the configuration file of the model in the tools/rknpu2/config to achieve rapid model conversion

git clone https://github.com/PaddlePaddle/FastDeploy.git
wget https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx

python -m paddle2onnx.optimize --input_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \
                               --output_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \
                               --input_shape_dict "{'data':[1,3,112,112]}"
                               
python  /Path/To/FastDeploy/tools/rknpu2/export.py \
        --config_path tools/rknpu2/config/arcface_unquantized.yaml \
        --target_platform rk3588

Workflow

# Download CoolPI-AI
git clone https://github.com/yanyitech/coolpi-ai.git

# Go to face detection demo
cd example/face_recognition

# Build and make
mkdir build
cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../../../third_party/fastdeploy-develop
make -j8

# 下载官方转换好的ArcFace模型文件和测试图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/face_demo.zip
unzip face_demo.zip

# CPU推理
./infer_arcface_demo ms1mv3_arcface_r100.onnx face_0.jpg face_1.jpg face_2.jpg 0
# RKNPU推理
./infer_arcface_demo ms1mv3_arcface_r100.onnx face_0.jpg face_1.jpg face_2.jpg 1

Navigation

• Introduction to FastDeploy
• Introduction to ArcFace

Brief Introduction

With the improvement of AI computing power, Face Detection algorithms have gradually shifted from machine learning to deep learning. Through this tutorial, you will learn how to use FastDeploy to quickly implement face detection on Coolpi.

What is SCRFD

SCRFD is an efficient high accuracy face detection approach which initially described in Arxiv, and accepted by ICLR-2022. In order to facilitate you to quickly understand the parameters of the model, we only give the parameters of common models. If you need to view the detailed data of the model, please go to InsightFace's official Github.

Workflow

# Download CoolPI-AI
git clone https://github.com/yanyitech/coolpi-ai.git

# Go to face detection demo
cd example/face_detection

# Build and make
mkdir build
cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../../../third_party/fastdeploy-develop
make -j8

# Download model and picture
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/scrfd_500m_bnkps_shape640x640_rknpu2.zip
unzip scrfd_500m_bnkps_shape640x640_rknpu2.zip
wget https://raw.githubusercontent.com/DefTruth/lite.ai.toolkit/main/examples/lite/resources/test_lite_face_detector_3.jpg

# Run
./infer_with_scrfd scrfd_500m_bnkps_shape640x640_rknpu2/scrfd_500m_bnkps_shape640x640_rk3588_quantized.rknn \
              test_lite_face_detector_3.jpg \
              1

How to transform the model

If you need to convert your ONNX model to RKNN model, FastDeploy also provides the corresponding method. You only need to modify the configuration file of the model in the tools/rknpu2/config to achieve rapid model conversion

git clone https://github.com/PaddlePaddle/FastDeploy.git
wget  https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/scrfd_500m_bnkps_shape640x640.zip
unzip scrfd_500m_bnkps_shape640x640.zip
python  /Path/To/FastDeploy/tools/rknpu2/export.py \
        --config_path tools/rknpu2/config/scrfd_quantized.yaml \
        --target_platform rk3588

Result

Input Data

Output Data

Navigation

• Introduction to FastDeploy
• Introduction to SCRFD

Brief Introduction

FastDeploy is an easy-to-use and high performance AI model deployment toolkit for Cloud, Mobile and Edge with out-of-the-box and unified experience, end-to-end optimization for over 150+ Text, Vision, Speech and Cross-modal AI models. Including image classification, object detection, image segmentation, face detection, face recognition, keypoint detection, matting, OCR, NLP, TTS and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform.

We spent a lot of time on the adaptation of FastDeploy for Coolpi. The model we have adapted can be used in Detection, Face Detection, Face Recognition, Face Alignment, Segmentation, OCR and other fields. These models will greatly improve Coolpi's ability in edge computing.

Model List

In order to facilitate you to choose a reasonable model to apply to your project, we have provided the running speed of the model.

Demo

• Face Detection with SCRFD
• Face Recognition with ArcFace
• Try to deploy YOLOv5 on coolpi

Contribution

Thanks to Baidu FastDeploy team for making such a good tool so that we can easily deploy AI model.

@yihuan1984 rk系列的板子解码应该不是用gpu吧，应该是mpp

@kingpin1cn 这个你可以尝试一下，因为我不太了解opencv 。但是肯定和 npu 是没有啥关系的

@kingpin1cn 不行哦，只支持RKNN框架

@1103854485 详细安装文档参考https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/install_rknn_toolkit2.md

@1103854485 rknn-tookit-lite是用于3399Pro这一类RK一代NPU上的python库。RK3588是rknn-tookit-lite2，最高版本就是1.4.0，不需要更新。只需要把RK1代的转换仓库从rknn_toolkit改成rknn_toolkit2转换一下模型就能跑了

@ccjjww1222 这个应该是不可以的，只能按照头文件里设定的进行配置。

@ccjjww1222 感谢您的支持。百度的fastdeploy仓库我已经初步适配完毕了，目前计划11.11在bilibili将有个直播，有Picodet（Detection）和PPHumanSeg的部署过程，有时间可以来看看。

@ccjjww1222 您先要学会怎么用onnxruntime跑通模型，然后再学习Rknn。您自己从头到尾把后处理写一遍，就知道问题出在哪里了。只跑代码，不学原理，不是长久之计。:-)

@ccjjww1222 我在获取推理结果坐标那个问题底下不是回复您如何排查问题了吗？您流程走一遍了吗

@ccjjww1222 这个应该是暂时查看不了的。和显卡不一样，rknn目前好像没办法实时查看NPU的信息。

@ccjjww1222

有问题无非两种情况。

• 第一种情况是模型本身就有问题。
• 第二种情况是模型没问题，rknn推理有问题。

排查的方法有两种。

• 第一种:针对RKNN推理精度问题，rknn-toolkit提供了连板调试的方法，但是我们还未开放这个方法。
• 第二种：请先用onnxruntime跑一遍onnx模型的推理结果，如果onnx推理出来的结果出错，说明是模型本身的问题。如果onnx推理没错，rknn错了，你再留言。

针对这个仓库的话，RKNN精度应该是没问题的，这个RK那边已经测试过了。只可能是模型不匹配，毕竟不可能100%都检测出来。

@ccjjww1222 正常，确实就是100ms左右

@ccjjww1222 这个yolov5应该是没问题的呀，你训练的yolov5是哪个版本的yolov5？

@ccjjww1222 yolov5计划在11月底前实现适配，下周可能会推出轻量级目标检测框架Picodet的适配，您可以尝试一下。

@ccjjww1222 NHWC这个配置不需要更改，输入图片不做nhwc转nchw的preprocess就行了。重要的是你需要更改目标板子的型号，你导出的模型是RK356X的，所以在RK3588上跑不通。请查询一下rknn-toolkit的文档，没记错的话应该是config函数存在这个配置选项。