Posts made by george

george

@wuming lsusb 发一下结果，然后内核要同步到最新。

george

@Pharizna Try using a USB disk or SDcard to start

george

@wuming 恭喜！！有空的话可以整理一下你的调试步骤，方便后面进来的小伙伴学习。

george

@顾真牛他如果需要环境调试，我们可以提供。

george

This example is based on the COOLPI CM5 32G memory version, and the system is ubuntu20.04.

Download ChatGLM-MNN project

git clone https://github.com/wangzhaode/ChatGLM-MNN.git

Compile MNN library

git clone https://github.com/alibaba/MNN.git
cd MNN
mkdir build && cd build
cmake .. # CPU only
make -j8
cp -r ../include ../../ChatGLM-MNN/
cp libMNN.so ../../ChatGLM-MNN/libs
cp express/libMNN_Express.so ../../ChatGLM-MNN/libs

Download Models

cd ChatGLM-MNN/resource/models
./download_models.sh int8
cp int8/* ./

Build

mkdir build
cd build
cmake ..
make -j8

Run

coolpi@Ubuntu:~/share/ChatGLM-MNN/build$ ./cli_demo    
The device support i8sdot:1, support fp16:1, support i8mm: 0
Can't Find type=2 backend, use 3 instead
load ../resource/tokenizer/slim_vocab.txt ... Done!
load ../resource/models/glm_block_0.mnn model ... Done!
load ../resource/models/glm_block_1.mnn model ... Done!
load ../resource/models/glm_block_2.mnn model ... Done!
load ../resource/models/glm_block_3.mnn model ... Done!
load ../resource/models/glm_block_4.mnn model ... Done!
load ../resource/models/glm_block_5.mnn model ... Done!
load ../resource/models/glm_block_6.mnn model ... Done!
load ../resource/models/glm_block_7.mnn model ... Done!
load ../resource/models/glm_block_8.mnn model ... Done!
load ../resource/models/glm_block_9.mnn model ... Done!
load ../resource/models/glm_block_10.mnn model ... Done!
load ../resource/models/glm_block_11.mnn model ... Done!
load ../resource/models/glm_block_12.mnn model ... Done!
load ../resource/models/glm_block_13.mnn model ... Done!
load ../resource/models/glm_block_14.mnn model ... Done!
load ../resource/models/glm_block_15.mnn model ... Done!
load ../resource/models/glm_block_16.mnn model ... Done!
load ../resource/models/glm_block_17.mnn model ... Done!
load ../resource/models/glm_block_18.mnn model ... Done!
load ../resource/models/glm_block_19.mnn model ... Done!
load ../resource/models/glm_block_20.mnn model ... Done!
load ../resource/models/glm_block_21.mnn model ... Done!
load ../resource/models/glm_block_22.mnn model ... Done!
load ../resource/models/glm_block_23.mnn model ... Done!
load ../resource/models/glm_block_24.mnn model ... Done!
load ../resource/models/glm_block_25.mnn model ... Done!
load ../resource/models/glm_block_26.mnn model ... Done!
load ../resource/models/glm_block_27.mnn model ... Done!
load ../resource/models/lm.mnn model ... Done!

Q: 写一首描绘冬天的诗

A:  漫天雪花飞舞,
寒风呼啸,大地一片沉寂。
冬天的气息弥漫,
银装素裹的世界令人陶醉。

树枝上挂满晶莹剔透的冰霜,
寒风把它们吹得摇曳生姿。
湖面上结满了厚厚的冰,
划着小船的人们欢笑着前行。

冬天是一个美丽而神秘的季节,
让人感受到宁静和祥和。
虽然它带来了一些不便,
比如冷的天气和冰雪的覆盖,
但它也是一年中最美好的时光。<eop>

Q:

george

@george Add 5.5inch HDMI AMOLED edid

https://www.waveshare.net/shop/5.5inch-HDMI-AMOLED.htm

diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c
index 4fdfb41b91e1..b1171d8cbf2d 100644
--- a/drivers/gpu/drm/drm_edid.c
+++ b/drivers/gpu/drm/drm_edid.c
@@ -575,6 +575,14 @@ static const struct drm_display_mode drm_dmt_modes[] = {
        { DRM_MODE("4096x2160", DRM_MODE_TYPE_DRIVER, 556188, 4096, 4104,
                   4136, 4176, 0, 2160, 2208, 2216, 2222, 0,
                   DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_NVSYNC) },
+       /* 0x59 - 3840x1100@59.998Hz RB */
+       { DRM_MODE("3840x1100", DRM_MODE_TYPE_DRIVER, 282890, 3840, 3888,
+                  3920, 4100, 0, 1100, 1103, 1108, 1150, 0,
+                  DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_NVSYNC) },
+       /* 0x60 - 1080x1920@60Hz RB */
+       { DRM_MODE("1080x1920", DRM_MODE_TYPE_DRIVER, 137520, 1080, 1112,
+                  1144, 1200, 0, 1920, 1928, 1932, 1936, 0,
+                  DRM_MODE_FLAG_NHSYNC | DRM_MODE_FLAG_NVSYNC) },
 };

george

Users may have various resolutions of displays on their hands, some of which are not included in the xf86EdidModes support list. In this case, the software needs to do some processing to support this type of display. The specific steps are as follows:

Obtaining device's edid information

cat /sys/class/drm/card0-HDMI*/edid > /home/coolpi/share/edid.bin

Parsing EDID to obtain timings for various resolutions

coolpi@Ubuntu:~/share$  sudo apt-get install read-edid edid-decode
coolpi@Ubuntu:~/share$  parse-edid < edid.bin
Checksum Correct

Section "Monitor"
        Identifier "HDMI"
        ModelName "HDMI"
        VendorName "ICB"
        # Monitor Manufactured week 50 of 2017
        # EDID version 1.3
        # Digital Display
        DisplaySize 1200 340
        Gamma 2.20
        Option "DPMS" "true"
        Horizsync 30-180
        VertRefresh 48-120
        # Maximum pixel clock is 600MHz
        #Not giving standard mode: 1152x864, 75Hz
        #Not giving standard mode: 1280x800, 60Hz
        #Not giving standard mode: 1280x720, 60Hz
        #Not giving standard mode: 1280x1024, 60Hz
        #Not giving standard mode: 1440x900, 60Hz
        #Not giving standard mode: 1600x900, 60Hz
        #Not giving standard mode: 1680x1050, 60Hz

        #Extension block found. Parsing...
#WARNING: I may have missed a mode (CEA mode 97)
#DOUBLE WARNING: It's your first mode, too, so this may actually be important.
#WARNING: I may have missed a mode (CEA mode 96)
#DOUBLE WARNING: It's your first mode, too, so this may actually be important.
#WARNING: I may have missed a mode (CEA mode 90)
#DOUBLE WARNING: It's your first mode, too, so this may actually be important.
#WARNING: I may have missed a mode (CEA mode 95)
        Modeline        "Mode 11" 297.00 1920 1968 2000 2200 2160 2163 2168 2250 +hsync +vsync 
        Modeline        "Mode 0" 282.89 3840 3888 3920 4100 1100 1103 1108 1150 +hsync -vsync 
        Modeline        "Mode 1" 540.00 3840 3888 3952 4000 1080 1083 1093 1125 +hsync -vsync 
        Modeline        "Mode 2" 148.500 1920 2008 2052 2200 1080 1084 1089 1125 +hsync +vsync
        Modeline        "Mode 3" 74.250 1920 2008 2052 2200 1080 1082 1087 1125 +hsync +vsync interlace
        Modeline        "Mode 4" 74.250 1280 1390 1420 1650 720 725 730 750 +hsync +vsync
        Modeline        "Mode 5" 27.027 720 736 798 858 480 489 495 525 -hsync -vsync
        Modeline        "Mode 6" 27.027 720 736 798 858 480 489 495 525 -hsync -vsync
        Modeline        "Mode 7" 148.500 1920 2448 2492 2640 1080 1084 1089 1125 +hsync +vsync
        Modeline        "Mode 8" 74.250 1280 1720 1760 1980 720 725 730 750 +hsync +vsync
        Modeline        "Mode 9" 27.000 720 732 796 864 576 581 586 625 -hsync -vsync
        Modeline        "Mode 10" 25.200 640 656 752 800 480 490 492 525 -hsync -vsync
        Modeline        "Mode 12" 241.50 2560 2608 2640 2720 1440 1443 1448 1481 +hsync +vsync 
        Option "PreferredMode" "Mode 11"
EndSection

Increase special resolution timming to DRM_ dmt_ Modes List

diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c
index 4fdfb41b91e1..132dd01f28c9 100644
--- a/drivers/gpu/drm/drm_edid.c
+++ b/drivers/gpu/drm/drm_edid.c
@@ -575,6 +575,10 @@ static const struct drm_display_mode drm_dmt_modes[] = {
        { DRM_MODE("4096x2160", DRM_MODE_TYPE_DRIVER, 556188, 4096, 4104,
                   4136, 4176, 0, 2160, 2208, 2216, 2222, 0,
                   DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_NVSYNC) },
+       /* 0x59 - 3840x1100@59.998Hz RB */
+       { DRM_MODE("3840x1100", DRM_MODE_TYPE_DRIVER, 282890, 3840, 3888,
+                  3920, 4100, 0, 1100, 1103, 1108, 1150, 0,
+                  DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_NVSYNC) },
 };

Modify HDMI clock division method

diff --git a/arch/arm64/boot/dts/rockchip/rk3588-cpcm5-evb-v11.dts b/arch/arm64/boot/dts/rockchip/rk3588-cpcm5-evb-v11.dts
index 9eee8a7133f0..41155fcfb4fe 100755
--- a/arch/arm64/boot/dts/rockchip/rk3588-cpcm5-evb-v11.dts
+++ b/arch/arm64/boot/dts/rockchip/rk3588-cpcm5-evb-v11.dts
@@ -1308,6 +1308,16 @@ &uart2 {
 &display_subsystem {
        memory-region = <&drm_logo>;
        memory-region-names = "drm-logo";
+       clocks = <&hdptxphy_hdmi_clk0>, <&hdptxphy_hdmi_clk1>;
+       clock-names = "hdmi0_phy_pll", "hdmi1_phy_pll";
+};
+
+&hdptxphy_hdmi_clk0 {
+status = "okay";
+};
+
+&hdptxphy_hdmi_clk1 {
+status = "okay";
 };
 
 &rng {

Recompile and replace the kernel, reboot

george

@wuming onedrive上0421最新的ubuntu22.04固件已经默认集成了intel的wifi/bt驱动和固件，可以下载测试。

george

@edwardzhou : good

george

@jugg
不同的格式对齐要求不同，RGA硬件本身是对图像每行的数据是按照字（world）对齐的方式进行取数的，即4个字节32个bit。例如RGBA格式本身单个像素存储大小为32（4 × 8）bit，所以没有对齐要求；RGB565格式存储大小为16（5 + 6 +5）bit，所以需要2对齐；RGB888格式存储大小为24（8 × 3）bit，所以该格式需要4对齐才能满足RGA硬件的32bit取数要求；YUV格式存储相对较为特殊，本身排列要求需要2对齐，Y通道单像素存储大小为8bit，UV通道根据420/422决定每四个像素的存储大小，所以YUV格式Y通道需要4对齐才能满足RGA的硬件取数要求，则YUV格式需要4对齐；其他的未提及的格式对齐要求原理相通。注意，该题中对齐均指width stride的对齐要求，YUV格式本身实际宽高、偏移量由于格式本身特性也是要求2对齐的。

george

@wuming 那两个文件的权限改一下

george

@wuming 提供dmesg信息

george

@wuming
iwlwifi-ty-a0-gf-a0-59.ucode
iwlwifi-ty-a0-gf-a0.pnvm

george

@wuming

git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
https://kernel.googlesource.com/pub/scm/linux/kernel/git/firmware/linux-firmware.git

george

@wuming 确认你的/lib/firmware路径有没有这个文件iwlwifi-ty-a0-gf-a0-59.ucode

george

@wuming 你的模组需要的固件是iwlwifi-ty-a0-gf-a0-59.ucode
sudo cp ./linux-firmware/iwlwifi* /lib/firmware

george

JD bought a cheaper one
The kernel code has been updated to the latest version, including the following submission records: commit 4235ffbfee944d67d14d8143762e0a9314af8ee3
Compile the kernel and copy it to the device，Remember that modules.tar.gz needs to be decompressed

sudo cp ./out  /boot/firmware -R
sudo tar -zxvf ./out/modules.tar.gz -C /lib
sudo reboot

Confirm that the network card is properly recognized，Confirm Bluetooth recognition is normal.
Download WiFi/BT firmware

git clone git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
sudo cp -rfp ./linux-firmware/iwlwifi-cc-a0* /lib/firmware/ 
sudo cp -rfp ./linux-firmware/intel /lib/firmware/ -R
sudo reboot

Actual measurement effect

george

@jugg 关于I420格式的问题，环境变量增加GST_MPP_NO_RGA=0
关于黑边和数据大小不对的问题，编码器数据需要四字节对齐，比如：1920X1080 需要设置为1920X1088

george

Some users have been asking what kind of LCD can be compatible with the 4b machine. Due to compatibility with Raspberry Pi, the MIPI interface cannot support high resolution screens. However, the 4b machine has a DP interface, which can drive the eDP interface LCD with a simple transfer, and the software does not need to be modified. The actual debugging effect of the adapter board is as follows:

george

@wuming

编译完成out目录文件拷贝到机器/boot/firmware路径

sudo cp ./out/* /boot/firmware -R

解压modules.tar.gz 到机器/lib 路径

sudo tar -zxvf /boot/firmware/modules.tar.gz -C /lib

george

@AugustRobot_Zou

george

To facilitate users' development based on RKNN, the RKNN C and Python environments have been made into PPA installation sources. Running the following command can complete the installation and setup.Currently, only the ubuntu22.04 environment is supported, and other versions will be added in the future.

sudo add-apt-repository ppa:george-coolpi/rknpu
sudo apt update
sudo apt-get install rknpu2

Environmental testing

Python

cd /usr/share/rknn-toolkit2/examples/inference_with_lite/
python3 test.py

The following results indicate that the environment was successfully built

--> Load RKNN model
done
--> Init runtime environment
I RKNN: [11:36:49.473] RKNN Runtime Information: librknnrt version: 1.4.0 (a10f100eb@2022-09-09T09:07:14)
I RKNN: [11:36:49.474] RKNN Driver Information: version: 0.8.2
I RKNN: [11:36:49.475] RKNN Model Information: version: 1, toolkit version: 1.4.0-c15f5e0b(compiler version: 1.4.0 (c73777b51@2022-09-05T12:06:01)), target: RKNPU v2, target platform: rk3588, framework name: PyTorch, framework layout: NCHW
done
--> Running model
resnet18
-----TOP 5-----
[812]: 0.9996696710586548
[404]: 0.0002492684288881719
[657]: 1.632158637221437e-05
[833]: 1.0159346857108176e-05
[466 895]: 9.02384545042878e-06

done

C

sudo apt-get install git cmake -y
git clone https://gitee.com/yanyitech/rknpu2.git
cd rknpu2/examples/rknn_mobilenet_demo
./build-linux_RK3588.sh
cd install/rknn_mobilenet_demo_Linux/
./rknn_mobilenet_demo model/RK3588/mobilenet_v1.rknn model/dog_224x224.jpg

The following results indicate that the environment was successfully built

model input num: 1, output num: 1
input tensors:
  index=0, name=input, n_dims=4, dims=[1, 224, 224, 3], n_elems=150528, size=150528, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=0, scale=0.007812
output tensors:
  index=0, name=MobilenetV1/Predictions/Reshape_1, n_dims=2, dims=[1, 1001, 0, 0], n_elems=1001, size=1001, fmt=UNDEFINED, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003906
rknn_run
 --- Top5 ---
156: 0.984375
155: 0.007812
205: 0.003906
 -1: 0.000000
 -1: 0.000000

george

@AugustRobot_Zou 如下图：

george

@Mz3D-0 The new machine will default to the WIFI&BT module of AP6275S. You can also replace other models of 1T1R modules, and there will be multiple models for users to choose from.

george

@jugg 那就是RGA有限制

george

@jugg 环境变量里边尝试关闭RGA使能。这个应该是RGB部分有限制。

george

@jugg GST_VIDEO_CONVERT_PREFERRED_FORMAT 环境变量改一下，I420改为默认格式。

george

@edwardzhou 先确认WIN是否OK，MAC暂时还没有测试，这个回头我们跟一下。

george

@顾真牛

george

@wuming