Apr 24, 2023, 3:14 AM
This example is based on the COOLPI CM5 32G memory version, and the system is ubuntu20.04.
- Download ChatGLM-MNN project
git clone https://github.com/wangzhaode/ChatGLM-MNN.git
- Compile MNN library
git clone https://github.com/alibaba/MNN.git
cd MNN
mkdir build && cd build
cmake .. # CPU only
make -j8
cp -r ../include ../../ChatGLM-MNN/
cp libMNN.so ../../ChatGLM-MNN/libs
cp express/libMNN_Express.so ../../ChatGLM-MNN/libs
- Download Models
cd ChatGLM-MNN/resource/models
./download_models.sh int8
cp int8/* ./
- Build
mkdir build
cd build
cmake ..
make -j8
- Run
coolpi@Ubuntu:~/share/ChatGLM-MNN/build$ ./cli_demo
The device support i8sdot:1, support fp16:1, support i8mm: 0
Can't Find type=2 backend, use 3 instead
load ../resource/tokenizer/slim_vocab.txt ... Done!
load ../resource/models/glm_block_0.mnn model ... Done!
load ../resource/models/glm_block_1.mnn model ... Done!
load ../resource/models/glm_block_2.mnn model ... Done!
load ../resource/models/glm_block_3.mnn model ... Done!
load ../resource/models/glm_block_4.mnn model ... Done!
load ../resource/models/glm_block_5.mnn model ... Done!
load ../resource/models/glm_block_6.mnn model ... Done!
load ../resource/models/glm_block_7.mnn model ... Done!
load ../resource/models/glm_block_8.mnn model ... Done!
load ../resource/models/glm_block_9.mnn model ... Done!
load ../resource/models/glm_block_10.mnn model ... Done!
load ../resource/models/glm_block_11.mnn model ... Done!
load ../resource/models/glm_block_12.mnn model ... Done!
load ../resource/models/glm_block_13.mnn model ... Done!
load ../resource/models/glm_block_14.mnn model ... Done!
load ../resource/models/glm_block_15.mnn model ... Done!
load ../resource/models/glm_block_16.mnn model ... Done!
load ../resource/models/glm_block_17.mnn model ... Done!
load ../resource/models/glm_block_18.mnn model ... Done!
load ../resource/models/glm_block_19.mnn model ... Done!
load ../resource/models/glm_block_20.mnn model ... Done!
load ../resource/models/glm_block_21.mnn model ... Done!
load ../resource/models/glm_block_22.mnn model ... Done!
load ../resource/models/glm_block_23.mnn model ... Done!
load ../resource/models/glm_block_24.mnn model ... Done!
load ../resource/models/glm_block_25.mnn model ... Done!
load ../resource/models/glm_block_26.mnn model ... Done!
load ../resource/models/glm_block_27.mnn model ... Done!
load ../resource/models/lm.mnn model ... Done!
Q: 写一首描绘冬天的诗
A: 漫天雪花飞舞,
寒风呼啸,大地一片沉寂。
冬天的气息弥漫,
银装素裹的世界令人陶醉。
树枝上挂满晶莹剔透的冰霜,
寒风把它们吹得摇曳生姿。
湖面上结满了厚厚的冰,
划着小船的人们欢笑着前行。
冬天是一个美丽而神秘的季节,
让人感受到宁静和祥和。
虽然它带来了一些不便,
比如冷的天气和冰雪的覆盖,
但它也是一年中最美好的时光。<eop>
Q: