Lecture

AmebaPro2

06 Oct 2024 • Richard Kuo

Introduction to AMB82-mini, HUB 8735 Ultra

EdgeAI AmebaPro2

教材網址：https://github.com/rkuo2000/EdgeAI-AmebaPro2

LLM服務器程式範例： AmebaPro2_server (.py)
Arduino 程式範例： Arduino/AMB82-mini (.ino)

自強基金會 WiFi

SSID: TCFSTWIFI.ALL
Pass: 035623116

1. AI 介紹

AI 簡介

AI 硬體介紹

2. 開發板介紹

RTL8735B晶片簡介

32-bit Arm v8M, up to 500MHz, 768KB ROM, 512KB RAM, 16MB Flash (MCM embedded DDR2/DDR3L up to 128MB)
802.11 a/b/g/n WiFi 2.4GHz/5GHz, BLE 5.1, NN Engine 0.4 TOPS, Crypto Engine, Audio Codec, …

HUB 8735 Ultra

https://github.com/ideashatch/HUB-8735

AMB82-mini

Ameba Arduino

3. Arduino IDE使用介紹

Arduino IDE 2.3.2 下載 & 安裝

偏好設定 (Preferences)

Hub8735 ultra
https://raw.githubusercontent.com/ideashatch/HUB-8735/main/amebapro2_arduino/Arduino_package/ideasHatch.json

AMB82-mini
https://github.com/ambiot/ambpro2_arduino/raw/main/Arduino_package/package_realtek_amebapro2_index.json

Ameba AIoT 影片

—

選定開發板 AMB82-MINI

Tools > Board Manager > AMB82 package > 4.0.7

Getting Started

首先將AMB82-mini板子用MicroUSB線連接至電腦的USB port

Arduino examples 範例練習

01.Basics> Blink

02.Digitial> GPIO> Button

修改程式碼：

const int buttonPin = 1;  // the number of the pushbutton pin
const int ledPin = LED_BUILTIN;    // the number of the LED pin

AMB82-Mini 程式範例

瀏覽器打開 EdgeAI-AmebaPro2, 點[Code]並選 [Download ZIP]
解壓縮.zip, 並將 Arduino/AMB82-mini 複製到 Documents/Arduino底下

4. 感測器範例練習

Serial-monitor baud rate = 115200

紅外線測距模組

VL53L0X v2
VL53L1X v2

Datasheet: VL53L0X - Time-of-Flight ranging sensor

Sketch: IR_VL53L0X

慣性感測模組

MPU6050

慣性元件介紹

Sketch: IMU MPU6050-DMP6v12

5.彩色顯示器 TFTLCD

AMB82-mini + ILI9341 TFT-LCD

SPI - LCD Screen ILI9341 TFT

Interface signal names:

MOSI: Standard SPI Pin
MISO: Standard SPI Pin
SLK: Standard SPI Pin
CS: Standard SPI Pin
RESET: Used to reboot LCD.
D/C: Data/Command. When it is at LOW, the signal transmitted are commands, otherwise the data transmitted are data.
LED (or BL): Adapt the screen backlight. Can be controlled by PWM or connected to VCC for 100% backlight.
VCC: Connected to 3V or 5V, depends on its spec.
GND: Connected to GND.

AMB82 MINI and QVGA TFT LCD Wiring Diagram:

Sketch: Camera_TFTLCD

Exmples/AmebaSPI:

LCD_Screen_ILI9341_TFT : LCD Draw Tests
Camera_2_Lcd : Camera output , then Jpeg Decoder to TFT-LCD
Camera_2_Lcd_JPEGDEC : Camera output, saved to SDcard, then Jpeg Decoder to read to TFT-LCD

6. 影像串流範例練習

影像串流

Sketch: RTSP_VideoOnly

Amebapro2 AMB82-mini Arduino Example Guides

動作偵測 (motion detection)

Examples: AmebaMultimedia / MotionDetection / LoopPostProcessing.ino

修改ssid, passwd, 後燒錄到AMB82-mini,
按reset後程式即開始運行, 用serial-monitor 查看顯示串流網址
啟動手機或電腦上之VLC player, 設定RTSP串流網址

Motion Detection Google Line Notify

Examples: AmebaMultimedia / MotionDetection / MotionDetectGoogleLineNotify

7. 音頻應用

Audio & Mic

音頻環回測試

Examples: AmebaMultimedia / Audio /LoopbackTest

MP3 播放

Skatch: MP3_Player

store a song.mp3 onto SDcard

安裝 Hub8735 ultra

cd ~/.arduino15/packages/ideasHatch/hardware/AmebaPro2/4.0.13-Release/
copy variants/common_libs/MP3_Player.a to ~/.arduino15/packages/realtek/hardware/AmebaPro2/4.0.7/variants/common_libs
edit platform.txt (~/.arduino15/packages/realtek/hardware/AmebaPro2/4.0.7/platform.txt) to add “{com_lib.path}/MP3_Player.a” into Ameba archive list (compiler.ameba.ar.list)
copy libraries/MP3_Player to ~/.arduino15/packages/realtek/hardware/AmebaPro2/4.0.7/libraries

Upload & Reset to run

音頻串流範例

Examples: AmebaMultimedia / Audio / RTSPAudioStream

RTSP Audio Stream

MP4錄音範例

Examples: AmebaMultimedia / RecordMP4 / AudioOnly

Multimedia - MP4 Recording

音頻分類範例

Examples: AmebaNN / AudioClassification

YAMNet

8. 人臉辨識與識別

人臉辨識與識別介紹

人臉檢測範例

Examples: AmebaNN / RTSPFaceDetection

人臉識別範例

Examples: AmebaNN / RTSPFaceRecognition

Serial_monitor: REG=RKUO

Enter the command REG=Name to give the targeted face a name.
Enter the command DEL=Name to delete a certain registered face. For example, DEL=SAM
Enter the command BACKUP to save a copy of registered faces to flash.
If a backup exists, enter the command RESTORE to load registered faces from flash.
Enter the command RESET to forget all previously registered faces.

9. 影像分類 (Image Classification)

影像分類範例

RTSP_ImageClassification.ino

Garbage模型訓練與檔案轉換

模型訓練： kaggle.com/rkuo2000/garbage-cnn

required in kaggle for AmebaPro2 1) pip install tensorflow==2.14.1 2) model.save(‘garbage_cnn.h5’, include_optimizer=False)

模型轉換：AMB82 AI Model Conversion

Download garbage_cnn.h5 from kaggle.com/rkuo2000/garbage-cnn Output
Compress garbage_cnn.h5 to garbage_cnn.zip
Go to Amebapro2 AI convert model, fill up your E-mail
Upload garbage_cnn.zip
Upload one (.jpg) test picture (EX. glass100.jpg from Garbage dataset)
Email will be sent to you for the link of network_binary.nb

程式範例：RTSP_GarbageClassification.ino

click the recieved Email link to download network_binary.nb
create NN_MDL folder in SDcard, save network_binary.nb under NN_MDL folder, and rename it to imgclassification.nb
plugin SDcard back to AMB82-MINI
modify Sketch RTSP_GarbageClassification.ino 1) modify SSID and PASSWD 2) modify imgclass.modelSelect (change DEFAULT_IMGCLASS to CUSTOMIZED_IMGCLASS)
burn code into board AMB82-MINI, and run it with VLC player streaming

10. 物件偵測 (Object Detection)

Public Dataset

Roboflow

物件檢測介紹

Kaggle範例:

Pothole模型訓練與檔案轉換

模型訓練： kaggle.com/rkuo2000/yolov7-pothole

1) repro https://github.com/WongKinYiu/yolov7 2) create pothole.yaml %%writefile data/pothole.yaml

train: ./Datasets/pothole/train/images
val:  ./Datasets/pothole/valid/images
test: ./Datasets/pothole/test/images

# Classes
nc: 1  # number of classes
names: ['pothole']  # class names

3) YOLOv7-Tiny Fixed Resolution Training

!sed -i "s/nc: 80/nc: 1/" cfg/training/yolov7-tiny.yaml
!sed -i "s/IDetect/Detect/" cfg/training/yolov7-tiny.yaml

模型轉換：AMB82 AI Model Conversion

Download best.pt from kaggle.com/rkuo2000/yolov7-pothole
Compress best.pt to best.zip
Go to Amebapro2 AI convert model, fill up your E-mail
Upload best.zip
Upload one (.jpg) test picture (EX. pothole_test.jpg from Pothole dataset)
Email will be sent to you for the link of network_binary.nb

程式範例：RTSP_YOLOv7_Pothole_Detection.ino

click the recieved Email link to download network_binary.nb
create NN_MDL folder in SDcard, save network_binary.nb under NN_MDL folder, and rename it to yolov7_tiny.nb
plugin SDcard back to AMB82-MINI
modify Sketch RTSP_YOLOv7_Pothole_Detection.ino 1) modify SSID and PASSWD 2) modify ObjDet.modelSelect(OBJECT_DETECTION, CUSTOMIZED_YOLOV7TINY, NA_MODEL, NA_MODEL);
burn code into board AMB82-MINI, and run it with VLC player streaming