EdgeAI-AMB82mini

EdgeAI AMB82mini

教材網址：https://github.com/rkuo2000/EdgeAI-AMB82mini

LLM服務器程式範例： [AmebaPro2_server/*.py](https://github.com/rkuo2000/EdgeAI-AMB82mini/tree/main/AmebaPro2_server)
Arduino 程式範例： [Arduino/AMB82-mini/](https://github.com/rkuo2000/EdgeAI-AMB82mini/tree/main/Arduino/AMB82-mini)

自強基金會 WiFi

SSID: TCFSTWIFI.ALL
Pass: 035623116

1. AI 介紹

AI 簡介

AI 硬體介紹

---

2. 開發板介紹

RTL8735B晶片簡介

32-bit Arm v8M, up to 500MHz, 768KB ROM, 512KB RAM, 16MB Flash (MCM embedded DDR2/DDR3L up to 128MB)
802.11 a/b/g/n WiFi 2.4GHz/5GHz, BLE 5.1, NN Engine 0.4 TOPS, Crypto Engine, Audio Codec, …

---

HUB 8735 Ultra

https://github.com/ideashatch/HUB-8735

---

AMB82-mini

Ameba Arduino

---

3. Arduino IDE使用介紹

Arduino IDE 2.3.4 下載 & 安裝

---

偏好設定 (Preferences)

Hub8735 ultra
https://raw.githubusercontent.com/ideashatch/HUB-8735/main/amebapro2_arduino/Arduino_package/ideasHatch.json

AMB82-mini
main https://github.com/Ameba-AIoT/ameba-arduino-pro2/raw/main/Arduino_package/package_realtek_amebapro2_index.json
dev https://github.com/Ameba-AIoT/ameba-arduino-pro2/raw/dev/Arduino_package/package_realtek_amebapro2_early_index.json

Ameba AIoT 影片

—

選定開發板 AMB82-MINI

Tools > Board Manager > AMB82 package > 4.0.9

---

Getting Started

首先將AMB82-mini板子用MicroUSB線 連接至電腦的USB port
確認UART com port (Ubuntu OS需 sudo chown usrname /dev/ttyUSB0)

燒錄程式碼：

• 按下UART_DOWNLOAD按鈕, 再按下RESET按鈕, 放開RESET按鈕, 再放開UART_DOWNLOAD按鈕,板子就進入燒錄模式.
• 然後于Arduino IDE上按下燒錄按鍵Upload

---

Arduino examples 範例練習

• 01.Basics> Blink

• 02.Digitial> GPIO> Button

修改程式碼：

const int buttonPin = 1; // the number of the pushbutton pin
const int ledPin = LED_BUILTIN; // the number of the LED pin

---

AMB82-Mini 程式範例

1. 瀏覽器打開 EdgeAI-AMB82mini, 點[Code]並選 [Download ZIP]
2. 解壓縮.zip, 並將 Arduino/AMB82-mini 複製到 文件/Arduino下, 成為 文件/Arduino/AMb82-mini

---

BLE example codes

Exmples > AmebaBLE > BLEV7RC_CAR_VIDEO

BLE V7RC

• V7RC APP 介紹
  
• 安裝手機App V7RC
  

---

4. 感測器範例練習

Serial-monitor baud rate = 115200

紅外線測距模組

VL53L0X v2
VL53L1X v2

Datasheet: VL53L0X - Time-of-Flight ranging sensor

Sketch: IR_VL53L0X

---

慣性感測模組

MPU6050

慣性元件介紹

Sketch: MPU6050-DMP6v12

---

5.彩色顯示器 TFTLCD

AMB82-mini + ILI9341 TFT-LCD

SPI - LCD Screen ILI9341 TFT

Interface signal names:

• MOSI: Standard SPI Pin
• MISO: Standard SPI Pin
• SLK: Standard SPI Pin
• CS: Standard SPI Pin
• RESET: Used to reboot LCD.
• D/C: Data/Command. When it is at LOW, the signal transmitted are commands, otherwise the data transmitted are data.
• LED (or BL): Adapt the screen backlight. Can be controlled by PWM or connected to VCC for 100% backlight.
• VCC: Connected to 3V or 5V, depends on its spec.
• GND: Connected to GND.

---

AMB82 MINI and QVGA TFT LCD Wiring Diagram:

Sketch: HTTP_Post_ImageText_TFTLCD

Exmples/AmebaSPI:

• LCD_Screen_ILI9341_TFT : LCD Draw Tests
• Camera_2_Lcd : Camera output , then Jpeg Decoder to TFT-LCD
• Camera_2_Lcd_JPEGDEC : Camera output, saved to SDcard, then Jpeg Decoder to read to TFT-LCD

---

6. 影像串流範例練習

影像串流

Examples: AmebaMultimedia > StreamRTSP > RTSP_VideoOnly
Sketch: RTSP_VideoOnly

• Amebapro2 AMB82-mini Arduino Example Guides

---

動作偵測 (motion detection)

Examples: AmebaMultimedia > MotionDetection > LoopPostProcessing

• 修改ssid, passwd, 後燒錄到AMB82-mini,
• 按reset後程式即開始運行, 用serial-monitor 查看顯示串流網址
• 啟動手機或電腦上之VLC player, 設定RTSP串流網址

---

Motion Detection Google Line Notify

Examples: AmebaMultimedia > MotionDetection > MotionDetectGoogleLineNotify

---

7. 音頻應用

Audio & Mic

音頻環回測試

Examples: AmebaMultimedia > Audio >LoopbackTest

---

MP3 播放

AMB82-mini + PAM8403 + 4ohm 3W speaker

Skatch: SDcardMP3

• .mp3 files stored under mp3 directory

Skatch: SDcardMP3_PlayAll

---

音頻串流範例

Examples: AmebaMultimedia > Audio > RTSPAudioStream

RTSP Audio Stream

---

MP4錄音範例

Examples: AmebaMultimedia > RecordMP4 > AudioOnly

Multimedia - MP4 Recording

---

音頻分類範例

Examples: AmebaNN > AudioClassification

YAMNet

---

8. 人臉辨識與識別

人臉辨識與識別介紹

---

人臉檢測範例

Examples: AmebaNN > RTSPFaceDetection

---

人臉識別範例

Examples: AmebaNN > RTSPFaceRecognition

Serial_monitor: REG=RKUO

• Enter the command REG=Name to give the targeted face a name.
• Enter the command DEL=Name to delete a certain registered face. For example, DEL=SAM
• Enter the command BACKUP to save a copy of registered faces to flash.
• If a backup exists, enter the command RESTORE to load registered faces from flash.
• Enter the command RESET to forget all previously registered faces.

---

9. 影像分類 (Image Classification)

• Kaggle平台使用介紹

• 卷積層神經網路介紹

• 影像分類介紹

---

影像分類範例

RTSP_GarbageClassification.ino

---

Garbage模型訓練與檔案轉換

模型訓練： kaggle.com/rkuo2000/garbage-cnn

required in kaggle for AmebaPro2 1) pip install tensorflow==2.14.1 2) model.save(‘garbage_cnn.h5’, include_optimizer=False)

模型轉換：AMB82 AI Model Conversion

1. Download garbage_cnn.h5 from kaggle.com/rkuo2000/garbage-cnn Output
2. Compress garbage_cnn.h5 to garbage_cnn.zip
3. Go to Amebapro2 AI convert model, fill up your E-mail
4. Upload garbage_cnn.zip
5. Upload one (.jpg) test picture (EX. glass100.jpg from Garbage dataset)
6. Email will be sent to you for the link of network_binary.nb

程式範例：RTSP_GarbageClassification.ino

1. click the recieved Email link to download network_binary.nb
2. create NN_MDL folder in SDcard, save network_binary.nb under NN_MDL folder, and rename it to imgclassification.nb
3. plugin SDcard back to AMB82-MINI
4. modify Sketch RTSP_GarbageClassification.ino 1) modify SSID and PASSWD 2) modify imgclass.modelSelect (change DEFAULT_IMGCLASS to CUSTOMIZED_IMGCLASS)
5. burn code into board AMB82-MINI, and run it with VLC player streaming

---

10. 物件偵測 (Object Detection)

Public Dataset

Roboflow

---

物件檢測介紹

Kaggle範例:

• YOLOv7 Facemask detection
• YOLOv7 Pothole detection
• YOLOv7 Sushi detection
• YOLOv7 refrigeratoryfood
  
• YOLOv7 reparm
  

---

Pothole模型訓練與檔案轉換

模型訓練： kaggle.com/rkuo2000/yolov7-pothole

1) repro https://github.com/WongKinYiu/yolov7 2) create pothole.yaml %%writefile data/pothole.yaml

train: ./Datasets/pothole/train/images
val:  ./Datasets/pothole/valid/images
test: ./Datasets/pothole/test/images

# Classes
nc: 1  # number of classes
names: ['pothole']  # class names

3) YOLOv7-Tiny Fixed Resolution Training

!sed -i "s/nc: 80/nc: 1/" cfg/training/yolov7-tiny.yaml
!sed -i "s/IDetect/Detect/" cfg/training/yolov7-tiny.yaml

模型轉換：AMB82 AI Model Conversion

1. Download best.pt from kaggle.com/rkuo2000/yolov7-pothole
2. Compress best.pt to best.zip
3. Go to Amebapro2 AI convert model, fill up your E-mail
4. Upload best.zip
5. Upload one (.jpg) test picture (EX. pothole_test.jpg from Pothole dataset)
6. Email will be sent to you for the link of network_binary.nb

程式範例：RTSP_YOLOv7_Pothole_Detection.ino

1. click the recieved Email link to download network_binary.nb
2. create NN_MDL folder in SDcard, save network_binary.nb under NN_MDL folder, and rename it to yolov7_tiny.nb
3. plugin SDcard back to AMB82-MINI
4. modify Sketch RTSP_YOLOv7_Pothole_Detection.ino 1) modify SSID and PASSWD 2) modify ObjDet.modelSelect(OBJECT_DETECTION, CUSTOMIZED_YOLOV7TINY, NA_MODEL, NA_MODEL);
5. burn code into board AMB82-MINI, and run it with VLC player streaming

---

AMB82 Mini 物件偵測範例

RTSP_YOLOv7_Pothole
RTSP_YOLOv7_Sushi

---

AMB82 Mini SD卡加載模型範例

RTPS_ObjectDetection_AudioClassification.ino

---

Online NN Conversion Tool (客製化模型轉換工具)

---

11. 大型語言模型範例 (LLM)

• 大型語言模型介紹

---

語音辨識範例

ffmpeg.exe is needed for Windows to run Whisper!

• AmebaPro2 Whisper server

• HTTP_Post_Audio.ino

---

語音交談範例

ffmpeg.exe is needed for Windows to run Whisper!

Download ffmpeg-master-latest-win64-gpl.zip, extract & put ffmpeg.exe into where you run Whisper server.

AmebaPro2 Whisper LLM_server

AmebaPro2_Whisper_Gemini_server

• 到 https://aistudio.google.com/app/apikey 取得API_Key 填入GOOGLE_API_KEY, 再執行AmebaPro2_Whisper_Gemini_server.py

HTTP_Post_Audio.ino

• 修改server IP位址 in RecordMP4_HTTP_Post_Audio.ino server IP位址, then 燒錄到 AMB82-MINI
• reset AMB82-MINI 來啟動, 按鍵兩秒後即可錄音詢問 LLM/Gemini

---

12. 視覺語言模型 (VLM)

• 視覺語言模型介紹

---

影像+語音交談範例

• AmebaPro2_whisper_llava_server.py

• HTTP_Post_ImageAudio

---

13. GenAI API

Examples: AmebaNN > GenAIVision

---

14. EdgeAI 應用專題實作

Portable ChatGPT

---

RoboCar

---

Vaccum Robot

AI Front-Door

---

Children’s Intelligent Interactive Robot

---

Automatic License-Plate Recognition (ALPR)

1. YOLOv7 Alphabet Detection
2. GenAI Vision

---

Recycle Assistant

GenAI Vision + SDcardMP3 + Audio amplifier

---

Bread POS

Github: https://github.com/youjunjer/AMB82_Bread_POS

1. YOLOv7 Object Detection
2. GenAI Vision