🛠 NPU 0,8 TOPS : inférence RKNN (YOLO, Whisper) sur la ZERO 3W

Objectives

The ZERO 3W features a 0.8 TOPS RKNN neural accelerator — extremely rare in a Pi Zero form factor at this price point — and Radxa documents a complete official AI stack: RKNN-Toolkit-Lite2 with YOLOv5/v8 examples (object detection), Whisper (speech transcription), and DeepLab (semantic segmentation). This tutorial explains how to achieve your first inference on the NPU, how to choose between the official image and the community Qengineering image, and how to manage the thermal constraints of a 65 × 30 mm form factor.

Hardware

• 1 Radxa ZERO 3W — 2 GB minimum for vision tasks, 4/8 GB for Whisper; the 1 GB SKU will be too limited
• eMMC (8–64 GB soldered) strongly recommended: loading models on a slow microSD card is frustrating
• Heatsink + airflow for sustained inference: the RK3566 is already underclocked to 1.6 GHz (instead of the usual 1.8 GHz) for thermal reasons according to CNX — the NPU will not be an exception
• Stable 5 V/2 A power supply (CPU + NPU + Wi-Fi peaks)

1) Choosing your software base

• Official path: Radxa OS bookworm b1 (fully tested image) + the RKNN example stack from the Radxa documentation (app-development/ai/rknn-quick-start).
• Community path: Qengineering Radxa-Zero-3-NPU-Ubuntu22 — an Ubuntu 22 image with OpenCV, NCNN, and RKNPU drivers preinstalled (github.com/Qengineering/Radxa-Zero-3-NPU-Ubuntu22).

# Verifier que le driver NPU est present et vu par le noyau :
sudo dmesg | grep -i rknpu
ls /sys/class/devfreq/ | grep -i npu        # le NPU apparait comme peripherique freq
cat /sys/class/thermal/thermal_zone*/temp   # temperature de reference au repos

2) First inference (YOLO example)

The RKNN workflow is split into two worlds: model CONVERSION (.rknn format) and INFERENCE on the board via the Lite2 runtime. The Radxa examples (YOLOv5/v8, Whisper, DeepLab) provide pre-converted models and ready-to-run Python scripts.

# Apries application de la procédure Radxa rknn-quick-start :
python3 -c 'import rknn_lite; print("runtime present")' 2>/dev/null || \
python3 -c 'import rknnlite; print("runtime present")'
# Lancer l'exemple YOLO fourni (nom de script selon l'exemple telecharge) :
python3 detect.py --model yolov5s.rknn --img test.jpg

3) Measure, don't hope

# Pendant l'inférence, dans un second terminal :
watch -n1 'cat /sys/class/thermal/thermal_zone*/temp'
time python3 detect.py --model yolov5s.rknn --img test.jpg

If the temperature rises too high, throttling reduces fps — hence the need for a heatsink and ventilation. For a continuous camera stream, combine this with the CSI tutorial: real-world projects mentioned in the sheet (rk3566-ipc-car camera car, USBridge 2.0 KVM with OCR/AI) run exactly on this CSI + NPU duo.

Expected verification

dmesg mentions the RKNPU driver at boot; the YOLO example displays detection boxes and a frames-per-second count significantly higher than what the CPU alone would provide (the CPU vs NPU toggle test is the best indicator); Whisper transcribes an audio sample. The temperature stabilizes with a heatsink.

Common errors

• Running the model on the CPU by forgetting to target the NPU: fps collapse — verify that the runtime correctly attaches the model to the RKNPU device.
• Using the 1 GB SKU for Whisper: OCR/ASR saturates the RAM; use 4 GB+ or lighten the model.
• Converting models on the board: the conversion chain is designed for x86 workstations; on the ZERO 3W you execute, you do not convert. >
• Closed case without a heatsink during continuous inference: throttling, fps drop after a few minutes.
• Relying on the Mali G52 GPU (OpenCL 2.0) instead of the NPU: the RKNN stack is the documented and optimized path; the GPU remains a Plan B.
• Low-quality microSD card: model loading dominates startup time; HS200 eMMC changes everything (see the eMMC tutorial).